CN113379219A - Quality evaluation method and device for emergency management data - Google Patents

Quality evaluation method and device for emergency management data Download PDF

Info

Publication number
CN113379219A
CN113379219A CN202110623948.2A CN202110623948A CN113379219A CN 113379219 A CN113379219 A CN 113379219A CN 202110623948 A CN202110623948 A CN 202110623948A CN 113379219 A CN113379219 A CN 113379219A
Authority
CN
China
Prior art keywords
data
emergency management
management data
parameter
parameters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110623948.2A
Other languages
Chinese (zh)
Inventor
林梓鹏
周亮
杜劲松
赵仕嘉
刘郁恒
郑睿
郑绵彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Planning and Designing Institute of Telecommunications Co Ltd
Original Assignee
Guangdong Planning and Designing Institute of Telecommunications Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Planning and Designing Institute of Telecommunications Co Ltd filed Critical Guangdong Planning and Designing Institute of Telecommunications Co Ltd
Priority to CN202110623948.2A priority Critical patent/CN113379219A/en
Publication of CN113379219A publication Critical patent/CN113379219A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0639Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06393Score-carding, benchmarking or key performance indicator [KPI] analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/067Enterprise or organisation modelling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/26Government or public services
    • G06Q50/265Personal security, identity or safety

Landscapes

  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Engineering & Computer Science (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • Tourism & Hospitality (AREA)
  • Development Economics (AREA)
  • Educational Administration (AREA)
  • Entrepreneurship & Innovation (AREA)
  • General Physics & Mathematics (AREA)
  • Marketing (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Game Theory and Decision Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a quality evaluation method and a device of emergency management data, wherein the method comprises the following steps: acquiring target emergency management data, and performing data preprocessing on the target emergency management data; inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result; the evaluation benchmark parameters of the data quality evaluation model comprise one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data. Therefore, the method and the device can accurately evaluate the data quality of the emergency management data, are beneficial to establishing a set of perfect emergency management data evaluation specifications, and further can improve the emergency treatment efficiency and reduce the damage caused by large-scale emergency events.

Description

Quality evaluation method and device for emergency management data
Technical Field
The invention relates to the technical field of smart cities, in particular to a quality evaluation method and device for emergency management data.
Background
Along with the intellectualization of cities, the importance degree of government departments on emergency management of public safety events is also gradually improved. Emergency management refers to the process of government and other public institutions in the processes of pre-prevention, incident response, disposal in the incident and good recovery of emergencies, and by establishing a necessary response mechanism, a series of necessary measures are taken, measures such as science, technology, planning and management are applied, the public life, health and property safety are guaranteed, and the relevant activities of social harmony and healthy development are promoted.
And the emergency management is carried out based on the emergency management data, and the comprehensive information required by the emergency management is analyzed to cover dangerous transportation vehicle information, enterprise information, population density information, key personnel information, ship information, natural disaster weather information, material reserve information, avoidance place information and the like. The emergency management data provides basic data support for the construction of the emergency command comprehensive management platform, but because the existing emergency management data has many and complex sources, and a set of standard evaluation standards are not established in the existing emergency management data quality evaluation technology, the defects of the existing technology are overcome, and a solution is needed.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a quality evaluation method and device for emergency management data, which can accurately evaluate the data quality of the emergency management data, is beneficial to establishing a set of perfect emergency management data evaluation specifications, can further improve the efficiency of emergency treatment, and reduce the damage caused by large-scale emergency events.
In order to solve the technical problem, a first aspect of the present invention discloses a quality evaluation method for emergency management data, including:
acquiring target emergency management data, and performing data preprocessing on the target emergency management data;
inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result; the evaluation benchmark parameters of the data quality evaluation model comprise one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data.
As an alternative implementation, in the first aspect of the present invention, the method further includes:
judging whether the data quality evaluation result meets a preset data alarm rule or not to obtain a first judgment result;
and when the first judgment result is yes, sending the data quality evaluation result to a corresponding terminal according to a preset notification sending rule.
As an alternative embodiment, in the first aspect of the present invention, the data preprocessing includes one or more of data cleaning, data integration, data transformation, data desensitization, and data reduction.
As an optional implementation manner, in the first aspect of the present invention, the inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result includes:
calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data;
multiplying the parameter value by the parameter weight corresponding to the parameter type of the parameter value to obtain an evaluation reference parameter result value;
and determining the sum of all evaluation reference parameter result values as a data quality evaluation result.
As an optional implementation manner, in the first aspect of the present invention, the calculating a parameter value of at least one of the evaluation reference parameters corresponding to the preprocessed target emergency management data includes:
matching and calculating the missing data proportion and/or the invalid data proportion in the target emergency management data according to a preset data integrity template so as to calculate the integrity parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
according to a preset data specification rule, calculating the proportion of non-standard format data in the target emergency management data in a matching manner, so as to calculate the normative parameters corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
according to a preset data verification rule, calculating the error data proportion and/or the overdue data proportion in the target emergency management data in a matching manner, so as to calculate the accuracy parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
calculating the proportion of repeated data and/or the proportion of attribute repeated data in the target emergency management data so as to calculate the uniqueness parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
calculating a loss proportion and/or an unassociated data proportion of associated data in the target emergency management data according to a preset data association rule, so as to calculate an association parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
determining attacked information of a database corresponding to the target emergency management data, and determining security parameters corresponding to the target emergency management data according to the attacked information of the database;
and/or the presence of a gas in the gas,
determining the proportion of conflict data in the target emergency management data, and determining consistency parameters corresponding to the target emergency management data according to the proportion of the conflict data; the conflict data are data with conflicting information meanings in the target emergency data;
and/or the presence of a gas in the gas,
acquiring a plurality of instruction execution times corresponding to the target emergency management data in a historical time period, and determining timeliness parameters corresponding to the target emergency management data according to the instruction execution times; the instruction execution time is the time difference between the triggered time and the executed time of the triggered instruction when the target emergency management data in the historical time period accords with a preset emergency instruction triggering rule.
As an optional implementation manner, in the first aspect of the present invention, the data quality evaluation model is:
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S;
wherein P is a data quality evaluation result, and I, N, C, A, U, R, S are the integrity parameter, the normative parameter, the consistency parameter, the accuracy parameter, the uniqueness parameter, the relevance parameter, and the security parameter corresponding to the target emergency management data, respectively; a is1-a7The integrity parameter, the normative parameter, the consistency parameter, the accuracy parameter, the uniqueness parameter, the relevance parameter and the parameter weight corresponding to the safety parameter corresponding to the target emergency management data are respectively weighted;
or the like, or, alternatively,
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S+α8T;
wherein T is the timeliness parameter corresponding to the target emergency management data, a8And weighting the parameter corresponding to the timeliness parameter.
As an alternative implementation, in the first aspect of the present invention, the terminal includes one or more of a data consumer terminal, a data provider terminal, or a data management server terminal in combination.
As an alternative implementation, in the first aspect of the present invention, the method further includes:
determining a notification response time of the terminal; the notification response time is a time difference between the time when the terminal receives the data quality evaluation result and the time when the terminal executes corresponding operation according to the data quality evaluation result;
and judging whether the notification response time of the terminal meets a preset punishment rule or not, and if so, executing corresponding punishment operation on the terminal.
The second aspect of the invention discloses a quality evaluation device for emergency management data, which comprises:
the system comprises a preprocessing module, a data preprocessing module and a data processing module, wherein the preprocessing module is used for acquiring target emergency management data and preprocessing the target emergency management data;
the evaluation module is used for inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation so as to obtain a data quality evaluation result; the evaluation benchmark parameters of the data quality evaluation model comprise one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data.
As an alternative embodiment, in the second aspect of the present invention, the apparatus further comprises:
the first judgment module is used for judging whether the data quality evaluation result meets a preset data alarm rule or not so as to obtain a first judgment result;
and the sending module is used for sending the data quality evaluation result to a corresponding terminal according to a preset notification sending rule when the first judgment result is yes.
As an alternative embodiment, in the second aspect of the present invention, the data preprocessing includes one or more of data cleaning, data integration, data transformation, data desensitization, and data reduction.
As an optional implementation manner, in the second aspect of the present invention, a specific manner in which the evaluation module inputs the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result includes:
calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data;
multiplying the parameter value by the parameter weight corresponding to the parameter type of the parameter value to obtain an evaluation reference parameter result value;
and determining the sum of all evaluation reference parameter result values as a data quality evaluation result.
As an optional implementation manner, in the second aspect of the present invention, the calculating a parameter value of at least one of the evaluation reference parameters corresponding to the preprocessed target emergency management data includes:
matching and calculating the missing data proportion and/or the invalid data proportion in the target emergency management data according to a preset data integrity template so as to calculate the integrity parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
according to a preset data specification rule, calculating the proportion of non-standard format data in the target emergency management data in a matching manner, so as to calculate the normative parameters corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
according to a preset data verification rule, calculating the error data proportion and/or the overdue data proportion in the target emergency management data in a matching manner, so as to calculate the accuracy parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
calculating the proportion of repeated data and/or the proportion of attribute repeated data in the target emergency management data so as to calculate the uniqueness parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
calculating a loss proportion and/or an unassociated data proportion of associated data in the target emergency management data according to a preset data association rule, so as to calculate an association parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
determining attacked information of a database corresponding to the target emergency management data, and determining security parameters corresponding to the target emergency management data according to the attacked information of the database;
and/or the presence of a gas in the gas,
determining the proportion of conflict data in the target emergency management data, and determining consistency parameters corresponding to the target emergency management data according to the proportion of the conflict data; the conflict data are data with conflicting information meanings in the target emergency data;
and/or the presence of a gas in the gas,
acquiring a plurality of instruction execution times corresponding to the target emergency management data in a historical time period, and determining timeliness parameters corresponding to the target emergency management data according to the instruction execution times; the instruction execution time is the time difference between the triggered time and the executed time of the triggered instruction when the target emergency management data in the historical time period accords with a preset emergency instruction triggering rule.
As an optional implementation manner, in the second aspect of the present invention, the data quality evaluation model is:
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S;
wherein P is a data quality evaluation result, and I, N, C, A, U, R, S are the integrity parameter, the normative parameter, the consistency parameter, the accuracy parameter, the uniqueness parameter, the relevance parameter, and the security parameter corresponding to the target emergency management data, respectively; a is1-a7The integrity parameter, the normative parameter, the consistency parameter, the accuracy parameter, the uniqueness parameter, the relevance parameter and the parameter weight corresponding to the safety parameter corresponding to the target emergency management data are respectively weighted;
or the like, or, alternatively,
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S+α8T;
wherein T is the timeliness parameter corresponding to the target emergency management data, a8And weighting the parameter corresponding to the timeliness parameter.
As an alternative embodiment, in the second aspect of the present invention, the terminal includes one or a combination of a plurality of data consumer terminals, data provider terminals, or data management server terminals.
As an alternative embodiment, in the second aspect of the present invention, the apparatus further comprises:
a determining module, configured to determine a notification response time of the terminal; the notification response time is a time difference between the time when the terminal receives the data quality evaluation result and the time when the terminal executes corresponding operation according to the data quality evaluation result;
the second judgment module is used for judging whether the notification response time of the terminal meets a preset punishment rule or not;
and the execution module is used for executing corresponding punishment operation on the terminal when the judgment result of the second judgment module is yes.
The third aspect of the present invention discloses another quality evaluation device for emergency management data, the device comprising:
a memory storing executable program code;
a processor coupled with the memory;
the processor calls the executable program code stored in the memory to execute part or all of the steps of the quality evaluation method of the emergency management data disclosed in the first aspect of the embodiment of the invention.
A fourth aspect of the present invention discloses a computer storage medium, where a computer instruction is stored, and when the computer instruction is called, the computer instruction is used to execute part or all of the steps in the method for evaluating quality of emergency management data disclosed in the first aspect of the present invention.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, target emergency management data is obtained, and data preprocessing is carried out on the target emergency management data; inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result; the evaluation benchmark parameters of the data quality evaluation model comprise one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data. Therefore, the quality evaluation method and the system can evaluate the quality of the emergency management data through the data quality evaluation model which considers various evaluation parameters of the emergency management data, so that the data quality of the emergency management data can be accurately evaluated, a set of complete emergency management data evaluation specifications can be established, the emergency treatment efficiency can be improved, and the damage caused by large-scale emergency events can be reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for evaluating the quality of emergency management data according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a quality evaluation device for emergency management data according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of another emergency management data quality evaluation device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," and the like in the description and claims of the present invention and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, article, or article that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or article.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The invention discloses a quality evaluation method and a quality evaluation device for emergency management data, which can evaluate the quality of the emergency management data through a data quality evaluation model considering various evaluation parameters of the emergency management data, thereby accurately evaluating the data quality of the emergency management data, being beneficial to establishing a set of complete emergency management data evaluation specifications, further improving the efficiency of emergency treatment and reducing the damage caused by large-scale emergency events. The following are detailed below.
Example one
Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a method for evaluating quality of emergency management data according to an embodiment of the present invention. The quality evaluation method can be applied to an emergency management data terminal or a server, and the invention is not limited. Preferably, the execution subject of the method can be a server with processing and storage functions and other electronic devices, which can be composed by software or/and hardware, referred to as an emergency management data platform, and exemplarily implement a data governance process by using a cloud server. As shown in fig. 1, the method for evaluating the quality of emergency management data may include the following operations:
101. and acquiring target emergency management data, and performing data preprocessing on the target emergency management data.
In the embodiment of the invention, the target emergency management data can be acquired in a data extraction mode, and the extracted data can comprise government affair data and internet data.
For extraction of internet data, a data provider monitors information of social media such as government websites, industry websites, enterprise websites, business portals, micro blogs and micro letters in real time according to business application requirements, and acquires required internet data regularly, particularly data of government affairs services such as government affair disclosure, government affair handling and interaction in a target area (province, city, county and the like), dynamic data of enterprises in a target area prefecture and the like.
Optionally, the collected internet data may be subjected to intelligent preliminary processing, for example, manual + computer-assisted identification information operation of the warehousing data, so as to realize functions including text content extraction, automatic attribute extraction, automatic format conversion, automatic Chinese internal code conversion, automatic classification, and the like. The automatic attribute extraction refers to automatically extracting attributes such as unit name, system name, title, edition, date, author, column and classification in the webpage. The automatic classification function is a function of performing automatic classification based on collected data. The automatic classification function should implement two methods, automatic classification based on semantic rules (machine inspection classification) and automatic classification based on statistical principles (with content).
For the extraction of the government affair data, various heterogeneous data source collecting and processing means can be adopted according to various data sources such as a government affair system interface, a government affair application database, a government affair electronic text, a government affair archive and the like provided by a buyer, and the government affair data is collected and summarized according to a certain format and rules.
Optionally, the data extraction mode may support cross-network multi-source heterogeneous data extraction processing, and a uniform interface access is provided to the outside through multiple heterogeneous data source extraction components, so as to support but not limited to the following database resources to obtain data: oracle, GP (greenplus), TeraData, DB2, MySQL, PostgreSQL, Hive, Hbase, Hadoop, Xcloud, damamway database, optionally supporting but not limited to the following file service or message service resource acquisition data: HDFS, FTP/SFTP, Kafka. Optionally, but not limited to, the following ways are supported to acquire data: socket, crawler, WebService, JDBC, front-end processor acquisition, OCR recognition, and the like, optionally, support but not limited to acquiring data in the following file formats: CSV, XML, HTML, JSON, TXT, Word, Excel, picture, video, audio, OCR recognition data.
Optionally, the extraction mode can be automatic extraction or manual entry for the system. The system automatic extraction supports full data extraction, incremental data extraction, batch data extraction, real-time extraction, timing extraction, one-time extraction and timing cycle extraction, and supports synchronization of data structures. When data is extracted, a data check and error correction mechanism can be established to ensure that the extracted data is consistent and complete with the data of the data source.
102. And inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result.
In the embodiment of the invention, the evaluation reference parameters of the data quality evaluation model comprise one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data.
Therefore, by implementing the embodiment of the invention, the quality of the emergency management data can be evaluated through the data quality evaluation model which takes various evaluation parameters of the emergency management data into consideration, so that the data quality of the emergency management data can be accurately evaluated, a set of complete emergency management data evaluation specifications can be established, the emergency treatment efficiency can be improved, and the damage caused by large-scale emergency events can be reduced.
As an alternative embodiment, the data preprocessing in step 101 includes one or more of data cleaning, data integration, data transformation, data desensitization, and data reduction. Wherein data cleaning is mainly to "clean" data by filling in missing values, smoothing noisy data, identifying or deleting outliers and resolving inconsistencies; the method mainly achieves the following aims of format standardization, abnormal data clearing, error correction and repeated data clearing. Data integration is the process of combining and uniformly storing data in multiple data sources to build a data warehouse. Data transformation is the conversion of data into a suitable format by means of smooth aggregation, data generalization, normalization, etc., and in the preferred embodiment of the present invention, the data format is defined in a standardized manner. Data desensitization refers to data deformation of some sensitive information through desensitization rules to realize reliable protection of sensitive private data, and the data desensitization includes but is not limited to random offset, mask replacement, truncation, precision reduction, reverse processing and random replacement. Data reduction techniques can be used to obtain a reduced representation of a data set that is much smaller than the original data prior to reduction, yet close to maintaining the integrity of the original data and the result is the same or nearly the same as the result prior to reduction.
Therefore, by implementing the optional embodiment, the target emergency management data can be preprocessed in a plurality of data preprocessing modes, so that more standard and more standard emergency management data can be obtained, a reliable data base is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an alternative embodiment, in step 102, inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result, where the data quality evaluation result includes:
calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data;
multiplying the parameter value by the parameter weight corresponding to the parameter type of the parameter value to obtain an evaluation reference parameter result value;
and determining the sum of all evaluation reference parameter result values as a data quality evaluation result.
In the embodiment of the present invention, the parameter weight is used to indicate the importance of the corresponding evaluation reference parameter, and preferably, when the data quality evaluation model includes a plurality of evaluation reference parameters, the sum of the parameter weights of all the evaluation reference parameters should be equal to 1.
In the embodiment of the invention, the data quality evaluation result may be the sum of evaluation reference parameter result values corresponding to one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data.
Therefore, by implementing the optional embodiment, the sum of all evaluation reference parameter result values can be determined as the data quality evaluation result, so that the priority and the importance among different evaluation reference parameters are considered, and the final data quality evaluation result is balanced in a parameter weight mode to obtain a more accurate data quality evaluation result.
As an optional embodiment, in the step, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
and matching and calculating the missing data proportion and/or the invalid data proportion in the target emergency management data according to a preset data integrity template so as to calculate the integrity parameter corresponding to the target emergency management data.
Optionally, the data integrity template may be used to indicate the data type, data amount, and data availability criteria in the completed emergency management data for subsequent matching calculations. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Therefore, by implementing the optional embodiment, the missing data proportion and/or the invalid data proportion in the target emergency management data can be matched and calculated, so that the information related to the integrity of the emergency management data can be calculated more accurately, a reliable data basis is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional embodiment, in the step, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
and according to a preset data specification rule, calculating the non-standard format data proportion in the target emergency management data in a matching manner, so as to calculate the normative parameters corresponding to the target emergency management data. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Optionally, the data specification rule may be used to indicate the data type, data format, and data requirements in the specified emergency management data for subsequent matching calculations.
Therefore, by implementing the optional embodiment, the non-standard format data proportion in the target emergency management data can be matched and calculated, so that the information related to the standardization of the emergency management data can be calculated more accurately, a reliable data base is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional embodiment, in the step, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
and matching and calculating the error data proportion and/or the overdue data proportion in the target emergency management data according to a preset data verification rule, so as to calculate the accuracy parameter corresponding to the target emergency management data.
Optionally, the data validation rules may be used to indicate the data type, data format, and data duration in the correct emergency management data for subsequent matching calculations. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Therefore, by implementing the optional embodiment, the error data proportion and/or the overdue data proportion in the target emergency management data can be matched and calculated, so that the information related to the accuracy of the emergency management data can be calculated more accurately, a reliable data basis is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional embodiment, in the step, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
and calculating the proportion of the repeated data and/or the proportion of the attribute repeated data in the target emergency management data so as to calculate the uniqueness parameter corresponding to the target emergency management data.
Optionally, the proportion of the duplicated data is the proportion of duplicated data in the emergency management data, and the proportion of the attribute duplicated data is the proportion of duplicated data of the attribute type or the attribute value in the emergency management data. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Therefore, by implementing the optional embodiment, the proportion of the repeated data and/or the proportion of the attribute repeated data in the target emergency management data can be calculated, so that the information related to uniqueness of the emergency management data can be calculated more accurately, a reliable data basis is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional embodiment, in the step, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
and calculating the loss proportion and/or the unassociated data proportion of the associated data in the target emergency management data according to a preset data association rule so as to calculate the association parameter corresponding to the target emergency management data.
Optionally, the data association rule is used to indicate a data type, a data value or a data format that should be associated to the same association in the emergency management data, so as to facilitate subsequent matching calculation. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Therefore, by implementing the optional embodiment, the loss proportion of the associated data and/or the proportion of the unassociated data in the target emergency management data can be calculated, so that the information related to the association of the emergency management data can be calculated more accurately, a reliable data basis is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional embodiment, in the step, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
and determining the attacked information of the database corresponding to the target emergency management data, and determining the security parameters corresponding to the target emergency management data according to the attacked information of the database.
Optionally, the attacked information of the database may be information such as the attacked times and frequency of the database corresponding to the emergency management data, or information such as the attacked times of the successful defense of the database after the attack. Alternatively, the parameter value of the parameter may be given a certain score according to the number of times.
Therefore, by implementing the optional embodiment, the security parameters corresponding to the target emergency management data can be determined according to the attacked information of the database, so that the security-related information of the emergency management data can be calculated more accurately, a reliable data basis is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional embodiment, in the step, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
and determining the proportion of conflict data in the target emergency management data, and determining the consistency parameter corresponding to the target emergency management data according to the proportion of the conflict data.
Optionally, the conflict data is data in which the meaning of the information in the target emergency data conflicts. Alternatively, the parameter value of the parameter may be given a certain score according to the number of times.
Therefore, by implementing the optional embodiment, the consistency parameter corresponding to the target emergency management data can be determined according to the proportion of the conflict data, so that the information related to the consistency of the emergency management data can be calculated more accurately, a reliable data basis is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional embodiment, in the step, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
the method comprises the steps of obtaining a plurality of instruction execution times corresponding to target emergency management data in a historical time period, and determining timeliness parameters corresponding to the target emergency management data according to the instruction execution times.
Optionally, the instruction execution time is a time difference between a triggered time and an executed time of the triggered instruction when the target emergency management data in the historical time period meets a preset emergency instruction triggering rule. Optionally, the timeliness parameter may be an average, median, or mode of the execution times of the multiple instructions, or may be an average of the execution times of the multiple instructions based on a weight.
Therefore, by implementing the optional embodiment, the timeliness parameters corresponding to the target emergency management data can be determined according to the execution time of the multiple instructions, so that the timeliness related information of the emergency management data can be calculated more accurately, a reliable data basis is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional embodiment, the data quality evaluation model in the present invention is:
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S;
wherein, P is a data quality evaluation result, and I, N, C, A, U, R, S is an integrity parameter, a normative parameter, a consistency parameter, an accuracy parameter, a uniqueness parameter, an association parameter and a security parameter corresponding to the target emergency management data respectively; a is1-a7And respectively weighing parameters corresponding to the integrity parameter, the normative parameter, the consistency parameter, the accuracy parameter, the uniqueness parameter, the relevance parameter and the safety parameter corresponding to the target emergency management data.
As an optional embodiment, the data quality evaluation model in the present invention is:
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S+α8T;
wherein T is a timeliness parameter corresponding to the target emergency management data, a8The parameter weight corresponding to the timeliness parameter.
As an alternative embodiment, the method further comprises:
judging whether the data quality evaluation result meets a preset data alarm rule or not to obtain a first judgment result;
and when the first judgment result is yes, sending the data quality evaluation result to the corresponding terminal according to a preset notification sending rule.
Optionally, the terminal includes one or more combinations of a data consumer terminal, a data provider terminal, or a data management server terminal, that is, the terminal may send a notification to terminals corresponding to the data consumer, the data provider, and the data management server at the same time, or send a notification to a terminal corresponding to one or two of the data consumer, the data provider, and the data management server, where the corresponding terminal may be a terminal used by a person in charge or/and a contact of a relevant party (the data consumer, the data provider, and the data management server), for example, send a short message notification to a mobile phone of the relevant party through a mobile phone number.
For example, taking integrity as an example, when the integrity score determined by the integrity quality evaluation model is 90, a reminder notification is simultaneously transmitted to the data consumer, the data provider, and the data management server, when the integrity score determined by the integrity quality evaluation model is 80, an early warning notification is simultaneously transmitted to the data consumer, the data provider, and the data management server, and when the integrity score determined by the integrity quality evaluation model is 70, an emergency notification is simultaneously transmitted to the data consumer, the data provider, and the data management server.
Similar to the notification of other constraint elements, the notification content includes the corresponding constraint element, the number or proportion of data not meeting the requirement corresponding to the constraint element, and the scoring result, for example, the content of a certain emergency notification is: presetting elements: data integrity, number or proportion: 12%, scoring results: and 70 minutes. When the related party executes the notification instruction, the problem data corresponding to the constraint elements are obtained so as to optimize or/and process the problem data.
As an alternative embodiment, the method further comprises:
determining notification response time of the terminal;
and judging whether the notification response time of the terminal meets a preset punishment rule or not, and if so, executing corresponding punishment operation on the terminal.
Optionally, the notification response time is a time difference between a time when the terminal receives the data quality evaluation result and a time when the terminal performs a corresponding operation according to the data quality evaluation result.
For different notification situations, a notification response time threshold can be preset, and the time threshold can be used as a response speed evaluation for a data user, a data provider and a data management server to give a certain penalty when the response speed evaluation fails, for example, when the data provider of an emergency notification needs to process the emergency notification at an appointed time, for example, 10min, if the emergency notification does not process the emergency notification, a corresponding penalty, for example, an alarm penalty, is given, and when the number of the alarm penalties reaches a certain degree, the data provider is likely to be relieved.
Because the emergency management platform is mainly proposed for the dangerous problems of the very serious accident disaster, the response speed of each party to the quality problem of the emergency management data, namely the time for triggering and informing the relevant party to execute the relevant operation, can also be used as a part of data quality evaluation, thereby being more beneficial to ensuring the subsequent smooth and quick response of the emergency management platform.
Example two
Referring to fig. 2, fig. 2 is a schematic structural diagram of a quality evaluation device for emergency management data according to an embodiment of the present invention. As shown in fig. 2, the apparatus may include:
the preprocessing module 201 is configured to acquire target emergency management data and perform data preprocessing on the target emergency management data.
In the embodiment of the invention, the target emergency management data can be acquired in a data extraction mode, and the extracted data can comprise government affair data and internet data.
For extraction of internet data, a data provider monitors information of social media such as government websites, industry websites, enterprise websites, business portals, micro blogs and micro letters in real time according to business application requirements, and acquires required internet data regularly, particularly data of government affairs services such as government affair disclosure, government affair handling and interaction in a target area (province, city, county and the like), dynamic data of enterprises in a target area prefecture and the like.
Optionally, the collected internet data may be subjected to intelligent preliminary processing, for example, manual + computer-assisted identification information operation of the warehousing data, so as to realize functions including text content extraction, automatic attribute extraction, automatic format conversion, automatic Chinese internal code conversion, automatic classification, and the like. The automatic attribute extraction refers to automatically extracting attributes such as unit name, system name, title, edition, date, author, column and classification in the webpage. The automatic classification function is a function of performing automatic classification based on collected data. The automatic classification function should implement two methods, automatic classification based on semantic rules (machine inspection classification) and automatic classification based on statistical principles (with content).
For the extraction of the government affair data, various heterogeneous data source collecting and processing means can be adopted according to various data sources such as a government affair system interface, a government affair application database, a government affair electronic text, a government affair archive and the like provided by a buyer, and the government affair data is collected and summarized according to a certain format and rules.
Optionally, the data extraction mode may support cross-network multi-source heterogeneous data extraction processing, and a uniform interface access is provided to the outside through multiple heterogeneous data source extraction components, so as to support but not limited to the following database resources to obtain data: oracle, GP (greenplus), TeraData, DB2, MySQL, PostgreSQL, Hive, Hbase, Hadoop, Xcloud, damamway database, optionally supporting but not limited to the following file service or message service resource acquisition data: HDFS, FTP/SFTP, Kafka. Optionally, but not limited to, the following ways are supported to acquire data: socket, crawler, WebService, JDBC, front-end processor acquisition, OCR recognition, and the like, optionally, support but not limited to acquiring data in the following file formats: CSV, XML, HTML, JSON, TXT, Word, Excel, picture, video, audio, OCR recognition data.
Optionally, the extraction mode can be automatic extraction or manual entry for the system. The system automatic extraction supports full data extraction, incremental data extraction, batch data extraction, real-time extraction, timing extraction, one-time extraction and timing cycle extraction, and supports synchronization of data structures. When data is extracted, a data check and error correction mechanism can be established to ensure that the extracted data is consistent and complete with the data of the data source.
The evaluation module 202 is configured to input the preprocessed target emergency management data into the data quality evaluation model for evaluation, so as to obtain a data quality evaluation result.
In the embodiment of the invention, the evaluation reference parameters of the data quality evaluation model comprise one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data.
Therefore, by implementing the embodiment of the invention, the quality of the emergency management data can be evaluated through the data quality evaluation model which takes various evaluation parameters of the emergency management data into consideration, so that the data quality of the emergency management data can be accurately evaluated, a set of complete emergency management data evaluation specifications can be established, the emergency treatment efficiency can be improved, and the damage caused by large-scale emergency events can be reduced.
As an alternative embodiment, the data preprocessing includes one or more of data cleansing, data integration, data transformation, data desensitization, and data reduction. Wherein data cleaning is mainly to "clean" data by filling in missing values, smoothing noisy data, identifying or deleting outliers and resolving inconsistencies; the method mainly achieves the following aims of format standardization, abnormal data clearing, error correction and repeated data clearing. Data integration is the process of combining and uniformly storing data in multiple data sources to build a data warehouse. Data transformation is the conversion of data into a suitable format by means of smooth aggregation, data generalization, normalization, etc., and in the preferred embodiment of the present invention, the data format is defined in a standardized manner. Data desensitization refers to data deformation of some sensitive information through desensitization rules to realize reliable protection of sensitive private data, and the data desensitization includes but is not limited to random offset, mask replacement, truncation, precision reduction, reverse processing and random replacement. Data reduction techniques can be used to obtain a reduced representation of a data set that is much smaller than the original data prior to reduction, yet close to maintaining the integrity of the original data and the result is the same or nearly the same as the result prior to reduction.
Therefore, by implementing the optional embodiment, the target emergency management data can be preprocessed in a plurality of data preprocessing modes, so that more standard and more standard emergency management data can be obtained, a reliable data base is provided for subsequent data evaluation, and the evaluation accuracy is improved.
As an optional implementation manner, the specific manner in which the evaluation module 202 inputs the preprocessed target emergency management data into the data quality evaluation model for evaluation to obtain a data quality evaluation result includes:
calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data;
multiplying the parameter value by the parameter weight corresponding to the parameter type of the parameter value to obtain an evaluation reference parameter result value;
and determining the sum of all evaluation reference parameter result values as a data quality evaluation result.
In the embodiment of the present invention, the parameter weight is used to indicate the importance of the corresponding evaluation reference parameter, and preferably, when the data quality evaluation model includes a plurality of evaluation reference parameters, the sum of the parameter weights of all the evaluation reference parameters should be equal to 1.
In the embodiment of the invention, the data quality evaluation result may be the sum of evaluation reference parameter result values corresponding to one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data.
Therefore, by implementing the optional embodiment, the sum of all evaluation reference parameter result values can be determined as the data quality evaluation result, so that the priority and the importance among different evaluation reference parameters are considered, and the final data quality evaluation result is balanced in a parameter weight mode to obtain a more accurate data quality evaluation result.
As an optional implementation manner, calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
matching and calculating the missing data proportion and/or the invalid data proportion in the target emergency management data according to a preset data integrity template so as to calculate the integrity parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
according to a preset data specification rule, calculating the proportion of non-standard format data in the target emergency management data in a matching manner, so as to calculate the normative parameters corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
according to a preset data verification rule, calculating the error data proportion and/or the overdue data proportion in the target emergency management data in a matching manner, so as to calculate the accuracy parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
calculating the proportion of repeated data and/or the proportion of attribute repeated data in the target emergency management data, and calculating the uniqueness parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
calculating the loss proportion and/or the unassociated data proportion of the associated data in the target emergency management data according to a preset data association rule, so as to calculate the association parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
determining attacked information of a database corresponding to the target emergency management data, and determining security parameters corresponding to the target emergency management data according to the attacked information of the database;
and/or the presence of a gas in the gas,
determining the proportion of conflict data in the target emergency management data, and determining consistency parameters corresponding to the target emergency management data according to the proportion of the conflict data; the conflict data are data with conflicting information meanings in the target emergency data;
and/or the presence of a gas in the gas,
acquiring a plurality of instruction execution times corresponding to target emergency management data in a historical time period, and determining timeliness parameters corresponding to the target emergency management data according to the instruction execution times; the instruction execution time is the time difference between the triggered time and the executed time of the triggered instruction when the target emergency management data in the historical time period accords with the preset emergency instruction triggering rule.
Optionally, the data integrity template may be used to indicate the data type, data amount, and data availability criteria in the completed emergency management data for subsequent matching calculations. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Optionally, the data specification rule may be used to indicate the data type, data format, and data requirements in the specified emergency management data for subsequent matching calculations.
Optionally, the data validation rules may be used to indicate the data type, data format, and data duration in the correct emergency management data for subsequent matching calculations. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Optionally, the proportion of the duplicated data is the proportion of duplicated data in the emergency management data, and the proportion of the attribute duplicated data is the proportion of duplicated data of the attribute type or the attribute value in the emergency management data. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Optionally, the data association rule is used to indicate a data type, a data value or a data format that should be associated to the same association in the emergency management data, so as to facilitate subsequent matching calculation. Optionally, a certain fraction may be given to the parameter value of the parameter according to the number of times, for example, the ratio is within 2%, the score is 100, when 2% -5%, the score is 90, and so on.
Optionally, the attacked information of the database may be information such as the attacked times and frequency of the database corresponding to the emergency management data, or information such as the attacked times of the successful defense of the database after the attack. Alternatively, the parameter value of the parameter may be given a certain score according to the number of times.
Optionally, the timeliness parameter may be an average, median, or mode of the execution times of the multiple instructions, or may be an average of the execution times of the multiple instructions based on a weight.
As an alternative implementation, the data quality evaluation model is:
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S;
wherein, P is a data quality evaluation result, and I, N, C, A, U, R, S is an integrity parameter, a normative parameter, a consistency parameter, an accuracy parameter, a uniqueness parameter, an association parameter and a security parameter corresponding to the target emergency management data respectively; a is1-a7Respectively weighing parameters corresponding to an integrity parameter, a normative parameter, a consistency parameter, an accuracy parameter, a uniqueness parameter, an association parameter and a safety parameter corresponding to the target emergency management data;
as an alternative implementation, the data quality evaluation model is:
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S+α8T;
wherein T is a timeliness parameter corresponding to the target emergency management data, a8The parameter weight corresponding to the timeliness parameter.
As an optional implementation, the apparatus further comprises:
the first judgment module is used for judging whether the data quality evaluation result meets a preset data alarm rule or not so as to obtain a first judgment result;
and the sending module is used for sending the data quality evaluation result to the corresponding terminal according to a preset notification sending rule when the first judgment result is yes.
Optionally, the terminal includes one or more combinations of a data consumer terminal, a data provider terminal, or a data management server terminal, that is, the terminal may send a notification to terminals corresponding to the data consumer, the data provider, and the data management server at the same time, or send a notification to a terminal corresponding to one or two of the data consumer, the data provider, and the data management server, where the corresponding terminal may be a terminal used by a person in charge or/and a contact of a relevant party (the data consumer, the data provider, and the data management server), for example, send a short message notification to a mobile phone of the relevant party through a mobile phone number.
For example, taking integrity as an example, when the integrity score determined by the integrity quality evaluation model is 90, a reminder notification is simultaneously transmitted to the data consumer, the data provider, and the data management server, when the integrity score determined by the integrity quality evaluation model is 80, an early warning notification is simultaneously transmitted to the data consumer, the data provider, and the data management server, and when the integrity score determined by the integrity quality evaluation model is 70, an emergency notification is simultaneously transmitted to the data consumer, the data provider, and the data management server.
Similar to the notification of other constraint elements, the notification content includes the corresponding constraint element, the number or proportion of data not meeting the requirement corresponding to the constraint element, and the scoring result, for example, the content of a certain emergency notification is: presetting elements: data integrity, number or proportion: 12%, scoring results: and 70 minutes. When the related party executes the notification instruction, the problem data corresponding to the constraint elements are obtained so as to optimize or/and process the problem data.
As an optional implementation, the apparatus further comprises:
the determining module is used for determining the notification response time of the terminal; the notification response time is the time difference between the time when the terminal receives the data quality evaluation result and the time when the terminal executes corresponding operation according to the data quality evaluation result;
the second judgment module is used for judging whether the notification response time of the terminal meets the preset punishment rule or not;
and the execution module is used for executing corresponding punishment operation on the terminal when the judgment result of the second judgment module is yes.
For different notification situations, a notification response time threshold can be preset, and the time threshold can be used as a response speed evaluation for a data user, a data provider and a data management server to give a certain penalty when the response speed evaluation fails, for example, when the data provider of an emergency notification needs to process the emergency notification at an appointed time, for example, 10min, if the emergency notification does not process the emergency notification, a corresponding penalty, for example, an alarm penalty, is given, and when the number of the alarm penalties reaches a certain degree, the data provider is likely to be relieved.
EXAMPLE III
Referring to fig. 3, fig. 3 is a schematic structural diagram of another emergency management data quality evaluation device according to an embodiment of the present invention. As shown in fig. 3, the apparatus may include:
a memory 301 storing executable program code;
a processor 302 coupled to the memory 301;
the processor 302 calls the executable program code stored in the memory 301 to execute some or all of the steps of the method for evaluating the quality of emergency management data disclosed in the embodiment of the present invention.
Example four
The embodiment of the invention discloses a computer storage medium, which stores computer instructions, and when the computer instructions are called, the computer instructions are used for executing part or all of the steps in the quality evaluation method of emergency management data disclosed by the embodiment of the invention.
EXAMPLE five
The embodiment of the invention discloses a quality evaluation system of emergency management data, which comprises a data transmission module. The data transmission module is used for executing part or all of the steps in the emergency management data quality evaluation method disclosed by the embodiment of the invention.
The above-described embodiments of the apparatus are merely illustrative, and the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above detailed description of the embodiments, those skilled in the art will clearly understand that the embodiments may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. Based on such understanding, the above technical solutions may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, where the storage medium includes a Read-Only Memory (ROM), a Random Access Memory (RAM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), a One-time Programmable Read-Only Memory (OTPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc-Read-Only Memory (CD-ROM), or other disk memories, CD-ROMs, or other magnetic disks, A tape memory, or any other medium readable by a computer that can be used to carry or store data.
Finally, it should be noted that: the quality evaluation method and apparatus for emergency management data disclosed in the embodiments of the present invention are only the preferred embodiments of the present invention, and are only used for illustrating the technical solutions of the present invention, not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art; the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method for quality assessment of emergency management data, the method comprising:
acquiring target emergency management data, and performing data preprocessing on the target emergency management data;
inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result; the evaluation benchmark parameters of the data quality evaluation model comprise one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data.
2. The method for quality assessment of emergency management data according to claim 1, further comprising:
judging whether the data quality evaluation result meets a preset data alarm rule or not to obtain a first judgment result;
and when the first judgment result is yes, sending the data quality evaluation result to a corresponding terminal according to a preset notification sending rule.
3. The method of claim 1, wherein the data preprocessing comprises one or more of data cleaning, data integration, data transformation, data desensitization, and data reduction.
4. The method for evaluating the quality of emergency management data according to claim 1, wherein the inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation to obtain a data quality evaluation result comprises:
calculating a parameter value of at least one evaluation reference parameter corresponding to the preprocessed target emergency management data;
multiplying the parameter value by the parameter weight corresponding to the parameter type of the parameter value to obtain an evaluation reference parameter result value;
and determining the sum of all evaluation reference parameter result values as a data quality evaluation result.
5. The method for evaluating the quality of emergency management data according to claim 4, wherein the calculating the parameter value of the at least one evaluation reference parameter corresponding to the preprocessed target emergency management data includes:
matching and calculating the missing data proportion and/or the invalid data proportion in the target emergency management data according to a preset data integrity template so as to calculate the integrity parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
according to a preset data specification rule, calculating the proportion of non-standard format data in the target emergency management data in a matching manner, so as to calculate the normative parameters corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
according to a preset data verification rule, calculating the error data proportion and/or the overdue data proportion in the target emergency management data in a matching manner, so as to calculate the accuracy parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
calculating the proportion of repeated data and/or the proportion of attribute repeated data in the target emergency management data so as to calculate the uniqueness parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
calculating a loss proportion and/or an unassociated data proportion of associated data in the target emergency management data according to a preset data association rule, so as to calculate an association parameter corresponding to the target emergency management data;
and/or the presence of a gas in the gas,
determining attacked information of a database corresponding to the target emergency management data, and determining security parameters corresponding to the target emergency management data according to the attacked information of the database;
and/or the presence of a gas in the gas,
determining the proportion of conflict data in the target emergency management data, and determining consistency parameters corresponding to the target emergency management data according to the proportion of the conflict data; the conflict data are data with conflicting information meanings in the target emergency data;
and/or the presence of a gas in the gas,
acquiring a plurality of instruction execution times corresponding to the target emergency management data in a historical time period, and determining timeliness parameters corresponding to the target emergency management data according to the instruction execution times; the instruction execution time is the time difference between the triggered time and the executed time of the triggered instruction when the target emergency management data in the historical time period accords with a preset emergency instruction triggering rule.
6. The method for quality assessment of emergency management data according to claim 1, wherein said data quality assessment model is:
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S;
wherein P is a data quality evaluation result, and I, N, C, A, U, R, S are the integrity parameter, the normative parameter, the consistency parameter, the accuracy parameter, the uniqueness parameter, the relevance parameter, and the security parameter corresponding to the target emergency management data, respectively; a is1-a7The integrity parameter, the normative parameter, the consistency parameter, the accuracy parameter, the uniqueness parameter, the relevance parameter and the parameter weight corresponding to the safety parameter corresponding to the target emergency management data are respectively weighted;
or the like, or, alternatively,
P=α1I+α2N+α3C+α4A+α5U+α6R+α7S+α8T;
wherein T is the target emergency management numberAccording to the corresponding timeliness parameter, a8And weighting the parameter corresponding to the timeliness parameter.
7. The method of claim 2, wherein the terminal comprises one or more of a data consumer terminal, a data provider terminal, or a data management server terminal.
8. The method for quality assessment of emergency management data according to claim 2, further comprising:
determining a notification response time of the terminal; the notification response time is a time difference between the time when the terminal receives the data quality evaluation result and the time when the terminal executes corresponding operation according to the data quality evaluation result;
and judging whether the notification response time of the terminal meets a preset punishment rule or not, and if so, executing corresponding punishment operation on the terminal.
9. An apparatus for evaluating the quality of emergency management data, comprising:
the system comprises a preprocessing module, a data preprocessing module and a data processing module, wherein the preprocessing module is used for acquiring target emergency management data and preprocessing the target emergency management data;
the evaluation module is used for inputting the preprocessed target emergency management data into a data quality evaluation model for evaluation so as to obtain a data quality evaluation result; the evaluation benchmark parameters of the data quality evaluation model comprise one or more of integrity parameters, normative parameters, consistency parameters, accuracy parameters, uniqueness parameters, relevance parameters, safety parameters and execution timeliness parameters of the target emergency management data.
10. An apparatus for quality assessment of emergency management data, the apparatus comprising:
a memory storing executable program code;
a processor coupled with the memory;
the processor calls the executable program code stored in the memory to execute the method for quality assessment of emergency management data according to any of claims 1 to 8.
CN202110623948.2A 2021-06-04 2021-06-04 Quality evaluation method and device for emergency management data Pending CN113379219A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110623948.2A CN113379219A (en) 2021-06-04 2021-06-04 Quality evaluation method and device for emergency management data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110623948.2A CN113379219A (en) 2021-06-04 2021-06-04 Quality evaluation method and device for emergency management data

Publications (1)

Publication Number Publication Date
CN113379219A true CN113379219A (en) 2021-09-10

Family

ID=77575745

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110623948.2A Pending CN113379219A (en) 2021-06-04 2021-06-04 Quality evaluation method and device for emergency management data

Country Status (1)

Country Link
CN (1) CN113379219A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114091842A (en) * 2021-10-29 2022-02-25 上海聚音信息科技有限公司 Commodity data quality evaluation method, commodity data replenishment method, commodity data quality evaluation apparatus, and storage medium
CN114742417A (en) * 2022-04-15 2022-07-12 北京科杰科技有限公司 Data quality evaluation method and device, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107730115A (en) * 2017-10-17 2018-02-23 云南大学 A kind of method for evaluating quality of the multi-source location track data based on AHP
CN109597805A (en) * 2018-11-07 2019-04-09 平安科技(深圳)有限公司 A kind of data processing method, electronic equipment and storage medium
CN111949647A (en) * 2020-09-03 2020-11-17 深圳市安亿通科技发展有限公司 Emergency management service data cleaning method, system, terminal and readable storage medium
CN112463773A (en) * 2019-09-06 2021-03-09 佛山市顺德区美的电热电器制造有限公司 Data quality determination method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107730115A (en) * 2017-10-17 2018-02-23 云南大学 A kind of method for evaluating quality of the multi-source location track data based on AHP
CN109597805A (en) * 2018-11-07 2019-04-09 平安科技(深圳)有限公司 A kind of data processing method, electronic equipment and storage medium
CN112463773A (en) * 2019-09-06 2021-03-09 佛山市顺德区美的电热电器制造有限公司 Data quality determination method and device
CN111949647A (en) * 2020-09-03 2020-11-17 深圳市安亿通科技发展有限公司 Emergency management service data cleaning method, system, terminal and readable storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114091842A (en) * 2021-10-29 2022-02-25 上海聚音信息科技有限公司 Commodity data quality evaluation method, commodity data replenishment method, commodity data quality evaluation apparatus, and storage medium
CN114742417A (en) * 2022-04-15 2022-07-12 北京科杰科技有限公司 Data quality evaluation method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN110275965B (en) False news detection method, electronic device and computer readable storage medium
CN113379219A (en) Quality evaluation method and device for emergency management data
CN112508311A (en) Data processing method and device for risk prediction
CN111143394B (en) Knowledge data processing method, device, medium and electronic equipment
Jain Real-time social network data mining for predicting the path for a disaster
CN114579653A (en) Data set-based CDA document generation method and device and electronic equipment
CN114219596A (en) Data processing method based on decision tree model and related equipment
CN111782803B (en) Work order processing method and device, electronic equipment and storage medium
CN112052346A (en) Method and device for updating real personnel library, electronic equipment and storage medium
CN112712369A (en) Method and device for monitoring suspicious transactions of anti-money laundering
US20220327147A1 (en) Method for updating information of point of interest, electronic device and storage medium
CN116109202A (en) Intelligent product quality safety supervision system, method and computer equipment
Neto et al. Disease surveillance big data platform for large scale event processing
CN112712270B (en) Information processing method, device, equipment and storage medium
CN111291029B (en) Data cleaning method and device
CN115330103A (en) Intelligent analysis method and device for urban operation state, computer equipment and storage medium
CN117035557B (en) Highway infrastructure digital degree evaluation method for business scene
Sato et al. A cross-cultural newspaper content analysis: smart cities in Japanese and slovenian newspapers
CN115269879B (en) Knowledge structure data generation method, data search method and risk warning method
CN115827606A (en) Data quality monitoring method and device
CN111858716B (en) Automatic flow processing technology for massive financial payment data
CN113379076A (en) Access network maintenance method, access network maintenance device and electronic equipment
CN116975305A (en) Information acquisition reminding method, device, equipment and storage medium
CN114298848A (en) Method, apparatus, device and computer readable medium for checking policy
CN115034736A (en) Service processing method and device, electronic equipment and computer readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210910

RJ01 Rejection of invention patent application after publication