CN111435483A - Data quality inspection method and device, storage medium and terminal - Google Patents
Data quality inspection method and device, storage medium and terminal Download PDFInfo
- Publication number
- CN111435483A CN111435483A CN201910033035.8A CN201910033035A CN111435483A CN 111435483 A CN111435483 A CN 111435483A CN 201910033035 A CN201910033035 A CN 201910033035A CN 111435483 A CN111435483 A CN 111435483A
- Authority
- CN
- China
- Prior art keywords
- data
- operator
- quality inspection
- quality
- proportion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06313—Resource planning in a project environment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A data quality inspection method and device, a storage medium and a terminal are provided, and the data quality inspection method comprises the following steps: acquiring data to be quality-checked, determining the data type of the data to be quality-checked and modifying an operator of the data to be quality-checked; dividing the data to be quality-tested into a plurality of data packets according to the operator and/or data type of the data to be quality-tested, and dividing the data to be quality-tested of different operators or data types into different data packets; determining the quality inspection proportion of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of an operator; and extracting the data in each data packet according to the quality inspection proportion of each data packet for auditing. The quality inspection proportion can be determined through the technical scheme of the invention, and the data auditing quality can be improved while the cost is reduced.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data quality inspection method and apparatus, a storage medium, and a terminal.
Background
At present, many products in the market have quality inspection links for data modification, and a sampling quality inspection mode is generally adopted in order to save cost.
In the prior art, sampling quality inspection is generally carried out by adopting a fixed proportion. Fixed sampling quality inspection ratios can be adopted according to different processing stages.
However, in the prior art, when the sampling inspection is performed according to the fixed sampling inspection ratio, the cost is high if the quality inspection ratio is too high, and the data quality is affected if the quality inspection ratio is too low.
Disclosure of Invention
The invention solves the technical problem of how to determine the quality inspection proportion so as to reduce the cost and improve the data auditing quality.
In order to solve the above technical problem, an embodiment of the present invention provides a data quality inspection method, where the data quality inspection method includes: acquiring data to be quality-checked, determining the data type of the data to be quality-checked and modifying an operator of the data to be quality-checked; dividing the data to be quality-tested into a plurality of data packets according to the operator and/or data type of the data to be quality-tested, and dividing the data to be quality-tested of different operators or data types into different data packets; determining the quality inspection proportion of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of an operator; and extracting the data in each data packet according to the quality inspection proportion of each data packet for auditing.
Optionally, the attribute information of the operator includes one or more of the following: the data packet modification operation type comprises a historical work accuracy of an operator, a professional level of the operator, a modification operation type of the operator for the data packet and a work time length of the operator for various data types.
Optionally, the determining the quality inspection ratio of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of the operator includes one or more of the following: judging the accuracy of the historical operation of the operator, wherein the higher the accuracy of the historical operation of the operator is, the lower the quality inspection proportion is; judging the professional grade of the operator, wherein the higher the professional grade of the operator is, the lower the quality inspection proportion is; judging the modification operation type of the operator, wherein if the modification operation type belongs to a preset operation type, the quality inspection proportion is a first proportion, otherwise, the quality inspection proportion is a second proportion, and the first proportion is higher than the second proportion; and judging the data type and the working time length of the operator for the data type, wherein the longer the working time length is, the lower the quality inspection proportion is.
Optionally, the determining the quality inspection ratio of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of the operator includes: determining the quality inspection proportion of the data packet according to the preset accuracy corresponding to the data type of the data to be inspected in the data packet, wherein the preset accuracy is positively correlated with the quality inspection proportion.
Optionally, the data to be quality-checked includes modified map data, reimbursement receipt and document data.
Optionally, the attribute information of the operator is determined according to historical statistical information of the operator, and the method further includes: and updating the historical statistical information of the operator according to the auditing result.
In order to solve the above technical problem, an embodiment of the present invention further discloses a data quality inspection apparatus, where the data quality inspection apparatus includes: the quality inspection data acquisition module is suitable for acquiring quality inspection data, determining the data type of the quality inspection data and modifying an operator of the quality inspection data; the data packet dividing module is suitable for dividing the data to be subjected to quality inspection into a plurality of data packets according to the operator and/or the data type of the data to be subjected to quality inspection, and dividing the data to be subjected to quality inspection of different operators or data types into different data packets; the quality inspection ratio determining module is suitable for determining the quality inspection ratio of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of an operator; and the auditing module is suitable for extracting the data in each data packet according to the quality inspection proportion of the data packet to audit.
Optionally, the attribute information of the operator includes one or more of the following: the data packet modification operation type comprises a historical work accuracy of an operator, a professional level of the operator, a modification operation type of the operator for the data packet and a work time length of the operator for various data types.
Optionally, the quality inspection ratio determining module includes one or more of the following units: a historical operation accuracy judging unit, adapted to judge the operator historical operation accuracy of the operator, wherein the higher the operator historical operation accuracy is, the lower the quality inspection ratio is; the professional grade judging unit is suitable for judging the professional grade of the operator, and the higher the professional grade of the operator is, the lower the quality inspection proportion is; the modification operation type judging unit is suitable for judging the modification operation type of the operator, if the modification operation type belongs to a preset operation type, the quality inspection proportion is a first proportion, and if not, the quality inspection proportion is a second proportion, and the first proportion is higher than the second proportion; and the working duration judging unit is suitable for judging the data type and the working duration of the operator for the data type, and the quality inspection proportion is lower as the working duration is longer.
Optionally, the quality inspection ratio determining module includes: the quality inspection proportion determining unit is suitable for determining the quality inspection proportion of the data packet according to the preset accuracy corresponding to the data type of the data to be inspected in the data packet, and the preset accuracy is positively correlated with the quality inspection proportion.
Optionally, the data to be quality-checked includes modified map data, reimbursement receipt and document data.
Optionally, the attribute information of the operator is determined according to historical statistical information of the operator, and the apparatus further includes: and the updating module is suitable for updating the historical statistical information of the operator according to the auditing result.
In order to solve the above technical problem, an embodiment of the present invention further discloses a storage medium, on which computer instructions are stored, and the computer instructions execute the steps of the data quality inspection method when running.
In order to solve the above technical problem, an embodiment of the present invention further discloses a terminal, including a memory and a processor, where the memory stores a computer instruction capable of running on the processor, and the processor executes the steps of the data quality inspection method when running the computer instruction.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
the technical scheme of the invention is that data to be quality-checked are obtained, the data type of the data to be quality-checked is determined, and an operator for modifying the data to be quality-checked is determined; dividing the data to be quality-tested into a plurality of data packets according to the operator and/or data type of the data to be quality-tested, and dividing the data to be quality-tested of different operators or data types into different data packets; determining the quality inspection proportion of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of an operator; and extracting the data in each data packet according to the quality inspection proportion of each data packet for auditing. According to the technical scheme, the quality inspection proportion can be dynamically adjusted according to the data to be inspected modified by different operators and/or the data to be inspected of different data types by combining the data types of the data to be inspected and the difference of the attribute information of the operators, so that a more appropriate quality inspection proportion can be determined, the cost is reduced, and the data auditing quality is improved.
Further, the attribute information of the operator includes one or more of: the data packet modification operation type comprises a historical work accuracy of an operator, a professional level of the operator, a modification operation type of the operator for the data packet and a work time length of the operator for various data types. The technical scheme of the invention combines different attribute information of operators, specifically combines the accuracy of the historical operation of the operators, the professional grade of the operators, the modification operation type of the operators for the data packet and the operation duration of the operators for various data types, and determines the quality inspection proportion as a consideration factor, thereby further ensuring the suitability of the quality inspection proportion, reducing the cost and improving the data auditing quality.
Drawings
FIG. 1 is a flow chart of a data quality inspection method according to an embodiment of the present invention;
FIG. 2 is a partial flow chart of a data quality inspection method according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a data quality inspection apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an embodiment of the quality inspection ratio determining module 303 shown in fig. 3.
Detailed Description
However, as described in the background art, when the prior art performs the sampling at a fixed sampling rate, if the quality inspection rate is too high, the cost is high, and if the quality inspection rate is too low, the data quality is affected.
According to the technical scheme, the quality inspection proportion can be dynamically adjusted according to the data to be inspected modified by different operators and/or the data to be inspected of different data types by combining the data types and the difference of the attribute information of the operators, so that a more appropriate quality inspection proportion can be determined, the cost is reduced, and the data auditing quality is improved.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
Fig. 1 is a flowchart of a data quality inspection method according to an embodiment of the present invention.
The data quality inspection method shown in fig. 1 may include the following steps:
step S101: acquiring data to be quality-checked, determining the data type of the data to be quality-checked and modifying an operator of the data to be quality-checked;
step S102: dividing the data to be quality-tested into a plurality of data packets according to the operator and/or data type of the data to be quality-tested, and dividing the data to be quality-tested of different operators or data types into different data packets;
step S103: determining the quality inspection proportion of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of an operator;
step S104: and extracting the data in each data packet according to the quality inspection proportion of each data packet for auditing.
In this embodiment, the data to be quality checked may be data that needs to be subjected to data quality audit. The data to be quality-checked can have different expression forms in different application scenes.
In a specific implementation of step S101, the data to be quality-tested may be obtained from a system that forms the data to be quality-tested. Map data modified by an operator may be acquired from an electronic map system, for example; the reimbursement documents passing the initial review can be obtained from the reimbursement system. And can determine the data type of the data to be inspected and the operator for modifying the data to be inspected. In particular, the data type may refer to an attribute to be inspected. More specifically, it may be real-time data such as bus lines, traffic conditions, etc.; but also static data such as hotel locations, restaurant locations, etc.
It is to be understood that the data type may be any other implementable type, and the embodiment of the present invention is not limited thereto.
In the specific implementation of step S102 and step S103, the data to be quality-tested may be divided into a plurality of data packets according to the operator of the data to be quality-tested, or the data to be quality-tested may be divided into a plurality of data packets according to the data type, or the data to be quality-tested may be divided into a plurality of data packets according to the operator of the data to be quality-tested and the data type.
The data to be inspected of different operators or different data types are divided into different data packets. Because different data packets have differences in operators or data types, after the quality inspection proportion of each data packet is determined according to the data type of the data to be inspected in the data packet and/or the attribute information of the operators, each data packet can have different quality inspection proportions.
Further, in the embodiment of step S104, the data may be extracted and checked according to the quality inspection ratio of the data packet. For example, if the quality inspection ratio of a packet is 60%, data of 60% of the total amount of data can be extracted from the packet.
According to the embodiment of the invention, by combining the data types of the data to be quality-inspected and the difference of the attribute information of the operators, the quality inspection proportion can be dynamically adjusted according to the data to be quality-inspected modified by different operators and/or the data to be quality-inspected of different data types, so that a more appropriate quality inspection proportion can be determined, the cost is reduced, and the data auditing quality is improved.
In a specific application scenario of the present invention, the data to be quality-checked may include modified map data. In the present embodiment, in the electronic map system, the map data needs to be updated and modified. In order to ensure the correctness of the modified data, quality inspection and verification need to be performed on the modified map data. In order to reduce the quality inspection cost and improve the quality of the audit, the quality inspection method shown in fig. 1 may be used to determine the quality inspection ratio of the modified map data and perform the audit.
In another specific application scenario of the present invention, the data to be quality-checked may include reimbursement documents and documentation. In this embodiment, in order to ensure the accuracy of the reimbursement document and the document data, quality inspection and verification are required to be performed on the reimbursement document and the document data. In order to reduce the quality inspection cost and improve the quality of the audit, the data quality inspection method shown in fig. 1 can be used to determine the quality inspection ratio of the reimbursed document and the document data, and the quality inspection ratio is checked.
In a preferred embodiment of the present invention, the attribute information of the operator includes one or more of the following: the data packet modification operation type comprises a historical work accuracy of an operator, a professional level of the operator, a modification operation type of the operator for the data packet and a work time length of the operator for various data types.
In a specific implementation, the attribute information of the operator may be obtained statistically in advance. Wherein, the historical operation accuracy rate of the operator refers to the percentage of correct data modified in the data which is checked by the operator; the operator specialty level may represent the specialty and proficiency of the operator; the modification operation type of the data packet by the operator refers to specific modification operations, such as updating, deleting, adding and the like; the length of a job by a worker for various data types refers to the time the worker takes to modify the data of the various data types.
The attribute information of the operator can influence the quantity of the data to be quality checked to be extracted, namely the quality check proportion. Therefore, when the quality inspection proportion is determined, the attribute information of the operator is considered in the determination process of the quality inspection proportion, so that the cost is reduced, and the data auditing quality is improved.
Further, referring to fig. 1 and fig. 2, step S103 shown in fig. 1 may include one or more of the following:
step S201: judging the accuracy of the historical operation of the operator, wherein the higher the accuracy of the historical operation of the operator is, the lower the quality inspection proportion is;
step S202: judging the professional grade of the operator, wherein the higher the professional grade of the operator is, the lower the quality inspection proportion is;
step S203: judging the modification operation type of the operator, wherein if the modification operation type belongs to a preset operation type, the quality inspection proportion is a first proportion, otherwise, the quality inspection proportion is a second proportion, and the first proportion is higher than the second proportion;
step S204: and judging the data type and the working time length of the operator for the data type, wherein the longer the working time length is, the lower the quality inspection proportion is.
In the present embodiment, the relationship between the attribute information of the operator and the quality control ratio quality control is described. And dividing the data to be inspected into a plurality of data packets according to the operator of the data to be inspected. After the data packet is divided, the operators of the data to be quality-checked in the same data packet are the same.
In the specific implementation of step S201, the operator history job accuracy of the operator is determined, and the higher the operator history job accuracy is, the lower the quality inspection ratio is. In other words, the higher the accuracy of the historical operation of the operator is, the higher the accuracy of the data modified by the operator is, so that less data to be inspected can be extracted for auditing, that is, a lower quality inspection ratio is determined.
In the specific implementation of step S202, the higher the professional level of the operator is, the higher the professional level and the proficiency level of the operator are, the lower the error rate of the operator is, so that less data to be inspected can be extracted for auditing, that is, the lower quality inspection ratio can be determined.
In the implementation of step S203, the preset modification operation type has a set relationship with the quality inspection ratio. Specifically, the data error rate of the preset operation type is higher, so that more data to be quality inspected can be extracted for auditing, that is, a higher quality inspection ratio is determined. In this case, the quality inspection ratio is a first ratio if the modification operation type belongs to the preset operation type, and is a second ratio otherwise, and the first ratio is higher than the second ratio.
Or the data error rate of the preset operation type is lower, so that less data to be inspected can be extracted for auditing, namely, a smaller quality inspection ratio is determined. In this case, the quality inspection ratio is a first ratio if the modification operation type belongs to the preset operation type, and is a second ratio otherwise, and the first ratio is lower than the second ratio.
In the specific implementation of step S204, since the job time lengths of the operators for different types of data are different, the data type and the job time length of the operator for the data type can be determined. Longer operation time indicates that the longer the time taken by the operator to modify the data type, the lower the error rate of the modification, so a lower quality control ratio can be determined.
It should be noted that the specific calculation relationship between the quality inspection ratio and the accuracy of the historical work of the operator, the professional level of the operator, the preset modification operation type, or the work duration may be configured according to the actual application environment, which is not limited in the embodiment of the present invention.
In another preferred embodiment of the present invention, step S103 shown in fig. 1 may include the steps of: determining the quality inspection proportion of the data packet according to the preset accuracy corresponding to the data type of the data to be inspected in the data packet, wherein the preset accuracy is positively correlated with the quality inspection proportion.
And dividing the data to be subjected to quality inspection into a plurality of data packets according to the data type of the data to be subjected to quality inspection. And after the data packets are divided, the data types of the data to be quality-tested in the same data packet are the same.
In this embodiment, different types of data to be quality-checked can correspond to different preset accuracy rates. The preset accuracy rate represents the accuracy requirement on the modified data to be inspected. The higher the preset accuracy of the data to be quality-tested is, the higher the quality-testing proportion of the data to be quality-tested is. That is, the higher the preset accuracy, the more data to be quality-checked is to be extracted.
It should be understood by those skilled in the art that the specific value of the preset accuracy may be adaptively configured according to an actual application scenario, and the embodiment of the present invention is not limited thereto.
In another preferred embodiment of the present invention, the data to be quality-checked is divided into a plurality of data packets according to the operator and the data type of the data to be quality-checked. After the data packets are divided, the operators and the data types of the data to be quality-checked in the same data packet are the same.
And when the quality inspection proportion is determined, the quality inspection proportion can be determined according to the accuracy of the historical operation of the operator, the professional grade of the operator, the preset modification operation type or the operation duration and the preset accuracy corresponding to the data type. For the specific implementation, reference may be made to the foregoing embodiments, which are not described in detail herein.
In another preferred embodiment of the present invention, the attribute information of the operator is determined according to historical statistical information of the operator, and the data quality inspection method shown in fig. 1 may further include the following steps: and updating the historical statistical information of the operator according to the auditing result.
In this embodiment, the attribute information of the operator is determined based on the historical statistical information of the operator. After the audit is completed, the audit result can be updated to the historical statistical information of the operator, so that the attribute information of the operator can be updated, the quality inspection proportion determined in the next data quality inspection process is more accurate, the cost is further reduced, and the data audit quality is improved.
Fig. 3 is a schematic structural diagram of a data quality inspection apparatus according to an embodiment of the present invention.
The data quality inspection device 30 may include a data to be inspected acquisition module 301, a data packet dividing module 302, a quality inspection ratio determination module 303, and an auditing module 304.
The data to be quality-tested acquisition module 301 is adapted to acquire data to be quality-tested, determine the data type of the data to be quality-tested, and modify an operator of the data to be quality-tested; the data packet dividing module 302 is adapted to divide the data to be quality-tested into a plurality of data packets according to the operator and/or the data type of the data to be quality-tested, and divide the data to be quality-tested of different operators or data types into different data packets; the quality inspection ratio determining module 303 is adapted to determine the quality inspection ratio of each data packet according to the data type of the data to be quality inspected in each data packet and/or the attribute information of the operator; the auditing module 304 is adapted to extract the data in each data packet for auditing according to the quality inspection ratio of the data packet.
According to the embodiment of the invention, by combining the data types and the difference of the attribute information of the operators, the quality inspection proportion can be dynamically adjusted according to the data to be inspected modified by different operators and/or the data to be inspected of different data types, so that a more appropriate quality inspection proportion can be determined, the cost is reduced, and the data auditing quality is improved
Preferably, the attribute information of the operator includes one or more of: the data packet modification operation type comprises a historical work accuracy of an operator, a professional level of the operator, a modification operation type of the operator for the data packet and a work time length of the operator for various data types.
Further, as shown in fig. 4, the quality inspection ratio determining module 303 shown in fig. 3 may include one or more of the following units:
a historical operation accuracy determining unit 3031, adapted to determine the operator historical operation accuracy of the operator, wherein the higher the operator historical operation accuracy is, the lower the quality inspection ratio is;
a professional grade determination unit 3032 adapted to determine a professional grade of the operator, wherein the higher the professional grade of the operator is, the lower the quality inspection ratio is;
a modification operation type determining unit 3033, adapted to determine a modification operation type of the operator, wherein if the modification operation type belongs to a preset operation type, the quality inspection ratio is a first ratio, otherwise, the quality inspection ratio is a second ratio, and the first ratio is higher than the second ratio;
the working duration judging unit 3034 is adapted to judge the data type and the working duration of the operator for the data type, wherein the longer the working duration is, the lower the quality inspection proportion is.
In another preferred embodiment of the present invention, the quality inspection ratio determining module 303 shown in fig. 3 may include a quality inspection ratio determining unit (not shown) adapted to determine a quality inspection ratio of the data packet according to a preset accuracy corresponding to a data type of data to be quality inspected in the data packet, where the preset accuracy is positively correlated with the quality inspection ratio.
In another preferred embodiment of the present invention, the attribute information of the operator is determined according to historical statistical information of the operator, and the data quality inspection apparatus 30 may further include: and the updating module (not shown) is suitable for updating the historical statistical information of the operator according to the auditing result.
For more details of the operation principle and the operation mode of the data quality inspection device 30, reference may be made to the description in fig. 1 to fig. 2, and details are not repeated here.
The embodiment of the invention also discloses a storage medium, wherein computer instructions are stored on the storage medium, and when the computer instructions are operated, the steps of the data quality inspection method shown in the figure 1 or the figure 2 can be executed. The storage medium may include ROM, RAM, magnetic or optical disks, etc. The storage medium may further include a non-volatile memory (non-volatile) or a non-transitory memory (non-transient), and the like.
The embodiment of the invention also discloses a terminal which can comprise a memory and a processor, wherein the memory is stored with computer instructions capable of running on the processor. The processor, when executing the computer instructions, may perform the steps of the data quality inspection method shown in fig. 1 or fig. 2. The terminal includes, but is not limited to, a mobile phone, a computer, a tablet computer and other terminal devices.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A method for data quality inspection, comprising:
acquiring data to be quality-checked, determining the data type of the data to be quality-checked and modifying an operator of the data to be quality-checked;
dividing the data to be quality-tested into a plurality of data packets according to the operator and/or data type of the data to be quality-tested, and dividing the data to be quality-tested of different operators or data types into different data packets;
determining the quality inspection proportion of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of an operator;
and extracting the data in each data packet according to the quality inspection proportion of each data packet for auditing.
2. The data quality inspection method according to claim 1, wherein the attribute information of the operator includes one or more of: the data packet modification operation type comprises a historical work accuracy of an operator, a professional level of the operator, a modification operation type of the operator for the data packet and a work time length of the operator for various data types.
3. The data quality inspection method according to claim 2, wherein the determining of the quality inspection ratio of each data packet according to the data type of the data to be quality inspected and/or the attribute information of the operator in each data packet comprises one or more of the following:
judging the accuracy of the historical operation of the operator, wherein the higher the accuracy of the historical operation of the operator is, the lower the quality inspection proportion is;
judging the professional grade of the operator, wherein the higher the professional grade of the operator is, the lower the quality inspection proportion is;
judging the modification operation type of the operator, wherein if the modification operation type belongs to a preset operation type, the quality inspection proportion is a first proportion, otherwise, the quality inspection proportion is a second proportion, and the first proportion is higher than the second proportion;
and judging the data type and the working time length of the operator for the data type, wherein the longer the working time length is, the lower the quality inspection proportion is.
4. The data quality inspection method according to claim 1, wherein the determining the quality inspection ratio of each data packet according to the data type of the data to be quality inspected and/or the attribute information of the operator in each data packet comprises:
determining the quality inspection proportion of the data packet according to the preset accuracy corresponding to the data type of the data to be inspected in the data packet, wherein the preset accuracy is positively correlated with the quality inspection proportion.
5. The data quality inspection method according to any one of claims 1 to 4, wherein the attribute information of the operator is determined based on historical statistical information of the operator, the method further comprising:
and updating the historical statistical information of the operator according to the auditing result.
6. A data quality inspection apparatus, comprising:
the quality inspection data acquisition module is suitable for acquiring quality inspection data, determining the data type of the quality inspection data and modifying an operator of the quality inspection data;
the data packet dividing module is suitable for dividing the data to be subjected to quality inspection into a plurality of data packets according to the operator and/or the data type of the data to be subjected to quality inspection, and dividing the data to be subjected to quality inspection of different operators or data types into different data packets;
the quality inspection ratio determining module is suitable for determining the quality inspection ratio of each data packet according to the data type of the data to be inspected in each data packet and/or the attribute information of an operator;
and the auditing module is suitable for extracting the data in each data packet according to the quality inspection proportion of the data packet to audit.
7. The data quality inspection device of claim 6, wherein the attribute information of the operator includes one or more of: the data packet modification operation type comprises a historical work accuracy of an operator, a professional level of the operator, a modification operation type of the operator for the data packet and a work time length of the operator for various data types.
8. The data quality inspection device of claim 7, wherein the quality inspection ratio determination module comprises one or more of the following:
a historical operation accuracy judging unit, adapted to judge the operator historical operation accuracy of the operator, wherein the higher the operator historical operation accuracy is, the lower the quality inspection ratio is;
the professional grade judging unit is suitable for judging the professional grade of the operator, and the higher the professional grade of the operator is, the lower the quality inspection proportion is;
the modification operation type judging unit is suitable for judging the modification operation type of the operator, if the modification operation type belongs to a preset operation type, the quality inspection proportion is a first proportion, and if not, the quality inspection proportion is a second proportion, and the first proportion is higher than the second proportion;
and the working duration judging unit is suitable for judging the data type and the working duration of the operator for the data type, and the quality inspection proportion is lower as the working duration is longer.
9. A storage medium having stored thereon computer instructions, wherein the computer instructions when executed perform the steps of the data quality inspection method of any one of claims 1 to 5.
10. A terminal comprising a memory and a processor, the memory having stored thereon computer instructions executable on the processor, wherein the processor, when executing the computer instructions, performs the steps of the data quality inspection method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910033035.8A CN111435483A (en) | 2019-01-14 | 2019-01-14 | Data quality inspection method and device, storage medium and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910033035.8A CN111435483A (en) | 2019-01-14 | 2019-01-14 | Data quality inspection method and device, storage medium and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111435483A true CN111435483A (en) | 2020-07-21 |
Family
ID=71580593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910033035.8A Pending CN111435483A (en) | 2019-01-14 | 2019-01-14 | Data quality inspection method and device, storage medium and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111435483A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114240139A (en) * | 2021-12-15 | 2022-03-25 | 高德软件有限公司 | Data processing method and device, electronic equipment and computer program product |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956940A (en) * | 2016-06-02 | 2016-09-21 | 广东电网有限责任公司 | Electric power service hotline quality inspection sampling method and system |
| CN107798461A (en) * | 2017-09-15 | 2018-03-13 | 平安科技（深圳）有限公司 | Attend a banquet monitoring method, device, equipment and computer-readable recording medium |
-
2019
- 2019-01-14 CN CN201910033035.8A patent/CN111435483A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956940A (en) * | 2016-06-02 | 2016-09-21 | 广东电网有限责任公司 | Electric power service hotline quality inspection sampling method and system |
| CN107798461A (en) * | 2017-09-15 | 2018-03-13 | 平安科技（深圳）有限公司 | Attend a banquet monitoring method, device, equipment and computer-readable recording medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114240139A (en) * | 2021-12-15 | 2022-03-25 | 高德软件有限公司 | Data processing method and device, electronic equipment and computer program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102129425B (en) | The access method of big object set table and device in data warehouse | |
| JP2019512126A (en) | Method and system for training a machine learning system | |
| CN105183669A (en) | Data storage method and device | |
| CN110928802A (en) | Test method, device, equipment and storage medium based on automatic generation of case | |
| CN109981715B (en) | Session management method and device | |
| CN110019367B (en) | Method and device for counting data characteristics | |
| CN104820663A (en) | Method and device for discovering low performance structural query language (SQL) statements, and method and device for forecasting SQL statement performance | |
| CN110750443A (en) | Webpage testing method and device, computer equipment and storage medium | |
| JP2019215847A (en) | Reducing buffer overflow | |
| CN114564390A (en) | Performance test method, device, equipment and product of application program | |
| CN106649210A (en) | Data conversion method and device | |
| US20180157797A1 (en) | Cost of healthcare analytics platform | |
| CN113128168A (en) | Pad parameter checking and correcting method and device, computer equipment and storage medium | |
| CN111435483A (en) | Data quality inspection method and device, storage medium and terminal | |
| CN113032258A (en) | Electronic map testing method and device, electronic equipment and storage medium | |
| US20170083531A1 (en) | Selecting an incremental backup approach | |
| CN113093702B (en) | Fault data prediction method and device, electronic equipment and storage medium | |
| CN115865753A (en) | Data processing method and device and electronic equipment | |
| CN106326310B (en) | Resource encryption updating method for mobile phone client software | |
| CN114416895A (en) | Map data processing method and device, electronic equipment and storage medium | |
| CN113051882A (en) | Data difference comparison method, device, medium and product | |
| CN113449062A (en) | Track processing method and device, electronic equipment and storage medium | |
| US9256626B2 (en) | Maintaining multiple copy versions of component values in a system | |
| CN113886745B (en) | Page picture testing method and device and electronic equipment | |
| CN112363927B (en) | Code testing method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |