CN111897803A - Database integrity evaluation method for power industry business system - Google Patents
Database integrity evaluation method for power industry business system Download PDFInfo
- Publication number
- CN111897803A CN111897803A CN202010826563.1A CN202010826563A CN111897803A CN 111897803 A CN111897803 A CN 111897803A CN 202010826563 A CN202010826563 A CN 202010826563A CN 111897803 A CN111897803 A CN 111897803A
- Authority
- CN
- China
- Prior art keywords
- database
- field
- data table
- values
- full
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a database integrity evaluation method for a power industry service system, which comprises the steps of reading a first field from a first function table, reading a second field and a third field from a second function table, and establishing an affiliated relationship between a data table and the fields according to the second field; reading a data table from the database, and accumulating the values of the third field to obtain the number of the none values; calculating the number of none values and the number of full field records for removing all empty fields; and traversing all the data tables, accumulating the obtained data index values, calculating each index value, and evaluating the data integrity of the database. The method takes a database system table as the basis of evaluation, and evaluates the integrity of the database; in the evaluation process, the reading frequency and the workload of the database service table are reduced, and the evaluation efficiency is improved; when the attributes such as the blank space, the empty column and the empty row of the data table are searched, the method has higher discovery rate and accuracy.
Description
Technical Field
The embodiment of the invention relates to the field of computer data processing, in particular to a database integrity evaluation method for a power industry business system.
Background
For the research of the data integrity evaluation method, the key point is to solve the problem of relation data integrity measurement. Most of existing data integrity evaluation methods are implemented by providing an abstract data integrity measurement framework, then realizing the influence of function dependence on data integrity on the basis of the framework, and providing a concrete relational data integrity measurement index and a system scheme. Under the current environment, the emphasis of research on data integrity focuses on definition, assurance technology, assurance model and application thereof, while the research on data integrity evaluation methods mostly focuses on verification of data, which is very slow in working efficiency for power industry business systems with huge data volume.
The following problems exist for database integrity assessment:
1) to evaluate database integrity without quantification;
2) with the rapid increase of the data volume, the integrity evaluation according to the database has low efficiency and high cost;
3) no specific database integrity evaluation method is given.
Disclosure of Invention
According to the embodiment of the invention, a database integrity evaluation scheme oriented to a power industry business system is provided.
In a first aspect of the invention, a database integrity evaluation method for a power industry service system is provided. The method comprises the following steps:
reading a first field from a first function table, wherein the first field is the total number of records contained in a data table; reading a second field and a third field from the second function table, and establishing an affiliation relationship between the data table and the fields according to the second field; the second field represents the name of the data table to which the field visible to the user belongs; the third field is a null value number contained in each field in the data table;
reading a data table from a database, and accumulating the values of a third field of the data table to obtain the number of none values of the data table; calculating the number of none values and the number of full field records for removing all empty fields;
traversing all data tables in the database, accumulating the obtained data index values, calculating the proportion of the full empty fields of the database, the recording proportion of the full fields of the database, the proportion of the none values of the database and the proportion of the none values of the removed full empty fields of the database, and evaluating the data integrity of the database.
Further, the calculating the number of none values of the removed all-empty fields includes:
Q＝P-M*N
q is the number of none values of all empty fields removed from the data table; p is the number of none values in the data table; m is the number of all-empty fields in the data table; and N is the total number of records in the data table.
Further, the full field record number is:
K＝N-F
k is the number of full field records in the data table; n is the total number of records in the data table; f is the number of records of the non-full field in the data table; and the number of the records of the non-full field is the total number of the records in the field containing the null value in the data table.
Further, the accumulating the obtained data values includes:
accumulating the number of fields of each data table to obtain the total number of fields of the database;
accumulating the product of the number of fields of each data table and the number of records to obtain the total number of the numerical values of the database;
accumulating the total record number of each data table to obtain the total record number of the database;
accumulating the number of none values of each data table to obtain the number of the none values of the database;
accumulating the number of none values of all removed empty fields of each data table to obtain the number of all removed empty field none values of the database;
accumulating the full field record number of each data table to obtain the full field record number of the database;
and accumulating the numerical values of all the data tables after all the empty fields are removed to obtain the total number of the numerical values of the database after all the empty fields are removed.
Further, the values after removing the all-blank field are:
H＝(R-M)*N
h is a numerical value of the data table after all empty fields are removed; r is the number of fields in the data table; m is the number of all-empty fields in the data table; and N is the total number of records in the data table.
Further, the calculating the proportion of the fully empty fields of the database, the proportion of the full field records of the database, the proportion of the none values of the database and the proportion of the none values of the removed fully empty fields of the database comprises:
the proportion of the database full-empty fields is the ratio of the number of the database full-empty fields to the total number of the database fields;
the database full field record proportion is the ratio of the database full field record number to the total database record number;
the proportion of the database none values is the ratio of the number of the database none values to the total number of the database values;
the proportion of the none values of the database without the full-empty fields is the ratio of the number of the none values of the database without the full-empty fields to the total number of the numerical values of the database without the full-empty fields.
Further, the evaluating the data integrity of the database includes:
the data integrity of the database is in negative correlation with the proportion of the full empty fields of the database, the proportion of the none values of the database and the proportion of the none values of the removed full empty fields of the database;
the data integrity of the database is positively correlated with the full field record proportion of the database.
In a second aspect of the present invention, a database integrity evaluation device for a power industry business system is provided. The device includes:
the reading module is used for reading a first field from the first function table, reading a second field and a third field from the second function table, and establishing the affiliated relationship between the data table and the fields according to the second field; the first field is the total record number contained in a data table; the second field represents the name of the data table to which the field visible to the user belongs; the third field is a null value number contained in each field in the data table;
the first accumulation calculation module is used for reading a data table from a database, accumulating the value of a third field of the data table and obtaining the number of the none values of the data table; calculating the number of none values and the number of full field records for removing all empty fields;
the second accumulation calculation module is used for traversing all data tables in the database, accumulating the obtained data index values, and calculating the proportion of the full-empty fields of the database, the recording proportion of the full fields of the database, the proportion of the none values of the database and the proportion of the none values of the removed full-empty fields of the database;
and the evaluation module is used for evaluating the data integrity of the database.
In a third aspect of the invention, an electronic device is provided. The electronic device includes: a memory having a computer program stored thereon and a processor implementing the method as described above when executing the program.
In a fourth aspect of the invention, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the method as according to the first aspect of the invention.
It should be understood that the statements herein reciting aspects are not intended to limit the critical or essential features of any embodiment of the invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.
The invention evaluates the rationality and integrity of the database by analyzing the proportion of the number of records with null attributes in each data table to the total number of records and the proportion of the number of attributes with null values in the table to the total number of attributes, and has higher discovery rate and accuracy when searching the attributes of blank spaces, null columns, null rows and the like of the data tables.
Drawings
The above and other features, advantages and aspects of various embodiments of the present invention will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements, and wherein:
fig. 1 shows a flow chart of a database integrity evaluation method for an electric power industry business system according to an embodiment of the invention;
fig. 2 shows a block diagram of a database integrity evaluation apparatus for an electric power industry business system according to an embodiment of the present invention;
FIG. 3 illustrates a block diagram of an exemplary electronic device capable of implementing embodiments of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
In addition, the term "and/or" herein is only one kind of association relationship describing an associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
In the invention, through a dbms _ stats module of an Oracle database, the basic information of the database is counted, the basic information is processed and processed, the data table is detected, and when the attributes such as spaces, empty columns, empty rows and the like of the data table are searched, the method has higher discovery rate and accuracy, analyzes the proportion of the number of records with empty attributes in the table to the total number of records and the proportion of the number of attributes with empty values in the table to the total number of attributes, calculates the data integrity of the data table, and accumulates the evaluation values of the data table, thereby evaluating the rationality and the integrity of the database.
Fig. 1 shows a flowchart of a database integrity evaluation method for a power industry business system according to an embodiment of the present invention.
The method comprises the following steps:
s110, reading a first field from a first function table, wherein the first field is the total record number contained in a data table; reading a second field and a third field from the second function table, and establishing an affiliation relationship between the data table and the fields according to the second field; the second field represents the name of the data table to which the field visible to the user belongs; the third field is the number of null values contained in each field in the data table.
As an embodiment of the present invention, a dbms _ stats function module is called in an Oracle database, and after the dbms _ stats function module is called, a first field is read from a first function table, such as a user _ tables table, which represents a current user visible table; the first field is, for example, a num _ row field, which indicates the total number of records contained in the data table. The second function table, for example, a user _ tab _ columns table, indicates the fields visible to the current user, that is, the fields visible to the current user; a second field, for example, a table _ name field, indicating the name of the data table to which the field visible to the user belongs; a third field, such as a num _ null field, indicates the number of null values contained in each field of the data table. And establishing the relationship between the data table and the data field according to the table _ name field.
In the embodiment, the dbms _ stats functional module carried by Oracle is utilized to realize the basic data collection of the data table, the difficulty and the workload of the development of the integrity method are reduced, and the accuracy is improved.
And S120, reading a data table from the database, and accumulating the values of the third field of the data table to obtain the number of the none values of the data table.
The data table read from the database is represented as an A table, and the values of num _ null fields in the A table are accumulated to obtain the number of none values of the A table; a num _ null field is a null value number contained in each field of the a table, for example, if num _ null is equal to num _ row, the field is indicated to be a completely null field, where num _ row is a total record number contained in the a table; if num _ null is 0, it indicates that there is no null value in this field, which is a full data field. The number of none values is the cumulative sum of the num _ null values of the fields in the a table.
And S130, calculating the number of none values of all empty fields.
The number of none values of the removed all-empty fields is the number of none values of an A table-the number of all-empty fields of the A table-the total number of records of the A table, namely the number Q of none values of the removed all-empty fields is as follows:
Q＝P-M*N
q is the number of none values of all empty fields removed from the data table; p is the number of none values in the data table; m is the number of all-empty fields in the data table; and N is the total number of records in the data table.
And S140, calculating the full field record number.
The full field record number is calculated, and the following logic is realized through an sql statement: set of precedent instructions { a1,a2,...,anAnd (4) respectively judging whether each set element is a none or not for each field of the A table, if so, counting the number of records by +1, and traversing the whole set to obtain the number of records of the non-full field. The calculation method is as follows:
K＝N-F
k is the number of full field records in the data table; n is the total number of records in the data table; f is the number of records of the non-full field in the data table; and the number of the records of the non-full field is the total number of the records in the field containing the null value in the data table.
And S110 to S140, counting the basic information of a single data table through a dbms _ stats module of the Oracle database, and counting to obtain data index values such as the number of fields, the number of records, the number of all-empty fields, the number of none values without all-empty fields, the number of records of full fields and the like of the data table.
S150, traversing all data tables in the database, and accumulating the obtained data index values.
As an embodiment of the present invention, the total data index value of the entire database is obtained by traversing all the data tables in the database and accumulating the data index values of each data table.
The method specifically comprises the following steps:
accumulating the number of the fields of each table and multiplying the number of records to obtain the total number of the numerical values of the database;
accumulating the number of the fields of each table to obtain the total number of the fields of the database;
accumulating the total record number of each table to obtain the total record number of the database;
accumulating the number of all null fields of each table to obtain the total number of all null fields of the database;
accumulating the number of the none values of each table to obtain the number of the none values of the database;
accumulating the number of none values of all removed empty fields of each table to obtain the number of the none values of all removed empty fields of the database;
accumulating the full field record number of each table to obtain the full field record number of the database;
accumulating the number of records in each table (field number-full empty field number) to obtain the total number of the values of the database after the full empty fields are removed, namely the values after the full empty fields are removed are as follows:
H＝(R-M)*N
h is a numerical value of the data table after all empty fields are removed; r is the number of fields in the data table; m is the number of all-empty fields in the data table; and N is the total number of records in the data table.
And S160, calculating the proportion of the full empty fields of the database, the proportion of the full fields of the database, the proportion of the none values of the database and the proportion of the none values of the removed full empty fields of the database, and evaluating the data integrity of the database.
The proportion of the all-empty fields of the database is the number of all-empty fields/the total number of the fields of the database;
the full field record proportion of the database is the total number of full field records/the total number of database records;
the proportion of the database none values is the number of the none values/the total number of the database values;
the proportion of the none value of the database without the full-empty field is the number of the none values after the full-empty field is removed/the total number of the numerical values of the database without the full-empty field.
Further, 170, the integrity of the database can be evaluated through the four evaluation indexes, including:
the data integrity of the database is in negative correlation with the proportion of the all-empty field of the database, namely the higher the proportion of the all-empty field of the database is, the worse the integrity of the database is; conversely, the lower the proportion of the full fields of the database is, the better the integrity of the database is.
The data integrity of the database is in negative correlation with the proportion of the database none value, namely the higher the proportion of the database none value is, the worse the integrity of the database is; conversely, the lower the proportion of the database none values, the better the integrity of the database.
The data integrity of the database is in negative correlation with the proportion of the none value of the database with the removed full-empty field, namely the higher the proportion of the none value of the database with the removed full-empty field is, the worse the integrity of the database is; conversely, the lower the proportion of the none value of the database excluding the all-empty field is, the better the integrity of the database is.
The data integrity of the database is positively correlated with the full field record proportion of the database; namely, the higher the full field record proportion of the database is, the better the integrity of the database is; conversely, the lower the full field record proportion of the database, the worse the integrity of the database.
The integrity of the database can be evaluated from the perspective of quantitative evaluation of data quality through the index evaluation.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are exemplary embodiments and that the acts and modules illustrated are not necessarily required to practice the invention.
The above is a description of method embodiments, and the embodiments of the present invention are further described below by way of apparatus embodiments.
As shown in fig. 2, the apparatus 200 includes:
a reading module 201, configured to read a first field from a first function table, read a second field and a third field from a second function table, and establish an affiliation relationship between a data table and the fields according to the second field; the first field is the total record number contained in a data table; the second field represents the name of the data table to which the field visible to the user belongs; the third field is a null value number contained in each field in the data table;
the first accumulation calculation module 202 is configured to read a data table from a database, accumulate values of a third field of the data table, and obtain a number of none values of the data table; calculating the number of none values and the number of full field records for removing all empty fields;
further, the method also includes a first calculating module, configured to calculate the number of none values of the removed all-blank fields, including:
Q＝P-M*N
q is the number of none values of all empty fields removed from the data table; p is the number of none values in the data table; m is the number of all-empty fields in the data table; and N is the total number of records in the data table.
Further, the system also comprises a second calculating module, configured to calculate the number of full field records, including:
K＝N-F
k is the number of full field records in the data table; n is the total number of records in the data table; f is the number of records of the non-full field in the data table; and the number of the records of the non-full field is the total number of the records in the field containing the null value in the data table.
And the second accumulation calculation module 203 is used for traversing all the data tables in the database, accumulating the obtained data index values, and calculating the proportion of the full-empty fields of the database, the proportion of the full-field records of the database, the proportion of the none values of the database and the proportion of the none values of the removed full-empty fields of the database.
Further, the second accumulation calculating module 203 further includes a first accumulation module, configured to accumulate the data index values of each data table to obtain a total data index value of the entire database, and specifically includes:
accumulating the number of fields of each data table to obtain the total number of fields of the database;
accumulating the product of the number of fields of each data table and the number of records to obtain the total number of the numerical values of the database;
accumulating the total record number of each data table to obtain the total record number of the database;
accumulating the number of none values of each data table to obtain the number of the none values of the database;
accumulating the number of none values of all removed empty fields of each data table to obtain the number of all removed empty field none values of the database;
accumulating the full field record number of each data table to obtain the full field record number of the database;
accumulating the numerical values of all the data tables after all the empty fields are removed to obtain the total number of the numerical values of the database after all the empty fields are removed; namely, the value after removing the all-blank field is:
H＝(R-M)*N
h is a numerical value of the data table after all empty fields are removed; r is the number of fields in the data table; m is the number of all-empty fields in the data table; and N is the total number of records in the data table.
The second accumulation calculating module 203 further includes a third calculating module, configured to calculate a proportion of all empty fields in the database, a proportion of full field records in the database, a proportion of none values in the database, and a proportion of none values in all empty fields removed in the database, including:
the proportion of the database full-empty fields is the ratio of the number of the database full-empty fields to the total number of the database fields;
the database full field record proportion is the ratio of the database full field record number to the total database record number;
the proportion of the database none values is the ratio of the number of the database none values to the total number of the database values;
the proportion of the none values of the database without the full-empty fields is the ratio of the number of the none values of the database without the full-empty fields to the total number of the numerical values of the database without the full-empty fields.
An evaluation module 204, configured to evaluate data integrity of the database; the method specifically comprises the following steps:
the data integrity of the database is in negative correlation with the proportion of the full empty fields of the database, the proportion of the none values of the database and the proportion of the none values of the removed full empty fields of the database;
the data integrity of the database is positively correlated with the full field record proportion of the database.
The method specifically comprises the following steps:
the data integrity of the database is in negative correlation with the proportion of the all-empty field of the database, namely the higher the proportion of the all-empty field of the database is, the worse the integrity of the database is; conversely, the lower the proportion of the full fields of the database is, the better the integrity of the database is.
The data integrity of the database is in negative correlation with the proportion of the database none value, namely the higher the proportion of the database none value is, the worse the integrity of the database is; conversely, the lower the proportion of the database none values, the better the integrity of the database.
The data integrity of the database is in negative correlation with the proportion of the none value of the database with the removed full-empty field, namely the higher the proportion of the none value of the database with the removed full-empty field is, the worse the integrity of the database is; conversely, the lower the proportion of the none value of the database excluding the all-empty field is, the better the integrity of the database is.
The data integrity of the database is positively correlated with the full field record proportion of the database; namely, the higher the full field record proportion of the database is, the better the integrity of the database is; conversely, the lower the full field record proportion of the database, the worse the integrity of the database.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the described module may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
As shown in fig. 3, the electronic device includes a Central Processing Unit (CPU) that can perform various appropriate actions and processes according to computer program instructions stored in a Read Only Memory (ROM) or computer program instructions loaded from a storage unit into a Random Access Memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input/output (I/O) interface is also connected to the bus.
A plurality of components in an electronic device are connected to an I/O interface, including: an input unit such as a keyboard, a mouse, etc.; an output unit such as various types of displays, speakers, and the like; storage units such as magnetic disks, optical disks, and the like; and a communication unit such as a network card, modem, wireless communication transceiver, etc. The communication unit allows the electronic device to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The processing unit executes the respective methods and processes described above, such as the methods S110 to S160. For example, in some embodiments, methods S110-S160 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device via ROM and/or the communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more of the steps of methods S110-S160 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to perform methods S110-S160 in any other suitable manner (e.g., by way of firmware).
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a load programmable logic device (CPLD), and the like.
Program code for implementing the methods of the present invention may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Further, while operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the invention. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims (10)
1. A database integrity evaluation method for a power industry service system is characterized by comprising the following steps:
reading a first field from a first function table, wherein the first field is the total number of records contained in a data table; reading a second field and a third field from the second function table, and establishing an affiliation relationship between the data table and the fields according to the second field; the second field represents the name of the data table to which the field visible to the user belongs; the third field is a null value number contained in each field in the data table;
reading a data table from a database, and accumulating the values of a third field of the data table to obtain the number of none values of the data table; calculating the number of none values and the number of full field records for removing all empty fields;
traversing all data tables in the database, accumulating the obtained data index values, calculating the proportion of the full empty fields of the database, the recording proportion of the full fields of the database, the proportion of the none values of the database and the proportion of the none values of the removed full empty fields of the database, and evaluating the data integrity of the database.
2. The method of claim 1, wherein the calculating the number of none values excluding all empty fields comprises:
Q＝P-M*N
q is the number of none values of all empty fields removed from the data table; p is the number of none values in the data table; m is the number of all-empty fields in the data table; and N is the total number of records in the data table.
3. The method of claim 1, wherein the full field record number is:
K＝N-F
k is the number of full field records in the data table; n is the total number of records in the data table; f is the number of records of the non-full field in the data table; and the number of the records of the non-full field is the total number of the records in the field containing the null value in the data table.
4. The method of claim 1, wherein accumulating the resulting data values comprises:
accumulating the number of fields of each data table to obtain the total number of fields of the database;
accumulating the product of the number of fields of each data table and the number of records to obtain the total number of the numerical values of the database;
accumulating the total record number of each data table to obtain the total record number of the database;
accumulating the number of none values of each data table to obtain the number of the none values of the database;
accumulating the number of none values of all removed empty fields of each data table to obtain the number of all removed empty field none values of the database;
accumulating the full field record number of each data table to obtain the full field record number of the database;
and accumulating the numerical values of all the data tables after all the empty fields are removed to obtain the total number of the numerical values of the database after all the empty fields are removed.
5. The method of claim 4, wherein the values after removing the full empty field are:
H＝(R-M)*N
h is a numerical value of the data table after all empty fields are removed; r is the number of fields in the data table; m is the number of all-empty fields in the data table; and N is the total number of records in the data table.
6. The method of claim 1, wherein the calculating the proportion of all empty fields in the database, the proportion of full field records in the database, the proportion of none values in the database, and the proportion of none values in the removed all empty fields in the database comprises:
the proportion of the database full-empty fields is the ratio of the number of the database full-empty fields to the total number of the database fields;
the database full field record proportion is the ratio of the database full field record number to the total database record number;
the proportion of the database none values is the ratio of the number of the database none values to the total number of the database values;
the proportion of the none values of the database without the full-empty fields is the ratio of the number of the none values of the database without the full-empty fields to the total number of the numerical values of the database without the full-empty fields.
7. The method of claim 1, wherein said evaluating the data integrity of said database comprises:
the data integrity of the database is in negative correlation with the proportion of the full empty fields of the database, the proportion of the none values of the database and the proportion of the none values of the removed full empty fields of the database;
the data integrity of the database is positively correlated with the full field record proportion of the database.
8. A database integrity evaluation device for a power industry business system is characterized by comprising:
the reading module is used for reading a first field from the first function table, reading a second field and a third field from the second function table, and establishing the affiliated relationship between the data table and the fields according to the second field; the first field is the total record number contained in a data table; the second field represents the name of the data table to which the field visible to the user belongs; the third field is a null value number contained in each field in the data table;
the first accumulation calculation module is used for reading a data table from a database, accumulating the value of a third field of the data table and obtaining the number of the none values of the data table; calculating the number of none values and the number of full field records for removing all empty fields;
the second accumulation calculation module is used for traversing all data tables in the database, accumulating the obtained data index values, and calculating the proportion of the full-empty fields of the database, the recording proportion of the full fields of the database, the proportion of the none values of the database and the proportion of the none values of the removed full-empty fields of the database;
and the evaluation module is used for evaluating the data integrity of the database.
9. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the processor, when executing the program, implements the method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010826563.1A CN111897803B (en) | 2020-08-17 | 2020-08-17 | Database integrity evaluation method for power industry service system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010826563.1A CN111897803B (en) | 2020-08-17 | 2020-08-17 | Database integrity evaluation method for power industry service system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111897803A true CN111897803A (en) | 2020-11-06 |
| CN111897803B CN111897803B (en) | 2023-10-20 |
Family
ID=73229656
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010826563.1A Active CN111897803B (en) | 2020-08-17 | 2020-08-17 | Database integrity evaluation method for power industry service system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111897803B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115203300A (en) * | 2022-06-22 | 2022-10-18 | 中电金信软件有限公司 | Data verification method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002351875A (en) * | 2001-05-21 | 2002-12-06 | Requisite Technology Inc | Database contents quality analyzing method and device |
| KR20040023376A (en) * | 2002-09-11 | 2004-03-18 | 한국과학기술정보연구원 | Real-time quality measurement method of bibliographic database |
| CN102156922A (en) * | 2011-04-20 | 2011-08-17 | 重庆市电力公司綦南供电局 | Data integrity determination method for application assessment expansion platform of IT (Information Technology) system |
| CN109299062A (en) * | 2018-07-02 | 2019-02-01 | 北京市天元网络技术股份有限公司 | A kind of quality evaluating method and system towards document category digital resource metadata |
-
2020
- 2020-08-17 CN CN202010826563.1A patent/CN111897803B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002351875A (en) * | 2001-05-21 | 2002-12-06 | Requisite Technology Inc | Database contents quality analyzing method and device |
| KR20040023376A (en) * | 2002-09-11 | 2004-03-18 | 한국과학기술정보연구원 | Real-time quality measurement method of bibliographic database |
| CN102156922A (en) * | 2011-04-20 | 2011-08-17 | 重庆市电力公司綦南供电局 | Data integrity determination method for application assessment expansion platform of IT (Information Technology) system |
| CN109299062A (en) * | 2018-07-02 | 2019-02-01 | 北京市天元网络技术股份有限公司 | A kind of quality evaluating method and system towards document category digital resource metadata |
Non-Patent Citations (1)
| Title |
|---|
| 李登道, 苏娜: "关于数据表中字段属性"NULL"的理解与使用", 计算机应用与软件, no. 12 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115203300A (en) * | 2022-06-22 | 2022-10-18 | 中电金信软件有限公司 | Data verification method and device |
| CN115203300B (en) * | 2022-06-22 | 2023-10-31 | 中电金信软件有限公司 | Data verification method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111897803B (en) | 2023-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108829638B (en) | Business data fluctuation processing method and device | |
| US9600559B2 (en) | Data processing for database aggregation operation | |
| CN114461644A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN111897803A (en) | Database integrity evaluation method for power industry business system | |
| CN109271453B (en) | Method and device for determining database capacity | |
| CN113792084A (en) | Data heat analysis method, device, equipment and storage medium | |
| CN110458581B (en) | Method and device for identifying business turnover abnormality of commercial tenant | |
| CN103136440A (en) | Method and device of data processing | |
| CN111949644B (en) | Data quality evaluation method and equipment for power industry service system | |
| CN111897889B (en) | Data table integrity evaluation method for power industry service system | |
| CN116955413A (en) | Data query method, device, medium and equipment based on online analysis processing | |
| CN115098740B (en) | Data quality detection method and device based on multi-source heterogeneous data source | |
| CN115879031A (en) | Load classification method for adjustable load area and related equipment | |
| CN115238815A (en) | Abnormal transaction data acquisition method, device, equipment, medium and program product | |
| CN110955547B (en) | Management method, system, equipment and readable storage medium for interface abnormal information | |
| CN112163127B (en) | Relationship graph construction method and device, electronic equipment and storage medium | |
| US20130191071A1 (en) | System and method for automatic modal parameter extraction in structural dynamics analysis | |
| CN110427558B (en) | Resource processing event pushing method and device | |
| CN114077532A (en) | SQL statement execution efficiency detection method and device | |
| CN113792961A (en) | Economic responsibility auditing decision method and equipment based on big audit data | |
| CN112634004A (en) | Blood margin map analysis method and system for credit investigation data | |
| CN107153651A (en) | A kind of multidimensional intersects data processing method and processing device | |
| CN117077598B (en) | 3D parasitic parameter optimization method based on Mini-batch gradient descent method | |
| CN112148764B (en) | Feature screening method, device, equipment and storage medium | |
| CN112836924B (en) | Method, device, electronic equipment and storage medium for resource allocation audit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |