CN113722333A - Data checking method, device, electronic equipment, storage medium and program product - Google Patents
Data checking method, device, electronic equipment, storage medium and program product Download PDFInfo
- Publication number
- CN113722333A CN113722333A CN202111062543.2A CN202111062543A CN113722333A CN 113722333 A CN113722333 A CN 113722333A CN 202111062543 A CN202111062543 A CN 202111062543A CN 113722333 A CN113722333 A CN 113722333A
- Authority
- CN
- China
- Prior art keywords
- target field
- data
- checking
- target
- rule information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000007689 inspection Methods 0.000 claims abstract description 74
- 238000004590 computer program Methods 0.000 claims description 13
- 238000013506 data mapping Methods 0.000 claims description 13
- 230000004044 response Effects 0.000 claims description 5
- 230000001172 regenerating effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 11
- 238000013507 mapping Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000001960 triggered effect Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the disclosure discloses a data checking method, a data checking device, an electronic device, a storage medium and a program product, wherein the method comprises the following steps: acquiring a target field; inquiring to obtain the storage position information of the target field according to the target field; inquiring according to the target field to obtain the check rule information corresponding to the target field; and checking the target data corresponding to the target field according to the storage position information and the checking rule information of the target field. According to the technical scheme, the automatic inspection of the data quality is realized, so that the automation degree and the inspection efficiency of the data inspection can be greatly improved, the labor cost is greatly reduced, and meanwhile, the accuracy of the data inspection can also be effectively guaranteed.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of data processing, in particular to a data checking method, a data checking device, electronic equipment, a storage medium and a program product.
Background
The data warehouse is a data set which stores a large amount of data and is used for providing various types of data support for users, data processing tasks and the like, and therefore the quality of the data stored in the data warehouse is the basis for guaranteeing data application. In order to guarantee the data quality and further guarantee the correctness of subsequent data processing tasks, the data quality needs to be checked before the data is used, but due to the fact that the data quantity stored in a data warehouse is huge, if the data quality is checked by manually or manually compiling commands, the data checking efficiency is greatly reduced, the labor cost is greatly improved, and the accuracy of data checking cannot be guaranteed.
Disclosure of Invention
The embodiment of the disclosure provides a data checking method, a data checking device, electronic equipment, a storage medium and a program product.
In a first aspect, a data checking method is provided in the embodiments of the present disclosure.
Specifically, the data checking method includes:
acquiring a target field;
inquiring to obtain the storage position information of the target field according to the target field;
inquiring according to the target field to obtain the check rule information corresponding to the target field;
and checking the target data corresponding to the target field according to the storage position information and the checking rule information of the target field.
With reference to the first aspect, in a first implementation manner of the first aspect, the obtaining, according to the target field query, storage location information of the target field includes:
and querying a data mapping table by using the target field as an index value to obtain the storage position information of the target field.
With reference to the first aspect and the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the obtaining, according to the target field query, the check rule information corresponding to the target field includes:
and querying a data checking table by using the target field as an index value to obtain checking rule information corresponding to the target field.
With reference to the first aspect, the first implementation manner of the first aspect, and the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the checking, according to the storage location information and the checking rule information of the target field, target data corresponding to the target field includes:
generating a data check command corresponding to the target field based on the storage location information and the check rule information of the target field;
and executing the data inspection command, and inspecting the target data corresponding to the target field.
With reference to the first aspect, the first implementation manner of the first aspect, the second implementation manner of the first aspect, and the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the present disclosure further includes:
and in response to detecting that the storage position information of the target field and/or the checking rule information corresponding to the target field are changed, regenerating the data checking command corresponding to the target field.
In a second aspect, a data checking method is provided in the disclosed embodiments.
Specifically, the data checking method includes:
an acquisition module configured to acquire a target field;
the first query module is configured to obtain the storage position information of the target field according to the query of the target field;
the second query module is configured to obtain the check rule information corresponding to the target field according to the query of the target field;
and the checking module is configured to check the target data corresponding to the target field according to the storage position information and the checking rule information of the target field.
With reference to the second aspect, in a first implementation manner of the second aspect, the first query module is configured to:
and querying a data mapping table by using the target field as an index value to obtain the storage position information of the target field.
With reference to the second aspect and the first implementation manner of the second aspect, in a second implementation manner of the second aspect, the second query module is configured to:
and querying a data checking table by using the target field as an index value to obtain checking rule information corresponding to the target field.
With reference to the second aspect, the first implementation manner of the second aspect, and the second implementation manner of the second aspect, in a third implementation manner of the second aspect, the checking module is configured to:
generating a data check command corresponding to the target field based on the storage location information and the check rule information of the target field;
and executing the data inspection command, and inspecting the target data corresponding to the target field.
With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, and the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect, the present disclosure further includes:
the generating module is configured to regenerate the data checking command corresponding to the target field in response to detecting that the storage location information of the target field and/or the checking rule information corresponding to the target field changes.
In a third aspect, the disclosed embodiments provide an electronic device, including a memory for storing one or more computer instructions that support a data checking apparatus to perform the above data checking method, and a processor configured to execute the computer instructions stored in the memory. The data inspection apparatus may further comprise a communication interface for the data inspection apparatus to communicate with other devices or a communication network.
In a fourth aspect, the disclosed embodiments provide a computer-readable storage medium for storing computer instructions for a data inspection apparatus, which includes computer instructions for performing the data inspection method described above as a data inspection apparatus.
In a fifth aspect, the disclosed embodiments provide a computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the above-described data inspection method.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
according to the technical scheme, the storage position information and the check rule information of the predetermined target field are inquired, and the target data corresponding to the target field are automatically checked based on the inquiry result. According to the technical scheme, the automatic inspection of the data quality is realized, so that the automation degree and the inspection efficiency of the data inspection can be greatly improved, the labor cost is greatly reduced, and meanwhile, the accuracy of the data inspection can also be effectively guaranteed.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of embodiments of the disclosure.
Drawings
Other features, objects, and advantages of embodiments of the disclosure will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 shows a flow diagram of a data inspection method according to an embodiment of the present disclosure;
FIG. 2 shows a block diagram of a data inspection device according to an embodiment of the present disclosure;
FIG. 3 shows a block diagram of an electronic device according to an embodiment of the present disclosure;
FIG. 4 is a schematic block diagram of a computer system suitable for use in implementing a data inspection method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, exemplary embodiments of the disclosed embodiments will be described in detail with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.
In the disclosed embodiments, it is to be understood that terms such as "including" or "having," etc., are intended to indicate the presence of the disclosed features, numbers, steps, behaviors, components, parts, or combinations thereof, and are not intended to preclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or combinations thereof may be present or added.
It should be further noted that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict. The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
According to the technical scheme provided by the embodiment of the disclosure, the storage position information and the check rule information of the predetermined target field are inquired, and the target data corresponding to the target field is automatically checked based on the inquiry result. According to the technical scheme, the automatic inspection of the data quality is realized, so that the automation degree and the inspection efficiency of the data inspection can be greatly improved, the labor cost is greatly reduced, and meanwhile, the accuracy of the data inspection can also be effectively guaranteed.
Fig. 1 shows a flowchart of a data inspection method according to an embodiment of the present disclosure, as shown in fig. 1, the data inspection method includes the following steps S101-S104:
in step S101, a target field is acquired;
in step S102, obtaining storage location information of the target field according to the target field query;
in step S103, obtaining inspection rule information corresponding to the target field according to the target field query;
in step S104, the target data corresponding to the target field is checked according to the storage location information and the check rule information of the target field.
As mentioned above, a data warehouse is a data set that stores a large amount of data and provides various types of data support for users, data processing tasks, and the like, and therefore, the quality of data stored in the data warehouse is the basis for guaranteeing data applications. In order to guarantee the data quality and further guarantee the correctness of subsequent data processing tasks, the data quality needs to be checked before the data is used, but due to the fact that the data quantity stored in a data warehouse is huge, if the data quality is checked by manually or manually compiling commands, the data checking efficiency is greatly reduced, the labor cost is greatly improved, and the accuracy of data checking cannot be guaranteed.
In view of the above, in this embodiment, a data inspection method is proposed that performs an inquiry of storage location information and inspection rule information for a predetermined target field and implements automatic inspection of target data corresponding to the target field based on the inquiry result. According to the technical scheme, the automatic inspection of the data quality is realized, so that the automation degree and the inspection efficiency of the data inspection can be greatly improved, the labor cost is greatly reduced, and meanwhile, the accuracy of the data inspection can also be effectively guaranteed.
In an embodiment of the present disclosure, the data inspection method may be applied to a computer, an electronic device, a server cluster, and the like that perform data inspection.
In an embodiment of the present disclosure, the target field refers to a field that needs to be checked for data, such as a product name, a product ID, and the like, wherein the target field may be predetermined by a user or by other data checking parties.
In an embodiment of the present disclosure, the storage location information of the target field refers to description information of a storage location where the target field is located. For example, if the target field exists in the data table a and the data file B, when there is a difference between the names of the data table a and the data file B, the storage location information of the target field may be the names of the data table a and the data file B, and when there is a duplication between the names of the data table a and the data file B, the storage location information of the target field may be the storage path of the data table a and the data file B.
In an embodiment of the present disclosure, the check rule information corresponding to the target field refers to rule information that is applicable to the target field and is used for checking whether the data quality of the data of the target field meets requirements. For example, if the target field is an article ID, the inspection rule information corresponding to the field may be information for inspecting whether the data format of the field meets a preset requirement, and the inspection rule information corresponding to the field may be, for example, "1 st to 2 th digits are selected from a set of letter combinations representing article categories, 3 rd to 4 th digits are selected from a set of letter combinations representing article attributes, 4 th to 8 th digits are selected from a set of number combinations representing article numbers, and 9 th to 10 th digits are selected from a set of number combinations representing article storage locations", and so on.
In the above embodiment, a predetermined target field to be checked is first obtained, then storage location information of the target field and check rule information corresponding to the target field are obtained according to the target field query, and finally, target data corresponding to the target field is automatically checked according to the storage location information of the target field and the check rule information.
In an embodiment of the present disclosure, the step S102, namely, the step of obtaining the storage location information of the target field according to the target field query, may include the following steps:
and querying a data mapping table by using the target field as an index value to obtain the storage position information of the target field.
The data mapping table refers to a data table for storing mapping relationships between field information and corresponding storage locations, and if a field a exists in a data table a and a data file B and a field B exists in a data table B and a data table C, the data mapping table can record mapping relationships between the field a and the data table a and the data file B and mapping relationships between the field B and the data table C.
In this embodiment, the querying of the target field storage location information is accomplished by means of a data mapping table. Specifically, the target field is used as an index value to be retrieved in the data mapping table, so that the storage location information of the target field can be obtained.
In an embodiment of the present disclosure, the step S103 of obtaining the check rule information corresponding to the target field according to the target field query may include the following steps:
and querying a data checking table by using the target field as an index value to obtain checking rule information corresponding to the target field.
The data check table refers to a data table for storing mapping relationships between field information and corresponding check rules. Based on the data checking table, the checking rule information corresponding to a certain field can be known, that is, if the data corresponding to the field is checked, which parts of the data of the field are to be checked, how to check, and the like.
In this embodiment, the query of the target field check rule information is implemented by means of a data check table. Specifically, the target field is used as an index value to be retrieved in the data check table, so that the check rule information corresponding to the target field can be obtained.
In an embodiment of the present disclosure, the step S104 of checking the target data corresponding to the target field according to the storage location information of the target field and the checking rule information may include the following steps:
generating a data check command corresponding to the target field based on the storage location information and the check rule information of the target field;
and executing the data inspection command, and inspecting the target data corresponding to the target field.
In this embodiment, in order to implement automatic inspection of data, after obtaining the storage location information of the target field and the inspection rule information corresponding to the target field by querying, a data inspection command corresponding to the target field is generated based on the storage location information of the target field and the inspection rule information, and then, by executing the data inspection command, automatic inspection of the target data corresponding to the target field can be implemented, so that the automation degree and the inspection efficiency of data inspection are greatly improved, and the labor cost is greatly reduced.
Wherein, the data checking command refers to a command which can be executed or automatically executed and is used for realizing the purpose of data checking. The data check command may be, for example, an SQL command statement or the like. The execution of the data check command may be triggered based on a preset command execution trigger condition, where the command execution trigger condition may be, for example, a time trigger condition, that is, the execution of the data check command is triggered after a certain time point is detected or a preset time interval elapses; it may also be an event trigger condition, i.e. triggering the execution of the data check command upon detection of a certain event or events.
In an embodiment of the present disclosure, the method may further include the steps of:
and in response to detecting that the storage position information of the target field and/or the checking rule information corresponding to the target field are changed, regenerating the data checking command corresponding to the target field.
In consideration of the fact that in practical applications, the storage location information of the target field and/or the check rule information corresponding to the target field may change, in order to effectively guarantee the accuracy of data check, the data check command needs to be updated according to the actually-occurring change. That is, in this embodiment, whether or not the storage location information of the target field and/or the check rule information corresponding to the target field has changed is detected at regular time, and if it is detected that the storage location information of the target field and/or the check rule information corresponding to the target field has changed, the data check command corresponding to the target field is newly generated in accordance with the data check command generation step.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods.
Fig. 2 shows a block diagram of a data checking apparatus according to an embodiment of the present disclosure, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 2, the data inspection apparatus includes:
an obtaining module 201 configured to obtain a target field;
a first query module 202, configured to obtain storage location information of the target field according to the query of the target field;
the second query module 203 is configured to obtain the check rule information corresponding to the target field according to the query of the target field;
and the checking module 204 is configured to check the target data corresponding to the target field according to the storage location information and the checking rule information of the target field.
As mentioned above, a data warehouse is a data set that stores a large amount of data and provides various types of data support for users, data processing tasks, and the like, and therefore, the quality of data stored in the data warehouse is the basis for guaranteeing data applications. In order to guarantee the data quality and further guarantee the correctness of subsequent data processing tasks, the data quality needs to be checked before the data is used, but due to the fact that the data quantity stored in a data warehouse is huge, if the data quality is checked by manually or manually compiling commands, the data checking efficiency is greatly reduced, the labor cost is greatly improved, and the accuracy of data checking cannot be guaranteed.
In view of the above, in this embodiment, a data inspection apparatus is proposed that performs an inquiry of storage location information and inspection rule information for a predetermined target field and realizes automatic inspection of target data corresponding to the target field based on the inquiry result. According to the technical scheme, the automatic inspection of the data quality is realized, so that the automation degree and the inspection efficiency of the data inspection can be greatly improved, the labor cost is greatly reduced, and meanwhile, the accuracy of the data inspection can also be effectively guaranteed.
In an embodiment of the present disclosure, the data inspection apparatus may be implemented as a computer, an electronic device, a server cluster, or the like that performs data inspection.
In an embodiment of the present disclosure, the target field refers to a field that needs to be checked for data, such as a product name, a product ID, and the like, wherein the target field may be predetermined by a user or by other data checking parties.
In an embodiment of the present disclosure, the storage location information of the target field refers to description information of a storage location where the target field is located. For example, if the target field exists in the data table a and the data file B, when there is a difference between the names of the data table a and the data file B, the storage location information of the target field may be the names of the data table a and the data file B, and when there is a duplication between the names of the data table a and the data file B, the storage location information of the target field may be the storage path of the data table a and the data file B.
In an embodiment of the present disclosure, the check rule information corresponding to the target field refers to rule information that is applicable to the target field and is used for checking whether the data quality of the data of the target field meets requirements. For example, if the target field is an article ID, the inspection rule information corresponding to the field may be information for inspecting whether the data format of the field meets a preset requirement, and the inspection rule information corresponding to the field may be, for example, "1 st to 2 th digits are selected from a set of letter combinations representing article categories, 3 rd to 4 th digits are selected from a set of letter combinations representing article attributes, 4 th to 8 th digits are selected from a set of number combinations representing article numbers, and 9 th to 10 th digits are selected from a set of number combinations representing article storage locations", and so on.
In the above embodiment, a predetermined target field to be checked is first obtained, then storage location information of the target field and check rule information corresponding to the target field are obtained according to the target field query, and finally, target data corresponding to the target field is automatically checked according to the storage location information of the target field and the check rule information.
In an embodiment of the present disclosure, the first query module 202 may be configured to:
and querying a data mapping table by using the target field as an index value to obtain the storage position information of the target field.
The data mapping table refers to a data table for storing mapping relationships between field information and corresponding storage locations, and if a field a exists in a data table a and a data file B and a field B exists in a data table B and a data table C, the data mapping table can record mapping relationships between the field a and the data table a and the data file B and mapping relationships between the field B and the data table C.
In this embodiment, the querying of the target field storage location information is accomplished by means of a data mapping table. Specifically, the target field is used as an index value to be retrieved in the data mapping table, so that the storage location information of the target field can be obtained.
In an embodiment of the present disclosure, the second query module 203 may be configured to:
and querying a data checking table by using the target field as an index value to obtain checking rule information corresponding to the target field.
The data check table refers to a data table for storing mapping relationships between field information and corresponding check rules. Based on the data checking table, the checking rule information corresponding to a certain field can be known, that is, if the data corresponding to the field is checked, which parts of the data of the field are to be checked, how to check, and the like.
In this embodiment, the query of the target field check rule information is implemented by means of a data check table. Specifically, the target field is used as an index value to be retrieved in the data check table, so that the check rule information corresponding to the target field can be obtained.
In an embodiment of the present disclosure, the checking module 204 may be configured to:
generating a data check command corresponding to the target field based on the storage location information and the check rule information of the target field;
and executing the data inspection command, and inspecting the target data corresponding to the target field.
In this embodiment, in order to implement automatic inspection of data, after obtaining the storage location information of the target field and the inspection rule information corresponding to the target field by querying, a data inspection command corresponding to the target field is generated based on the storage location information of the target field and the inspection rule information, and then, by executing the data inspection command, automatic inspection of the target data corresponding to the target field can be implemented, so that the automation degree and the inspection efficiency of data inspection are greatly improved, and the labor cost is greatly reduced.
Wherein, the data checking command refers to a command which can be executed or automatically executed and is used for realizing the purpose of data checking. The data check command may be, for example, an SQL command statement or the like. The execution of the data check command may be triggered based on a preset command execution trigger condition, where the command execution trigger condition may be, for example, a time trigger condition, that is, the execution of the data check command is triggered after a certain time point is detected or a preset time interval elapses; it may also be an event trigger condition, i.e. triggering the execution of the data check command upon detection of a certain event or events.
In an embodiment of the present disclosure, the apparatus may further include:
the generating module is configured to regenerate the data checking command corresponding to the target field in response to detecting that the storage location information of the target field and/or the checking rule information corresponding to the target field changes.
In consideration of the fact that in practical applications, the storage location information of the target field and/or the check rule information corresponding to the target field may change, in order to effectively guarantee the accuracy of data check, the data check command needs to be updated according to the actually-occurring change. That is, in this embodiment, whether or not the storage location information of the target field and/or the check rule information corresponding to the target field has changed is detected at regular time, and if it is detected that the storage location information of the target field and/or the check rule information corresponding to the target field has changed, the data check command corresponding to the target field is newly generated in accordance with the data check command generation step.
The present disclosure also discloses an electronic device, fig. 3 shows a block diagram of an electronic device according to an embodiment of the present disclosure, and as shown in fig. 3, the electronic device 300 includes a memory 301 and a processor 302; wherein,
the memory 301 is used to store one or more computer instructions, which are executed by the processor 302 to implement the above-described method steps.
FIG. 4 is a schematic block diagram of a computer system suitable for use in implementing a data inspection method according to an embodiment of the present disclosure.
As shown in fig. 4, the computer system 400 includes a processing unit 401 that can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM)402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. In the RAM403, various programs and data necessary for the operation of the system 400 are also stored. The processing unit 401, the ROM402, and the RAM403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. A driver 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as necessary, so that a computer program read out therefrom is mounted into the storage section 408 as necessary. The processing unit 401 may be implemented as a CPU, a GPU, a TPU, an FPGA, an NPU, or other processing units.
In particular, the above described methods may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the data inspection method. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 409, and/or installed from the removable medium 411.
A computer program product is also disclosed in embodiments of the present disclosure, the computer program product comprising computer programs/instructions which, when executed by a processor, implement any of the above method steps.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present disclosure may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.
As another aspect, the disclosed embodiment also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the foregoing embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the embodiments of the present disclosure.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept. For example, the above features and (but not limited to) the features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (10)
1. A method of data inspection, comprising:
acquiring a target field;
inquiring to obtain the storage position information of the target field according to the target field;
inquiring according to the target field to obtain the check rule information corresponding to the target field;
and checking the target data corresponding to the target field according to the storage position information and the checking rule information of the target field.
2. The method of claim 1, wherein the obtaining of the storage location information of the target field according to the target field query comprises:
and querying a data mapping table by using the target field as an index value to obtain the storage position information of the target field.
3. The method according to claim 1 or 2, wherein the obtaining of the inspection rule information corresponding to the target field according to the target field query includes:
and querying a data checking table by using the target field as an index value to obtain checking rule information corresponding to the target field.
4. The method according to any one of claims 1 to 3, wherein the checking the target data corresponding to the target field according to the storage location information and the checking rule information of the target field includes:
generating a data check command corresponding to the target field based on the storage location information and the check rule information of the target field;
and executing the data inspection command, and inspecting the target data corresponding to the target field.
5. The method of any of claims 1-4, further comprising:
and in response to detecting that the storage position information of the target field and/or the checking rule information corresponding to the target field are changed, regenerating the data checking command corresponding to the target field.
6. A data inspection apparatus comprising:
an acquisition module configured to acquire a target field;
the first query module is configured to obtain the storage position information of the target field according to the query of the target field;
the second query module is configured to obtain the check rule information corresponding to the target field according to the query of the target field;
and the checking module is configured to check the target data corresponding to the target field according to the storage position information and the checking rule information of the target field.
7. The apparatus of claim 6, the checking module configured to:
generating a data check command corresponding to the target field based on the storage location information and the check rule information of the target field;
and executing the data inspection command, and inspecting the target data corresponding to the target field.
8. An electronic device comprising a memory and a processor; wherein,
the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method of any one of claims 1-5.
9. A computer readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, implement the steps of the method of any one of claims 1-5.
10. A computer program product comprising computer programs/instructions which, when executed by a processor, carry out the steps of the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111062543.2A CN113722333A (en) | 2021-09-10 | 2021-09-10 | Data checking method, device, electronic equipment, storage medium and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111062543.2A CN113722333A (en) | 2021-09-10 | 2021-09-10 | Data checking method, device, electronic equipment, storage medium and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113722333A true CN113722333A (en) | 2021-11-30 |
Family
ID=78683205
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111062543.2A Pending CN113722333A (en) | 2021-09-10 | 2021-09-10 | Data checking method, device, electronic equipment, storage medium and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113722333A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180189330A1 (en) * | 2017-01-04 | 2018-07-05 | Salesforce.Com, Inc. | Database schema for efficient data assessment |
| US20180218041A1 (en) * | 2017-02-02 | 2018-08-02 | International Business Machines Corporation | Validation of query plan |
| CN109510824A (en) * | 2018-11-12 | 2019-03-22 | 中国银行股份有限公司 | A kind of method of calibration and device of interface packets |
| CN111209538A (en) * | 2020-01-03 | 2020-05-29 | 北京明略软件系统有限公司 | Table data quality probing method and device |
| CN112650762A (en) * | 2021-03-15 | 2021-04-13 | 腾讯科技（深圳）有限公司 | Data quality monitoring method and device, electronic equipment and storage medium |
-
2021
- 2021-09-10 CN CN202111062543.2A patent/CN113722333A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180189330A1 (en) * | 2017-01-04 | 2018-07-05 | Salesforce.Com, Inc. | Database schema for efficient data assessment |
| US20180218041A1 (en) * | 2017-02-02 | 2018-08-02 | International Business Machines Corporation | Validation of query plan |
| CN109510824A (en) * | 2018-11-12 | 2019-03-22 | 中国银行股份有限公司 | A kind of method of calibration and device of interface packets |
| CN111209538A (en) * | 2020-01-03 | 2020-05-29 | 北京明略软件系统有限公司 | Table data quality probing method and device |
| CN112650762A (en) * | 2021-03-15 | 2021-04-13 | 腾讯科技（深圳）有限公司 | Data quality monitoring method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 方利, 刘生权, 钟耳顺: "基于元数据和质量规则的土地数据检查", 地球信息科学, no. 03 * |
| 李雨: "中文版Visual FoxPro 6.0编程指南", 30 September 1999, 人民邮电出版社, pages: 141 - 144 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2019253860B2 (en) | Data quality analysis | |
| CN110908997B (en) | Data blood relationship construction method and device, server and readable storage medium | |
| US8584112B2 (en) | Open application lifecycle management framework | |
| CN111414352A (en) | Database information management method and device | |
| CN111813804A (en) | Data query method and device, electronic equipment and storage medium | |
| CN110750969B (en) | Report generation method and device | |
| CN113268500B (en) | Service processing method and device and electronic equipment | |
| CN113434527B (en) | Data processing method, device, electronic equipment and storage medium | |
| AU2021244852B2 (en) | Offloading statistics collection | |
| CN113010208B (en) | Version information generation method, device, equipment and storage medium | |
| CN109871408B (en) | Multi-type database adaptation method, device, electronic equipment and storage medium | |
| CN110737655A (en) | Method and device for reporting data | |
| CN113138906A (en) | Call chain data acquisition method, device, equipment and storage medium | |
| CN110515967B (en) | Spark calculation framework-based data analysis method and electronic equipment | |
| CN111753141B (en) | Data management method and related equipment | |
| CN112463872A (en) | Visual index data acquisition method, device, system and medium | |
| CN113722333A (en) | Data checking method, device, electronic equipment, storage medium and program product | |
| CN112346994B (en) | Test information association method, device, computer equipment and storage medium | |
| CN111262727B (en) | Service capacity expansion method, device, equipment and storage medium | |
| CN114048231A (en) | Data processing method, device and computer program product | |
| US8321844B2 (en) | Providing registration of a communication | |
| CN112181825A (en) | Test case library construction method and device, electronic equipment and medium | |
| CN113485890A (en) | Flight inquiry system service monitoring method, device, equipment and storage medium | |
| CN112363944A (en) | Method and equipment for comparing return values of multiple environment interfaces | |
| CN113254328B (en) | White box testing method, system, mobile terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |