CN116680579A - Error data determining method and device and electronic equipment - Google Patents

Error data determining method and device and electronic equipment Download PDF

Info

Publication number
CN116680579A
CN116680579A CN202310617947.6A CN202310617947A CN116680579A CN 116680579 A CN116680579 A CN 116680579A CN 202310617947 A CN202310617947 A CN 202310617947A CN 116680579 A CN116680579 A CN 116680579A
Authority
CN
China
Prior art keywords
data
target
field name
determining
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310617947.6A
Other languages
Chinese (zh)
Inventor
严光兵
顾振华
吴林峰
廖程
谢家宝
文艺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peoples Insurance Company of China
Original Assignee
Peoples Insurance Company of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peoples Insurance Company of China filed Critical Peoples Insurance Company of China
Priority to CN202310617947.6A priority Critical patent/CN116680579A/en
Publication of CN116680579A publication Critical patent/CN116680579A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application discloses a method and a device for determining error data and electronic equipment, wherein the method for determining the error data comprises the following steps: receiving a plurality of data tables to be tested, wherein the data tables to be tested comprise a plurality of target field names and target data corresponding to the target field names; acquiring target matching rules corresponding to the target field names according to the corresponding relation between the pre-configured field names and the matching rules; and under the condition that the first data is matched with a first matching rule, determining the first data as error data, wherein the first data is data corresponding to a first field name, the first matching rule is a matching rule corresponding to the first field name, and the first field name is any field name in each target field name.

Description

Error data determining method and device and electronic equipment
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method and an apparatus for determining error data, and an electronic device.
Background
Due to the reasons of business logic errors of the front-end system, noise of a transmission link, imperfect functions of the system and the like, error data (namely abnormal business logic data) can be generated, and based on the error data, the back-end system cannot complete data processing in time when performing serial processing according to the data acquired from the front-end system, and further cannot complete the process of data accounting on time.
At present, in order to process error data, data inspection is required to be performed according to manual SQL (structured query language ) sentences, and the problems that the efficiency is low, the efficient operation of a system cannot be ensured, and the data aging requirement of a back-end system is affected exist.
Disclosure of Invention
The application discloses a method and a device for determining error data and electronic equipment, and aims to solve the problem that the efficiency of determining the error data is low in the related technology.
In order to solve the problems, the application adopts the following technical scheme:
in a first aspect, an embodiment of the present application discloses a method for determining error data, including: receiving a plurality of data tables to be tested, wherein the data tables to be tested comprise a plurality of target field names and target data corresponding to the target field names; acquiring target matching rules corresponding to the target field names according to the corresponding relation between the pre-configured field names and the matching rules; and under the condition that the first data is matched with a first matching rule, determining the first data as error data, wherein the first data is data corresponding to a first field name, the first matching rule is a matching rule corresponding to the first field name, and the first field name is any field name in each target field name.
In a second aspect, an embodiment of the present application discloses an apparatus for determining error data, including: the receiving module is used for receiving a plurality of data tables to be detected, wherein the data tables to be detected comprise a plurality of target field names and target data corresponding to the target field names; the acquisition module is used for acquiring target matching rules corresponding to the target field names according to the corresponding relation between the preset field names and the matching rules; and the determining module is used for determining the first data as error data under the condition that the first data is determined to be matched with a first matching rule, wherein the first data is data corresponding to a first field name, the first matching rule is a matching rule corresponding to the first field name, and the first field name is any field name in each target field name.
In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method as described in the first aspect.
In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the method according to the first aspect.
The embodiment of the application provides a method for determining error data, which comprises the steps of receiving a plurality of data tables to be detected, wherein the data tables to be detected comprise a plurality of target field names and target data corresponding to each target field name, then acquiring a target matching rule corresponding to each target field name according to the corresponding relation between a pre-configured field name and the matching rule, and determining the first data as the error data under the condition that the first data is determined to be matched with the first matching rule, wherein the first data is the data corresponding to the first field name, the first matching rule is the matching rule corresponding to the first field name, and the first field name is any field name in each target field name. The application judges the error data based on the pre-configured matching rule, so that the efficiency of determining the error data can be improved, and the problem of lower efficiency of determining the error data in the related technology is solved. In addition, the error data is determined through the scheme disclosed by the application, and the manual participation degree can be reduced.
Drawings
FIG. 1 is a flow chart of a method for determining error data according to an embodiment of the present application;
FIG. 2 is a flow chart of a data process according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a device for determining error data according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which are obtained by a person skilled in the art based on the embodiments of the present application, fall within the scope of protection of the present application.
The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type, and are not limited to the number of objects, such as the first object may be one or more. In addition, in the specification and claims, "and/or" means at least one of the electrically connected objects, and the character "/", generally means that the associated object is an "or" relationship.
The method, the device and the electronic equipment for determining error data disclosed by the embodiment of the application are described in detail through specific embodiments and application scenes thereof with reference to the accompanying drawings.
Fig. 1 is a flow chart of a method for determining error data according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
s120, receiving a plurality of data tables to be tested.
The data table to be tested comprises a plurality of target field names and target data corresponding to the target field names.
By way of example, the destination field name may be a business type, a warranty passing date, a rechecking date, and the like.
S140, according to the corresponding relation between the preset field names and the matching rules, obtaining the target matching rules corresponding to the target field names.
In the application, the matching rules of the error data can be summarized by carrying out characteristic analysis on the problem data, then the corresponding relation between the field names and the matching rules is stored, and when a plurality of data tables to be detected are received, the target matching rules corresponding to the field names of all targets in the data tables to be detected are obtained.
For example, as shown in table 1, the correspondence between the field names and the matching rules may be that the target field name is "service type", the corresponding matching rule is "null", that is, the data corresponding to the service type is null, the data is error data, the target field name is "correction type", the corresponding matching rule is "null", that is, the data corresponding to the correction type is null, the data is error data, the target field name is "underwriting passing date", the corresponding matching rule is "date does not satisfy the format requirement, and the format requirement is: YYYY-MM-DD ", i.e., the date of the underwriting passing date does not meet the format requirement yyyyy-MM-DD, the date is erroneous data.
TABLE 1
The modification type of the social security service is an operation model corresponding to the social security service.
And S160, determining the first data as error data when the first data is determined to be matched with the first matching rule.
The first data is data corresponding to a first field name, the first matching rule is a matching rule corresponding to the first field name, and the first field name is any field name in the target field names.
For example, the first field name is "service type", the first matching rule is "null", and if the service type has no data corresponding to it (i.e., is null), the first data is matched with the first matching rule and is erroneous data; the first field name is 'check passing date', the first matching rule is 'date does not meet the format requirement YYYY-MM-DD', and if the first data corresponding to the check passing date is YYY-MM, the first data is matched with the first matching rule and is error data.
If the first matching rule corresponding to the first field name is acquired, but the first data is not matched with the first matching rule, the first data is correct data, for example, if the first field name is "the date of the passing of the check", the first matching rule is "the date does not satisfy the format requirement YYYY-MM-DD", and if the first data corresponding to the date of the passing of the check is YYYY-MM-DD, the first data is not matched with the first matching rule, and if the first data is correct data, the data processing is continued according to the predetermined processing flow.
The embodiment of the application provides a method for determining error data, which comprises the steps of receiving a plurality of data tables to be detected, wherein the data tables to be detected comprise a plurality of target field names and target data corresponding to each target field name, then acquiring a target matching rule corresponding to each target field name according to the corresponding relation between a pre-configured field name and the matching rule, and determining the first data as the error data under the condition that the first data is determined to be matched with the first matching rule, wherein the first data is the data corresponding to the first field name, the first matching rule is the matching rule corresponding to the first field name, and the first field name is any field name in each target field name. The application judges the error data based on the pre-configured matching rule, so that the efficiency of determining the error data can be improved, and the problem of lower efficiency of determining the error data in the related technology is solved. In addition, the error data is determined through the scheme disclosed by the application, and the manual participation degree can be reduced.
In an embodiment of the present application, after the determining the first data as error data, the method may further include: acquiring a first correction rule corresponding to the first field name according to the corresponding relation between the pre-configured field name and the correction rule; and correcting the first data according to the first correction rule.
For example, as shown in table 2, the corresponding relationship between the field names and the correction rules may be that the field names are of the service type, the corresponding correction rules are not processed, the field names are of the correction type, the corresponding correction rules are default values, the field names are of the underwriting date, and the corresponding correction rules are replaced by a certain field value.
TABLE 2
For example, the first field name is "the date of passing the check" and the first matching rule is "the date does not satisfy the format requirement yyyyy-MM-DD", and if the first data corresponding to the date of passing the check is YYYY-MM, the first data is matched with the first matching rule and is erroneous data. At this time, according to the correspondence between the preconfigured field name and the correction rule, the first correction rule corresponding to the first field name is obtained and replaced by a certain field value, and then the field value is used for replacing YYYY-MM, so as to complete the correction of the first data, for example, the field value is the date of start and hold.
According to the application, error data is corrected according to the preset correction rule, and the correction efficiency is higher.
In the present application, after the first data is corrected according to the first correction rule, the method may further include: and carrying out data processing on the corrected data according to a preset processing flow. That is, the present application detects data first, corrects error data according to a preset correction rule after determining the error data according to a preset matching rule, and performs data processing according to a predetermined processing flow after correcting the error data, and can improve the timeliness of data processing compared with the advanced data processing, which detects error data and corrects after abnormality occurs.
The method provided by the application can adopt a multithreading mechanism, a plurality of tasks are started each time, a plurality of matching rules are obtained, and the matching rules are executed in parallel, so that the processing efficiency is further improved. In one implementation manner, the obtaining, according to the correspondence between the preconfigured field names and the matching rules, the target matching rules corresponding to the target field names may include: and for each to-be-tested data table in the plurality of to-be-tested data tables, obtaining target matching rules corresponding to each target field name in the to-be-tested data table in parallel according to the corresponding relation between the pre-configured field names and the matching rules. That is, in the case of receiving a plurality of data tables to be tested, the data of the same table may be preferentially processed, the target matching rules corresponding to the respective target field names in the same data table to be tested are obtained in parallel according to the correspondence between the pre-configured field names and the matching rules, and then whether the target data is matched with the obtained target matching rules is determined in parallel, as shown in fig. 2. By the method, the error detection of the data table to be detected can be completed at one time, for example, the service type, the modification type and the verification passing date of the direct-protection main data fee-protection table can be completed at one time.
In another possible implementation manner, the obtaining, according to the correspondence between the preconfigured field names and the matching rules, the target matching rules corresponding to the target field names may include: and obtaining the target matching rules corresponding to the target field names in parallel according to the corresponding relation between the pre-configured field names and the matching rules. That is, after receiving the multiple data tables to be tested, the target matching rules corresponding to the respective target field names in the multiple data tables to be tested can be obtained in parallel according to the corresponding relation between the preconfigured field names and the matching rules, and whether the respective target data are matched with the target matching rules is judged, so that the efficiency of determining the error data is further improved.
In an embodiment of the present application, after the receiving a plurality of data tables to be tested, the method may further include: sending prompt information to a display interface under the condition that a target matching rule corresponding to the target field name cannot be obtained; and correcting the target data corresponding to the target field name in response to the received correction operation. That is, under the condition that the target matching rule corresponding to the target field name cannot be obtained according to the corresponding relation between the pre-configured field name and the matching rule (that is, the corresponding relation between the pre-configured field name and the matching rule does not include the corresponding relation between the target field name and the target matching rule), prompt information is sent to the display interface to prompt an operator to perform manual intervention, and the operator corrects the target data corresponding to the target field name after seeing the prompt information of the display interface. After the data correction is completed, a target matching rule corresponding to the target field name may be determined and stored based on the data characteristics of the corrected data. That is, by the method provided by the application, along with the accumulation of data problems, the matching rule can be continuously perfected, so that the accumulation of the processing efficiency is improved.
In the present application, after the first data is determined as error data, it may further include: and sending the first data to a display interface to display error information of the first data.
The method for determining the error data can be applied to the data processing of the new insurance contract criterion.
According to the method for determining the error data provided by the embodiment of the application, the execution main body can be the device for determining the error data. In the embodiment of the present application, a method for determining error data by using a determining device for error data is taken as an example, and the determining device for error data provided in the embodiment of the present application is described.
Fig. 3 is a schematic structural diagram of an apparatus for determining error data according to an embodiment of the present application. As shown in fig. 3, the error data determining apparatus 300 includes: a receiving module 310, an obtaining module 320 and a determining module 330.
In the present application, a receiving module 310 is configured to receive a plurality of to-be-measured data tables, where the to-be-measured data tables include a plurality of target field names and target data corresponding to each of the target field names; an obtaining module 320, configured to obtain target matching rules corresponding to the target field names according to the correspondence between the preconfigured field names and the matching rules; the determining module 330 is configured to determine, when it is determined that the first data matches a first matching rule, where the first data is data corresponding to a first field name, the first matching rule is a matching rule corresponding to the first field name, and the first field name is any field name in the target field names.
In one implementation, the determining module 330 is further configured to: after the first data is determined to be error data, acquiring a first correction rule corresponding to the first field name according to a corresponding relation between a pre-configured field name and the correction rule; and correcting the first data according to the first correction rule.
In one implementation, the determining module 330 is further configured to: and after the first data is corrected according to the first correction rule, performing data processing on the corrected data according to a preset processing flow.
In one implementation, the obtaining module 320 obtains the target matching rule corresponding to each target field name according to the correspondence between the preconfigured field name and the matching rule, including: and for each to-be-tested data table in the plurality of to-be-tested data tables, obtaining target matching rules corresponding to each target field name in the to-be-tested data table in parallel according to the corresponding relation between the pre-configured field names and the matching rules.
In one implementation, the obtaining module 320 obtains the target matching rule corresponding to each target field name according to the correspondence between the preconfigured field name and the matching rule, including: and obtaining the target matching rules corresponding to the target field names in parallel according to the corresponding relation between the pre-configured field names and the matching rules.
In one implementation, the receiving module 310 is further configured to: after receiving a plurality of data tables to be tested, sending prompt information to a display interface under the condition that a target matching rule corresponding to the target field name cannot be obtained; and correcting the target data corresponding to the target field name in response to the received correction operation.
In one implementation, the determining module 330 is further configured to: and after the first data is determined to be the error data, the first data is sent to a display interface.
The error data determining device provided by the embodiment of the application can realize each process realized by the error data determining method embodiment, and in order to avoid repetition, the description is omitted.
Optionally, as shown in fig. 4, the embodiment of the present application further provides an electronic device 400, including a processor 401 and a memory 402, where the memory 402 stores a program or an instruction that can be executed on the processor 401, and the program or the instruction implements each step of the above embodiment of the method for determining error data when executed by the processor 401, and the steps achieve the same technical effects, so that repetition is avoided, and no further description is given here.
The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device.
The embodiment of the application also provides a readable storage medium, on which a program or an instruction is stored, which when executed by a processor, implements each process of the above-mentioned error data determining method embodiment, and can achieve the same technical effects, so that repetition is avoided, and no further description is given here.
Wherein the processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes computer readable storage medium such as computer readable memory ROM, random access memory RAM, magnetic or optical disk, etc.
The foregoing embodiments of the present application mainly describe differences between the embodiments, and as long as there is no contradiction between different optimization features of the embodiments, the embodiments may be combined to form a better embodiment, and in view of brevity of line text, no further description is provided herein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.

Claims (10)

1. A method for determining error data, comprising:
receiving a plurality of data tables to be tested, wherein the data tables to be tested comprise a plurality of target field names and target data corresponding to the target field names;
acquiring target matching rules corresponding to the target field names according to the corresponding relation between the pre-configured field names and the matching rules;
and under the condition that the first data is matched with a first matching rule, determining the first data as error data, wherein the first data is data corresponding to a first field name, the first matching rule is a matching rule corresponding to the first field name, and the first field name is any field name in each target field name.
2. The determination method according to claim 1, further comprising, after said determining said first data as erroneous data:
acquiring a first correction rule corresponding to the first field name according to the corresponding relation between the pre-configured field name and the correction rule;
and correcting the first data according to the first correction rule.
3. The determination method according to claim 2, further comprising, after said correcting said first data according to said first correction rule:
and carrying out data processing on the corrected data according to a preset processing flow.
4. The method according to claim 1, wherein the obtaining, according to the correspondence between the preconfigured field names and the matching rules, the target matching rules corresponding to the target field names includes:
and for each to-be-tested data table in the plurality of to-be-tested data tables, obtaining target matching rules corresponding to each target field name in the to-be-tested data table in parallel according to the corresponding relation between the pre-configured field names and the matching rules.
5. The method according to claim 1, wherein the obtaining, according to the correspondence between the preconfigured field names and the matching rules, the target matching rules corresponding to the target field names includes:
and obtaining the target matching rules corresponding to the target field names in parallel according to the corresponding relation between the pre-configured field names and the matching rules.
6. The method of determining according to claim 1, further comprising, after said receiving a plurality of tables of data to be measured:
sending prompt information to a display interface under the condition that a target matching rule corresponding to the target field name cannot be obtained;
and correcting the target data corresponding to the target field name in response to the received correction operation.
7. The determination method according to claim 1, further comprising, after said determining said first data as erroneous data:
and sending the first data to a display interface.
8. An apparatus for determining error data, comprising:
the receiving module is used for receiving a plurality of data tables to be detected, wherein the data tables to be detected comprise a plurality of target field names and target data corresponding to the target field names;
the acquisition module is used for acquiring target matching rules corresponding to the target field names according to the corresponding relation between the preset field names and the matching rules;
and the determining module is used for determining the first data as error data under the condition that the first data is determined to be matched with a first matching rule, wherein the first data is data corresponding to a first field name, the first matching rule is a matching rule corresponding to the first field name, and the first field name is any field name in each target field name.
9. An electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method of determining error data as claimed in any one of claims 1 to 7.
10. A readable storage medium, characterized in that the readable storage medium has stored thereon a program or instructions which, when executed by a processor, implement the steps of the error data determination method according to any of claims 1-7.
CN202310617947.6A 2023-05-29 2023-05-29 Error data determining method and device and electronic equipment Pending CN116680579A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310617947.6A CN116680579A (en) 2023-05-29 2023-05-29 Error data determining method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310617947.6A CN116680579A (en) 2023-05-29 2023-05-29 Error data determining method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN116680579A true CN116680579A (en) 2023-09-01

Family

ID=87788319

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310617947.6A Pending CN116680579A (en) 2023-05-29 2023-05-29 Error data determining method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN116680579A (en)

Similar Documents

Publication Publication Date Title
CN108563214B (en) Vehicle diagnosis method, device and equipment
CN106780045B (en) Policy information corrects method and apparatus
CN111382740B (en) Text picture analysis method, text picture analysis device, computer equipment and storage medium
CN112084109A (en) System test method, apparatus, device and medium
CN110034948B (en) Method and device for troubleshooting system faults
CN104869019A (en) Feedback method and apparatus for positioning faults, and server
CN110046086B (en) Expected data generation method and device for test and electronic equipment
CN112860957A (en) Method, medium and system for checking fixed value list
CN113158988A (en) Financial statement processing method and device and computer readable storage medium
CN116680579A (en) Error data determining method and device and electronic equipment
CN104504029B (en) Data convert information processing method and device
CN113779620A (en) Program file checking method and device, storage medium and computer equipment
CN110807148B (en) Equipment identifier filtering method and device, electronic equipment and storage medium
CN110298018B (en) Text data processing method, device, computer equipment and storage medium
CN110807037B (en) Data modification method and device, electronic equipment and storage medium
CN113743200A (en) Method and system for checking target network segment signal information
CN111143225A (en) Vulnerability processing method of automobile diagnosis software and related product
CN111953544B (en) Fault detection method, device, equipment and storage medium of server
CN113360491B (en) Data quality inspection method, device, electronic equipment and storage medium
CN113778804B (en) Log printing method and device, computer equipment and storage medium
CN116610734A (en) ETL-based data processing method and device, storage medium and electronic equipment
CN115860657A (en) Electric power inspection method and system based on data analysis
CN114501514A (en) Information processing method, device and equipment
CN116643978A (en) Terminal software version error correction method and device, electronic equipment and storage medium
CN117743269A (en) Bus description file error correction processing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination