CN109241074B - Automatic auditing data processing method and device, computer equipment and storage medium - Google Patents
Automatic auditing data processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109241074B CN109241074B CN201811137859.1A CN201811137859A CN109241074B CN 109241074 B CN109241074 B CN 109241074B CN 201811137859 A CN201811137859 A CN 201811137859A CN 109241074 B CN109241074 B CN 109241074B
- Authority
- CN
- China
- Prior art keywords
- data
- imported
- task node
- writing
- importing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The application discloses an automatic auditing data processing method, an automatic auditing data processing device, computer equipment and a storage medium. The method comprises the following steps: acquiring a preset data importing process, and importing data to be imported into a corresponding target table in sequence according to the sequence of task nodes in the data importing process; judging whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence; if the data to be imported is successfully imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the state of the data to be imported into the target form into the data to be imported corresponding to the task node, and filling the identification bit which is successfully written into the writing state identification field; if the data to be imported is successfully imported into the target form according to the data importing process, sending notification information of successful data importing to an uploading end of the data to be imported. According to the method, the report to be sent to the auditing terminal is automatically filled according to the corresponding requirement through a data processing technology, and the data processing efficiency is improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to an automatic processing method and apparatus for audit data, a computer device, and a storage medium.
Background
At present, service data is processed, filled into a designated audit transmission form and then sent to an audit mechanism for audit, a semi-automatic filling mode is generally adopted, and when errors occur in the process of filling the service data, the process of filling the service data is generally restarted, so that the efficiency of data processing is low.
Content of the application
The embodiment of the application provides an automatic processing method, an automatic processing device, computer equipment and a storage medium for auditing data, and aims to solve the problem that in the prior art, service data are processed, filled into a designated auditing table and then sent to an auditing mechanism for auditing in a semi-automatic filling mode, so that the efficiency of data processing is low.
In a first aspect, an embodiment of the present application provides a method for automatically processing audit data, including:
acquiring a preset data importing process, and importing data to be imported into a corresponding target table in sequence according to the sequence of task nodes in the data importing process;
Judging whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence;
if the data to be imported is successfully imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the state of the data to be imported into the target form into the data to be imported corresponding to the task node, and filling the identification bit which is successfully written into the writing state identification field;
if the data to be imported is successfully imported into the target form according to the data importing process, sending notification information of successful data importing to an uploading end of the data to be imported.
In a second aspect, an embodiment of the present application provides an audit data automatic processing apparatus, including:
the data importing unit is used for acquiring a preset data importing process and importing data to be imported into a corresponding target form in sequence according to the sequence of task nodes in the data importing process;
the judging unit is used for judging whether the data to be imported is successfully imported into the corresponding target form in sequence according to the task node;
a data import success control unit, configured to, if importing data to be imported into a corresponding target table according to a task node is successful, add a writing state identification field for identifying a state of writing the data into the target table into the data to be imported corresponding to the task node, and fill an identification bit that is successfully written into the writing state identification field;
And the introduction success notification unit is used for sending notification information of successful data introduction to the uploading end of the data to be introduced if the data to be introduced is successfully introduced into the target form according to the data introduction process.
In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the method for automatically processing audit data according to the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present application further provides a storage medium, where the storage medium stores a computer program, where the computer program includes program instructions, where the program instructions when executed by a processor cause the processor to perform the method for automatically processing audit data according to the first aspect.
The embodiment of the application provides an audit data automatic processing method, an audit data automatic processing device, computer equipment and a storage medium. According to the method, a preset data importing flow is obtained, and data to be imported are imported into corresponding target forms in sequence according to the sequence of task nodes in the data importing flow; judging whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence; if the data to be imported is successfully imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the state of the data to be imported into the target form into the data to be imported corresponding to the task node, and filling the identification bit which is successfully written into the writing state identification field; if the data to be imported is successfully imported into the target form according to the data importing process, sending notification information of successful data importing to an uploading end of the data to be imported. The method realizes automatic filling of the report to be sent to the auditing terminal according to the corresponding requirement, and improves the efficiency of data processing.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of an automatic processing method for audit data according to an embodiment of the present application;
fig. 2 is a schematic sub-flowchart of an automatic processing method for audit data according to an embodiment of the present application;
FIG. 3 is another flow chart of the method for automatically processing audit data according to the embodiment of the present application;
fig. 4 is another flow chart of an automatic processing method for audit data according to an embodiment of the present application;
FIG. 5 is another flow chart of the method for automatically processing audit data according to the embodiment of the present application;
FIG. 6 is a schematic block diagram of an audit data automatic processing apparatus provided by an embodiment of the present application;
FIG. 7 is a schematic block diagram of a subunit of an audit data automatic processing apparatus according to an embodiment of the present application;
FIG. 8 is another schematic block diagram of an audit data automatic processing apparatus provided by an embodiment of the present application;
FIG. 9 is another schematic block diagram of an audit data automatic processing apparatus provided by an embodiment of the present application;
FIG. 10 is another schematic block diagram of an audit data automatic processing apparatus provided by an embodiment of the present application;
fig. 11 is a schematic block diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Referring to fig. 1, fig. 1 is a schematic flow chart of an automatic processing method for audit data according to an embodiment of the present application. The method is applied to the management server, and is executed through application software installed in the management server, and a plurality of user terminals are connected with the management server through a network to transmit information. The user terminal is terminal equipment with an internet access function, such as a desktop computer, a notebook computer, a tablet computer or a mobile phone; the management server is an enterprise terminal for auditing data automatic processing.
As shown in fig. 1, the method includes steps S101 to S104.
S101, acquiring a preset data import flow, and importing data to be imported into a corresponding target table in sequence according to the sequence of task nodes in the data import flow.
In the present embodiment, step S101 is performed in the load side of the management server. The method comprises the steps of pre-configuring a data import flow at each load end, wherein a plurality of data import steps can be set in the data import flow, namely a plurality of corresponding task nodes, and each task node automatically fills data meeting the conditions into a target table through a database script (such as an SQL script) (generally, the conditions met by screened data are edited in the SQL script corresponding to each task node, for example, a certain field value is equal to a value A). The source data (namely the data to be imported) is automatically imported into the target form through the data importing process, manual intervention is not needed, and the efficiency of data processing is improved.
For example, the data import flow includes N task nodes (e.g., 10 task nodes), where a task corresponding to a first task node is to write a part of the specified fields in table 1 of the data to be imported into the target table 1, a task corresponding to a second task node is to write a part of the specified fields in table 2 of the data to be imported into the target table 2, … …, and a task corresponding to an nth task node is to write a part of the specified fields in table N of the data to be imported into the target table N. Through the data import process, the data to be imported (i.e. the data to be submitted for auditing) can be written into the target form, and then the target form is submitted to a server (i.e. an auditing end) of the auditing institution for auditing.
In one embodiment, as shown in fig. 2, step S101 includes:
s1011, acquiring source end information of data to be imported, adding a source identification field into the data to be imported according to the source end information, and filling the source end information into the source identification field to obtain data after source identification;
s1012, dividing the data after the source identification according to the task nodes to obtain partitioned data corresponding to each task node one by one, adding a step sequence number field to each partitioned data, and filling the task node sequence number corresponding to each partitioned data into the step sequence number field to obtain the data after the step identification;
S1013, importing the data of the data identified in the step into a corresponding target table in sequence.
In this embodiment, the load side obtains source side information of the data to be imported, not only writes the source side information of the data to be imported into the data source identification field in the created storage table, but also adds a source identification field in the data to be imported, and fills the source side information into the source identification field. The source identification field is added to the data to be imported, so that when the data to be imported is sequentially imported into the corresponding target form according to the sequence of task nodes in the data import flow, if a write error occurs, the data to be imported can be directly informed to an uploading end of the data to be imported according to the source identification field to re-upload the data, or the data to be imported can be re-acquired from a backup of the data to be imported of a load end (namely, when the data to be imported is acquired by a certain load end, a form for storing the data to be imported can be established to store the data to be imported).
And dividing the data after the source identification according to the task nodes to obtain block data corresponding to each task node one by one, wherein the aim of the method is that the load end writes the corresponding data into the corresponding table according to the task nodes. For example, 10000 pieces of data are included in the source identification data, the data importing process includes 10 task nodes, the task corresponding to the first task node is to import the data 1-1000 into the target table 1, a step sequence number field is added in the data 1-1000, and 1 is filled in the step sequence number field (which indicates the first task node, and the execution sequence is the first execution sequence); the next 9000 pieces of data are also added with step sequence number fields referring to the first 1000 pieces of data and correspond to filling task node sequence numbers.
S102, judging whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence;
in this embodiment, it is determined whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence, and the criterion is whether the automatic importing process of the data is interrupted, if not, the importing is successful, and if interrupted, the importing is failed. Judging whether the automatic importing process is interrupted, specifically judging whether the number of data strips screened by the data script corresponding to the task node is the same as the number of target data strips expected by the task node, if the number of data strips screened by the data script corresponding to the task node is the same as the number of target data strips expected by the task node, indicating that the automatic importing process of the data is not interrupted, and if the number of data strips screened by the data script corresponding to the task node is smaller than the number of target data strips expected by the task node, indicating that the automatic importing process of the data is not interrupted.
S103, if the data to be imported is successfully imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the writing state of the data into the target form into the data to be imported corresponding to the task node, and filling the identification bit which is successfully written into the writing state identification field;
In this embodiment, if the writing status identifier field added to the imported data is all the successfully written identifier bits, it indicates that the whole automatic importing process has no error. For example, when the task corresponding to the first task node is to import the data 1-1000 into the target table 1, if writing the data 1 into all the 1000 pieces of data is successful, a writing state identification field for identifying the state of writing the data into the target table is added into the data after the step identification corresponding to the data 1-1000, and an identification bit (such as success or number 1) for indicating the writing success is filled into the writing state identification field, so as to quickly acquire which data has been successfully written into the target table in the subsequent flow through writing the writing state identification bit.
And S104, if the data to be imported is successfully imported into the target form according to the data importing process, sending notification information of successful data importing to an uploading end of the data to be imported.
In this embodiment, if the data to be imported is successfully imported into the target table according to the data import procedure, the automatic filling process is completed, and the uploading end of the data to be imported needs to be notified in time. At this time, the uploading end of the data to be imported can choose to ignore the notification information, and the data of the target form is imported into the audit data reporting template form for automatic uploading after the system time reaches the preset uploading time; or the uploading end of the data to be imported can send out an instruction for immediately importing the data of the target form into the audit data reporting template form in a multiplexing way so as to automatically upload the data.
In one embodiment, as shown in fig. 3, step S104 includes:
s1041, acquiring a prestored audit data reporting template table;
s1042, importing the data in the target form into an audit data reporting template form to obtain an audit data form.
In this embodiment, the sub-table included in the target table and the fields included in each sub-table are the same as the audit data reporting template table. However, compared with the target form, the audit data reporting template form may have more template unnecessary contents (the contents are to meet the format requirement of the audit report), so that the temporary form, i.e. the data to be imported is written into the target form for storage, so that the time for locating the target area in the automatic data writing process can be reduced. If the data to be imported is directly written into the audit data reporting template form, the area where the target field is required to be located is written, and the locating process is greatly interfered by the template unnecessary content.
In one embodiment, as shown in fig. 4, step S1042 includes:
s1043, sending the auditing data form to the corresponding auditing end according to the preset uploading time.
In this embodiment, when the system time reaches the preset uploading time, the data of the target form is led into the audit data reporting template form again for automatic uploading. Therefore, the network non-congestion time (such as 20:00 a day to 6:00 a day) of the management server can be set, and the auditing data table is sent to the corresponding auditing end, so that the problems of uploading failure or low uploading rate caused by network congestion are avoided.
In one embodiment, as shown in fig. 5, step S102 includes:
s105, if the data to be imported fails to be imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the writing state of the data into the target form into the data to be imported corresponding to the task node, filling an identification bit of the writing success corresponding to the data of the writing success of the target form into the data to be imported into the writing state identification field, and filling an identification bit of the writing failure corresponding to the data of the writing failure of the target form into the data to be imported into the writing state identification field;
s106, positioning the abnormal position of the data to be imported in the target table according to the identification bit of the writing failure, and if the task node corresponding to the abnormal position is detected to be successful in obstacle removal, writing the data to be imported corresponding to the task node corresponding to the abnormal position in the target table.
For example, when the task corresponding to the first task node is to import the data of the number 1 to the target table 1, if the first 500 pieces of data in the 1000 pieces of data are successfully written into the target table 1, a writing state identification field for identifying the state of the data to be written into the target table is added in the data after the step identification corresponding to the data of the number 1 to 500, and an identification bit (such as success or number 1) which is successfully written into the writing state identification field is filled; if writing of the 501 st data into the target table 1 fails, adding a writing state identification field for identifying the writing state of the data into the target table in the step identification data corresponding to the 501 st data, and filling an identification bit (such as fail or digital 0) with the writing failure into the writing state identification field.
Obviously, a sequence number field may be added to the data after the step identifier to indicate that each data in the step is assigned a sequence number, and the sequence number is assigned to each data in the step in ascending order. Therefore, once the data has faults in the data writing process, the user can quickly know which number of data has problems in the writing process, and therefore the quick fault positioning is realized.
If the writing failure identification bit is detected in the writing state identification field, the task node corresponding to the step is indicated to have a fault, an automatic writing script corresponding to the task node needs to be modified to remove the fault, after the fault removal is successful, data in a target table corresponding to the task node is firstly cleared, and then all data corresponding to the task node is written into the target table again.
According to the method, the report to be sent to the auditing terminal is automatically filled according to the corresponding requirement through the preset data import procedure, and the data processing efficiency is improved.
The embodiment of the application also provides an audit data automatic processing device which is used for executing any embodiment of the audit data automatic processing method. Specifically, referring to fig. 6, fig. 6 is a schematic block diagram of an audit data automatic processing apparatus according to an embodiment of the present application. The audit data automatic processing apparatus 100 may be configured in a management server.
As shown in fig. 6, the audit data automatic processing apparatus 100 includes a data import unit 101, a judgment unit 102, a data import success control unit 103, and an import success notification unit 104.
The data importing unit 101 is configured to obtain a preset data importing process, and import data to be imported into a corresponding target table sequentially according to the order of task nodes in the data importing process.
In the present embodiment, step S101 is performed in the load side of the management server. The method comprises the steps of pre-configuring a data import flow at each load end, wherein a plurality of data import steps can be set in the data import flow, namely a plurality of task nodes are corresponding, and data meeting the conditions are automatically filled into a target table through a database script (such as an SQL script) in each task node. The source data (namely the data to be imported) is automatically imported into the target form through the data importing process, manual intervention is not needed, and the efficiency of data processing is improved.
For example, the data import flow includes N task nodes (e.g., 10 task nodes), where a task corresponding to a first task node is to write a part of the specified fields in table 1 of the data to be imported into the target table 1, a task corresponding to a second task node is to write a part of the specified fields in table 2 of the data to be imported into the target table 2, … …, and a task corresponding to an nth task node is to write a part of the specified fields in table N of the data to be imported into the target table N. Through the data import process, the data to be imported (i.e. the data to be submitted for auditing) can be written into the target form, and then the target form is submitted to a server (i.e. an auditing end) of the auditing institution for auditing.
In one embodiment, as shown in fig. 7, the data import unit 101 includes:
the source identification unit 1011 is configured to obtain source information of data to be imported, add a source identification field to the data to be imported according to the source information, and fill the source information into the source identification field to obtain data after source identification;
step identification unit 1012, configured to divide the data after source identification according to task nodes to obtain block data corresponding to each task node one by one, add a step sequence number field to each block data, and fill the task node sequence number corresponding to each block data into the step sequence number field to obtain data after step identification;
a sequential writing unit 1013 for sequentially importing the data of the data identified by the step into the corresponding target table.
In this embodiment, the load side obtains source side information of the data to be imported, not only writes the source side information of the data to be imported into the data source identification field in the created storage table, but also adds a source identification field in the data to be imported, and fills the source side information into the source identification field. The source identification field is added to the data to be imported, so that when the data to be imported is sequentially imported into the corresponding target form according to the sequence of task nodes in the data import flow, if a write error occurs, the data to be imported can be directly informed to an uploading end of the data to be imported according to the source identification field to re-upload the data, or the data to be imported can be re-acquired from a backup of the data to be imported of a load end (namely, when the data to be imported is acquired by a certain load end, a form for storing the data to be imported can be established to store the data to be imported).
And dividing the data after the source identification according to the task nodes to obtain block data corresponding to each task node one by one, wherein the aim of the method is that the load end writes the corresponding data into the corresponding table according to the task nodes. For example, 10000 pieces of data are included in the source identification data, the data importing process includes 10 task nodes, the task corresponding to the first task node is to import the data 1-1000 into the target table 1, a step sequence number field is added in the data 1-1000, and 1 is filled in the step sequence number field (which indicates the first task node, and the execution sequence is the first execution sequence); the next 9000 pieces of data are also added with step sequence number fields referring to the first 1000 pieces of data and correspond to filling task node sequence numbers.
And the judging unit 102 is used for judging whether the data to be imported is successfully imported into the corresponding target table in sequence according to the task node.
In this embodiment, it is determined whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence, and the criterion is whether the automatic importing process of the data is interrupted, if not, the importing is successful, and if interrupted, the importing is failed. Judging whether the automatic importing process is interrupted, specifically judging whether the number of data strips screened by the data script corresponding to the task node is the same as the number of target data strips expected by the task node, if the number of data strips screened by the data script corresponding to the task node is the same as the number of target data strips expected by the task node, indicating that the automatic importing process of the data is not interrupted, and if the number of data strips screened by the data script corresponding to the task node is smaller than the number of target data strips expected by the task node, indicating that the automatic importing process of the data is not interrupted.
And the data import success control unit 103 is configured to, if the importing of the data to be imported into the corresponding target table according to the task node is successful, add a writing state identification field for identifying the state of writing the data into the target table to the data to be imported corresponding to the task node, and fill the writing successful identification bit into the writing state identification field.
In this embodiment, if the writing status identifier field added to the imported data is all the successfully written identifier bits, it indicates that the whole automatic importing process has no error. For example, when the task corresponding to the first task node is to import the data 1-1000 into the target table 1, if writing the data 1 into all the 1000 pieces of data is successful, a writing state identification field for identifying the state of writing the data into the target table is added into the data after the step identification corresponding to the data 1-1000, and an identification bit (such as success or number 1) for indicating the writing success is filled into the writing state identification field, so as to quickly acquire which data has been successfully written into the target table in the subsequent flow through writing the writing state identification bit.
And the import success notification unit 104 is configured to send notification information of successful data import to the uploading end of the data to be imported if the data to be imported is successfully imported into the target table according to the data import procedure.
In this embodiment, if the data to be imported is successfully imported into the target table according to the data import procedure, the automatic filling process is completed, and the uploading end of the data to be imported needs to be notified in time. At this time, the uploading end of the data to be imported can choose to ignore the notification information, and the data of the target form is imported into the audit data reporting template form for automatic uploading after the system time reaches the preset uploading time; or the uploading end of the data to be imported can send out an instruction for immediately importing the data of the target form into the audit data reporting template form in a multiplexing way so as to automatically upload the data.
In one embodiment, as shown in fig. 8, the audit data automatic processing apparatus 100 further includes:
the audit form template acquiring unit 1041 is configured to acquire a pre-stored audit data reporting template form;
the audit form filling unit 1042 is used for guiding the data in the target form into the audit data reporting template form again to obtain an audit data form.
In this embodiment, the sub-table included in the target table and the fields included in each sub-table are the same as the audit data reporting template table. However, compared with the target form, the audit data reporting template form may have more template unnecessary contents (the contents are to meet the format requirement of the audit report), so that the temporary form, i.e. the data to be imported is written into the target form for storage, so that the time for locating the target area in the automatic data writing process can be reduced. If the data to be imported is directly written into the audit data reporting template form, the area where the target field is required to be located is written, and the locating process is greatly interfered by the template unnecessary content.
In one embodiment, as shown in fig. 9, the audit data automatic processing apparatus 100 further includes:
the timing uploading unit 1043 is configured to send the audit data table to the corresponding audit end according to a preset uploading time.
In this embodiment, when the system time reaches the preset uploading time, the data of the target form is led into the audit data reporting template form again for automatic uploading. Therefore, the network non-congestion time (such as 20:00 a day to 6:00 a day) of the management server can be set, and the auditing data table is sent to the corresponding auditing end, so that the problems of uploading failure or low uploading rate caused by network congestion are avoided.
In one embodiment, as shown in fig. 10, the audit data automatic processing apparatus 100 further includes:
a data import failure control unit 105, configured to, if there is a failure to import data to be imported into a corresponding target table according to a task node, add a writing state identification field for identifying a writing state of the data to the target table to the data to be imported corresponding to the task node, fill an identification bit of a writing success corresponding to the data to be imported, which is written into the target table successfully, into the writing state identification field, and fill an identification bit of a writing failure corresponding to the data to be imported, which is written into the target table failed, into the writing state identification field;
And the exception handling unit 106 is configured to locate an exception position in the target table for writing the data to be imported according to the identification bit of the writing failure, and if it is detected that the task node corresponding to the exception position is successful in obstacle removal, write the data to be imported corresponding to the task node corresponding to the exception position into the target table.
For example, when the task corresponding to the first task node is to import the data of the number 1 to the target table 1, if the first 500 pieces of data in the 1000 pieces of data are successfully written into the target table 1, a writing state identification field for identifying the state of the data to be written into the target table is added in the data after the step identification corresponding to the data of the number 1 to 500, and an identification bit (such as success or number 1) which is successfully written into the writing state identification field is filled; if writing of the 501 st data into the target table 1 fails, adding a writing state identification field for identifying the writing state of the data into the target table in the step identification data corresponding to the 501 st data, and filling an identification bit (such as fail or digital 0) with the writing failure into the writing state identification field.
Obviously, a sequence number field may be added to the data after the step identifier to indicate that each data in the step is assigned a sequence number, and the sequence number is assigned to each data in the step in ascending order. Therefore, once the data has faults in the data writing process, the user can quickly know which number of data has problems in the writing process, and therefore the quick fault positioning is realized.
If the writing failure identification bit is detected in the writing state identification field, the task node corresponding to the step is indicated to have a fault, an automatic writing script corresponding to the task node needs to be modified to remove the fault, after the fault removal is successful, data in a target table corresponding to the task node is firstly cleared, and then all data corresponding to the task node is written into the target table again.
The device realizes automatic filling of the report to be sent to the auditing terminal according to the corresponding requirement through the preset data import procedure, and improves the data processing efficiency.
The above-described audit data automatic processing means may be implemented in the form of a computer program which may be run on a computer device as shown in fig. 11.
Referring to fig. 11, fig. 11 is a schematic block diagram of a computer device according to an embodiment of the present application.
With reference to FIG. 11, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform an audit data automated processing method.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of a computer program 5032 in the non-volatile storage medium 503, which computer program 5032, when executed by the processor 502, causes the processor 502 to perform an audit data automated processing method.
The network interface 505 is used for network communications, such as providing for the transmission of data traffic for network access, etc. It will be appreciated by those skilled in the art that the structure shown in FIG. 11 is merely a block diagram of some of the structures associated with the present inventive arrangements and does not constitute a limitation of the computer device 500 to which the present inventive arrangements may be applied, and that a particular computer device 500 may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
Wherein the processor 502 is configured to execute a computer program 5032 stored in a memory to perform the following functions: acquiring a preset data importing process, and importing data to be imported into a corresponding target table in sequence according to the sequence of task nodes in the data importing process; judging whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence; if the data to be imported is successfully imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the state of the data to be imported into the target form into the data to be imported corresponding to the task node, and filling the identification bit which is successfully written into the writing state identification field; if the data to be imported is successfully imported into the target form according to the data importing process, sending notification information of successful data importing to an uploading end of the data to be imported.
In one embodiment, when the processor 502 performs the following operations in order to import the data to be imported into the corresponding target table according to the order of the task nodes in the data import flow: acquiring source end information of data to be imported, adding a source identification field into the data to be imported according to the source end information, and filling the source end information into the source identification field to obtain data after source identification; dividing the data after the source identification according to the task nodes to obtain partitioned data corresponding to each task node one by one, adding a step sequence number field to each partitioned data, and filling the task node sequence number corresponding to each partitioned data into the step sequence number field to obtain the data after the step identification; and sequentially importing the data of the data identified in the step into a corresponding target table.
In one embodiment, after executing the sending of the notification information that the data import is successful to the uploading end of the data to be imported, the processor 502 further executes the following operations: acquiring a prestored audit data reporting template form; and importing the data in the target form into an audit data reporting template form to obtain an audit data form.
In one embodiment, after executing the above-mentioned importing the data in the target table into the audit data reporting template table to obtain the audit data table, the processor 502 further executes the following operations: and sending the auditing data form to a corresponding auditing end according to the preset uploading time.
In one embodiment, the processor 502 further performs the following operations after performing the determination as to whether the sequential importing of the data to be imported into the corresponding target table according to the task node is successful: if the data to be imported fails to be imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the writing state of the data to be imported into the target form into the data to be imported corresponding to the task node, filling an identification bit of the writing success corresponding to the data of the writing success of the target form into the data to be imported into the writing state identification field, and filling an identification bit of the writing failure corresponding to the data of the writing failure of the target form into the data to be imported into the writing state identification field; and positioning the abnormal position of the data to be imported in the target table according to the identification bit of the writing failure, and if the task node corresponding to the abnormal position is detected to be successful in obstacle removal, writing the data to be imported corresponding to the task node corresponding to the abnormal position in the target table.
Those skilled in the art will appreciate that the embodiment of the computer device shown in fig. 11 is not limiting of the specific construction of the computer device, and in other embodiments, the computer device may include more or less components than those shown, or certain components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may include only a memory and a processor, and in such embodiments, the structure and function of the memory and the processor are consistent with the embodiment shown in fig. 11, and will not be described again.
It should be appreciated that in embodiments of the present application, the processor 502 may be a central processing unit (CentralProcessing Unit, CPU), the processor 502 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific IntegratedCircuit, ASIC), off-the-shelf Programmable gate arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In another embodiment of the present application, a storage medium is provided. The storage medium may be a non-volatile computer readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. The program instructions, when executed by a processor, implement the steps of: acquiring a preset data importing process, and importing data to be imported into a corresponding target table in sequence according to the sequence of task nodes in the data importing process; judging whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence; if the data to be imported is successfully imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the state of the data to be imported into the target form into the data to be imported corresponding to the task node, and filling the identification bit which is successfully written into the writing state identification field; if the data to be imported is successfully imported into the target form according to the data importing process, sending notification information of successful data importing to an uploading end of the data to be imported.
In one embodiment, the program instructions when executed by a processor perform the steps of: acquiring source end information of data to be imported, adding a source identification field into the data to be imported according to the source end information, and filling the source end information into the source identification field to obtain data after source identification; dividing the data after the source identification according to the task nodes to obtain partitioned data corresponding to each task node one by one, adding a step sequence number field to each partitioned data, and filling the task node sequence number corresponding to each partitioned data into the step sequence number field to obtain the data after the step identification; and sequentially importing the data of the data identified in the step into a corresponding target table.
In one embodiment, the program instructions when executed by a processor perform the steps of: acquiring a prestored audit data reporting template form; and importing the data in the target form into an audit data reporting template form to obtain an audit data form.
In one embodiment, the program instructions when executed by a processor perform the steps of: and sending the auditing data form to a corresponding auditing end according to the preset uploading time.
In one embodiment, the program instructions when executed by a processor perform the steps of: if the data to be imported fails to be imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the writing state of the data to be imported into the target form into the data to be imported corresponding to the task node, filling an identification bit of the writing success corresponding to the data of the writing success of the target form into the data to be imported into the writing state identification field, and filling an identification bit of the writing failure corresponding to the data of the writing failure of the target form into the data to be imported into the writing state identification field; and positioning the abnormal position of the data to be imported in the target table according to the identification bit of the writing failure, and if the task node corresponding to the abnormal position is detected to be successful in obstacle removal, writing the data to be imported corresponding to the task node corresponding to the abnormal position in the target table.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus, device and unit described above may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein. Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the units is merely a logical function division, there may be another division manner in actual implementation, or units having the same function may be integrated into one unit, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices, or elements, or may be an electrical, mechanical, or other form of connection.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present application.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units may be stored in a storage medium if implemented in the form of software functional units and sold or used as stand-alone products. Based on such understanding, the technical solution of the present application is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
While the application has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (7)
1. An automatic processing method for audit data is characterized by comprising the following steps:
acquiring a preset data importing process, and importing data to be imported into a corresponding target table in sequence according to the sequence of task nodes in the data importing process;
judging whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence;
if the data to be imported is successfully imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the state of the data to be imported into the target form into the data to be imported corresponding to the task node, and filling the identification bit which is successfully written into the writing state identification field;
if the data to be imported is successfully imported into the target form according to the data importing process, sending notification information of successful data importing to an uploading end of the data to be imported;
After the notification information that the data is successfully imported is sent to the uploading end of the data to be imported, the method further comprises the following steps:
acquiring a prestored audit data reporting template form;
the data in the target form is led into an audit data reporting template form in a multiplexing way to obtain an audit data form;
the judging whether the data to be imported is successfully imported into the corresponding target table according to the task node in sequence comprises the following steps:
judging whether the number of data screened by the data script corresponding to the task node is the same as the expected target number of data of the task node;
after judging whether the data to be imported is successfully imported into the corresponding target table in sequence according to the task node, the method further comprises the following steps:
if the data to be imported fails to be imported into the corresponding target form according to the task node, adding a writing state identification field for identifying the writing state of the data to be imported into the target form into the data to be imported corresponding to the task node, filling an identification bit of the writing success corresponding to the data of the writing success of the target form into the data to be imported into the writing state identification field, and filling an identification bit of the writing failure corresponding to the data of the writing failure of the target form into the data to be imported into the writing state identification field;
And positioning the abnormal position of the data to be imported in the target table according to the identification bit of the writing failure, and if the task node corresponding to the abnormal position is detected to be successful in obstacle removal, writing the data to be imported corresponding to the task node corresponding to the abnormal position in the target table.
2. The method for automatically processing audit data according to claim 1, wherein the sequentially importing the data to be imported into the corresponding target table according to the order of the task nodes in the data importing flow includes:
acquiring source end information of data to be imported, adding a source identification field into the data to be imported according to the source end information, and filling the source end information into the source identification field to obtain data after source identification;
dividing the data after the source identification according to the task nodes to obtain partitioned data corresponding to each task node one by one, adding a step sequence number field to each partitioned data, and filling the task node sequence number corresponding to each partitioned data into the step sequence number field to obtain the data after the step identification;
and sequentially importing the data of the data identified in the step into a corresponding target table.
3. The method for automatically processing audit data according to claim 1, wherein after the data in the target form is reintroduced into the audit data reporting template form to obtain the audit data form, the method further comprises:
And sending the auditing data form to a corresponding auditing end according to the preset uploading time.
4. An audit data automatic processing device, comprising:
the data importing unit is used for acquiring a preset data importing process and importing data to be imported into a corresponding target form in sequence according to the sequence of task nodes in the data importing process;
the judging unit is used for judging whether the data to be imported is successfully imported into the corresponding target form in sequence according to the task node;
a data import success control unit, configured to, if importing data to be imported into a corresponding target table according to a task node is successful, add a writing state identification field for identifying a state of writing the data into the target table into the data to be imported corresponding to the task node, and fill an identification bit that is successfully written into the writing state identification field;
the data transmission unit is used for transmitting the data transmission information to the data transmission unit according to the data transmission process;
the audit form template acquisition unit is used for acquiring a prestored audit data reporting template form;
the auditing form filling unit is used for guiding the data in the target form into the auditing data reporting template form again to obtain an auditing data form;
The judging unit includes:
judging whether the number of data screened by the data script corresponding to the task node is the same as the expected target number of data of the task node;
after the judging unit, the method further comprises:
a data import failure control unit, configured to, if there is a failure to import data to be imported into a corresponding target table according to a task node, add a writing state identification field for identifying a state of writing the data into the target table to the data to be imported corresponding to the task node, fill an identification bit of a successful writing corresponding to the data of the successful writing into the target table in the data to be imported into the writing state identification field, and fill an identification bit of a writing failure corresponding to the data of the failure writing into the target table in the data to be imported into the writing state identification field;
the exception handling unit is used for locating the exception position of the data to be imported in the target table according to the identification bit of the writing failure, and if the task node corresponding to the exception position is detected to be successful in obstacle elimination, the data to be imported corresponding to the task node corresponding to the exception position is written in the target table.
5. The automated audit data processing apparatus according to claim 4 wherein the data importing unit includes:
The source identification unit is used for acquiring source end information of the data to be imported, adding a source identification field into the data to be imported according to the source end information, and filling the source end information into the source identification field to obtain data after source identification;
the step identification unit is used for dividing the data after the source identification according to the task nodes to obtain partitioned data corresponding to each task node one by one, adding a step sequence number field to each partitioned data, and filling the task node sequence number corresponding to each partitioned data into the step sequence number field to obtain the data after the step identification;
and the sequential writing unit is used for sequentially importing the data of the data identified by the steps into the corresponding target table.
6. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the audit data automatic processing method according to any one of claims 1 to 3 when executing the computer program.
7. A storage medium storing a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the audit data automated processing method according to any of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811137859.1A CN109241074B (en) | 2018-09-28 | 2018-09-28 | Automatic auditing data processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811137859.1A CN109241074B (en) | 2018-09-28 | 2018-09-28 | Automatic auditing data processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109241074A CN109241074A (en) | 2019-01-18 |
| CN109241074B true CN109241074B (en) | 2023-09-26 |
Family
ID=65057649
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811137859.1A Active CN109241074B (en) | 2018-09-28 | 2018-09-28 | Automatic auditing data processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109241074B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110728128A (en) * | 2019-08-30 | 2020-01-24 | 平安证券股份有限公司 | Data report generation method, device, medium and electronic equipment |
| CN112749533B (en) * | 2019-10-30 | 2023-08-04 | 腾讯科技（深圳）有限公司 | Data input control method and device, electronic equipment and storage medium |
| CN116319718B (en) * | 2023-03-10 | 2023-12-12 | 北京振中电子技术有限公司 | Cloud data storage processing method, system, equipment and medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4658351A (en) * | 1984-10-09 | 1987-04-14 | Wang Laboratories, Inc. | Task control means for a multi-tasking data processing system |
| AU2645501A (en) * | 2000-03-15 | 2001-09-20 | Ilaunch Pty Ltd | Data importer |
| CN103995908A (en) * | 2014-06-17 | 2014-08-20 | 山东中创软件工程股份有限公司 | Method and device for importing data |
| CN105069149A (en) * | 2015-08-24 | 2015-11-18 | 电子科技大学 | Structured line data-oriented distributed parallel data importing method |
| CN105095240A (en) * | 2014-05-04 | 2015-11-25 | 中国银联股份有限公司 | Database data sample acquisition |
| CN106126374A (en) * | 2016-06-22 | 2016-11-16 | 腾讯科技（深圳）有限公司 | Method for writing data, method for reading data and device |
| CN106776962A (en) * | 2016-12-05 | 2017-05-31 | 济南浪潮高新科技投资发展有限公司 | A kind of general Excel data import multiple database physical table methods |
| CN107025289A (en) * | 2017-04-14 | 2017-08-08 | 腾讯科技（深圳）有限公司 | The method and relevant device of a kind of data processing |
| CN107864209A (en) * | 2017-11-17 | 2018-03-30 | 北京联想超融合科技有限公司 | The method, apparatus and server of data write-in |
| CN108171468A (en) * | 2017-12-15 | 2018-06-15 | 东软集团股份有限公司 | The method, apparatus and storage medium and electronic equipment of data processing |
| CN108228730A (en) * | 2017-12-11 | 2018-06-29 | 深圳市买买提信息科技有限公司 | Data lead-in method, device, computer equipment and readable storage medium storing program for executing |
| CN108460068A (en) * | 2017-11-28 | 2018-08-28 | 平安科技（深圳）有限公司 | Method, apparatus, storage medium and the terminal that report imports and exports |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1591914A1 (en) * | 2004-04-26 | 2005-11-02 | Sap Ag | Method, computer program product and device for importing a plurality of data sets into a system |
| US10503718B2 (en) * | 2016-07-06 | 2019-12-10 | Micro Focus Llc | Parallel transfers of electronic data |
-
2018
- 2018-09-28 CN CN201811137859.1A patent/CN109241074B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4658351A (en) * | 1984-10-09 | 1987-04-14 | Wang Laboratories, Inc. | Task control means for a multi-tasking data processing system |
| AU2645501A (en) * | 2000-03-15 | 2001-09-20 | Ilaunch Pty Ltd | Data importer |
| CN105095240A (en) * | 2014-05-04 | 2015-11-25 | 中国银联股份有限公司 | Database data sample acquisition |
| CN103995908A (en) * | 2014-06-17 | 2014-08-20 | 山东中创软件工程股份有限公司 | Method and device for importing data |
| CN105069149A (en) * | 2015-08-24 | 2015-11-18 | 电子科技大学 | Structured line data-oriented distributed parallel data importing method |
| CN106126374A (en) * | 2016-06-22 | 2016-11-16 | 腾讯科技（深圳）有限公司 | Method for writing data, method for reading data and device |
| CN106776962A (en) * | 2016-12-05 | 2017-05-31 | 济南浪潮高新科技投资发展有限公司 | A kind of general Excel data import multiple database physical table methods |
| CN107025289A (en) * | 2017-04-14 | 2017-08-08 | 腾讯科技（深圳）有限公司 | The method and relevant device of a kind of data processing |
| CN107864209A (en) * | 2017-11-17 | 2018-03-30 | 北京联想超融合科技有限公司 | The method, apparatus and server of data write-in |
| CN108460068A (en) * | 2017-11-28 | 2018-08-28 | 平安科技（深圳）有限公司 | Method, apparatus, storage medium and the terminal that report imports and exports |
| CN108228730A (en) * | 2017-12-11 | 2018-06-29 | 深圳市买买提信息科技有限公司 | Data lead-in method, device, computer equipment and readable storage medium storing program for executing |
| CN108171468A (en) * | 2017-12-15 | 2018-06-15 | 东软集团股份有限公司 | The method, apparatus and storage medium and electronic equipment of data processing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109241074A (en) | 2019-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109241074B (en) | Automatic auditing data processing method and device, computer equipment and storage medium | |
| US10476849B2 (en) | Monitoring and alerting for improper memory device replacement | |
| US20170286088A1 (en) | Configuration of automated update routines | |
| US8892712B2 (en) | Upgrading enterprise managers | |
| US20170060686A1 (en) | Intelligent read strategy within a dispersed storage network (dsn) | |
| CN104092718A (en) | Distributed system and configuration information updating method in distributed system | |
| CN108345617B (en) | Data synchronization method and device and electronic equipment | |
| KR20110074096A (en) | Method, device and system for monitoring trouble of distributed home-network | |
| CN109799948B (en) | Data storage method and device | |
| CN104202201A (en) | Log processing method and device and terminal | |
| CN111124722B (en) | Method, equipment and medium for isolating fault memory | |
| US8086909B1 (en) | Automatic core file upload | |
| CN111104283B (en) | Fault detection method, device, equipment and medium of distributed storage system | |
| CN111338869A (en) | Configuration parameter management method, device, equipment and storage medium | |
| CN110879756B (en) | Collaborative process exception handling method, collaborative process exception handling device, computer equipment and storage medium | |
| CN107172113B (en) | Processing method and device in abnormal service call | |
| CN111639053A (en) | Authorization file expiration alarm prompting method and device and computer equipment | |
| CN111367740B (en) | BMC debugging system and method and computer readable storage medium | |
| US11429537B2 (en) | Method, device, and computer program product for managing storage system | |
| US11023307B2 (en) | Automatic remediation of distributed storage system node components through visualization | |
| CN110113187B (en) | Configuration updating method and device, configuration server and configuration system | |
| US10481980B2 (en) | Enabling segmented source data introspection within dispersed storage network (DSN) memory | |
| JP5670935B2 (en) | Distributed data management system and operation method thereof | |
| CN103220162B (en) | The fault-tolerant optimization method and device of SCSI based on HDFS | |
| CN110955647A (en) | Database assistance method, database assistance device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |