CN109241074A - Check data automatic processing method, device, computer equipment and storage medium - Google Patents
Check data automatic processing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109241074A CN109241074A CN201811137859.1A CN201811137859A CN109241074A CN 109241074 A CN109241074 A CN 109241074A CN 201811137859 A CN201811137859 A CN 201811137859A CN 109241074 A CN109241074 A CN 109241074A
- Authority
- CN
- China
- Prior art keywords
- data
- imported
- target table
- task node
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
This application discloses check data automatic processing method, device, computer equipment and storage medium.This method comprises: obtaining pre-set data guiding flow, data to be imported are sequentially imported by corresponding target table according to the sequence of task node in data guiding flow；Judge that data to be imported are sequentially imported whether corresponding target table succeeds according to task node；If data to be imported are imported corresponding target table success according to task node, the write state identification field for being used for mark data write-in target table state will be increased in the corresponding data to be imported of the task node, successful marker will be written and fill to write state identification field；If data to be imported successfully import target table according to data guiding flow, the upload end that successful notification information is sent to data to be imported is imported data to.This method realizes that being sent to the report for checking terminal requires automatic filling according to corresponding, improves the efficiency of data mart modeling by data processing technique.
Description
Technical field
This application involves data mart modeling technical fields more particularly to one kind to check data automatic processing method, device, calculating
Machine equipment and storage medium.
Background technique
Currently, business datum is processed, it is sent to audit mechanism after filling into specified core table of submitting to a higher level for approval or revision and carries out
Audit, generally by the way of semi-automatic filling, and business datum during filling when the error occurs, usually again
Start to be filled again, leads to the inefficiency of data mart modeling.
Apply for content
The embodiment of the present application provides one kind and checks data automatic processing method, device, computer equipment and storage medium,
It aims to solve the problem that and processes business datum in existing technical method, be sent to after filling into specified core table of submitting to a higher level for approval or revision careful
The problem of core mechanism carries out audit by the way of semi-automatic filling, leads to the inefficiency of data mart modeling.
In a first aspect, the embodiment of the present application, which provides one kind, checks data automatic processing method comprising:
Pre-set data guiding flow is obtained, according to the sequence of task node in data guiding flow by number to be imported
According to sequentially importing corresponding target table；
Judge that data to be imported are sequentially imported whether corresponding target table succeeds according to task node；
It is if data to be imported are imported corresponding target table success according to task node, the task node is corresponding
Increase a write state identification field for mark data write-in target table state in data to be imported, will be written successfully
Marker fill to write state identification field；
If data to be imported successfully import target table according to data guiding flow, successful notification information is imported data to
It is sent to the upload end of data to be imported.
Second aspect, the embodiment of the present application provide one kind and check data automatic machining device comprising:
Data import unit, for obtaining pre-set data guiding flow, according to task section in data guiding flow
Data to be imported are sequentially imported corresponding target table by the sequence of point；
Judging unit, for judge according to task node by data to be imported sequentially import corresponding target table whether at
Function；
Data import successfully control unit, if for data to be imported to be imported corresponding target table according to task node
Success is used for writing for mark data write-in target table state for increasing by one in the corresponding data to be imported of the task node
Enter status indicator field, successful marker will be written and fill to write state identification field；
Success notification unit is imported, it, will if successfully importing target table according to data guiding flow for data to be imported
Data import the upload end that successful notification information is sent to data to be imported.
The third aspect, the embodiment of the present application provide a kind of computer equipment again comprising memory, processor and storage
On the memory and the computer program that can run on the processor, the processor execute the computer program
Data automatic processing method is checked described in the above-mentioned first aspect of Shi Shixian.
Fourth aspect, the embodiment of the present application also provides a kind of storage mediums, wherein the storage medium is stored with calculating
Machine program, the computer program include program instruction, and described program instruction when being executed by a processor holds the processor
Data automatic processing method is checked described in the above-mentioned first aspect of row.
The embodiment of the present application provides one kind and checks data automatic processing method, device, computer equipment and storage medium.
This method, will be to be imported according to the sequence of task node in data guiding flow by obtaining pre-set data guiding flow
Data sequentially import corresponding target table；Judge that data to be imported are sequentially imported corresponding target table according to task node
Whether succeed；It is if data to be imported are imported corresponding target table success according to task node, the task node is corresponding
Data to be imported in increase by one for mark data write-in target table state write state identification field, will be written to
The marker of function is filled to write state identification field；If data to be imported successfully import object table according to data guiding flow
Lattice import data to the upload end that successful notification information is sent to data to be imported.The method achieve be sent to check
The report of terminal requires automatic filling according to corresponding, improves the efficiency of data mart modeling.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to needed in embodiment description
Attached drawing is briefly described, it should be apparent that, the accompanying drawings in the following description is some embodiments of the present application, general for this field
For logical technical staff, without creative efforts, it is also possible to obtain other drawings based on these drawings.
Fig. 1 is the flow diagram provided by the embodiments of the present application for checking data automatic processing method；
Fig. 2 is the sub-process schematic diagram provided by the embodiments of the present application for checking data automatic processing method；
Fig. 3 is another flow diagram provided by the embodiments of the present application for checking data automatic processing method；
Fig. 4 is another flow diagram provided by the embodiments of the present application for checking data automatic processing method；
Fig. 5 is another flow diagram provided by the embodiments of the present application for checking data automatic processing method；
Fig. 6 is the schematic block diagram provided by the embodiments of the present application for checking data automatic machining device；
Fig. 7 is the subelement schematic block diagram provided by the embodiments of the present application for checking data automatic machining device；
Fig. 8 is another schematic block diagram provided by the embodiments of the present application for checking data automatic machining device；
Fig. 9 is another schematic block diagram provided by the embodiments of the present application for checking data automatic machining device；
Figure 10 is another schematic block diagram provided by the embodiments of the present application for checking data automatic machining device；
Figure 11 is the schematic block diagram of computer equipment provided by the embodiments of the present application.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete
Site preparation description, it is clear that described embodiment is some embodiments of the present application, instead of all the embodiments.Based on this Shen
Please in embodiment, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall in the protection scope of this application.
It should be appreciated that ought use in this specification and in the appended claims, term " includes " and "comprising" instruction
Described feature, entirety, step, operation, the presence of element and/or component, but one or more of the other feature, whole is not precluded
Body, step, operation, the presence or addition of element, component and/or its set.
It is also understood that mesh of the term used in this present specification merely for the sake of description specific embodiment
And be not intended to limit the application.As present specification and it is used in the attached claims, unless on
Other situations are hereafter clearly indicated, otherwise " one " of singular, "one" and "the" are intended to include plural form.
It will be further appreciated that the term "and/or" used in present specification and the appended claims is
Refer to any combination and all possible combinations of one or more of associated item listed, and including these combinations.
Referring to Fig. 1, Fig. 1 is the schematic flow diagram provided by the embodiments of the present application for checking data automatic processing method.It should
Check that data automatic processing method is applied in management server, this method passes through the application software that is installed in management server
It is executed, more user terminals are by establishing network connection with management server to carry out information transmission.Wherein, user terminal
It is terminal device, such as desktop computer, laptop, tablet computer or mobile phone etc. with access internet function；Management
Server is the enterprise terminal processed automatically for checking data.
As shown in Figure 1, the method comprising the steps of S101~S104.
S101, pre-set data guiding flow is obtained, it will be to according to the sequence of task node in data guiding flow
It imports data and sequentially imports corresponding target table.
In the present embodiment, step S101 is executed in the load end of management server.It is i.e. first equal in each load end
It is pre-configured with data guiding flow, multiple data steps for importing, namely corresponding multiple tasks can be set in data guiding flow
Node is filled the data for the condition that meets automatically to object table by database script (such as SQL script) in each task node
Lattice (edit the condition that garbled data meets in the corresponding SQL script of general each task node, such as a certain field value is equal to
Numerical value A).It is realized by data guiding flow and source data (data i.e. to be imported) is automatically imported to target table, be not necessarily to
Artificial interference improves the efficiency of data mart modeling processing.
Such as including N number of task node (such as 10 task nodes) in data guiding flow, first task node is corresponding
Task is that target table 1 is written in the part specific field in data to be imported in table 1, and the corresponding task of the second task node is
By in data to be imported in table 2 part specific field be written target table 2 ... ..., the corresponding task of N task node be by
Target table N is written in part specific field in data to be imported in table N.It, can will be wait lead by above-mentioned data guiding flow
Enter data (namely data of audit to be committed) write-in target table, target table is committed to the server of audit mechanism later
(checking end) is audited.
In one embodiment, as shown in Fig. 2, step S101 includes:
S1011, the source client information for obtaining data to be imported, increase by one according to source client information in data to be imported
Source identification field, and source terminal information is filled to source identification field, the data after obtaining source identification；
S1012, the data behind source identification are divided according to task node, is obtained a pair of with each task node one
The block data answered increases a step number field in each block data, by the corresponding task node of each block data
Serial number fills the data after obtaining step mark to step number field；
S1013, step is identified after the data of data sequentially import corresponding target table.
In the present embodiment, load end obtains the source client information of data to be imported, not only carrys out data to be imported
The data source identification field in storage table that the write-in of source information has created increases a source mark also in data to be imported
Character learning section, and source terminal information is filled to source identification field.By by data to be imported increase source identification field, be for
It prevents from sequentially being imported data to be imported according to the sequence of task node in data guiding flow in data to be imported corresponding
It, directly can be according to source identification field signals data to be imported if occurring write error when target table need to re-write
It uploads end and uploads data again, or from the data backup to be imported of load end to reacquire data to be imported (i.e. to be imported
When data are obtained by a certain load end, the table of a storage data to be imported can establish, to save data to be imported).
The data behind source identification are divided according to task node later, obtain corresponding with each task node
Block data, be that corresponding data is written by corresponding table according to task node for the ease of load end.For example, behind source identification
Data have 10000 datas, include 10 task nodes in data guiding flow, the corresponding task of first task node is by 1-
1000 numbers are directed into target table 1, increase a step number field, and the step number word in 1-1000 number
Duan Zhongjun filling 1 (indicates first task node, execution sequence is first to execute sequence)；Subsequent 9000 data is also before referring to
1000 datas increase step number field and corresponding filling task node serial number.
S102, judge that data to be imported are sequentially imported whether corresponding target table succeeds according to task node；
In the present embodiment, judge according to task node by data to be imported sequentially import corresponding target table whether at
Function, judgment criteria are that data are automatically imported whether process interrupts, and are not interrupted always, indicate to import successfully, if by
Interruption then indicates to import failure.Judge whether the process being automatically imported interrupts, specifically judges the corresponding data pin of task node
Whether this number of data screened and the desired target data item number of the task node are identical, if the corresponding number of task node
The number of data screened according to script is identical as the desired target data item number of the task node, indicates leading automatically for data
Enter process not interrupt, if the number of data that the corresponding data script of task node is screened is less than desired by the task node
Target data item number then indicate that the process that is automatically imported of data is not interrupted.
If data to be imported S103, are imported corresponding target table success according to task node, by the task node
Increase a write state identification field for mark data write-in target table state in corresponding data to be imported, will write
Enter successful marker to fill to write state identification field；
In the present embodiment, if being all write-in successfully mark to increased write state identification field in data is imported
Position, then it represents that be entirely automatically imported process zero defect.For example, being by 1-1000 number by the corresponding task of first task node
It is corresponding in 1-1000 number if 1 success of target table is written in above-mentioned 1000 data when being directed into target table 1
The step of mark after increase by a write state identification field for mark data write-in target table state in data, will be written
Successful marker (1) fill to the write state identification field, by the way that this write state is written by such as success, or number
Marker is to quickly know which data is successfully written into target table in follow-up process.
If S104, data to be imported successfully import target table according to data guiding flow, import data to successfully logical
Know that information is sent to the upload end of data to be imported.
In the present embodiment, if data to be imported successfully import target table according to data guiding flow, expression is filled out automatically
Filling process has terminated, and need to timely notify the upload end of data to be imported.The upload end of data to be imported can choose suddenly at this time
The slightly notification information reaches preset uplink time to system time, and the data duplication of target table is imported and is checked in data
It reports in template form to upload automatically；Or the upload end of data to be imported can issue and immediately replicate the data of target table
Importing checks that data report in template form the instruction to upload automatically.
In one embodiment, as shown in figure 3, including: after step S104
S1041, it obtains and pre-stored checks that data report template form；
S1042, the data duplication in target table is imported and checks that data report template form, obtain checking tables of data
Lattice.
In the present embodiment, field included in sublist included by target table and each sublist with check
Data report template table identical.But it checks that data report template form compared with target table, some template properties may be had more
Inessential content (these contents are the call formats in order to meet audit report), therefore data to be imported will be stored up, mesh is first written
Mark this interim table of table stored, it is possible to reduce data automatically write during positioning target area time.Because
If data to be imported are write direct and check that data report template form, the region that need to position aiming field is being written, and
The interference of inessential content in position fixing process by template property is larger.
In one embodiment, as shown in figure 4, including: after step S1042
S1043, preset uplink time is pressed, will checks that data form is sent to and corresponding checks end.
In the present embodiment, preset uplink time is reached to system time, the data duplication of target table is imported and is checked
Nuclear Data reports in template form to upload automatically.In this way, the network that the may be set in management server non-congested time is (as often
Day 20:00- next day 6:00), will check data form be sent to it is corresponding check end, avoid uploading mistake caused by because of network congestion
It loses or problem that uploading rate is low.
In one embodiment, as shown in figure 5, including: after step S102
S105, data to be imported are imported by corresponding target table according to task node if it exists fail, by the task
Increase by a write state identification field for mark data write-in target table state in the corresponding data to be imported of node, it will
The corresponding successful marker of write-in of the successful data of target table is written in data to be imported to fill to said write state mark
The corresponding marker that failure is written of the data that target table failure is written in data to be imported is filled to described and is write by character learning section
Enter status indicator field；
S106, the abnormal position in data write-in target table to be imported is positioned according to the marker of write-in failure, if inspection
The corresponding task node troubleshooting success of abnormal position is measured, data to be imported corresponding to the corresponding task node of abnormal position are write
Enter target table.
For example, the corresponding task of first task node is when 1-1000 number is directed into target table 1, if above-mentioned
1 success of target table is written in preceding 500 data in 1000 datas, then the data after step mark corresponding to 1-500 number
It is middle to increase by a write state identification field for mark data write-in target table state, successful marker will be written (such as
1) success, or number are filled to the write state identification field；If fail from the 501st number write-in target table 1,
It will increase by a write state for being used for mark data write-in target table state in data after step mark corresponding to 501 numbers
Identification field, by the marker of write-in failure, (0) such as fail, or number are filled to the write state identification field.
Obviously, it can also increase by a sequence number field in data after step mark, to indicate that each data are all in the step
It is endowed a serial number, this serial number is endowed each data in the step by ascending order.In this way, once in data writing process
There are when failure, which number goes wrong in writing process when can quickly know, to realize quick positioning failure.
Wherein, if once write state identification field detect write-in failure marker, then it represents that the step for pair
The task node answered is to need the script that automatically writes corresponding to the task node to modify to debug there are failure,
And after troubleshooting success, data in the corresponding target table of the task node are first emptied, then again by the task node pair
All data answered are written to target table.
This method is realized by pre-set data guiding flow and is sent to the report for checking terminal according to correspondence
It is required that automatic filling, improves the efficiency of data mart modeling.
The embodiment of the present application also provides one kind and checks data automatic machining device, this checks that data automatic machining device is used for
Execute the aforementioned any embodiment for checking data automatic processing method.Specifically, referring to Fig. 6, Fig. 6 is that the embodiment of the present application mentions
The schematic block diagram for checking data automatic machining device supplied.Check that data automatic machining device 100 can be configured at management clothes
It is engaged in device.
As shown in fig. 6, checking that data automatic machining device 100 includes data import unit 101, judging unit 102, data
Import successfully control unit 103, importing success notification unit 104.
Data import unit 101, for obtaining pre-set data guiding flow, according to task in data guiding flow
Data to be imported are sequentially imported corresponding target table by the sequence of node.
In the present embodiment, step S101 is executed in the load end of management server.It is i.e. first equal in each load end
It is pre-configured with data guiding flow, multiple data steps for importing, namely corresponding multiple tasks can be set in data guiding flow
Node is filled the data for the condition that meets automatically to object table by database script (such as SQL script) in each task node
Lattice.It is realized by data guiding flow and source data (data i.e. to be imported) is automatically imported to target table, without artificial
Interference improves the efficiency of data mart modeling processing.
Such as including N number of task node (such as 10 task nodes) in data guiding flow, first task node is corresponding
Task is that target table 1 is written in the part specific field in data to be imported in table 1, and the corresponding task of the second task node is
Target table 2 ... ... is written into part specific field in data to be imported in table 2, is by the corresponding task of N task node
Target table N is written into part specific field in data to be imported in table N.It, can will be to by above-mentioned data guiding flow
It imports data (namely data of audit to be committed) and target table is written, target table is committed to the service of audit mechanism later
Device (checking end) is audited.
In one embodiment, as shown in fig. 7, data import unit 101 includes:
Source identification unit 1011, for obtaining the source client information of data to be imported, according to source client information wait lead
Enter in data one source identification field of increase, and source terminal information is filled to source identification field, after obtaining source identification
Data；
Step identifies unit 1012, for the data behind source identification to be divided according to task node, obtain with often
The one-to-one block data of one task node increases a step number field in each block data, by each block count
The data after obtaining step mark to step number field are filled according to corresponding task node serial number；
Sequentially writing unit 1013, the data for the data after identifying step sequentially import corresponding target table.
In the present embodiment, load end obtains the source client information of data to be imported, not only carrys out data to be imported
The data source identification field in storage table that the write-in of source information has created increases a source mark also in data to be imported
Character learning section, and source terminal information is filled to source identification field.By by data to be imported increase source identification field, be for
It prevents from sequentially being imported data to be imported according to the sequence of task node in data guiding flow in data to be imported corresponding
It, directly can be according to source identification field signals data to be imported if occurring write error when target table need to re-write
It uploads end and uploads data again, or from the data backup to be imported of load end to reacquire data to be imported (i.e. to be imported
When data are obtained by a certain load end, the table of a storage data to be imported can establish, to save data to be imported).
The data behind source identification are divided according to task node later, obtain corresponding with each task node
Block data, be that corresponding data is written by corresponding table according to task node for the ease of load end.For example, behind source identification
Data have 10000 datas, include 10 task nodes in data guiding flow, the corresponding task of first task node is by 1-
1000 numbers are directed into target table 1, increase a step number field, and the step number word in 1-1000 number
Duan Zhongjun filling 1 (indicates first task node, execution sequence is first to execute sequence)；Subsequent 9000 data is also before referring to
1000 datas increase step number field and corresponding filling task node serial number.
Judging unit 102, for judging that data to be imported, which are sequentially imported corresponding target table, according to task node is
No success.
In the present embodiment, judge according to task node by data to be imported sequentially import corresponding target table whether at
Function, judgment criteria are that data are automatically imported whether process interrupts, and are not interrupted always, indicate to import successfully, if by
Interruption then indicates to import failure.Judge whether the process being automatically imported interrupts, specifically judges the corresponding data pin of task node
Whether this number of data screened and the desired target data item number of the task node are identical, if the corresponding number of task node
The number of data screened according to script is identical as the desired target data item number of the task node, indicates leading automatically for data
Enter process not interrupt, if the number of data that the corresponding data script of task node is screened is less than desired by the task node
Target data item number then indicate that the process that is automatically imported of data is not interrupted.
Data import successfully control unit 103, if for data to be imported to be imported corresponding target according to task node
Table success will increase by one and be used for mark data write-in target table state in the corresponding data to be imported of the task node
Write state identification field, successful marker will be written and fill to write state identification field.
In the present embodiment, if being all write-in successfully mark to increased write state identification field in data is imported
Position, then it represents that be entirely automatically imported process zero defect.For example, being by 1-1000 number by the corresponding task of first task node
It is corresponding in 1-1000 number if 1 success of target table is written in above-mentioned 1000 data when being directed into target table 1
The step of mark after increase by a write state identification field for mark data write-in target table state in data, will be written
Successful marker (1) fill to the write state identification field, by the way that this write state is written by such as success, or number
Marker is to quickly know which data is successfully written into target table in follow-up process.
Success notification unit 104 is imported, if successfully importing target table according to data guiding flow for data to be imported,
Import data to the upload end that successful notification information is sent to data to be imported.
In the present embodiment, if data to be imported successfully import target table according to data guiding flow, expression is filled out automatically
Filling process has terminated, and need to timely notify the upload end of data to be imported.The upload end of data to be imported can choose suddenly at this time
The slightly notification information reaches preset uplink time to system time, and the data duplication of target table is imported and is checked in data
It reports in template form to upload automatically；Or the upload end of data to be imported can issue and immediately replicate the data of target table
Importing checks that data report in template form the instruction to upload automatically.
In one embodiment, as shown in figure 8, checking data automatic machining device 100 further include:
It checks form template acquiring unit 1041, pre-stored checks that data report template form for obtaining；
It checks table fills unit 1042, checks that data report template table for importing the data duplication in target table
Lattice obtain checking data form.
In the present embodiment, field included in sublist included by target table and each sublist with check
Data report template table identical.But it checks that data report template form compared with target table, some template properties may be had more
Inessential content (these contents are the call formats in order to meet audit report), therefore data to be imported will be stored up, mesh is first written
Mark this interim table of table stored, it is possible to reduce data automatically write during positioning target area time.Because
If data to be imported are write direct and check that data report template form, the region that need to position aiming field is being written, and
The interference of inessential content in position fixing process by template property is larger.
In one embodiment, as shown in figure 9, checking data automatic machining device 100 further include:
Timing uploading unit 1043 will check that data form is sent to corresponding check for pressing preset uplink time
End.
In the present embodiment, preset uplink time is reached to system time, the data duplication of target table is imported and is checked
Nuclear Data reports in template form to upload automatically.In this way, the network that the may be set in management server non-congested time is (as often
Day 20:00- next day 6:00), will check data form be sent to it is corresponding check end, avoid uploading mistake caused by because of network congestion
It loses or problem that uploading rate is low.
In one embodiment, as shown in Figure 10, data automatic machining device 100 is checked further include:
Data import failure control unit 105, for according to task node that data to be imported importing is corresponding if it exists
Target table failure will increase by one and be used for mark data write-in object table trellis in the corresponding data to be imported of the task node
The corresponding write-in of the successful data of target table successfully mark will be written in the write state identification field of state in data to be imported
Position filling loses the corresponding write-in of data that target table failure is written in data to be imported to said write status indicator field
The marker lost is filled to said write status indicator field；
Exception processing unit 106, for being positioned in data write-in target table to be imported according to the marker of write-in failure
Abnormal position, if abnormal position corresponding task node troubleshooting success is detected, by the corresponding task node institute of abnormal position
Target table is written in corresponding data to be imported.
For example, the corresponding task of first task node is when 1-1000 number is directed into target table 1, if above-mentioned
1 success of target table is written in preceding 500 data in 1000 datas, then the data after step mark corresponding to 1-500 number
It is middle to increase by a write state identification field for mark data write-in target table state, successful marker will be written (such as
1) success, or number are filled to the write state identification field；If fail from the 501st number write-in target table 1,
It will increase by a write state for being used for mark data write-in target table state in data after step mark corresponding to 501 numbers
Identification field, by the marker of write-in failure, (0) such as fail, or number are filled to the write state identification field.
Obviously, it can also increase by a sequence number field in data after step mark, to indicate that each data are all in the step
It is endowed a serial number, this serial number is endowed each data in the step by ascending order.In this way, once in data writing process
There are when failure, which number goes wrong in writing process when can quickly know, to realize quick positioning failure.
Wherein, if once write state identification field detect write-in failure marker, then it represents that the step for pair
The task node answered is to need the script that automatically writes corresponding to the task node to modify to debug there are failure,
And after troubleshooting success, data in the corresponding target table of the task node are first emptied, then again by the task node pair
All data answered are written to target table.
The present apparatus is realized by pre-set data guiding flow and is sent to the report for checking terminal according to correspondence
It is required that automatic filling, improves the efficiency of data mart modeling.
The above-mentioned form checked data automatic machining device and can be implemented as computer program, the computer program can be
It is run in computer equipment as shown in figure 11.
Figure 11 is please referred to, Figure 11 is the schematic block diagram of computer equipment provided by the embodiments of the present application.
Refering to fig. 11, which includes processor 502, memory and the net connected by system bus 501
Network interface 505, wherein memory may include non-volatile memory medium 503 and built-in storage 504.
The non-volatile memory medium 503 can storage program area 5031 and computer program 5032.The computer program
5032 include program instruction, which is performed, and the execution of processor 502 may make to check data automatic processing method.
The processor 502 supports the operation of entire computer equipment 500 for providing calculating and control ability.
The built-in storage 504 provides environment for the operation of the computer program 5032 in non-volatile memory medium 503, should
When computer program 5032 is executed by processor 502, the execution of processor 502 may make to check data automatic processing method.
The network interface 505 provides the transmission etc. of data traffic for carrying out network communication, for example network access.This field
Technical staff is appreciated that structure shown in Figure 11, only the block diagram of part-structure relevant to application scheme, not
The restriction for the computer equipment 500 being applied thereon to application scheme is constituted, specific computer equipment 500 may include
Than more or fewer components as shown in the figure, certain components are perhaps combined or with different component layouts.
Wherein, the processor 502 is for running computer program 5032 stored in memory, to realize following function
Can: obtain pre-set data guiding flow, according to the sequence of task node in data guiding flow by data to be imported according to
Sequence imports corresponding target table；Judge according to task node by data to be imported sequentially import corresponding target table whether at
Function；It is if data to be imported are imported corresponding target table success according to task node, the task node is corresponding wait lead
Enter to increase in data a write state identification field for mark data write-in target table state, by the successful mark of write-in
Know position to fill to write state identification field；If data to be imported successfully import target table according to data guiding flow, will count
The upload end for being sent to data to be imported according to successful notification information is imported.
In one embodiment, processor 502 execute the sequence according to task node in data guiding flow will be to
When importing data sequentially import corresponding target table, performs the following operations: obtaining the source client information of data to be imported, according to
Source client information increases a source identification field in data to be imported, and source terminal information is filled to source identification word
Section, the data after obtaining source identification；The data behind source identification are divided according to task node, are obtained and each task
The one-to-one block data of node increases a step number field in each block data, and each block data is corresponding
Task node serial number fill obtained to step number field step mark after data；The number of data after step is identified
According to sequentially importing corresponding target table.
In one embodiment, processor 502 import data to described in the execution successful notification information be sent to it is to be imported
After the upload end of data, also perform the following operations: acquisition is pre-stored to check that data report template form；By target table
In data duplication import check that data report template form, obtain checking data form.
In one embodiment, processor 502 is imported in the execution data duplication by target table and is checked in data
Template form is reported, obtains after checking data form, also performs the following operations: by preset uplink time, will check tables of data
Lattice, which are sent to, corresponding checks end.
In one embodiment, processor 502 in execution judges that data to be imported are sequentially imported correspondence according to task node
Target table it is whether successful after, also perform the following operations: data to be imported being imported according to task node correspond to if it exists
Target table failure, will in the corresponding data to be imported of the task node increase by one be used for mark data write-in target table
The corresponding write-in of the successful data of target table successfully mark will be written in the write state identification field of state in data to be imported
Know position to fill to said write status indicator field, the corresponding write-in of data of target table failure will be written in data to be imported
The marker of failure is filled to said write status indicator field；Data write-in to be imported is positioned according to the marker of write-in failure
Abnormal position in target table, it is if detecting the corresponding task node troubleshooting success of abnormal position, abnormal position is corresponding
Data write-in target table to be imported corresponding to task node.
It will be understood by those skilled in the art that the embodiment of computer equipment shown in Figure 11 is not constituted to computer
The restriction of equipment specific composition, in other embodiments, computer equipment may include components more more or fewer than diagram, or
Person combines certain components or different component layouts.For example, in some embodiments, computer equipment can only include depositing
Reservoir and processor, in such embodiments, the structure and function of memory and processor are consistent with embodiment illustrated in fig. 11,
Details are not described herein.
It should be appreciated that in the embodiment of the present application, processor 502 can be central processing unit
(CentralProcessing Unit, CPU), which can also be other general processors, digital signal processor
(Digital Signal Processor, DSP), specific integrated circuit (Application Specific
IntegratedCircuit, ASIC), ready-made programmable gate array (Field-Programmable GateArray, FPGA) or
Person other programmable logic device, discrete gate or transistor logic, discrete hardware components etc..Wherein, general processor
It can be microprocessor or the processor be also possible to any conventional processor etc..
Storage medium is provided in another embodiment of the application.The storage medium can be that non-volatile computer can
Read storage medium.The storage medium is stored with computer program, and wherein computer program includes program instruction.The program instruction quilt
Processor performs the steps of when executing obtains pre-set data guiding flow, according to task section in data guiding flow
Data to be imported are sequentially imported corresponding target table by the sequence of point；Judge sequentially to be led data to be imported according to task node
Enter whether corresponding target table succeeds；It, will if data to be imported are imported corresponding target table success according to task node
Increase a write state for mark data write-in target table state in the corresponding data to be imported of the task node
Identification field will be written successful marker and fill to write state identification field；It is flowed if data to be imported are imported according to data
Cheng Chenggong imports target table, imports data to the upload end that successful notification information is sent to data to be imported.
In one embodiment, it is performed the steps of when which is executed by processor and obtains coming for data to be imported
Source information increases a source identification field according to source client information in data to be imported, and source terminal information is filled
Data to source identification field, after obtaining source identification；The data behind source identification are divided according to task node, are obtained
To with the one-to-one block data of each task node, each block data increase a step number field, will be each
The corresponding task node serial number of block data fills the data after obtaining step mark to step number field；Step is identified
The data of data afterwards sequentially import corresponding target table.
In one embodiment, it is performed the steps of when which is executed by processor and obtains pre-stored check
Data report template form；Data duplication in target table is imported and checks that data report template form, obtains checking data
Table.
In one embodiment, it is performed the steps of when which is executed by processor by preset uplink time, it will
It checks that data form is sent to and corresponding checks end.
In one embodiment, it is performed the steps of when which is executed by processor if it exists according to task node
Data to be imported are imported into corresponding target table failure, are used for increasing by one in the corresponding data to be imported of the task node
The write state identification field of target table state is written in mark data, successfully counts target table is written in data to be imported
It fills according to the corresponding successful marker of write-in to said write status indicator field, target table will be written in data to be imported
The marker of the corresponding write-in failure of the data of failure is filled to said write status indicator field；According to the mark of write-in failure
Position positions the abnormal position in data to be imported write-in target table, if detect the corresponding task node troubleshooting of abnormal position at
Target table is written in data to be imported corresponding to the corresponding task node of abnormal position by function.
It is apparent to those skilled in the art that for convenience of description and succinctly, foregoing description is set
The specific work process of standby, device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
Those of ordinary skill in the art may be aware that unit described in conjunction with the examples disclosed in the embodiments of the present disclosure and algorithm
Step can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and software
Interchangeability generally describes each exemplary composition and step according to function in the above description.These functions are studied carefully
Unexpectedly the specific application and design constraint depending on technical solution are implemented in hardware or software.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
Scope of the present application.
In several embodiments provided herein, it should be understood that disclosed unit and method, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only logical function partition, there may be another division manner in actual implementation, can also will be with the same function
Unit set is at a unit, such as multiple units or components can be combined or can be integrated into another system or some
Feature can be ignored, or not execute.In addition, shown or discussed mutual coupling, direct-coupling or communication connection can
Be through some interfaces, the indirect coupling or communication connection of device or unit, be also possible to electricity, mechanical or other shapes
Formula connection.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.Some or all of unit therein can be selected to realize the embodiment of the present application scheme according to the actual needs
Purpose.
It, can also be in addition, each functional unit in each embodiment of the application can integrate in one processing unit
It is that each unit physically exists alone, is also possible to two or more units and is integrated in one unit.It is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in one storage medium.Based on this understanding, the technical solution of the application is substantially in other words to existing
The all or part of part or the technical solution that technology contributes can be embodied in the form of software products, should
Computer software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be
Personal computer, server or network equipment etc.) execute all or part of step of each embodiment the method for the application
Suddenly.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), magnetic disk or
The various media that can store program code such as person's CD.
The above, the only specific embodiment of the application, but the protection scope of the application is not limited thereto, it is any
Those familiar with the art within the technical scope of the present application, can readily occur in various equivalent modifications or replace
It changes, these modifications or substitutions should all cover within the scope of protection of this application.Therefore, the protection scope of the application should be with right
It is required that protection scope subject to.
Claims (10)
1. one kind checks data automatic processing method characterized by comprising
Obtain pre-set data guiding flow, according to the sequence of task node in data guiding flow by data to be imported according to
Sequence imports corresponding target table；
Judge that data to be imported are sequentially imported whether corresponding target table succeeds according to task node；
It is if data to be imported are imported corresponding target table success according to task node, the task node is corresponding wait lead
Enter to increase in data a write state identification field for mark data write-in target table state, by the successful mark of write-in
Know position to fill to write state identification field；
If data to be imported successfully import target table according to data guiding flow, imports data to successful notification information and send
To the upload end of data to be imported.
2. according to claim 1 check data automatic processing method, which is characterized in that described according to data guiding flow
Data to be imported are sequentially imported corresponding target table by the sequence of middle task node, comprising:
The source client information for obtaining data to be imported increases a source identification word according to source client information in data to be imported
Section, and source terminal information is filled to source identification field, the data after obtaining source identification；
The data behind source identification are divided according to task node, are obtained and the one-to-one block count of each task node
According to, each block data increase a step number field, by the corresponding task node serial number of each block data fill to
Step number field, the data after obtaining step mark；
The data of data after step is identified sequentially import corresponding target table.
3. according to claim 1 check data automatic processing method, which is characterized in that described to import data to successfully
Notification information is sent to after the upload end of data to be imported, further includes:
It obtains and pre-stored checks that data report template form；
Data duplication in target table is imported and checks that data report template form, obtains checking data form.
4. according to claim 3 check data automatic processing method, which is characterized in that the number by target table
It is imported according to duplication and checks that data report template form, after obtaining checking data form, further includes:
By preset uplink time, it will check that data form is sent to and corresponding check end.
5. according to claim 1 check data automatic processing method, which is characterized in that the judgement is according to task node
By data to be imported sequentially import corresponding target table it is whether successful after, further includes:
Data to be imported corresponding target table is imported according to task node if it exists to fail, the task node is corresponding
Increase by a write state identification field for mark data write-in target table state in data to be imported, by data to be imported
The successful marker of the corresponding write-in of the successful data of middle write-in target table is filled to said write status indicator field, will be to
The marker for importing the corresponding write-in failure of data of write-in target table failure in data is filled to said write status indicator
Field；
The abnormal position in data write-in target table to be imported is positioned according to the marker of write-in failure, if detecting exception bits
Corresponding task node troubleshooting success is set, object table is written into data to be imported corresponding to the corresponding task node of abnormal position
Lattice.
6. one kind checks data automatic machining device characterized by comprising
Data import unit, for obtaining pre-set data guiding flow, according to task node in data guiding flow
Data to be imported are sequentially imported corresponding target table by sequence；
Data to be imported are sequentially imported whether corresponding target table succeeds according to task node for judging by judging unit；
Data import successfully control unit, if for according to task node by data to be imported import corresponding target table at
Function will increase the write-in for being used for mark data write-in target table state in the corresponding data to be imported of the task node
Status indicator field will be written successful marker and fill to write state identification field；
Success notification unit is imported, if target table is successfully imported according to data guiding flow for data to be imported, by data
Import the upload end that successful notification information is sent to data to be imported.
7. according to claim 6 check data automatic machining device, which is characterized in that the data import unit, packet
It includes:
Source identification unit, for obtaining the source client information of data to be imported, according to source client information in data to be imported
Increase a source identification field, and source terminal information is filled to source identification field, the data after obtaining source identification；
Step mark unit obtains and each task section for being divided the data behind source identification according to task node
The one-to-one block data of point increases a step number field in each block data, and each block data is corresponding
Task node serial number fills the data after obtaining step mark to step number field；
Sequentially writing unit, the data for the data after identifying step sequentially import corresponding target table.
8. according to claim 6 check data automatic machining device, which is characterized in that further include:
It checks form template acquiring unit, pre-stored checks that data report template form for obtaining；
It checks table fills unit, checks that data report template form for importing the data duplication in target table, obtain
Check data form.
9. a kind of computer equipment, including memory, processor and it is stored on the memory and can be on the processor
The computer program of operation, which is characterized in that the processor realizes such as claim 1 to 5 when executing the computer program
Any one of described in check data automatic processing method.
10. a kind of storage medium, which is characterized in that the storage medium is stored with computer program, the computer program packet
Program instruction is included, described program instruction executes the processor such as any one of claim 1 to 5 institute
That states checks data automatic processing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811137859.1A CN109241074B (en) | 2018-09-28 | 2018-09-28 | Automatic auditing data processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811137859.1A CN109241074B (en) | 2018-09-28 | 2018-09-28 | Automatic auditing data processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109241074A true CN109241074A (en) | 2019-01-18 |
| CN109241074B CN109241074B (en) | 2023-09-26 |
Family
ID=65057649
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811137859.1A Active CN109241074B (en) | 2018-09-28 | 2018-09-28 | Automatic auditing data processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109241074B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110728128A (en) * | 2019-08-30 | 2020-01-24 | 平安证券股份有限公司 | Data report generation method, device, medium and electronic equipment |
| CN112749533A (en) * | 2019-10-30 | 2021-05-04 | 腾讯科技（深圳）有限公司 | Data input control method and device, electronic equipment and storage medium |
| CN116319718A (en) * | 2023-03-10 | 2023-06-23 | 北京振中电子技术有限公司 | Cloud data storage processing method, system, equipment and medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4658351A (en) * | 1984-10-09 | 1987-04-14 | Wang Laboratories, Inc. | Task control means for a multi-tasking data processing system |
| AU2645501A (en) * | 2000-03-15 | 2001-09-20 | Ilaunch Pty Ltd | Data importer |
| US20050240562A1 (en) * | 2004-04-26 | 2005-10-27 | Joerg Steinmann | Method, computer program product and device for importing a plurality of data sets into a system |
| CN103995908A (en) * | 2014-06-17 | 2014-08-20 | 山东中创软件工程股份有限公司 | Method and device for importing data |
| CN105069149A (en) * | 2015-08-24 | 2015-11-18 | 电子科技大学 | Structured line data-oriented distributed parallel data importing method |
| CN105095240A (en) * | 2014-05-04 | 2015-11-25 | 中国银联股份有限公司 | Database data sample acquisition |
| CN106126374A (en) * | 2016-06-22 | 2016-11-16 | 腾讯科技（深圳）有限公司 | Method for writing data, method for reading data and device |
| CN106776962A (en) * | 2016-12-05 | 2017-05-31 | 济南浪潮高新科技投资发展有限公司 | A kind of general Excel data import multiple database physical table methods |
| CN107025289A (en) * | 2017-04-14 | 2017-08-08 | 腾讯科技（深圳）有限公司 | The method and relevant device of a kind of data processing |
| US20180011886A1 (en) * | 2016-07-06 | 2018-01-11 | Hewlett Packard Enterprise Development Lp | Parallel transfers of electronic data |
| CN107864209A (en) * | 2017-11-17 | 2018-03-30 | 北京联想超融合科技有限公司 | The method, apparatus and server of data write-in |
| CN108171468A (en) * | 2017-12-15 | 2018-06-15 | 东软集团股份有限公司 | The method, apparatus and storage medium and electronic equipment of data processing |
| CN108228730A (en) * | 2017-12-11 | 2018-06-29 | 深圳市买买提信息科技有限公司 | Data lead-in method, device, computer equipment and readable storage medium storing program for executing |
| CN108460068A (en) * | 2017-11-28 | 2018-08-28 | 平安科技（深圳）有限公司 | Method, apparatus, storage medium and the terminal that report imports and exports |
-
2018
- 2018-09-28 CN CN201811137859.1A patent/CN109241074B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4658351A (en) * | 1984-10-09 | 1987-04-14 | Wang Laboratories, Inc. | Task control means for a multi-tasking data processing system |
| AU2645501A (en) * | 2000-03-15 | 2001-09-20 | Ilaunch Pty Ltd | Data importer |
| US20050240562A1 (en) * | 2004-04-26 | 2005-10-27 | Joerg Steinmann | Method, computer program product and device for importing a plurality of data sets into a system |
| CN105095240A (en) * | 2014-05-04 | 2015-11-25 | 中国银联股份有限公司 | Database data sample acquisition |
| CN103995908A (en) * | 2014-06-17 | 2014-08-20 | 山东中创软件工程股份有限公司 | Method and device for importing data |
| CN105069149A (en) * | 2015-08-24 | 2015-11-18 | 电子科技大学 | Structured line data-oriented distributed parallel data importing method |
| CN106126374A (en) * | 2016-06-22 | 2016-11-16 | 腾讯科技（深圳）有限公司 | Method for writing data, method for reading data and device |
| US20180011886A1 (en) * | 2016-07-06 | 2018-01-11 | Hewlett Packard Enterprise Development Lp | Parallel transfers of electronic data |
| CN106776962A (en) * | 2016-12-05 | 2017-05-31 | 济南浪潮高新科技投资发展有限公司 | A kind of general Excel data import multiple database physical table methods |
| CN107025289A (en) * | 2017-04-14 | 2017-08-08 | 腾讯科技（深圳）有限公司 | The method and relevant device of a kind of data processing |
| CN107864209A (en) * | 2017-11-17 | 2018-03-30 | 北京联想超融合科技有限公司 | The method, apparatus and server of data write-in |
| CN108460068A (en) * | 2017-11-28 | 2018-08-28 | 平安科技（深圳）有限公司 | Method, apparatus, storage medium and the terminal that report imports and exports |
| CN108228730A (en) * | 2017-12-11 | 2018-06-29 | 深圳市买买提信息科技有限公司 | Data lead-in method, device, computer equipment and readable storage medium storing program for executing |
| CN108171468A (en) * | 2017-12-15 | 2018-06-15 | 东软集团股份有限公司 | The method, apparatus and storage medium and electronic equipment of data processing |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110728128A (en) * | 2019-08-30 | 2020-01-24 | 平安证券股份有限公司 | Data report generation method, device, medium and electronic equipment |
| CN112749533A (en) * | 2019-10-30 | 2021-05-04 | 腾讯科技（深圳）有限公司 | Data input control method and device, electronic equipment and storage medium |
| CN112749533B (en) * | 2019-10-30 | 2023-08-04 | 腾讯科技（深圳）有限公司 | Data input control method and device, electronic equipment and storage medium |
| CN116319718A (en) * | 2023-03-10 | 2023-06-23 | 北京振中电子技术有限公司 | Cloud data storage processing method, system, equipment and medium |
| CN116319718B (en) * | 2023-03-10 | 2023-12-12 | 北京振中电子技术有限公司 | Cloud data storage processing method, system, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109241074B (en) | 2023-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100412802C (en) | Planned computer problem diagnosis and solvement and its automatic report and update | |
| CN109241074A (en) | Check data automatic processing method, device, computer equipment and storage medium | |
| CN107145437B (en) | Java annotation test method and device | |
| CN108762966A (en) | System exception hold-up interception method, device, computer equipment and storage medium | |
| CN109189640A (en) | Monitoring method, device, computer equipment and the storage medium of server | |
| CN104021081B (en) | A kind of adjustment method for browser and device | |
| CN109656767A (en) | A kind of acquisition methods, system and the associated component of CPLD status information | |
| CN108470004A (en) | Test script generation method, test method, device, equipment and storage medium | |
| CN110175165A (en) | Transaction account checking method, device, computer equipment and storage medium | |
| CN103745010A (en) | Method and device for determining object attribute value based on CSV (Comma Separated Values) file | |
| CN110147378A (en) | Verification of data method, apparatus, computer equipment and storage medium | |
| CN109753490A (en) | Database optimizing method, system, equipment and medium based on loophole reparation | |
| CN107180040A (en) | The management method and device of a kind of business datum | |
| CN107704365B (en) | Monitoring data reporting method and device for mobile terminal hybrid application | |
| CN106358220A (en) | Detection method of abnormal contact person information, device and system | |
| WO2011058581A2 (en) | An improved performance testing tool for financial applications | |
| CN109117302A (en) | A kind of internal memory data acquiring method, system, Memory Management Middleware and medium | |
| CN116680055A (en) | Asynchronous task processing method and device, computer equipment and storage medium | |
| CN110515842A (en) | A kind of program debugging method, device and electronic equipment and storage medium | |
| CN109885483A (en) | Log method for visualizing and system for operating status in Real-time Feedback game | |
| CN108920338A (en) | A kind of method and terminal indicating file system state | |
| CN109710187A (en) | Read command accelerated method, device, computer equipment and the storage medium of NVMe SSD main control chip | |
| CN111427328B (en) | Method for reducing household system faults | |
| CN109324887A (en) | Check data parallel processing method and system | |
| CN110019321A (en) | Problem delimits method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |