CN110457327A - A kind of data processing method, device and storage medium - Google Patents
A kind of data processing method, device and storage medium Download PDFInfo
- Publication number
- CN110457327A CN110457327A CN201910754056.9A CN201910754056A CN110457327A CN 110457327 A CN110457327 A CN 110457327A CN 201910754056 A CN201910754056 A CN 201910754056A CN 110457327 A CN110457327 A CN 110457327A
- Authority
- CN
- China
- Prior art keywords
- data
- manufacturing side
- service data
- target service
- tables
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003860 storage Methods 0.000 title claims abstract description 22
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 238000004519 manufacturing process Methods 0.000 claims abstract description 171
- 230000001360 synchronised effect Effects 0.000 claims abstract description 66
- 238000000034 method Methods 0.000 claims abstract description 48
- 238000012545 processing Methods 0.000 claims abstract description 47
- 238000012216 screening Methods 0.000 claims description 52
- 238000001914 filtration Methods 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 7
- 230000008859 change Effects 0.000 description 12
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 230000006872 improvement Effects 0.000 description 7
- 230000002093 peripheral effect Effects 0.000 description 6
- 238000012217 deletion Methods 0.000 description 5
- 230000037430 deletion Effects 0.000 description 5
- 238000004140 cleaning Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000000977 initiatory effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000032258 transport Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000010926 purge Methods 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/02—Banking, e.g. interest calculation or account maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
Abstract
This specification embodiment provides a kind of data processing method, device and storage medium.The method is applied to mirror image end, and the mirror image has supported the first tables of data, which comprises receives the synchronous service data that the manufacturing side is sent；The synchronous service data are stored to first tables of data；The target service data for meeting and needing are filtered out from first tables of data according to task requests；The target service data are transmitted to the manufacturing side, so that the manufacturing side handles the target service data in the second tables of data.The method that this specification embodiment provides can reduce the manufacturing side in data processing to the occupancy of resource, improve data-handling efficiency.
Description
Technical field
This specification embodiment is related to field of computer technology, in particular to a kind of data processing method, device and storage
Medium.
Background technique
Core banking system is exactly the transaction processing system towards all kinds of business of bank, by trading processing, drives accounting
It calculates and payment is cleared, be finally reached the target of integrated approach Batch Processing.Core banking system is based primarily upon following characteristics:
First is that being allowed for the serial transaction event after contract signing with regard to the right occurred inside bank and client, bank or between client
The information such as behavior and profit and loss relationship are crossed, registered, read or whole maintenance；Second is that the transaction completed, which will become, arbitrarily to be changed
Dynamic information, transaction maloperation can only be solved with positive way is rushed；Third is that trading situation can be inquired by transaction journal, hand over
Easy flowing water is that the most thin business of particle collects object, can carry out statistics remittance to it from angle any one or more based on this
Always.The system is customer-centric, carries out accounting processing, provides the integrated service of 7 × 24 hours core banks.
In the prior art, core banking system data handling procedure in, it usually needs filtered out from mass data
Meet the data for the treatment of conditions to be increased to data, be deleted, change operation, these data need to expend before treatment vast resources with
And the logic judgment of time is screened, so that vast resources is occupied, so that the efficiency of data processing reduces.Such as at present
In the data purge of core banking system operation, the operation traversed to most tables of data is needed, it is every to judge
Whether in vain data, to delete from system, above data manipulation, needs to handle the other data of tera-scale, right
The resources such as CPU, the memory of host are greatly to expend.
Summary of the invention
The purpose of this specification embodiment is to provide a kind of data processing method, device and storage medium, to reduce production
To the occupancy of resource, improve data-handling efficiency in data processing in end.
To solve the above problems, it is this that this specification embodiment, which provides a kind of data processing method, device and storage medium,
What sample was realized.
A kind of data processing method, is applied to mirror image end, and the mirror image has supported the first tables of data；The described method includes:
Receive the synchronous service data that the manufacturing side is sent；The synchronous service data are stored to first tables of data；According to task
Request filters out the target service data for meeting and needing from first tables of data；The target service data are transmitted to life
Is produced from end, so that the manufacturing side handles the target service data in the second tables of data.
A kind of data processing equipment is applied to mirror image end, and the mirror image has supported the first tables of data, and described device includes:
Receiving module, the synchronous service data sent for receiving the manufacturing side；Memory module, for storing the synchronous service data
To first tables of data；Screening module meets needs for filtering out from first tables of data according to task requests
Target service data；Transmission module, for the target service data to be transmitted to the manufacturing side, so that the manufacturing side is to the second data
The target service data in table are handled.
A kind of computer readable storage medium is stored thereon with computer program instructions, the computer program instructions quilt
It is realized when execution: receiving the synchronous service data that the manufacturing side is sent；The synchronous service data are stored to the first tables of data；Root
The target service data for meeting and needing are filtered out from first tables of data according to task requests；The target service data are passed
The manufacturing side is transported to, so that the manufacturing side handles the target service data in the second tables of data.
The technical solution provided by above this specification embodiment is as it can be seen that the data processing side that this specification embodiment provides
Method is applied to mirror image end, and the mirror image has supported the first tables of data, and the method can receive the synchronous service that the manufacturing side is sent
Data；The synchronous service data are stored to first tables of data；It is sieved from first tables of data according to task requests
Select the target service data for meeting and needing；The target service data are transmitted to the manufacturing side, so that the manufacturing side is to the second number
It is handled according to the target service data in table.The method that this specification embodiment provides, the screening process of data do not exist
It is carried out in the manufacturing side, can make the manufacturing side only need to handle the data after screening, carry out simple data to the data after screening
Processing, such as deletion, change of data etc., consumption resource is seldom, and the data volume and workload of processing are all greatly reduced, and reduces
The manufacturing side to the occupancy of manufacturing side resource, improves data-handling efficiency in data processing.
Detailed description of the invention
In order to illustrate more clearly of this specification embodiment or technical solution in the prior art, below will to embodiment or
Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only
The some embodiments recorded in this specification, for those of ordinary skill in the art, before not making the creative labor
It puts, is also possible to obtain other drawings based on these drawings.
Fig. 1 is a kind of data processing system schematic diagram of this specification embodiment；
Fig. 2 is a kind of flow chart of data processing method of this specification embodiment；
Fig. 3 is a kind of flow chart of data processing method of this specification embodiment；
Fig. 4 is a kind of schematic diagram of data processing method of this specification embodiment；
Fig. 5 is a kind of functional block diagram of data processing equipment of this specification embodiment.
Specific embodiment
Below in conjunction with the attached drawing in this specification embodiment, the technical solution in this specification embodiment is carried out clear
Chu is fully described by, it is clear that described embodiment is only this specification a part of the embodiment, rather than whole implementation
Example.The embodiment of base in this manual, those of ordinary skill in the art are obtained without making creative work
Every other embodiment, all should belong to this specification protection range.
Referring to Fig. 1, this specification embodiment provides a kind of data processing system 100.In some embodiments, the number
It may include production server 101 according to processing system 100.The production server 101 can be a server；Alternatively, also
It can be the server cluster for including multiple servers.The production server 101 may include the second tables of data, the data
Table is used for storage service data, and the production server 101 can be handled the business datum in data, such as to data
The operation such as increased, deleted, being modified.In some embodiments, the production server 101 can be used for handling banking
Business, provides the integrated service of 7 × 24 hours core banks, the data may include transaction data, accounting data etc..
In some embodiments, the data processing system 100 can also include mirror image server 102.The mirror image clothes
Business device 102 can be a server；Alternatively, can also be the server cluster for including multiple servers.The mirroring service
Device 102 can be used for receiving the synchronous business datum of production server 101, and the business datum is stored in the first tables of data
In；The mirror image server 102 can process the data in the first tables of data, but not in production server 101
Data in two tables of data are modified, for example, synchronous data can be generated with report during handling banking, added
Work file is that business personnel or other peripheries carry out data service.In some embodiments, the mirror image server 102 can
To receive the synchronous business datum of production server 101 every preset time, such as synchronous in daily specific time reception
Business datum, when not having business need, the data in the mirror image server 102 are in stationary state, the mirror image clothes
Business device 102, which does not increase data, deletes, modify etc., to be operated.
Fig. 2 is a kind of flow chart of data processing method of this specification embodiment.The data processing method specifically can be with
Include the following steps.
S210: the manufacturing side initiates synchronizing traffic data.
It is simple synchronization that data, which synchronize, means and is performed simultaneously same operation, and object is positioned as counting by data synchronization
According to being interpreted as different storage device or terminal and terminal to it most of the time, the backup operation between terminal and server.
But complete data, which synchronize, should be in real time, i.e. current operation both sides should be mirror image each other, for example, backup process until knot
Beam is that data are synchronous, but this operation sample then should belong to backup.
In some embodiments, it due to banking demand or system architecture adjustment requirement, needs bank's magnanimity number
It is synchronized to mirror image end system according to by manufacturing side end system, when needing to spend a large amount of during mass data is synchronized
Between, thus it is possible to produce end can be initiated every preset time, data are synchronous, such as same in daily specific time initiation data
Step, daily incremental data synchronize, wherein the incremental data is the new data occurred after last time data synchronize.
S220: mirror image end receives synchronous business datum, and the synchronous service data are stored to first data
Table.
In some embodiments, mirror image end can be after the synchronizing traffic data for receiving manufacturing side initiation, to synchronous
Business datum is received, and the synchronous service data are stored to first tables of data.
S230: mirror image end filters out the target service number for meeting and needing according to task requests from first tables of data
According to.
The task requests can be business personnel's sending, can also be issued with other peripheral systems, such as request pair
The data of specific date generate the tasks such as report, reconciliation, collection.The peripheral system can be the Workflow Management of loan examination & approval
System, the platform data warehouse Knowledge Management System of integrated data, and the general ledger system of opposition.Workflow Management System be for
The application system that the business such as bank loan provide leading portion examination and approval workflow, transmit information；Though data warehouse Knowledge Management System is not
Belong to core system operation system scope, but the synchronized construction of such system, is conducive to excavate and play core business system
Effect constructs unified data platform for bank, the integration of transaction data, accounting data, management information is realized, for administrative analysis
Department is inquired, is analyzed, report is processed；The general ledger system of opposition, it is generally direct by core system in core business system
Account inside control and managing customer account and bank by the general ledger system administration bank ledger to oppose relatively, and provides corresponding meeting
Count the application framework of management function.The peripheral system can also be some interface systems, such as counter-employee system, ATM/POS, phone
Bank, Web bank, Customer Service Center etc., the transaction of these systems can all be passed to the core banking system on backstage, real
Now carry out the update in relation to customer account and general ledger.
In some embodiments, the data that mirror image end is initiated according to the manufacturing side every preset time are synchronous, to synchronous industry
Business data are received, and the mirror image end can be sent same after this data synchronously completes in the reception manufacturing side next time
Before walking business datum, the target service data for meeting and needing are filtered out from first tables of data according to task requests.Tool
Body, the data screening can be completed in data sync interval twice, for example, mirror image end carries out client certificate in a certain day
The overdue data screening of client certificate in synchronous data is come out, needs client all in scan data by overdue search,
Whether the certificate of client is judged expired or will be a few days ago expired at certain, mirror image end obtains the information of this portions of client,
It can be saved with the file of specific format, the mirror image end can carry out data rule according to interface after completing data screening
It is whole, but database will not be modified.
In some embodiments, described to filter out the target for meeting and needing from first tables of data according to task requests
Business datum can be realized by following steps.
Step 1: determining data screening rule according to the task requests；
Step 2: filtering out the target service for meeting and needing from first tables of data according to the data screening rule
Data.
In above-mentioned steps, first task requests can be analyzed first, determine data screening rule.For example, task is asked
Cleaning of the Seeking Truth to invalid data can then determine a data screening rule, to the operation that the first tables of data is traversed, benefit
Judge every data whether in vain with the rule, wherein the rule of the determination can be, by every data and backup
Data compare, if the data from backup are different, judge that the data is invalid.
It in some embodiments, can be true if the task requests are the cleanings to transaction data in the specific date
A fixed data screening rule, to the operation that the first tables of data is traversed, judged using the rule every data whether be
Data in specific date, then judge whether the data are transaction data, or first filter out the data in the specific date, then
Judge whether target service data are transaction data.
In some embodiments, if the task requests are to pick out client certificate in overdue data of a certain day,
It can determine a data screening rule, to the operation that the first tables of data is traversed, the card of client is judged using the rule
Whether part expires in a certain day.
Above-mentioned task requests can also be any other request, and this specification embodiment is not limited thereto, above-mentioned basis
The screening rule that task requests determine is also possible to any other rule that can satisfy task requests, this specification embodiment pair
This is not construed as limiting.
S240: the target service data are transmitted to the manufacturing side by mirror image end.
In some embodiments, the target service data can be transmitted to the manufacturing side by the mirror image end, can also be incited somebody to action
The file of the target service data specific format is saved, then the file is transmitted to the manufacturing side.
In some embodiments, the data that mirror image end is initiated according to the manufacturing side every preset time are synchronous, to synchronous number
According to being received, the mirror image end can receive the synchronous service data that the manufacturing side is sent next time before, by the target industry
Business data are transmitted to the manufacturing side and are handled.Specifically, by the target service data be transmitted to the manufacturing side carry out processing can be with
It is realized by following steps:
Step 1: judging whether the target service data have been handled by the manufacturing side；
In some embodiments, the step for judging the target service data and whether having been handled by the manufacturing side
Suddenly, can be realized by flow scheduling tool: target service data can be sent to by mirror image end after completing data screening
Flow scheduling tool, flow scheduling tool can detect target in the manufacturing side after receiving the target service data
Whether business datum has been handled by the manufacturing side.
In some embodiments, described that the step of whether the target service data are handled by the manufacturing side judged
Can also include: the manufacturing side in processing data procedures, can will treated that Data Filename is modified, mirror image end is passing
Whether before defeated target service data, can detecte in the manufacturing side has filename identical with the target service data, if
Have, then judges that the manufacturing side is not also handled the target service data, if it is not, judging the manufacturing side to the mesh
Mark business datum is handled.
In this specification embodiment, judge method that whether the target service data have been handled by the manufacturing side also
It can be any other method, this specification embodiment is not construed as limiting this.
Step 2: if processed, the target service data being transmitted to the manufacturing side and are handled；If not by
The target service data then then will be transmitted to the manufacturing side after the completion of manufacturing side processing and handled by processing.
In above-mentioned steps, before the target service data to be transmitted to the manufacturing side and are handled, it can be determined that described
Whether target service data have been handled by the manufacturing side.For example, the data of the screening are arrived in a certain day client certificate
The customer profile data of phase first has to these customer information numbers that judgement filters out before the data are transmitted to the manufacturing side
According to whether being handled by the manufacturing side, renewed as client has carried out certificate, then the manufacturing side can extend the term of validity of client certificate, to life
The certificate information data for producing the client in end are modified, and mirror image end will pass the target service data after the data change
It transports to the manufacturing side to be handled, be handled if the target service data are transmitted to the manufacturing side before the data change,
It is capped then to will lead to untreated data, the change to client certificate renewal is caused mistake occur.
In some embodiments, mirror image end can be after filtering out data, in the synchronization that the reception manufacturing side is sent next time
Before business datum, judge whether the target service data have been handled by the manufacturing side.Wherein, the target service number
According to including the increase to the data, modification, deletion by manufacturing side processing, the mirror image end can detecte out the production
Whether end has been handled the target service data.
S250: the manufacturing side handles the target service data in the second tables of data.
The technical solution provided by above this specification embodiment is as it can be seen that the data processing side that this specification embodiment provides
Method is applied to mirror image end, and the mirror image has supported the first tables of data, and the method can receive the synchronous service that the manufacturing side is sent
Data；The synchronous service data are stored to first tables of data；It is sieved from first tables of data according to task requests
Select the target service data for meeting and needing；The target service data are transmitted to the manufacturing side, so that the manufacturing side is to the second number
It is handled according to the target service data in table.The method that this specification embodiment provides, the screening process of data do not exist
It is carried out in the manufacturing side, can make the manufacturing side only need to handle the data after screening, carry out simple data to the data after screening
Processing, such as deletion, change of data etc., consumption resource is seldom, and the data volume and workload of processing are all greatly reduced, and reduces
The manufacturing side to the occupancy of manufacturing side resource, improves data-handling efficiency in data processing.
Method to further understand this specification embodiment, we are illustrated by taking the processing of D days data as an example.Such as
Shown in Fig. 3, Fig. 3 is the manufacturing side and mirror image end the data time point of axis and main at any time in the method for this specification embodiment
Work, it is synchronous that the manufacturing side carries out data at the end of daily.Wherein, perpendicular dotted line indicates new daystart, while the manufacturing side will
Data at this time are synchronized to mirror image end, and between 2 perpendicular dotted lines, the data source data at mirror image end is static, the number at mirror image end
According to comparing the delay of the data of the manufacturing side one day.
It is synchronous that data are carried out after manufacturing side D end of day, after data synchronously complete, mirror image end start according to task requests into
The work of row data screening.In the present embodiment, the time limit of the work theoretically has 24 hours, before manufacturing side D+1 end of day
The work of screening is completed, such as mirror image end carries out client certificate in the overdue search of a certain day, and client in synchronous data is demonstrate,proved
The overdue data screening of part comes out, and needs client all in scan data, judge client certificate whether expired or
Person will be a few days ago expired at certain, and mirror image end obtains the information of this portions of client, can be saved with the file of specific format, institute
Mirror image end is stated after completing data screening, it is regular data to be carried out according to interface, but database will not be modified.
As indicated by the arrows in fig. 3, this is the process that the selection result file at mirror image end is transferred to the manufacturing side.Mirror
After completing screening, whether the screening file that can detect day has been handled by P1, if having handled completion, is stood at picture end
The screening file is transferred to the manufacturing side, otherwise needs that manufacturing side processing is waited to be transmitted to the manufacturing side again after the completion, avoids
Cover untreated file of upper day.
In the present embodiment, when mirror image end carries out data screening, the manufacturing side just obtained additionally in data processing in D+1 days
Resource carry out other important work, the manufacturing side can be further processed the knot screened at mirror image end in beginning in D+2 days
Fruit file, so that the data of manufacturing side processing greatly reduce, it is only necessary to carry out simple data mart modeling, such as the increasing of data
Add, delete, change, consumption resource is seldom, provides data-handling efficiency.
Fig. 4 is a kind of flow chart of data processing method of this specification embodiment.The data processing method is applied to mirror
As end, the mirror image has supported the first tables of data, can specifically include following steps.
S410: the synchronous service data that the manufacturing side is sent are received.
It is simple synchronization that data, which synchronize, means and is performed simultaneously same operation, and object is positioned as counting by data synchronization
According to being interpreted as different storage device or terminal and terminal to it most of the time, the backup operation between terminal and server.
But complete data, which synchronize, should be in real time, i.e. current operation both sides should be mirror image each other, for example, backup process until knot
Beam is that data are synchronous, but this operation sample then should belong to backup.
In some embodiments, it due to banking demand or system architecture adjustment requirement, needs bank's magnanimity number
It is synchronized to mirror image end system according to by manufacturing side end system, when needing to spend a large amount of during mass data is synchronized
Between, thus it is possible to produce end can be initiated every preset time, data are synchronous, such as same in daily specific time initiation data
Step, daily incremental data synchronize, wherein the incremental data is the new data occurred after last time data synchronize.
In some embodiments, mirror image end can receive the synchronous service data that the manufacturing side is sent every preset time.
S420: the synchronous service data are stored to first tables of data.
The mirror image end may include the first tables of data, and first tables of data is used for storage service data.
S430: the target service data for meeting and needing are filtered out from first tables of data according to task requests.
The task requests can be business personnel's sending, can also be issued with other peripheral systems, such as request pair
The data of specific date generate the tasks such as report, reconciliation, collection.The peripheral system can be the Workflow Management of loan examination & approval
System, the platform data warehouse Knowledge Management System of integrated data, and the general ledger system of opposition.Workflow Management System be for
The application system that the business such as bank loan provide leading portion examination and approval workflow, transmit information；Though data warehouse Knowledge Management System is not
Belong to core system operation system scope, but the synchronized construction of such system, is conducive to excavate and play core business system
Effect constructs unified data platform for bank, the integration of transaction data, accounting data, management information is realized, for administrative analysis
Department is inquired, is analyzed, report is processed；The general ledger system of opposition, it is generally direct by core system in core business system
Account inside control and managing customer account and bank by the general ledger system administration bank ledger to oppose relatively, and provides corresponding meeting
Count the application framework of management function.The peripheral system can also be some interface systems, such as counter-employee system, ATM/POS, phone
Bank, Web bank, Customer Service Center etc., the transaction of these systems can all be passed to the core banking system on backstage, real
Now carry out the update in relation to customer account and general ledger.
In some embodiments, the synchronous service data that mirror image end is sent according to the manufacturing side every preset time, to synchronization
Business datum is received, and the mirror image end can be after this data synchronously completes, and the reception manufacturing side is sent same next time
Before walking business datum, the target service data for meeting and needing are filtered out from first tables of data according to task requests.Tool
Body, the data screening can be completed in data sync interval twice, for example, mirror image end carries out client certificate in a certain day
The overdue data screening of client certificate in synchronous data is come out, needs client all in scan data by overdue search,
Whether the certificate of client is judged expired or will be a few days ago expired at certain, mirror image end obtains the information of this portions of client,
It can be saved with the file of specific format, the mirror image end can carry out data rule according to interface after completing data screening
It is whole, but database will not be modified.
In some embodiments, described to filter out the target for meeting and needing from first tables of data according to task requests
Business datum can be realized by following steps.
Step 1: determining data screening rule according to the task requests；
Step 2: filtering out the target service for meeting and needing from first tables of data according to the data screening rule
Data.
In above-mentioned steps, first task requests can be analyzed first, determine data screening rule.For example, task is asked
Cleaning of the Seeking Truth to invalid data can then determine a data screening rule, to the operation that the first tables of data is traversed, benefit
Judge every data whether in vain with the rule, wherein the rule of the determination can be, by every data and backup
Data compare, if the data from backup are different, judge that the data is invalid.
It in some embodiments, can be true if the task requests are the cleanings to transaction data in the specific date
A fixed data screening rule, to the operation that the first tables of data is traversed, judged using the rule every data whether be
Data in specific date, then judge whether the data are transaction data, or first filter out the data in the specific date, then
Judge whether target service data are transaction data.
In some embodiments, if the task requests are to pick out client certificate in overdue data of a certain day,
It can determine a data screening rule, to the operation that the first tables of data is traversed, the card of client is judged using the rule
Whether part expires in a certain day.
Above-mentioned task requests can also be any other request, and this specification embodiment is not limited thereto, above-mentioned basis
The screening rule that task requests determine is also possible to any other rule that can satisfy task requests, this specification embodiment pair
This is not construed as limiting.
S440: being transmitted to the manufacturing side for the target service data, so that the manufacturing side is to the mesh in the second tables of data
Mark business datum is handled.
In some embodiments, the target service data can be transmitted to the manufacturing side by the mirror image end, can also be incited somebody to action
The file of the target service data specific format is saved, then the file is transmitted to the manufacturing side.
In some embodiments, the data that mirror image end is initiated according to the manufacturing side every preset time are synchronous, to synchronous number
According to being received, the mirror image end can receive the synchronous service data that the manufacturing side is sent next time before, by the target industry
Business data are transmitted to the manufacturing side and are handled.Specifically, by the target service data be transmitted to the manufacturing side carry out processing can be with
It is realized by following steps:
Step 1: judging whether the target service data have been handled by the manufacturing side；
In some embodiments, the step for judging the target service data and whether having been handled by the manufacturing side
Suddenly, can be realized by flow scheduling tool: target service data can be sent to by mirror image end after completing data screening
Flow scheduling tool, flow scheduling tool can detect target in the manufacturing side after receiving the target service data
Whether business datum has been handled by the manufacturing side.
In some embodiments, described that the step of whether the target service data are handled by the manufacturing side judged
Can also include: the manufacturing side in processing data procedures, can will treated that Data Filename is modified, mirror image end is passing
Whether before defeated target service data, can detecte in the manufacturing side has filename identical with the target service data, if
Have, then judges that the manufacturing side is not also handled the target service data, if it is not, judging the manufacturing side to the mesh
Mark business datum is handled.
In this specification embodiment, judge method that whether the target service data have been handled by the manufacturing side also
It can be any other method, this specification embodiment is not construed as limiting this.
Step 2: if processed, the target service data being transmitted to the manufacturing side and are handled；If not by
The target service data then then will be transmitted to the manufacturing side after the completion of manufacturing side processing and handled by processing.
In above-mentioned steps, before the target service data to be transmitted to the manufacturing side and are handled, it can be determined that described
Whether target service data have been handled by the manufacturing side.For example, the data of the screening are arrived in a certain day client certificate
The customer profile data of phase first has to these customer information numbers that judgement filters out before the data are transmitted to the manufacturing side
According to whether being handled by the manufacturing side, renewed as client has carried out certificate, then the manufacturing side can extend the term of validity of client certificate, to life
The certificate information data for producing the client in end are modified, and mirror image end will pass the target service data after the data change
It transports to the manufacturing side to be handled, be handled if the target service data are transmitted to the manufacturing side before the data change,
It is capped then to will lead to untreated data, the change to client certificate renewal is caused mistake occur.
In some embodiments, mirror image end can be after filtering out data, in the synchronization that the reception manufacturing side is sent next time
Before business datum, judge whether the target service data have been handled by the manufacturing side.Wherein, the target service number
According to including the increase to the data, modification, deletion by manufacturing side processing, the mirror image end can detecte out the production
Whether end has been handled the target service data.
The technical solution provided by above this specification embodiment is as it can be seen that the data processing side that this specification embodiment provides
Method is applied to mirror image end, and the mirror image has supported the first tables of data, and the method can receive the synchronous service that the manufacturing side is sent
Data；The synchronous service data are stored to first tables of data；It is sieved from first tables of data according to task requests
Select the target service data for meeting and needing；The target service data are transmitted to the manufacturing side, so that the manufacturing side is to the second number
It is handled according to the target service data in table.The method that this specification embodiment provides, the screening process of data do not exist
It is carried out in the manufacturing side, can make the manufacturing side only need to handle the data after screening, carry out simple data to the data after screening
Processing, such as deletion, change of data etc., consumption resource is seldom, and the data volume and workload of processing are all greatly reduced, and reduces
The manufacturing side to the occupancy of manufacturing side resource, improves data-handling efficiency in data processing.
This specification embodiment additionally provides a kind of computer readable storage medium, is stored thereon with computer program and refers to
It enables, the computer program instructions are performed realization: receiving the synchronous service data that the manufacturing side is sent；By the synchronous service
Data are stored to the first tables of data；The target service number for meeting and needing is filtered out from first tables of data according to task requests
According to；The target service data are transmitted to the manufacturing side, so that the manufacturing side is to the target service data in the second tables of data
It is handled.
In the present embodiment, above-mentioned storage medium includes but is not limited to random access memory (Random Access
Memory, RAM), read-only memory (Read-Only Memory, ROM), caching (Cache), hard disk (Hard DiskDrive,
) or storage card (Memory Card) HDD.The memory can be used for storing the computer program and/or module, described
Memory can mainly include storing program area and storage data area, wherein storing program area can storage program area, at least one
Application program needed for function (such as sound-playing function, text conversion function etc.) etc.；Storage data area can store according to
Family terminal uses created data (such as audio data, text message data etc.) etc..In addition, memory may include height
Fast random access memory can also include nonvolatile memory.In the present embodiment, the computer readable storage medium
The function and effect of the program instruction specific implementation of storage, can compare explanation, details are not described herein with other embodiment.
Refering to Fig. 5, on software view, this specification embodiment additionally provides a kind of data processing equipment, device tool
Body may include construction module below.
Receiving module 510, the synchronous service data sent for receiving the manufacturing side；
Memory module 520, for storing the synchronous service data to first tables of data；
Screening module 530, for filtering out the target industry for meeting and needing from first tables of data according to task requests
Business data；
Transmission module 540, for the target service data to be transmitted to the manufacturing side, so that the manufacturing side is to the second tables of data
In the target service data handled.
In some embodiments, described device further include: receive next time the synchronous service data sent of the manufacturing side it
Before, the target service data for meeting and needing are filtered out from first tables of data according to task requests, by the target service
Data are transmitted to the manufacturing side, so that the manufacturing side handles the target service data in the second tables of data.
In some embodiments, the screening module 530 further include: submodule is determined, for according to the task requests
Determine data screening rule；Submodule is screened, for filtering out from first tables of data according to the data screening rule
Meet the target service data needed.
In some embodiments, the transmission module 540 includes: judging submodule, for judging the target service number
According to whether having been handled by the manufacturing side；Submodule is transmitted, is used in the target service data by the manufacturing side
In the case where processing, the target service data are transmitted to the manufacturing side and are handled；In the target service data not by institute
In the case where stating manufacturing side processing, the target service data then will be transmitted to the manufacturing side after the completion of manufacturing side processing
It is handled.
It should be noted that all the embodiments in this specification are described in a progressive manner, each embodiment it
Between the same or similar part may refer to each other, the highlights of each of the examples are it is different from other embodiments it
Place.For Installation practice and apparatus embodiments, since it is substantially similar to the method embodiment, so description
Fairly simple, the relevent part can refer to the partial explaination of embodiments of method.
Those skilled in the art may not need creative work and expect this specification after reading this specification file
It some or all of enumerates embodiment and carries out any combination, these combinations are also in the range of this disclosure and protection.
In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example,
Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So
And with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.
Designer nearly all obtains corresponding hardware circuit by the way that improved method flow to be programmed into hardware circuit.Cause
This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device
(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate
Array, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designer
Voluntarily programming comes a digital display circuit " integrated " on a piece of PLD, designs and makes without asking chip maker
Dedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " is patrolled
Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,
And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language
(Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL
(Advanced Boolean Expression Language)、AHDL(AlteraHardware Description
Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL
(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby
Hardware Description Language) etc., VHDL (Very-High-Speed is most generally used at present
Integrated Circuit Hardware Description Language) and Verilog2.Those skilled in the art
It will be apparent to the skilled artisan that only needing method flow slightly programming in logic and being programmed into integrated circuit with above-mentioned several hardware description languages
In, so that it may it is readily available the hardware circuit for realizing the logical method process.
System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by the product with certain function.It is a kind of typically to realize that equipment is computer.Specifically, computer for example may be used
Think personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
It is any in device, navigation equipment, electronic mail equipment, game console, tablet computer, wearable device or these equipment
The combination of equipment.
As seen through the above description of the embodiments, those skilled in the art can be understood that this specification
It can realize by means of software and necessary general hardware platform.Based on this understanding, the technical solution of this specification
Substantially the part that contributes to existing technology can be embodied in the form of software products in other words, the computer software
Product can store in storage medium, such as ROM/RAM, magnetic disk, CD, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) executes each embodiment of this specification or embodiment
Certain parts described in method.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality
For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method
Part explanation.
This specification can be used in numerous general or special purpose computing system environments or configuration.Such as: personal computer,
Server computer, handheld device or portable device, laptop device, multicomputer system, microprocessor-based system,
Set top box, programmable consumer-elcetronics devices, network PC, minicomputer, mainframe computer including any of the above system are set
Standby distributed computing environment etc..
This specification can describe in the general context of computer-executable instructions executed by a computer, such as journey
Sequence module.Generally, program module include routines performing specific tasks or implementing specific abstract data types, programs, objects,
Component, data structure etc..This specification can also be practiced in a distributed computing environment, in these distributed computing environment
In, by executing task by the connected remote processing devices of communication network.In a distributed computing environment, program module
It can be located in the local and remote computer storage media including storage equipment.
Although depicting this specification by embodiment, it will be appreciated by the skilled addressee that there are many become for this specification
Shape and the spirit changed without departing from this specification, it is desirable to which the attached claims include these deformations and change without departing from this
The spirit of specification.
Claims (12)
1. a kind of data processing method, which is characterized in that be applied to mirror image end, the mirror image has supported the first tables of data；It is described
Method includes:
Receive the synchronous service data that the manufacturing side is sent；
The synchronous service data are stored to first tables of data；
The target service data for meeting and needing are filtered out from first tables of data according to task requests；
The target service data are transmitted to the manufacturing side, so that the manufacturing side is to the target service data in the second tables of data
It is handled.
2. the method as described in claim 1, which is characterized in that the synchronous service data sent of the manufacturing side that receive include:
The synchronous service data that the manufacturing side is sent are received every preset time.
3. according to the method described in claim 2, it is characterized in that, the method also includes: next time receive the manufacturing side hair
Before the synchronous service data come, the target service number for meeting and needing is filtered out from first tables of data according to task requests
According to the target service data are transmitted to the manufacturing side.
4. method according to claim 1-3, which is characterized in that described to be counted according to task requests from described first
Include: according to the target service data for meeting and needing are filtered out in table
Data screening rule is determined according to the task requests；
The target service data for meeting and needing are filtered out from first tables of data according to the data screening rule.
5. method according to claim 1-3, which is characterized in that described by the business data transmission to production
End includes:
Judge whether the target service data have been handled by the manufacturing side；
If processed, the target service data are transmitted to the manufacturing side.
6. according to the method described in claim 5, it is characterized in that, being completed if not processed in manufacturing side processing
After the target service data then will be transmitted to the manufacturing side.
7. a kind of data processing equipment, which is characterized in that be applied to mirror image end, the mirror image has supported the first tables of data, described
Device includes:
Receiving module, the synchronous service data sent for receiving the manufacturing side；
Memory module, for storing the synchronous service data to first tables of data；
Screening module, for filtering out the target service data for meeting and needing from first tables of data according to task requests；
Transmission module, for the target service data to be transmitted to the manufacturing side, so that the manufacturing side is to the institute in the second tables of data
Target service data are stated to be handled.
8. device according to claim 7, which is characterized in that the receiving module is also used to receive every preset time and give birth to
Produce the synchronous service data that end is sent.
9. device according to claim 8, which is characterized in that described device further include: receiving manufacturing side hair next time
Before the synchronous service data come, the target service number for meeting and needing is filtered out from first tables of data according to task requests
According to the target service data being transmitted to the manufacturing side, so that the manufacturing side is to the target service data in the second tables of data
It is handled.
10. according to the described in any item devices of claim 7-9, which is characterized in that the screening module includes:
Submodule is determined, for determining data screening rule according to the task requests；
Submodule is screened, for filtering out the target for meeting and needing from first tables of data according to the data screening rule
Business datum.
11. according to the described in any item devices of claim 7-9, which is characterized in that the transmission module includes:
Judging submodule, for judging whether the target service data have been handled by the manufacturing side；
Submodule is transmitted, is used in the case where the target service data are handled by the manufacturing side, by the target
Business data transmission to the manufacturing side is handled；In the case where the target service data are not handled by the manufacturing side, In
The target service data then will be transmitted to the manufacturing side after the completion of the manufacturing side processing to handle.
12. a kind of computer readable storage medium, is stored thereon with computer program instructions, the computer program instructions are held
Row Shi Shixian: the synchronous service data that the manufacturing side is sent are received；The synchronous service data are stored to the first tables of data；According to
Task requests filter out the target service data for meeting and needing from first tables of data；The target service data are transmitted
To the manufacturing side, so that the manufacturing side handles the target service data in the second tables of data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910754056.9A CN110457327A (en) | 2019-08-15 | 2019-08-15 | A kind of data processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910754056.9A CN110457327A (en) | 2019-08-15 | 2019-08-15 | A kind of data processing method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110457327A true CN110457327A (en) | 2019-11-15 |
Family
ID=68486833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910754056.9A Pending CN110457327A (en) | 2019-08-15 | 2019-08-15 | A kind of data processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110457327A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015738A (en) * | 2020-08-28 | 2020-12-01 | 支付宝(杭州)信息技术有限公司 | Method and device for realizing linked list processing of multiple data detail lists |
| CN117078139A (en) * | 2023-10-16 | 2023-11-17 | 国家邮政局邮政业安全中心 | Cross-border express supervision method, system, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101102233A (en) * | 2007-06-27 | 2008-01-09 | 中国移动通信集团四川有限公司 | Detection processing method for data inconsistency |
| CN101859319A (en) * | 2009-02-12 | 2010-10-13 | 阿森图尔环球服务有限责任公司 | The data system framework of analysis distribution formula data set |
| CN102937931A (en) * | 2012-09-25 | 2013-02-20 | 北京奇虎科技有限公司 | Method and system of log data transmission |
| CN107818431A (en) * | 2016-09-14 | 2018-03-20 | 北京京东尚科信息技术有限公司 | A kind of method and system that order track data is provided |
| CN108415792A (en) * | 2018-01-15 | 2018-08-17 | 阿里巴巴集团控股有限公司 | Disaster tolerance system, method, apparatus and equipment |
| CN108629014A (en) * | 2018-05-07 | 2018-10-09 | 中国平安人寿保险股份有限公司 | A kind of data processing method, device, computer equipment and storage medium |
-
2019
- 2019-08-15 CN CN201910754056.9A patent/CN110457327A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101102233A (en) * | 2007-06-27 | 2008-01-09 | 中国移动通信集团四川有限公司 | Detection processing method for data inconsistency |
| CN101859319A (en) * | 2009-02-12 | 2010-10-13 | 阿森图尔环球服务有限责任公司 | The data system framework of analysis distribution formula data set |
| CN102937931A (en) * | 2012-09-25 | 2013-02-20 | 北京奇虎科技有限公司 | Method and system of log data transmission |
| CN107818431A (en) * | 2016-09-14 | 2018-03-20 | 北京京东尚科信息技术有限公司 | A kind of method and system that order track data is provided |
| CN108415792A (en) * | 2018-01-15 | 2018-08-17 | 阿里巴巴集团控股有限公司 | Disaster tolerance system, method, apparatus and equipment |
| CN108629014A (en) * | 2018-05-07 | 2018-10-09 | 中国平安人寿保险股份有限公司 | A kind of data processing method, device, computer equipment and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015738A (en) * | 2020-08-28 | 2020-12-01 | 支付宝(杭州)信息技术有限公司 | Method and device for realizing linked list processing of multiple data detail lists |
| CN117078139A (en) * | 2023-10-16 | 2023-11-17 | 国家邮政局邮政业安全中心 | Cross-border express supervision method, system, electronic equipment and storage medium |
| CN117078139B (en) * | 2023-10-16 | 2024-02-09 | 国家邮政局邮政业安全中心 | Cross-border express supervision method, system, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Muniswamaiah et al. | Big data in cloud computing review and opportunities | |
| US20230144450A1 (en) | Multi-partitioning data for combination operations | |
| CN109034993A (en) | Account checking method, equipment, system and computer readable storage medium | |
| US20210112115A1 (en) | Integrating logic in micro batch based event processing systems | |
| CN107818431B (en) | Method and system for providing order track data | |
| US20150379429A1 (en) | Interactive interfaces for machine learning model evaluations | |
| JP2017530469A (en) | Enriching events with dynamically typed big data for event processing | |
| WO2018169429A1 (en) | Framework for the deployment of event-based applications | |
| US20080249981A1 (en) | Systems and methods for federating data | |
| CN102236705A (en) | Fine grain synchronization for database replay | |
| CN111339073A (en) | Real-time data processing method and device, electronic equipment and readable storage medium | |
| US20120084253A1 (en) | Transaction in memory object store | |
| CN106537386A (en) | Identifying files for data write operations | |
| CN113094434A (en) | Database synchronization method, system, device, electronic equipment and medium | |
| JP2011258122A (en) | Data transfer unit, data transfer method, data transfer program, and data cooperation system | |
| CN110457327A (en) | A kind of data processing method, device and storage medium | |
| Oleson et al. | Operational information systems: An example from the airline industry | |
| CN112527801A (en) | Data synchronization method and system between relational database and big data system | |
| CN108733515A (en) | Dispatching method, file backup method, device and the storage medium of file backup | |
| CN110955719B (en) | Data access processing equipment, system and method | |
| AU2015265595B2 (en) | System and method for dynamic collection of system management data in a mainframe computing environment | |
| CN109901918A (en) | A kind of method and apparatus handling overtime task | |
| Dunning et al. | Real-world hadoop | |
| US20150112947A1 (en) | System and method for database flow management | |
| CN111753015A (en) | Data query method and device of payment clearing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191115 |