CN109816481A - Bill processing method, device and computer readable storage medium - Google Patents
Bill processing method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN109816481A CN109816481A CN201910007213.XA CN201910007213A CN109816481A CN 109816481 A CN109816481 A CN 109816481A CN 201910007213 A CN201910007213 A CN 201910007213A CN 109816481 A CN109816481 A CN 109816481A
- Authority
- CN
- China
- Prior art keywords
- transaction data
- ordering system
- channel side
- transaction
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 72
- 238000000034 method Methods 0.000 claims abstract description 25
- 239000012634 fragment Substances 0.000 claims description 22
- 238000013467 fragmentation Methods 0.000 claims 2
- 238000006062 fragmentation reaction Methods 0.000 claims 2
- 230000008569 process Effects 0.000 description 13
- 230000009977 dual effect Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of bill processing methods, this method comprises: obtaining the transaction request of ordering system, the relevant transaction data of the transaction request of ordering system is stored in the database of ordering system, the transaction data of channel side is read from the file server of channel side, the transaction data of channel side is stored on the NAS disk of ordering system, call the transaction data of ordering system and the transaction data of channel side, the transaction data of the transaction data of ordering system and channel side is compared, when the transaction data of the transaction data of ordering system and channel side is compared successfully, file in the NAS disk of ordering system is uploaded in the file server of channel side.The present invention also proposes a kind of bill processing unit and a kind of computer readable storage medium.The present invention realizes reconciliation performance and stablizes, and as the data volume of transaction is increasing, reconciliation and export performance will not significantly decrease, and scalability is strong.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of bill processing method, device and computer-readable deposit
Storage media.
Background technique
There are two transaction systems currently on the market, and one is ordering system, one be channel side system, the two are
The reconciliation of system is then done in the database to the data of the two by landing the reconciliation file of channel side into database
Reconciliation operation and export, however it is very big if necessary to the data volume of reconciliation, and history account data amount is very big, this reconciliation
Mode will be very slow, and as the increasing this reconciliation performance of data volume can be worse and worse.
Summary of the invention
The present invention provides a kind of bill processing method, device and computer readable storage medium, main purpose and is reality
Show reconciliation performance to stablize, as the data volume of transaction is increasing, reconciliation and export performance will not significantly decrease；
Scalability is strong.
To achieve the above object, the present invention also provides a kind of bill processing methods, which comprises
The transaction request for obtaining ordering system, is stored in order system for the relevant transaction data of the transaction request of ordering system
In the database of system；
The transaction data that channel side is read from the file server of channel side, is stored in order for the transaction data of channel side
On the network attached storage NAS disk of system；
The transaction data of ordering system and the transaction data of channel side are called, by the transaction data and channel side of ordering system
Transaction data be compared；
When the transaction data of the transaction data of ordering system and channel side is compared successfully, by the NAS of ordering system
File in disk uploads in the file server of channel side.
Optionally, the relevant transaction data of the transaction request by ordering system is stored in the database of ordering system
Include:
According to sliding window strategy, start timed task in each time window, by the ordering system in time window
Transaction data fragment batch write-in Hadoop distributed file system, according to order application odd numbers Hash codes hashcode into
Row fragment.
Optionally, the transaction data for calling ordering system and the transaction data of channel side include:
Ordering system calls the Driver of Spark cluster, starts Spark task schedule, each task task of Spark from
The transaction data of channel side is read in Spark node on NAS disk, wherein the transaction data of channel side includes channel source mark
Know；
And based on channel source identification, according in the transaction data of channel side order application odd numbers Hash codes
Hashcode carries out repartition processing, and Spark node reads in ordering system from Hadoop distributed file system after having handled
Transaction data, read the corresponding fragment of task task number ID of Spark respectively.
Optionally, described when the transaction data of the transaction data of ordering system and channel side is compared successfully, it will order
File in the NAS disk of single system uploads in the file server of channel side, comprising: when the transaction data and canal of ordering system
When the transaction data of road side is compared successfully, the Map object of ordering system is changed into String object and is written to HDFS by Spark
System；
After the completion of all fragment write-ins, the API of Hadoop distributed file system is called, the file of each distribution is closed
And and upload to the NAS disk of ordering system, the file under the NAS disk is uploaded to the sftp server of channel side by ordering system
On.
Optionally, when the transaction data of the transaction data of ordering system and channel side is compared unsuccessful, reception pair
The processing of problem data, and re-start reconciliation.
Optionally, the Hadoop distributed file system be configured with fault-tolerant architecture, wherein fault-tolerant architecture by metadata with
Timestamp composition, same order number, the big person of timestamp are valid data.
To achieve the above object, the present invention also provides a kind of bill processing unit, described device includes memory and processing
Device is stored with the bill processing routine that can be run on the processor on the memory, and the bill processing routine is by institute
It states when processor executes and realizes following steps:
The transaction request for obtaining ordering system, is stored in order system for the relevant transaction data of the transaction request of ordering system
In the database of system；
The transaction data that channel side is read from the file server of channel side, is stored in order for the transaction data of channel side
On the network attached storage NAS disk of system；
The transaction data of ordering system and the transaction data of channel side are called, by the transaction data and channel side of ordering system
Transaction data be compared；
When the transaction data of the transaction data of ordering system and channel side is compared successfully, by the NAS of ordering system
File in disk upload in the file server of channel side when the transaction data of ordering system and the transaction data of channel side into
When row is compared successfully, the file in the NAS disk of ordering system is uploaded in the file server of channel side.
Optionally, the bill processing routine can also be executed by the processor, also realization following steps:
According to sliding window strategy, start timed task in each time window, by the ordering system in time window
Transaction data fragment batch write-in Hadoop distributed file system (HDFS) distributed file system, according to the request slip of order
Number Hash codes hashcode carry out fragment.
Optionally, the bill processing routine can also be executed by the processor, realize following steps:
Ordering system calls the Driver of Spark cluster, starts Spark task schedule, each task task of Spark from
The transaction data of channel side is read in Spark node on NAS disk, wherein the transaction data of channel side includes channel source mark
Know；
And based on channel source identification, according in the transaction data of channel side order application odd numbers Hash codes
Hashcode carries out repartition processing, and Spark node reads in ordering system from Hadoop distributed file system after having handled
Transaction data, read the corresponding fragment of task task number ID of Spark respectively.
In addition, to achieve the above object, it is described computer-readable the present invention also provides a kind of computer readable storage medium
Bill processing routine is stored on storage medium, the bill processing routine can be executed by one or more processor, with reality
Now the step of bill processing method as described above.
The present invention obtains the transaction request of ordering system, and the relevant transaction data of the transaction request of ordering system is stored in
In the database of ordering system, the transaction data of channel side is read from the file server of channel side, by the number of deals of channel side
According on the NAS disk for being stored in ordering system, the transaction data of ordering system and the transaction data of channel side are called, by ordering system
Transaction data and the transaction data of channel side be compared, when ordering system transaction data and channel side transaction data into
When row is compared successfully, the file in the NAS disk of ordering system is uploaded in the file server of channel side.Reconciliation of the present invention
It can stablize, as the data volume of transaction is increasing, reconciliation and export performance will not significantly decrease；Scalability
It is strong: performance can be improved by way of increasing machine node；Reduce system pressure: thought handled using class sliding window formula,
Reduce reconciliation and the processing of the export pressure caused by database；Business is fault-tolerant: being directed to Hadoop distributed file system
Only newly-increased and additional feature designs fault-tolerant architecture, avoids data modification that big performance is brought to be lost.
Detailed description of the invention
Fig. 1 is the flow diagram for the bill processing method that one embodiment of the invention provides；
Fig. 2 is the schematic diagram of internal structure for the bill processing unit that one embodiment of the invention provides；
The module diagram of bill processing routine in the bill processing unit that Fig. 3 provides for one embodiment of the invention.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
The present invention provides a kind of bill processing method.Shown in referring to Fig.1, handled for the bill that one embodiment of the invention provides
The flow diagram of method.This method can be executed by a device, which can be by software and or hardware realization.
In the present embodiment, bill processing method includes:
S10, the transaction request for obtaining ordering system, the relevant transaction data of the transaction request of ordering system is stored in and is ordered
In the database of single system.
In the present embodiment, channel side sends real-time deal request and is given to ordering system, and ordering system in real time asks transaction
Relevant transaction data is asked to be stored in database.
Preferably, the relevant transaction data of the transaction request by ordering system is stored in the database of ordering system
Include:
According to sliding window strategy, start timed task in each time window, by the ordering system in time window
Transaction data fragment batch write-in Hadoop distributed file system (HDFS), according to the Hash codes of the application odd numbers of order
Hashcode carries out fragment.Such as filename starts with 0, step-length 1 is incremented by.
S20, the transaction data that channel side is read from the file server of channel side, the transaction data of channel side is stored
In on the NAS disk of ordering system.
In the present embodiment, periodically (such as daily) number of deals for reading channel side from the file server of channel side
According to the transaction data of channel side is stored on the network attached storage NAS disk of ordering system.The temporal frequency periodically read can
With configured in advance.
Wherein, fragment is carried out according to the Hash codes hashcode of the application odd numbers of the order of channel side.Such as filename is with 0
Start, step-length 1, is incremented by.
The transaction data of S30, the transaction data for calling ordering system and channel side, by the transaction data and canal of ordering system
The transaction data of road side is compared.
Wherein with the transaction data of the architectural form management ordering system of cluster and the transaction data of channel side.Spark is answered
It, with the operation of independent process collection, is adjusted on cluster with SparkContext object in main program (referred to as driver)
Section.Particularly, in order to run on cluster, SparkContext can be with cluster manager dual system (Spark itself list of several types
Only cluster manager dual system or Mesos/YARN) it is connected, these cluster manager dual systems can distribute resource between application.Once even
Connect, Spark needs thread pool child node on cluster, that is, those execute calculate and storage using data work into
Journey.Then, it will send your application code (with JAR or the Python file defined and being transmitted to SparkContext)
To thread pool.Thread pool is allowed to run finally, SparkContext sends task.
Each application has the thread pool process of oneself, can keep in the operational process entirely applied and in multiple threads
Operation task.The advantage of doing so is that application is mutually isolated, both in scheduling aspect (task of each driving scheduling own)
Also in terms of execution (task of different application is run on different JVM).
For potential cluster manager dual system, Spark is unknowable.As long as it need thread pool process and they
Between communication, even when be run on the cluster manager dual system (for example, Mesos/YARN) for also supporting other application it is also opposite
Simply.
Because of driving scheduler task on cluster, it should run and be close to working node, in identical local area network more
It is good.
Preferably, the transaction data for calling ordering system and the transaction data of channel side include:
Ordering system calls the Driver of Spark cluster, starts Spark task schedule, each task task of Spark from
The transaction data of channel side is read in Spark node on NAS disk, wherein the transaction data of channel side includes channel source mark
Know；And based on channel source identification, according in the transaction data of channel side order application odd numbers Hash codes hashcode into
Row repartition step process, Spark node reads in the transaction data of ordering system from HDFS system after having handled, and reads respectively
To the corresponding fragment of task task number ID of Spark.
Preferably, after by processing step above, there is the transaction of ordering system in each fragment in Spark
The transaction data of data and channel side, each task task start to compare the transaction of the transaction data and channel side to ordering system
Data processing.
S40, when the transaction data of the transaction data of ordering system and channel side is compared successfully, by ordering system
File in NAS disk uploads in the file server of channel side.
Preferably, it when the transaction data of the transaction data of ordering system and channel side is compared successfully, that is, orders
When the transaction data of single system and the completely the same transaction data of channel side, Spark changes into the Map object of ordering system
String object is written to HDFS system；After the completion of all fragment write-ins, the API of HDFS system is called, by the text of each distribution
Part merges, and uploads to the NAS disk of ordering system, and ordering system takes the sftp that the file under the NAS disk uploads to channel side
It is engaged on device.
When the transaction data of the transaction data of ordering system and channel side is compared unsuccessful, that is to say, that reconciliation has
Problem then needs artificial treatment data, after the completion of data processing, repeats above step reconciliation again secondary.
It is therefore preferred that being connect when the transaction data of the transaction data of ordering system and channel side is compared unsuccessful
The processing to problem data is received, and re-starts reconciliation.
Preferably, the HDFS system configuration has fault-tolerant architecture, and wherein fault-tolerant architecture is made of metadata and timestamp.Phase
With order number, the big person of timestamp is valid data.Because may be modified for data on HDFS, while HDFS does not support to repair
The problem of changing is related to a fault-tolerant architecture, avoids importing database data again because of data modification.Fault-tolerant architecture is by metadata
It is formed with timestamp, same order number, the big person of timestamp is valid data.
The present invention obtains the transaction request of ordering system, and the relevant transaction data of the transaction request of ordering system is stored in
In the database of ordering system, the transaction data of channel side is read from the file server of channel side, by the transaction of channel side
Data are stored on the NAS disk of ordering system, call the transaction data of ordering system and the transaction data of channel side, by order system
The transaction data of system and the transaction data of channel side are compared, when the transaction data of ordering system and the transaction data of channel side
When being compared successfully, the file in the NAS disk of ordering system is uploaded in the file server of channel side.Reconciliation of the present invention
Performance is stablized, and as the data volume of transaction is increasing, reconciliation and export performance will not significantly decrease；Scalability
It is strong: performance can be improved by way of increasing machine node；Reduce system pressure: thought handled using class sliding window formula,
Reduce reconciliation and the processing of the export pressure caused by database；Business is fault-tolerant: only newly-increased and additional for HDFS
Feature designs fault-tolerant architecture, avoids data modification that big performance is brought to be lost.
The present invention also provides a kind of bill processing units.At the bill shown in Fig. 2, provided for one embodiment of the invention
Manage the schematic diagram of internal structure of device.
In the present embodiment, bill processing unit 1 can be PC (Personal Computer, PC), can also be with
It is the terminal devices such as smart phone, tablet computer, portable computer.The bill processing unit 1 includes at least memory 11, processing
Device 12, communication bus 13 and network interface 14.
Wherein, memory 11 include at least a type of readable storage medium storing program for executing, the readable storage medium storing program for executing include flash memory,
Hard disk, multimedia card, card-type memory (for example, SD or DX memory etc.), magnetic storage, disk, CD etc..Memory 11
It can be the internal storage unit of bill processing unit 1, such as the hard disk of the bill processing unit 1 in some embodiments.It deposits
Reservoir 11 is also possible in further embodiments on the External memory equipment of bill processing unit 1, such as bill processing unit 1
The plug-in type hard disk of outfit, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD)
Card, flash card (Flash Card) etc..Further, memory 11 can also both include the storage inside of bill processing unit 1
Unit also includes External memory equipment.Memory 11 can be not only used for the application software that storage is installed on bill processing unit 1
And Various types of data, such as the code of bill processing routine 01 etc., it can be also used for temporarily storing and exported or will be defeated
Data out.
Processor 12 can be in some embodiments a central processing unit (Central Processing Unit,
CPU), controller, microcontroller, microprocessor or other data processing chips, the program for being stored in run memory 11
Code or processing data, such as execute bill processing routine 01 etc..
Communication bus 13 is for realizing the connection communication between these components.
Network interface 14 optionally may include standard wireline interface and wireless interface (such as WI-FI interface), be commonly used in
Communication connection is established between the device 1 and other electronic equipments.
Optionally, which can also include user interface, and user interface may include display (Display), input
Unit such as keyboard (Keyboard), optional user interface can also include standard wireline interface and wireless interface.It is optional
Ground, in some embodiments, display can be light-emitting diode display, liquid crystal display, touch-control liquid crystal display and OLED
(Organic Light-Emitting Diode, Organic Light Emitting Diode) touches device etc..Wherein, display can also be appropriate
Referred to as display screen or display unit, for being shown in the information handled in bill processing unit 1 and for showing visually
User interface.
Fig. 2 illustrates only the bill processing unit 1 with component 11-14 and bill processing routine 01, art technology
Personnel may include than illustrating more it is understood that structure shown in fig. 1 does not constitute the restriction of reconciliation processing apparatus 1
Perhaps more component perhaps combines certain components or different component layouts less.
In 1 embodiment of device shown in Fig. 2, bill processing routine 01 is stored in memory 11；Processor 12 executes
Following steps are realized when the bill processing routine 01 stored in memory 11:
The transaction request for obtaining ordering system, is stored in order system for the relevant transaction data of the transaction request of ordering system
In the database of system.
In the present embodiment, channel side sends real-time deal request and is given to ordering system, and ordering system in real time asks transaction
Relevant transaction data is asked to be stored in database.
Further, in another embodiment of apparatus of the present invention, bill processing routine can also be called by processor, with reality
Existing following steps:
According to sliding window strategy, start timed task in each time window, by the ordering system in time window
Transaction data fragment batch write-in Hadoop distributed file system (HDFS), according to the Hash codes of the application odd numbers of order
Hashcode carries out fragment.Such as filename starts with 0, step-length 1 is incremented by.
The transaction data that channel side is read from the file server of channel side, the transaction data of channel side is stored in and is ordered
On the NAS disk of single system.
In the present embodiment, periodically (such as daily) number of deals for reading channel side from the file server of channel side
According to the transaction data of channel side is stored on the network attached storage NAS disk of ordering system.The temporal frequency periodically read can
With configured in advance.
Wherein, fragment is carried out according to the Hash codes hashcode of the application odd numbers of the order of channel side.Such as filename is with 0
Start, step-length 1, is incremented by.
The transaction data of ordering system and the transaction data of channel side are called, by the transaction data and channel side of ordering system
Transaction data be compared.
Wherein with the transaction data of the architectural form management ordering system of cluster and the transaction data of channel side.Spark is answered
It, with the operation of independent process collection, is adjusted on cluster with SparkContext object in main program (referred to as driver)
Section.Particularly, in order to run on cluster, SparkContext can be with cluster manager dual system (Spark itself list of several types
Only cluster manager dual system or Mesos/YARN) it is connected, these cluster manager dual systems can distribute resource between application.Once even
Connect, Spark needs thread pool child node on cluster, that is, those execute calculate and storage using data work into
Journey.Then, it will send your application code (with JAR or the Python file defined and being transmitted to SparkContext)
To thread pool.Thread pool is allowed to run finally, SparkContext sends task.
Each application has the thread pool process of oneself, can keep in the operational process entirely applied and in multiple threads
Operation task.The advantage of doing so is that application is mutually isolated, both in scheduling aspect (task of each driving scheduling own)
Also in terms of execution (task of different application is run on different JVM).
For potential cluster manager dual system, Spark is unknowable.As long as it need thread pool process and they
Between communication, even when be run on the cluster manager dual system (for example, Mesos/YARN) for also supporting other application it is also opposite
Simply.
Because of driving scheduler task on cluster, it should run and be close to working node, in identical local area network more
It is good.
Preferably, bill processing routine can also be called by processor, to realize that following steps include:
Ordering system calls the Driver of Spark cluster, starts Spark task schedule, each task task of Spark from
The transaction data of channel side is read in Spark node on NAS disk, wherein the transaction data of channel side includes channel source mark
Know；And based on channel source identification, according in the transaction data of channel side order application odd numbers Hash codes hashcode into
Row repartition step process, Spark node reads in the transaction data of ordering system from HDFS system after having handled, and reads respectively
To the corresponding fragment of task task number ID of Spark.
Preferably, after by processing step above, there is the transaction of ordering system in each fragment in Spark
The transaction data of data and channel side, each task task start to compare the transaction of the transaction data and channel side to ordering system
Data processing.
When the transaction data of the transaction data of ordering system and channel side is compared successfully, by the NAS of ordering system
File in disk uploads in the file server of channel side.
Preferably, it when the transaction data of the transaction data of ordering system and channel side is compared successfully, that is, orders
When the transaction data of single system and the completely the same transaction data of channel side, Spark changes into the Map object of ordering system
String object is written to HDFS system；After the completion of all fragment write-ins, the API of HDFS system is called, by the text of each distribution
Part merges, and uploads to the NAS disk of ordering system, and ordering system takes the sftp that the file under the NAS disk uploads to channel side
It is engaged on device.
When the transaction data of the transaction data of ordering system and channel side is compared unsuccessful, that is to say, that reconciliation has
Problem then needs artificial treatment data, after the completion of data processing, repeats above step reconciliation again secondary.
It is therefore preferred that being connect when the transaction data of the transaction data of ordering system and channel side is compared unsuccessful
The processing to problem data is received, and re-starts reconciliation.
Preferably, the HDFS system configuration has fault-tolerant architecture, and wherein fault-tolerant architecture is made of metadata and timestamp.Phase
With order number, the big person of timestamp is valid data.Because may be modified for data on HDFS, while HDFS does not support to repair
The problem of changing is related to a fault-tolerant architecture, avoids importing database data again because of data modification.Fault-tolerant architecture is by metadata
It is formed with timestamp, same order number, the big person of timestamp is valid data.
The present invention obtains the transaction request of ordering system, and the relevant transaction data of the transaction request of ordering system is stored in
In the database of ordering system, the transaction data of channel side is read from the file server of channel side, by the transaction of channel side
Data are stored on the NAS disk of ordering system, call the transaction data of ordering system and the transaction data of channel side, by order system
The transaction data of system and the transaction data of channel side are compared, when the transaction data of ordering system and the transaction data of channel side
When being compared successfully, the file in the NAS disk of ordering system is uploaded in the file server of channel side.Reconciliation of the present invention
Performance is stablized, and as the data volume of transaction is increasing, reconciliation and export performance will not significantly decrease；Scalability
It is strong: performance can be improved by way of increasing machine node；Reduce system pressure: thought handled using class sliding window formula,
Reduce reconciliation and the processing of the export pressure caused by database；Business is fault-tolerant: only newly-increased and additional for HDFS
Feature designs fault-tolerant architecture, avoids data modification that big performance is brought to be lost.
Optionally, in other embodiments, bill processing routine can also be divided into one or more module, and one
Or multiple modules are stored in memory 11, and performed by one or more processors (the present embodiment is processor 12)
To complete the present invention, the so-called module of the present invention is the series of computation machine program instruction section for referring to complete specific function, is used
In implementation procedure of the description bill processing routine in bill processing unit.
It is the program mould of the bill processing routine in one embodiment of bill processing unit of the present invention for example, referring to shown in Fig. 3
Block schematic diagram, in the embodiment, bill processing routine can be divided into plate and obtain module 10, memory module 20, compare mould
Block 30 and sending module 40, illustratively:
It obtains module 10 to be used for: the transaction request of ordering system is obtained, by the relevant transaction of the transaction request of ordering system
Data are stored in the database of ordering system；
Memory module 20 is used for: the transaction data of channel side is read from the file server of channel side, by channel side
Transaction data is stored on the network attached storage NAS disk of ordering system；
Comparison module 30 is used for: the transaction data of ordering system and the transaction data of channel side is called, by ordering system
Transaction data and the transaction data of channel side are compared；
Sending module 40 is used for: when the transaction data of the transaction data of ordering system and channel side is compared successfully,
File in the NAS disk of ordering system is uploaded in the file server of channel side.
The program modules such as above-mentioned acquisition module 10, memory module 20, comparison module 30 and sending module 40 are performed institute
Functions or operations step and above-described embodiment of realization are substantially the same, and details are not described herein.
In addition, the embodiment of the present invention also proposes a kind of computer readable storage medium, the computer readable storage medium
On be stored with bill processing routine, the bill processing routine can be executed by one or more processors, to realize following operation:
The transaction request for obtaining ordering system, is stored in order system for the relevant transaction data of the transaction request of ordering system
In the database of system；
The transaction data that channel side is read from the file server of channel side, the transaction data of channel side is stored in and is ordered
On the network attached storage NAS disk of single system；
The transaction data of ordering system and the transaction data of channel side are called, by the transaction data and channel side of ordering system
Transaction data be compared；
When the transaction data of the transaction data of ordering system and channel side is compared successfully, by the NAS of ordering system
File in disk uploads in the file server of channel side.
Computer readable storage medium specific embodiment of the present invention and above-mentioned bill processing unit and each embodiment of method
It is essentially identical, do not make tired state herein.
It should be noted that the serial number of the above embodiments of the invention is only for description, do not represent the advantages or disadvantages of the embodiments.And
The terms "include", "comprise" herein or any other variant thereof is intended to cover non-exclusive inclusion, so that packet
Process, device, article or the method for including a series of elements not only include those elements, but also including being not explicitly listed
Other element, or further include for this process, device, article or the intrinsic element of method.Do not limiting more
In the case where, the element that is limited by sentence "including a ...", it is not excluded that including process, device, the article of the element
Or there is also other identical elements in method.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in one as described above
In storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that terminal device (it can be mobile phone,
Computer, server or network equipment etc.) execute method described in each embodiment of the present invention.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of bill processing method, which is characterized in that the described method includes:
The transaction request for obtaining ordering system, is stored in ordering system for the relevant transaction data of the transaction request of ordering system
In database；
The transaction data that channel side is read from the file server of channel side, is stored in ordering system for the transaction data of channel side
Network attached storage NAS disk on；
The transaction data of ordering system and the transaction data of channel side are called, by the friendship of the transaction data of ordering system and channel side
Easy data are compared；
It, will be in the NAS disk of ordering system when the transaction data of the transaction data of ordering system and channel side is compared successfully
File upload in the file server of channel side.
2. bill processing method as described in claim 1, which is characterized in that the transaction request by ordering system is relevant
Transaction data is stored in the database of ordering system
According to sliding window strategy, start timed task in each time window, by the transaction of the ordering system in time window
Data fragmentation batch write-in Hadoop distributed file system, is divided according to the Hash codes hashcode of the application odd numbers of order
Piece.
3. bill processing method as described in claim 1, which is characterized in that the transaction data and canal for calling ordering system
The transaction data of road side includes:
Ordering system calls the Driver of Spark cluster, starts Spark task schedule, and each task task of Spark is from NAS
The transaction data of channel side is read in Spark node on disk, wherein the transaction data of channel side includes channel source identification；
And based on channel source identification, according in the transaction data of channel side order application odd numbers Hash codes hashcode into
The processing of row repartition, Spark node reads in the transaction data of ordering system from Hadoop distributed file system after having handled,
The corresponding fragment of task task number ID of Spark is read respectively.
4. bill processing method as described in claim 1, which is characterized in that the transaction data and channel when ordering system
When the transaction data of side is compared successfully, the file in the NAS disk of ordering system is uploaded to the file server of channel side
In, comprising:
When the transaction data of the transaction data of ordering system and channel side is compared successfully, Spark is by the Map of ordering system
Object changes into String object and is written to Hadoop distributed file system；
After the completion of all fragment write-ins, the API of Hadoop distributed file system is called, by the file mergences of each distribution, and
The NAS disk of ordering system is uploaded to, ordering system uploads to the file under the NAS disk on the sftp server of channel side.
5. bill processing method as described in claim 1, which is characterized in that when the transaction data and channel side of ordering system
When transaction data is compared unsuccessful, the processing to problem data is received, and re-start reconciliation.
6. such as the described in any item bill processing methods of claim 2 to 4, which is characterized in that the Hadoop distributed document
System configuration has fault-tolerant architecture, and wherein fault-tolerant architecture is made of metadata and timestamp, same order number, the big person of timestamp
For valid data.
7. a kind of bill processing unit, which is characterized in that described device includes memory and processor, is stored on the memory
There is the bill processing routine that can be run on the processor, is realized such as when the bill processing routine is executed by the processor
Lower step:
The transaction request for obtaining ordering system, is stored in ordering system for the relevant transaction data of the transaction request of ordering system
In database；
The transaction data that channel side is read from the file server of channel side, is stored in ordering system for the transaction data of channel side
Network attached storage NAS disk on；
The transaction data of ordering system and the transaction data of channel side are called, by the friendship of the transaction data of ordering system and channel side
Easy data are compared；
It, will be in the NAS disk of ordering system when the transaction data of the transaction data of ordering system and channel side is compared successfully
File upload in the file server of channel side.
8. bill processing unit as claimed in claim 7, which is characterized in that the bill processing routine can also be by the processing
Device executes, also realization following steps:
According to sliding window strategy, start timed task in each time window, by the transaction of the ordering system in time window
Data fragmentation batch write-in Hadoop distributed file system, is divided according to the Hash codes hashcode of the application odd numbers of order
Piece.
9. bill processing unit as claimed in claim 7, which is characterized in that the bill processing routine can also be by the processing
Device executes, and realizes following steps:
Ordering system calls the Driver of Spark cluster, starts Spark task schedule, and each task task of Spark is from NAS
The transaction data of channel side is read in Spark node on disk, wherein the transaction data of channel side includes channel source identification；
And based on channel source identification, according in the transaction data of channel side order application odd numbers Hash codes hashcode into
The processing of row repartition, Spark node reads in the transaction data of ordering system from Hadoop distributed file system after having handled,
The corresponding fragment of task task number ID of Spark is read respectively.
10. a kind of computer readable storage medium, which is characterized in that be stored at bill on the computer readable storage medium
Program is managed, the bill processing routine can be executed by one or more processor, to realize as any in claim 1 to 6
The step of bill processing method described in item.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910007213.XA CN109816481A (en) | 2019-01-04 | 2019-01-04 | Bill processing method, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910007213.XA CN109816481A (en) | 2019-01-04 | 2019-01-04 | Bill processing method, device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109816481A true CN109816481A (en) | 2019-05-28 |
Family
ID=66603944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910007213.XA Pending CN109816481A (en) | 2019-01-04 | 2019-01-04 | Bill processing method, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109816481A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110544080A (en) * | 2019-07-03 | 2019-12-06 | 威富通科技有限公司 | payment reconciliation method and server |
| CN111143401A (en) * | 2019-12-27 | 2020-05-12 | 中国银行股份有限公司 | Query information processing method and device |
| CN111899011A (en) * | 2020-07-29 | 2020-11-06 | 广州海鹚网络科技有限公司 | Medical bill reconciliation system, computer equipment and storage medium |
| CN112052141A (en) * | 2020-09-02 | 2020-12-08 | 平安科技（深圳）有限公司 | Data fragment verification method and device, computer equipment and readable storage medium |
| CN112258191A (en) * | 2020-12-22 | 2021-01-22 | 深圳市深圳通有限公司 | Data reconciliation method, device, equipment and storage medium |
| CN113177400A (en) * | 2021-04-28 | 2021-07-27 | 中国工商银行股份有限公司 | Keyword comparison method and device based on large-data-volume file |
| EP4213037A4 (en) * | 2020-09-09 | 2024-10-09 | The Peoples Bank Of China Nat Clearing Center | Data storage and reconciliation method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105989462A (en) * | 2015-02-02 | 2016-10-05 | 卓望数码技术（深圳）有限公司 | Settlement method capable of supporting different payment channels and settlement system |
| CN107665460A (en) * | 2016-07-24 | 2018-02-06 | 平安科技（深圳）有限公司 | Accounting method and device based on real-time deal |
| CN107993151A (en) * | 2018-01-17 | 2018-05-04 | 平安科技（深圳）有限公司 | Fund exchange settlement method, apparatus, equipment and computer-readable recording medium |
| CN109034993A (en) * | 2018-09-29 | 2018-12-18 | 深圳前海微众银行股份有限公司 | Account checking method, equipment, system and computer readable storage medium |
-
2019
- 2019-01-04 CN CN201910007213.XA patent/CN109816481A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105989462A (en) * | 2015-02-02 | 2016-10-05 | 卓望数码技术（深圳）有限公司 | Settlement method capable of supporting different payment channels and settlement system |
| CN107665460A (en) * | 2016-07-24 | 2018-02-06 | 平安科技（深圳）有限公司 | Accounting method and device based on real-time deal |
| CN107993151A (en) * | 2018-01-17 | 2018-05-04 | 平安科技（深圳）有限公司 | Fund exchange settlement method, apparatus, equipment and computer-readable recording medium |
| CN109034993A (en) * | 2018-09-29 | 2018-12-18 | 深圳前海微众银行股份有限公司 | Account checking method, equipment, system and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 袁景凌: "《Spark案例与实验教程》", 30 April 2017, 武汉大学出版社, pages: 136 - 137 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110544080A (en) * | 2019-07-03 | 2019-12-06 | 威富通科技有限公司 | payment reconciliation method and server |
| CN111143401A (en) * | 2019-12-27 | 2020-05-12 | 中国银行股份有限公司 | Query information processing method and device |
| CN111899011A (en) * | 2020-07-29 | 2020-11-06 | 广州海鹚网络科技有限公司 | Medical bill reconciliation system, computer equipment and storage medium |
| CN112052141A (en) * | 2020-09-02 | 2020-12-08 | 平安科技（深圳）有限公司 | Data fragment verification method and device, computer equipment and readable storage medium |
| EP4213037A4 (en) * | 2020-09-09 | 2024-10-09 | The Peoples Bank Of China Nat Clearing Center | Data storage and reconciliation method and system |
| CN112258191A (en) * | 2020-12-22 | 2021-01-22 | 深圳市深圳通有限公司 | Data reconciliation method, device, equipment and storage medium |
| CN113177400A (en) * | 2021-04-28 | 2021-07-27 | 中国工商银行股份有限公司 | Keyword comparison method and device based on large-data-volume file |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109816481A (en) | Bill processing method, device and computer readable storage medium | |
| CN107908472A (en) | Data synchronization unit, method and computer-readable recording medium | |
| CN107515933A (en) | Change method, server, system and the storage medium of source data database table structure | |
| CN110019267A (en) | A kind of metadata updates method, apparatus, system, electronic equipment and storage medium | |
| CN109309712A (en) | Data transmission method, server and the storage medium called based on interface asynchronous | |
| CN107656729A (en) | Updating device, method and the computer-readable recording medium of List View | |
| CN109634915A (en) | File dispositions method, Cloud Server, system and storage medium | |
| CN108958881A (en) | Data processing method, device and computer readable storage medium | |
| CN110457311A (en) | Automatically generate method, server and the storage medium of reconciliation file | |
| CN111813573B (en) | Communication method of management platform and robot software and related equipment thereof | |
| CN109271410A (en) | Extracting method, device and the computer readable storage medium of bank receipt | |
| CN110457346A (en) | Data query method, apparatus and computer readable storage medium | |
| CN110019263A (en) | Information storage means and device | |
| CN112860662A (en) | Data blood relationship establishing method and device, computer equipment and storage medium | |
| US20160232307A1 (en) | Server in visiting service management support system, control method thereof, and control program thereof | |
| CN108073698B (en) | Real-time animation display methods, device, electric terminal and readable storage medium storing program for executing | |
| CN114157679A (en) | Cloud-native-based distributed application monitoring method, device, equipment and medium | |
| CN113254106B (en) | Task execution method and device based on Flink, computer equipment and storage medium | |
| CN106850838A (en) | The control method and system of mobile terminal cloud application | |
| CN110149313A (en) | Data sharing method, electronic device and computer readable storage medium | |
| CN109582888A (en) | Web page bookmark method for sorting and system | |
| CN109639801A (en) | Back end distribution and data capture method and system | |
| US20160019602A1 (en) | Advertisement method of electronic device and electronic device thereof | |
| CN117271122A (en) | Task processing method, device, equipment and storage medium based on separation of CPU and GPU | |
| CN109241727B (en) | Permission setting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |