CN110191182A - Distributed document batch processing method, device, equipment and readable storage medium storing program for executing - Google Patents
Distributed document batch processing method, device, equipment and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN110191182A CN110191182A CN201910465966.5A CN201910465966A CN110191182A CN 110191182 A CN110191182 A CN 110191182A CN 201910465966 A CN201910465966 A CN 201910465966A CN 110191182 A CN110191182 A CN 110191182A
- Authority
- CN
- China
- Prior art keywords
- file
- export
- downloading
- downloading end
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of distributed document batch processing methods, it include: when detecting file download notice, determine that the file download notifies corresponding downloading end, and the file download is notified by corresponding file by the downloading end, the corresponding local disk of the downloading end is downloaded to from corresponding cloud storage；The file content of the file is read in the local disk by the downloading end, and is based on the file content, is imported the file in the corresponding database of the downloading end by the downloading end；The file in the database is handled by the corresponding processing end of the database, to obtain to export；It is exported to export to the local disk by the processing end by described, and is sent to the cloud storage to export for described.The invention also discloses a kind of distributed document batch processing device, equipment and a kind of storage mediums.The present invention replaces NAS disk by database, realizes Distributed data exchange.
Description
Technical field
The present invention relates to financial technology (Fintech) technical field more particularly to distributed document batch processing methods, dress
It sets, equipment and readable storage medium storing program for executing.
Background technique
In recent years, with financial technology (Fintech), the especially continuous development of internet finance, data processing technique
It is introduced into the daily service of bank and other financial mechanism.During financial institution's daily service, generally require to do race batch work
Make, running batch main is exactly to generate ledgers, carry out total score verification and either carry out volume business, such as settles interests, count and mention, withhold pair
Deng；Report is either generated, export pipelined data etc. namely system needs criticize the distributed document from each sub-agencies
Processing, then how to carry out batch processing to distributed document.
In the prior art, number is stored using shared NAS disk (Network Attached Storage, network memory)
According to, realize distributed structure/architecture data exchange, specifically, local disk is downloaded files into from cloud storage by local machine, and by
NAS disk is written in local machine, then reads file from NAS disk by processing system, and handle file, finally by processing system
By treated, NAS disk is written in file again, and is synchronized in cloud storage, realizes data exchange.
But at present to the batch processing mode of distributed document, rely on NAS disk and realize, needs higher hardware cost, and NAS
Performance write-in is slow, slower when especially large amount of small documents is written.Therefore, race batch processing mode of the prior art to distributed document
It could be improved.
Summary of the invention
It is a primary object of the present invention to propose a kind of distributed document batch processing method, device, equipment and readable storage
Medium, it is intended to improve the high efficiency of processing distributed document.
To achieve the above object, the present invention provides a kind of distributed document batch processing method, at the distributed document batch
Reason method includes the following steps:
When detecting file download notice, determine that the file download notifies corresponding downloading end, and by under described
It carries end and the file download is notified into corresponding file, the corresponding local of the downloading end is downloaded to from corresponding cloud storage
Disk；
The file content of the file is read in the local disk by the downloading end, and based in the file
Hold, is imported the file in the corresponding database of the downloading end by the downloading end；
The file in the database is handled by the corresponding processing end of the database, to obtain text to be exported
Part；
It is exported to export to the local disk, and described will be sent described to export by the processing end
To the cloud storage.
Preferably, described to determine that the file download notifies corresponding downloading end when detecting file download notice, and
The file download is notified into corresponding file by the downloading end, the downloading end pair is downloaded to from corresponding cloud storage
The step of local disk answered includes:
When detecting file download notice, determine that the file download notifies corresponding downloading end；
The file download is notified that file status is text to be downloaded in corresponding listed files by the downloading end
Part is downloaded to the corresponding local disk of the downloading end from corresponding cloud storage；
The file status of the file is changed in downloading or has been downloaded by the downloading end, and according to ought be above
Part state updates the listed files.
Preferably, the file content for reading the file in the local disk by the downloading end, and pass through
The step that the file imports in the corresponding database of the downloading end includes: by the downloading end based on the file content
The transmitting-receiving allocation list of the file is read in the local disk by the downloading end, and is matched based on the transmitting-receiving
It sets table and determines whether the file needs to import；
If desired, then it is based on the transmitting-receiving allocation list, file type belonging to the file is determined by the processing end,
And it is based on the file type, determine the lead-in type of the file；
Based on the lead-in type, the file is imported by the corresponding database of the downloading end by the downloading end
In.
Preferably, described to be based on the lead-in type if the lead-in type is single thread importing, pass through the downloading
The step that the file imports in the corresponding database of the downloading end is included: by end
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and reads the file configuration of the file by the downloading end, with true
The fixed file header for whether needing to skip the file；
If desired, then by the downloading end that the file importing downloading end for skipping the file header is corresponding
In database.
Preferably, described to be based on the lead-in type if the lead-in type is multithreading importing, pass through the downloading
The step that the file imports in the corresponding database of the downloading end is included: by end
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and is the first present count by the file declustering by the downloading end
The first of amount splits file；
File in parallel is split by described first by the downloading end to import in the corresponding database of the downloading end.
Preferably, described to be exported to export to the local disk by the processing end by described, and will it is described to
Export is sent to the step of cloud storage and includes:
It is described to the corresponding transmitting-receiving allocation list of export by processing end acquisition, and based on described to export
Corresponding transmitting-receiving allocation list determines the init state to export；
Based on the init state, whether need to export to export by the way that processing end determination is described；
If desired, then by the processing end determine it is described to export in the local disk and in the cloud storage pair
The first catalogue answered, and by the second fractionation file for being split as the second preset quantity to export, and export to first
Catalogue；
Splitting file mergences for described second in first catalogue is to merge file, and be merged into the text to be exported
Corresponding second catalogue of part.
Preferably, it is described if desired, then by the processing end determine it is described to export the local disk and
Corresponding first catalogue of the cloud storage, and be split as the second of the second preset quantity by described to export and split file,
And it exports to the step of the first catalogue and includes:
If desired, it is determined that it is described to export the local disk in corresponding first catalogue of the cloud storage,
And determine that first catalogue whether there is Stub File；
If it does not exist, then by the processing end by it is described to export be split as the second preset quantity second split
File, and export to first catalogue；
If it exists, then the Stub File is deleted, and is split as second to export for described by the processing end
The second of preset quantity splits file, and exports to first catalogue.
In addition, to achieve the above object, the present invention also provides a kind of distributed document batch processing device, the distributed text
Part batch processing device includes:
Download module, for determining that the file download notifies corresponding downloading end when detecting file download notice,
And the file download is notified by corresponding file by the downloading end, the downloading end is downloaded to from corresponding cloud storage
Corresponding local disk；
Import modul, for reading the file content of the file, and base in the local disk by the downloading end
In the file content, the file is imported in the corresponding database of the downloading end by the downloading end；
Processing module, for handling the file in the database by the corresponding processing end of the database, with
It obtains to export；
Export module, for being exported to export to the local disk by the processing end by described, and will be described
The cloud storage is sent to export.
Preferably, the download module is also used to:
When detecting file download notice, determine that the file download notifies corresponding downloading end；
The file download is notified that file status is text to be downloaded in corresponding listed files by the downloading end
Part is downloaded to the corresponding local disk of the downloading end from corresponding cloud storage；
The file status of the file is changed in downloading or has been downloaded by the downloading end, and according to ought be above
Part state updates the listed files.
Preferably, the import modul is also used to:
The transmitting-receiving allocation list of the file is read in the local disk by the downloading end, and is matched based on the transmitting-receiving
It sets table and determines whether the file needs to import；
If desired, then it is based on the transmitting-receiving allocation list, file type belonging to the file is determined by the processing end,
And it is based on the file type, determine the lead-in type of the file；
Based on the lead-in type, the file is imported by the corresponding database of the downloading end by the downloading end
In.
Preferably, if the lead-in type is single thread importing, the import modul is also used to:
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and reads the file configuration of the file by the downloading end, with true
The fixed file header for whether needing to skip the file；
If desired, then by the downloading end that the file importing downloading end for skipping the file header is corresponding
In database.
Preferably, if the lead-in type is multithreading importing, the import modul is also used to:
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and is the first present count by the file declustering by the downloading end
The first of amount splits file；
File in parallel is split by described first by the downloading end to import in the corresponding database of the downloading end.
Preferably, the export module is also used to:
It is described to the corresponding transmitting-receiving allocation list of export by processing end acquisition, and based on described to export
Corresponding transmitting-receiving allocation list determines the init state to export；
Based on the init state, whether need to export to export by the way that processing end determination is described；
If desired, then by the processing end determine it is described to export in the local disk and in the cloud storage pair
The first catalogue answered, and by the second fractionation file for being split as the second preset quantity to export, and export to first
Catalogue；
Splitting file mergences for described second in first catalogue is to merge file, and be merged into the text to be exported
Corresponding second catalogue of part.
Preferably, the export module is also used to:
If desired, it is determined that it is described to export the local disk in corresponding first catalogue of the cloud storage,
And determine that first catalogue whether there is Stub File；
If it does not exist, then by the processing end by it is described to export be split as the second preset quantity second split
File, and export to first catalogue；
If it exists, then the Stub File is deleted, and is split as second to export for described by the processing end
The second of preset quantity splits file, and exports to first catalogue.
In addition, to achieve the above object, the present invention also provides a kind of distributed document batch processing equipment, the distributed text
Part batch processing equipment includes: memory, processor and point that is stored on the memory and can run on the processor
Cloth file batch program realizes as described above point when the distributed document batch program is executed by the processor
The step of cloth file batch processing method.
In addition, to achieve the above object, it is described computer-readable the present invention also provides a kind of computer readable storage medium
Distributed document batch program is stored on storage medium, it is real when the distributed document batch program is executed by processor
Now the step of distributed document batch processing method as described above.
Distributed document batch processing method proposed by the present invention determines the file when detecting file download notice
Downloading notifies corresponding downloading end, and the file download is notified corresponding file by the downloading end, from corresponding cloud
The corresponding local disk of the downloading end is downloaded in storage；The file is read in the local disk by the downloading end
File content, and it is based on the file content, the file is imported by the corresponding data of the downloading end by the downloading end
In library；The file in the database is handled by the corresponding processing end of the database, to obtain to export；It is logical
It crosses the processing end to export described to export to the local disk, and is sent to the cloud by described to export and deposits
Storage.The present invention replaces NAS disk by database, realizes Distributed data exchange.
Detailed description of the invention
Fig. 1 is the device structure schematic diagram for the hardware running environment that the embodiment of the present invention is related to；
Fig. 2 is the flow diagram of distributed document batch processing method first embodiment of the present invention；
Fig. 3 is the flow diagram of distributed document batch processing method second embodiment of the present invention.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
As shown in Figure 1, Fig. 1 is the device structure schematic diagram for the hardware running environment that the embodiment of the present invention is related to.
The terminal of that embodiment of the invention can be PC machine or server apparatus.
As shown in Figure 1, the terminal may include: processor 1001, such as CPU, network interface 1004, user interface
1003, memory 1005, communication bus 1002.Wherein, communication bus 1002 is for realizing the connection communication between these components.
User interface 1003 may include display screen (Display), input unit such as keyboard (Keyboard), optional user interface
1003 can also include standard wireline interface and wireless interface.Network interface 1004 optionally may include that the wired of standard connects
Mouth, wireless interface (such as WI-FI interface).Memory 1005 can be high speed RAM memory, be also possible to stable memory
(non-volatile memory), such as magnetic disk storage.Memory 1005 optionally can also be independently of aforementioned processor
1001 storage device.
It will be understood by those skilled in the art that device structure shown in Fig. 1 does not constitute the restriction to equipment, can wrap
It includes than illustrating more or fewer components, perhaps combines certain components or different component layouts.
As shown in Figure 1, as may include that operating system, network are logical in a kind of memory 1005 of computer storage medium
Believe module, Subscriber Interface Module SIM and distributed document batch program.
Wherein, operating system is to manage and control the program of distributed document batch processing equipment and software resource, supports net
The operation of network communication module, Subscriber Interface Module SIM, distributed document batch program and other programs or software；Network communication
Module is for managing and controlling network interface 1002；Subscriber Interface Module SIM is for managing and controlling user interface 1003.
In distributed document batch processing equipment shown in Fig. 1, the distributed document batch processing equipment passes through processor
The distributed document batch program stored in 1001 calling memories 1005, and execute following distributed document batch processing methods
Operation in each embodiment.
Based on above-mentioned hardware configuration, distributed document batch processing method embodiment of the present invention is proposed.
It is the flow diagram of distributed document batch processing method first embodiment of the present invention, the side referring to Fig. 2, Fig. 2
Method includes:
Step S10 determines that the file download notifies corresponding downloading end, and lead to when detecting file download notice
It crosses the downloading end and the file download is notified into corresponding file, it is corresponding that the downloading end is downloaded to from corresponding cloud storage
Local disk；
Step S20 reads the file content of the file by the downloading end in the local disk, and based on described
File content is imported the file in the corresponding database of the downloading end by the downloading end；
Step S30 handles the file in the database by the corresponding processing end of the database, with obtain to
Export；
Step S40 is exported to export to the local disk by the processing end by described, and by described wait export
File is sent to the cloud storage.
The present embodiment replaces NAS disk using database, downloads files into local disk by downloading end, and import database
In, then file is read from database by processing end and file is handled, the file of processing is finally exported to local
Disk, and it is synchronized to database, realize the batch processing of distributed document.
Each step will be described in detail below:
Step S10 determines that the file download notifies corresponding downloading end, and lead to when detecting file download notice
It crosses the downloading end and the file download is notified into corresponding file, it is corresponding that the downloading end is downloaded to from corresponding cloud storage
Local disk.
The present embodiment distributed document batch processing method is applied to point of the financing financial institutions such as mechanism or banking system
In cloth file batch processing equipment, for convenience of description, distributed document batch processing equipment hereinafter referred to as processing equipment, wherein place
Reason equipment includes several downloading ends, several processing ends and database etc., and file download notice is issued by messaging bus, and any one
Downloading end can receive file download notice, and the downloading end for receiving file download notice is responsible for notifying to correspond to by file download
File fall library.
Specifically, determining in the file download notice for detecting messaging bus publication and receiving file download notice
Downloading end by the downloading end notifies that it is corresponding to downloading end to download corresponding file in corresponding cloud storage from file download
In local disk.
It should be noted that the IP information of the downloading end need to be recorded when determining the downloading end for receiving file download notice,
And when by corresponding file download to local disk, the IP information of record is associated with current file, i.e., record current file by
Any platform downloading end downloading.
Further, step S10 includes:
Step a determines that the file download notifies corresponding downloading end when detecting file download notice；
In this step, to prevent Single Point of Faliure, any downloading end can receive file download notice, specifically
, start polling procedure, the file status of each file in each downloading end wheel flow monitoring listed files.In the case where detecting file
When carrying notice, downloading end in the monitoring state is determined as this downloading end, and records the IP information of the downloading end.
The file download is notified that file status is under in corresponding listed files by the downloading end by step b
The file of load is downloaded to the corresponding local disk of the downloading end from corresponding cloud storage；
In this step, there is the file status of each file in listed files, under indicating whether each file needs
It carries, notifies the file status indicated in corresponding listed files especially by downloading end scanning file download, and by file status
It is file to be downloaded from being downloaded in corresponding cloud storage in the corresponding local disk of downloading end.
The file status of the file is changed in downloading or has been downloaded by the downloading end by step c, and according to
Current file state updates the listed files.
In this step, during downloading end downloads current file, the file status of current file is changed to download
In, and after the download is complete, the file status of current file is changed to download, and the file status of current file is real
Shi Gengxin is into listed files, to update listed files.
It is appreciated that in the present embodiment, it is file to be downloaded that downloading end, which only downloads file status, and works as current file
File status be downloading in when, even if in polling interval do not complete downloading, next downloading end detect file download notify
When, it does not need to be downloaded it yet.
Certainly, to avoid downloading end delay machine or downloading process from interrupting, file is caused to be constantly in the state in downloading, when
The preceding downloading end for executing downloading movement also detects other than it will be in file download to local disk to be downloaded in listed files
Whether the corresponding downloading end of file that file status is in downloading is normal, specifically can determine the text that file status is in downloading
The associated IP information of part sends survival detection packet to the corresponding downloading end of the IP information by current downloading end, determines the
Whether corresponding response bag is received in one preset time；Or in the second preset time, determine that file status is in downloading
In the download progress of file whether change, if so, showing the corresponding downloading of file that file status is in downloading
Rectify often, without being adjusted to the file in downloading；If not, it is determined that under file of the file status in downloading is corresponding
It carries end delay machine or downloading process is interrupted, at this point, switching action will be executed, specifically switch to current downloading end and execute downloading behaviour
Make, and the IP information of file association corresponding in listed files is changed to the IP information of current downloading end.
Step S20 reads the file content of the file by the downloading end in the local disk, and based on described
File content is imported the file in the corresponding database of the downloading end by the downloading end.
In the present embodiment, after downloading files into local disk, being read in local disk by downloading end ought be above
The file content of part, this document content include the associated IP information of current file, are based on file content, whether determine current file
It is that current downloading end is downloaded, if it is not, not handling then；If so, current file is imported in the corresponding database of downloading end.
That is the file status of current file to have downloaded is downloaded by current downloading end, and current downloading end can be just conducted into
In database, for file status be it is to be downloaded or downloading in or other downloading ends downloading file, current downloading end
It does not need to be conducted into database.
Further, after the file content for reading file, it is based on file content, determines importing opportunity, it is possible to understand that,
Some files do not need emergent management, therefore, the file of downloading can be divided into urgent file and non-emergent file in advance, promptly
File can import in database in real time, for non-emergent file, then can be processed in batches, i.e., the non-emergent file that will currently download
Retain, and when non-emergent file reaches quantity set in advance, by all one piece of importing databases of non-emergent file, or
Person is based on preset interval time, will be in one piece of importing database of non-emergent file in interval time.
Specifically during current file is imported database, mark the file status of current file to import in, and
After current file is imported into database, by the file status of current file labeled as having imported, avoid repeating to import.
Step S30 handles the file in the database by the corresponding processing end of the database, with obtain to
Export.
In the present embodiment, using the file in the corresponding processing end reading database of database, and to file at
Reason, concrete processing procedure are determined according to the type of service of file, if current file is the file that keeps accounts of refunding, belong to accounting class, then
Refund account in file is kept accounts to refund and refund number is calculated, to take off the corresponding amount of money of refund account, and by institute
The amount of money taken off keeps accounts；If current file is to settle interests file, belong to class of settling interests, then obtain the interest rate of each account in file of settling interests,
The savings amount of money and storage time of each account, and it is based on interest rate, the amount of money and storage time, calculate the interest of each account；Such as work as
Preceding document is chronological file, belongs to flowing water account, then exports pipelined data, generates flowing water report etc..
That is, reading current file using processing end, and determine type of service belonging to current file, be based on type of service,
Current file is handled using corresponding processing mode, to obtain to export.
Step S40 is exported to export to the local disk by the processing end by described, and by described wait export
File is sent to the cloud storage.
In the present embodiment, it is obtaining after export, will exported to export to local disk by processing end, with
Corresponding file in local disk is updated, and cloud storage will be sent to export, to update corresponding file in cloud storage.
Further, can will retain to export, and after being spaced third preset time, by institute in third preset time
Some is exported to one piece of export to local disk, and all synchronize to export is sent to cloud storage.
Further, it can will retain to export, and when export reaches quantity set in advance, will own
Export to one piece of export to local disk, and all synchronize to export is sent to cloud storage.
The present embodiment determines that the file download notifies corresponding downloading end, and lead to when detecting file download notice
It crosses the downloading end and the file download is notified into corresponding file, it is corresponding that the downloading end is downloaded to from corresponding cloud storage
Local disk；The file content of the file is read in the local disk by the downloading end, and based in the file
Hold, is imported the file in the corresponding database of the downloading end by the downloading end；It is corresponding by the database
Processing end handles the file in the database, to obtain to export；By the processing end by described wait export
File is exported to the local disk, and is sent to the cloud storage to export for described.The present invention is replaced by database
NAS disk by downloading end downloading object file, and imports database, carries out data processing to file destination by corresponding processing end,
And after treatment, by treated, file destination is exported to local disk, and is synchronized to cloud storage, realizes Distributed data exchange.
Further, it is based on distributed document batch processing method first embodiment of the present invention, proposes the distributed text of the present invention
Part batch processing method second embodiment.
The first embodiment of the second embodiment and distributed document batch processing method of distributed document batch processing method
Difference is, includes: referring to Fig. 3, step S20
Step S21 is read the transmitting-receiving allocation list of the file in the local disk by the downloading end, and is based on institute
It states transmitting-receiving allocation list and determines whether the file needs to import；
Step S22, if desired, be then based on the transmitting-receiving allocation list, determined belonging to the file by the processing end
File type, and it is based on the file type, determine the lead-in type of the file；
Step S23 is based on the lead-in type, by the downloading end that the file importing downloading end is corresponding
In database.
The present embodiment need to judge the file type of file and import class when the file that will have been downloaded imports database
Correct file is smoothly imported database by type, improves the intelligence that file imports.
Each step will be described in detail below:
Step S21 is read the transmitting-receiving allocation list of the file in the local disk by the downloading end, and is based on institute
It states transmitting-receiving allocation list and determines whether the file needs to import.
In the present embodiment, each file is corresponding with transmitting-receiving allocation list, receives and dispatches in allocation list and sets correspondence in advance
File imports and exports situation, and specifically including need to not need to import database, need to not need export database and corresponding text
Therefore part type etc. can read the transmitting-receiving allocation list of current file, so that it is determined that whether current file needs to lead by downloading end
Enter database.
Step S22, if desired, be then based on the transmitting-receiving allocation list, determined belonging to the file by the processing end
File type, and it is based on the file type, determine the lead-in type of the file.
In the present embodiment, however, it is determined that need to import current file into database, then it is corresponding further to read current file
Transmitting-receiving allocation list, determine the file type of current file, wherein file type includes fixed-length file and non-fixed-length file, can
With understanding, however, it is determined that current file is fixed-length file, then needs to further determine that the corresponding fixed length character of fixed-length file, with after an action of the bowels
Continue when carrying out importing database to current file, is imported according to determining fixed length character.
After the file type of current file has been determined, lead-in type can be determined according to file type, specifically, if
The file type of current file is fixed-length file, then corresponding lead-in type is multithreading importing；If the files classes of current file
Type is non-fixed-length file, then corresponding lead-in type is single thread importing.
Step S23 is based on the lead-in type, by the downloading end that the file importing downloading end is corresponding
In database.
It in the present embodiment, ought be above by downloading end according to corresponding lead-in mode according to determining lead-in type
Part imports in the corresponding database of downloading end.
Specifically, step S23 is specifically included if lead-in type is single thread importing:
It determines in the database with the presence or absence of the corresponding failure record of the file；
In this step, first determining whether there is the corresponding failure record of current file in database, it is possible to understand that,
It is likely occurred the failure such as delay machine or process interrupt during file is imported database, file is caused to import failure,
And the data remained in the database are failure record, are the files for importing failure before to avoid current file, and cause
Data duplication occupies the too many space of database, and therefore, first need to determine database currently whether there is the corresponding mistake of current file
Lose record.
If it exists, then the failure record is cleared up, and reads the file configuration of the file by the downloading end, with true
The fixed file header for whether needing to skip the file；
In this step, however, it is determined that there are the corresponding failure record of current file in database, then the failure record is cleared up,
And the file configuration of current file is read by downloading end, it is possible to understand that, when file is imported database, have with class file
There is identical file header, therefore, certain files can be determined in advance whether need to import file header according to the actual situation, if when leading
When the line number entered is the first row of current file, and file configuration is when skipping file header, and the file header of current file does not import
Otherwise file header is also introduced into database by database.
It should be understood that being not present, then the file configuration that the file is read by the downloading end is directly executed, with
Determine the need for the step of skipping the file header of the file.
If desired, then by the downloading end that the file importing downloading end for skipping the file header is corresponding
In database.
In this step, however, it is determined that need, then file header is skipped by downloading end, downloading will be imported except the file of file header
It holds in corresponding database.
It should be understood that directly executing if not needing and importing current file in corresponding database.
If lead-in type is multithreading importing, step S23 is specifically included:
It determines in the database with the presence or absence of the corresponding failure record of the file；
In this step, first determining whether there is the corresponding failure record of current file in database, it is possible to understand that,
It is likely occurred the failure such as delay machine or process interrupt during file is imported database, file is caused to import failure,
And the data remained in the database are failure record, are the files for importing failure before to avoid current file, and cause
Data duplication occupies the too many space of database, and therefore, first need to determine database currently whether there is the corresponding mistake of current file
Lose record.
If it exists, then the failure record is cleared up, and is the first present count by the file declustering by the downloading end
The first of amount splits file；
In this step, however, it is determined that there are the corresponding failure record of current file in database, then the failure record is cleared up,
And current file is split as to the first fractionation file of the first preset quantity by downloading end, wherein the first preset quantity can root
It is configured according to actual conditions.
Further, when current file is split as the first fractionation file, the corresponding fixed-length word of current file can be passed through
The file content of symbol and current file, determines corresponding fractionation quantity, and according to the fractionation quantity, current file is torn open
It is divided into and splits the corresponding fractionation file of quantity.
File in parallel is split by described first by the downloading end to import in the corresponding database of the downloading end.
In this step, file in parallel is split for the first of fractionation by downloading end and imports the corresponding database of downloading end,
Multiple threads are used, while being conducted into database parallel.
Finally, corresponding file record is generated, specifically, the file status of current file is labeled as having imported.
The present embodiment need to judge the file type of file and import class when the file that will have been downloaded imports database
Correct file is smoothly imported database by type, improves the intelligence that file imports.
Further, it is based on the first, second embodiment of distributed document batch processing method of the present invention, proposes present invention distribution
Formula file batch processing method 3rd embodiment.
The 3rd embodiment of distributed document batch processing method and the first, second of distributed document batch processing method are implemented
The difference of example is that step S40 includes:
It is described to the corresponding transmitting-receiving allocation list of export by processing end acquisition, and based on described to export
Corresponding transmitting-receiving allocation list determines the init state to export；
Based on the init state, whether need to export to export by the way that processing end determination is described；
If desired, then by the processing end determine it is described to export in the local disk and in the cloud storage pair
The first catalogue answered, and by the second fractionation file for being split as the second preset quantity to export, and export to first
Catalogue；
Splitting file mergences for described second in first catalogue is to merge file, and be merged into the text to be exported
Corresponding second catalogue of part.
The present embodiment will need to treat export and carry out further when export exports to local disk and cloud storage
Judgement namely it is not all require to export to export, and only export it is qualified to export, and
In export when export, correctly export mode is selected, so that the export of file is more rapidly and convenient.
Each step will be described in detail below:
It is described to the corresponding transmitting-receiving allocation list of export by processing end acquisition, and based on described to export
Corresponding transmitting-receiving allocation list determines the init state to export.
In the present embodiment, it will first pass through processing end when export exports and obtain to the corresponding receipts of export
Send out allocation list, it is possible to understand that, each file is downloading files into local to beginning to a transmitting-receiving allocation list is had eventually
During file is either imported database or carries out processing generation to export to file, carries transmitting-receiving always and match
Set table.Transmitting-receiving allocation list includes the init state to export.It therefore, can be by obtaining the transmitting-receiving configuration to export
Table determines the init state to export.
Based on the init state, whether need to export to export by the way that processing end determination is described.
In the present embodiment, init state is specially export state, therefore, can be by reading to the initial of export
Change state, determine it is current whether need to export to export, i.e., init state is derived to lead to export needs
Out.
It is export state by init state specifically, reading the init state to export by processing end
It is determined as needing to export to export.It should be understood that processing end can be one either it is multiple, be multiple in processing end
In the case where, polling mechanism can also be used, poll reads the init state to export, and is export by init state
State, and be present treatment end processing be determined as needing to export to export.I.e. processing end is handled by current file,
During generating to export, the IP information of record processing end is also needed, to determine which waits exporting text by IP information
Part is the machine processing.
If desired, then by the processing end determine it is described to export in the local disk and in the cloud storage pair
The first catalogue answered, and by the second fractionation file for being split as the second preset quantity to export, and export to first
Catalogue.
In the present embodiment, it currently needs to export to export if determining by processing end, it is determined that export
In local disk and corresponding first catalogue in cloud storage, that is, determine to which which position export will be exported to, and will
To export be split as the second preset quantity second split file, wherein the second preset quantity can according to the actual situation into
Row setting will then be split as 7 fractionation files if preset quantity is 7 to export.
Further, can be by the corresponding fixed length character of export, and to the file content of export, determine
Corresponding fractionation quantity, and according to the fractionation quantity, it will be split as splitting the corresponding fractionation file of quantity to export.
Multi-threaded parallel export will started, by multiple fractionation files after export is split as multiple fractionation files
It exports to the first determining catalogue.
Further, the second fractionation file was exported to the step of first catalogue and includes:
If desired, it is determined that it is described to export the local disk in corresponding first catalogue of the cloud storage,
And determine that first catalogue whether there is Stub File；
If it does not exist, then by the processing end by it is described to export be split as the second preset quantity second split
File, and export to first catalogue；
If it exists, then the Stub File is deleted, and is split as second to export for described by the processing end
The second of preset quantity splits file, and exports to the first catalogue.
In this step, determine first catalogue with the presence or absence of Stub File, and if it exists, then to delete the residual text
Part, and exported the second fractionation file to first catalogue by processing end；If it does not exist, then direct by processing end
The second fractionation file is exported to first catalogue.
It should be understood that will be when export exports, it is possible that the failures such as delay machine or process interrupt, herein
In the case of kind, completely derived file, i.e. Stub File are certainly existed in corresponding catalogue, therefore, it is necessary to by Stub File
It deletes, then file export is split for second by processing end, avoid repeating to export, if there is no residual texts in corresponding catalogue
Part then directly splits file export for second by processing end.
Splitting file mergences for described second in first catalogue is to merge file, and be merged into the text to be exported
Corresponding second catalogue of part.
Second in the first catalogue is torn open after the second fractionation file declustering is exported to the first catalogue in the present embodiment
Single cent part merges into merging file, and is merged into the second catalogue, and concrete mode can be torn open that will be split as second to export
It when single cent part, is numbered to file is split, such as part1, part2, part3, exports by the second fractionation file to first
After catalogue, according to the number for splitting file, will split file mergences one by one is to merge file to be so far synchronized to export
In export all local disk and cloud storage.
The present embodiment will need to treat export and carry out further when export exports to local disk and cloud storage
Judgement namely it is not all require to export to export, and only export it is qualified to export, and
In export when export, correctly export mode is selected, so that the export of file is more rapidly and convenient.
It should be noted that in the above-described embodiments, if delay machine or process interrupt etc. occur for downloading end and/or processing end
Failure then records the corresponding failure file of the failure and the corresponding fault actions of failure file, and switch current downloading end and/
Or the corresponding spare end in processing end is continued to execute according to the failure file and fault actions of record.
The present invention also provides a kind of distributed document batch processing devices.Distributed document batch processing device device packet of the present invention
It includes:
Download module, for determining that the file download notifies corresponding downloading end when detecting file download notice,
And the file download is notified by corresponding file by the downloading end, the downloading end is downloaded to from corresponding cloud storage
Corresponding local disk；
Import modul, for reading the file content of the file, and base in the local disk by the downloading end
In the file content, the file is imported in the corresponding database of the downloading end by the downloading end；
Processing module, for handling the file in the database by the corresponding processing end of the database, with
It obtains to export；
Export module, for being exported to export to the local disk by the processing end by described, and will be described
The cloud storage is sent to export.
Further, the download module is also used to:
When detecting file download notice, determine that the file download notifies corresponding downloading end；
The file download is notified that file status is text to be downloaded in corresponding listed files by the downloading end
Part is downloaded to the corresponding local disk of the downloading end from corresponding cloud storage；
The file status of the file is changed in downloading or has been downloaded by the downloading end, and according to ought be above
Part state updates the listed files.
Further, the import modul is also used to:
The transmitting-receiving allocation list of the file is read in the local disk by the downloading end, and is matched based on the transmitting-receiving
It sets table and determines whether the file needs to import；
If desired, then it is based on the transmitting-receiving allocation list, file type belonging to the file is determined by the processing end,
And it is based on the file type, determine the lead-in type of the file；
Based on the lead-in type, the file is imported by the corresponding database of the downloading end by the downloading end
In.
Further, if the lead-in type is single thread importing, the import modul is also used to:
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and reads the file configuration of the file by the downloading end, with true
The fixed file header for whether needing to skip the file；
If desired, then by the downloading end that the file importing downloading end for skipping the file header is corresponding
In database.
Further, if the lead-in type is multithreading importing, the import modul is also used to:
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and is the first present count by the file declustering by the downloading end
The first of amount splits file；
File in parallel is split by described first by the downloading end to import in the corresponding database of the downloading end.
Further, the export module is also used to:
It is described to the corresponding transmitting-receiving allocation list of export by processing end acquisition, and based on described to export
Corresponding transmitting-receiving allocation list determines the init state to export；
Based on the init state, whether need to export to export by the way that processing end determination is described；
If desired, then by the processing end determine it is described to export in the local disk and in the cloud storage pair
The first catalogue answered, and by the second fractionation file for being split as the second preset quantity to export, and export to first
Catalogue；
Splitting file mergences for described second in first catalogue is to merge file, and be merged into the text to be exported
Corresponding second catalogue of part.
Further, the export module is also used to:
If desired, it is determined that it is described to export the local disk in corresponding first catalogue of the cloud storage,
And determine that first catalogue whether there is Stub File；
If it does not exist, then by the processing end by it is described to export be split as the second preset quantity second split
File, and export to first catalogue；
If it exists, then the Stub File is deleted, and is split as second to export for described by the processing end
The second of preset quantity splits file, and exports to first catalogue.
The present invention also provides a kind of computer readable storage mediums.
Distributed document batch program, the distributed document batch are stored on computer readable storage medium of the present invention
The step of distributed document batch processing method as described above is realized when processing routine is executed by processor.
Wherein, the distributed document batch program run on the processor, which is performed realized method, to join
According to each embodiment of distributed document batch processing method of the present invention, details are not described herein again.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or the system that include a series of elements not only include those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or system institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including being somebody's turn to do
There is also other identical elements in the process, method of element, article or system.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in one as described above
In storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that terminal device (it can be mobile phone,
Computer, server, air conditioner or network equipment etc.) execute method described in each embodiment of the present invention.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content, is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (16)
1. a kind of distributed document batch processing method, which is characterized in that the distributed document batch processing method includes following step
It is rapid:
When detecting file download notice, determine that the file download notifies corresponding downloading end, and pass through the downloading end
The file download is notified into corresponding file, the corresponding local disk of the downloading end is downloaded to from corresponding cloud storage；
The file content of the file is read in the local disk by the downloading end, and is based on the file content, is led to
The downloading end is crossed to import the file in the corresponding database of the downloading end；
The file in the database is handled by the corresponding processing end of the database, to obtain to export；
It is exported to export to the local disk by the processing end by described, and is sent to institute to export for described
State cloud storage.
2. distributed document batch processing method as described in claim 1, which is characterized in that described to detect that file download is logical
When knowing, determine that the file download notifies corresponding downloading end, and notify to correspond to by the file download by the downloading end
File, the step of corresponding local disk of the downloading end is downloaded to from corresponding cloud storage includes:
When detecting file download notice, determine that the file download notifies corresponding downloading end；
The file download is notified that file status is file to be downloaded in corresponding listed files by the downloading end, from
The corresponding local disk of the downloading end is downloaded in corresponding cloud storage；
The file status of the file is changed in downloading or has been downloaded by the downloading end, and according to current file shape
State updates the listed files.
3. distributed document batch processing method as described in claim 1, which is characterized in that it is described by the downloading end in institute
The file content for reading the file in local disk is stated, and the file content is based on by the downloading end and leads the file
The step entered in the corresponding database of the downloading end includes:
The transmitting-receiving allocation list of the file is read in the local disk by the downloading end, and is based on the transmitting-receiving allocation list
Determine whether the file needs to import；
If desired, then it is based on the transmitting-receiving allocation list, file type belonging to the file, and base are determined by the processing end
In the file type, the lead-in type of the file is determined；
Based on the lead-in type, the file is imported in the corresponding database of the downloading end by the downloading end.
4. distributed document batch processing method as claimed in claim 3, which is characterized in that if the lead-in type is single thread
It imports, then it is described to be based on the lead-in type, the file is imported by the corresponding data of the downloading end by the downloading end
Step in library includes:
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and reads the file configuration of the file by the downloading end, is with determination
The no file header for needing to skip the file；
If desired, then the file for skipping the file header is imported by the corresponding data of the downloading end by the downloading end
In library.
5. distributed document batch processing method as claimed in claim 3, which is characterized in that if the lead-in type is multithreading
It imports, then it is described to be based on the lead-in type, the file is imported by the corresponding data of the downloading end by the downloading end
Step in library includes:
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and is the first preset quantity by the file declustering by the downloading end
First splits file；
File in parallel is split by described first by the downloading end to import in the corresponding database of the downloading end.
6. distributed document batch processing method as described in any one in claim 1-5, which is characterized in that described to pass through the place
Reason end is exported described to export to the local disk, and by described the step of being sent to the cloud storage to export
Include:
It is obtained by the processing end described to the corresponding transmitting-receiving allocation list of export and corresponding to export based on described in
Transmitting-receiving allocation list determine the init state to export；
Based on the init state, whether need to export to export by the way that processing end determination is described；
If desired, then it is determined by the processing end described corresponding in the cloud storage in the local disk to export
First catalogue, and by the second fractionation file for being split as the second preset quantity to export, and export to the first catalogue；
Splitting file mergences for described second in first catalogue is to merge file, and be merged into described to export pair
The second catalogue answered.
7. distributed document batch processing method as claimed in claim 6, which is characterized in that it is described if desired, then by described
Processing end determine it is described to export the local disk in corresponding first catalogue of the cloud storage, and by described wait lead
File declustering is the second fractionation file of the second preset quantity out, and exports to the step of the first catalogue and include:
If desired, it is determined that it is described to export the local disk in corresponding first catalogue of the cloud storage, and really
Fixed first catalogue whether there is Stub File；
If it does not exist, then by the processing end that second fractionation for being split as the second preset quantity to export is literary
Part, and export to first catalogue；
If it exists, then it deletes the Stub File, and is split as second by described by the processing end to export and presets
The second of quantity splits file, and exports to first catalogue.
8. a kind of distributed document batch processing device, which is characterized in that the distributed document batch processing device includes:
Download module, for determining that the file download notifies corresponding downloading end, and lead to when detecting file download notice
It crosses the downloading end and the file download is notified into corresponding file, it is corresponding that the downloading end is downloaded to from corresponding cloud storage
Local disk；
Import modul for reading the file content of the file in the local disk by the downloading end, and is based on institute
File content is stated, is imported the file in the corresponding database of the downloading end by the downloading end；
Processing module, for handling the file in the database by the corresponding processing end of the database, to obtain
To export；
Export module, for being exported to export to the local disk by the processing end by described, and by described wait lead
File is sent to the cloud storage out.
9. distributed document batch processing device as claimed in claim 8, which is characterized in that the download module is also used to:
When detecting file download notice, determine that the file download notifies corresponding downloading end；
The file download is notified that file status is file to be downloaded in corresponding listed files by the downloading end, from
The corresponding local disk of the downloading end is downloaded in corresponding cloud storage；
The file status of the file is changed in downloading or has been downloaded by the downloading end, and according to current file shape
State updates the listed files.
10. distributed document batch processing device as claimed in claim 8, which is characterized in that the import modul is also used to:
The transmitting-receiving allocation list of the file is read in the local disk by the downloading end, and is based on the transmitting-receiving allocation list
Determine whether the file needs to import；
If desired, then it is based on the transmitting-receiving allocation list, file type belonging to the file, and base are determined by the processing end
In the file type, the lead-in type of the file is determined；
Based on the lead-in type, the file is imported in the corresponding database of the downloading end by the downloading end.
11. distributed document batch processing device as claimed in claim 10, which is characterized in that if the lead-in type is single line
Journey imports, then the import modul is also used to:
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and reads the file configuration of the file by the downloading end, is with determination
The no file header for needing to skip the file；
If desired, then the file for skipping the file header is imported by the corresponding data of the downloading end by the downloading end
In library.
12. distributed document batch processing device as claimed in claim 10, which is characterized in that if the lead-in type is multi-thread
Journey imports, then the import modul is also used to:
It determines in the database with the presence or absence of the corresponding failure record of the file；
If it exists, then the failure record is cleared up, and is the first preset quantity by the file declustering by the downloading end
First splits file；
File in parallel is split by described first by the downloading end to import in the corresponding database of the downloading end.
13. such as the described in any item distributed document batch processing devices of claim 8-12, which is characterized in that the export module
It is also used to:
It is obtained by the processing end described to the corresponding transmitting-receiving allocation list of export and corresponding to export based on described in
Transmitting-receiving allocation list determine the init state to export；
Based on the init state, whether need to export to export by the way that processing end determination is described；
If desired, then it is determined by the processing end described corresponding in the cloud storage in the local disk to export
First catalogue, and by the second fractionation file for being split as the second preset quantity to export, and export to the first catalogue；
Splitting file mergences for described second in first catalogue is to merge file, and be merged into described to export pair
The second catalogue answered.
14. distributed document batch processing device as claimed in claim 13, which is characterized in that the export module is also used to:
If desired, it is determined that it is described to export the local disk in corresponding first catalogue of the cloud storage, and really
Fixed first catalogue whether there is Stub File；
If it does not exist, then by the processing end that second fractionation for being split as the second preset quantity to export is literary
Part, and export to first catalogue；
If it exists, then it deletes the Stub File, and is split as second by described by the processing end to export and presets
The second of quantity splits file, and exports to first catalogue.
15. a kind of distributed document batch processing equipment, which is characterized in that the distributed document batch processing equipment includes: storage
Device, processor and it is stored in the distributed document batch program that can be run on the memory and on the processor, institute
State the distribution realized as described in any one of claims 1 to 7 when distributed document batch program is executed by the processor
The step of formula file batch processing method.
16. a kind of computer readable storage medium, which is characterized in that be stored with distribution on the computer readable storage medium
File batch program is realized when the distributed document batch program is executed by processor as any in claim 1 to 7
The step of distributed document batch processing method described in item.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910465966.5A CN110191182B (en) | 2019-05-30 | 2019-05-30 | Distributed file batch processing method, device, equipment and readable storage medium |
| PCT/CN2020/092139 WO2020238860A1 (en) | 2019-05-30 | 2020-05-25 | Distributed file batch processing method and apparatus, and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910465966.5A CN110191182B (en) | 2019-05-30 | 2019-05-30 | Distributed file batch processing method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110191182A true CN110191182A (en) | 2019-08-30 |
| CN110191182B CN110191182B (en) | 2023-04-21 |
Family
ID=67719137
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910465966.5A Active CN110191182B (en) | 2019-05-30 | 2019-05-30 | Distributed file batch processing method, device, equipment and readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110191182B (en) |
| WO (1) | WO2020238860A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020238860A1 (en) * | 2019-05-30 | 2020-12-03 | 深圳前海微众银行股份有限公司 | Distributed file batch processing method and apparatus, and readable storage medium |
| CN113722277A (en) * | 2020-05-25 | 2021-11-30 | 中兴通讯股份有限公司 | Data import method, device, service platform and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112597228B (en) * | 2020-12-26 | 2024-06-07 | 中国农业银行股份有限公司 | File processing method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307219A (en) * | 2011-03-18 | 2012-01-04 | 北京神州数码思特奇信息技术股份有限公司 | File access system, file uploading method and file downloading method |
| US20170024410A1 (en) * | 2015-07-24 | 2017-01-26 | Salesforce.Com, Inc. | Synchronize collaboration entity files |
| CN108830715A (en) * | 2018-05-30 | 2018-11-16 | 平安科技（深圳）有限公司 | Batch file part disk returning processing method and system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103379149A (en) * | 2012-04-19 | 2013-10-30 | 北京智慧风云科技有限公司 | Cloud service system providing function of processing files according to received commands |
| CN104580437A (en) * | 2014-12-30 | 2015-04-29 | 创新科存储技术（深圳）有限公司 | Cloud storage client and high-efficiency data access method thereof |
| CN105956481B (en) * | 2015-09-17 | 2019-05-10 | 中国银联股份有限公司 | A kind of data processing method and its device |
| CN107870814B (en) * | 2016-09-23 | 2022-02-22 | 伊姆西Ip控股有限责任公司 | Method and apparatus for content management batching |
| CN107967347B (en) * | 2017-12-07 | 2021-01-22 | 湖北三新文化传媒有限公司 | Batch data processing method, server, system and storage medium |
| CN110191182B (en) * | 2019-05-30 | 2023-04-21 | 深圳前海微众银行股份有限公司 | Distributed file batch processing method, device, equipment and readable storage medium |
-
2019
- 2019-05-30 CN CN201910465966.5A patent/CN110191182B/en active Active
-
2020
- 2020-05-25 WO PCT/CN2020/092139 patent/WO2020238860A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307219A (en) * | 2011-03-18 | 2012-01-04 | 北京神州数码思特奇信息技术股份有限公司 | File access system, file uploading method and file downloading method |
| US20170024410A1 (en) * | 2015-07-24 | 2017-01-26 | Salesforce.Com, Inc. | Synchronize collaboration entity files |
| CN108830715A (en) * | 2018-05-30 | 2018-11-16 | 平安科技（深圳）有限公司 | Batch file part disk returning processing method and system |
Non-Patent Citations (1)
| Title |
|---|
| 徐杨等: "基于SpringBatch框架的数据批量处理在决策分析系统中的应用", 《电子测量技术》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020238860A1 (en) * | 2019-05-30 | 2020-12-03 | 深圳前海微众银行股份有限公司 | Distributed file batch processing method and apparatus, and readable storage medium |
| CN113722277A (en) * | 2020-05-25 | 2021-11-30 | 中兴通讯股份有限公司 | Data import method, device, service platform and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110191182B (en) | 2023-04-21 |
| WO2020238860A1 (en) | 2020-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108647049B (en) | Configurable system, method, equipment and storage medium based on rule engine | |
| CN109034993A (en) | Account checking method, equipment, system and computer readable storage medium | |
| CN110191182A (en) | Distributed document batch processing method, device, equipment and readable storage medium storing program for executing | |
| CN107678907A (en) | Database business logic monitoring method, system and storage medium | |
| CN110347716A (en) | Daily record data processing method, device, terminal and storage medium | |
| CN111400011B (en) | Real-time task scheduling method, system, equipment and readable storage medium | |
| CN102752371B (en) | In client, realize method and the client of dodging screen | |
| CN107608798A (en) | A kind of method for processing business and equipment | |
| CN110084561A (en) | Breakpoint follow-up method, electronic device and readable storage medium storing program for executing | |
| CN106251508A (en) | The method for processing business of a kind of self-aided terminal and system | |
| CN106096034A (en) | application log management method and device | |
| CN110908837B (en) | Application program exception handling method and device, electronic equipment and storage medium | |
| CN110442819A (en) | Data processing method, device, storage medium and terminal | |
| CN110111162A (en) | Consume discount processing method, device, equipment and computer readable storage medium | |
| CN109039695B (en) | Service fault processing method, device and equipment | |
| CN115455957A (en) | User touch method, device, electronic equipment and computer readable storage medium | |
| CN102053862B (en) | Method and device for inspecting material files in instant messenger | |
| CN110019023A (en) | A kind of method for pushing, device and the equipment of organization information message | |
| CN112333262A (en) | Data updating prompting method and device, computer equipment and readable storage medium | |
| CN111767703A (en) | Form data acquisition method, device and system | |
| CN109714381A (en) | Information-pushing method, equipment, storage medium and device based on consumption tracking | |
| CN107678737A (en) | Method for processing business, device and self-help terminal equipment | |
| CN109657179A (en) | A kind of method for processing business, system and storage medium | |
| CN106789361A (en) | Event-handling method and device under a kind of cloud computing environment | |
| CN108345461A (en) | Using update method and device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |