CN110177154B - File interaction processing method, device and system - Google Patents

File interaction processing method, device and system Download PDF

Info

Publication number
CN110177154B
CN110177154B CN201910520528.4A CN201910520528A CN110177154B CN 110177154 B CN110177154 B CN 110177154B CN 201910520528 A CN201910520528 A CN 201910520528A CN 110177154 B CN110177154 B CN 110177154B
Authority
CN
China
Prior art keywords
file
identifier
hash value
server
fps
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910520528.4A
Other languages
Chinese (zh)
Other versions
CN110177154A (en
Inventor
于泽棋
江旻
王大飞
文海荣
旷波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
WeBank Co Ltd
Original Assignee
WeBank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by WeBank Co Ltd filed Critical WeBank Co Ltd
Priority to CN201910520528.4A priority Critical patent/CN110177154B/en
Publication of CN110177154A publication Critical patent/CN110177154A/en
Priority to PCT/CN2020/091929 priority patent/WO2020253465A1/en
Application granted granted Critical
Publication of CN110177154B publication Critical patent/CN110177154B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/12—Applying verification of the received information
    • H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/55—Push-based network services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions

Abstract

The invention discloses a file interactive processing method, a device and a system, which relate to the field of file transmission, wherein the method comprises the following steps: acquiring a first channel identifier, a first file identifier of a first file and a first hash value of the first file; the first channel identification is used for indicating a Secure File Transfer Protocol (SFTP) server corresponding to a first file receiver; downloading the first file from a file transfer protocol (FPS) server according to the first file identifier of the first file and the first hash value of the first file; and storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier. The method can be applied to financial technology (Fintech).

Description

File interaction processing method, device and system
Technical Field
The invention relates to the field of file transmission of financial technology (Fintech), in particular to a file interactive processing method, device and system.
Background
With the development of computer technology, more and more technologies (big data, distributed, Blockchain (Blockchain), artificial intelligence, etc.) are applied in the financial field, and the traditional financial industry is gradually changing to financial technology (Fintech). At present, in the field of financial science and technology, as the processing of information is more and more frequent, the cross analysis and the comprehensive application of various files are increased, and the interactive processing among the files is more and more important. In a File interaction development process in the prior art, two systems of a File interaction communicate with each other to align File formats of the two systems, a sender packs files, provides verification information such as File digests and a Message Digest Algorithm (MD 5), uploads the files to an SFTP server through a File Transfer Protocol (FTP)/Secure File Transfer Protocol (SFTP)/Hypertext Transfer Protocol (HTTP), and the like, and a receiver takes the files out of the SFTP server through the SFTP Protocol.
The interactive mode in the prior art not only needs to develop a set of complete file packaging and transmission module, but also cannot realize the separation of configuration and service, thereby increasing the development difficulty and the complexity of actual use.
Disclosure of Invention
The embodiment of the application provides a method, a device and a system for file interactive processing, which are used for providing a file interactive processing method.
The embodiment of the application provides a file interaction processing method, which specifically comprises the following steps:
acquiring a first channel identifier, a first file identifier of a first file and a first hash value of the first file; the first channel identification is used for indicating a Secure File Transfer Protocol (SFTP) server corresponding to a first file receiver;
downloading the first file from a file transfer protocol (FPS) server according to the first file identifier of the first file and the first hash value of the first file;
and storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier.
A possible implementation manner, where downloading the first file from a file transfer protocol FPS server according to the first file identifier of the first file and the first hash value of the first file includes:
acquiring a second hash value of the first file in the FPS server;
when the first hash value is equal to the second hash value;
and downloading the first file from a file transfer protocol (FPS) server according to the first file identifier.
A possible implementation manner, after the storing the first file to the SFTP server corresponding to the first channel identifier according to the first channel identifier, further includes:
feeding back a push completion message to the first file sender; the push completion message is used for notifying the first file sender that the first file is stored to the SFTP server corresponding to the first channel identifier.
An embodiment of the present application provides a file interaction processing system, which specifically includes:
the data warehouse is used for acquiring data, processing the data, packaging the processed data into a first file, and sending the first file to the FPS server;
the FPS server is used for storing the acquired first file, distributing a first file identifier to the first file and generating a first hash value of the first file; sending the first file identification and the first hash value to the data warehouse;
the data warehouse is used for acquiring the first file identifier and the first hash value, determining a first channel identifier of a first file receiver, and sending the first channel identifier, the first file identifier and the first hash value to a batch file interaction system CAP;
the CAP system is used for acquiring the first channel identifier, the first file identifier and the first hash value; the first channel identification is used for indicating an SFTP server corresponding to the first file receiver; downloading the first file from an FPS server according to the first file identifier and the first hash value; storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier;
the SFTP server is used for storing the first file.
An embodiment of the present application provides a file interaction processing system, which specifically includes:
the batch file interaction system CAP is used for acquiring a second channel identifier; the second channel identification is used for indicating an SFTP server corresponding to a second file sender; acquiring a second file from the SFTP server according to the second channel identifier; sending the second file to an FPS server;
the FPS server is used for storing the obtained second file, distributing a second file identifier to the second file and generating a third hash value of the second file; sending the second file identifier and the third hash value to the CAP system;
the CAP system is used for transmitting the second file identifier and the third hash value to a message communication queue port;
a file receiving system, configured to monitor the message communication queue port, and obtain the second file identifier and the third hash value from the message communication queue port; and downloading the second file from the FPS server according to the second file identifier and the third hash value.
An embodiment of the present application provides a file interaction processing apparatus, which specifically includes:
the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring a first channel identifier, a first file identifier of a first file and a first hash value of the first file; the first channel identification is used for indicating a Secure File Transfer Protocol (SFTP) server corresponding to a first file receiver;
the processing unit is used for downloading the first file from a file transfer protocol (FPS) server according to a first file identifier of the first file and a first hash value of the first file; and storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier.
In one possible implementation, the processing unit is further configured to:
acquiring a second hash value of the first file in the FPS server;
when the first hash value is equal to the second hash value;
and downloading the first file from a file transfer protocol (FPS) server according to the first file identifier.
In one possible implementation manner, the processing unit is further configured to:
feeding back a push completion message to the first file sender; the push completion message is used for notifying the first file sender that the first file is stored to the SFTP server corresponding to the first channel identifier.
Embodiments of the present application provide a computer device comprising a program or instructions that, when executed, cause a computer to perform the method of any of the above possible designs.
Embodiments of the present application provide a storage medium containing a program or instructions that, when executed, cause a computer to perform the method of any of the above possible designs.
The file interactive processing method, the device and the system provided by the invention have the following beneficial effects: by mapping the environment information into the channel identifier, a file developer can realize the separation of configuration and service without paying attention to the specific environment information, thereby reducing the complexity of actual use.
Drawings
FIG. 1 is a schematic diagram of a technical architecture of a file interaction processing method in the prior art;
fig. 2 is a schematic diagram of a technical architecture of file pushing in a file interaction processing method according to an embodiment of the present application;
fig. 3 is a schematic flowchart of file pushing in a file interaction processing method according to an embodiment of the present application;
fig. 4 is a schematic diagram of a technical architecture for pulling a file in a file interaction processing method according to an embodiment of the present application;
fig. 5 is a schematic flowchart of file pulling in a file interaction processing method according to an embodiment of the present application.
Fig. 6 is a schematic flowchart of an alternative method for pushing a file in a file interaction processing method according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions, the technical solutions will be described in detail below with reference to the drawings and the specific embodiments of the specification, and it should be understood that the specific features in the embodiments and examples of the present application are detailed descriptions of the technical solutions of the present application, but not limitations of the technical solutions of the present application, and the technical features in the embodiments and examples of the present application may be combined with each other without conflict.
With the development of computer technology, more and more technologies (big data, distributed, Blockchain (Blockchain), artificial intelligence, etc.) are applied in the financial field, and the traditional financial industry is gradually changing to financial technology (Fintech). In the prior art, in the process of developing business (such as loan business, deposit business and the like of a bank) of a financial institution (a banking institution, an insurance institution or a security institution), the development flow of file interaction mainly utilizes a program to package files to provide verification information such as file summaries and MD5, and the like, and carries out a file transmission process through protocols such as FTP/SFTP/HTTP and the like.
As shown in fig. 1, which is a schematic diagram of a technical architecture of a file interaction processing method in the prior art, when a sender a transmits a file to a receiver B, the two parties communicate and align file formats, then the sender a packages the file to provide an MD5 file and a digest file, then the sender a uploads the file to an SFTP server, and the receiver B takes the file out of the SFTP server through an SFTP protocol.
The file interactive processing method needs to develop a set of complete file packaging and transmission modules, and needs to develop uploading and downloading functions of protocols such as FTP (file transfer protocol) and the like, and even needs to provide a complete file monitoring function. Meanwhile, the control capability of the file content is not enough, and the flexible configuration of the file data and the shielding function of sensitive data are difficult to realize. The separation of configuration and service cannot be realized, so that the environment information and the application service are mixed together, thereby increasing the complexity of practical use.
In the face of increasingly important file interaction processes, the embodiment of the application provides a file interaction processing method, which can simplify the development process of file interaction and reduce the complexity of actual use, and the specific technical scheme is as follows.
Fig. 2 is a schematic diagram of a technical architecture of file pushing in a file interaction processing method according to an embodiment of the present application. The technical architecture of file pushing shown in fig. 2 includes an application system 201, an application 202, a Distributed database (TDSQL) 203, a big data warehouse 204, a data warehouse 205, a data file 206, a file interaction system 207, a batch file interaction system (CAP)208, a file storage system (FPS)209, an SFTP server group 210, an SFTP server 1211, and an SFTP server 2212.
The application system 201 includes an application 202 and a distributed database 203.
The application 202 is connected to the distributed database 203, and is configured to store data to be transmitted in the application 202 in the distributed database 203.
The distributed database 203 is connected to the application 202 and the data warehouse 205, and is used for pushing data to be transmitted in the application 202 to the data warehouse 205 through an extract, transform and loading (ETL) process to generate a data table.
The big data repository 204, which contains a data repository 205 and data files 206, is coupled to a file interaction system 207, and more specifically, to a batch file interaction system 208 and a file storage system 209.
The data warehouse 205 is connected to the distributed database 203, the data file 206 and the file interaction system 207, and is used for processing the generated data table according to the business level requirement.
Optionally, the field of the processed data is sent to the batch file interaction system 208, and the batch file interaction system 208 receives the field list configured by itself, and according to the field list, the field of the processed data is screened, and the data table corresponding to the screened field is packaged to generate the data file 206.
Optionally, the data file 206 is sent to the file storage system 209, and a file identifier and a hash value corresponding to the data file 206 returned by the file storage system 209 are received.
Optionally, on the basis of the file identifier and the hash value corresponding to the data file 206, a channel identifier of a specific server in the SFTP server group 210 to be transmitted is added, and these pieces of information are sent to the batch file interaction system 208 together, where the channel identifier includes information such as an SFTP address, an account number, and a password of a partner.
The data file 206 is a package file connected to the data repository 205 for later generation in the flow.
The file interaction system 207, which includes a batch file interaction system 208 and a file storage system 209, is coupled to the big data repository 204.
The batch file interaction system 208 is connected to the big data warehouse 204, the file storage system 209, and the SFTP server group 210, and is configured to receive fields of processed data sent by the data warehouse 205 in the big data warehouse 204, feed back the fields to the data warehouse 205 according to a field list configured by the batch file interaction system, filter the fields of the processed data by the data warehouse 205 according to the field list, and package data tables corresponding to the filtered fields to generate the data file 206.
Optionally, a file identifier and a hash value corresponding to the data file 206 sent by the data warehouse 205, and a channel identifier of a specific server in the SFTP server group 210 are received.
Optionally, the data file is downloaded from file storage system 209 based on the file identification and the hash value.
Optionally, after downloading the data file, the specific information in the channel identifier is read, the specific SFTP server in the SFTP server group 210 is determined according to the specific information, and then the data file is pushed to the server.
Optionally, after the data file is pushed to the SFTP server group 210, the push completion message is fed back to the application system 201.
The file storage system 209 is connected to the big data warehouse 204 and the batch file interaction system 208, and is configured to receive and store the data file 206 in the big data warehouse 204, and then send a file identifier and a hash value corresponding to the data file 206.
Optionally, the file identifier and the hash value sent by the batch file interaction system 208 are received.
The SFTP server group 210 includes an SFTP server 1211, an SFTP server 2212, and the like. Is connected with the batch file interaction system 208, and is configured to receive the data file 206 pushed by the batch file interaction system 208, and store the data file to the corresponding specific SFTP server.
When there is an internal file to be transferred to the outside, a specific pushing process will be described in detail below.
As shown in fig. 3, a flowchart of file pushing in a file interaction processing method provided in the embodiment of the present application is schematically shown, and specific steps will be described in detail below.
Step 301, the data warehouse acquires data and processes the data.
Specifically, the data of the first file is generally stored in a distributed database of the application system, when the first file needs to be transmitted, a data table interface where the data of the first file in the distributed database is located is determined, the data table is pushed to a data warehouse through an ETL process, and the data table is processed in the data warehouse according to business level requirements.
And then the data warehouse sends the processed data fields to the batch file interaction system, the batch file interaction system returns the field list configured by the batch file interaction system to the data warehouse, and the data warehouse screens the processed data fields according to the field list and generates a data table corresponding to the screened fields.
The batch file interactive system stores fields of each channel and each service, and can control a field list required to be contained in the file packaging process through the field information. For example, a data table structure, such as a data warehouse to be transmitted, has five fields, including: ID. NAME, ADDRESS, ID _ NO, and LAST _ MODIFIED _ data, where ID is a sequence number field in the data table structure, NAME is a NAME field in the data table structure, ADDRESS is an ADDRESS field in the data table structure, ID _ NO is an identity number field in the data table structure, and LAST _ MODIFIED _ data is a LAST MODIFIED time field in the data table structure. In the process of file transmission, the file interaction process only needs ID, NAME and LAST _ MODIFIED _ data, and then ID, NAME and LAST _ MODIFIED _ data can be configured in the batch file interaction system as a screened field list and returned to the data warehouse. The batch file interaction system realizes the functions of adjusting data fields, screening sensitive fields and the like in file interaction processing in the flexibly configurable mode, thereby ensuring the controllability of data.
Step 302: and packaging the processed data into a first file, and sending the first file to the FPS server.
Specifically, the data warehouse packs the data table corresponding to the screened field into a first file, and sends the first file to the FPS server.
Because the management mechanism of the internal file is uniformly stored in the FPS server and then uploaded to the SFTP server, the following contents are how to acquire the first file from the FPS server and then upload the first file to the SFTP server. The detailed description is as follows.
Step 303: the FPS server stores the acquired first file, allocates a first file identifier to the first file, and generates a first hash value of the first file; and sending the first file identification and the first hash value to the data warehouse.
Specifically, the FPS server receives and stores the first file, allocates a first file identifier to the first file, and generates a first hash value of the first file. The first file identification and the first hash value are then sent to the data warehouse.
The FPS server herein has two functions, including: 1. receiving and storing a first file; 2. and returning the first file identifier and the first hash value. In the point 2 function, the first file identifier is a 64-bit character string and has uniqueness; the first hash value is a 32-bit character string, records the first hash value of the first file, and if the file is slightly changed, for example, a bit is changed, the first hash value is changed, so that whether the first file is damaged or not can be identified.
Step 304: the data warehouse acquires the first file identifier and the first hash value, determines a first channel identifier of a first file receiver, and sends the first channel identifier, the first file identifier and the first hash value to a batch file interaction system CAP.
Specifically, on the basis of receiving a first file identifier and a first hash value sent by the FPS server, the data warehouse determines, according to a transmission requirement of a first file, a first channel identifier of a specific server in the SFTP server group corresponding to the receiving party, and sends the first channel identifier, the first file identifier, and the first hash value together to the batch file interaction system CAP.
Step 305: and the CAP system acquires the first channel identifier, the first file identifier and the first hash value.
The first channel identification is used for indicating an SFTP server corresponding to the first file receiver; downloading the first file from an FPS server according to the first file identifier and the first hash value; and storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier.
Specifically, the CAP system receives a first channel identifier, a first file identifier and a first hash value, and downloads a first file from the FPS server according to the first file identifier and the first hash value; and acquiring the SFTP server corresponding to the first file receiver according to the first channel identifier, and storing the first file to the SFTP server corresponding to the first channel identifier.
By the method, the environmental information in the traditional file interaction process is mapped into the channel identifier, so that a developer of the file can decouple the problem of mixing the application system and the environmental information without paying attention to the specific environmental information, namely the condition of a receiver transmitting the first file; meanwhile, unified management of the environmental information is realized, and timely change can be realized when the environmental information changes.
More specifically, in step 305, the details of downloading the first file from the FPS server according to the first file identifier and the first hash value are described as follows.
Acquiring a second hash value of the first file in the FPS server;
when the first hash value is equal to the second hash value;
and downloading the first file from a file transfer protocol (FPS) server according to the first file identifier.
Optionally, a second hash value of the first file in the FPS server is obtained;
and downloading the first file from a file transfer protocol (FPS) server according to the first file identifier.
When the first hash value is equal to the second hash value;
and confirming that the first file is intact, otherwise, indicating that the file is damaged due to network conditions and other problems in the downloading process, and needing to download the first file again.
And step 306, the SFTP server is used for storing the first file.
Specifically, the SFTP server receives and stores the first file, waiting for the recipient to retrieve it from.
Step 307: feeding back a push completion message to the first file sender; the push completion message is used for notifying the first file sender that the first file is stored to the SFTP server corresponding to the first channel identifier.
Specifically, after step 306 is completed, the push completion message is fed back to the first file sender. The push completion message informs the first file sender that the first file is stored in the SFTP server corresponding to the first channel identifier, so that the sender can know the processing state of the first file, the first file is monitored, and the closed-loop management of the interactive process is ensured.
The above detailed step descriptions are explained from the perspective of internal files to be transmitted to an external system, and the present application provides a method for file interaction processing, so the following is explained from the perspective of acquiring external files for use by the internal system.
As shown in fig. 4, fig. 4 is a schematic diagram of a technical architecture of a file pull in a file interaction processing method according to an embodiment of the present application. The technical architecture of file pulling shown in fig. 4 includes a scheduling system 401, a file interaction system 207, a bulk file interaction system (CAP)208, a file storage system (FPS)209, an SFTP server farm 210, an SFTP server 1211, an SFTP server 2212, and a receiving file system 402.
The scheduling system 401 is connected to the batch file interaction system 208, and is configured to send the channel identifier of the sender to the batch file interaction system 208 within a specified configuration time.
Optionally, after the file is successfully pulled, a pull success message fed back by the batch file interaction system 208 is received.
The file interaction system 207 comprises a bulk file interaction system 208 and a file storage system 209.
The batch file interaction system 208 is connected with the scheduling system 401, the file storage system 209, the SFTP server group 210 and the file receiving system 402, and is configured to receive a channel identifier of a sender sent by the scheduling system 401, read specific information in the channel identifier, determine a position of a file in the SFTP server group 210, and obtain the file uploaded by the sender.
Optionally, since the general file is transmitted in a compressed package form, the file package can be unpacked according to the corresponding format through configuration, so as to obtain the file.
Optionally, the file is uploaded to the file storage system 209, and a file identifier and a hash value returned by the file storage system 209 are received.
Optionally, the file identifier and the hash value are transmitted to a message communication queue port configured in the pull channel management.
Optionally, after the file is successfully pulled, a pulling success message is fed back to the scheduling system 401.
The file storage system 209 is connected to the batch file interaction system 208 and the receiving file system 202, and is configured to receive and store the file uploaded by the batch file interaction system 208, and send the file identifier and the hash value to the batch file interaction system 208.
The SFTP server group 210 includes an SFTP server 1211, an SFTP server 2212, and the like, and is connected to the batch file interaction system 208, and is configured to receive a search of the batch file interaction system 208 on the SFTP server group 210, and acquire a file uploaded by a sender from the search.
The receiving file system 402 is connected to the batch file interaction system 208 and the file storage system 209, and is configured to monitor a file identifier and a hash value in a message communication queue port configured by the batch file interaction system, and call a Software Development Kit (SDK) of the file storage system to download a file after receiving the file identifier and the hash value.
The specific pulling process when an external file needs to be obtained for use by the internal system is described in detail below.
As shown in fig. 5, a flowchart of file pulling in a file interaction processing method provided in the embodiment of the present application is shown, and specific steps will be described in detail below.
Step 501: the batch file interaction system CAP acquires a second channel identifier; the second channel identification is used for indicating an SFTP server corresponding to a second file sender; acquiring a second file from the SFTP server according to the second channel identifier; sending the second file to an FPS server;
specifically, before step 501, the sender uploads the second file to the SFTP server, the developer configures the pull time and the pull frequency of the second file uploaded by the sender in the scheduling platform, and the scheduling platform runs the pull script within the configured pull time and sends the second channel identifier of the sender to the CAP system.
And after acquiring the second channel identifier, the CAP system determines an SFTP server corresponding to a second file sender according to the second channel identifier, and then acquires a second file uploaded by the sender. Since the general file is transmitted in a compressed packet form, the CAP system obtains the second file by configuring and unpacking the second file according to the corresponding format. The second file is then uploaded to the FPS server.
The management mechanism of the internal files is uniformly stored in the FPS server, and then the receiving file system downloads the corresponding files from the file storage system according to the file identification and the hash value as required. What follows is how the receiving file system retrieves the second file from the FPS server. The detailed description is as follows.
Step 502: the FPS server stores the acquired second file, allocates a second file identifier to the second file, and generates a third hash value of the second file; and sending the second file identifier and the third hash value to the CAP system.
Specifically, the FPS server receives and stores the second file, assigns a second file identifier to the second file, generates a third hash value of the second file, and returns the second file identifier and the third hash value to the CAP system.
Step 503: and the CAP system transmits the second file identifier and the third hash value to a message communication queue port.
Step 504: monitoring the message communication queue port by a receiving file system, and acquiring the second file identifier and the third hash value from the message communication queue port; and downloading the second file from the FPS server according to the second file identifier and the third hash value.
Specifically, a fourth hash value of a second file in the FPS server is obtained;
when the third hash value is equal to the fourth hash value;
and downloading the second file from a file transfer protocol (FPS) server according to the second file identifier.
Optionally, a fourth hash value of the second file in the FPS server is obtained;
and downloading the second file from a file transfer protocol (FPS) server according to the second file identifier.
When the third hash value is equal to the fourth hash value;
and confirming that the second file is intact, otherwise, indicating that the file is damaged due to network conditions and other problems in the downloading process, and needing to download the second file again.
More specifically, after step 504, the CAP system feeds back the pull success message to the scheduling system, so as to form closed-loop management.
Fig. 6 is a schematic flow chart of an alternative method for file pushing in another file interaction processing method, as shown in the figure, in the alternative, java SDK may be used to push data instead of ETL process, that is, the first file does not need to be pushed to the data warehouse through the ETL process, but the data of the first file of the sender and the first channel identifier of the first file receiver are introduced into the SDK through a third-party java SDK package of the CAP system, and the subsequent process is the same as the schematic flow chart of file pushing in fig. 3. In this way, the link of pushing to the data warehouse through the ETL process in fig. 3 can be simplified.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (10)

1. A file interactive processing method is characterized by comprising the following steps:
acquiring a first channel identifier, a first file identifier of a first file and a first hash value of the first file; the first channel identification is used for indicating a Secure File Transfer Protocol (SFTP) server corresponding to a first file receiver;
downloading the first file from a file storage system (FPS) server according to a first file identifier of the first file and a first hash value of the first file;
and storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier.
2. The method of claim 1, wherein said downloading said first file from a file storage system (FPS) server based on a first file identification of said first file and a first hash value of said first file comprises:
acquiring a second hash value of the first file in the FPS server;
and when the first hash value is equal to the second hash value, downloading the first file from a file storage system (FPS) server according to the first file identifier.
3. The method of claim 1, wherein after storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier, further comprising:
feeding back the push completion message to the first file sender; the push completion message is used for notifying the first file sender that the first file is stored to the SFTP server corresponding to the first channel identifier.
4. A file interaction processing system, comprising:
the data warehouse is used for acquiring data, processing the data, packaging the processed data into a first file, and sending the first file to the FPS server;
the FPS server is used for storing the acquired first file, distributing a first file identifier to the first file and generating a first hash value of the first file; sending the first file identification and the first hash value to the data warehouse;
the data warehouse is used for acquiring the first file identifier and the first hash value, determining a first channel identifier of a first file receiver, and sending the first channel identifier, the first file identifier and the first hash value to a batch file interaction system CAP;
the CAP system is used for acquiring the first channel identifier, the first file identifier and the first hash value; the first channel identification is used for indicating an SFTP server corresponding to the first file receiver; downloading the first file from an FPS server according to the first file identifier and the first hash value; storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier;
the SFTP server is used for storing the first file.
5. A file interaction processing system, comprising:
the batch file interaction system CAP is used for acquiring a second channel identifier; the second channel identification is used for indicating an SFTP server corresponding to a second file sender; acquiring a second file from the SFTP server according to the second channel identifier; sending the second file to an FPS server;
the FPS server is used for storing the obtained second file, distributing a second file identifier to the second file and generating a third hash value of the second file; sending the second file identifier and the third hash value to the CAP system;
the CAP system is used for transmitting the second file identifier and the third hash value to a message communication queue port;
a file receiving system, configured to monitor the message communication queue port, and obtain the second file identifier and the third hash value from the message communication queue port; and downloading the second file from the FPS server according to the second file identifier and the third hash value.
6. A file interaction processing apparatus, comprising:
the communication unit is used for acquiring a first channel identifier, a first file identifier of a first file and a first hash value of the first file; the first channel identification is used for indicating a Secure File Transfer Protocol (SFTP) server corresponding to a first file receiver;
the processing unit is used for downloading the first file from a file storage system (FPS) server according to a first file identifier of the first file and a first hash value of the first file; and storing the first file to an SFTP server corresponding to the first channel identifier according to the first channel identifier.
7. The apparatus as recited in claim 6, said processing unit to further:
acquiring a second hash value of the first file in the FPS server;
and when the first hash value is equal to the second hash value, downloading the first file from a file storage system (FPS) server according to the first file identifier.
8. The apparatus of claim 6, wherein the communication unit is further to:
feeding back the push completion message to the first file sender; the push completion message is used for notifying the first file sender that the first file is stored to the SFTP server corresponding to the first channel identifier.
9. A computer device, comprising:
a memory for storing a computer program;
a processor for calling a computer program stored in said memory, for executing the method according to any one of claims 1-3 in accordance with the obtained program.
10. A computer-readable storage medium having stored thereon computer-executable instructions for causing a computer to perform the method of any one of claims 1-3.
CN201910520528.4A 2019-06-17 2019-06-17 File interaction processing method, device and system Active CN110177154B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201910520528.4A CN110177154B (en) 2019-06-17 2019-06-17 File interaction processing method, device and system
PCT/CN2020/091929 WO2020253465A1 (en) 2019-06-17 2020-05-22 File interaction processing method, apparatus, and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910520528.4A CN110177154B (en) 2019-06-17 2019-06-17 File interaction processing method, device and system

Publications (2)

Publication Number Publication Date
CN110177154A CN110177154A (en) 2019-08-27
CN110177154B true CN110177154B (en) 2021-07-02

Family

ID=67698352

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910520528.4A Active CN110177154B (en) 2019-06-17 2019-06-17 File interaction processing method, device and system

Country Status (2)

Country Link
CN (1) CN110177154B (en)
WO (1) WO2020253465A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110177154B (en) * 2019-06-17 2021-07-02 深圳前海微众银行股份有限公司 File interaction processing method, device and system
CN110597764A (en) * 2019-10-10 2019-12-20 深圳前海微众银行股份有限公司 File management method and device
CN113726830B (en) * 2020-05-25 2023-09-12 网联清算有限公司 Message identifier generation method and device
CN113037710A (en) * 2021-02-03 2021-06-25 南京物通物语科技有限公司 Processing method for big data transmission for industrial Internet of things
CN114945020B (en) * 2022-05-17 2023-08-08 宝信软件(武汉)有限公司 Cloud edge cooperative data access method and system based on cloud platform

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102231742A (en) * 2011-06-27 2011-11-02 中国建设银行股份有限公司 File uploading and downloading methods, system and related equipment based on SFTP (Secure File Transfer Protocol)
CN104184740A (en) * 2014-09-04 2014-12-03 中电长城网际系统应用有限公司 Credible transmission method, credible third party and credible transmission system
CN108984560A (en) * 2017-06-01 2018-12-11 杭州海康威视数字技术股份有限公司 File memory method and device

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9419801B2 (en) * 2009-05-12 2016-08-16 Infrascale Inc. System and method for transmitting needed portions of a data file between networked computers
CN102497366A (en) * 2011-12-09 2012-06-13 常州英贝通信息技术有限公司 Method for securely transmitting U disk file on counter of financial network and file transmission system
US10069817B2 (en) * 2015-09-28 2018-09-04 Mastercard International Incorporated Secure file transfer with secure SSH key authentication
CN106162212B (en) * 2016-07-01 2019-10-15 江苏省广播电视集团有限公司 A kind of news content Field Site Production System realized based on cloud computing
CN109240726B (en) * 2018-09-30 2022-04-22 湖北华联博远科技有限公司 Software updating method and system
CN109639819B (en) * 2018-12-28 2022-01-14 腾讯科技（深圳）有限公司 File transmission method, client, server and system
CN110177154B (en) * 2019-06-17 2021-07-02 深圳前海微众银行股份有限公司 File interaction processing method, device and system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102231742A (en) * 2011-06-27 2011-11-02 中国建设银行股份有限公司 File uploading and downloading methods, system and related equipment based on SFTP (Secure File Transfer Protocol)
CN104184740A (en) * 2014-09-04 2014-12-03 中电长城网际系统应用有限公司 Credible transmission method, credible third party and credible transmission system
CN108984560A (en) * 2017-06-01 2018-12-11 杭州海康威视数字技术股份有限公司 File memory method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于SFTP的GNSS数据采集软件的设计与实现;黄经国等;《震灾防御技术》;20170615(第02期);全文 *

Also Published As

Publication number Publication date
WO2020253465A1 (en) 2020-12-24
CN110177154A (en) 2019-08-27

Similar Documents

Publication Publication Date Title
CN110177154B (en) File interaction processing method, device and system
US8819698B2 (en) Cross-platform web-based native device feature access
CN104754062B (en) Information popularization method, server and user terminal based on internet
CN104462562B (en) Data migration system and method based on data warehouse automation
CN112052013A (en) Software package generation method and device, storage medium and electronic device
WO2017124736A1 (en) Method, device and system for transmitting upgrade abnormality information
CN107977260B (en) Task submitting method and device
CN111339471B (en) Method and device for dynamically loading page buried point data
CN111367685B (en) Interface calling method and device, computer equipment and storage medium
CN110532182B (en) Automatic testing method and device for virtualization platform
US20040139147A1 (en) System and method for the dynamic distribution of data and/or services
CN109343970B (en) Application program-based operation method and device, electronic equipment and computer medium
CN103646015B (en) Transmission, the method and system for receiving and transmitting XML message
CN111736844B (en) Database cloud service standard interface and implementation method
CN105915613A (en) Resource supply method and device based on cloud services
CN116048533B (en) Implementation method and system for achieving dependency isolation in Flink task running
CN110008086B (en) Log generation method and device and client
CN110417860A (en) File transfer management method, apparatus, equipment and storage medium
CN113296987B (en) Interface calling method and device for calling module, computer equipment and storage medium
CN114816785A (en) Service data processing method and device, electronic equipment and storage medium
CN110007937B (en) System updating method and system
CN113254448A (en) Data recording method and system
CN112153148A (en) Message accumulation processing method and device based on message receiving end
CN110430279A (en) File download control method and device
CN114765606B (en) Container mirror image transmission method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant