CN117675840A

CN117675840A - File synchronization method and device, electronic equipment and storage medium

Info

Publication number: CN117675840A
Application number: CN202311631586.7A
Authority: CN
Inventors: 褚占峰; 王心怡; 周鹏程; 张鹏; 张强强
Original assignee: Hangzhou Dt Dream Technology Co Ltd
Current assignee: Hangzhou Dt Dream Technology Co Ltd
Priority date: 2023-11-30
Filing date: 2023-11-30
Publication date: 2024-03-08

Abstract

The specification provides a file synchronization method, a file synchronization device, electronic equipment and a storage medium. The method comprises the following steps: after detecting file updating, reading a first file to be synchronized; generating a first storage path in a transit file server for a first file; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file; storing the first file to a first storage path in a transit file server, and recording a first identification, the first storage path and a synchronous operation type of the first file, so that a downstream file exchange platform acquires the first file according to the first storage path and carries out synchronous operation on the first file according to the first identification and the synchronous operation type of the first file; the first identifier represents a storage path of the first file at the downstream business file server. According to the scheme, each file synchronization operation is distinguished from other file synchronization operations, and errors caused by different file synchronization operations of an upstream exchange platform and a downstream exchange platform are avoided.

Description

File synchronization method and device, electronic equipment and storage medium

Technical Field

One or more embodiments of the present disclosure relate to the field of file synchronization technologies, and in particular, to a method, an apparatus, an electronic device, and a storage medium for file synchronization.

Background

A cascading file exchange platform is a network platform that allows users to directly share files, and performs file transfer and sharing through a plurality of interconnected nodes. For example, the file exchange platform in the section and the file exchange platform in the province realize reporting of the data in the province to the section through cross-level and cross-platform cooperative linkage, and the data in the section is issued to the province.

The cascade file exchange platform comprises an upstream file exchange platform, a downstream file exchange platform, a transfer file server and the like, and when the upstream file exchange platform synchronizes the updated file in the upstream business file server to the downstream file exchange platform, the updated file is stored to the transfer file server for use by the downstream file exchange platform.

It can be seen that in the above-illustrated embodiment, when the upstream file exchange platform and the downstream file exchange platform perform different operations on the same file in the transit file server, an error may be generated due to an operation conflict.

Disclosure of Invention

The application provides a file synchronization method, which comprises the following steps:

after detecting that an upstream service file server generates file updating, reading a first file to be synchronized from the upstream service file server;

Generating a first storage path in the transit file server for the first file; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

storing the first file to a first storage path in the transit file server, and recording a first identification of the first file, the first storage path and a synchronous operation type for the first file, so that a downstream file exchange platform acquires the first file according to the recorded first storage path, and synchronously operates the first file according to the first identification of the first file and the synchronous operation type for the first file;

wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

Optionally, generating a first storage path in the transit file server for the first file includes:

adding a suffix to the file name of the first file to generate a temporary file name of the first file;

and generating a first storage path in the transit file server for the first file based on the temporary file name of the first file.

Optionally, the first identifier is an original file name of the first file; the storing the first file in a first storage path in the transit file server, and recording a first identifier of the first file, the first storage path and a synchronization operation type for the first file, includes:

storing the first file to a first storage path in the transit file server, recording an original file name of the first file, the first storage path and a synchronous operation type for the first file, so that the downstream file exchange platform obtains the first file according to the recorded first storage path, replaces the storage path of the first file with the storage path of the first file of a downstream service file server according to the recorded original file name of the first file, and performs synchronous operation on the first file according to the synchronous operation type for the first file.

Optionally, the synchronization operation types include:

file insert type, file update type, file delete type.

Optionally, the method further comprises:

and recording the update time of the first file so that the downstream file exchange platform determines that the first file exists in the transit file server when the update time is polled to have update.

Optionally, the method further comprises:

reading a second file to be synchronized from an upstream service file server after detecting that a new file is inserted into the upstream service file server;

and storing the second file into the transit file server, and recording a storage path of a downstream service file server and a file insertion type for the second file, so that a downstream file exchange platform synchronously operates the second file according to the storage path of the downstream service file server and the file insertion type for the second file.

Optionally, the method further comprises:

reading a third file to be synchronized from an upstream service file server after detecting that the upstream service file server is subjected to file deletion;

and recording the storage path of the third file at the downstream service file server and the file deletion type aiming at the third file, so that the downstream file exchange platform synchronously operates the third file according to the storage path of the third file at the downstream service file server and the file deletion type aiming at the third file.

receiving a first file to be synchronized and a first storage path of the first file, which are sent by an upstream file exchange platform; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

storing the first file to the first storage path, and storing a first identifier of the first file, the first storage path and a synchronous operation type for the first file, so that a downstream file exchange platform obtains the first file according to the recorded first storage path, and performs synchronous operation on the first file according to the first identifier of the first file and the synchronous operation type for the first file;

after detecting that an upstream business file server generates file updating, reading a first identification, a first storage path and a synchronous operation type aiming at a first file to be synchronized from a pre-recorded metadata table; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

Acquiring the first file according to the first storage path, and carrying out synchronous operation on the first file according to a first identifier of the first file and a synchronous operation type aiming at the first file;

The application also provides a file synchronization device, which comprises:

the reading unit is used for reading the first file to be synchronized from the upstream business file server after detecting that the upstream business file server is updated;

the generation unit is used for generating a first storage path in the transit file server for the first file; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

the storage unit is used for storing the first file to a first storage path in the transit file server, recording a first identification of the first file, the first storage path and a synchronous operation type aiming at the first file, so that a downstream file exchange platform acquires the first file according to the recorded first storage path, and synchronously operating the first file according to the first identification of the first file and the synchronous operation type aiming at the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

The application also provides a file synchronization device, which comprises:

the receiving unit is used for receiving a first file to be synchronized and a first storage path of the first file, which are sent by the upstream file exchange platform; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

the storage unit is used for storing the first file to the first storage path, storing a first identification of the first file, the first storage path and a synchronous operation type aiming at the first file, so that a downstream file exchange platform obtains the first file according to the recorded first storage path and carries out synchronous operation on the first file according to the first identification of the first file and the synchronous operation type aiming at the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

The application also provides a file synchronization device, which comprises:

the reading unit is used for reading a first identifier of a first file to be synchronized, a first storage path and a synchronization operation type aiming at the first file from a pre-recorded metadata table after detecting that the upstream business file server is updated; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

The synchronization unit is used for acquiring the first file according to the first storage path and performing synchronization operation on the first file according to a first identifier of the first file and a synchronization operation type aiming at the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

The application also provides electronic equipment, which comprises a communication interface, a processor, a memory and a bus, wherein the communication interface, the processor and the memory are connected with each other through the bus;

the memory stores machine readable instructions and the processor performs the method by invoking the machine readable instructions.

The present application also provides a machine-readable storage medium storing machine-readable instructions that, when invoked and executed by a processor, implement the above-described methods.

In the above manner, after detecting that the upstream file server generates file update, first reading a first file to be synchronized, then generating a first storage path for the first file (for distinguishing multiple synchronization operations for the same file), and storing the first file into the first storage path in the transit file server; recording a first identifier (representing a storage path of the first file in a downstream service file server), the first storage path and a synchronous operation type of the first file, so that the downstream file exchange platform acquires the first file according to the first storage path and synchronously operates the first file according to the first identifier of the first file and the synchronous operation type of the first file; in addition, a first identifier representing a storage path of the file at the downstream service file server and a synchronous operation type of the file are recorded, so that the downstream exchange platform reads the file and performs corresponding synchronous operation on the file. Accordingly, because the first storage path distinguishes multiple synchronous operations for the same file, the upstream switching platform stores the file according to the generated first storage path, and the downstream switching platform acquires the file according to the first storage path, so that each file synchronous operation can be distinguished from other file synchronous operations, and the upstream switching platform and the downstream switching platform are prevented from conflicting and causing errors when performing different file synchronous operations.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present disclosure, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an architecture of a cascaded file sharing exchange platform according to an exemplary embodiment;

FIG. 2 is a schematic diagram of the architecture of another cascaded file sharing exchange platform, shown in an exemplary embodiment;

FIG. 3 is a schematic diagram of a file synchronization method according to an exemplary embodiment;

FIG. 4 is a schematic diagram of another file synchronization method shown in an exemplary embodiment;

FIG. 5 is a schematic diagram of another file synchronization method shown in an exemplary embodiment;

FIG. 6 is a hardware block diagram of an electronic device in which a file synchronization apparatus is located, according to an exemplary embodiment;

FIG. 7 is a block diagram of a file synchronization device shown in an exemplary embodiment;

FIG. 8 is a block diagram of another file synchronization device shown in an exemplary embodiment;

FIG. 9 is a block diagram of another file synchronization device shown in an exemplary embodiment.

Detailed Description

In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.

It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. Furthermore, individual steps described in this specification, in other embodiments, may be described as being split into multiple steps; while various steps described in this specification may be combined into a single step in other embodiments.

For example, referring to fig. 1, fig. 1 is a schematic diagram of an architecture of a cascaded file sharing switching platform according to an exemplary embodiment. As shown in fig. 1, after detecting that there is a file update in the upstream service file server 101, the upstream file exchange platform 102 reads a file to be synchronized in the upstream service file server 101, then writes the read file to be synchronized to the cascade middle-to-file server 103 (e.g. in path/dir/file 1), and writes metadata (file storage path filepath and file update time) corresponding to the read file to be synchronized to the cascade metadata table 104; when the downstream file exchange platform 105 polls the cascade metadata table 104, it finds that there is an updated file operation, reads a file from the cascade file transfer server 103 according to the file storage path recorded in the cascade metadata table 104, and synchronizes the read file to the downstream service file server 106.

In view of this, the present disclosure aims to propose a technical solution that, when files are synchronized by an upstream and a downstream file exchange platform, makes each file synchronization operation relatively independent by generating a first storage path in a transit file server for a file to be synchronized, where the first storage path can distinguish multiple synchronization operations for the same file, so as to avoid operation conflicts.

When the method is realized, after the upstream service file server is detected to be updated, a first file to be synchronized can be read from the upstream service file server; then, generating a first storage path in the transit file server for the first file; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file; further, the first file is stored to a first storage path in the transit file server, a first identification of the first file, the first storage path and a synchronous operation type for the first file are recorded, so that a downstream file exchange platform obtains the first file according to the recorded first storage path, and synchronous operation is carried out on the first file according to the first identification of the first file and the synchronous operation type for the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

For example, referring to fig. 2, fig. 2 is a schematic diagram of an architecture of another cascaded file sharing switching platform according to an exemplary embodiment. As shown in fig. 2, after detecting that there is a file update in the upstream service file server 101, the upstream file exchange platform 102 reads a file to be synchronized in the upstream service file server 101, and then generates a path (a first storage path) for the read file to be synchronized, where the file is to be stored in the transit file server 103; the path of the file to be synchronized stored in the downstream service file server 106 is obtained and represented by a first identifier, and the synchronization operation type, the first storage path and the first identifier of the file to be synchronized are recorded, so that the downstream file exchange platform 105 reads the file to be synchronized in the transit file server according to the recorded first storage path, and performs synchronization operation on the file to be synchronized according to the recorded synchronization operation type and the first identifier of the file to be synchronized, so that the file to be synchronized is synchronized into the downstream service file server 106.

The path of the file to be synchronized stored in the downstream service file server is generally the same as the path of the file to be synchronized stored in the upstream service file server.

It can be seen that, in the technical solution in the present specification, after detecting that the file update occurs in the upstream file server, first reading the first file to be synchronized, then generating a first storage path for the first file (for distinguishing multiple synchronization operations for the same file), and storing the first file into the first storage path in the transit file server; recording a first identifier (representing a storage path of the first file in a downstream service file server), the first storage path and a synchronous operation type of the first file, so that the downstream file exchange platform acquires the first file according to the first storage path and synchronously operates the first file according to the first identifier of the first file and the synchronous operation type of the first file; in addition, a first identifier representing a storage path of the file at the downstream service file server and a synchronous operation type of the file are recorded, so that the downstream exchange platform reads the file and performs corresponding synchronous operation on the file. Accordingly, because the first storage path distinguishes multiple synchronous operations for the same file, the upstream switching platform stores the file according to the generated first storage path, and the downstream switching platform acquires the file according to the first storage path, so that each file synchronous operation can be distinguished from other file synchronous operations, and the upstream switching platform and the downstream switching platform are prevented from conflicting and causing errors when performing different file synchronous operations.

The following describes the present application with reference to specific application scenarios by means of specific embodiments.

Referring to fig. 3, fig. 3 is a schematic diagram illustrating a file synchronization method according to an exemplary embodiment. The method may perform the steps of:

step 302: and after detecting that the upstream service file server is updated, reading a first file to be synchronized from the upstream service file server.

For example, as shown in fig. 2, after detecting that there is a file update in the upstream service file server 101, the upstream file exchange platform 102 reads a file to be synchronized stored in the upstream service file server 101, and uses the file to be synchronized as a first file.

The upstream service file server refers to a file storage server of a data provider, and is generally used for storing service files. The upstream file exchange platform refers to an intermediary service system located between a data source (or a data provider) and a data target (or a data receiver) in the data exchange process, and is generally supported by a data platform provider and is responsible for coordinating functions of data interaction, data transmission and the like. The upstream service file server refers to a file storage server of a data provider, which is generally used to store service files. When there is file update in the upstream service file server, the upstream file exchange platform detects the change by means of regular polling or real-time monitoring, and then obtains a file list stored in the upstream service file server, compares the file list with the file list obtained last time to identify newly added, deleted or updated files, and takes the file with the state change as a file to be synchronized.

Step 304: generating a first storage path in the transit file server for the first file; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file.

For example, as shown in fig. 2, the upstream file exchange platform 102 generates, for the read file to be synchronized, a path/dir/file1_1 (first storage path) for the file to be stored in the transit file server 103; the storage locations of the files to be synchronized in the intermediate file server 103 are not unified as/dir/file 1, but a first storage path for distinguishing multiple synchronization operations for the same file is generated each time the files are stored in the intermediate file server.

The transfer file server is an intermediate server between the upstream file exchange platform and the downstream file exchange platform, and comprises a server or a cluster formed by a plurality of servers and the like, and is used for temporarily storing files to be synchronized, coordinating data transmission between different data sources and targets, and reducing network flow and load pressure caused by direct transmission. The transit file server may be an independent server, or may be implemented as part of an upstream and downstream file exchange platform, which is not limited in this specification with respect to a specific form of the transit file server.

Step 306: storing the first file to a first storage path in the transit file server, and recording a first identification of the first file, the first storage path and a synchronous operation type for the first file, so that a downstream file exchange platform acquires the first file according to the recorded first storage path, and synchronously operates the first file according to the first identification of the first file and the synchronous operation type for the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

For example, as shown in fig. 2, the upstream file exchange platform 102 stores the read file to be synchronized as a first file to a first storage path/dir/file1_1 in the transit file server 103, the storage path/dir/file1 of the first file in the downstream service file server is represented by a first identifier, and the first identifier of the first file, the first storage path/dir/file1_1, and a synchronization operation type (optflag) of the first file to perform a synchronization operation are recorded in a metadata table. The downstream file exchange platform 105 obtains a first file to be synchronized from the intermediate file server 103 according to a first storage path/dir/file1_1 recorded in the metadata table, and determines whether to perform a new synchronization operation, an update synchronization operation or a delete synchronization operation on the first file according to an operation type (optflag) of insert, update, delete.

Wherein the first identifier is used for indicating a storage path of the file in the downstream service file server, and if the downstream service file server generates the storage path by using the file name, the first identifier is the original file name. If the downstream service file server is to generate a storage path with a file purpose, then the first identification is the file purpose. The present specification does not limit the generation manner of the storage path and the expression form of the first identifier in the downstream service file server.

In this embodiment, in order to reduce the backlog of the file in the transit file server, after the downstream file exchange platform completes the synchronization operation of the file to be synchronized, the downstream file exchange platform sends a file deletion instruction to the transit file server, and after receiving the file deletion instruction, the transit file server deletes the synchronized file, thereby reducing its own storage pressure.

In one embodiment shown, generating a first storage path in the staging file server for the first file includes:

For example, as shown in fig. 2, the upstream file exchange platform 102 takes the read file to be synchronized as a first file, the original file name of the first file is file1, the suffix_1 is added to the original file name file1 of the first file, the temporary file name file1_1 of the first file is generated, and then a first storage path/dir/file 1_1 in the transit file server 103 is generated for the first file based on the temporary file name file1_1 of the first file.

It should be noted that, the method of generating the first storage path in the transit file server for the first file is not limited to adding a suffix to the file name of the first file to obtain a temporary file name, and generating the first storage path based on the temporary file name; in fact, the first storage path only needs to be able to distinguish between multiple synchronization operations of the first file to be synchronized, and the generation of the first storage path may be generated according to at least one of the following information: a timestamp for identifying a file update time; batch identification of file synchronization operation; a random identification (e.g., a random number) that is not repeated, and the like.

In one embodiment shown, the first identifier is an original file name of the first file; the storing the first file in a first storage path in the transit file server, and recording a first identifier of the first file, the first storage path and a synchronization operation type for the first file, includes:

For example, as shown in fig. 2, the upstream file exchange platform 102 stores a file to be synchronized (i.e., a first file) read from the upstream service file server 101 into a first storage path/dir/file1_1 in the transit file server 103, and records an original file name file1, a first storage path/dir/file1_1, and a synchronization operation type insert (insert) for the first file in a metadata table; the downstream file exchange platform 105 obtains a first file from the intermediate file server 103 according to the first storage path/dir/file1_1 recorded in the metadata table, replaces the storage path dir/file1_1 of the first file with the storage path dir/file1 of the first file of the downstream service file server according to the original file name file1 of the first file, and performs a synchronization operation related to file insertion on the first file according to a synchronization operation type insert (insertion) of the first file.

The metadata table is a database table for storing related information of files or data, and contains descriptive information about the files, such as file names, file storage paths, operation types, file update time, and the like. In addition to storing descriptive information of the file in a metadata table, storage may also be performed at locations such as a file system log.

In one embodiment shown, the synchronization operation types include:

file insert type, file update type, file delete type.

For example, as shown in fig. 2, the file synchronization operation type (optflag) recorded in the metadata table may specifically include, but is not limited to, insert, update, delete, which respectively performs an add synchronization operation, an update synchronization operation, and a delete synchronization operation on the first file.

The file newly-added synchronization operation means that newly-added files appear in an upstream service file server, and an exchange platform is required to synchronize the newly-added new files into a downstream service file server; the file updating synchronization operation means that the existing file in the upstream service file server is changed, and the exchange platform is required to synchronize the changed file to the downstream service file server; the file deletion synchronization operation means that deleted files appear in the upstream service file server, and the exchange platform is required to synchronize the deleted files to the downstream service file server.

In one embodiment shown, the method further comprises:

For example, as shown in fig. 2, the update time of the first file of the files to be synchronized is recorded in the form of a metadata table, and for three different synchronization operations of the first file, the update time is respectively: as can be seen from the fact that the upstream service file server 101 adds the first file at the moment 07/01 12:00:00, the upstream service file server updates the first file at the moment 07/01 15:00:01, and the upstream service file server 101 deletes the first file at the moment 07/0117:00:02, respectively, the first file is newly added at the moment 07/01 12:00:00:00, 07/01 15:00:01, and 07/01:0117:00:02. When the update time recorded in the metadata table is polled for the update, the downstream file exchange platform 105 determines that the first file to be synchronized exists in the transit file server 103.

The manner in which the downstream file exchange platform 105 knows that the first file to be synchronized exists may be that the metadata table is periodically polled, or that the upstream file exchange platform 102 sends related information of file update to the downstream file exchange platform 105 when detecting that file update occurs in the upstream service file server 101; the present specification does not limit the manner in which the downstream file exchange platform knows that there is a first file to be synchronized.

When a new file is inserted into the upstream service file server, the file is not synchronized before, so that the storage path of the file is not required to be replaced and is stored to the transit file server, the new file is directly stored in the transit file server according to the storage path of the new file in the upstream service file server, steps are saved, and the operation efficiency of file synchronization is improved.

In one embodiment shown, the method further comprises:

For example, as shown in fig. 2, after the upstream file exchange platform 102 detects that a new file is inserted into the upstream service file server 101, a second file to be synchronized is read from the upstream service file server 101; since the second file belongs to new file insertion, file synchronization is not performed before, and there is no need to distinguish different synchronization operations of the second file, so that there is no need to change a storage path of the second file, the second file is directly stored into the transit file server according to a storage path/dir/file 1 in the upstream service file server 101, a storage path of the downstream service file server 106 and a file insertion type (optflag is insert) for the second file are also recorded, and the downstream file exchange platform 105 can obtain the second file from the transit file server 103 according to the storage path of the second file in the downstream service file server 106, and perform synchronization operations on the second file according to the file insertion type of the second file.

The upstream file exchange platform can detect that new files are inserted into the upstream service file server in various modes, for example, periodically initiate a query request to the upstream service file server to obtain the latest file list; monitoring the change of an upstream business file server in real time; and sending a notification to a designated process when the file system event occurs, wherein the upstream file exchange platform can acquire the file system event in real time through registering inotify monitoring, thereby detecting new file insertion and the like. The present specification does not limit the manner in which the upstream file exchange platform detects the file update in the upstream service file server.

When the file is deleted in the upstream service file server, the file is not required to be transmitted and stored any more, and only the information of the deleted file is required to be recorded, so that the deleted file is not required to be stored in the transit file server, the information of the deleted file is directly recorded in the metadata table, and the downstream file exchange platform performs file deletion operation according to the recorded information of the deleted file, so that steps are saved, and the operation efficiency of file synchronization is improved.

In one embodiment shown, the method further comprises:

For example, as shown in fig. 2, after the upstream file exchange platform 102 detects that the upstream service file server 101 performs file deletion, a third file to be synchronized is read from the upstream service file server 101; since the third file belongs to file deletion, file transmission and file storage are not required, so that the deleted file does not need to be stored in the transit file server 103, only the storage path of the third file in the downstream service file server 106 and the file deletion type (optflag is delete) for the third file need to be recorded in the metadata table, after the third file is periodically polled to the metadata table, the downstream file exchange platform 105 may execute a deletion-related synchronization operation on the third file according to the recorded storage path of the third file in the downstream service file server 106 and the file deletion type of the third file, specifically, the downstream file exchange platform 105 may determine the position of the third file to be deleted in the downstream service file server 106 according to the storage path of the third file in the downstream service file server 106, and then execute the deletion operation on the third file according to the file operation type of the third file as delete (optflag is delete).

It should be noted that, the upstream and the downstream merely represent data flow directions, and are not used to distinguish between the upper and lower relationships. The upstream of the data flow direction can be an upper-level platform or a lower-level platform, and the downstream of the data flow direction can be an upper-level platform or a lower-level platform. Taking a scenario of file data sharing between provinces and cities as an example, if a provincial platform (lower-level platform) reports a file to a provincial platform (upper-level platform), the provincial platform (lower-level platform) belongs to the upstream of the file, and the provincial platform (upper-level platform) belongs to the downstream of the file; if the municipal platform (lower platform) receives the file synchronized by the provincial platform (upper platform), the provincial platform belongs to the upstream of the file, and the municipal platform belongs to the downstream of the file.

Taking a city level platform (a lower level platform) as an example to report files to a provincial level platform (an upper level platform), the scheme is introduced as follows:

firstly, storing the own file of a municipal platform (lower platform) in a lower service file server, and storing the own file of a provincial platform (upper platform) in an upper service file server; then, the cascade file exchange platform (at least comprising an upstream file exchange platform, a downstream file exchange platform, a transit file server and the like) is utilized to realize the file synchronization of the city level platform to the provincial level platform according to the following Step1-Step6 by using the method:

Step1: when the lower file exchange platform detects that updated file1 exists in the/dir/file 1 path of the lower business file server, the newly added file1 is read.

Step2: the subordinate file exchange platform adds a file name suffix_1 according to the file name file1 of the newly added file, generates a temporary file name file1_1, generates a first storage path/dir/file 1_1 for distinguishing different synchronous operations on the same file based on the temporary file name, and stores the updated file into the path/dir/file 1_1 of the transit file server.

Step3: the lower-level file exchange platform writes the metadata corresponding to the updated file into a metadata table, wherein the metadata table comprises a filepath field, a filename field, an optflag field and an updatetime field. Writing a first storage path/dir/file1_1 of the updated file under the filepath field; writing an original file name file1 of the updated file under the filename field; an operation type insert/update/delete of the update file is written in the optflag field; the update time of the file is written in the update time field, such as 2023/07/18 16:54:01.

Step4: the upper level file exchange platform polls an 'update time' field in the metadata table, discovers a new record and determines that a file to be synchronized exists.

Step5: and the upper-level file exchange platform performs different operations according to the optflag field in the updated file metadata, if the optflag field is insert or update, reads the first storage path/dir/file1_1 according to the filepath field, reads the updated file in the transit file server according to the first storage path/dir/file1_1, restores the storage path of the updated file to/dir/file1 according to the file1 in the filename field when writing or covering the updated file in the upper-level service file server, then performs the operations, and issues a file deletion instruction to the transit file server after synchronization is completed, so that the file backlog of the transit file server is reduced.

Step6: if the optflag field is delete, the upper file exchange platform does not need to acquire the stored file in the transit file server, and directly confirms that the storage path of the file is restored to/dir/file 1 according to the information in the metadata table, and then performs synchronous deletion operation.

Referring to fig. 4, fig. 4 is a schematic diagram illustrating another file synchronization method according to an exemplary embodiment. The method may perform the steps of:

step 402: receiving a first file to be synchronized and a first storage path of the first file, which are sent by an upstream file exchange platform; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file.

For example, as shown in fig. 2, the intermediate file server 103 receives a first file that needs to be synchronized and is sent by the upstream file exchange platform 102, and a first storage path/dir/file1_1 of the first file, where the first storage path is used to distinguish multiple synchronization operations for the same file to be synchronized.

Step 404: storing the first file to the first storage path, and storing a first identifier of the first file, the first storage path and a synchronous operation type for the first file, so that a downstream file exchange platform obtains the first file according to the recorded first storage path, and performs synchronous operation on the first file according to the first identifier of the first file and the synchronous operation type for the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

For example, as shown in fig. 2, the intermediate file server 103 stores the first file in its own first storage path/dir/file1_1, and stores a first identifier of the first file, the first storage path/dir/file1_1, and a synchronization operation type (optflag) of the first file to be synchronized, so that the downstream file exchange platform 105 may acquire the first file to be synchronized according to the recorded first storage path/dir/file1_1, and perform a synchronization operation on the first file according to the first identifier of the first file and the synchronization operation type (optflag) of the first file to be synchronized. The first identifier is used for indicating a storage path/dir/file 1 of the first file in the downstream service file server.

In this specification, the specific implementation of steps 402 to 404 is similar to that of steps 302 to 306 described above, and will not be repeated here.

In the above embodiment, after receiving the first file to be synchronized sent by the upstream file exchange platform and the first storage path generated by the upstream file exchange platform for the first file and used for distinguishing multiple synchronization operations for the same file, the transit file server stores the first file in the first storage path, records the first storage path of the first file, the first identifier used for indicating the storage path of the downstream service file server of the first file, and the synchronization operation type of the first file, so that the downstream file exchange platform obtains the first file according to the first storage path and performs synchronization operations, and each synchronization operation of the downstream exchange platform on the file is performed on different paths, thereby avoiding errors when the same file is subjected to different operations on the same path.

It should be noted that, the transit file server may store only the file, and not store the first identifier of the first file, the first storage path and the type of synchronization operation for the first file, where these information may be stored separately in the form of metadata table, and the specific storage manner of the file description information is not limited in this specification.

In one embodiment shown, the first storage path is generated based on a temporary file name of the first file; the temporary file name is obtained by adding a suffix to the file name of the first file.

In one embodiment shown, the first identifier is an original file name of the first file; storing the first file to the first storage path, and storing a first identification of the first file, the first storage path, and a type of synchronization operation for the first file, including:

storing the first file to the first storage path, storing an original file name of the first file, the first storage path and a synchronous operation type aiming at the first file, so that the downstream file exchange platform obtains the first file according to the recorded first storage path, replaces the storage path of the first file with the storage path of the first file of the downstream service file server according to the recorded original file name of the first file, and carries out synchronous operation on the first file according to the synchronous operation type aiming at the first file.

In one embodiment shown, the synchronization operation types include:

file insert type, file update type, file delete type.

In one embodiment shown, the method further comprises:

and storing the update time of the first file so that the downstream file exchange platform determines that the first file exists when the update time is polled to have the update.

In one embodiment shown, the method further comprises:

receiving a second file to be inserted and a storage path of a downstream business file server, wherein the second file is sent by an upstream file exchange platform;

and storing the second file to a storage path of a downstream business file server, and storing a synchronous operation type of the second file so that the downstream file exchange platform can synchronously operate the second file according to the synchronous operation type aiming at the second file.

Referring to fig. 5, fig. 5 is a schematic diagram illustrating another file synchronization method according to an exemplary embodiment. The method may perform the steps of:

step 502: after detecting that an upstream business file server generates file updating, reading a first identification, a first storage path and a synchronous operation type aiming at a first file to be synchronized from a pre-recorded metadata table; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file.

For example, as shown in fig. 2, after knowing that the file synchronization operation needs to be performed, the downstream file exchange platform 105 reads a first identifier of a first file to be synchronized, a first storage path/dir/file1_1 of the first file, and a synchronization operation type (optflag) for the first file from the recorded metadata table; the first storage path/dir/file1_1 of the first file can distinguish multiple synchronous operations aiming at the same file to be synchronized.

Step 504: acquiring the first file according to the first storage path, and carrying out synchronous operation on the first file according to a first identifier of the first file and a synchronous operation type aiming at the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

For example, as shown in fig. 2, the downstream file exchange platform 105 obtains the first file according to a first storage path/dir/file1_1 of the first file, and performs a synchronization operation on the first file according to a first identifier of the first file and a synchronization operation type (optflag) for the first file, where the first identifier is used to indicate that the first file is in the storage path/dir/file1 of the downstream service file server.

In this specification, the specific implementation manners of step 502 to step 504 are similar to those of step 302 to step 306 described above, and will not be repeated here.

In the above embodiment, after knowing that the upstream service file server has a file update, the downstream file exchange platform acquires a first file to be synchronized according to a first storage path for distinguishing multiple synchronization operations for the same file, and synchronizes the first file according to a first identifier for indicating the storage path of the downstream service file server and a synchronization operation type for the first file; the method and the device realize that each synchronous operation of the downstream exchange platform on the file is carried out in different paths, and avoid errors when the same file is carried out in different operations in the same path.

In one embodiment shown, the first identifier is an original file name of the first file; the obtaining the first file according to the first storage path, and performing synchronization operation on the first file according to the first identifier of the first file and the synchronization operation type for the first file, including:

Acquiring the first file at a file transfer server according to the first storage path;

replacing a storage path of the first file with a storage path of the first file of a downstream business file server according to the original file name of the first file;

and carrying out synchronous operation on the first file according to the synchronous operation type aiming at the first file.

In one embodiment shown, the synchronization operation types include:

file insert type, file update type, file delete type.

In one embodiment shown, the method further comprises:

and when the file updating moment is polled, determining that the first file exists in the transit file server.

In one embodiment shown, the method further comprises:

after detecting that a new file is inserted into the upstream service file server, acquiring a second file according to a recorded inserted second file in a storage path of the downstream service file server, and performing synchronous operation on the second file according to a file insertion type of the second file.

In one embodiment shown, the method further comprises:

After detecting that the upstream service file server performs file deletion, performing synchronous operation on a third file according to the recorded storage path of the deleted third file in the downstream service file server and the file deletion type of the third file.

Corresponding to the embodiment of the file synchronization method, the present specification also provides an embodiment of a file synchronization device.

Referring to fig. 6, fig. 6 is a hardware configuration diagram of an electronic device where a file synchronization device is located in an exemplary embodiment. At the hardware level, the device includes a processor 602, an internal bus 604, a network interface 606, memory 608, and non-volatile storage 610, although other hardware requirements are possible. One or more embodiments of the present description may be implemented in a software-based manner, such as by the processor 602 reading a corresponding computer program from the non-volatile memory 610 into the memory 608 and then running. Of course, in addition to software implementation, one or more embodiments of the present disclosure do not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but may also be hardware or a logic device.

Referring to fig. 7, fig. 7 is a block diagram illustrating a file synchronization apparatus according to an exemplary embodiment. The file synchronization device can be applied to the electronic equipment shown in fig. 6 to realize the technical scheme of the specification. The apparatus may include:

a reading unit 702, configured to read a first file to be synchronized from an upstream service file server after detecting that the upstream service file server is updated with a file;

a generating unit 704, configured to generate a first storage path in the transit file server for the first file; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

a first storage unit 706, configured to store the first file to a first storage path in the intermediate file server, record a first identifier of the first file, the first storage path, and a synchronization operation type for the first file, so that a downstream file exchange platform obtains the first file according to the recorded first storage path, and perform a synchronization operation on the first file according to the first identifier of the first file and the synchronization operation type for the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

In this embodiment, the generating unit includes:

a first generation subunit, configured to add a suffix to a file name of the first file, and generate a temporary file name of the first file;

and the second generation subunit is used for generating a first storage path in the transit file server side for the first file based on the temporary file name of the first file.

In this embodiment, the first identifier is an original file name of the first file; the first storage unit includes:

the storage subunit is configured to store the first file to a first storage path in the intermediate file server, record an original file name of the first file, the first storage path, and a synchronization operation type for the first file, so that the downstream file exchange platform obtains the first file according to the recorded first storage path, replaces the storage path of the first file with the storage path of the first file in the downstream service file server according to the recorded original file name of the first file, and performs a synchronization operation on the first file according to the synchronization operation type for the first file.

In this embodiment, the synchronization operation types include:

file insert type, file update type, file delete type.

In this embodiment, the apparatus further includes:

the first recording unit is used for recording the update time of the first file so that the downstream file exchange platform determines that the first file exists in the transit file server when the update time is polled to have update.

In this embodiment, the apparatus further includes:

the first detection unit is used for reading a second file to be synchronized from the upstream business file server after detecting that the upstream business file server is subjected to new file insertion;

and the second storage unit is used for storing the second file into the transit file server and recording a storage path of the downstream service file server and a file insertion type for the second file, so that the downstream file exchange platform synchronously operates the second file according to the storage path of the downstream service file server and the file insertion type for the second file.

In this embodiment, the apparatus further includes:

The second detection unit is used for reading a third file to be synchronized from the upstream service file server after detecting that the upstream service file server performs file deletion;

and the second recording unit is used for recording the storage path of the third file at the downstream service file server and the file deletion type aiming at the third file, so that the downstream file exchange platform synchronously operates the third file according to the storage path of the third file at the downstream service file server and the file deletion type aiming at the third file.

Referring to fig. 8, fig. 8 is a block diagram illustrating another file synchronization apparatus according to an exemplary embodiment. The file synchronization device can be applied to the electronic equipment shown in fig. 6 to realize the technical scheme of the specification. The apparatus may include:

a first receiving unit 802, configured to receive a first file to be synchronized and a first storage path of the first file, where the first file is sent by an upstream file exchange platform; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

a first storage unit 804, configured to store the first file to the first storage path, and store a first identifier of the first file, the first storage path, and a synchronization operation type for the first file, so that a downstream file exchange platform obtains the first file according to the recorded first storage path, and performs a synchronization operation on the first file according to the first identifier of the first file and the synchronization operation type for the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

In this embodiment, the first storage path is generated based on a temporary file name of the first file; the temporary file name is obtained by adding a suffix to the file name of the first file.

the storage subunit is configured to store the first file to the first storage path, store an original file name of the first file, the first storage path, and a synchronization operation type for the first file, so that the downstream file exchange platform obtains the first file according to the recorded first storage path, replaces the storage path of the first file with the storage path of the first file of the downstream service file server according to the recorded original file name of the first file, and performs a synchronization operation on the first file according to the synchronization operation type for the first file.

In this embodiment, the synchronization operation types include:

file insert type, file update type, file delete type.

In this embodiment, the apparatus further includes:

And the second storage unit is used for storing the update time of the first file so that the downstream file exchange platform determines that the first file exists when the update time is polled to have the update.

In this embodiment, the apparatus further includes:

the second receiving unit is used for receiving a second file to be inserted and a storage path of a downstream business file server, wherein the second file is sent by the upstream file exchange platform;

and the third storage unit is used for storing the second file to a storage path of the second file on a downstream service file server and storing the synchronous operation type of the second file so that the downstream file exchange platform can carry out synchronous operation on the second file according to the synchronous operation type of the second file.

Referring to fig. 9, fig. 9 is a block diagram of another file synchronization apparatus according to an exemplary embodiment. The file synchronization device can be applied to the electronic equipment shown in fig. 6 to realize the technical scheme of the specification. The apparatus may include:

a reading unit 902, configured to read, after detecting that a file update occurs in an upstream service file server, a first identifier of a first file to be synchronized, a first storage path, and a synchronization operation type for the first file from a pre-recorded metadata table; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

A synchronizing unit 904, configured to obtain the first file according to the first storage path, and perform a synchronization operation on the first file according to a first identifier of the first file and a synchronization operation type for the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

In this embodiment, the first identifier is an original file name of the first file; the synchronization unit includes:

the obtaining subunit is used for obtaining the first file at the transit file server according to the first storage path;

a replacing subunit, configured to replace, according to an original file name of the first file, a storage path of the first file with a storage path of the first file on a downstream service file server;

and the synchronization subunit is used for performing synchronization operation on the first file according to the type of the synchronization operation on the first file.

In this embodiment, the synchronization operation types include:

File insert type, file update type, file delete type.

In this embodiment, the apparatus further includes:

and the determining unit is used for determining that the first file exists in the transit file server when the file update moment is polled and the file update exists.

In this embodiment, the apparatus further includes:

the acquisition unit is used for acquiring a second file according to a recorded inserted second file after detecting that the new file is inserted into the upstream service file server;

and the first synchronization unit is used for synchronizing the second file according to the file insertion type of the second file.

In this embodiment, the apparatus further includes:

and the second synchronization unit is used for performing synchronization operation on the third file according to the recorded storage path of the third file deleted in the downstream service file server and the file deletion type aiming at the third file after detecting that the file deletion occurs in the upstream service file server.

The implementation process of the functions and roles of each unit in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.

For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are illustrative only, in that the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present description. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or a combination of any of these devices.

In a typical configuration, a computer includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, read only compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by the computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

User information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to herein are both user-authorized or fully authorized information and data by parties, and the collection, use and processing of relevant data requires compliance with relevant laws and regulations and standards of the relevant country and region, and is provided with corresponding operation portals for user selection of authorization or denial.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The terminology used in the one or more embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the specification. As used in this specification, one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of the present description to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.

The foregoing description of the preferred embodiment(s) is (are) merely intended to illustrate the embodiment(s) of the present invention, and it is not intended to limit the embodiment(s) of the present invention to the particular embodiment(s) described.

Claims

1. A method of file synchronization, the method comprising:

2. The method of claim 1, wherein generating a first storage path in the staging file server for the first file comprises:

3. The method of claim 2, wherein the first identification is an original file name of the first file; the storing the first file in a first storage path in the transit file server, and recording a first identifier of the first file, the first storage path and a synchronization operation type for the first file, includes:

4. A method of file synchronization, the method comprising:

5. A method of file synchronization, the method comprising:

6. A file synchronization device, the device comprising:

the first storage unit is used for storing the first file to a first storage path in the transit file server, recording a first identification of the first file, the first storage path and a synchronous operation type aiming at the first file, so that a downstream file exchange platform obtains the first file according to the recorded first storage path, and synchronously operates the first file according to the first identification of the first file and the synchronous operation type aiming at the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

7. A file synchronization device, the device comprising:

the first receiving unit is used for receiving a first file to be synchronized and a first storage path of the first file, which are sent by the upstream file exchange platform; the first storage path is used for distinguishing multiple synchronous operations aiming at the same file;

the first storage unit is used for storing the first file to the first storage path, storing a first identification of the first file, the first storage path and a synchronous operation type aiming at the first file, so that a downstream file exchange platform obtains the first file according to the recorded first storage path, and synchronously operating the first file according to the first identification of the first file and the synchronous operation type aiming at the first file; wherein the first identifier is used for representing a storage path of the first file at a downstream service file server.

8. A file synchronization device, the device comprising:

9. An electronic device comprises a communication interface, a processor, a memory and a bus, wherein the communication interface, the processor and the memory are connected with each other through the bus;

the memory stores machine readable instructions, the processor executing the method of any of claims 1 to 5 by invoking the machine readable instructions.

10. A machine-readable storage medium storing machine-readable instructions which, when invoked and executed by a processor, implement the method of any one of claims 1 to 5.