CN112492001A - Data transmission method, device and equipment and computer storage medium - Google Patents

Data transmission method, device and equipment and computer storage medium Download PDF

Info

Publication number
CN112492001A
CN112492001A CN202011295099.4A CN202011295099A CN112492001A CN 112492001 A CN112492001 A CN 112492001A CN 202011295099 A CN202011295099 A CN 202011295099A CN 112492001 A CN112492001 A CN 112492001A
Authority
CN
China
Prior art keywords
storage device
file
data
information
state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202011295099.4A
Other languages
Chinese (zh)
Inventor
何炳川
吴勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Sensetime Technology Co Ltd
Original Assignee
Shenzhen Sensetime Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Sensetime Technology Co Ltd filed Critical Shenzhen Sensetime Technology Co Ltd
Priority to CN202011295099.4A priority Critical patent/CN112492001A/en
Publication of CN112492001A publication Critical patent/CN112492001A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G06F16/162Delete operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files

Abstract

The application discloses a data transmission method, a device, equipment and a computer storage medium, wherein the method comprises the following steps: reading a specific state file from a first storage device, wherein the specific state file comprises first quantity information, and the specific state file is read from a second storage device by a gatekeeper and ferred to the first storage device; the first quantity information is used for representing the quantity of the data files stored in the second storage device; pushing a data file to the first storage device if the first amount information is less than a first specific amount; the data file in the first storage device is used for: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.

Description

Data transmission method, device and equipment and computer storage medium
Technical Field
The present application relates to, but not limited to, data transmission technology, and in particular, to a data transmission method, apparatus, device, and computer storage medium.
Background
In many scenarios, in order to improve the security of data transmission, it is common to physically isolate the data sender from the data receiver.
In the related art, a data sender continuously pushes data to a data receiver regardless of whether the data receiver is capable of receiving the data, however, in case that the data receiver does not have enough capacity to receive the data, the pushed data is lost.
Disclosure of Invention
The application provides a data transmission method, a data transmission device, data transmission equipment and a computer storage medium.
In a first aspect, a data transmission method is provided, which is applied to a data sending device, and the method includes: reading a specific state file from a first storage device, wherein the specific state file comprises first quantity information, and the specific state file is read from a second storage device by a gatekeeper and ferred to the first storage device; the first quantity information is used for representing the quantity of the data files stored in the second storage device; pushing a data file to the first storage device if the first amount information is less than a first specific amount; the data file in the first storage device is used for: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
Therefore, the data sending device can determine the first quantity information of the data files in the second storage device of the data receiving party, so that the second storage device can receive the data files under the condition that the first quantity information is smaller than the first specific quantity, the data files can be pushed to the first storage device, and the situation that the data files are easy to lose due to the fact that the data files are pushed to the first storage device when the second storage device does not receive the data files.
In some embodiments, the specific state file further comprises: first connection state information between the data receiving device and the second storage device; the pushing a data file to the first storage device in the case that the first amount information is less than a first specific amount includes: and under the condition that the first quantity information is smaller than the first specific quantity and the first connection state information represents the connection between the data receiving equipment and the second storage equipment, pushing a data file to the first storage equipment.
Therefore, the data sending device pushes the data files to the first storage device based on the first quantity information of the data files in the second storage device and the connection state between the data receiving device and the second storage device, so that the data sending device can determine that the second storage device has the ability to receive the data files and can also determine that the data files in the second storage device can be extracted, and the situation that the data files in the second storage device cannot be extracted due to disconnection between the data receiving device and the second storage device, so that more and more stored data of the second storage device are obtained, and the second storage device is prone to damage is avoided.
In some embodiments, the method further comprises: determining second quantity information of the data files stored in the first storage device and second connection state information between the data sending device and the first storage device; the pushing a data file to the first storage device under the condition that the first quantity information is smaller than the first specific quantity and the first connection state information represents the connection between the data receiving device and the second storage device includes: and pushing a data file to the first storage device under the condition that the first quantity information is smaller than the first specific quantity, the first connection state information represents the connection between the data receiving device and the second storage device, the second quantity information is smaller than the second specific quantity, and the second connection state information represents the connection between the data sending device and the first storage device.
In this way, the data sending device pushes the data files to the first storage device not only based on the number and the connection state of the data files stored by the data receiving party, but also based on the number and the connection state of the data files stored by the data sending party, so that the data sending device can push the data files to the first storage device not only when the receiving state of the data receiving party is determined to be good, but also when the sending state of the data sending party is determined to be good, and the pushing of the data files can be effectively carried out.
In some embodiments, the method further comprises: stopping pushing data files to the first storage device in one of the following cases; in a case where the first amount information is greater than or equal to the first specific amount; in the event that the first connection status information characterizes a disconnection between the data receiving device and the second storage device; in a case where the second number information is greater than or equal to the second specific number; and in the case that the second connection state information represents disconnection between the data transmission device and the first storage device.
Therefore, when the data sending equipment determines that the state of the data sending party or the state of the data receiving party is bad, the data file is stopped being pushed to the first storage equipment, so that the data pushing can be effectively carried out, and the situation that the data file is easy to lose and the storage equipment is easy to damage due to the fact that the data file is continuously pushed under the condition that the state is bad is avoided.
In some embodiments, the method further comprises: reading the specific state file from the first storage device every a first time interval; determining the second quantity information and the second connection state information every second duration; wherein the first duration is greater than or equal to the second duration.
In this way, the data sending device can regularly determine the state of the data sending party and the state of the data receiving party, so that whether the state of the data receiving party and the state of the data sending party are normal or not can be continuously determined, and in the case that any state is abnormal, the data file sending to the first storage device is stopped, so that the data pushing can be effectively carried out. In addition, the state of the data sender is easy to determine, so that the state of the data sender can be determined at a higher frequency, and the hidden danger of data insecurity can be improved because the data receiver sends a state file to the data sender, so that the state of the data receiver can be determined at a lower frequency, and the safety of data transmission and the accuracy of state determination can be considered at the same time.
In some embodiments, the method further comprises: determining that a first state file is the specific state file under the condition that the first state file is stored in the first storage device, wherein the first state file comprises the first quantity information; in a case where at least two second state files are stored in the first storage device, determining time information of each of the at least two second state files, the time information including: file creation time information or file generation time information, the second state file including the first amount information; determining a latest second state file of the at least two second state files as the specific state file based on the time information.
In this way, in the case where at least two second state files exist in the first storage device, the second state file with the latest time most indicates the state information of the data receiving side, so that the data transmitting device can determine the state of the data receiving side only by the second state file with the latest time in order to reduce the amount of calculation by the data transmitting side.
In some embodiments, after reading the specific state file from the first storage device, the method further comprises: deleting the one first state file or the at least two second state files in the first storage device.
In this way, by deleting the first state file or the at least two second state files on the first storage device, the storage occupation of the first storage device can be reduced, and the data transmission device is not easily affected when reading the state files next time.
In some embodiments, at least one of the following files is a file transfer protocol, FTP, file: the specific state file, the data file stored in the first storage device, and the data file stored in the second storage device.
Therefore, at least one of the files transmitted through the gateway is the FTP file, and the FTP file has the characteristic of high safety, so that the safety of the data transmission process is improved.
In a second aspect, a data transmission method is provided, which is applied to a data sending device, and the method includes: determining second quantity information of the data files stored in the first storage device; pushing a data file to the first storage device under the condition that the second amount information is smaller than a second specific amount, wherein the data file in the first storage device is used for: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
Therefore, the data sending device can push the data files to the first storage device based on the first quantity information of the data files in the first storage device connected with the data sending device, and the data sending device can determine that the state of the data sending party is good through the data files in the first storage device, so that the situation that when the number of the data files in the first storage device of the data sending party is large, the files are pushed to the first storage device, the data quantity in the first storage device is easy to exceed the storage capacity of the first storage device, and the first storage device is easy to damage and the data files are easy to lose can be avoided.
In a third aspect, a data transmission method is provided, which is applied to a data receiving device, and includes: determining first quantity information of the stored data files from the second storage device; the data file in the second storage device is read from the first storage device by the gatekeeper and ferred; generating a specific state file based on the first quantity information; storing the specific state file into the second storage device, wherein the specific state file in the second storage device is used for: the gatekeeper reads and ferries to the first storage device, and a specific state file in the first storage device is used for: the data sending device pushes the data file to the first storage device based on the first amount of information in the specific state file.
Therefore, the data receiving device can generate the specific state file based on the first quantity information, and when the specific state file is stored in the second storage device, the specific state file can be acquired by the data sending device, so that the data sending device can determine that the second storage device has the ability to receive the data file under the condition that the first quantity information in the specific state file is smaller than the first specific quantity, and then can push the data file to the first storage device, and the situation that the data file is easy to lose due to the fact that the data file is pushed to the first storage device when the second storage device does not have the ability to receive the data file is avoided.
In some embodiments, the method further comprises: determining first connection state information between the data receiving device and the second storage device; generating a specific state file based on the first amount of information comprises: generating the specific state file based on the first quantity information and the first connection state information.
In this way, the data receiving device generates the specific state file based on not only the first amount information but also the first connection state information between the data receiving device and the second storage device, so that when the data sending device acquires the specific state file, the data receiving device pushes the data file to the first storage device based on not only the first amount information of the data file in the second storage device but also the connection state information between the data receiving device and the second storage device.
In some embodiments, said generating said particular state file based on said first quantity information and said first connection state information comprises: creating a first temporary file; the first temporary file is a file which cannot be read by the gatekeeper; writing the first quantity information and the first connection state information into the first temporary file to obtain a second temporary file; and converting the format of the second temporary file to obtain the specific state file.
Therefore, a temporary file which cannot be read by the gateway is established, the first quantity information and the first connection state information are written into the temporary file, and the format of the temporary file is changed into a file which can be read by the gateway, so that the file read by the gateway inevitably comprises the first quantity information and the first connection state information.
In some embodiments, the method further comprises: determining the first quantity information and the first connection state information every first time length.
In this way, the data receiving device can periodically determine the first quantity information and the first connection state information, so as to continuously generate a specific state file, thereby enabling the data transmission to continuously determine the state of the data receiving party.
In some embodiments, before said storing said particular state file into said second storage device, said method further comprises: determining at least one third state file stored in the second storage device, and deleting the at least one third state file, wherein the third state file comprises the first quantity information.
In this way, only the latest status file is retained in the second storage device, so that the latest status file can be transported when the gatekeeper transports the status file, the load of the gatekeeper is reduced, and the data transmission device can also easily determine the specific status file.
In a fourth aspect, there is provided a data transmission apparatus, the apparatus comprising: the reading unit is used for reading a specific state file from a first storage device, wherein the specific state file comprises first quantity information, and the specific state file is read from a second storage device by a gateway and ferred to the first storage device; the first quantity information is used for representing the quantity of the data files stored in the second storage device; the pushing unit is used for pushing the data files to the first storage device under the condition that the first quantity information is smaller than a first specific quantity; the data file in the first storage device is used for: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
In a fifth aspect, a data transmission apparatus is provided, the apparatus comprising: a determining unit configured to determine second quantity information of the data files stored in the first storage device; a pushing unit, configured to push a data file to the first storage device when the second amount information is smaller than a second specific amount, where the data file in the first storage device is used to: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
In a sixth aspect, there is provided a data transmission apparatus, the apparatus comprising: a determining unit configured to determine first quantity information of the stored data files from the second storage device; the data file in the second storage device is read from the first storage device by the gatekeeper and ferred; a generating unit configured to generate a specific state file based on the first amount information; a storage control unit, configured to store the specific state file into the second storage device, where the specific state file in the second storage device is used to: the gatekeeper reads and ferries to the first storage device, and a specific state file in the first storage device is used for: the data sending device pushes the data file to the first storage device based on the first amount of information in the specific state file.
A seventh aspect provides a data transmission apparatus, including: a memory storing a computer program operable on a processor and a processor implementing the steps of any of the methods described above when executing the program.
In an eighth aspect, a computer storage medium is provided that stores one or more programs, the one or more programs being executable by one or more processors to implement the steps of any of the methods described above.
Drawings
Fig. 1 is a schematic architecture diagram of a data transmission system according to an embodiment of the present application;
fig. 2 is a schematic flow chart illustrating an implementation of a data transmission method according to an embodiment of the present application;
fig. 3 is a schematic flow chart illustrating an implementation of another data transmission method according to an embodiment of the present application;
fig. 4 is a schematic flow chart illustrating an implementation of another data transmission method according to an embodiment of the present application;
fig. 5 is a schematic flow chart illustrating an implementation of another data transmission method according to an embodiment of the present application;
fig. 6 is a schematic flow chart illustrating an implementation of a data transmission method according to another embodiment of the present application;
fig. 7 is a schematic flow chart illustrating an implementation of a data transmission method according to another embodiment of the present application;
fig. 8 is a schematic flow chart illustrating an implementation of a data transmission method according to still another embodiment of the present application;
fig. 9 is a schematic flow chart illustrating another implementation of a data transmission method according to another embodiment of the present application;
fig. 10 is a schematic structural diagram of a data transmission device according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of another data transmission device according to an embodiment of the present disclosure;
fig. 12 is a schematic structural diagram of another data transmission apparatus according to an embodiment of the present application;
fig. 13 is a hardware entity diagram of a data transmission device according to an embodiment of the present application.
Detailed Description
The technical solution of the present application will be specifically described below by way of examples with reference to the accompanying drawings. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
It should be noted that: in the present examples, "first", "second", etc. are used for distinguishing similar objects and are not necessarily used for describing a particular order or sequence. The technical means described in the embodiments of the present application may be arbitrarily combined without conflict.
In a public security system, for security and confidentiality, the public security system is usually physically separated from a video network system, but various monitoring devices (such as cameras and capturing machines) are all deployed in the video network, the public security system needs to use data shot by the monitoring devices to operate various services and technical and tactics, in order to obtain the shot data, the public security network usually ferries the data from the video network to the public security network through a network gate, and in order to achieve security, the public security network is generally only allowed to ferry the data to the public security network, and the public security network is not allowed to ferry the data to the video network. A File Transfer Protocol (FTP) server or an FTP storage device needs to be deployed in both the public security network system and the video network system as a storage medium for data.
FTP is an english abbreviation of File Transfer Protocol (File Transfer Protocol), and is one of protocols in the Transmission Control Protocol/Internet Protocol (TCP/IP) Protocol suite. The FTP protocol includes two components, one being an FTP server and the other being an FTP client. The FTP server is used for storing files, and a user can use the FTP client to access resources located on the FTP server through an FTP protocol.
It should be noted that the networks corresponding to the data sending party and the data receiving party in the embodiment of the present application are a video network and a public security network, respectively, but other two networks to which the data transmission method in the embodiment of the present application can be applied should be within the scope of the present application, for example, the networks corresponding to the data sending party and the data receiving party may be a video network and a government network, respectively, or a file network and a public security network, respectively.
In this embodiment, the network corresponding to the data sending party may be an extranet, and the network corresponding to the data receiving party may be an intranet. When the gatekeeper continuously moves data from the external network FTP server to the internal network FTP server, the data consumption program of the internal network may not lift the data files due to faults, so that the residual space of the disk of the internal network FTP server is smaller and smaller, and finally the fault of the internal network FTP server is caused, and the data push program of the external network has no knowledge about the fault, so that the data is continuously pushed, and the data loss may be caused.
Data is a very important resource of a service system, and the normal development of services is seriously affected by data loss, so a scheme is urgently needed to actively protect the system and avoid data loss. In order to solve at least the problems in the related art, embodiments of the present application provide the following data transmission schemes.
Fig. 1 is a schematic architecture diagram of a data transmission system according to an embodiment of the present application, and as shown in fig. 1, the data transmission system 100 includes: a data transmission device 101 and a first storage device 102 in the external network, a gatekeeper 103, and a second storage device 104 and a data reception device 105 in the internal network.
The data sending device 101 may connect to a database and extract relevant data from the database and push the data to the first storage device 102 in the form of a data file. The database may be one or a combination of at least two of a video database, a file database, a voice database, or other database. For example, in the case where the database is a video database, the data transmission apparatus 101 may extract video data from the video database and store the video data in the form of a file in the first storage apparatus 102. In some embodiments, each video file size stored in the first storage device 102 may be a first set size file, or each video file size stored in the first storage device 102 may be a file that is smaller than the first set size. For another example, in the case where the database is a file database, the data transmission apparatus 101 may extract file data from the file database and store the file data in a file of a second set size or less in the first storage apparatus 102.
After the first storage device 102 stores the data file, the gatekeeper 103 may periodically or aperiodically read the data file from the first storage device 102 and ferry the read data file to the second storage device 104.
The data receiving device 105 may read the data file from the second storage device 104 or extract the data file from the second storage device 104 to another storage device connected to the data receiving device 105. In some embodiments, not only may the data receiving device read data from the second storage device, there may be other devices that read data files from the second storage device. When the data receiving device and/or other devices read the data file from the second storage device, in one case, the data file can be read only, the data file is still in the second storage device, and after the read data file is used up, the read data file can be deleted; in another case, the data file may be retrieved and stored in another storage device.
The above description is made on the process of ferrying the data file from the external network to the internal network, and on the basis of the architecture of the data transmission system 100 provided in the embodiment of the present application, the present application may provide the following technical solutions of the data transmission method.
Fig. 2 is a schematic diagram of an implementation flow of a data transmission method provided in an embodiment of the present application, and as shown in fig. 2, the method is applied to a data sending device, and the method includes:
s201, reading a specific state file from a first storage device, wherein the specific state file comprises first quantity information, and the specific state file is read from a second storage device by a gatekeeper and ferred to the first storage device; the first quantity information is used to characterize the quantity of data files stored in the second storage device.
In some embodiments, the first amount information in the particular state file may characterize the amount of data files currently stored in the second storage device. For example, the first amount of information in a particular state file may characterize the amount of data files currently stored in the second storage device at the time the particular state file was generated or created.
The combination of the data transmission device and the first storage device may form a data sender. The data sending device can be a first FTP client, and the first storage device can be a first FTP server. The data transmission device may read the specific state file from the first storage device through the FTP protocol.
The data transmitting device may read the specific state file from the first specific directory of the first storage device. The first specific directory may be a preset directory, which may be used to exclusively store the specific state file, or may store not only the specific state file but also other files such as a data file such as at least one of a video file, an audio file, a document file, and the like.
In some embodiments, the data sending device may read a particular status file from the first storage device every first duration. The first time period may be between 10 seconds and 2 minutes, for example the first time period may be 10 seconds, 1 minute, 2 minutes, or the like. In other embodiments, the data sending device may determine to use different interval durations to read the specific status file according to different actual scenarios. Taking the example that the data sending party sends the video file to the data receiving party, under the scenes of holidays, commuting time and the like, the number of monitored people is large, the monitoring condition needing to be checked by the public security network is also large, and therefore the number of data files sent to the public security network by the video network is increased, and therefore the data party equipment can read the specific state file from the first storage equipment at short time intervals; during the period from morning to morning, fewer scenes need to be monitored by public security screening, so that the specific state file can be read from the first storage space at longer time intervals.
In some embodiments, the particular state file may include: the first amount of information. In other embodiments, the particular state file may include not only the first amount of information, but also other information, which may include at least one of: first connection state information between the data reception device and the second storage device, a type of the specific state file, attribute information of the specific state file, a first specific number, attribute information of the second storage device, attribute information of the data reception device, and the like.
The specific state file in the embodiment of the present application may be an FTP file. In other embodiments, the particular state file may be a file in other formats, such as HyperText Transfer Protocol (HTTP), and the like.
The specific state file may be generated by a data receiver of the intranet. For example, in some embodiments, a data receiving device of a data recipient may determine first amount information of data files stored in a second storage device connected to the data receiving device, and generate a particular state file based on the first amount information.
In some embodiments, the data receiving device generating the particular state file based on the first amount of information may include: and writing the first quantity information into the first temporary file to obtain a second temporary file, wherein both the first temporary file and the second temporary file are files which cannot be read by the gatekeeper, and then performing format conversion on the second temporary file to obtain a specific state file which is a file which can be read by the gatekeeper.
In other embodiments, the data receiving device generating the particular state file based on the first amount of information may include: generating a target file, and writing the first amount of information into the target file to obtain a specific state file; wherein the target file is a file capable of being read by the gatekeeper.
The first temporary file and/or the target file may be a blank file having a certain format or a file having a certain format and attribute class information so that the data receiving apparatus can write the related information including the first amount of information into the corresponding attribute class.
The file size of a particular state file may be in the range of 10 to 100 bytes, for example, the file size of a particular state file may be 10 bytes, 50 bytes, 100 bytes, or the like. In some embodiments, the gatekeeper may support a file filtering function, for example, the gatekeeper may filter the status file in the second storage device, so that the gatekeeper may only read the status file in the second storage device whose file size is smaller than or equal to a specific byte, which may be in a range of 10 to 100 bytes, for example, which may be 10 bytes, 50 bytes, or 100 bytes, etc. The state file in the specific byte is ferried from the internal network to the external network through the network gate, and even under the condition of information leakage, little data is leaked, so that the data safety is influenced as little as possible.
In some embodiments, the file extension (filename extension) of a particular state file may be ". stat". The file corresponding to the file extension ". stat" is a file that can be read by the gatekeeper. The first amount information in the specific state file may be in json (javascript Object notification) format. In other embodiments, the file extension of the specific state file may be other, or the first amount of information may be encapsulated in other formats, which is not limited in this embodiment.
The network gate is an information safety device which uses a solid-state switch read-write medium with multiple control functions to connect two independent host systems. Because two independent host systems are isolated through the gatekeeper, no physical connection, logical connection and information transmission protocol for communication exist between the systems, no information exchange according to the protocol exists, and only no protocol ferry is performed in a data file form.
The data files stored in the second storage device may be data files stored in a second specific directory in the second storage device. The second specific directory may be a preset directory.
Without specific description, the file extension of the data file in the embodiment of the present application may be ". bak", and the file corresponding to ". bak" may be a backup file. Without being particularly described, the data file may be a file having a file size smaller than or equal to a specific size, and the specific size may be a predefined size.
S202, under the condition that the first quantity information is smaller than the first specific quantity, pushing a data file to the first storage device; the data file in the first storage device is used for: the network gate reads and ferries the data to a second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
And the pushed data file is read by the gateway and ferred to the second storage device, so that the data receiving device can read the pushed data file from the second storage device.
The first specific number may be pre-stored in the data transmitting device or the first storage device. The first specific amount may be determined according to a storage capacity size of the second storage device, or according to a storage capacity size under a second specific directory in the second storage device. In this embodiment, the storage size occupied by the data file in the second storage device or in the second specific directory of the second storage device may be predicted by the first quantity information, for example, the size of the data file is defined to be less than or equal to a Megabytes (MB), and when the first quantity information is B, the data sending device may determine the storage size occupied by the data file in the second specific directory of the second storage device to be less than or equal to a × B Megabytes. By determining whether the first quantity information is smaller than the first specific quantity, it can be determined whether the storage size occupied by the data files in the second specific directory in the second storage device is smaller than a specific storage size. If the occupied storage size is larger than or equal to the specific storage size, the remaining storage space of the second storage device is smaller, and the data sending device can stop pushing the data file to the first storage device; if the occupied storage size is smaller than the specific storage size, the remaining storage space of the second storage device is larger, and the data sending device can continue to push the data file to the first storage device.
The gatekeeper can transport a specific state file or data file from one side to the other side in the case of ferrying the specific state file or data file, or can transport a copied file to the other side after copying the specific state file or data file on one side.
In the embodiment of the application, the data sending device can determine the first quantity information of the data files in the second storage device of the data receiving party, so that the second storage device can receive the data files under the condition that the first quantity information is smaller than the first specific quantity, the data files can be pushed to the first storage device, and the situation that the data files are easy to lose due to the fact that the data files are pushed to the first storage device when the second storage device does not receive the data files under the condition that the first quantity information is smaller than the first specific quantity information is avoided.
In some implementation scenarios, in order to ensure data security as much as possible, the system may prohibit the gatekeeper from ferrying data from the internal network to the external network, in this case, in order to avoid data loss as much as possible, the data transmission method may be implemented by the embodiment corresponding to fig. 3:
fig. 3 is a schematic implementation flow diagram of another data transmission method provided in the embodiment of the present application, and as shown in fig. 3, the method is applied to a data sending device, and the method includes:
s301, determining second quantity information of the data files stored in the first storage device.
In some embodiments, the data transmitting device may determine the second quantity information every second duration. The first duration of the interval between reading a particular state file may be greater than or equal to the second duration. The second time period may be between 5 seconds and 1 minute, for example, the second time period may be 5 seconds, 10 seconds, 30 seconds, 1 minute, or the like. In other embodiments, the data transmitting device may determine the second amount of information by determining different interval durations according to different actual scenarios. The interval duration of the second quantity of information may be determined to be positively correlated with the interval duration of reading the specific status file.
S302, pushing data files to a first storage device under the condition that the second quantity information is smaller than a second specific quantity, wherein the data files in the first storage device are used for: the network gate reads and ferries the data to a second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
And the pushed data file is read by the gateway and ferred to the second storage device, so that the data receiving device can read the pushed data file from the second storage device.
The second specific number may be pre-stored in the data transmitting device or the first storage device. The second specific amount may be determined based on a storage capacity size of the first storage device, or based on a storage capacity size of the first storage device under the first specific directory. In the embodiment of the present application, the storage size occupied by the data file in the first storage device or in the first specific directory of the first storage device may be predicted through the second quantity information. The first specific number and the second specific number may be the same or different. The first specific number and the second specific number may be the same number.
The second quantity information is smaller than the second specific quantity, and may indicate that the remaining storage capacity in the first storage device or the first specific directory of the first storage device is larger, at this time, the data sending device may continue to push the data file to the first storage device. Conversely, the second quantity information is greater than or equal to the second specific quantity, which may indicate that the remaining storage capacity in the first storage device or the first specific directory of the first storage device is smaller, and at this time, the data sending device may stop pushing the data file to the first storage device.
In the application embodiment, the data sending device may push the data file to the first storage device based on the information of the first number of data files in the first storage device connected to the data sending device, and the data file in the first storage device may determine that the status of the data sender is good, so that it may be avoided that the file is pushed to the first storage device when there are many data files in the first storage device of the data sender, which may easily cause the data amount in the first storage device to exceed the storage capacity thereof, thereby causing the situation that the first storage device is easily damaged and the data file is easily lost.
The embodiment corresponding to fig. 3 may also have an application scenario as follows: under the condition that the gatekeeper has a fault, the gatekeeper stops ferrying the data files in the first storage device to the second storage space, so that more and more data files in the first storage device are caused, and the situation that the data is lost due to insufficient storage capacity of the first storage device caused by continuously pushing the data files to the first storage device can be avoided by the mode of judging the second quantity information of the data files in the first storage device in the embodiment of the application.
Based on the embodiment corresponding to fig. 3, the present application may further provide a data transmission method, in which the data sending device may determine not only the second amount information but also second connection state information between the data sending device and the first storage device. The time interval of the second connection state information determined by the data transmission device may be greater than, less than, or equal to the time interval of the second amount information.
The second connection state information may characterize whether the data transmission device is connected to the first storage device. The data sending device may push the data file to the first storage device upon determining that the second amount information is less than the second specific amount and that the second connection state information characterizes a connection between the data sending device and the first storage device.
In some embodiments, the data sending device may stop pushing the data file to the first storage device if it is determined that the second amount of information is greater than or equal to a second specific amount, or the second connection state information characterizes a disconnection between the data sending device and the first storage device.
In some embodiments, the connection between the data sending device and the first storage device may be determined when the data sending device is able to read a second amount of information or a particular state file or other data.
The embodiment corresponding to fig. 2 and the embodiment corresponding to fig. 3 both provide a scheme that the data sending device pushes the data files to the first storage device by using the number information of the data files in the storage device being less than a certain number. The present application further provides a data transmission method corresponding to a data receiving device, please refer to the embodiment corresponding to fig. 4.
Fig. 4 is a schematic implementation flow diagram of another data transmission method provided in the embodiment of the present application, and as shown in fig. 4, the method is applied to a data receiving device, and the method includes:
s401, determining first quantity information of stored data files from second storage equipment; the data file in the second storage device is read from the first storage device by the gatekeeper and ferred.
The data receiving device may determine the first amount of information every first time period. In other embodiments, the data receiving device may determine the first amount of information by determining different interval durations according to different actual scenarios. The data receiving device may determine first quantity information for data files in a second particular directory in the second storage device.
S402, generating a specific state file based on the first quantity information.
The data receiving device may create a first temporary file; the first temporary file is a file which can not be read by the gatekeeper; then writing the first quantity information into the first temporary file to obtain a second temporary file; and finally, converting the format of the second temporary file to obtain a specific state file.
The first temporary file may be an empty file or the first temporary file may be a file with certain attribute parameters. The file extension of the first temporary file may be ". stat. In other embodiments, the file extension of the first temporary file may be other, and the present application is not limited thereto, as long as the file with the file extension cannot be read by the gatekeeper. The first temporary file may be created on the data receiving device or the second storage device.
The data receiving device may assemble the first amount of information into a JSON format to write into the first temporary file, thereby obtaining a second temporary file. The extension of the second temporary file is the same as the extension of the first temporary file. After obtaining the second temporary file, the data reception apparatus may modify the file extension ". stat.part" of the second temporary file to ". stat", thereby obtaining the specific state file.
S403, storing the specific state file into the second storage device, where the specific state file stored in the second storage device is used to: the gatekeeper reads and ferries to a first storage device, and a specific state file in the first storage device is used for: the data transmission device reads.
And the specific state file in the second storage device is read by the gateway and ferried to the first storage device, so that the data sending device pushes the data file to the first storage device based on the first amount of information.
The data receiving device may generate the specific state file every third duration, and then upload the specific state file to the second storage device, where the third duration may be less than or equal to the first duration. In some embodiments, before uploading the currently obtained specific state file, the data receiving device checks whether the originally uploaded state file with the file extension name of ". stat" exists in the second storage device, and if so, the ". stat" state file may be deleted. In other embodiments, the data receiving device may view the state file with the file extension of ". stat" on the second storage device at certain time intervals, and only keep the latest state file, and delete the state files other than the latest state file.
In the embodiment of the application, the data receiving device can generate the specific state file based on the first amount of information, and when the specific state file is stored in the second storage device, the specific state file can be acquired by the data sending device, so that the data sending device can determine that the second storage device has the ability to receive the data file under the condition that the first amount of information in the specific state file is smaller than the first specific amount, and then push the data file to the first storage device, thereby avoiding the occurrence of the condition that the data file is easy to lose due to the fact that the data file is also pushed to the first storage device when the second storage device does not have the ability to receive the data file.
Fig. 5 is a schematic flow chart of an implementation of another data transmission method provided in the embodiment of the present application, and as shown in fig. 5, the method includes:
s501, the data receiving device determines first quantity information of the stored data files from the second storage device.
The data receiving device may determine the first quantity information from the second storage device based on the FTP protocol.
S502, the data receiving device generates a specific state file based on the first quantity information.
S503, the data receiving device stores the specific state file into the second storage device.
The data receiving device may store a specific state file into the second storage device based on the FTP protocol.
In some embodiments, prior to S503, the data receiving device may determine whether a state file exists in the second storage device, and if not, perform S503, and if so, may determine at least one third state file stored in the second storage device, delete the at least one third state file, the third state file including the first amount of information.
In some embodiments, the first amount information in the third state file may characterize the amount of data files currently stored in the second storage device at the time the third state file was generated or created.
S504, the gatekeeper reads the specific state file stored in the second storage device.
And S505, the network brake ferries the specific state file to a second storage device.
S506, the data sending device reads the specific state file from the first storage device.
The specific state file includes a first amount of information. The data transmission device may read the specific state file from the first storage device based on the FTP protocol.
In general, only one first state file exists on the first storage device, and the data sending device may determine that the first state file is a specific state file when the first storage device stores one first state file, where the first state file includes the first amount information. In this case, the first state file and the specific state file are the same file.
In some cases, it may happen that there are multiple second state files on the first storage device, for example, the gatekeeper fails to operate, the gatekeeper is unstable to operate, or other reasons. The data sending device may determine, in a case where the first storage device stores at least two second state files, time information of each of the at least two second state files, the time information including: file creation time information or file generation time information, the second state file including first quantity information; and determining the latest second state file of the at least two second state files as the specific state file based on the time information. In this case, the latest second state file is the same file as the specific state file.
The first amount information included in the different second state files may be the same or different. In some embodiments, the first amount information included in each second state file may characterize the amount of data files currently stored in the second storage device at the time the each second state file was generated or created. Different second state files may be generated or created at different times.
In some embodiments, the attribute category information of the files in the particular state file, the first state file, and the second state file may be the same, and the attribute category information may include: a data file quantity attribute, and/or a connection state attribute between the data receiving device and the second storage device, and the like.
Any one of the specific state file, the first state file, and the second state file in the embodiment of the present application may all include: the number information of the data files in the second storage device, or, may both include: the number information of the data files in the second storage device and the connection state information between the data receiving device and the second storage device. In this embodiment of the application, the number information of the data files in different state files of the at least two state files may be the same or different, or the connection state information may be the same or different.
In this embodiment, in the case that there are at least two second state files in the first storage device, the second state file with the latest time most indicates the state information of the data receiving side, so that to reduce the calculation amount of the data sending side, the data sending device may determine the state of the data receiving side only by the second state file with the latest time.
In some embodiments, after S506, the data transmission device may delete one first state file or at least two second state files in the first storage device.
By deleting the first state file or the at least two second state files on the first storage device, the storage occupation of the first storage device can be reduced, and the data sending device is not easily affected when the state files are read next time.
In some embodiments, after S506, the data transmitting device may determine whether the first amount of information is less than a first specific amount. After S506, steps S507 to S508, or steps S509 to S513 may be executed.
S507, the data transmission device determines that the first quantity information is greater than or equal to the first specific quantity.
S508, the data sending device stops pushing the data file to the first storage device.
S509, the data transmission device determines that the first quantity information is smaller than the first specific quantity.
S510, the data sending device pushes the data file to the first storage device.
The data sending device may push the data file to the first storage device based on the FTP protocol.
S511, the gatekeeper reads the data file in the first storage device.
And S512, the network brake ferry data file is sent to a second storage device.
S513, the data receiving device reads the data file in the second storage device.
The data receiving device may read the data file in the second storage device based on the FTP protocol.
The specific state files and data files on the first storage device or the second storage device may be under the same directory, or may be under different directories.
In some embodiments, at least one of the following files is a file transfer protocol, FTP, file: a specific state file, a data file stored in the first storage device, a data file stored in the second storage device.
Therefore, at least one of the files transmitted through the gateway is the FTP file, and the FTP file has the characteristic of high safety, so that the safety of the data transmission process is improved.
In the embodiment of the application, the specific state file includes the first amount of information, so that the data sending device can determine the storage condition of the second storage device based on the first amount of information when obtaining the specific state file, and thus can adaptively push the data file to the first storage device according to the storage condition.
Fig. 6 is a schematic flow chart of an implementation of a data transmission method according to another embodiment of the present application, as shown in fig. 6, the method includes:
s601, the data receiving device determines first quantity information of the stored data files from the second storage device, and determines first connection state information between the data receiving device and the second storage device.
The data receiving device may determine that the first status information characterizes a connection between the data receiving device and the second storage device, in case the data receiving device is able to read the first amount of information or other data from the second storage device, indicating that the connection between the data receiving device and the second storage device is normal. The data receiving device may determine that the first status information indicates that the data receiving device is disconnected from the second storage device in a case where the first amount of information is not available from the second storage device indicating that the connection between the data receiving device and the second storage device is not normal. In a case where the data receiving apparatus is not able to obtain the first amount information from the second storage apparatus, the data receiving apparatus may determine that the first amount information is empty.
S602, the data receiving device generates a specific state file based on the first quantity information and the first connection state information.
In some embodiments, the data receiving device may create a first temporary file; the first temporary file is a file which can not be read by the gatekeeper; writing the first quantity information and the first connection state information into a first temporary file to obtain a second temporary file; and converting the format of the second temporary file to obtain a specific state file.
In other embodiments, the data receiving device may generate a target file, and write the first amount information and the first connection state information into the target file to obtain a specific state file; wherein the target file is a file capable of being read by the gatekeeper.
S603, the data receiving device stores the specific state file into the second storage device.
S604, the gatekeeper reads the specific state file stored in the second storage device.
S605, the network brake ferries the specific state file to the second storage device.
S606, the data sending device reads the specific state file from the first storage device.
The specific state file includes first quantity information and first connection state information.
In some embodiments, after S606, the data transmitting device may determine whether the first amount information is less than a first specific amount, and the first connection state information characterizes whether a connection between the data receiving device and the second storage device is present. After S606, the steps of S607 to S608, or the steps of S609 to S613 may be performed.
S607, the data sending device determines that the first quantity information is greater than or equal to the first specific quantity, or the first connection state information represents that the data receiving device and the second storage device are disconnected.
S608, the data sending device stops pushing the data file to the first storage device.
S609, the data sending device determines that the first quantity information is smaller than the first specific quantity, and the first connection state information represents the connection between the data receiving device and the second storage device.
In some embodiments, the data sending device may read the specific state file from the first storage device every first time period; the data sending equipment can determine second quantity information and second connection state information every second time length; wherein the first duration is greater than or equal to the second duration.
In this way, the data sending device can regularly determine the state of the data sending party and the state of the data receiving party, so that whether the state of the data receiving party and the state of the data sending party are normal or not can be continuously determined, and in the case that any state is abnormal, the data file sending to the first storage device is stopped, so that the data pushing can be effectively carried out. In addition, the state of the data sender is easy to determine, so that the state of the data sender can be determined at a higher frequency, and the hidden danger of data insecurity can be improved because the data receiver sends a state file to the data sender, so that the state of the data receiver can be determined at a lower frequency, and the safety of data transmission and the accuracy of state determination can be considered at the same time.
S610, the data sending equipment pushes the data file to the first storage equipment.
S611, the gatekeeper reads the data file in the first storage device.
And S612, the network gate ferry data file is sent to a second storage device.
S613, the data receiving device reads the data file in the second storage device.
In the embodiment of the application, the data sending device continues to push the data file to the first storage device under the condition that the first connection state represents that the data receiving device is connected with the second storage device, and stops pushing the data file to the first storage device under the condition that the first connection state represents that the data receiving device is disconnected from the second storage device, so that the situation that the data receiving device cannot read the data file from the second storage device and also pushes the data file to the first storage device, the occupied space of the data file of the second storage device is more and more, and finally the data cannot be stored is avoided.
Fig. 7 is a schematic flow chart of an implementation of a data transmission method according to another embodiment of the present application, as shown in fig. 7, the method includes:
s701, the data receiving device determines first quantity information of the stored data files from the second storage device, and determines first connection state information between the data receiving device and the second storage device.
S702, the data receiving device generates a specific state file based on the first quantity information and the first connection state information.
S703, the data receiving device stores the specific state file into the second storage device.
S704, the gatekeeper reads the specific state file stored in the second storage device.
S705, the network gate ferries the specific state file to the second storage device.
S706, the data sending device reads the specific state file from the first storage device.
The specific state file includes first quantity information and first connection state information.
In some embodiments, after S706, the data transmitting device may determine whether the first amount information is less than a first specific amount, and the first connection state information characterizes whether a connection between the data receiving device and the second storage device is present. After S706, steps S707 to S708, or steps S709 to S712, or steps S709 to S710, and S713 to S717 may be executed.
S707, the data sending device determines that the first quantity information is greater than or equal to a first specific quantity, or the first connection state information represents that the data receiving device and the second storage device are disconnected.
S708, the data sending device stops pushing the data file to the first storage device.
S709, the data sending device determines that the first quantity information is smaller than the first specific quantity, and the first connection state information represents a connection between the data receiving device and the second storage device.
S710, the data sending device determines second quantity information of the data files stored in the first storage device and second connection state information between the data sending device and the first storage device.
S711, the data transmitting device determines that the first quantity information is greater than or equal to a first specific quantity, or the first connection state information represents that the data receiving device is disconnected from the second storage device.
And S712, the data sending device stops pushing the data file to the first storage device.
S713, the data sending device determines that the first quantity information is smaller than the first specific quantity, and the first connection state information represents a connection between the data receiving device and the second storage device.
S714, the data sending device pushes the data file to the first storage device.
S715, the gatekeeper reads the data file in the first storage device.
And S716, the network brake ferry data file is sent to a second storage device.
S717, the data receiving device reads the data file in the second storage device.
In the embodiment of the application, the data file is pushed to the first storage device under the condition that the following conditions are all met. Stopping pushing the data file to the first storage device in the event that one of: in the case where the first amount information is smaller than the first specific amount; in the case that the first connection state information characterizes a connection between the data receiving device and the second storage device; in the case where the second quantity information is smaller than the second specific quantity; in case the second connection state information characterizes a connection between the data transmitting device and the first storage device.
In this embodiment of the application, the data sending device may push the data file to the first storage device when it is determined that the first connection state information represents the connection between the data receiving device and the second storage device, thereby avoiding a situation that the data file cannot be pushed to the first storage device due to disconnection between the data receiving device and the second storage device.
Fig. 8 is a schematic flow chart of an implementation of a data transmission method according to still another embodiment of the present application, and as shown in fig. 8, the method is applied to a data receiving device, and the method includes:
s801, reading the number of files with the suffix of.bak on the direct-connection FTP server.
The suffix can be understood the same as the file extension.
And S802, creating a stat file on the FTP server.
And S803, writing the file number with the determined suffix of.bak into the.stat.part file.
S804, the stage.
And S805, reading a file list with a suffix of.stat on the FTP server.
S806, the latest file of the stat on the FTP server is reserved, and the files of other stats except the latest file of the stat are deleted.
In some embodiments, S806 may be performed before S802.
Fig. 9 is a schematic implementation flow diagram of another data transmission method according to another embodiment of the present application, and as shown in fig. 9, the method is applied to a data sending device, and the method includes:
and S901, reading the number of files with the suffix of bak on the direct-connection FTP server.
S902, reading a file list with a suffix of stat on the direct-connected FTP server, and acquiring a file with a latest suffix of stat to obtain first quantity information and first connection state information of a data receiver.
S903, obtaining second quantity information and second connection state information of the data sender, comparing the smaller one of the first quantity information and the second quantity information with a threshold value, continuing data file pushing if the smaller one is smaller than the threshold value, and stopping data file pushing if the smaller one is larger than or equal to the threshold value.
And S904, under the condition that the first connection state and the second connection state are both connected, continuing to push the data file, and under the condition that one of the first connection state and the second connection state is disconnected, stopping pushing the data file.
S905, deleting the files with the suffixes of 'stat'.
S906, under the condition that the pushing of the data file is stopped, if the smaller one of the first quantity information and the second quantity information is smaller than the threshold value and the first connection state and the second connection state are both connected, the pushing of the data file is resumed.
In the embodiment of the application, the system has a regular state detection mechanism, the pushing of the data file can be automatically stopped when the system is automatically found to be unsuitable for pushing the data file, and the pushing of the data file can be automatically recovered when the system is automatically found to be recovered. In addition, the detection program of the outer network can simultaneously detect the states of the inner network and the outer network, and a relatively comprehensive protection system is formed.
Based on the foregoing embodiments, the present application provides a data transmission apparatus, where the apparatus includes units and modules included in the units, and the data transmission apparatus and the modules may be implemented by a processor in a data transmission device/a data reception device; of course, it may be implemented by a specific logic circuit. The processor here may be the same processor as the processor 1301 described below.
Fig. 10 is a schematic structural diagram of a data transmission device according to an embodiment of the present application, and as shown in fig. 10, the data transmission device 1000 includes: a reading unit 1001, configured to read a specific state file from a first storage device, where the specific state file includes first quantity information, and the specific state file is read from a second storage device by a gatekeeper and ferred to the first storage device; the first quantity information is used for representing the quantity of the data files stored in the second storage device; a pushing unit 1002, configured to push a data file to the first storage device if the first amount information is smaller than the first specific amount; the data file in the first storage device is used for: the network gate reads and ferries the data to a second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
In some embodiments, the specific state file further comprises: first connection state information between the data receiving device and the second storage device; the pushing unit 1002 is further configured to push the data file to the first storage device when the first quantity information is smaller than the first specific quantity and the first connection state information indicates that the data receiving device is connected to the second storage device.
In some embodiments, the data transmission apparatus 1000 further comprises: a determining unit 1003 for determining second quantity information of the data files stored in the first storage device and second connection state information between the data transmitting device and the first storage device; the pushing unit 1002 is further configured to push a data file to the first storage device when the first quantity information is smaller than the first specific quantity, the first connection state information indicates that the data receiving device is connected to the second storage device, the second quantity information is smaller than the second specific quantity, and the second connection state information indicates that the data sending device is connected to the first storage device.
In some embodiments, the pushing unit 1002 is further configured to stop pushing the data file to the first storage device in case of one of the following; in the case where the first amount information is greater than or equal to the first specific amount; in the case that the first connection state information characterizes a disconnection between the data receiving device and the second storage device; in the case where the second amount information is greater than or equal to the second specific amount; in case the second connection state information characterizes a disconnection between the data transmitting device and the first storage device.
In some embodiments, the reading unit 1001 is further configured to read the specific status file from the first storage device every first duration; determining second quantity information and second connection state information every second duration; wherein the first duration is greater than or equal to the second duration.
In some embodiments, the determining unit 1003 is further configured to determine, in a case that one first status file is stored in the first storage device, that the first status file is a specific status file, where the first status file includes the first amount information; in the case that at least two second state files are stored in the first storage device, determining time information of each of the at least two second state files, the time information including: file creation time information or file generation time information, the second state file including first quantity information; and determining the latest second state file of the at least two second state files as the specific state file based on the time information.
In some embodiments, the data transmission apparatus 1000 further comprises: a deleting unit 1004 configured to delete one first state file or at least two second state files in the first storage device.
In some embodiments, at least one of the following files is a file transfer protocol, FTP, file: a specific state file, a data file stored in the first storage device, a data file stored in the second storage device.
Fig. 11 is a schematic structural diagram of another data transmission device according to an embodiment of the present application, and as shown in fig. 11, a data transmission device 1100 includes: a determining unit 1101 for determining second quantity information of the data files stored in the first storage device; a pushing unit 1102, configured to, in a case that the second amount information is smaller than the second specific amount, push a data file to the first storage device, where the data file in the first storage device is used to: the network gate reads and ferries the data to a second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
Fig. 12 is a schematic structural diagram of another data transmission device according to an embodiment of the present application, and as shown in fig. 12, a data transmission device 1200 includes: a determining unit 1201, configured to determine first quantity information of the stored data files from the second storage device; the data file in the second storage device is read from the first storage device by the gatekeeper and ferred; a generating unit 1202 for generating a specific state file based on the first amount information; a storage control unit 1203, configured to store the specific state file into the second storage device, where the specific state file in the second storage device is used for: the gatekeeper reads and ferries to a first storage device, and a specific state file in the first storage device is used for: the data sending device pushes the data file to the first storage device based on the first amount of information in the specific state file.
In some embodiments, the determining unit 1201 is further configured to determine first connection state information between the data receiving apparatus and the second storage apparatus; the generating unit 1202 is further configured to generate a specific state file based on the first quantity information and the first connection state information.
In some embodiments, the generating unit 1202 is further configured to create a first temporary file; the first temporary file is a file which can not be read by the gatekeeper; writing the first quantity information and the first connection state information into a first temporary file to obtain a second temporary file; and converting the format of the second temporary file to obtain a specific state file.
In some embodiments, the determining unit 1201 is further configured to determine the first amount information and the first connection state information every first duration.
In some embodiments, the data transmission apparatus 1200 further comprises: a deleting unit 1204, configured to determine at least one third status file stored in the second storage device, and delete the at least one third status file, where the third status file includes the first quantity information.
The above description of the apparatus embodiments, similar to the above description of the method embodiments, has similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.
It should be noted that, in the embodiment of the present application, if the data transmission method is implemented in the form of a software functional module and sold or used as a standalone product, the data transmission method may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be substantially or partially embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a data transmitting apparatus/data receiving apparatus to execute all or part of the methods of the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
It should be noted that fig. 13 is a schematic diagram of a hardware entity of a data transmission apparatus according to an embodiment of the present application, and as shown in fig. 13, the hardware entity of the data transmission apparatus 1300 includes: a processor 1301 and a memory 1302, wherein the memory 1302 stores a computer program operable on the processor 1301, and the processor 1301 executes the program to implement the steps of the method of any of the embodiments described above. The data transmission device 1300 in the embodiment of the present application may be a data transmission device or a data reception device.
The Processor 1301 may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understood that the electronic device implementing the above-mentioned processor function may be other electronic devices, and the embodiments of the present application are not particularly limited.
The Memory 1302 stores a computer program operable on the processor, and the Memory 1302 is configured to store instructions and applications executable by the processor 1301, and may also buffer data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by each module in the processor 1301 and the data transmission apparatus 1300, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).
The processor 1301 implements the steps of the data transmission method of any one of the above when executing the program. The processor 1301 generally controls the overall operation of the data transmission apparatus 1300.
Embodiments of the present application provide a computer-readable storage medium, which stores one or more programs, where the one or more programs are executable by one or more processors to implement the steps of the data transmission method according to any one of the above embodiments.
Here, it should be noted that: the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.
The computer storage medium/Memory may be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic Random Access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM), and the like.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment of the present application" or "a previous embodiment" or "some embodiments" or "some implementations" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrase "one embodiment" or "an embodiment of the present application" or "the preceding embodiments" or "some implementations" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
In a case where no specific description is given, the data transmitting apparatus/the data receiving apparatus may perform any step in the embodiments of the present application, and the step may be performed by a processor of the data transmitting apparatus/the data receiving apparatus. Unless otherwise specified, the embodiment of the present application does not limit the sequence of steps executed by the data sending device/the data receiving device. In addition, the data may be processed in the same way or in different ways in different embodiments. It should be further noted that any step in this embodiment of the present application may be executed independently by the data sending device/the data receiving device, that is, when the data sending device/the data receiving device executes any step in the foregoing embodiment, it may not depend on the execution of other steps.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of a unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment. In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The methods disclosed in the several method embodiments provided in the present application may be combined arbitrarily without conflict to obtain new method embodiments. Features disclosed in several of the product embodiments provided in the present application may be combined in any combination to yield new product embodiments without conflict.
The features disclosed in the several method or apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new method embodiments or apparatus embodiments.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.
Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be substantially implemented or portions thereof that contribute to the related art may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.
In the embodiments of the present application, the descriptions of the same steps and the same contents in different embodiments may be mutually referred to. In the embodiment of the present application, the term "and" does not affect the order of the steps, for example, the data sending device/the data receiving device performs a and performs B, where the data sending device/the data receiving device performs a first and then performs B, or the data sending device/the data receiving device performs B first and then performs a, or the data sending device/the data receiving device performs B while performing a.
The above description is only for the embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (19)

1. A data transmission method, applied to a data transmission apparatus, the method comprising:
reading a specific state file from a first storage device, wherein the specific state file comprises first quantity information, and the specific state file is read from a second storage device by a gatekeeper and ferred to the first storage device; the first quantity information is used for representing the quantity of the data files stored in the second storage device;
pushing a data file to the first storage device if the first amount information is less than a first specific amount; the data file in the first storage device is used for: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
2. The method of claim 1, wherein the specific state file further comprises: first connection state information between the data receiving device and the second storage device;
the pushing a data file to the first storage device in the case that the first amount information is less than a first specific amount includes:
and under the condition that the first quantity information is smaller than the first specific quantity and the first connection state information represents the connection between the data receiving equipment and the second storage equipment, pushing a data file to the first storage equipment.
3. The method of claim 2, further comprising:
determining second quantity information of the data files stored in the first storage device and second connection state information between the data sending device and the first storage device;
the pushing a data file to the first storage device under the condition that the first quantity information is smaller than the first specific quantity and the first connection state information represents the connection between the data receiving device and the second storage device includes:
and pushing a data file to the first storage device under the condition that the first quantity information is smaller than the first specific quantity, the first connection state information represents the connection between the data receiving device and the second storage device, the second quantity information is smaller than the second specific quantity, and the second connection state information represents the connection between the data sending device and the first storage device.
4. The method of claim 3, further comprising:
stopping pushing data files to the first storage device in one of the following cases;
in a case where the first amount information is greater than or equal to the first specific amount;
in the event that the first connection status information characterizes a disconnection between the data receiving device and the second storage device;
in a case where the second number information is greater than or equal to the second specific number;
and in the case that the second connection state information represents disconnection between the data transmission device and the first storage device.
5. The method according to claim 3 or 4, characterized in that the method further comprises:
reading the specific state file from the first storage device every a first time interval;
determining the second quantity information and the second connection state information every second duration; wherein the first duration is greater than or equal to the second duration.
6. The method according to any one of claims 1 to 5, further comprising:
determining that a first state file is the specific state file under the condition that the first state file is stored in the first storage device, wherein the first state file comprises the first quantity information;
in a case where at least two second state files are stored in the first storage device, determining time information of each of the at least two second state files, the time information including: file creation time information or file generation time information, the second state file including the first amount information;
determining a latest second state file of the at least two second state files as the specific state file based on the time information.
7. The method of claim 6, wherein after reading the particular state file from the first storage device, the method further comprises:
deleting the one first state file or the at least two second state files in the first storage device.
8. The method according to any of claims 1 to 7, characterized in that at least one of the following files is a file transfer protocol, FTP, file:
the specific state file, the data file stored in the first storage device, and the data file stored in the second storage device.
9. A data transmission method, applied to a data transmission apparatus, the method comprising:
determining second quantity information of the data files stored in the first storage device;
pushing a data file to the first storage device under the condition that the second amount information is smaller than a second specific amount, wherein the data file in the first storage device is used for: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
10. A data transmission method applied to a data receiving device, the method comprising:
determining first quantity information of the stored data files from the second storage device; the data file in the second storage device is read from the first storage device by the gatekeeper and ferred;
generating a specific state file based on the first quantity information;
storing the specific state file into the second storage device, wherein the specific state file in the second storage device is used for: the gatekeeper reads and ferries to the first storage device, and a specific state file in the first storage device is used for: the data sending device pushes the data file to the first storage device based on the first amount of information in the specific state file.
11. The method of claim 10, further comprising: determining first connection state information between the data receiving device and the second storage device;
generating a specific state file based on the first amount of information comprises:
generating the specific state file based on the first quantity information and the first connection state information.
12. The method of claim 11, wherein generating the particular state file based on the first quantity information and the first connection state information comprises:
creating a first temporary file; the first temporary file is a file which cannot be read by the gatekeeper;
writing the first quantity information and the first connection state information into the first temporary file to obtain a second temporary file;
and converting the format of the second temporary file to obtain the specific state file.
13. The method according to claim 11 or 12, characterized in that the method further comprises:
determining the first quantity information and the first connection state information every first time length.
14. The method according to any of claims 10 to 12, wherein prior to storing the particular state file in the second storage device, the method further comprises:
determining at least one third state file stored in the second storage device, and deleting the at least one third state file, wherein the third state file comprises the first quantity information.
15. A data transmission apparatus, characterized in that the apparatus comprises:
the reading unit is used for reading a specific state file from a first storage device, wherein the specific state file comprises first quantity information, and the specific state file is read from a second storage device by a gateway and ferred to the first storage device; the first quantity information is used for representing the quantity of the data files stored in the second storage device;
the pushing unit is used for pushing the data files to the first storage device under the condition that the first quantity information is smaller than a first specific quantity; the data file in the first storage device is used for: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
16. A data transmission apparatus, characterized in that the apparatus comprises:
a determining unit configured to determine second quantity information of the data files stored in the first storage device;
a pushing unit, configured to push a data file to the first storage device when the second amount information is smaller than a second specific amount, where the data file in the first storage device is used to: the network gate reads and ferries to the second storage device; the data file in the second storage device is used for: and reading by the data receiving device.
17. A data transmission apparatus, characterized in that the apparatus comprises:
a determining unit configured to determine first quantity information of the stored data files from the second storage device; the data file in the second storage device is read from the first storage device by the gatekeeper and ferred;
a generating unit configured to generate a specific state file based on the first amount information;
a storage control unit, configured to store the specific state file into the second storage device, where the specific state file in the second storage device is used to: the gatekeeper reads and ferries to the first storage device, and a specific state file in the first storage device is used for: the data sending device pushes the data file to the first storage device based on the first amount of information in the specific state file.
18. A data transmission device, characterized in that the device comprises: a memory and a processor, wherein the processor is capable of,
the memory stores a computer program operable on the processor,
the processor implementing the steps of the method of any one of claims 1 to 8 or implementing the steps of the method of claim 9 when executing the program,
alternatively, the steps in the method of any one of claims 10 to 14 are implemented.
19. A computer storage medium storing one or more programs executable by one or more processors to perform the steps of the method of any one of claims 1 to 8 or to perform the steps of the method of claim 9 to perform the steps of the method of any one of claims 10 to 14.
CN202011295099.4A 2020-11-18 2020-11-18 Data transmission method, device and equipment and computer storage medium Withdrawn CN112492001A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011295099.4A CN112492001A (en) 2020-11-18 2020-11-18 Data transmission method, device and equipment and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011295099.4A CN112492001A (en) 2020-11-18 2020-11-18 Data transmission method, device and equipment and computer storage medium

Publications (1)

Publication Number Publication Date
CN112492001A true CN112492001A (en) 2021-03-12

Family

ID=74931381

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011295099.4A Withdrawn CN112492001A (en) 2020-11-18 2020-11-18 Data transmission method, device and equipment and computer storage medium

Country Status (1)

Country Link
CN (1) CN112492001A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080005121A1 (en) * 2006-06-30 2008-01-03 Moka5, Inc. Network-extended storage
CN103929455A (en) * 2013-01-15 2014-07-16 北京东方通科技股份有限公司 Asynchronous file transmission system and method comprising network storage equipment
CN106506510A (en) * 2016-11-18 2017-03-15 江苏方天电力技术有限公司 Dynamic vibration signal data inter-network lock Transmission system and its method
CN108092971A (en) * 2017-12-13 2018-05-29 新华三信息安全技术有限公司 A kind of method and device of processing business message
CN110489484A (en) * 2019-07-11 2019-11-22 视联动力信息技术股份有限公司 Method of data synchronization, device, readable storage medium storing program for executing and electronic equipment
CN111200624A (en) * 2018-11-19 2020-05-26 浙江宇视科技有限公司 Cross-gatekeeper data transmission method and system
CN111355752A (en) * 2018-12-20 2020-06-30 阿里巴巴集团控股有限公司 File transmission method, device and equipment based on gatekeeper

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080005121A1 (en) * 2006-06-30 2008-01-03 Moka5, Inc. Network-extended storage
CN103929455A (en) * 2013-01-15 2014-07-16 北京东方通科技股份有限公司 Asynchronous file transmission system and method comprising network storage equipment
CN106506510A (en) * 2016-11-18 2017-03-15 江苏方天电力技术有限公司 Dynamic vibration signal data inter-network lock Transmission system and its method
CN108092971A (en) * 2017-12-13 2018-05-29 新华三信息安全技术有限公司 A kind of method and device of processing business message
CN111200624A (en) * 2018-11-19 2020-05-26 浙江宇视科技有限公司 Cross-gatekeeper data transmission method and system
CN111355752A (en) * 2018-12-20 2020-06-30 阿里巴巴集团控股有限公司 File transmission method, device and equipment based on gatekeeper
CN110489484A (en) * 2019-07-11 2019-11-22 视联动力信息技术股份有限公司 Method of data synchronization, device, readable storage medium storing program for executing and electronic equipment

Similar Documents

Publication Publication Date Title
CN109739810B (en) File synchronization method, server, client and device with storage function
CN103733187A (en) Token based file operations
KR20150111952A (en) Method and system for using a recursive event listener on a node in hierarchical data structure
CN108667635B (en) Disaster recovery processing method, equipment and system
US10558547B2 (en) Methods for proactive prediction of disk failure in a RAID group and devices thereof
CN102468982B (en) Element terminal and communication system
KR102563888B1 (en) Method, apparatus and computer program for deduplicating data frame
CN112925759A (en) Data file processing method and device, storage medium and electronic device
CN115277727B (en) Data disaster recovery method, system, device and storage medium
JP2013251588A (en) Network camera system
JP2019515595A (en) Data transmission method, apparatus and system
KR20120030938A (en) Method of data replication in a distributed data storage system and corresponding device
KR20170125252A (en) Message Fragmentation Method using a MQTT Protocol in M2M/IoT Platforms
CN112492001A (en) Data transmission method, device and equipment and computer storage medium
CN113330426A (en) Method, device and system for backing up data
CN111211993A (en) Incremental persistence method and device for streaming computation
JP5913196B2 (en) Computer system and fault identification method
JPH1127342A (en) Method and system for matching network information
CN110708355A (en) File uploading method, system, device and readable storage medium
JPWO2016129275A1 (en) Information processing apparatus, log management system, log management method, and program
CN110737543A (en) method, device and storage medium for recovering distributed file system data
CN115145473A (en) Data processing method, device and equipment between heterogeneous storage media
JP2010211494A (en) Information processing system, message management method, and message management program
US8086567B2 (en) Apparatus and method for storing data
JP2015177481A (en) Monitoring camera apparatus and monitoring camera system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20210312

WW01 Invention patent application withdrawn after publication