CN114238240B - Distributed multi-cluster data storage method and device and storage medium - Google Patents
Distributed multi-cluster data storage method and device and storage medium Download PDFInfo
- Publication number
- CN114238240B CN114238240B CN202210131409.1A CN202210131409A CN114238240B CN 114238240 B CN114238240 B CN 114238240B CN 202210131409 A CN202210131409 A CN 202210131409A CN 114238240 B CN114238240 B CN 114238240B
- Authority
- CN
- China
- Prior art keywords
- file
- stored
- file data
- server
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Abstract
The invention discloses a distributed multi-cluster data storage method, a device and a storage medium, wherein the method comprises the following steps: acquiring file data to be stored, and searching file data matched with the file data to be stored in the cluster server according to the file data to be stored; if the cluster server does not have file data matched with the file data to be stored, encrypting the file to be stored, and storing the encrypted file data to be stored in the target server; and generating a shortcut file link for downloading the file to be stored, and storing the shortcut file link into a stable node in the cluster server. According to the invention, the data of the file to be stored is compared and checked for duplication, and then is stored in the target server, so that the storage efficiency is saved. And the shortcut file link of the file to be stored is stored in the stable node in the cluster server, so that the data between the servers can be conveniently and rapidly searched and downloaded.
Description
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a distributed multi-cluster data storage method, device, and storage medium.
Background
The cluster storage means: the cluster for storage is composed of a plurality of 'universal storage devices', and the performance and the capacity of each storage system composing the cluster storage can be superposed and expanded in a 'cluster' mode. The traditional data storage method is limited by the physical composition (such as controller performance, bus performance, the number of disk drives, the number of connected servers, memory size, the performance of a NAS header, and the like) and functional limitations (such as the capacity of supporting a file system, the coupling of metadata and a data processing path, the number of snapshots or copies, and the like), so that a data storage bottleneck occurs, and the storage efficiency is affected.
Thus, there is a need for improvements and enhancements in the art.
Disclosure of Invention
The present invention provides a distributed multi-cluster data storage method, apparatus and storage medium, aiming at solving the above-mentioned drawbacks of the prior art, and aiming at solving the problem that the storage efficiency is affected by the limitation of physical composition to function in the data storage method in the prior art.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
in a first aspect, the present invention provides a distributed multi-cluster data storage method, where the method includes:
acquiring file data to be stored, and searching file data matched with the file data to be stored in a cluster server according to the file data to be stored;
if the cluster server does not have file data matched with the file data to be stored, encrypting the file data to be stored, and storing the encrypted file data to be stored in a target server;
and generating a shortcut file link for downloading the file data to be stored, and storing the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node.
In one implementation manner, the obtaining file data to be stored, and searching file data in a cluster server, which is matched with the file data to be stored, according to the file data to be stored includes:
acquiring the file data to be stored, and extracting file information from the file data to be stored, wherein the file information comprises: file name, keyword, file type, file data volume;
and searching the cluster server for the file data matched with the file information according to the file information.
In one implementation manner, the searching, in the cluster server, for the file data matching the file information according to the file information includes:
and searching the cluster server for file data with the same file information according to the file information.
In an implementation manner, if the cluster server does not have the file data to be stored, the encrypting the file data to be stored includes:
if the cluster server does not have file data matched with the file data to be stored, generating a storage key, wherein the storage key carries a unique storage identifier;
and binding the storage key with the file data to be stored to realize encryption processing.
In one implementation manner, the storing the encrypted file data to be stored in the target server includes:
acquiring a storage space of each server in the cluster server;
screening out the server with the largest storage space, and taking the server with the largest storage space as the target server;
and storing the encrypted file data to be stored in the target server.
In one implementation manner, the generating a shortcut file link for downloading the file data to be stored and storing the shortcut file link into a stable node in the cluster server includes:
generating the shortcut file link according to the address of the target server, wherein the shortcut file link is used for downloading the file data to be stored;
storing the shortcut file link into a stable node in the cluster server;
and storing the shortcut file link into each server through the data storage module of the stable node of each server, and ending the journey.
In one implementation, the method further comprises:
and if the cluster server has file data matched with the file data to be stored, prompting that the file data to be stored is uploaded, and ending the stroke.
In a second aspect, an embodiment of the present invention further provides a distributed multi-cluster data storage device, where the device includes:
the file data searching module is used for acquiring file data to be stored and searching file data matched with the file data to be stored in the cluster server according to the file data to be stored;
the file data storage module is used for encrypting the file data to be stored and storing the encrypted file data to be stored into a target server if the cluster server does not have file data matched with the file data to be stored;
and the file link generation module is used for generating a shortcut file link for downloading the file data to be stored and storing the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node.
In a third aspect, an embodiment of the present invention further provides a terminal device, where the terminal device includes a memory, a processor, and a distributed multi-cluster data storage program that is stored in the memory and is executable on the processor, and when the processor executes the distributed multi-cluster data storage program, the step of implementing the distributed multi-cluster data storage method in any one of the above solutions is implemented.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores thereon a distributed multi-cluster data storage program, and when the distributed multi-cluster data storage program is executed by a processor, the steps of the distributed multi-cluster data storage method in any one of the above solutions are implemented.
Has the advantages that: compared with the prior art, the invention provides a distributed multi-cluster data storage method, which comprises the steps of obtaining file data to be stored, and searching file data matched with the file data to be stored in a cluster server according to the file data to be stored; if the cluster server does not have file data matched with the file data to be stored, encrypting the file data to be stored, and storing the encrypted file data to be stored in a target server; and generating a shortcut file link for downloading the file data to be stored, and storing the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node. According to the invention, the file data to be stored is compared and checked for duplication and then stored in the target server, so that the storage efficiency is greatly saved, and the storage space is saved. And a quick file link of a file to be stored is generated and stored in a stable node in the cluster server, so that the data between the servers can be conveniently and quickly searched and downloaded.
Drawings
Fig. 1 is a flowchart of a specific implementation of a distributed multi-cluster data storage method according to an embodiment of the present invention.
Fig. 2 is a schematic block diagram of a distributed multi-cluster data storage device according to an embodiment of the present invention.
Fig. 3 is a schematic block diagram of an internal structure of a terminal device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and effects of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the invention provides a distributed multi-cluster data storage method. In specific implementation, the embodiment acquires file data to be stored, and searches file data matched with the file data to be stored in a cluster server according to the file data to be stored; if the cluster server does not have file data matched with the file data to be stored, encrypting the file data to be stored, and storing the encrypted file data to be stored in a target server; and generating a shortcut file link for downloading the file data to be stored, and storing the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node. According to the invention, the file data to be stored is compared and checked for duplication and then stored in the target server, so that the storage efficiency is greatly saved, and the storage space is saved. And a quick file link of a file to be stored is generated and stored in a stable node in the cluster server, so that the data between the servers can be conveniently and quickly searched and downloaded.
The distributed multi-cluster data storage method of the embodiment can be applied to terminal equipment, and the terminal equipment can be intelligent products such as computers, mobile phones and smart televisions. As shown in fig. 1, the distributed multi-cluster data storage method in this embodiment includes the following steps:
step S100, file data to be stored is obtained, and file data matched with the file data to be stored in a cluster server is searched according to the file data to be stored.
In this embodiment, file data to be stored is first obtained, where the file data to be stored is selected by a user through a terminal device to be file data that needs to be uploaded to a server. After the file data to be stored is acquired, in this embodiment, whether file data matching the file data to be stored exists or not is searched in the cluster server. The distributed cluster server in this embodiment is composed of a plurality of servers connected to each other via a network, and employs an open architecture, a distributed operating system, and a unified namespace.
In an implementation manner, in this embodiment, when determining whether the cluster server has the file data to be stored, the method includes the following steps:
step S101, obtaining the file data to be stored, and extracting file information from the file data to be stored, wherein the file information comprises: file name, keyword, file type, file data volume;
and S102, searching file data matched with the file information in the cluster server according to the file information.
In specific implementation, in this embodiment, first, file data to be stored is obtained, and then file information is extracted from the file data to be stored, where the file information includes: file name, keywords, file type, file data size. Then, the terminal device searches whether the file data of the same file information exists in the cluster server based on the file information so as to determine whether the file data to be stored needs to be stored. In another implementation manner, the terminal device in this embodiment may also generate a corresponding check code for the file data to be stored, and determine whether the same check code exists in a check code cluster in the cluster storage server through the check code. Specifically, the check code comprises a file name, a keyword, a file type and a file size type signal, the check code is sent to the cluster server through the terminal device, whether the same check code exists in the cluster server is searched, and the check code is fed back to the terminal device.
Step S200, if the cluster server does not have file data matched with the file data to be stored, encrypting the file data to be stored, and storing the encrypted file data to be stored in a target server.
After the terminal device searches whether the same file information exists in the cluster server based on the file information, it is determined that file data matched with the file data to be stored does not exist in the cluster server, and at this time, the file to be stored needs to be stored. In this embodiment, the file data to be stored is first encrypted, and then the encrypted file data to be stored is stored in the target server. Therefore, in this embodiment, when storing the file data to be stored in the target server, it is first determined whether the file data to be stored exists in the cluster server, and only when the file data to be stored does not exist in the cluster server, the file data to be stored is stored in the target server.
In an implementation manner, the embodiment includes the following steps when encrypting file data to be stored:
step S201, if the cluster server does not have file data matched with the file data to be stored, generating a storage key, wherein the storage key carries a unique storage identifier;
and S202, binding the storage key and the file data to be stored to realize encryption processing.
In specific implementation, if the cluster server does not have file data matched with the file data to be stored, a storage key is generated, the storage key has a unique storage identifier, and the storage key is a secret key, so that the file data can be downloaded, decrypted and used conveniently. After the terminal device generates the storage key, the embodiment binds the storage key with the file data to be stored, and performs encryption processing. Since the file data to be stored and the storage key are bound in the embodiment, in the subsequent decryption process, the storage key can be used for decrypting the file data to be stored only if the file data to be stored and the storage key have a binding relationship. In another implementation manner, if the cluster server has file data matching the file data to be stored, it is prompted that the file data to be stored has been uploaded, and the process is ended.
After encrypting the file data to be stored in this embodiment, the file data to be stored after the encryption processing is uploaded to the target server, which specifically includes the following steps:
step S21, obtaining the storage space of each server in the cluster server;
step S22, screening out the server with the largest storage space, and taking the server with the largest storage space as the target server;
and step S23, storing the encrypted file data to be stored into the target server.
In this embodiment, the terminal device first obtains the storage space of each server in the cluster server, then compares the storage spaces of the servers, screens out the server with the largest storage space, and takes the server with the largest storage space as the target server; and finally, storing the encrypted file data to be stored into the target server. That is to say, the file data to be stored is uploaded to the server with the largest storage space for storage in the embodiment, so that the storage space is reasonably utilized, and the storage efficiency is improved. In addition, when the file data to be stored is uploaded to the target server, the storage key is also uploaded to the target server.
Step S300, a shortcut file link used for downloading the file data to be stored is generated, and the shortcut file link is stored in a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node.
In this embodiment, after the file data to be stored is uploaded to the target server, a shortcut file link for downloading the file data to be stored is generated in this embodiment, where the shortcut file link carries a keyword, so that the shortcut file link is convenient to manage, and then the shortcut file link is stored in a stable node in the cluster server. The shortcut file link can be used for positioning the storage address of the file data to be stored so as to download the file data to be stored in the subsequent steps. In addition, the shortcut file is stored in a stable node in the cluster server in a linked manner, the stable node is a transition node and is equivalent to a data transmitter, so that each server in the cluster server can acquire the file data to be stored through the stable node.
In an implementation manner, the step S300 specifically includes the following steps:
step S301, generating the shortcut file link according to the address of the target server, wherein the shortcut file link is used for downloading the file data to be stored;
step S302, the shortcut file is stored in a stable node in the cluster server in a linked mode;
and step S303, storing the shortcut file link into each server through the data storage module of the stable node of each server, and ending the journey.
Specifically, the shortcut file link in this embodiment is used for downloading the file data to be stored, and therefore the shortcut file link needs to carry an address of the target server. And then, the terminal equipment transmits a quick file link and a file storage key of the file data to be stored to a stable node of the cluster server, wherein the quick file link is a download link convenient for file extraction, and each server is a cluster storage system, so that the file data to be stored of other servers can be downloaded and shared through any server.
Therefore, in the embodiment, the file data to be stored is firstly acquired, and the file data matched with the file data to be stored in the cluster server is searched according to the file data to be stored. If the cluster server does not have file data matched with the file data to be stored, encrypting the file data to be stored, and storing the encrypted file data to be stored in a target server. And finally, generating a shortcut file link for downloading the file data to be stored, and storing the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node. According to the embodiment, the file data to be stored are compared and the duplicate is checked, and then the file data are stored in the target server, so that the storage efficiency is greatly saved, and the storage space is saved. And a quick file link of a file to be stored is generated and stored in a stable node in the cluster server, so that the data between the servers can be conveniently and quickly searched and downloaded.
Based on the foregoing embodiments, the present invention further provides a distributed multi-cluster data storage apparatus, as shown in fig. 2, the apparatus includes: a file data search module 10, a file data storage module 20, and a file link generation module 30. Specifically, the file data searching module 10 is configured to obtain file data to be stored, and search file data in the cluster server, which is matched with the file data to be stored, according to the file data to be stored. The file data storage module 20 is configured to, if there is no file data matching the file data to be stored in the cluster server, encrypt the file data to be stored, and store the encrypted file data to be stored in a target server. The file link generating module 30 is configured to generate a shortcut file link for downloading the file data to be stored, and store the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node.
In one implementation, the file data search module 10 includes:
a data obtaining unit, configured to obtain the file data to be stored, and extract file information from the file data to be stored, where the file information includes: file name, keyword, file type, file data volume;
and the data matching unit is used for searching the file data matched with the file information in the cluster server according to the file information.
In one implementation, the data matching unit includes:
and the data comparison unit is used for searching the file data with the same file information in the cluster server according to the file information.
In one implementation, the file data storage module 20 includes:
the storage key generation unit is used for generating a storage key if the cluster server does not have file data matched with the file data to be stored, and the storage key carries a unique storage identifier;
and the encryption processing unit is used for binding the storage key and the file data to be stored so as to realize encryption processing.
In one implementation, the encryption processing unit includes:
a storage space obtaining subunit, configured to obtain a storage space of each server in the cluster servers;
the server determining subunit is used for screening out the server with the largest storage space and taking the server with the largest storage space as the target server;
and the encryption processing subunit is used for storing the encrypted file data to be stored into the target server.
In one implementation, the file link generation module 30 includes:
the link generation unit is used for generating the shortcut file link according to the address of the target server, and the shortcut file link is used for downloading the file data to be stored;
the link storage unit is used for storing the shortcut file into a stable node in the cluster server in a link mode;
and the link sharing unit is used for storing the shortcut file link into each server through the data storage module of the stable node of each server and finishing the journey.
In one implementation, the apparatus further comprises:
and the prompting module is used for prompting that the file data to be stored is uploaded and ending the stroke if the file data matched with the file data to be stored exists in the cluster server.
The working principle of each module in the distributed multi-cluster data storage device in this embodiment is the same as that of the distributed multi-cluster data storage method in the above embodiment, and is not described herein again.
Based on the above embodiments, the present invention further provides a terminal device, and a schematic block diagram thereof may be as shown in fig. 3. The terminal equipment comprises a processor and a memory which are connected through a system bus. Wherein the processor of the terminal device is configured to provide computing and control capabilities. The memory of the terminal equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the terminal device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a distributed multi-cluster data storage method.
It will be understood by those skilled in the art that the block diagram shown in fig. 3 is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation to the terminal device to which the solution of the present invention is applied, and a specific terminal device may include more or less components than those shown in the figure, or may combine some components, or have different arrangements of components.
In one embodiment, a terminal device is provided, where the terminal device includes a memory, a processor, and a distributed multi-cluster data storage program stored in the memory and executable on the processor, and when the processor executes the distributed multi-cluster data storage program, the following operation instructions are implemented:
acquiring file data to be stored, and searching file data matched with the file data to be stored in a cluster server according to the file data to be stored;
if the cluster server does not have file data matched with the file data to be stored, encrypting the file data to be stored, and storing the encrypted file data to be stored in a target server;
and generating a shortcut file link for downloading the file data to be stored, and storing the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, operational databases, or other media used in embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double-rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
In summary, the present invention discloses a distributed multi-cluster data storage method, apparatus and storage medium, the method comprising: acquiring file data to be stored, and searching file data matched with the file data to be stored in the cluster server according to the file data to be stored; if the cluster server does not have file data matched with the file data to be stored, encrypting the file to be stored, and storing the encrypted file data to be stored in the target server; and generating a shortcut file link for downloading the file to be stored, and storing the shortcut file link into a stable node in the cluster server. According to the invention, the data of the file to be stored is compared and checked for duplication, and then is stored in the target server, so that the storage efficiency is saved. And the shortcut file link of the file to be stored is stored in the stable node in the cluster server, so that the data between the servers can be conveniently and rapidly searched and downloaded.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (5)
1. A method for distributed multi-cluster data storage, the method comprising:
acquiring file data to be stored, and searching file data matched with the file data to be stored in a cluster server according to the file data to be stored;
if the cluster server does not have file data matched with the file data to be stored, encrypting the file data to be stored, and storing the encrypted file data to be stored in a target server;
generating a shortcut file link for downloading the file data to be stored, and storing the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node;
the acquiring file data to be stored and searching the file data matched with the file data to be stored in the cluster server according to the file data to be stored comprises the following steps:
acquiring the file data to be stored, and extracting file information from the file data to be stored, wherein the file information comprises: file name, keyword, file type and file data volume;
searching file data matched with the file information in the cluster server according to the file information;
or the terminal equipment further generates a corresponding check code for the file data to be stored, and judges whether the same check code exists in a check code cluster in the cluster storage server or not through the check code, wherein the check code comprises a file name, a keyword, a file type and a file size type signal;
sending the check code to a cluster server through the terminal equipment, searching whether the same check code exists in the cluster server or not, and feeding back the check code to the terminal equipment;
searching the cluster server for the file data matched with the file information according to the file information, wherein the searching comprises the following steps:
searching file data with the same file information in the cluster server according to the file information; if the cluster server does not have the file data to be stored, encrypting the file data to be stored, including:
if the cluster server does not have file data matched with the file data to be stored, generating a storage key, wherein the storage key carries a unique storage identifier;
binding the storage key with the file data to be stored to realize encryption processing;
the storing the encrypted file data to be stored in the target server includes:
acquiring a storage space of each server in the cluster server;
screening out the server with the largest storage space, and taking the server with the largest storage space as the target server;
storing the encrypted file data to be stored into the target server;
when file data to be stored is uploaded to a target server, a storage key is uploaded to the target server;
the generating a shortcut file link for downloading the file data to be stored and storing the shortcut file link to a stable node in the cluster server includes:
generating the shortcut file link according to the address of the target server, wherein the shortcut file link is used for downloading the file data to be stored, and the shortcut file link carries a keyword and is used for managing the shortcut file link;
storing the shortcut file link into a stable node in the cluster server;
the shortcut file is linked and stored into each server through a data storage module of the stable node of each server, and the journey is finished;
the stable nodes are transition nodes, which are equivalent to data transmitters, and each server in the cluster server acquires file data to be stored through the stable nodes.
2. The distributed, multi-cluster data storage method of claim 1, further comprising:
and if the cluster server has file data matched with the file data to be stored, prompting that the file data to be stored is uploaded, and ending the stroke.
3. A distributed, multi-cluster data storage apparatus, the apparatus comprising:
the file data searching module is used for acquiring file data to be stored and searching file data matched with the file data to be stored in the cluster server according to the file data to be stored;
the file data storage module is used for encrypting the file data to be stored and storing the encrypted file data to be stored into a target server if the cluster server does not have file data matched with the file data to be stored;
the file link generation module is used for generating a shortcut file link for downloading the file data to be stored, and storing the shortcut file link into a stable node in the cluster server, so that each server in the cluster server obtains the shortcut file link through the stable node;
the file data search module comprises:
a data obtaining unit, configured to obtain the file data to be stored, and extract file information from the file data to be stored, where the file information includes: file name, keyword, file type, file data volume;
the data matching unit is used for searching the file data matched with the file information in the cluster server according to the file information;
or the terminal equipment further generates a corresponding check code for the file data to be stored, and judges whether the same check code exists in a check code cluster in the cluster storage server or not through the check code, wherein the check code comprises a file name, a keyword, a file type and a file size type signal;
sending the check code to a cluster server through the terminal equipment, searching whether the same check code exists in the cluster server, and feeding back the check code to the terminal equipment;
the data matching unit includes:
the data comparison unit is used for searching the file data with the same file information in the cluster server according to the file information;
the file data storage module includes:
the storage key generation unit is used for generating a storage key if the cluster server does not have file data matched with the file data to be stored, and the storage key carries a unique storage identifier;
the encryption processing unit is used for binding the storage key and the file data to be stored so as to realize encryption processing;
the encryption processing unit includes:
a storage space obtaining subunit, configured to obtain a storage space of each server in the cluster servers;
the server determining subunit is used for screening out the server with the largest storage space and taking the server with the largest storage space as the target server;
the encryption processing subunit is configured to store the encrypted file data to be stored in the target server;
when file data to be stored is uploaded to a target server, a storage key is uploaded to the target server;
the file link generation module comprises:
the link generation unit is used for generating the shortcut file link according to the address of the target server, wherein the shortcut file link is used for downloading the file data to be stored, and the shortcut file link carries a keyword and is used for managing the shortcut file link;
the link storage unit is used for storing the shortcut file into a stable node in the cluster server in a link mode;
the link sharing unit is used for storing the shortcut file link into each server through the data storage module of the stable node of each server and ending the journey;
the stable nodes are transition nodes, which are equivalent to data transmitters, and each server in the cluster server acquires file data to be stored through the stable nodes.
4. A terminal device, characterized in that the terminal device comprises a memory, a processor and a distributed multi-cluster data storage program stored in the memory and operable on the processor, and when the processor executes the distributed multi-cluster data storage program, the steps of the distributed multi-cluster data storage method according to any one of claims 1-2 are implemented.
5. A computer-readable storage medium, on which a distributed multi-cluster data storage program is stored, which, when executed by a processor, carries out the steps of the distributed multi-cluster data storage method according to any one of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210131409.1A CN114238240B (en) | 2022-02-14 | 2022-02-14 | Distributed multi-cluster data storage method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210131409.1A CN114238240B (en) | 2022-02-14 | 2022-02-14 | Distributed multi-cluster data storage method and device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114238240A CN114238240A (en) | 2022-03-25 |
| CN114238240B true CN114238240B (en) | 2022-05-27 |
Family
ID=80747703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210131409.1A Active CN114238240B (en) | 2022-02-14 | 2022-02-14 | Distributed multi-cluster data storage method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114238240B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114840488B (en) * | 2022-07-04 | 2023-05-02 | 柏科数据技术（深圳）股份有限公司 | Distributed storage method, system and storage medium based on super fusion structure |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105025053A (en) * | 2014-04-24 | 2015-11-04 | 苏宁云商集团股份有限公司 | Distributed file upload method based on cloud storage technology and system |
| CN107852173A (en) * | 2015-06-15 | 2018-03-27 | 阿斯卡瓦公司 | Fetch and search based on keyword and fetch to performing multi-dimensional search, relevance using the data of the lossless simplification of master data screen |
| CN108255875A (en) * | 2016-12-29 | 2018-07-06 | 北京奇虎科技有限公司 | Message is stored to the method and apparatus of distributed file system |
| CN108475508A (en) * | 2015-12-10 | 2018-08-31 | 阿斯卡瓦公司 | The simplification of audio data and the data being stored in block processing storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110267060B (en) * | 2019-07-24 | 2021-03-26 | 北京视界云天科技有限公司 | Video file storage injection method and device, computer equipment and storage medium |
-
2022
- 2022-02-14 CN CN202210131409.1A patent/CN114238240B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105025053A (en) * | 2014-04-24 | 2015-11-04 | 苏宁云商集团股份有限公司 | Distributed file upload method based on cloud storage technology and system |
| CN107852173A (en) * | 2015-06-15 | 2018-03-27 | 阿斯卡瓦公司 | Fetch and search based on keyword and fetch to performing multi-dimensional search, relevance using the data of the lossless simplification of master data screen |
| CN108475508A (en) * | 2015-12-10 | 2018-08-31 | 阿斯卡瓦公司 | The simplification of audio data and the data being stored in block processing storage system |
| CN108255875A (en) * | 2016-12-29 | 2018-07-06 | 北京奇虎科技有限公司 | Message is stored to the method and apparatus of distributed file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114238240A (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4047487A1 (en) | File storage method, terminal, and storage medium | |
| CN108133008B (en) | Method, device, equipment and storage medium for processing service data in database | |
| CN110689349B (en) | Transaction hash value storage and searching method and device in blockchain | |
| CN107797817B (en) | Application updating method and device | |
| CN111831208B (en) | Information processing method and device, terminal equipment and storage medium | |
| CN105337941A (en) | Method and device for providing device identifier | |
| CN112015674A (en) | Cache resource access method and device based on multiple hierarchies and computer equipment | |
| CN107528917B (en) | File storage method and device | |
| US20200153889A1 (en) | Method for uploading and downloading file, and server for executing the same | |
| CN104937904A (en) | Copy offload for disparate offload providers | |
| EP4020265A1 (en) | Method and device for storing encrypted data | |
| CN113273163A (en) | File uploading method, file downloading method and file management device | |
| CN114238240B (en) | Distributed multi-cluster data storage method and device and storage medium | |
| EP4357950A1 (en) | Device management method, system and apparatus | |
| CN117421771A (en) | Structured data-oriented searchable encryption method and device and electronic equipment | |
| CN116011013A (en) | Database access method and apparatus, computing device and computer program product | |
| CN116192395A (en) | Trusted system for distributed data storage | |
| CN112416526B (en) | Direct storage access method, device and related equipment | |
| CN115098519A (en) | Data storage method and device | |
| CN111884818B (en) | Data file processing method, system, server and storage medium | |
| US11763038B2 (en) | Secured file storage | |
| US10296760B2 (en) | System of shared secure data storage and management | |
| CN112783866A (en) | Data reading method and device, computer equipment and storage medium | |
| CN113300833B (en) | Key management method and device | |
| CN114793156B (en) | Data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |