CN109254956B - Data downloading method and device and electronic equipment - Google Patents
Data downloading method and device and electronic equipment Download PDFInfo
- Publication number
- CN109254956B CN109254956B CN201810844119.5A CN201810844119A CN109254956B CN 109254956 B CN109254956 B CN 109254956B CN 201810844119 A CN201810844119 A CN 201810844119A CN 109254956 B CN109254956 B CN 109254956B
- Authority
- CN
- China
- Prior art keywords
- file
- shared
- storage
- network storage
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
Abstract
The embodiment of the invention relates to the technical field of network storage equipment, and discloses a data storage method, a data downloading device and electronic equipment. Establishing a shared storage group with at least one other network storage device, wherein all the network storage devices in the shared storage group are provided with a shared space; receiving a storage request, wherein the storage request carries a file to be stored; storing a file to be stored in network storage equipment; storing the copy of the file to be stored to the shared space of other network storage devices in the shared storage group; receiving a file reading request; identifying whether a file to be read matched with the file reading request exists in the network storage equipment or not; if yes, judging whether the file to be read is invalid; if the file is not invalid, directly reading the file to be read from the network storage equipment; and if the file to be read fails, reading the copy of the file to be read from other network storage equipment of the shared storage group. The invention reduces the storage cost and improves the safety of data storage.
Description
Technical Field
The embodiment of the invention relates to the technical field of network storage equipment, in particular to a data storage method, a data downloading device and electronic equipment.
Background
The cloud storage refers to a system which integrates a large number of storage devices of different types in a network through application software to cooperatively work through functions such as cluster application, network technology or a distributed file system and provides data storage and service access functions to the outside.
The cloud Storage is generally divided into a public cloud and a private cloud, the private cloud refers to a Network Storage device exclusive to a certain organization or an individual, the private cloud Storage can be applied to a region, an enterprise, or even a home, for example, a home NAS (Network Attached Storage) product, each user in the home independently configures the Network Storage device, and each user can only access the Network Storage device of the user.
In the process of implementing the invention, the inventor finds that: in the private cloud, each user is a network storage device exclusive to the user, and once the storage space of the network storage device is damaged, data is permanently lost due to the fact that the data is not backed up.
Disclosure of Invention
The embodiment of the invention provides a data storage method for improving the safety of data storage, a data downloading method, a data downloading device and electronic equipment.
The embodiment of the invention discloses the following technical scheme:
in a first aspect, an embodiment of the present invention provides a data storage method, which is applied to a network storage device, and the method includes:
establishing a shared storage group with at least one other network storage device, wherein all the network storage devices in the shared storage group are provided with a shared space;
receiving a storage request, wherein the storage request carries a file to be stored;
storing the file to be stored in the network storage equipment;
and storing the copy of the file to be stored to the shared space of other network storage devices in the shared storage group.
Optionally, the storing the copy of the file to be stored in the shared space of the other network storage devices in the shared storage group includes:
dividing the copy of the file to be stored into a plurality of data blocks according to a preset erasure correction algorithm;
and respectively storing the data blocks into the shared spaces of other different network storage devices in the shared storage group.
Optionally, the method further comprises: carrying out encryption processing on the data block;
the step of respectively storing the data blocks into the shared spaces of other different network storage devices in the shared storage group specifically includes: and respectively storing the encrypted data blocks into shared spaces of other different network storage devices in the shared storage group.
Optionally, the method further comprises:
after the copies of the files to be stored are stored in the shared spaces of other network storage devices in the shared storage group, marking the files to be stored as shared files in the network storage devices;
detecting whether a shared file in the network storage device or a file located in a shared space of the network storage device has failed;
if the file is invalid, sending a file replacement request to the shared storage group;
receiving a replacement file returned by the shared storage group according to the file replacement request;
and replacing the failed file with the replacement file.
In a second aspect, an embodiment of the present invention provides a data downloading method, which is applied to a network storage device, where the network storage device and at least one other network storage device form a shared storage group, and all network storage devices in the shared storage group set a shared space, where the method includes:
receiving a file reading request;
identifying whether a file to be read matched with the file reading request exists in the network storage equipment or not;
if yes, judging whether the file to be read is invalid or not;
if the file is not invalid, directly reading the file to be read from the network storage equipment;
and if the file to be read fails, reading the copy of the file to be read from other network storage equipment of the shared storage group.
Optionally, the reading the copy of the file to be read from the other network storage devices of the shared storage group includes:
judging whether the file to be read positioned in the shared storage group is invalid or not;
if the failure occurs, returning and reading failure information;
and if the file is not invalid, extracting the file to be read from the shared storage group, and returning the file to be read.
Optionally, if the storage mode of the copy of the file to be read in the shared storage group is block storage;
reading copies of the file to be read from other network storage devices of the shared storage group comprises:
respectively extracting data blocks of the file to be read from other network storage devices in the shared storage group;
and integrating the extracted data blocks to form a complete copy of the file to be stored according to a preset erasure algorithm.
In a third aspect, an embodiment of the present invention provides a data storage apparatus, which is applied to a network storage device, and the apparatus includes:
the system comprises a group unit, a storage unit and a control unit, wherein the group unit is used for establishing a shared storage group with at least one other network storage device, and all the network storage devices in the shared storage group are provided with shared spaces;
the device comprises a receiving unit, a storing unit and a processing unit, wherein the receiving unit is used for receiving a storage request which carries a file to be stored;
the first storage unit is used for storing the file to be stored in the network storage equipment;
and the second storage unit is used for storing the copy of the file to be stored to the shared space of other network storage devices in the shared storage group.
Optionally, the second storage unit includes:
the dividing module is used for dividing the copy of the file to be stored into a plurality of data blocks according to a preset erasure correction algorithm;
and the shared storage module is used for respectively storing the data blocks into shared spaces of other different network storage devices in the shared storage group.
Optionally, the second storage unit further includes: an encryption module;
the encryption module is used for encrypting the data block;
the shared storage module is specifically configured to store the encrypted data blocks into shared spaces of other different network storage devices in the shared storage group, respectively.
Optionally, the apparatus further comprises:
the marking unit is used for marking the file to be stored as a shared file in the network storage equipment after the copy of the file to be stored is stored in the shared space of other network storage equipment in the shared storage group;
the detection unit is used for detecting whether the shared file in the network storage equipment or the file positioned in the shared space of the network storage equipment is invalid or not;
a sending unit, configured to send a file replacement request to the shared storage group if the file replacement request fails;
the second receiving unit is used for receiving a replacement file returned by the shared storage group according to the file replacement request;
and the file replacing unit is used for replacing the replacement file with the invalid file.
In a fourth aspect, an embodiment of the present invention provides a data downloading apparatus, which is applied to a network storage device, where the network storage device and at least one other network storage device form a shared storage group, and all network storage devices in the shared storage group set a shared space, and the apparatus includes:
a receiving unit configured to receive a file reading request;
the identification unit is used for identifying whether the file to be read matched with the file reading request exists in the network storage equipment or not;
the judging unit is used for judging whether the file to be read is invalid or not if the file to be read exists;
the first reading unit is used for directly reading the file to be read from the network storage equipment if the file is not invalid;
and the second reading unit is used for reading the copy of the file to be read from other network storage equipment of the shared storage group if the file to be read fails.
Optionally, the second reading unit is specifically configured to:
judging whether the file to be read positioned in the shared storage group is invalid or not;
if the failure occurs, returning and reading failure information;
and if the file is not invalid, extracting the file to be read from the shared storage group, and returning the file to be read.
Optionally, if the storage manner of the copy of the file to be read in the shared storage group is block storage, the second reading unit is specifically configured to:
respectively extracting data blocks of the file to be read from other network storage devices in the shared storage group;
and integrating the extracted data blocks to form a complete copy of the file to be stored according to a preset erasure algorithm.
In a fifth aspect, an embodiment of the present invention provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described above.
In a sixth aspect, embodiments of the present invention provide a computer program product comprising a computer program stored on a non-volatile computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the above-mentioned method.
In a seventh aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, where the computer-executable instructions are configured to cause a computer to perform the foregoing method.
The invention has the beneficial effects that: compared with the prior art, the embodiment of the invention provides a data storage method, a shared storage group is established by at least one other network storage device, a shared space is arranged in all the network storage devices in the shared storage group, when files to be stored need to be stored, the files to be stored are stored in the network storage devices, and copies of the files to be stored are also stored in the shared spaces of the other network storage devices in the shared storage group, so that the files to be stored are backed up, when the storage space of the network storage device of a user has a problem, the files can be recovered from the shared space, and the safety of data storage is improved.
Drawings
One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.
Fig. 1 is a schematic structural diagram of a shared memory group according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a data storage method according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for storing data according to another embodiment of the present invention;
fig. 4 is a flowchart illustrating a method for downloading data according to an embodiment of the present invention;
FIG. 5 is a flow chart illustrating an apparatus for storing data according to an embodiment of the present invention;
fig. 6 is a schematic flowchart of an apparatus for downloading data according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a shared memory group according to an embodiment of the present invention. As shown in fig. 1, the shared storage group 100 includes network storage devices 10 and shared spaces 20, at least two network storage devices 10 are included in the shared storage group 100, and the shared spaces 20 are set in the network storage devices 10, so that the number of the shared spaces 20 is the same as the number of the network storage devices 10 in the shared storage group 100.
It is understood that the network storage device 10 refers to a storage device that is owned by a user alone and is capable of connecting to a network. In the information age of big data, various information is stored in the form of data, and as personal data is more and more, the network storage device 10 for storing the personal data is also required to have more and more storage capacity, or the personal data can be stored in a plurality of network storage devices 10. The network storage device 10 is connected to an electronic device, and the electronic device may store data in the network storage device, read data from the network storage device, or edit data in the network storage device 10. For example, in the home NAS, a user may share data (such as photos, videos, or files) in the network storage device 10 with a family in a cloud manner.
The storage medium on the network storage device 10 may specifically be a device such as a USB flash disk (USB flash disk), an SSD, an emmc, an EMCP, a ufs, an SD card, and a mobile hard disk. In some embodiments, the storage medium on the network storage device 10 includes two parts, namely, a storage controller and a storage, and the storage controller and the storage are electrically connected. The functions of the memory controller may be implemented by a control chip, for example: and a single chip microcomputer. The function of the memory may be specifically realized by a memory chip, the memory chip may be a flash chip, and the flash type chip may be specifically a NAND flash chip or the like.
The shared space 20 refers to a storage space that is opened on the network storage device 10 to other network storage devices 10 in the shared storage group 100. It is understood that the shared space 20 is a part of the storage space on the network storage device 10, and is used for storing copies of files to be stored of other network storage devices 10 in the shared storage group 100. That is, each of the network storage devices 10 stores the file to be stored corresponding to the storage request, and copies (i.e., file backups) of the file to be stored are stored in the shared spaces 20 of other network storage devices 10 in the shared storage group 100.
In the embodiment of the present invention, all the network storage devices 10 in the shared storage group 100 are provided with the shared space 20, and the shared storage group 100 may be created by one user, each user has respective identification and network access authentication (including a login account, a password, and the like), and correspondingly, the network storage devices 10 owned by each user also have the same identification and network access authentication. Preferably, when a user invites another network storage device 10 to join the shared storage group 100, authentication information is required to be input, so that the other network storage device 10 can identify the identity of the user, and the other network storage device 10 receives request information of the user, where the request information includes user information and authentication information of the user, and clicks to confirm, to join the shared storage group 100.
It is understood that a user corresponding to any network storage device 10 in the shared storage group 100 may invite other network storage devices 10, and in order to ensure that users corresponding to each network storage device 10 in the shared storage group 100 know each other, a newly invited network storage device 10 needs to be joinable after verification of corresponding users of all network storage devices 10 existing in the shared storage group 100. In some embodiments, the network storage device 10 is associated with an electronic device (such as a mobile phone) frequently used by the user, and when the newly invited network storage device 10 does not receive a reply of passing the verification for more than a preset time, all users corresponding to the network storage devices 10 that are not verified in the shared storage group 100 are reminded to perform verification immediately through a short message or a pop-up window. If the network storage device 10 is not associated with an electronic device, when the network storage device 10 is used again, verification information may be sent to the electronic device connected thereto. When at least one user in the shared storage group 100 selects to reject the request of the user to join the shared storage group 100, in order to avoid the possibility of the user misoperation, the system will issue the verification request to the user who rejects the request again, perform the second confirmation, and improve the reliability of the shared storage group 100.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating a data storage method according to an embodiment of the invention. As shown in fig. 2, the data storage method is applied to a network storage device, and includes:
s11: and establishing a shared storage group with at least one other network storage device, wherein all the network storage devices in the shared storage group are provided with a shared space.
The network storage device refers to a network storage device exclusively owned by a certain user or a certain organization, and in some embodiments, the implementation manner of the network storage device exclusively owned by a certain user or a certain organization is as follows: so that a user or an organization may register as a user of the network storage device and users that are not registered are not allowed to access the network storage device.
It can be understood that: the same network storage device can create or join a plurality of different shared storage groups to realize shared storage with different network storage devices.
S12: receiving a storage request, wherein the storage request carries a file to be stored.
In some embodiments, the storage request carries an access address link of a file to be stored, so that the size of a data packet of the storage request can be reduced, and the data processing efficiency is improved. It should be noted that, before storing personal data, a user may select a storage mode for data to be stored, where the storage mode includes shared storage or non-shared storage, and when selecting non-shared storage, the personal data of the user is all stored in a non-shared space; when the shared storage is selected, the personal data of the user may be stored in the shared space or the unshared space.
S13: and storing the file to be stored in the network storage equipment.
It can be understood that, when the storage mode of non-shared storage is selected, the file to be stored can only be stored in the network storage device, and the file to be stored cannot be stored in other network storage devices of the shared storage group.
S14: and storing the copy of the file to be stored to the shared space of other network storage devices in the shared storage group.
Each network storage device in the shared storage group is equal, no primary and secondary points exist, and the copy of the file to be stored can be randomly stored to the shared space of other network storage devices in the shared storage group; the copy of the file to be stored can be preferentially stored in other network storage devices of the same local area network according to whether the network storage device receiving the storage request and other network storage devices in the shared storage group belong to the same local area network; when other network storage devices belong to the same local area network, the copy of the file to be stored can be preferentially stored in other network storage devices with larger available space of the shared space according to the distance from the network storage device receiving the storage request and the available space size of the shared space of other network storage devices.
In some embodiments, in order to ensure storage privacy of the network storage devices, each of the network storage devices is provided with a non-shared space in addition to a shared space, when a user does not want to share and store data, the data may be directly stored in the non-shared space and not stored in the shared space, that is: without performing step S14, for data in the unshared space, other network storage devices cannot access, i.e. are members of the same group and are not allowed.
In some embodiments, when storing the copy, a block storage manner may also be used, and step S14 includes: dividing the copy of the file to be stored into a plurality of data blocks according to a preset erasure correction algorithm; and respectively storing the data blocks into the shared spaces of other different network storage devices in the shared storage group.
The erasure code technology is one of two common data redundancy technologies in the storage field, is a forward error correction technology, is mainly applied to network transmission to avoid packet loss, and is used by a storage system to improve storage reliability. The erasure code technology mainly encodes original data through an erasure code algorithm to obtain redundancy, and stores the data and the redundancy together to achieve the purpose of fault tolerance. The basic idea is to obtain m redundant elements (also called check blocks) by calculating n original data elements to a certain extent. For the elements of the n + m blocks, when any element smaller than the m blocks is in error (including original data and redundant data), the original n blocks of data can be restored through a corresponding reconstruction algorithm. The process of generating the parity is referred to as encoding and the process of recovering the missing data blocks is referred to as decoding. The disk utilization is n/(n + m). Compared with the multiple methods, the method based on the erasure codes has the advantages of low redundancy, high disk utilization rate and the like.
Or, step S14 includes dividing the copy of the file to be stored into a plurality of data blocks according to a preset erasure correction algorithm; carrying out encryption processing on the data block; and respectively storing the encrypted data blocks into shared spaces of other different network storage devices in the shared storage group.
Because the network storage equipment is owned by the user independently, the user does not need to be encrypted, and other network storage equipment is owned by other users independently, when the user stores data in the shared space of other network storage equipment, encryption processing is needed, the situation that other users can directly check the data of the user is avoided, and privacy is improved.
In the embodiment of the invention, the shared storage group is established with at least one other network storage device, the shared storage space is set in all the network storage devices in the shared storage group, when the file to be stored needs to be stored, the file to be stored is stored in the network storage device, and the copy of the file to be stored is also stored in the shared space of the other network storage devices in the shared storage group, so that the backup of the file to be stored is realized, when the storage space of the network storage device of a user has a problem, the recovery from the shared space can be realized, and the safety of data storage is improved.
Referring to fig. 3, fig. 3 is a schematic flow chart of a data storage method according to another embodiment of the present invention, which is different from the foregoing embodiment in that the method further includes:
s15: and marking the file to be stored as a shared file in the network storage equipment.
S16: detecting whether the shared file in the network storage device or the file in the shared space of the network storage device is invalid.
S17: and if the file is invalid, sending a file replacement request to the shared storage group.
S18: and receiving a replacement file returned by the shared storage group according to the file replacement request.
S19: and replacing the failed file with the replacement file.
In summary, after the file to be stored is stored in the network storage device, and the copy of the file to be stored is stored in the shared space of other network storage devices in the shared storage group, whether the stored file to be stored is valid is detected, and the invalid file is replaced in time, so that the failure of the file to be stored caused by the possible reasons in the network is avoided, and the accuracy of data storage is improved.
Referring to fig. 4, fig. 4 is a flowchart illustrating a data downloading method according to an embodiment of the present invention. The data downloading method is applied to network storage equipment, the network storage equipment and at least one other network storage equipment form a shared storage group, all the network storage equipment in the shared storage group are provided with shared spaces, and the data downloading method comprises the following steps:
s21: a file read request is received.
Reading files from file storage objects (including storage spaces of network storage devices and other network storage devices) through an interface provided by an operating system, and it can be understood that instructions, programs and the like of file reading requests are generated by the operating system (corresponding operations are performed by a user), for example, double-clicking a target folder and the like.
S22: and identifying whether the file to be read matched with the file reading request exists in the network storage equipment.
S23: if yes, judging whether the file to be read is invalid or not.
And if the file to be read matched with the file reading request does not exist, the file to be read does not exist in the shared space of the network storage equipment or other network storage equipment of the shared storage group.
S241: and if the file to be read does not fail, directly reading the file to be read from the network storage equipment.
In some embodiments, the local read operation is initiated by a processor in which a cache of the network storage device resides. It can be understood that when a user is in a private cloud storage network, reading the file to be read from the network storage device is preferentially adopted, so that the bandwidth of data transmission is reduced, and the efficiency of data downloading is improved.
S242: and if the file to be read fails, reading the copy of the file to be read from other network storage equipment of the shared storage group.
If the storage mode of the copy of the file to be read in the shared storage group is whole file storage, step S242 includes determining whether the file to be read in the shared storage group is invalid; if the failure occurs, returning and reading failure information; and if the file is not invalid, extracting the file to be read from the shared storage group, and returning the file to be read.
If the storage mode of the copy of the file to be read in the shared storage group is block storage, reading the copy of the file to be read from other network storage devices of the shared storage group comprises: respectively extracting data blocks of the file to be read from other network storage devices in the shared storage group; and integrating the extracted data blocks to form a complete copy of the file to be stored according to a preset erasure algorithm.
It can be understood that, when the storage mode of the copy of the file to be read in the shared storage group is block storage, the step of determining whether the file to be read located in the shared storage group has failed is omitted, and due to the adoption of the erasure correction algorithm, even if a plurality of nodes in the shared storage group have failed, the data can be recovered as long as the number of failure points is within an allowable range, so that the reliability of the data in the storage process is improved, for example, the data is prevented from being lost, mistaken or damaged due to network factors.
Referring to fig. 5, fig. 5 is a schematic flowchart illustrating a data storage device according to an embodiment of the present invention. As shown in fig. 5, the apparatus 300 for storing data is applied to a network storage device, and includes:
the group unit 301 is configured to establish a shared storage group with at least one other network storage device, where all the network storage devices in the shared storage group are provided with a shared space.
A receiving unit 302, configured to receive a storage request, where the storage request carries a file to be stored.
The first storage unit 303 is configured to store the file to be stored in the network storage device.
A second storage unit 304, configured to store a copy of the file to be stored to a shared space of other network storage devices in the shared storage group.
The second storage unit 304 includes a partitioning module 3041, configured to partition a copy of the file to be stored into a plurality of data blocks according to a preset erasure correction algorithm; a shared storage module 3042, configured to store the data blocks into shared spaces of other different network storage devices in the shared storage group respectively.
In some embodiments, the second storage unit 304 further includes an encryption module 3043, where the encryption module 3043 is configured to encrypt the data block; the shared storage module 3042 is specifically configured to store the encrypted data blocks into shared spaces of other different network storage devices in the shared storage group, respectively.
In some embodiments, the apparatus 300 further comprises:
a marking unit 305, configured to mark the file to be stored as a shared file in the network storage device after the copy of the file to be stored is completely stored in the shared space of the other network storage devices in the shared storage group.
A detecting unit 306, configured to detect whether the shared file in the network storage device or the file located in the shared space of the network storage device has failed.
A sending unit 307, configured to send a file replacement request to the shared storage group if the file replacement request fails.
A second receiving unit 308, configured to receive a replacement file returned by the shared storage group according to the file replacement request.
A file replacing unit 309, configured to replace the failed file with the replacement file.
Since the concept of the data storage device is the same as that of the data storage method described in the above embodiments, the contents of the above embodiments may be referred to by embodiments of the data storage device without conflicting with each other, and are not described herein again.
Referring to fig. 6, fig. 6 is a schematic flowchart illustrating a data downloading apparatus according to an embodiment of the present invention. As shown in fig. 6, the apparatus 400 for downloading data is applied to a network storage device, where the network storage device and at least one other network storage device form a shared storage group, and all network storage devices in the shared storage group are provided with a shared space, and the apparatus 400 includes:
a receiving unit 401, configured to receive a file reading request.
An identifying unit 402, configured to identify whether a file to be read matching the file read request exists in the network storage device.
The determining unit 403 is configured to determine whether the file to be read is invalid or not if the file exists.
A first reading unit 404, configured to directly read the file to be read from the network storage device if the file is not invalid.
A second reading unit 405, configured to read a copy of the file to be read from another network storage device of the shared storage group if the copy fails.
If the storage manner of the copy of the file to be read in the shared storage group is the whole file storage, the second reading unit 405 is specifically configured to: judging whether the file to be read positioned in the shared storage group is invalid or not; if the failure occurs, returning and reading failure information; and if the file is not invalid, extracting the file to be read from the shared storage group, and returning the file to be read.
If the storage manner of the copy of the file to be read in the shared storage group is block storage, the second reading unit 405 is specifically configured to: respectively extracting data blocks of the file to be read from other network storage devices in the shared storage group; and integrating the extracted data blocks to form a complete copy of the file to be stored according to a preset erasure algorithm.
Since the concept of the data downloading apparatus is the same as that of the data downloading method described in the foregoing embodiments, the contents of the foregoing embodiments may be referred to in the embodiments of the data downloading apparatus under the condition that the contents do not conflict with each other, and details are not described herein.
As shown in fig. 7, fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device 600 may be an independent electronic device, or may be integrated into a module or unit in a storage controller of a network storage device. The electronic device 600 includes: one or more processors 601 and a memory 602, one processor 601 being taken as an example in fig. 7.
The processor 601 and the memory 602 may be connected by a bus or other means, such as the bus connection shown in fig. 7.
The memory 602 is used as a non-volatile computer-readable storage medium and can be used for storing non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the data storage method and the data downloading method in the embodiment of the present invention (for example, the receiving unit 401, the identifying unit 402, the determining unit 403, the first reading unit 404, the second reading unit 405, and the like shown in fig. 6). The processor 601 executes various functional applications and data processing of the electronic device 600, namely, a method of data storage and a method of data download of the method embodiments, by executing nonvolatile software programs, instructions and units stored in the memory 602.
The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the electronic device 600, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 602 optionally includes memory located remotely from the processor 601, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more units are stored in the memory 602, and when executed by the one or more processors 601, perform the method and apparatus for data storage and the method and apparatus for data download in the above embodiments.
The electronic device 600 can execute the data storage method and the data downloading method in the above embodiments, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in the embodiment of the electronic device 600, reference may be made to the method and apparatus for data storage and the method and apparatus for data downloading in the above-described embodiment.
Embodiments of the present invention provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, where the computer program includes program instructions, and when the program instructions are executed by the electronic device 600, the electronic device 600 executes the method and apparatus for data storage and the method and apparatus for data download in the above-described embodiments. For example, the method steps S11-S14 in fig. 2 described above are executed to implement the functions of the 301-309 cell in fig. 5.
Embodiments of the present invention provide a non-volatile computer-readable storage medium, where computer-executable instructions are stored, and the computer-executable instructions are used to enable the electronic device 600 to perform the method and apparatus for data storage and the method and apparatus for data downloading in the foregoing embodiments.
The electronic device 600 of the embodiment of the present application may exist in various forms, including but not limited to a smart phone, a computer, a smart watch, a smart band, a tablet computer, a palm computer, and other electronic devices with a storage function.
It should be noted that the above-described device embodiments are merely illustrative, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
Through the above description of the embodiments, those skilled in the art will clearly understand that the embodiments may be implemented by software plus a general hardware platform, and may also be implemented by hardware. It will be understood by those skilled in the art that all or part of the processes in the methods for implementing the embodiments may be implemented by hardware associated with computer program instructions, and the programs may be stored in a computer readable storage medium, and when executed, may include processes of the embodiments of the methods as described. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and these modifications or substitutions do not depart from the spirit of the corresponding technical solutions of the embodiments of the present invention.
Claims (6)
1. A method for downloading data is applied to a network storage device, and is characterized in that the method comprises the following steps:
establishing a shared storage group with at least one other network storage device, wherein all the network storage devices in the shared storage group are provided with a shared space; each network storage device in the shared storage group is equal, and no primary and secondary scores exist; storing the copies of the files randomly to the shared space of other network storage devices in the shared storage group, or storing the copies according to whether the network storage device receiving the storage request and the other network storage devices in the shared storage group belong to the same local area network, the distance between the network storage device receiving the storage request and the network storage devices receiving the storage request, and the available space size of the shared space of the other network storage devices;
receiving a storage request, wherein the storage request carries a file to be stored;
storing the file to be stored in the network storage equipment;
dividing the copy of the file to be stored into a plurality of data blocks according to a preset erasure correction algorithm, encrypting the data blocks, and respectively storing the encrypted data blocks into shared spaces of other different network storage devices in the shared storage group;
marking the file to be stored as a shared file in the network storage equipment;
detecting whether a shared file in the network storage device or a file located in a shared space of the network storage device has failed; if the file is invalid, sending a file replacement request to the shared storage group; receiving a replacement file returned by the shared storage group according to a file replacement request; replacing the replacement file with a failed file;
receiving a file reading request;
identifying whether a file to be read matched with the file reading request exists in the network storage equipment or not;
if yes, judging whether the file to be read is invalid or not;
if the file to be read does not fail, directly reading the file to be read from the network storage equipment;
if the file fails, extracting the data blocks of the file to be read from other network storage devices in the shared storage group respectively, and integrating the extracted data blocks to form a complete copy of the file to be stored according to a preset erasure correction algorithm.
2. The method of claim 1, further comprising:
judging whether the file to be read positioned in the shared storage group is invalid or not;
if the failure occurs, returning and reading failure information;
and if the file is not invalid, extracting the file to be read from the shared storage group, and returning the file to be read.
3. An apparatus for downloading data, applied to a network storage device, the apparatus comprising:
the system comprises a group unit, a storage unit and a control unit, wherein the group unit is used for establishing a shared storage group with at least one other network storage device, and all the network storage devices in the shared storage group are provided with shared spaces; each network storage device in the shared storage group is equal, and no primary and secondary scores exist; storing the copies of the files randomly to the shared space of other network storage devices in the shared storage group, or storing the copies according to whether the network storage device receiving the storage request and the other network storage devices in the shared storage group belong to the same local area network, the distance between the network storage device receiving the storage request and the network storage devices receiving the storage request, and the available space size of the shared space of the other network storage devices;
the device comprises a first receiving unit, a second receiving unit and a processing unit, wherein the first receiving unit is used for receiving a storage request which carries a file to be stored;
the first storage unit is used for storing the file to be stored in the network storage equipment;
the dividing module is used for dividing the copy of the file to be stored into a plurality of data blocks according to a preset erasure correction algorithm;
the shared storage module is used for respectively storing the data blocks into shared spaces of other different network storage devices in the shared storage group;
the marking unit is used for marking the file to be stored as a shared file in the network storage equipment;
the detection unit is used for detecting whether the shared file in the network storage equipment or the file positioned in the shared space of the network storage equipment is invalid or not;
a sending unit, configured to send a file replacement request to the shared storage group if the file replacement request fails;
the second receiving unit is used for receiving a replacement file returned by the shared storage group according to the file replacement request;
the file replacement unit is used for replacing the failed file with the replacement file;
a receiving unit configured to receive a file reading request;
the identification unit is used for identifying whether a file to be read matched with the file reading request exists in the network storage equipment or not;
the judging unit is used for judging whether the file to be read is invalid or not if the file exists;
the first reading unit is used for directly reading the file to be read from the network storage equipment if the file is not invalid;
the second reading unit is used for respectively extracting the data blocks of the file to be read from other network storage devices in the shared storage group if the file to be read fails; and integrating the extracted data blocks to form a complete copy of the file to be stored according to a preset erasure algorithm.
4. The apparatus of claim 3, wherein the shared storage module is further specifically configured to:
judging whether the file to be read positioned in the shared storage group is invalid or not;
if the failure occurs, returning and reading failure information;
and if the file is not invalid, extracting the file to be read from the shared storage group, and returning the file to be read.
5. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of claim 1 or 2.
6. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for causing a computer to perform the method of claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810844119.5A CN109254956B (en) | 2018-07-27 | 2018-07-27 | Data downloading method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810844119.5A CN109254956B (en) | 2018-07-27 | 2018-07-27 | Data downloading method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109254956A CN109254956A (en) | 2019-01-22 |
| CN109254956B true CN109254956B (en) | 2022-09-02 |
Family
ID=65049851
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810844119.5A Active CN109254956B (en) | 2018-07-27 | 2018-07-27 | Data downloading method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109254956B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209670B (en) * | 2019-05-09 | 2022-03-25 | 北京猫盘技术有限公司 | Data processing method and device based on network storage device cluster |
| CN112272185A (en) * | 2020-10-30 | 2021-01-26 | 江苏智云领创信息咨询有限公司 | Method for sharing computer information and mobile terminal data |
| CN113885795B (en) * | 2021-09-23 | 2024-01-19 | 惠州Tcl移动通信有限公司 | Data storage method, device, terminal equipment and storage medium |
| CN114915461A (en) * | 2022-04-29 | 2022-08-16 | 联想(北京)有限公司 | Processing method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101188569A (en) * | 2006-11-16 | 2008-05-28 | 饶大平 | Method for constructing data quanta space in network and distributed file storage system |
| CN102270161A (en) * | 2011-06-09 | 2011-12-07 | 华中科技大学 | Methods for storing, reading and recovering erasure code-based multistage fault-tolerant data |
| CN103106124A (en) * | 2012-12-29 | 2013-05-15 | 华中科技大学 | Intersection reconstruction method based on erasure code cluster memory system |
| CN107861843A (en) * | 2017-11-30 | 2018-03-30 | 紫光股份有限公司 | A kind of file backup system based on LAN computer |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10769029B2 (en) * | 2016-11-18 | 2020-09-08 | International Business Machines Corporation | Accessing records of a backup file in a network storage |
-
2018
- 2018-07-27 CN CN201810844119.5A patent/CN109254956B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101188569A (en) * | 2006-11-16 | 2008-05-28 | 饶大平 | Method for constructing data quanta space in network and distributed file storage system |
| CN102270161A (en) * | 2011-06-09 | 2011-12-07 | 华中科技大学 | Methods for storing, reading and recovering erasure code-based multistage fault-tolerant data |
| CN103106124A (en) * | 2012-12-29 | 2013-05-15 | 华中科技大学 | Intersection reconstruction method based on erasure code cluster memory system |
| CN107861843A (en) * | 2017-11-30 | 2018-03-30 | 紫光股份有限公司 | A kind of file backup system based on LAN computer |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109254956A (en) | 2019-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109254956B (en) | Data downloading method and device and electronic equipment | |
| US11327840B1 (en) | Multi-stage data recovery in a distributed storage network | |
| CN110602239B (en) | Block chain information storage method and related equipment | |
| US10073652B2 (en) | Performance optimized storage vaults in a dispersed storage network | |
| US10042704B2 (en) | Validating stored encoded data slice integrity in a dispersed storage network | |
| CN110716895B (en) | Target data archiving method, device, computer equipment and medium | |
| CN109885256B (en) | Data storage method, device and medium based on data slicing | |
| CN109831487B (en) | Fragmented file verification method and terminal equipment | |
| US11074139B2 (en) | Dynamic block chain system using metadata for backing up data based on digest rules | |
| CN111163182B (en) | Block chain-based device registration method and apparatus, electronic device, and storage medium | |
| CN106776130B (en) | Log recovery method, storage device and storage node | |
| CN110995513A (en) | Data sending and receiving method in Internet of things system, Internet of things equipment and platform | |
| CN104468548A (en) | Data verification method, disaster recovery storage and production storage | |
| CN111262822A (en) | File storage method, device, block link point and system | |
| CN106201788A (en) | Copy restorative procedure and system for distributed storage cluster | |
| CN111339551B (en) | Data verification method and related device and equipment | |
| CN105550071A (en) | System file upgrading and detecting method and communication device | |
| KR20080050276A (en) | Distributed object-sharing system and method thereof | |
| CN111260475A (en) | Data processing method, block chain node point equipment and storage medium | |
| CN110908801B (en) | Data processing method and device based on block chain, computer equipment and storage medium | |
| CN112738249A (en) | File uploading method, device, equipment and storage medium based on quantitative transaction | |
| EP4195033A1 (en) | Method and apparatus for upgrading blockchain system, and terminal device | |
| CN115935414A (en) | Block chain based data verification method and device, electronic equipment and storage medium | |
| CN106020975B (en) | Data operation method, device and system | |
| US10122795B2 (en) | Consistency level driven data storage in a dispersed storage network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |