WO2018027466A1

WO2018027466A1 - Method and system for storing big data in distributed system

Info

Publication number: WO2018027466A1
Application number: PCT/CN2016/093941
Authority: WO
Inventors: 马岩
Original assignee: 马岩
Priority date: 2016-08-08
Filing date: 2016-08-08
Publication date: 2018-02-15

Abstract

A method and system for storing big data in a distributed system. The method comprises the following steps: receiving data needing to be stored (101); after the storage of the data is completed, searching for a distributed storage position thereof (102); and backing up, in the distributed storage position, the data needing to be stored (103). The method has the advantage of being highly secure.

Description

Big data storage method and system in distributed system

Technical field

The present invention relates to the field of big data, and in particular, to a method and system for storing big data in a distributed system.

Background technique

Big data (big Data), or massive data, refers to the massive, high-growth, and diverse information assets that require new processing models to have greater decision-making, insight, and process optimization capabilities. In Victor · Meyer Schonberg and Kenneth Big data in the Big Data Era written by Cookey refers to the use of all data for analysis without the use of random analysis (sample survey). 4V features of big data: Volume, Velocity, Variety, Value.

Existing big data cannot be backed up and data security is low.

technical problem

A method for storing big data in a distributed system is provided, which solves the shortcoming of low security of data in the prior art.

Technical solution

In one aspect, a method for storing big data in a distributed system is provided, the method comprising the steps of:

Receive data that needs to be stored;

After the data is stored, find the location of its distributed storage;

Back up the data that needs to be stored at the location of the distributed storage.

Optionally, the method further includes:

After the distributed storage is completed, a backup complete message is issued.

Optionally, the method further includes:

Prompt for remaining storage capacity.

In a second aspect, a storage system for providing big data in a distributed system is provided, the system comprising:

a receiving unit, configured to receive data that needs to be stored;

a search unit for finding the location of its distributed storage after the data is stored;

A backup unit for backing up data that needs to be stored at the location of the distributed storage.

Optionally, the system further includes:

A sending unit, configured to issue a backup completion message after the distributed storage is completed.

Optionally, the system further includes:

A prompting unit for prompting the remaining storage capacity.

Beneficial effect

The technical solution provided by the specific embodiment of the present invention receives data that needs to be stored. After the data is stored, the location of the distributed storage is searched, and the data to be stored is backed up in the distributed storage location, so that the backup processing can be implemented. advantage.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.

1 is a flowchart of a method for storing big data in a distributed system according to the present invention;

2 is a structural diagram of a big data storage system in a distributed system according to the present invention.

Embodiments of the invention

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Referring to FIG. 1, FIG. 1 is a flowchart of a method for storing big data in a distributed system according to a first preferred embodiment of the present invention. The method is implemented by a server. The method is as shown in FIG. 1 and includes the following steps:

Step S101: Receive data that needs to be stored;

Step S102: After the data is stored, search for the location of the distributed storage;

Step S103: Backing up data that needs to be stored at the location of the distributed storage.

Optionally, after the step S103, the foregoing method may further include:

Prompt for remaining storage capacity.

Referring to FIG. 2, FIG. 2 is a schematic diagram of a storage system for providing big data in a distributed system according to a second preferred embodiment of the present invention. The system includes:

The receiving unit 201 is configured to receive data that needs to be stored;

The searching unit 202 is configured to search for the location of the distributed storage after the data is stored.

The backup unit 203 is configured to back up data that needs to be stored at the location of the distributed storage.

Optionally, the above system may further include:

The sending unit 204 is configured to issue a backup completion message after the distributed storage is completed.

Optionally, the above system may further include:

The prompting unit 205 is configured to prompt the remaining storage capacity.

It should be noted that, for the foregoing method embodiments or embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not subject to the described action sequence. Limitations, as certain steps may be performed in other sequences or concurrently in accordance with the present invention. In the following, those skilled in the art should also understand that the embodiments or examples described in the specification are preferred embodiments, and the actions and units involved are not necessarily required by the present invention.

In the above embodiments, the descriptions of the various embodiments are different, and the details that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

The steps in the method of the embodiment of the present invention may be sequentially adjusted, merged, and deleted according to actual needs.

The units in the apparatus of the embodiment of the present invention may be combined, divided, and deleted according to actual needs. Those skilled in the art can combine or combine the different embodiments described in the specification and the features of the different embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented in hardware, firmware implementation, or a combination thereof. When implemented in software, the functions described above may be stored in or transmitted as one or more instructions or code on a computer readable medium. Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one location to another. A storage medium may be any available media that can be accessed by a computer. Taking this as an example, but not limited to: the computer readable medium may include random access memory (Random) Access Memory, RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), Compact Disc Read-Only Memory, CD-ROM, or other optical disc storage, magnetic storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also. Any connection may suitably be a computer readable medium. For example, if the software is using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (Digital Subscriber Line, DSL) or wireless technology such as infrared, radio and microwave transmission from a website, server or other remote source, then coaxial cable, fiber optic cable, twisted pair, DSL or such as infrared, wireless and microwave Wireless technology is included in the fixing of the associated medium. As used in the present invention, a disk and a disc include a compact disc (CD), a laser disc, a compact disc, a digital versatile disc (DVD), a floppy disk, and a Blu-ray disc, wherein the disc is usually magnetically copied, and the disc is The laser is used to optically replicate the data. Combinations of the above should also be included within the scope of the computer readable media.

In summary, the above description is only a preferred embodiment of the technical solution of the present invention, and is not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims

A method for storing big data in a distributed system, characterized in that the method comprises the following steps:

Receive data that needs to be stored;

After the data is stored, find the location of its distributed storage;

Back up the data that needs to be stored at the location of the distributed storage.
The method of claim 1 further comprising:

After the distributed storage is completed, a backup complete message is issued.
The method of claim 1 further comprising:

Prompt for remaining storage capacity.
A storage system for providing big data in a distributed system, the system comprising:

a receiving unit, configured to receive data that needs to be stored;

a search unit for finding the location of its distributed storage after the data is stored;

A backup unit for backing up data that needs to be stored at the location of the distributed storage.
The system of claim 4, wherein the system further comprises:

A sending unit, configured to issue a backup completion message after the distributed storage is completed.
The system of claim 4, wherein the system further comprises:

A prompting unit for prompting the remaining storage capacity.