CN109063121B

CN109063121B - Data storage method, device, computer equipment and computer storage medium

Info

Publication number: CN109063121B
Application number: CN201810865642.6A
Authority: CN
Inventors: 易仁杰
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2018-08-01
Filing date: 2018-08-01
Publication date: 2024-04-05
Anticipated expiration: 2038-08-01
Also published as: CN109063121A; WO2020024445A1

Abstract

The invention discloses a data storage method, a device, computer equipment and a computer storage medium, which relate to the technical field of Internet, and can determine the data to be stored with the nearest target storage node according to the physical position of a user, so that the users in different positions can realize distributed storage, the data of all users can not be stored in one storage cluster, the data can be quickly acquired, the time for inquiring the data is shortened, and the intelligence is better. The method comprises the following steps: receiving a data storage instruction of a user; positioning a terminal of a user based on a terminal identifier to acquire a first physical position of the user; determining at least one storage node according to the first physical location, and determining at least one first distance between the at least one storage node and the first physical location; extracting a target distance from at least one first distance, and determining a target storage node indicated by the target distance; and acquiring data to be stored in the data storage instruction, and storing the data to be stored to the target storage node.

Description

Data storage method, device, computer equipment and computer storage medium

Technical Field

The present invention relates to the field of internet technologies, and in particular, to a data storage method, a data storage device, a computer device, and a computer storage medium.

Background

With the explosive development of various applications such as mobile devices, social networks, and the internet of things, data generated by human society is explosively increased. Conventional data storage is typically disk storage, and the user stores the number average to be stored in the disk so that the data can be viewed anytime and anywhere. However, as the data volume of the data to be stored is larger, it is more and more difficult for the conventional disk to satisfy the storage requirements of users based on mass data in terms of capacity, performance and bandwidth, and thus, a data storage system supported by a cloud platform has been developed. The data storage system is provided with a data center, a user can upload data to be stored to the data center, and the data center stores the data to be stored.

In the related art, a storage cluster for storing data is disposed in a data center, and when a user uploads data to be stored to a data storage system, the data center receives the data to be stored, and the data center adds the received data to be stored to the storage cluster for storage.

In carrying out the present invention, the inventors have found that the related art has at least the following problems:

when the data center stores data, all received data to be stored are stored in the same storage cluster, so that massive data are stored in the storage cluster, and when a data query request of a user is received, the data query is needed in the massive data, the data which the user needs to query cannot be quickly obtained, the time for querying the data is long, and the intelligence is poor.

Disclosure of Invention

In view of the above, the present invention provides a data storage method, apparatus, computer device and computer storage medium, which mainly aims to solve the problems of long time consumption and poor intelligence of data query caused by the fact that data query needs to be performed in massive data and the data required to be queried by a user cannot be obtained quickly when a data query request of the user is received.

According to a first aspect of the present invention there is provided a data storage method comprising:

receiving a data storage instruction of a user, wherein the data storage instruction at least comprises data to be stored and a terminal identifier of a terminal of the user;

Positioning the terminal of the user based on the terminal identifier to acquire a first physical position of the user;

determining at least one storage node according to the first physical location, and determining at least one first distance between the at least one storage node and the first physical location, wherein the at least one storage node is deployed in a storage resource pool indicated by the first physical location;

extracting a target distance from the at least one first distance, and determining a target storage node indicated by the target distance, wherein the target distance is a first distance meeting a first distance standard in the at least one first distance;

and acquiring the data to be stored in the data storage instruction, and storing the data to be stored to the target storage node.

In another embodiment, the extracting the target distance from the at least one first distance, determining the target storage node indicated by the target distance, includes:

sequencing the at least one first distance from large to small to generate a first sequencing result;

and extracting a first distance arranged at the last position from the first sorting result as the target distance, and determining a storage node indicated by the target distance as the target storage node.

In another embodiment, the method further comprises:

determining a plurality of second distances between the target storage node and other storage nodes in the at least one storage node, wherein the other storage nodes are storage nodes except the target node in the at least one storage node;

determining a preset number of backup storage nodes of which the second distances meet a second distance standard according to the plurality of second distances;

and transmitting the data to be stored to the backup storage nodes with the preset number, and carrying out data backup.

In another embodiment, the method further comprises:

receiving a data adding instruction of a user, and acquiring a second physical position of the user, wherein the data adding instruction carries data to be added;

if the second physical location is different from the first physical location, and the storage node, the distance of which from the second physical location meets the distance standard, is any backup storage node in the preset number of backup storage nodes, the data to be added is stored in the backup storage nodes;

and synchronizing the data to be added to the target storage node based on the backup storage node.

In another embodiment, the method further comprises:

receiving a data query request of the user, and acquiring a query position of the user, wherein the data query request carries a data identifier of data to be queried;

if the query position is consistent with the first physical position, a first data acquisition request is sent to the target storage node, wherein the first data acquisition request is used for indicating the target storage node to acquire data to be queried indicated by the data identification, and the data to be queried is returned to the user;

if the query position is inconsistent with the first physical position, determining a query storage node according to the query position, and sending a second data acquisition request to the query storage node, wherein the second data acquisition request is used for indicating the query storage node to acquire the data to be queried through the target storage node and returning the data to be queried to the user.

In another embodiment, the method further comprises:

counting the data query of the user at the query position, and generating a query rule of the user at the query position, wherein the query rule at least comprises historical query times and historical query time;

And predicting the next query time of the user at the query position based on the query rule, and synchronizing all data in the target storage node to the query storage node at the next query time.

In another embodiment, the method further comprises:

counting the times of data query of the user at the query position to generate query times;

if the query times are greater than the times threshold, the query storage node is used as a new target storage node;

synchronizing all data in the target storage node to the new target storage node;

and determining a preset number of new backup storage nodes based on the new target storage nodes, and backing up all the data to the preset number of new backup storage nodes.

According to a second aspect of the present invention there is provided a data storage device comprising:

the first receiving module is used for receiving a data storage instruction of a user, wherein the data storage instruction at least comprises data to be stored and a terminal identifier of a terminal of the user;

the positioning module is used for positioning the terminal of the user based on the terminal identifier and acquiring a first physical position of the user;

A first distance determining module, configured to determine at least one storage node according to the first physical location, and determine at least one first distance between the at least one storage node and the first physical location, where the at least one storage node is disposed in a storage resource pool indicated by the first physical location;

a first node determining module, configured to extract a target distance from the at least one first distance, determine a target storage node indicated by the target distance, where the target distance is a first distance that meets a first distance criterion in the at least one first distance;

the first storage module is used for acquiring the data to be stored in the data storage instruction and storing the data to be stored to the target storage node.

In another embodiment, the first node determining module includes:

the sorting sub-module is used for sorting the at least one first distance from large to small to generate a first sorting result;

and the determining submodule is used for extracting the first distance arranged at the last position from the first sorting result as the target distance, and determining a storage node indicated by the target distance as the target storage node.

In another embodiment, the apparatus further comprises:

a second distance determining module, configured to determine a plurality of second distances between the target storage node and other storage nodes in the at least one storage node, where the other storage nodes are storage nodes in the at least one storage node except the target node;

the second node determining module is used for determining a preset number of backup storage nodes of which the second distances meet a second distance standard according to the plurality of second distances;

and the first backup module is used for transmitting the data to be stored to the backup storage nodes with the preset number and carrying out data backup.

In another embodiment, the apparatus further comprises:

the second receiving module is used for receiving a data adding instruction of a user and obtaining a second physical position of the user, wherein the data adding instruction carries data to be added;

the second storage module is used for storing the data to be added to the backup storage nodes if the second physical position is different from the first physical position and the storage node, the distance between which and the second physical position meet the distance standard, is any one of the backup storage nodes with the preset number;

And the first synchronization module is used for synchronizing the data to be added to the target storage node based on the backup storage node.

In another embodiment, the apparatus further comprises:

the position acquisition module is used for receiving a data query request of the user and acquiring a query position of the user, wherein the data query request carries a data identifier of data to be queried;

the first sending module is used for sending a first data acquisition request to the target storage node if the query position is consistent with the first physical position, wherein the first data acquisition request is used for indicating the target storage node to acquire the data to be queried indicated by the data identification, and returning the data to be queried to the user;

and the second sending module is used for determining a query storage node according to the query position if the query position is inconsistent with the physical position, sending a second data acquisition request to the query storage node, wherein the second data acquisition request is used for indicating the query storage node to acquire the data to be queried through the target storage node and returning the data to be queried to the user.

In another embodiment, the apparatus further comprises:

the first statistics module is used for counting the data query of the user at the query position and generating a query rule of the user at the query position, wherein the query rule at least comprises historical query times and historical query time;

and the prediction module is used for predicting the next query time of the user at the query position based on the query rule and synchronizing all data in the target storage node to the query storage node at the next query time.

In another embodiment, the apparatus further comprises:

the second statistics module is used for counting the times of data query of the user at the query position and generating query times;

a third node determining module, configured to take the query storage node as a new target storage node if the number of queries is greater than a number threshold;

a second synchronization module for synchronizing all data in the target storage node to the new target storage node;

and the second backup module is used for determining a preset number of new backup storage nodes based on the new target storage nodes and backing up all the data to the preset number of new backup storage nodes.

According to a third aspect of the present invention there is provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, characterised in that the processor when executing the computer program implements the steps of the method of the first aspect described above.

According to a fourth aspect of the present invention there is provided a computer storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor implements the steps of the method according to the first aspect described above.

By means of the technical scheme, compared with the mode that all received data to be stored are stored in the same storage cluster at present, the data storage method and device provided by the invention have the advantages that the physical position of data storage is carried out according to the user request, the target storage node closest to the physical position is determined to store the data to be stored of the user, so that the data stored by the user in different positions are distributed to be stored, the stored data of all the users are not stored in one storage cluster, the data required to be queried by the user can be quickly obtained when the data query request of the user is received, the time for querying the data is shortened, and the intelligence is better.

The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:

FIG. 1 is a schematic flow chart of a data storage method according to an embodiment of the present invention;

FIG. 2A is a schematic diagram of a data storage system according to an embodiment of the present invention;

FIG. 2B is a schematic flow chart of a data storage method according to an embodiment of the present invention;

FIG. 2C is a schematic flow chart of a data storage method according to an embodiment of the present invention;

FIG. 2D is a schematic flow chart of a data storage method according to an embodiment of the present invention;

FIG. 3A is a schematic diagram of a data storage device according to an embodiment of the present invention;

FIG. 3B is a schematic diagram of a data storage device according to an embodiment of the present invention;

FIG. 3C is a schematic diagram of a data storage device according to an embodiment of the present invention;

FIG. 3D is a schematic diagram of a data storage device according to an embodiment of the present invention;

FIG. 3E is a schematic diagram of a data storage device according to an embodiment of the present invention;

FIG. 3F is a schematic diagram of a data storage device according to an embodiment of the present invention;

fig. 3G illustrates a schematic structure of a data storage device according to an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The embodiment of the invention provides a data storage method, as shown in fig. 1, which comprises the following steps:

101. And receiving a data storage instruction of the user, wherein the data storage instruction at least comprises data to be stored and a terminal identifier of a terminal of the user.

For a specific procedure see step 203 in the following examples.

102. And positioning the terminal of the user based on the terminal identification to acquire a first physical position of the user.

For a specific procedure see step 203 in the following examples.

103. According to the first physical location, at least one storage node is determined, at least one first distance between the at least one storage node and the first physical location is determined, and the at least one storage node is deployed in a storage resource pool indicated by the first physical location.

For a specific procedure see step 204 in the examples below.

104. And extracting the target distance from the at least one first distance, and determining a target storage node indicated by the target distance, wherein the target distance is the first distance meeting the first distance standard in the at least one first distance.

For a specific procedure see step 205 in the examples described below.

105. And acquiring data to be stored in the data storage instruction, and storing the data to be stored to the target storage node.

For specific procedures, see step 206 in the examples below.

In another embodiment, extracting a target distance from at least one first distance, determining a target storage node indicated by the target distance, comprises:

Sequencing at least one first distance from large to small to generate a first sequencing result;

and extracting the first distance arranged at the last position from the first sorting result as a target distance, and determining a storage node indicated by the target distance as a target storage node.

In another embodiment, the method further comprises:

and transmitting the data to be stored to a preset number of backup storage nodes, and carrying out data backup.

In another embodiment, the method further comprises:

if the second physical location is different from the first physical location and the storage node with the distance meeting the distance standard from the second physical location is any backup storage node in the preset number of backup storage nodes, storing the data to be added to the backup storage nodes;

Based on the backup storage node, the data to be appended is synchronized to the target storage node.

In another embodiment, the method further comprises:

receiving a data query request of a user, and acquiring a query position of the user, wherein the data query request carries a data identifier of data to be queried;

if the query location is inconsistent with the first physical location, determining a query storage node according to the query location, and sending a second data acquisition request to the query storage node, wherein the second data acquisition request is used for indicating the query storage node to acquire data to be queried through the target storage node and returning the data to be queried to a user.

In another embodiment, the method further comprises:

counting the data query performed by the user at the query position to generate a query rule of the user at the query position, wherein the query rule at least comprises historical query times and historical query time;

based on the query rule, predicting the next query time of the user at the query position, and synchronizing all data in the target storage node to the query storage node at the next query time.

In another embodiment, the method further comprises:

counting the times of data inquiry of a user at an inquiry position to generate inquiry times;

synchronizing all data in the target storage node to a new target storage node;

and based on the new target storage nodes, determining a preset number of new backup storage nodes, and backing up all data to the preset number of new backup storage nodes.

According to the data storage method provided by the embodiment of the invention, the physical position for data storage can be determined according to the user request, the target storage node closest to the physical position stores the data to be stored of the user, so that the data stored by the users in different positions are all stored in a distributed mode, all the stored data of the user are not stored in one storage cluster, the data required to be queried by the user can be quickly obtained when the data query request of the user is received, the time for querying the data is shortened, and the intelligence is better.

Before explaining the embodiments of the present invention in detail, a simple description is given of the structure of the data storage system according to the embodiments of the present invention.

Referring to FIG. 2A, an Endpoint and a storage resource pool are deployed in a data storage system at multiple regions. Wherein, endpoint represents a directory system when a user's terminal is connected to a storage resource pool; the storage resource pool is used for storing the received data uploaded by the user, the storage resource pool comprises at least one storage node, the storage of the storage resource pool of each area is high-availability storage, and even if one storage node in the storage resource pool fails, the data stored in the storage node cannot be lost. In addition, the storage resource pool of each area is connected with a resource synchronization channel, and the resource synchronization channel realizes the synchronization of the data of one area from the storage resource pool of the current area to the storage resource pool of another area.

The embodiment of the invention provides a data storage method, which can achieve the purposes of quickly acquiring data to be queried by a user, shortening the time for querying the data and having better intelligence, and as shown in fig. 2B, the method comprises the following steps:

201. determining a plurality of storage resource pools, and establishing data connection between the storage resource pools and other storage resource pools based on a preset data channel for each storage resource pool in the plurality of storage resource pools.

In the embodiment of the present invention, each storage resource pool in the plurality of storage resource pools includes at least one storage node, and the preset data channel is at least a public network channel or an intranet channel, where the public network channel may specifically be a VPN (Virtual Private Network ) channel. The inventor realizes that in order to make the user more convenient when storing data, and select the most suitable storage node for the user to store the data, the storage node for storing the data to be stored of the user can be determined according to the geographic position of the user, so that the time for storing and acquiring the data of the user is shortened. In addition, considering that the user may query data in different areas, in order to facilitate the query of the user on the data, the data to be queried by the user needs to be synchronized from one area to another area, so that the data connection between the storage resource pool of one area and the storage resource pools of other areas can be established based on the preset data channel.

When establishing data connection between the storage resource pool of one area and the storage resource pool of another area, a preset data channel can be adopted to respectively establish data connection between the storage resource pool of one area and the storage resource pool of another area and the resource synchronization channel, so that the synchronization of data from the resource storage pool of one area to the resource storage pool of another area can be realized through the resource synchronization channel. For at least one storage node in an area, the at least one storage node can be connected with each other by adopting a preset data channel of the area, so that operations such as synchronization and backup of data stored in the storage nodes are guaranteed.

202. And grading the data connection between the storage resource pool and other storage resource pools, and generating a grading result.

In the embodiment of the invention, when the storage resource pool is connected with other storage resource pools, a public network channel or an intranet channel can be adopted for connection, and the security coefficient of the intranet channel is higher than that of the public network channel, so after the data connection between the storage resource pool and the other storage resource pools is successfully established, the data connection between the storage resource pool and the other storage resource pools can be rated, and a rating result is generated, so that when synchronous data can be determined according to the rating result, the safest storage node is determined to be synchronous, and the user can be reminded of executing the security of data synchronization.

If the data connection between the storage resource pool and other storage resource pools is a public network channel, the generated rating result is three-level, which can be specifically represented by level C, and the storage resource pool of the area and the storage resource pools of other areas only have the public network channel, so that the synchronization risk is higher; if the data connection between the storage resource pool and other storage resource pools is an intranet channel, the generated rating result is a secondary, and can be specifically represented by a B level, wherein the storage resource pool of the area and the storage resource pools of other areas only have the intranet channel, and the synchronization risk is low; if the data connection between the storage resource pool and other storage resource pools is a public network channel and an intranet channel, the generated rating result is a first level, which can be specifically represented by a level a, and the storage resource pool representing the area and the storage resource pools of other areas have two channels, namely the intranet channel and the public network channel, so that the synchronization risk is the lowest, and when the data synchronization is performed, the intranet channel is preferentially selected for the data synchronization.

203. And when the data storage instruction is received, extracting a terminal identifier from the data storage instruction, and positioning the terminal of the user based on the terminal identifier to acquire a first physical position of the user.

In the embodiment of the invention, the data storage instruction at least comprises data to be stored and a terminal identifier of a terminal of a user. The server carried by the data storage system can provide data storage service for users in the form of clients, a data storage entrance is provided in the clients, and the users can download the clients in the held terminals; further, in order to ensure that the user can accurately acquire the stored data later, the client can provide registration service for the user, and allocate a registration identifier for each registered user, and the registration identifier is also used as a terminal identifier.

Thus, when the user is detected to trigger the data storage entrance, the data storage system determines that the data storage instruction is received, and the terminal identification of the terminal held by the user is obtained, and specifically, the mobile phone number of the terminal held by the user and the registration identification of the user can be obtained. After the terminal identification of the terminal is obtained, the terminal can be positioned according to the terminal identification, and then the first physical position of the terminal is obtained. In positioning the terminal, GPRS (General Packet Radio Service ) may be used to position the terminal. In the process of actual application, when the terminal is positioned, the GPRS function in the operating system of the terminal can be called to position the terminal, so that the first physical position of the terminal is obtained.

204. At least one storage node is determined based on the first physical location, and at least one first distance between the at least one storage node and the first physical location is determined.

In an embodiment of the present invention, at least one storage node is deployed in a storage resource pool indicated by a first physical location. After determining the first physical location of the terminal, the area where the terminal is located can be determined according to the obtained first physical location, for example, the first physical location can be a certain area in Shanghai, a certain area in Shenzhen, and the like, so that the area where the first physical location is located can be determined to be Shanghai or Shenzhen. Since more than one storage node is included in the storage resource pool of each region, at least one storage node within the region needs to be determined in order to select the best storage node for subsequent use by the user.

After at least one storage node is determined, the position of each storage node is different, and the data to be stored of the user can be stored by determining at least one first distance between the at least one storage node and a first physical position of the terminal according to the at least one first distance in consideration of the fact that the data storage and query speeds are faster when the positions of the storage node and the user are closer. When the first distance is acquired, the node position of at least one storage node can be positioned; subsequently, a distance between the node location of the at least one storage node and the first physical location is calculated as a first distance. Since the storage resource pool of one area includes at least one storage node, at least one first distance can be acquired.

205. And sequencing at least one first distance from large to small to generate a first sequencing result, extracting the first distance ranked at the last position from the first sequencing result as a target distance, and determining a storage node indicated by the target distance as a target storage node.

In the embodiment of the invention, after at least one first distance is acquired, at least one first distance is sequenced from big to small, a first sequencing result is generated, the first distance in the last position is extracted from the first sequencing result as a target distance, namely the smallest first distance is used as the target distance, and a storage node indicated by the target distance is determined as a target storage node. In addition, at least one first distance can be ranked from small to large, the first distance ranked first is taken as the target distance, and the target storage node is further determined.

206. And acquiring data to be stored in the data storage instruction, and storing the data to be stored to the target storage node.

In the embodiment of the invention, after the target storage node is determined, the data to be stored can be obtained in the data storage instruction, the data to be stored is transmitted to the target storage node, and the target storage node stores the data to be stored.

207. Determining a plurality of second distances between the target storage node and other storage nodes in at least one storage node, sequencing the plurality of second distances from large to small, generating a second sequencing result, extracting a preset number of last second distances from the second sequencing result, and taking the storage nodes indicated by the preset number of last second distances as backup storage nodes.

In the embodiment of the invention, the other storage nodes are storage nodes except the target node in at least one storage node. In order to ensure the safety of data stored in a certain storage node, a plurality of backup storage nodes can be determined according to a target storage node so as to synchronize the data stored in the storage nodes to a plurality of backup storage nodes in the storage resource pool, and the data is backed up based on the plurality of backup storage nodes.

After the target storage node is determined, a plurality of second distances between the target storage node and other storage nodes in at least one storage node of the current storage resource pool can be determined, the second distances are sorted from large to small, a second sorting result is generated, the last second distances of the preset number are extracted from the second sorting result, namely the minimum second distances of the preset number are extracted, the storage node indicated by the last second distances of the preset number is used as a backup storage node, and the data to be stored is synchronized to the backup storage nodes of the preset number for data backup subsequently.

The preset number may be 2, so that 2 storage nodes with the smallest distance from the target storage node may be selected as backup storage nodes. For example, if the target storage node is P1, after storing the data to be stored in P1, storage nodes P2 and P3 closest to P1 are selected as backup storage nodes, so that the data to be stored is subsequently transmitted to the backup storage nodes P2 and P3 for backup based on the data channel between P1 and P2 and P3.

In the practical application process, since the data connection between the storage resource pool and the other storage resource pools is rated in step 202 and the rating result is generated, when the backup storage node is determined for the target storage node, the rating result between the target storage node and the other storage nodes can be selected in combination. For example, let P1 be the target storage node, the connection of P1 and P2 be level A, and the average delay of the network be 100ms; the connection of P1 and P3 is C level and the average delay is 300ms. Then we calculate the network weight for P2 using F (a) +g (100) and P3 using F (C) +g (300). In this way, when determining which storage node is to be used as the backup storage node, for any storage node in other storage nodes, the first weight and the second weight of the second distance and the network weight can be obtained, the first product of the second distance and the first weight is calculated, the second product of the network weight and the second weight is calculated, the sum of the first product and the second product is used as the comprehensive distance of the storage node, all the comprehensive distances of other storage nodes are calculated, and then the storage node with the minimum comprehensive distance and the preset number is used as the backup storage node.

It should be noted that, since the data channels between the storage resource pools may change, and the data connection between the storage nodes may also change, the above manner may be periodically adopted to determine a new backup storage node for each target storage node, and adjust the backup position of the data, for example, the new backup storage node may be determined for each target storage node every 30 days. The embodiment of the invention does not specifically limit the time for updating the backup storage node for the target storage node.

208. And transmitting the data to be stored to a preset number of backup storage nodes, and carrying out data backup.

In the embodiment of the invention, after the preset number of backup storage nodes are determined according to the target storage node, the data to be stored can be transmitted to the preset number of backup storage nodes, and the data backup is performed based on the preset number of backup storage nodes, so that the safety of the data stored in the target storage node is ensured.

In the practical process, the user needs to append the data stored in the data storage system from time to time in daily work, and the process of appending the data is shown in fig. 2C, and the process includes the contents in steps 209 to 211.

209. Receiving a data adding instruction of a user, acquiring a second physical position of the user, and executing the following step 210 if the second physical position is the same as the first physical position; if the second physical location is different from the first physical location, then the following step 211 is performed.

In the embodiment of the invention, the data append instruction carries data to be appended. In consideration of the fact that a user may want to append the previously stored data and append some new data to store, a client mounted on the data storage system can provide a data append entry, when the user is detected to trigger the data query entry, a data append page is displayed, data to be appended uploaded by the user on the data append page is acquired, and a data append instruction of the user is received currently.

Wherein, the user may move in the area due to business trip, travel, etc., so that the second physical location where the user is located when the user requests to perform data addition is different from the first physical location where the data storage is performed before, thus, the storage node which is the second physical location away from the user may not be the target storage node where the data storage is performed before, and the storage node closest to the second physical location may be any backup storage node of a plurality of backup storage nodes determined according to the target storage node, therefore, when receiving the data addition instruction of the user, the second physical location of the user needs to be acquired, the second physical location of the user is compared with the first physical location where the data storage is performed before, and if the second physical location is the same as the first physical location where the data storage is performed before, the data to be added may be directly added to the target storage node, that is, the following step 210 is performed; if the second physical location is not the same as the first physical location where the data storage was previously performed, then data addition based on the backup storage node is required, i.e., step 211 described below is performed.

210. If the second physical location is the same as the first physical location, the data to be appended is written in the target storage node.

In the embodiment of the present invention, if the second physical location where the user requests to perform data addition is the same as the first physical location where data storage is performed before, the data to be added may be directly written in the target storage node.

After the data to be added is added to the target storage node, the data to be added needs to be synchronized to other backup storage nodes, so that the data stored in the plurality of backup storage nodes is consistent with the data stored in the target storage node.

211. If the second physical location is different from the first physical location, the data to be added is stored to the backup storage node, and the data to be added is synchronized to the target storage node based on the backup storage node.

In the embodiment of the invention, if the second physical location is different from the first physical location, and the storage node whose distance from the second physical location meets the distance standard is the backup storage node of the target storage node, the data to be added can be stored in the backup storage node, and based on the backup storage node, the data to be added is synchronized to the target storage node, so that the data to be added is added to the target storage node. For example, let the target storage node be P1, the backup storage nodes be P2 and P3, if the first new physical location of the user is closest to P2, when a data append instruction of the user is received, append data carried in the data append instruction is stored in P2, and the append data is synchronized to P1 and P3 based on P2.

In addition, if the second physical location is different from the first physical location and is not in the same storage resource pool as the first physical location, that is, the node closest to the second physical location is not the backup storage node determined according to the target storage node, the above-described process of determining the target storage node shown in step 204 to step 205 is repeatedly performed, the temporary storage node is determined based on the second physical location, and the data to be appended is transferred to the temporary storage node, and the data to be appended is synchronized to the target storage node and the preset number of backup storage nodes based on the temporary storage node. For the process of determining the temporary storage node, a detailed description is omitted herein. For example, let the target storage node be P1, the backup storage nodes be P2 and P3, P1, P2 and P3 are all located in the area a, if the second physical location of the user is in the area B and the storage node closest to the second physical location is P5, the storage node P5 is used as a temporary storage node, the additional data is transferred to P5, the additional data is transferred to P1 based on P5, and the synchronization is performed to P2 and P3.

In the practical process, the user may need to query the data stored in the data storage system from time to time in daily work, and the process of data query is shown in fig. 2D, and the process includes the contents of steps 212 to 217.

212. Receiving a data query request of a user, acquiring a query position of the user, and executing the following step 213 if the query position is consistent with the first physical position; if the query location does not coincide with the first physical location, then the following step 214 is performed.

In the embodiment of the invention, the data query request carries the data identifier of the data to be queried. In consideration of the fact that the user may want to query the data stored in advance later, a client carried by the data storage system can provide a data query entry, when the fact that the user triggers the data query entry is detected, a data query page is displayed, the data identification of data to be queried, which is input by the user on the data query page, is obtained, and the data query request of the user is determined to be received currently.

The user may not perform the data query at the previous data storage position due to business trip, travel, etc., so that the storage node closest to the query position of the user for querying the data is not the previous target storage node for performing the data storage, therefore, when the data query request of the user is received, the query position of the user for querying the data needs to be acquired, the query position is compared with the previous first physical position for performing the data storage, whether the query position is consistent with the first physical position is determined, if the query position is consistent with the first physical position, the data to be queried can be directly acquired from the target storage node, that is, the following step 213 is executed; if the query location does not coincide with the first physical location, then it is necessary to determine the storage node closest to the query location and implement the data query based on the storage node, i.e., perform step 214 described below.

213. And if the query position is consistent with the first physical position, acquiring the data to be queried indicated by the data identification in the target storage node, and returning the data to be queried to the user.

In the embodiment of the invention, if the query position is consistent with the first physical position, the data to be queried indicated by the data identification is obtained from the target storage node, and the data to be queried is directly returned to the user. For example, if the user stores data in Shanghai before the user, and the target storage node is P1 in Shanghai, when a data query request of the user is received, if the current position of the user is detected to be Shanghai, the data to be queried is acquired in P1, and the data to be queried is returned to the user.

214. If the query position is inconsistent with the first physical position, the process of determining the target storage node is repeatedly executed, the query storage node is determined according to the query position, the data identification is transmitted to the target storage node based on the query storage node, the data to be queried returned by the target storage node is received based on the query storage node, and the data to be queried is returned to the user.

In the embodiment of the invention, if the query position is inconsistent with the first physical position, the storage node closest to the query position is not the target storage node, the storage node closest to the query position needs to be determined as the query storage node, and the data representation is transmitted to the target storage node based on the query storage node, so that the target storage node can acquire the data to be queried according to the data identification, the data to be queried is returned to the query storage node, and the query storage node returns the data to be queried to the user. For example, let the user store data in Shanghai before and the target storage node be P1 in Shanghai, when receiving the data query request of the user, if the current position of the user is detected to be Shenzhen, determining a storage node P5 closest to the current position of the user as a query storage node in Shenzhen, sending a data identifier of the data to be queried to P1 based on P5, and receiving the data to be queried returned by P1 based on P5.

It should be noted that, considering that the user may perform data query at the same query location multiple times, in order to shorten the time for the user to perform data query, the data storage system may count the data query performed by the user at the query location, and generate a query rule of the user, so as to predict the next query behavior of the user according to the query rule, thereby making query preparation before the user performs data query, and effectively shortening the time for the user to perform data query. Specifically, the generation of the query rule of the user and the prediction of the query behavior of the user may be achieved through the following steps one to two.

Step one, counting data inquiry performed by a user at an inquiry position to generate an inquiry rule of the user at the inquiry position, wherein the inquiry rule at least comprises historical inquiry times and historical inquiry time.

If it is detected that the user does not perform the data query at the previous data storage position, the query position for performing the data query at this time can be reserved, and the data query performed at the query position by the user is continuously counted at the subsequent time, so as to generate a query rule of the user at the query position.

For example, let the user store data in Shanghai before, and the target storage node be P1 in Shanghai, if it is detected that the user is located between work dates and Shenzhen requests to query data 10 times, then the query rule generated in Shenzhen may be 10 times and during work days.

And secondly, predicting the next query time of the user at the query position based on the query rule, and synchronizing all data in the target storage node to the query storage node at the next query time.

In the embodiment of the invention, after the query rule is generated, the next query time of the user at the query position can be preset based on the query rule, and all data in the target storage node is synchronized to the query storage node at the next query time, so that the data to be queried can be directly provided by the query data node without requesting the data to be queried from the target storage node when the user queries the data at the position next time, and the time consumed by data query is shortened.

For example, continuing with the above step one, in the case that the query rule generated by Shenzhen is 10 times and the working day period is taken as an example, it may be determined that the query node whose query position is nearest to Shenzhen is P5, and before the working day arrives, all the data in the target storage node P1 is synchronized into the query storage node P5.

215. Counting the number of times of data query performed by the user at the query position, generating the number of times of query, and executing the following step 216 if the number of times of query is greater than a number threshold; if the number of queries is less than the number threshold, then the following step 217 is performed.

In the embodiment of the invention, considering that the user is in other areas different from the area to which the first physical location where the data storage is previously performed is caused to work and live for a long time due to moving or work mobilization, so that the user can request data inquiry in other areas for a long time, therefore, the number of times of data inquiry performed by the user at the inquiry location can be counted, the inquiry times are generated, whether the inquiry times are greater than the time threshold is judged, if the inquiry times are greater than the time threshold, the data storage system can consider that the user is at the inquiry location for a long time, a new target storage node can be determined for the user at the inquiry location again, and all data stored by the user in the target storage node is migrated to the new target storage node, thereby ensuring high efficiency of data inquiry, namely, executing the following step 116; if the number of queries is less than the number threshold, it indicates that the current user's data query operation does not meet the criteria for performing data migration, and it is sufficient to keep the current data storage system for the data storage, data addition, and data query service provided by the user, that is, to execute step 217 described below.

216. If the query times are greater than the times threshold, the query storage node is used as a new target storage node, all data in the target storage node are synchronized to the new target storage node, a preset number of new backup storage nodes are determined based on the new target storage node, and all data are backed up to the preset number of new backup storage nodes.

In the embodiment of the present invention, if the number of queries is greater than the number threshold, it indicates that all data stored in the target storage node by the user can be migrated to the query location currently, so that the query storage node in the query location can be used as a new target storage node, all data stored in the target storage node by the user previously can be synchronized to the new target storage node, the content shown in the above step 207 is repeatedly executed, a preset number of new backup storage nodes are determined based on the new target storage node, and the data stored in the new target storage node is backed up to the new backup storage node, so that when the data query request of the user is received at the query location, the data to be queried can be directly acquired in the new target storage node.

For example, if the user stores data in the Shanghai before, the target storage node is P1 in the Shanghai, the query threshold is 100 times, if the user is detected to request for querying the data in Shenzhen more than 100 times, and the position of sending the data query request is closest to P5 of Shenzhen, then P5 can be used as a new target storage node, all data stored in P1 are copied into P5, and a new backup storage node is determined to backup the data near P5.

After all data is migrated to a new target storage node, all data about the user stored in the target storage node may be deleted, and if a user request is subsequently received to perform data addition, data addition may be performed directly based on the new target storage node.

217. If the number of times of inquiry is smaller than the number threshold, the data inquiry request of the user is kept to be continuously received, and data inquiry and data acquisition are carried out.

In the embodiment of the invention, if the query times are smaller than the time threshold, the condition that all the data related to the user in the target storage node are migrated to the query position is not met currently is indicated, so that the data query request of the user is kept to be received continuously, and the operations of data query and data acquisition are carried out based on the query storage node.

Further, as a specific implementation of the method of fig. 2B to 2D, an embodiment of the present invention provides a data storage device, as shown in fig. 3A, where the device includes: a first receiving module 301, a positioning module 302, a first distance determining module 303, a first node determining module 304 and a first storing module 302.

The first receiving module 301 is configured to receive a data storage instruction of a user, where the data storage instruction at least includes data to be stored and a terminal identifier of a terminal of the user;

the positioning module 302 is configured to position a terminal of a user based on a terminal identifier, and obtain a first physical location of the user;

the first distance determining module 303 is configured to determine at least one storage node according to a first physical location, and determine at least one first distance between the at least one storage node and the first physical location, where the at least one storage node is disposed in a storage resource pool indicated by the first physical location;

the first node determining module 304 is configured to extract a target distance from at least one first distance, determine a target storage node indicated by the target distance, where the target distance is a first distance that meets a first distance criterion in the at least one first distance;

The first storage module 305 is configured to obtain data to be stored in a data storage instruction, and store the data to be stored in a target storage node.

The data storage device provided by the embodiment of the invention can determine the target storage node closest to the physical position to store the data to be stored of the user according to the physical position of the data storage request of the user, so that the data stored by the users in different positions are all stored in a distributed mode, all the stored data of the users are not stored in one storage cluster, the data required to be queried by the user can be quickly obtained when the data query request of the user is received, the time for querying the data is shortened, and the intelligence is better.

In a specific application scenario, as shown in fig. 3B, the first node determining module 304 includes a sorting sub-module 3041 and a determining sub-module 3042.

The sorting submodule 3041 is used for sorting at least one first distance from big to small to generate a first sorting result;

the determining submodule 3042 is configured to extract a first distance ranked at the last position from the first ranking result as a target distance, and determine a storage node indicated by the target distance as a target storage node.

In a specific application scenario, as shown in fig. 3C, the apparatus further includes a second distance determining module 306, a second node determining module 307, and a first backup module 308.

The second distance determining module 306 is configured to determine a plurality of second distances between the target storage node and other storage nodes in the at least one storage node, where the other storage nodes are storage nodes in the at least one storage node except the target node;

the second node determining module 307 is configured to determine, according to the plurality of second distances, a preset number of backup storage nodes whose second distances meet a second distance criterion;

the first backup module 308 is configured to transmit data to be stored to a preset number of backup storage nodes, and perform data backup.

In a specific application scenario, as shown in fig. 3D, the apparatus further includes a second receiving module 309, a second storage module 310, and a first synchronization module 311.

The second receiving module 309 is configured to receive a data append instruction of a user, obtain a second physical location of the user, where the data append instruction carries data to be appended;

the second storage module 310 is configured to store the data to be added to the backup storage nodes if the second physical location is different from the first physical location and the storage node whose distance from the second physical location satisfies the distance criterion is any backup storage node of the preset number of backup storage nodes;

The first synchronization module 311 is configured to synchronize data to be appended to a target storage node based on a backup storage node.

In a specific application scenario, as shown in fig. 3E, the apparatus further includes a location obtaining module 312, a first sending module 313 and a second sending module 314.

The location obtaining module 312 is configured to receive a data query request from a user, obtain a query location of the user, where the data query request carries a data identifier of data to be queried;

the first sending module 313 is configured to send a first data acquisition request to the target storage node if the query location is consistent with the first physical location, where the first data acquisition request is used to instruct the target storage node to acquire data to be queried indicated by the data identifier, and return the data to be queried to the user;

the second sending module 314 is configured to determine, if the query location is inconsistent with the first physical location, a query storage node according to the query location, send a second data acquisition request to the query storage node, where the second data acquisition request is used to instruct the query storage node to acquire data to be queried through the target storage node and return the data to be queried to the user.

In a specific application scenario, as shown in fig. 3F, the apparatus further includes a first statistics module 315 and a prediction module 316.

The first statistics module 315 is configured to perform statistics on data query performed by a user at a query location, and generate a query rule of the user at the query location, where the query rule at least includes a historical query number and a historical query time;

the prediction module 316 is configured to predict a next query time of the user at the query location based on the query rule, and synchronize all data in the target storage node to the query storage node at the next query time.

In a specific application scenario, as shown in fig. 3G, the apparatus further includes a second statistics module 317, a third node determination module 318, a second synchronization module 319, and a second backup module 320.

The second statistics module is used for counting the times of data query of the user at the query position and generating the query times;

the third node determining module is used for taking the query storage node as a new target storage node if the query times are greater than the times threshold;

the second synchronization module is used for synchronizing all data in the target storage node to a new target storage node;

and the second backup module is used for determining a preset number of new backup storage nodes based on the new target storage nodes and backing up all data to the preset number of new backup storage nodes.

It should be noted that, for other corresponding descriptions of each functional unit related to the data storage device provided by the embodiment of the present invention, reference may be made to corresponding descriptions in fig. 2B to fig. 2D, and no further description is given here.

Based on the above-mentioned methods shown in fig. 2B to 2D, correspondingly, the embodiment of the present invention further provides a storage device, on which a computer program is stored, which when executed by a processor, implements the above-mentioned data storage method shown in fig. 2B to 2D.

Based on the above-mentioned method shown in fig. 2B to 2D and the embodiments of the virtual device shown in fig. 3A to 3G, in order to achieve the above-mentioned object, an embodiment of the present invention further provides a physical device for data storage, where the physical device includes a storage apparatus and a processor; the storage device is used for storing a computer program; the processor is configured to execute the computer program to implement the data storage method shown in fig. 2B to 2D.

By applying the technical scheme of the invention, the physical position for data storage can be determined according to the user request, the target storage node closest to the physical position stores the data to be stored of the user, so that the data stored by the users in different positions are all stored in a distributed mode, all the stored data of the user are not stored in one storage cluster, the data required to be queried by the user can be quickly obtained when the data query request of the user is received, the time for querying the data is shortened, and the intelligence is better.

From the above description of the embodiments, it will be apparent to those skilled in the art that the present application may be implemented in hardware, or may be implemented by means of software plus necessary general hardware platforms. Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.), and includes several instructions for causing a computer device (may be a personal computer, a server, or a network device, etc.) to perform the methods described in various implementation scenarios of the present application.

Those skilled in the art will appreciate that the drawings are merely schematic illustrations of one preferred implementation scenario, and that the modules or flows in the drawings are not necessarily required to practice the present application.

Those skilled in the art will appreciate that modules in an apparatus in an implementation scenario may be distributed in an apparatus in an implementation scenario according to an implementation scenario description, or that corresponding changes may be located in one or more apparatuses different from the implementation scenario. The modules of the implementation scenario may be combined into one module, or may be further split into a plurality of sub-modules.

The foregoing application serial numbers are merely for description, and do not represent advantages or disadvantages of the implementation scenario.

The foregoing disclosure is merely a few specific implementations of the present application, but the present application is not limited thereto and any variations that can be considered by a person skilled in the art shall fall within the protection scope of the present application.

Claims

1. A method of data storage, comprising:

extracting a last-row target distance from the at least one first distance, and determining a target storage node indicated by the target distance, wherein the target distance is a first distance meeting a first distance standard in the at least one first distance;

Acquiring the data to be stored in the data storage instruction, and storing the data to be stored to the target storage node;

if the second physical location is different from the first physical location, and the storage node with the distance from the second physical location meeting the distance standard is any backup storage node in a preset number of backup storage nodes, storing the data to be added to the backup storage nodes;

2. The method of claim 1, wherein the extracting a target distance from the at least one first distance, determining a target storage node indicated by the target distance, comprises:

3. The method according to claim 1, wherein the method further comprises:

4. The method according to claim 1, wherein the method further comprises:

5. The method according to claim 4, wherein the method further comprises:

6. The method according to claim 4, wherein the method further comprises:

7. A data storage device, comprising:

a first node determining module, configured to extract a last target distance from the at least one first distance, determine a target storage node indicated by the target distance, where the target distance is a first distance that meets a first distance criterion in the at least one first distance;

The first storage module is used for acquiring the data to be stored in the data storage instruction and storing the data to be stored to the target storage node;

8. A computer device comprising a memory and a processor, the memory having stored therein a computer program, characterized in that the processor, when executing the computer program, carries out the steps of the method according to any one of claims 1 to 6.

9. A computer storage medium having stored thereon a computer program, which when executed by a processor realizes the steps of the method according to any of claims 1 to 6.