CN112540960A

CN112540960A - Data storage management method and system

Info

Publication number: CN112540960A
Application number: CN202011426822.8A
Authority: CN
Inventors: 阳涉; 周婵贞; 李振刚
Original assignee: Individual
Current assignee: Individual
Priority date: 2020-12-09
Filing date: 2020-12-09
Publication date: 2021-03-23

Abstract

The invention discloses a data storage management method and a data storage management system. According to the method, firstly, corresponding associated data are obtained based on a received target data storage instruction, secondly, a plurality of data lists to be stored of the data to be stored are determined based on received current data type information, then, each data list to be stored is analyzed and coded, further, category information corresponding to each data list of the target data list to be stored in the data lists to be stored is obtained through identification, further, dynamic data are obtained through identification from the associated data based on the category information, after the dynamic data are stored, on one hand, redundant data in the dynamic data are removed, and on the other hand, multi-level data labels are added to residual data in the dynamic data. Therefore, the hierarchical property of the residual data in the multi-level search can be ensured by adding the multi-level data tags to the residual data, so that the user can conveniently search the stored data through the data tags of different levels, and the user experience is improved.

Description

Data storage management method and system

Technical Field

The present disclosure relates to the field of data storage management technologies, and in particular, to a data storage management method and system.

Background

With the development of network communication technology, data has been dramatically increased, and a huge data age has been entered. For this data age, the traditional way of manually recording data has been unable to meet the needs of users.

At present, a plurality of data storage management systems are already available in the market, and a user stores all data in the system, and when redundant data are excessive, it is difficult to classify and manage the data, further difficult to search data desired by the user, and meanwhile, the experience of the user is also influenced.

Disclosure of Invention

In order to solve the technical problems in the related art, the present disclosure provides a data storage management method and system.

The invention provides a data storage management method, which is applied to a data storage platform and comprises the following steps:

receiving a target data storage instruction, wherein the target data storage instruction comprises current data type information of data to be stored, and the current data type information refers to a system data type corresponding to the data storage platform;

acquiring corresponding associated data based on the received target data storage instruction; the associated data is key data of each data to be stored, and the data to be stored is uploaded to the data storage platform by the service terminal;

determining a plurality of data lists to be stored of the data to be stored based on the current data type information; each to-be-stored data list comprises a plurality of storage indexes;

acquiring a data list of each data list to be stored, encoding the plurality of data lists to be stored according to the data list, determining a target data list to be stored in the encoded plurality of data lists to be stored, and identifying category information corresponding to each data list in the target data list to be stored;

identifying dynamic data with a compression identifier from the associated data according to the category information, and determining a target storage path according to the dynamic data and a set storage path parameter so as to store the dynamic data according to the target storage path;

carrying out redundancy processing on the stored dynamic data to obtain redundant data in the dynamic data, removing the redundant data from the dynamic data, and adding multi-level data labels to the residual data in the dynamic data.

In an alternative embodiment, determining multiple to-be-stored data lists of the to-be-stored data based on the current data type information specifically includes:

acquiring the query times of data information corresponding to the data to be stored in a preset data set and a query time interval in the query process;

acquiring a user query data set acquired in a corresponding query time interval in each query;

determining a plurality of data nodes corresponding to the data information according to the query time interval of the data information in the query process and the preset shortest query time occupied by the data information in each query; each data node consists of a plurality of child nodes;

for each data node, determining a user query data set which is included in the data node and is acquired in a query time interval corresponding to the query process, obtaining a user query data set corresponding to the data node, and integrating query times corresponding to different user query data in the user query data set corresponding to the data node;

determining the user query data with the query times reaching a first preset time as the query data to be identified corresponding to the data node;

determining target user query data matched with the data to be stored according to the determined query data to be identified corresponding to each data node;

determining a plurality of data lists to be stored of the data to be stored based on the current data type information and the target user query data.

In an alternative embodiment, acquiring a data list of each to-be-stored data list, and encoding the plurality of to-be-stored data lists according to the data list, specifically includes:

acquiring list attribute information corresponding to each data list to be stored, and dividing a plurality of first data lists from each data list to be stored according to the list attribute information;

analyzing first data description information from the first data list; the first data description information is type information used for explaining a data list in the first data list;

analyzing the data list to be stored according to the first data description information to obtain a corresponding data list, and encoding the data lists to be stored according to the data list.

In an alternative implementation manner, determining a target to-be-stored data list in a plurality of encoded to-be-stored data lists, and identifying category information corresponding to each data list in the target to-be-stored data list specifically includes:

carrying out statistical processing on the coded multiple data lists to be stored to obtain a target data list to be stored;

analyzing the data occupancy determined according to the commonly occurring activity degree of a plurality of adjacent data list nodes in the target data list to be stored in a set data list set to obtain a frequency weight and a data structure sequence corresponding to the data occupancy;

judging a target data list to be stored according to the frequency weight, selecting a data structure sequence corresponding to the frequency weight of the target data list to be stored to determine a data distribution track, and acquiring a similarity fusion weight of each parameter in the data occupancy in the data distribution track; obtaining a data list node packet based on the similarity fusion weight;

and extracting the target data list to be stored according to the data list node packet to obtain a data list characteristic value, and identifying category information corresponding to each data list in the target data list to be stored based on the data list characteristic value.

In an alternative implementation, identifying, according to the category information, dynamic data with a compression identifier from the associated data, and determining a target storage path according to the dynamic data and a set storage path parameter specifically include:

according to first data to be compressed and second data to be compressed, which are used for marking the associated data, in the category information, determining element weights of a plurality of behavior element information to be selected for identifying the associated data and linear element matching coefficients among different behavior element information; each first data to be compressed is continuous data of associated data, and each second data to be compressed is discrete data of the associated data;

selecting the behavior element information based on the determined element weights of the behavior element information and linear element matching coefficients among different behavior element information, so that the element weight of the selected behavior element information is larger than a set weight value, and the linear element matching coefficient among the selected behavior element information is smaller than a preset matching coefficient;

and aiming at any preset reference data, according to element mapping data of the preset reference data under each behavior element information in the selected behavior element information, identifying the associated data to obtain dynamic data with a compressed identifier, and determining a target storage path according to the dynamic data and the set storage path parameters.

The invention also provides a data storage management system, which comprises a data storage platform and a user terminal which are communicated with each other;

the data storage platform is to:

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects.

The invention provides a data storage management method and a data storage management system, wherein corresponding associated data are obtained based on a received target data storage instruction, a plurality of data lists to be stored of data to be stored are determined based on received current data type information, each data list to be stored is analyzed and coded, category information corresponding to each data list of the target data list to be stored in the data lists to be stored is obtained through identification, dynamic data are obtained through identification from the associated data based on the category information, after the dynamic data are stored, redundant data in the dynamic data are removed, and multi-level data labels are added to residual data in the dynamic data.

Therefore, firstly, by determining the dynamic data, the dynamic data can be compressed and stored when the data is stored, so that the storage space can be saved, and the storage efficiency can be improved. And secondly, redundant data elimination is carried out on the stored dynamic data, useless data can be effectively filtered, and the data storage platform can be ensured to carry out good classification management on the simplified data. Furthermore, for classification management, the hierarchy of the residual data in the multi-level search can be ensured by adding the multi-level data tags to the residual data, so that a user can conveniently search the stored data through the data tags of different levels, and the user experience is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a schematic diagram of a communication architecture of a data storage management system according to an embodiment of the present invention.

Fig. 2 is a flowchart of a data storage management method according to an embodiment of the present invention.

Fig. 3 is a block diagram of a data storage management apparatus according to an embodiment of the present invention.

Fig. 4 is a schematic hardware structure diagram of a data storage platform according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

In order to solve the technical problem that a user is difficult to classify and manage data, embodiments of the present invention provide a data storage management method and system, which can ensure that dynamic data can be compressed and stored during data storage by determining the dynamic data, so that storage space can be saved and storage efficiency can be improved. And secondly, redundant data elimination is carried out on the stored dynamic data, useless data can be effectively filtered, and the data storage platform can be ensured to carry out good classification management on the simplified data. Furthermore, for classification management, the hierarchy of the residual data in the multi-level search can be ensured by adding the multi-level data tags to the residual data, so that a user can conveniently search the stored data through the data tags of different levels, and the user experience is improved.

To achieve the above object, a communication architecture diagram of a data storage management system as shown in fig. 1 is first provided. The data storage management system 100 may include, among other things, a data storage platform 200 and a user terminal 300. Wherein the data storage platform 200 is in communication with the user terminal 300.

In this embodiment, the data storage platform 200 may be a desktop computer, a notebook computer, or the like, and the user terminal 300 may be a mobile phone, an intelligent electronic device, or the like, which is not limited herein.

On the basis, please refer to fig. 2 in combination, a flow chart of a data storage management method is provided, the method may be applied to the data storage platform 200 in fig. 1, and the data storage platform 200 specifically executes the contents described in the following steps S210 to S260 when implementing the method.

Step S210, receiving a target data storage instruction, where the target data storage instruction includes current data type information of data to be stored, and the current data type information refers to a system data type corresponding to the data storage platform.

Step S220, acquiring corresponding associated data based on the received target data storage instruction; the associated data is key data of each data to be stored, and the data to be stored is uploaded to the data storage platform by the service terminal.

Step S230, determining a plurality of to-be-stored data lists of the to-be-stored data based on the current data type information. In this embodiment, each to-be-stored data list includes a plurality of storage indexes.

Step S240, obtaining a data list of each to-be-stored data list, encoding the plurality of to-be-stored data lists according to the data list, determining a target to-be-stored data list in the encoded plurality of to-be-stored data lists, and identifying category information corresponding to each data list in the target to-be-stored data list.

And step S250, identifying dynamic data with a compression identifier from the associated data according to the category information, and determining a target storage path according to the dynamic data and set storage path parameters, so as to store the dynamic data according to the target storage path.

Step S260, performing redundancy processing on the stored dynamic data to obtain redundant data in the dynamic data, removing the redundant data from the dynamic data, and performing multi-level data tag addition on the remaining data in the dynamic data.

The following advantageous effects can be achieved when the method described in the above steps S210 to S260 is executed:

the method comprises the steps of firstly obtaining corresponding associated data based on a received target data storage instruction, secondly determining a plurality of data lists to be stored of the data to be stored based on received current data type information, then analyzing and coding each data list to be stored, further identifying and obtaining category information corresponding to each data list of the target data list to be stored in the data lists to be stored, further identifying and obtaining dynamic data from the associated data based on the category information, and after the dynamic data are stored, removing redundant data in the dynamic data on one hand, and adding multi-level data labels to residual data in the dynamic data on the other hand.

In a specific implementation, in order to accurately determine a plurality of to-be-stored data lists of data to be stored, the determining, based on the current data type information, the plurality of to-be-stored data lists of data to be stored, which is described in step S230, may further include what is described in sub-step S2301-sub-step S2307, as follows:

step S2301, acquiring the query times of data information corresponding to the data to be stored in a preset data set and a query time interval in the query process;

step S2302, acquiring a user query data set acquired in a corresponding query time interval for each query;

step S2303, determining a plurality of data nodes corresponding to the data information according to the query time interval of the data information in the query process and the preset shortest query time interval occupied by each query; each data node consists of a plurality of child nodes;

step S2304, for each data node, determining a user query data set which is included in the data node and is acquired in a query time interval corresponding to the query process, obtaining a user query data set corresponding to the data node, and integrating query times corresponding to different user query data in the user query data set corresponding to the data node;

step S2305, determining the user query data with the query times reaching a first preset time as the query data to be identified corresponding to the data node;

step S2306, determining target user query data matched with the data to be stored according to the determined query data to be identified corresponding to each data node;

step S2307, determining a plurality of to-be-stored data lists of the to-be-stored data based on the current data type information and the target user query data.

Executing the contents described in the steps S2301 to S2307, first obtaining a user query data set according to the obtained query times and the query time interval, then determining a plurality of data nodes corresponding to the data information, further determining, for each data node, a data node for applying the user query data set, further integrating the query times corresponding to different user query data, then determining query data to be identified corresponding to the data node, further determining target user query data according to the query data to be identified, and then determining a plurality of data lists to be stored of the data to be stored. Therefore, the data nodes are determined to inquire the data set of the user through the data nodes corresponding to the data information, so that a large amount of time can be saved, the storage efficiency is improved in the process of determining the data storage list, and meanwhile, a plurality of data to be stored lists of the data to be stored can be accurately determined.

In a specific implementation, in order to encode the multiple data lists to be stored quickly based on the data list, and in order to add the multi-level data labels to the data after encoding the multiple data lists to be stored and facilitate searching for a user, the step S240 may further include obtaining the data list of each data list to be stored, and encoding the multiple data lists to be stored according to the data list, where the step S2401 to the step S2403 may specifically include the following contents:

step S2401, obtaining list attribute information corresponding to each data list to be stored, and dividing a plurality of first data lists from each data list to be stored according to the list attribute information;

step S2402, analyzing first data description information from the first data list; the first data description information is type information used for explaining a data list in the first data list;

step S2403, analyzing the data list to be stored according to the first data description information to obtain a corresponding data list, and encoding the data lists to be stored according to the data list.

Executing the content described in the above step S2401-0 and step S2403, parsing the first data description information from the divided first data list, further parsing the to-be-stored data list based on the first data description information, and encoding the plurality of to-be-stored data lists after obtaining the corresponding data list. On the one hand, the data list can be used for rapidly coding a plurality of data lists to be stored on the basis of the data list, on the other hand, after the data lists to be stored are coded, the data are added with multi-level data labels conveniently in the following process, and meanwhile, the user can search conveniently.

In a specific implementation, in order to accurately analyze the category information corresponding to each data list, the determining, which is described in step S240, a target to-be-stored data list in the multiple encoded to-be-stored data lists, and identifying the category information corresponding to each data list in the target to-be-stored data list may further include the following contents described in sub-step S2401 to sub-step S2404:

step S2401, carrying out statistical processing on a plurality of encoded data lists to be stored to obtain a target data list to be stored;

step S2402, analyzing the data occupation amount determined according to the commonly appeared activity degree of a plurality of adjacent data list nodes in the target data list to be stored in a set data list set, and obtaining a frequency weight and a data structure sequence corresponding to the data occupation amount;

step S2403, judging a target data list to be stored according to the frequency weight, selecting a data structure sequence corresponding to the frequency weight of the target data list to be stored to determine a data distribution track, and acquiring a similarity fusion weight of each parameter in the data occupancy in the data distribution track; obtaining a data list node packet based on the similarity fusion weight;

step S2404, extracting the target data list to be stored according to the data list node packet to obtain a data list characteristic value, and identifying category information corresponding to each data list in the target data list to be stored based on the data list characteristic value.

By executing the contents described in the steps S2401 to S2404, analyzing the determined data occupation amount, further judging a target data list to be stored according to the obtained frequency weight and determining a data distribution track, further obtaining a similarity fusion weight in the data distribution track, further obtaining a data list node packet, then extracting a target data list to be stored according to the data list node packet, and after extracting a characteristic value of the data list, identifying category information corresponding to each data list in the target data list to be stored. Therefore, each data list in the target data list to be stored can be identified based on the extracted characteristic value of the data list, and the category information corresponding to each data list can be accurately analyzed.

In specific implementation, in order to effectively save storage space, facilitate a user to directly store data in a determined target storage path, and facilitate the user to search for data in a later period, the dynamic data with a compression identifier is identified and obtained from the associated data according to the category information, and the target storage path is determined according to the dynamic data and set storage path parameters, which is described in step S250, and specifically, the method may further include the following contents described in substeps S2501 to substep S2503:

step S2501, determining element weights of a plurality of behavior element information to be selected for identifying the associated data and linear element matching coefficients among different behavior element information according to first data to be compressed and second data to be compressed which are used for marking the associated data in the category information;

in the present embodiment, each of the first data to be compressed is continuous data of associated data, and each of the second data to be compressed is discrete data of associated data.

Step S2502, selecting the behavior element information based on the determined element weights of the behavior element information and linear element matching coefficients among different behavior element information, so that the element weight of the selected behavior element information is larger than a set weight value, and the linear element matching coefficient among the selected behavior element information is smaller than a preset matching coefficient;

step S2503, aiming at any preset reference data, according to element mapping data of the preset reference data under each behavior element information in the selected behavior element information, identifying the associated data to obtain dynamic data with a compression identifier, and determining a target storage path according to the dynamic data and the set storage path parameters.

By executing the contents described in the above steps S2501 to S2503, the multiple behavior element information is selected based on the determined element weights of the multiple behavior element information and the linear element matching coefficients between different behavior element information, further, for any preset reference data, according to the element mapping data of the preset reference data under each behavior element information in the selected behavior element information, the associated data is identified to obtain dynamic data with a compressed identifier, and a target storage path is determined according to the dynamic data and the set storage path parameter. In this way, the dynamic data with the compressed identification is obtained by identifying the associated data, and the dynamic data is further compressed and stored, so that the storage space can be greatly saved. Furthermore, the target storage path is determined through the dynamic data and the set storage path parameters, and the user can directly store the data into the determined target storage path, so that the user can conveniently search the data in the later period.

Further, in order to obtain redundant data in the dynamic data by performing redundancy processing on the stored dynamic data in step S260, remove the redundant data from the dynamic data, and add multi-level data labels to the remaining data in the dynamic data, the method may specifically include the following steps described in substeps S2601 to substep S2604:

step S2601, before redundant processing is performed on the stored dynamic data, collecting data characteristic information of the dynamic data, and analyzing the data characteristic information through node equipment in the data storage platform to obtain corresponding analysis text information;

step S2602, performing redundancy processing on the stored dynamic data based on the parsed text information to obtain redundant data in the dynamic data, and removing the redundant data in the dynamic data through multiple different removing modes to obtain residual data in the dynamic data;

step S2603, dividing the remaining data in the dynamic data into multi-level data according to a preset hierarchical manner; determining target single-level data in the multi-level data of the same category; the multi-level data is used for representing data corresponding to the step of dividing the rest data in the dynamic data into a plurality of levels;

step S2604, adding data labels to the multi-level data of the same category by adopting a preset data label adding mode according to the target single-level data.

In this embodiment, each level of data corresponds to a data tag.

In this way, by executing the contents described in the above steps S2601 to S2604, the collected data feature information is firstly analyzed, and the stored dynamic data is further subjected to redundancy processing based on the obtained analysis text information to obtain redundant data in the dynamic data. And then, removing redundant data in the state data, further performing multi-level data division on the obtained residual data, and then adding data labels to the multi-level data of the same category based on the determined target single-level data and a preset data label adding mode.

Therefore, redundant data in the state data are removed, a large amount of storage space can be greatly saved, the utilization rate of the storage space is further improved, the storage and maintenance cost is reduced, and the requirement of a user on large data storage space is met.

Furthermore, the remaining data is divided into multiple levels of data, so that the hierarchy of the remaining data in the multi-level search can be ensured, a user can conveniently search the stored data through the data tags of different levels, and the user experience is improved.

Based on the same inventive concept, the invention also provides a data storage management system, which comprises a data storage platform and the user terminal; the data storage platform is communicated with the user terminal;

the data storage platform is to:

In an alternative embodiment, determining a plurality of to-be-stored data lists of the to-be-stored data based on the current data type information specifically includes:

In an alternative embodiment, the obtaining a data list of each to-be-stored data list, and encoding the plurality of to-be-stored data lists according to the data list specifically includes:

In an alternative embodiment, determining a target to-be-stored data list in a plurality of encoded to-be-stored data lists, and identifying category information corresponding to each data list in the target to-be-stored data list specifically includes:

In an alternative embodiment, identifying, according to the category information, dynamic data with a compression identifier from the associated data, and determining a target storage path according to the dynamic data and a set storage path parameter, specifically include:

On the basis, please refer to fig. 3, the present invention further provides a data storage management apparatus 400, applied to a data storage platform, the apparatus including:

a storage instruction receiving module 410, configured to receive a target data storage instruction, where the target data storage instruction includes current data type information of data to be stored, and the current data type information refers to a system data type corresponding to the data storage platform;

an associated data obtaining module 420, configured to obtain corresponding associated data based on the received target data storage instruction; the associated data is key data of each data to be stored, and the data to be stored is uploaded to the data storage platform by the service terminal;

a data list determining module 430, configured to determine multiple to-be-stored data lists of the to-be-stored data based on the current data type information; each to-be-stored data list comprises a plurality of storage indexes;

the data list encoding module 440 is configured to obtain a data list of each to-be-stored data list, encode the multiple to-be-stored data lists according to the data list, determine a target to-be-stored data list in the encoded multiple to-be-stored data lists, and identify category information corresponding to each data list in the target to-be-stored data list;

the dynamic data storage module 450 is configured to identify dynamic data with a compression identifier from the associated data according to the category information, and determine a target storage path according to the dynamic data and a set storage path parameter, so as to store the dynamic data according to the target storage path;

the data tag adding module 460 is configured to perform redundancy processing on stored dynamic data to obtain redundant data in the dynamic data, remove the redundant data from the dynamic data, and add multi-level data tags to remaining data in the dynamic data.

On the basis of the above, referring to fig. 4, a data storage platform 200 is provided, which includes a processor 210, and a memory 220 and a bus 230 connected to the processor 210; wherein, the processor 210 and the memory 220 complete communication with each other through the bus 230; the processor 210 is used to call the program instructions in the memory 220 to execute the above-mentioned method.

Further, a readable storage medium is provided, on which a program is stored, which when executed by a processor implements the method described above.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. A data storage management method is applied to a data storage platform, and comprises the following steps:

2. The method according to claim 1, wherein determining a plurality of to-be-stored data lists of the to-be-stored data based on the current data type information specifically includes:

3. The method according to claim 1, wherein obtaining a data list of each to-be-stored data list, and encoding the plurality of to-be-stored data lists according to the data list specifically includes:

4. The method according to claim 1, wherein determining a target data list to be stored in the encoded multiple data lists to be stored, and identifying category information corresponding to each data list in the target data list to be stored specifically includes:

5. The method according to claim 1, wherein the identifying of the dynamic data with the compressed identifier from the associated data according to the category information and determining the target storage path according to the dynamic data and the set storage path parameter specifically include:

6. A data storage management system is characterized by comprising a data storage platform and a user terminal which are communicated with each other;

the data storage platform is to:

7. The system according to claim 6, wherein determining a plurality of to-be-stored data lists of the to-be-stored data based on the current data type information specifically includes:

8. The system according to claim 6, wherein obtaining a data list of each to-be-stored data list, and encoding the plurality of to-be-stored data lists according to the data list specifically includes:

9. The system according to claim 6, wherein determining a target data list to be stored in the encoded multiple data lists to be stored, and identifying category information corresponding to each data list in the target data list to be stored specifically includes:

10. The system according to claim 6, wherein the identifying of the dynamic data with the compressed identifier from the associated data according to the category information and the determining of the target storage path according to the dynamic data and the set storage path parameter specifically include: