CN115543941A - Data storage optimization processing method - Google Patents
Data storage optimization processing method Download PDFInfo
- Publication number
- CN115543941A CN115543941A CN202211528212.8A CN202211528212A CN115543941A CN 115543941 A CN115543941 A CN 115543941A CN 202211528212 A CN202211528212 A CN 202211528212A CN 115543941 A CN115543941 A CN 115543941A
- Authority
- CN
- China
- Prior art keywords
- client
- folder
- interactive data
- compression
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
Abstract
The invention belongs to the technical field of data storage optimization, and discloses a data storage optimization processing method, which comprises the steps of storing interactive data stored in a client interactive data disk in a client folder mode, processing and analyzing each client folder, screening out target folders according to the client folders, further distributing required compression spaces of each target folder by combining the target folders with spaces to be compressed corresponding to the client interactive data disk, determining the interactive data to be compressed corresponding to each target folder, and realizing intelligent determination of the interactive data to be compressed.
Description
Technical Field
The invention belongs to the technical field of data storage optimization, and particularly relates to a data storage optimization processing method.
Background
The rapid development of the current internet causes the network data information to show an explosive growth trend, and in order to ensure the permanent security of data storage, data storage is more and more accepted by enterprises. Especially, the customer management type enterprise generates a large amount of interactive data such as communication records, purchase records, etc. due to its direct contact with the customer, and the interactive data is very useful for understanding the customer in the whole business cycle, so that the storage of the interactive data is very necessary.
With the rapid development of social economy and the continuous improvement of living standard of people, the number of customers served by customer management type enterprises is increased, new interactive data is generated almost every day, and the space of a customer interactive data disk is more and more insufficient. It is becoming more and more important to maximize the use of disk resources. The best method for improving the utilization rate of the disk at present is to compress interactive data stored in a client interactive data disk by adopting a disk compression technology so as to replace more disk space. Not all interactive data is suitable for compression, since it will cause some hindrance to the use of the data after compression. In this case, the primary operation of performing interactive data compression is to determine interactive data to be compressed.
However, in the prior art, the determination of the interactive data to be compressed is performed manually, the subjectivity is strong, scientific and objective reference is lacked, and phenomena of missing compression and unreasonable compression easily occur, so that the determination efficiency of the interactive data to be compressed is reduced, the compression effect may not meet the expectation, and great inconvenience is brought to subsequent interactive data calling.
Disclosure of Invention
Therefore, the present invention is directed to a data storage optimization processing method, which at least solves one of the technical problems in the related art to some extent.
The purpose of the invention can be realized by the following technical scheme: a data storage optimization processing method comprises the following steps: (1) And counting the number of client folders existing in the client interactive data disc, and numbering the client folders according to the sequence of creation time points, wherein each client folder corresponds to one client.
(2) And acquiring the total storage space corresponding to the customer interactive data disk and the storage space corresponding to each customer folder.
(3) And counting the space to be compressed corresponding to the interactive data disk of the client by combining the storage space corresponding to each client folder with the set compression ratio.
(4) And respectively extracting the client information corresponding to each client folder, and analyzing the client grade corresponding to each client folder according to the client information.
(5) And respectively counting the quantity of the interactive data stored in each client folder, numbering each piece of interactive data, and simultaneously acquiring the display attribute corresponding to each piece of interactive data.
(6) And respectively setting the storage time period corresponding to each client folder, thereby acquiring the use parameters corresponding to each client folder in the storage time period corresponding to each client folder.
(7) And judging whether each client folder is suitable for compression or not based on the display attribute of each interactive data in each client folder and the use parameter corresponding to each client folder, and recording the client folder which is judged to be suitable for compression as a target folder.
(8) And counting the number of the target folders, acquiring the number of each target folder, extracting the client grade, the storage space, the display attribute and the use parameter of each interactive data corresponding to each target folder, and distributing the required compression space of each target folder by combining the required compression space with the space to be compressed corresponding to the client interactive data disk.
(9) And determining interactive data to be compressed corresponding to each target folder.
Based on the improved technical scheme, the specific implementation manner of the space to be compressed corresponding to the customer interaction data disk in the step (3) is as follows: (31) And accumulating the storage spaces corresponding to the client folders to obtain the stored spaces corresponding to the client interactive data discs.
(32) The stored space corresponding to the customer interactive data disk and the set compression rate are formulated
Calculating the space to be compressed corresponding to the customer interactive data disk
Wherein
Represented as the corresponding stored space of the customer interaction data disc,
expressed as the set compression rate.
Based on the improved technical scheme, the client information comprises the client cooperation times and the cooperation amount corresponding to each cooperation.
Based on the improved technical scheme, the analysis of the client level corresponding to each client folder specifically refers to the following analysis steps: (41) Extracting the client cooperation times from the client information, and further calculating the client cooperation tightness corresponding to each client folder based on the client cooperation times corresponding to each client folder
In which
，
Expressed as the number of client collaborations corresponding to the ith client folder, i is expressed as the client folder number,
。
（42) Extracting the cooperation amount corresponding to each cooperation from the client information, further carrying out mean calculation on the cooperation amount corresponding to each cooperation in each client folder to obtain the average cooperation amount of the client corresponding to each client folder, and calculating the proportion of the client cooperation amount corresponding to each client folder according to the average cooperation amount
Wherein
，
Expressed as the average collaboration amount of the customer corresponding to the ith customer folder.
(43) Will be provided with
And
substituting into the evaluation formula of the degree of cooperation superiority of the client
Calculating the client cooperation dominance corresponding to each client folder
And a and b are respectively expressed as the proportion factors corresponding to the preset client cooperation compactness and the client cooperation amount proportion.
(44) And matching the client cooperation superiority corresponding to each client folder with the client cooperation superiority interval corresponding to each predefined client grade, and matching the client grade corresponding to each client folder from the client cooperation superiority interval.
Based on the improved technical scheme, the display attributes comprise display content categories and display formats, wherein the display formats comprise documents, pictures and videos.
Based on the improved technical scheme, the use parameters comprise use frequency, adjacent use interval duration and use duration corresponding to each use.
Based on the improved technical scheme, the specific setting mode for setting the storage time period corresponding to each client folder is as follows: and taking the time period between the creation time point corresponding to each client folder and the current time point as the storage time period corresponding to each client folder.
Based on the improved technical scheme, the judging whether each client folder is suitable for compression specifically comprises: (71) And extracting display content categories from the display attributes, comparing the display content categories corresponding to the interactive data in each client folder with the importance degrees corresponding to the display content categories stored in the optimized database, and screening out the importance degrees corresponding to the interactive data in each client folder.
(72) Extracting the maximum importance from the importance corresponding to each interactive data in each client folder to be used as the interactive data reference importance corresponding to each client folder, thereby centralizing the index through the importance
Calculating the importance concentration index corresponding to each client folder
Wherein
Expressed as the importance corresponding to the kth interactive data in the ith client folder, k is expressed as the interactive data number,
，
the reference importance of the interactive data corresponding to the ith client folder is expressed, z is the number of the interactive data, and e is a natural constant.
(73) Each clientThe use parameters corresponding to the folders are calculated by using the normal degree
Calculating the usage constant degree corresponding to each client folder
Wherein
Is expressed as the usage duration corresponding to the jth usage in the ith client folder, j is expressed as the usage number,
，
expressed as the length of time that the ith client folder corresponds to the storage period,
the interval duration between the (j + 1) th use and the (j) th use in the ith client folder is represented, and m is represented as the use frequency.
(74) Substituting the importance concentration index and the use normality corresponding to each client folder into a storage utility index evaluation formula
Evaluating the storage utility index corresponding to each client folder
Wherein
And is expressed as a weight factor corresponding to the set importance concentration index.
(75) And comparing the storage utility index corresponding to each client folder with the set critical storage utility index, and if the storage utility index corresponding to a certain client folder is smaller than the critical storage utility index, judging that the client folder is suitable for compression.
Based on the improved technical scheme, the step of allocating the required compressed space of each target folder specifically comprises the following steps: (81) And extracting the display format from the display attribute, further extracting the display format corresponding to each interactive data in each target folder based on the number of each target folder, matching the display format with the compression capability index corresponding to each display format stored in the optimized database, and matching the compression capability index corresponding to each interactive data in each target folder.
(82) And carrying out average calculation on the compression capacity indexes corresponding to the interactive data in the target folders to obtain the average compression capacity index corresponding to the target folders.
(83) And extracting the importance degree centralized index and the use constant degree corresponding to each target folder based on the number of each target folder.
(84) Substituting the client grade, average compression capability index, importance centralization index and use normality corresponding to each target folder into a formula
Calculating the compression value degree corresponding to each target folder
Where f is represented as the number of the target folder,
，
indicated as the customer rating corresponding to the f-th target folder,
expressed as the average compressibility index corresponding to the f-th target folder,
、
respectively expressed as the importance concentration index and the use normal degree corresponding to the f-th target folder.
(85) Calculating the ratio of the storage space corresponding to each target folder to the total storage space corresponding to the client interactive data disk to obtain the compression redundancy corresponding to each target folder, and recording the compression redundancy as
。
(86) Carrying out proportional operation on the compression value degree and the compression richness corresponding to each target folder
To obtain the compression ratio corresponding to each target folder
。
(87) The compression proportion corresponding to each target folder and the space to be compressed corresponding to the customer interaction data disk pass through a demand compression space formula
Calculating the required compression space corresponding to each target folder
。
Based on the improved technical scheme, the step of determining the interactive data to be compressed corresponding to each target folder specifically refers to the following steps: (91) Counting the use times corresponding to each interactive data in each target folder in the storage time period corresponding to each target folder, and analyzing the use frequency corresponding to each interactive data in each target folder according to the use times
Analysis formula thereof
，
Expressed as the usage times corresponding to the kth interactive data in the fth target folder,
indicated as the usage frequency corresponding to the f-th target folder.
(92) Calculating the compression utilization degree corresponding to each interactive data in each target folder according to the importance degree, the use frequency and the compression capability index corresponding to each interactive data in each target folder
The calculation formula is
，
Expressed as the importance corresponding to the kth interactive data in the fth target folder,
and the compression capacity index corresponding to the kth interactive data in the f-th target folder is shown.
(93) And sequencing the interactive data in each target folder according to the sequence of the compression utilization degrees from large to small to obtain the interactive data sequencing result corresponding to each target folder.
(94) Compressing according to the interactive data sequencing result corresponding to each target folder, obtaining the compression space corresponding to each target folder after each piece of interactive data is compressed, comparing the compression space with the demand compression space corresponding to the target folder, stopping the compression of the target folder when the compression space after the compression of a certain piece of interactive data in a certain target folder reaches the demand compression space corresponding to the target folder, and recording the interactive data as cut-off interactive data at the moment, thereby obtaining the cut-off interactive data corresponding to each target folder.
(95) And extracting all interactive data between the first piece of interactive data and the cut-off interactive data from the interactive data sequencing result corresponding to each target folder to be used as interactive data to be compressed corresponding to each target folder.
By combining all the technical schemes, the invention has the advantages and positive effects that: (1) The interactive data stored in the client interactive data disk is stored in the form of the client folders, and therefore, the target folders are screened out through processing and analyzing the client folders, the target folders are further distributed in combination with the spaces to be compressed corresponding to the client interactive data disk, and meanwhile, the interactive data to be compressed corresponding to the target folders are determined, so that the intelligent determination of the interactive data to be compressed is realized.
(2) In the process of screening the target folder, the influence of the display attribute of the interactive data in each client folder and the use parameters of each client folder on whether the client folder is suitable for compression is fully considered, the storage utility index corresponding to each client folder is comprehensively analyzed, and the storage utility index is used as the screening basis of the target folder, so that the screening precision of the target folder is improved to the maximum extent, a reliable determination range main body is provided for the determination of the interactive data to be compressed in the target folder, the range deviation of the compression main body is reduced, the occurrence of secondary compression is effectively avoided, and the method has higher practical operation advantages.
(3) In the process of determining the interactive data to be compressed corresponding to each target folder, the importance degree, the use frequency and the compression capability index of each interactive data stored in each target folder are analyzed, so that the compression utilization degree corresponding to each interactive data in each target folder is counted, and then the interactive data are sequenced according to the calculation result, so that the interactive data are used as the basis for determining the interactive data to be compressed, and the determination of the interactive data to be compressed is more convenient.
Drawings
The invention is further illustrated by means of the attached drawings, but the embodiments in the drawings do not constitute any limitation to the invention, and for a person skilled in the art, without inventive effort, further drawings may be derived from the following figures.
FIG. 1 is a flow chart of the method steps of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the present invention provides a data storage optimization processing method, including the following steps: (1) And counting the number of client folders existing in the client interactive data disc, and numbering the client folders according to the sequence of creation time points, wherein each client folder corresponds to one client.
(2) And acquiring a total storage space corresponding to the client interactive data disk and a storage space corresponding to each client folder.
(3) And (3) counting the space to be compressed corresponding to the client interactive data disk by combining the storage space corresponding to each client folder with the set compression ratio, wherein the specific implementation mode is as follows: (31) And accumulating the storage spaces corresponding to the client folders to obtain the stored spaces corresponding to the client interactive data discs.
(32) The stored space corresponding to the customer interactive data disk and the set compression rate are formulated
Calculating the space to be compressed corresponding to the customer interactive data disk
Wherein
Represented as the corresponding stored space of the customer interaction data disc,
expressed as a set compression rate.
(4) And respectively extracting client information corresponding to each client folder, and analyzing the client grade corresponding to each client folder according to the client information, wherein the client information comprises the client cooperation times and the cooperation amount corresponding to each cooperation.
The analysis of the client level corresponding to each client folder specifically refers to the following analysis steps: (41) Extracting the client cooperation times from the client information, and further calculating the client cooperation tightness corresponding to each client folder based on the client cooperation times corresponding to each client folder
Wherein
，
Expressed as the number of client collaborations corresponding to the ith client folder, i is expressed as the client folder number,
。
(42) Extracting cooperation fund corresponding to each cooperation from client informationAnd further carrying out average calculation on the cooperation amount corresponding to each cooperation in each client folder to obtain the average client cooperation amount corresponding to each client folder, and calculating the proportion of the client cooperation amount corresponding to each client folder according to the average client cooperation amount
Wherein
，
Expressed as the average collaboration amount of the customer corresponding to the ith customer folder.
(43) Will be provided with
And
substituting into the evaluation formula of the degree of cooperation superiority of the client
Calculating the client cooperation dominance degree corresponding to each client folder
And a and b are respectively expressed as the proportion factors corresponding to the preset client cooperation compactness and the client cooperation amount proportion, wherein the client cooperation compactness and the client cooperation amount proportion positively influence the client cooperation superiority.
(44) And matching the client cooperation superiority corresponding to each client folder with the client cooperation superiority interval corresponding to each predefined client grade, and matching the client grade corresponding to each client folder from the client cooperation superiority interval.
It should be noted that the customer ranks mentioned above are indicated by numbers, for example, 1 rank, 2 ranks, and the larger the number, the higher the customer rank is indicated.
(5) The method comprises the steps of respectively counting the quantity of interactive data stored in each client folder, numbering each piece of interactive data, and simultaneously obtaining display attributes corresponding to each piece of interactive data, wherein the display attributes comprise display content types and display formats, the display content types refer to the types of the content of each piece of interactive data, and specifically comprise pre-sale communication information types, purchase information types, post-sale feedback information types and the like, and the display formats comprise documents, pictures and videos.
(6) And respectively setting storage time periods corresponding to the client folders, and acquiring the use parameters corresponding to the client folders in the storage time periods corresponding to the client folders, wherein the use parameters comprise use frequency, adjacent use interval duration and use duration corresponding to each use.
In a specific embodiment, the specific setting manner of setting the storage time period corresponding to each client folder is as follows: and taking the time period between the creation time point corresponding to each client folder and the current time point as the storage time period corresponding to each client folder.
(7) And judging whether each client folder is suitable for compression or not based on the display attribute of each piece of interactive data in each client folder and the corresponding use parameter of each client folder, and recording the client folder which is judged to be suitable for compression as a target folder.
In an embodiment of the present invention, the determining whether each client folder is suitable for compression specifically comprises: (71) And extracting display content categories from the display attributes, comparing the display content categories corresponding to the interactive data in each client folder with the importance degrees corresponding to the display content categories stored in the optimized database, and screening out the importance degrees corresponding to the interactive data in each client folder.
(72) Extracting the maximum importance from the importance corresponding to each interactive data in each client folder to be used as the interactive data reference importance corresponding to each client folder, thereby centralizing the index through the importance
Calculating the importance concentration index corresponding to each client folder
Wherein
Expressed as the importance corresponding to the kth interactive data in the ith client folder, k is expressed as the interactive data number,
，
the reference importance of the interactive data corresponding to the ith client folder is expressed, z is the number of the interactive data, and e is a natural constant.
It should be explained that the above-mentioned importance concentration index calculation formula
And indicating the deviation degree between the importance degree of each piece of interactive data and the reference importance degree of the interactive data, wherein the smaller the deviation degree in a certain client folder is, the closer the importance degree of each piece of interactive data in the client folder is to the reference importance degree of the interactive data, indicating that the importance degree of the client folder is more concentrated, and further reflecting the importance degree of the interactive data in the client folder from the side.
(73) Calculating the use parameters corresponding to each client folder by using a normal degree calculation formula
Calculating the usage normality corresponding to each client folder
In which
Is expressed as the usage duration corresponding to the jth usage in the ith client folder, j is expressed as the usage number,
，
expressed as the length of time that the ith client folder corresponds to the storage period,
the interval duration between the (j + 1) th use and the (j) th use in the ith client folder is represented, and the use frequency is represented by m, wherein the longer the duration of each use of the client folder is, the shorter the duration of the adjacent time interval is, the more normal the use of the client folder is represented.
(74) Substituting the importance concentration index and the use normality corresponding to each client folder into a storage utility index evaluation formula
Evaluating the storage utility index corresponding to each client folder
In which
Expressed as a weighting factor corresponding to the set importance concentration index.
In the storage utility index evaluation formula, the larger the importance concentration index corresponding to a client folder is, the larger the usage normality is, the larger the storage utility index corresponding to the client folder is, which indicates that the storage use of the client folder is larger, and if a client folder with a larger storage use is compressed, the inconvenience in use of the client folder is caused, so that the client folder with a larger storage use is less suitable for compression.
(75) And comparing the storage utility index corresponding to each client folder with the set critical storage utility index, and if the storage utility index corresponding to a certain client folder is smaller than the critical storage utility index, judging that the client folder is suitable for compression.
In the process of screening the target folder, the display attribute of the interactive data in each client folder and the influence of the use parameters of each client folder on whether the client folder is suitable for compression are fully considered, the storage utility indexes corresponding to the client folders are comprehensively analyzed, and then the storage utility indexes are used as the screening basis of the target folder, so that the screening precision of the target folder is improved to the maximum extent, a reliable determination range main body is provided for determining the interactive data to be compressed in the target folder, the range deviation of the compression main body is reduced, the occurrence of secondary compression is effectively avoided, and the method and the device have high practical operation advantages.
(8) Counting the number of target folders, acquiring the number of each target folder, extracting the client grade, the storage space, the display attribute and the use parameter of each interactive data corresponding to each target folder, distributing the required compression space of each target folder by combining the required compression space with the to-be-compressed space corresponding to the client interactive data disc, determining the to-be-compressed interactive data corresponding to each target folder according to the required compression space, and specifically executing the following steps: (81) And extracting the display format from the display attribute, further extracting the display format corresponding to each interactive data in each target folder based on the number of each target folder, matching the display format with the compression capability index corresponding to each display format stored in the optimized database, and matching the compression capability index corresponding to each interactive data in each target folder.
It should be noted that the compression capabilities corresponding to different presentation formats are different, where the compression capability corresponding to the document format is the largest, which indicates that the space in which the interactive data in the document format can be compressed is very large, and the compression capability corresponding to the picture and video formats is smaller, which indicates that the space in which the interactive data in the picture and video formats can be compressed is limited.
(82) And carrying out average calculation on the compression capacity indexes corresponding to the interactive data in the target folders to obtain the average compression capacity index corresponding to the target folders.
(83) And extracting the importance degree centralized index and the use constant degree corresponding to each target folder based on the number of each target folder.
(84) Substituting the client grade, average compression capability index, importance centralization index and use normality corresponding to each target folder into a formula
Calculating the compression value degree corresponding to each target folder
Where f is represented as the number of the target folder,
，
indicated as the client level corresponding to the fth target folder,
expressed as the average compressibility index corresponding to the f-th target folder,
、
respectively expressed as the importance concentration index and the use constant corresponding to the f-th target folder.
It can be explained that the influence of the client level, the importance concentration index and the usage normality corresponding to the target folder on the compression value is negative, and the influence of the average compression capability index on the compression value is positive, because the higher the client level of the target folder is, the more important the interactive data content is, the more normal the usage is, the more prominent the importance of the interactive data in the target folder is, and the less suitable the deep compression is.
(85) Calculating the ratio of the storage space corresponding to each target folder to the total storage space corresponding to the customer interactive data disk to obtain the compression redundancy corresponding to each target folder, and recording the compression redundancy as
Wherein the compression margin reflects the size of the storage space of the target folder.
(86) Carrying out proportional operation on the compression value degree and the compression richness corresponding to each target folder
To obtain the compression ratio corresponding to each target folder
。
(87) The compression proportion corresponding to each target folder and the space to be compressed corresponding to the customer interaction data disk pass through a demand compression space formula
Calculating the required compression space corresponding to each target folder
。
(9) Determining interactive data to be compressed corresponding to each target folder, and specifically referring to the following steps: (91) Counting the use times corresponding to each interactive data in each target folder in the storage time period corresponding to each target folder, and analyzing the use frequency corresponding to each interactive data in each target folder according to the use times
Analysis formula thereof
，
Expressed as the usage times corresponding to the kth interactive data in the fth target folder,
expressed as f-th target folder correspondenceIs used frequently.
(92) Calculating the compression utilization degree corresponding to each interactive data in each target folder according to the importance degree, the use frequency and the compression capacity index corresponding to each interactive data in each target folder
The calculation formula is
，
Expressed as the importance corresponding to the kth interactive data in the fth target folder,
and the compression capacity index corresponding to the kth interactive data in the f-th target folder is shown.
(93) And sequencing the interactive data in each target folder according to the sequence of the compression utilization degrees from large to small to obtain the interactive data sequencing result corresponding to each target folder.
(94) Compressing according to the interactive data sequencing result corresponding to each target folder, obtaining the compression space corresponding to each target folder after each piece of interactive data is compressed, comparing the compression space with the demand compression space corresponding to the target folder, stopping the compression of the target folder when the compression space after the compression of a certain piece of interactive data in a certain target folder reaches the demand compression space corresponding to the target folder, and recording the interactive data as cut-off interactive data at the moment, thereby obtaining the cut-off interactive data corresponding to each target folder.
(95) And extracting all interactive data between the first interactive data and the cut-off interactive data from the interactive data sequencing result corresponding to each target folder to be used as the interactive data to be compressed corresponding to each target folder.
In the embodiment, in the process of determining the interactive data to be compressed corresponding to each target folder, importance, use frequency and compression capability index analysis are performed on each interactive data stored in each target folder, so that the compression availability corresponding to each interactive data in each target folder is counted, and then interactive data is sequenced according to the analysis, so that the interactive data is used as a basis for determining the interactive data to be compressed, and the determination of the interactive data to be compressed is facilitated.
The interactive data stored in the client interactive data disk is stored in the form of the client folders, and therefore, the target folders are screened out through processing and analyzing the client folders, the target folders are further distributed in combination with the spaces to be compressed corresponding to the client interactive data disk, and meanwhile, the interactive data to be compressed corresponding to the target folders are determined, so that the intelligent determination of the interactive data to be compressed is realized.
The foregoing is merely exemplary and illustrative of the present invention and various modifications, additions and substitutions may be made by those skilled in the art to the specific embodiments described without departing from the scope of the invention as defined in the following claims.
Claims (10)
1. A data storage optimization processing method is characterized by comprising the following steps:
(1) Counting the number of client folders existing in a client interactive data disc, and numbering the client folders according to the sequence of creation time points, wherein each client folder corresponds to one client;
(2) Acquiring a total storage space corresponding to a client interactive data disc and a storage space corresponding to each client folder;
(3) The storage space corresponding to each client folder is combined with the set compression rate to count the space to be compressed corresponding to the client interactive data disk;
(4) Respectively extracting client information corresponding to each client folder, and analyzing client grades corresponding to each client folder according to the client information;
(5) Respectively counting the quantity of the interactive data stored in each client folder, numbering each piece of interactive data, and simultaneously acquiring the display attribute corresponding to each piece of interactive data;
(6) Respectively setting storage time periods corresponding to the client folders, and acquiring the use parameters corresponding to the client folders in the storage time periods corresponding to the client folders;
(7) Judging whether each client folder is suitable for compression or not based on the display attribute of each interactive data in each client folder and the use parameter corresponding to each client folder, and recording the client folder which is judged to be suitable for compression as a target folder;
(8) Counting the number of the target folders, acquiring the number of each target folder, extracting the client grade, the storage space, the display attribute and the use parameter of each interactive data corresponding to each target folder, and further distributing the required compression space of each target folder by combining the required compression space with the space to be compressed corresponding to the client interactive data disc;
(9) And determining interactive data to be compressed corresponding to each target folder.
2. A data storage optimization processing method according to claim 1, characterized in that: the specific implementation manner of the space to be compressed corresponding to the customer interaction data disk in the step (3) is as follows:
(31) Accumulating the storage space corresponding to each client folder to obtain the stored space corresponding to the client interactive data disc;
(32) The stored space corresponding to the customer interactive data disk and the set compression rate are formulated
Calculating the space to be compressed corresponding to the customer interactive data disk
Wherein
Represented as the corresponding stored space of the customer interaction data disc,
expressed as the set compression rate.
3. The data storage optimization processing method according to claim 1, wherein: the client information comprises the number of times of client cooperation and the cooperation amount corresponding to each cooperation.
4. A data storage optimization processing method according to claim 3, wherein: the analysis of the client level corresponding to each client folder specifically refers to the following analysis steps:
(41) Extracting the client cooperation times from the client information, and further calculating the client cooperation tightness corresponding to each client folder based on the client cooperation times corresponding to each client folder
In which
，
Expressed as the number of client collaborations corresponding to the ith client folder, i is expressed as the client folder number,
；
（42）extracting the cooperation amount corresponding to each cooperation from the client information, further carrying out mean calculation on the cooperation amount corresponding to each cooperation in each client folder to obtain the average cooperation amount of the client corresponding to each client folder, and calculating the proportion of the client cooperation amount corresponding to each client folder according to the average cooperation amount
Wherein
，
The average collaboration amount of the clients corresponding to the ith client folder is expressed;
(43) Will be provided with
And
substituting into the evaluation formula of the degree of cooperation superiority of the client
Calculating the client cooperation dominance corresponding to each client folder
Wherein a and b are respectively expressed as the proportion factors corresponding to the preset client cooperation compactness and the client cooperation amount proportion;
(44) And matching the client cooperation superiority corresponding to each client folder with the client cooperation superiority interval corresponding to each predefined client grade, and matching the client grade corresponding to each client folder from the client cooperation superiority interval.
5. The data storage optimization processing method according to claim 4, wherein: the display attributes comprise a display content category and a display format, wherein the display format comprises a document, a picture and a video.
6. The data storage optimization processing method according to claim 5, wherein: the use parameters comprise use frequency, adjacent use interval duration and use duration corresponding to each use.
7. The data storage optimization processing method according to claim 1, wherein: the specific setting mode for setting the storage time period corresponding to each client folder is as follows: and taking the time period between the creation time point corresponding to each client folder and the current time point as the storage time period corresponding to each client folder.
8. The data storage optimization processing method according to claim 6, wherein: the step of judging whether each client folder is suitable for compression specifically comprises the following steps:
(71) Extracting display content categories from the display attributes, comparing the display content categories corresponding to the interactive data in each client folder with the importance degrees corresponding to the display content categories stored in the optimized database, and screening out the importance degrees corresponding to the interactive data in each client folder;
(72) Extracting the maximum importance from the importance corresponding to each interactive data in each client folder to be used as the interactive data reference importance corresponding to each client folder, thereby centralizing the index through the importance
Calculating the importance concentration index corresponding to each client folder
Wherein
Is expressed as the importance corresponding to the kth interactive data in the ith client folderDegree, k, is denoted as the interactive data number,
，
expressing the interactive data reference importance corresponding to the ith client folder, expressing z as the interactive data quantity, and expressing e as a natural constant;
(73) The use parameters corresponding to each client folder are calculated by using a normal degree calculation formula
Calculating the usage normality corresponding to each client folder
Wherein
Is expressed as the usage duration corresponding to the jth usage in the ith client folder, j is expressed as the usage number,
，
expressed as the length of time that the ith client folder corresponds to the storage period,
the interval duration between the (j + 1) th use and the (j) th use in the ith client folder is represented, and m is the use frequency;
(74) Substituting the importance concentration index and the use normality corresponding to each client folder into a storage utility index evaluation formula
Evaluating each client fileClip corresponding storage utility index
Wherein
The weight factor is expressed as the corresponding weight factor of the set importance concentration index;
(75) And comparing the storage utility index corresponding to each client folder with the set critical storage utility index, and if the storage utility index corresponding to a certain client folder is smaller than the critical storage utility index, judging that the client folder is suitable for compression.
9. The data storage optimization processing method according to claim 8, wherein: the step of allocating the required compressed space of each target folder specifically comprises the following steps:
(81) Extracting a display format from the display attribute, further extracting the display format corresponding to each interactive data in each target folder based on the number of each target folder, matching the display format with the compression capability index corresponding to each display format stored in the optimized database, and obtaining the compression capability index corresponding to each interactive data in each target folder through matching;
(82) Performing mean calculation on the compression capacity indexes corresponding to the interactive data in the target folders to obtain average compression capacity indexes corresponding to the target folders;
(83) Extracting an importance degree centralized index and a use normality corresponding to each target folder based on the number of each target folder;
(84) Substituting the client grade, average compression capability index, importance centralization index and use normality corresponding to each target folder into a formula
Calculating the compression value degree corresponding to each target folder
Where f is represented as the number of the target folder,
，
indicated as the client level corresponding to the fth target folder,
expressed as the average compressibility index corresponding to the f-th target folder,
、
respectively expressing the importance degree concentration index and the use normal degree corresponding to the f-th target folder;
(85) Calculating the ratio of the storage space corresponding to each target folder to the total storage space corresponding to the customer interactive data disk to obtain the compression redundancy corresponding to each target folder, and recording the compression redundancy as
；
(86) Carrying out proportional operation on the compression value degree and the compression richness corresponding to each target folder
To obtain the compression ratio corresponding to each target folder
；
(87) The compression proportion corresponding to each target folder and the space to be compressed corresponding to the customer interaction data disk pass through a demand compression space formula
Calculating the required compression space corresponding to each target folder
。
10. The data storage optimization processing method according to claim 9, wherein: the step of determining the interactive data to be compressed corresponding to each target folder specifically refers to the following steps:
(91) Counting the use times corresponding to each interactive data in each target folder in the storage time period corresponding to each target folder, and analyzing the use frequency corresponding to each interactive data in each target folder according to the use times
Analysis formula thereof
，
Expressed as the number of usage times corresponding to the kth interactive data in the fth target folder,
the usage frequency corresponding to the f-th target folder is expressed;
(92) Calculating the compression utilization degree corresponding to each interactive data in each target folder according to the importance degree, the use frequency and the compression capability index corresponding to each interactive data in each target folder
The calculation formula is
，
Expressed as the importance corresponding to the kth interactive data in the fth target folder,
expressing the compression capacity index corresponding to the kth interactive data in the f target folder;
(93) Sequencing all the interactive data in all the target folders according to the sequence of the compression availability from large to small to obtain interactive data sequencing results corresponding to all the target folders;
(94) Compressing according to the interactive data sequencing result corresponding to each target folder, acquiring a compression space corresponding to each target folder after each piece of interactive data is compressed, comparing the compression space with a demand compression space corresponding to the target folder, stopping the compression of the target folder when the compression space after the compression of a certain piece of interactive data in a certain target folder reaches the demand compression space corresponding to the target folder, and recording the interactive data as cut-off interactive data at the moment, thereby obtaining the cut-off interactive data corresponding to each target folder;
(95) And extracting all interactive data between the first interactive data and the cut-off interactive data from the interactive data sequencing result corresponding to each target folder to be used as the interactive data to be compressed corresponding to each target folder.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211528212.8A CN115543941B (en) | 2022-12-01 | 2022-12-01 | Data storage optimization processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211528212.8A CN115543941B (en) | 2022-12-01 | 2022-12-01 | Data storage optimization processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115543941A true CN115543941A (en) | 2022-12-30 |
| CN115543941B CN115543941B (en) | 2023-02-17 |
Family
ID=84721760
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211528212.8A Active CN115543941B (en) | 2022-12-01 | 2022-12-01 | Data storage optimization processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115543941B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117373600A (en) * | 2023-12-04 | 2024-01-09 | 邦盛高科特种车辆（天津）有限公司 | Medical detection vehicle data optimal storage method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006195588A (en) * | 2005-01-11 | 2006-07-27 | Sony Corp | Disk system, disk control method, and computer program |
| CN102833188A (en) * | 2012-09-04 | 2012-12-19 | 上海量明科技发展有限公司 | Method, client and system for displaying transmission file in instant messaging |
| CN103248632A (en) * | 2013-05-29 | 2013-08-14 | 中国人民解放军理工大学 | Synchronous disc data security protection writing and reading method |
| CN103327171A (en) * | 2013-03-15 | 2013-09-25 | 深圳市卡迪尔通讯技术有限公司 | Talking and writing method for portable terminal with function of talking and writing short message |
| CN105811994A (en) * | 2016-03-03 | 2016-07-27 | 云南大学 | Computer data zipping and processing system |
| US10169359B1 (en) * | 2015-09-28 | 2019-01-01 | EMC IP Holding Company LLC | Distribution content-aware compression and decompression of data |
| CN111770022A (en) * | 2020-06-28 | 2020-10-13 | 中国平安财产保险股份有限公司 | Link monitoring-based capacity expansion method, system, equipment and computer storage medium |
| CN111836052A (en) * | 2020-07-06 | 2020-10-27 | Oppo广东移动通信有限公司 | Image compression method, image compression device, electronic equipment and storage medium |
| JP2021161849A (en) * | 2020-04-03 | 2021-10-11 | 株式会社神垣組 | Adhesion method bonding weight object on surface of structure |
-
2022
- 2022-12-01 CN CN202211528212.8A patent/CN115543941B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006195588A (en) * | 2005-01-11 | 2006-07-27 | Sony Corp | Disk system, disk control method, and computer program |
| CN102833188A (en) * | 2012-09-04 | 2012-12-19 | 上海量明科技发展有限公司 | Method, client and system for displaying transmission file in instant messaging |
| CN103327171A (en) * | 2013-03-15 | 2013-09-25 | 深圳市卡迪尔通讯技术有限公司 | Talking and writing method for portable terminal with function of talking and writing short message |
| CN103248632A (en) * | 2013-05-29 | 2013-08-14 | 中国人民解放军理工大学 | Synchronous disc data security protection writing and reading method |
| US10169359B1 (en) * | 2015-09-28 | 2019-01-01 | EMC IP Holding Company LLC | Distribution content-aware compression and decompression of data |
| CN105811994A (en) * | 2016-03-03 | 2016-07-27 | 云南大学 | Computer data zipping and processing system |
| JP2021161849A (en) * | 2020-04-03 | 2021-10-11 | 株式会社神垣組 | Adhesion method bonding weight object on surface of structure |
| CN111770022A (en) * | 2020-06-28 | 2020-10-13 | 中国平安财产保险股份有限公司 | Link monitoring-based capacity expansion method, system, equipment and computer storage medium |
| CN111836052A (en) * | 2020-07-06 | 2020-10-27 | Oppo广东移动通信有限公司 | Image compression method, image compression device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| FENG LIANG 等: "Hardware Oriented Vision System of Logistics Robotics", 《ANTI-COUNTERFEITING, SECURITY, AND IDENTIFICATION》 * |
| 祁长兴: "面向异构平台的数据迁移系统的设计与实现", 《电子技术与软件工程》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117373600A (en) * | 2023-12-04 | 2024-01-09 | 邦盛高科特种车辆（天津）有限公司 | Medical detection vehicle data optimal storage method |
| CN117373600B (en) * | 2023-12-04 | 2024-02-20 | 邦盛高科特种车辆（天津）有限公司 | Medical detection vehicle data optimal storage method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115543941B (en) | 2023-02-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115543941B (en) | Data storage optimization processing method | |
| US10671926B2 (en) | Method and system for generating predictive models for scoring and prioritizing opportunities | |
| CN110427418A (en) | A kind of customer analysis grouping method based on client's energy value index system | |
| JP6251383B2 (en) | Calculating the probability of a defaulting company | |
| CN115309963B (en) | Intelligent archive management method, system and storage medium | |
| CN107977855B (en) | Method and device for managing user information | |
| CN115578027A (en) | Data quality evaluation method and device, electronic equipment and storage medium | |
| CN110866698A (en) | Device for assessing service score of service provider | |
| CN112634078B (en) | Large-industrial load interruption priority evaluation method based on multi-dimensional index fusion | |
| WO2021034852A1 (en) | Cryptocurrency valuation by processing near real-time and historical data from multiple cryptocurrency exchanges | |
| CN116611914A (en) | Salary prediction method and device based on grouping statistics | |
| CN116777652A (en) | Risk evaluation model-based financial analysis method | |
| CN116188050A (en) | Takeaway platform information processing system based on data analysis | |
| CN109241048A (en) | For the data processing method of data statistics, server and storage medium | |
| CN114663208A (en) | Enterprise tax intelligent management platform based on big data analysis | |
| CN114896285A (en) | Bank flow calculation service real-time index system based on multi-dimensional intermediate state aggregation | |
| CN114418322A (en) | Enterprise index management method and device and storage medium | |
| CN107545056B (en) | New technology potential information analysis system and information analysis method | |
| CN111815453A (en) | Electric power transaction operation system | |
| CN107330620B (en) | The method for carrying out resource adjustment based on business and Properties Correlation analysis and dynamic sensing | |
| CN113284007B (en) | Power consumption information processing system based on electric insurance package and processing method thereof | |
| CN111309758A (en) | Charging data verification and comparison method and device | |
| CN117252690B (en) | Loan contract online signing method and system | |
| CN115269277B (en) | Intelligent laboratory data collaborative comprehensive management system | |
| CN112926816B (en) | Vendor evaluation method, device, computer device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |