WO2020015047A1 - 基于cdn应用的存储容量评估方法和装置 - Google Patents
基于cdn应用的存储容量评估方法和装置 Download PDFInfo
- Publication number
- WO2020015047A1 WO2020015047A1 PCT/CN2018/101516 CN2018101516W WO2020015047A1 WO 2020015047 A1 WO2020015047 A1 WO 2020015047A1 CN 2018101516 W CN2018101516 W CN 2018101516W WO 2020015047 A1 WO2020015047 A1 WO 2020015047A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- file
- old
- cache server
- storage capacity
- files
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0896—Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Definitions
- the present invention relates to the technical field of content distribution networks, and in particular, to a method and device for evaluating storage capacity based on a CDN application.
- Service operators provide users with on-demand, live broadcast, and download services through the CDN system. These services can be implemented by using multiple cache servers in the CDN system.
- the cache server can cache data resources provided by the service source site locally. When users need When obtaining these data resources, the corresponding data resources can be obtained directly through the cache server without accessing the business source station.
- the storage capacity of the cache server can be evaluated.
- the storage capacity usage is used as an evaluation criterion to determine whether the storage capacity is full or still available, and then the storage capacity of the cache server is evaluated. Expansion or reduction.
- the current evaluation method only considers the storage capacity of the cache server and does not distinguish the types of services running on the cache server. Different services have different cache requirements. For example, live broadcast services only require the cache server to be able to store a certain amount of time in a short period of time. The data is sufficient, even if the storage capacity of the cache server is full, the capacity expansion processing may not be performed.
- the cache server is required to be able to store large amounts of data for a long time, so the cache server storage capacity is full Expansion should be performed in time to reduce the processing of data back to the source after the data is deleted.
- increasing the storage capacity of the cache server will cause a corresponding increase in storage costs, while reducing the storage capacity of the cache server will increase the amount of back-to-source data, which will lead to increased bandwidth costs. Therefore, the cache server is only based on the current storage capacity. Assessing the storage capacity has a high probability that the total cost will increase significantly. Therefore, there is an urgent need for a comprehensive and comprehensive storage capacity assessment method that can balance storage costs and bandwidth costs and reduce overall operating costs for different services.
- embodiments of the present invention provide a method and a device for evaluating a storage capacity based on a CDN application.
- the technical solution is as follows:
- a method for evaluating storage capacity based on a CDN application including:
- the cache server uses the file stored before the preset time node as the old file;
- the adjustment type of the storage capacity of the cache server Determining, according to the proportion of the old files and the popularity of the old files, the adjustment types of the storage capacity of the cache server, where the adjustment types include at least capacity to be expanded, capacity to be reduced, and capacity unchanged;
- the adjustment type of the storage capacity of the target service is determined according to the adjustment types of the storage capacity of all cache servers corresponding to the target service.
- using the file stored before the preset time node as the old file in the cache server includes:
- file operation information generated within the preset query time range according to the statistical log within the preset query time range of the cache server, the file operation information including file name, file size, file writing time, and last access time ;
- the file corresponding to the file operation information whose writing time is before the preset time node is used as an old file.
- determining the proportion of old files among all files stored by the cache server includes:
- the proportion of the old files is determined according to the total storage of the old files and the storage capacity of the cache server.
- determining the access popularity of an old file within a preset access time range includes:
- the access popularity of the old file is determined according to the total amount of active old file storage and the total amount of old file storage.
- determining the adjustment type of the storage capacity of the cache server according to the proportion of the old files and the popularity of the old files includes:
- a heat threshold is set to determine that the adjustment type of the storage capacity of the cache server is constant capacity.
- determining the adjustment type of the storage capacity of the target service according to the adjustment types of the storage capacity of all cache servers corresponding to the target service includes:
- the adjustment type of the storage capacity of the target service is set to be pending.
- a CDN application-based storage capacity evaluation device includes:
- a classification module configured to use any cache server corresponding to a target service to use a file stored before a preset time node as an old file in the cache server;
- a calculation module configured to determine the proportion of old files among all files stored by the cache server and the popularity of old files within a preset access time range
- An evaluation module configured to determine a type of adjustment of the storage capacity of the cache server according to the proportion of the old file and the popularity of the old file access, the adjustment type including at least capacity to be expanded, capacity to be reduced, and capacity unchanged;
- the adjustment types of storage capacity of all cache servers corresponding to the target service determine the adjustment types of storage capacity of the target service.
- classification module is specifically configured to:
- file operation information generated within the preset query time range according to the statistical log within the preset query time range of the cache server, the file operation information including file name, file size, file writing time, and last access time ;
- the file corresponding to the file operation information whose writing time is before the preset time node is used as an old file.
- calculation module is specifically configured to:
- the proportion of the old files is determined according to the total storage of the old files and the storage capacity of the cache server.
- calculation module is further specifically configured to:
- the access popularity of the old file is determined according to the total amount of active old file storage and the total amount of old file storage.
- evaluation module is specifically configured to:
- a heat threshold is set to determine that the adjustment type of the storage capacity of the cache server is constant capacity.
- evaluation module is specifically configured to:
- the adjustment type of the storage capacity of the target service is set to be pending.
- a cache server includes a processor and a memory.
- the memory stores at least one instruction, at least one program, code set, or instruction set.
- the at least one instruction, the at least one A program, the code set, or the instruction set is loaded and executed by the processor to implement the method for evaluating a storage capacity of a CDN-based application according to the first aspect.
- a computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, and the code.
- the set or instruction set is loaded and executed by the processor to implement the method for evaluating a storage capacity of a CDN-based application as described in the first aspect.
- the file stored before the preset time node is used as the old file in the cache server; the proportion of the old file among all files stored by the cache server and the preset access are determined.
- the adjustment types include at least capacity expansion, capacity reduction, and constant capacity.
- the adjustment type of the storage capacity of all cache servers determines the adjustment type of the storage capacity of the target service.
- the data access situation of the cache server can be judged by the proportion of the old files and the popularity of the old files, and the storage requirements and bandwidth requirements of the cache server can be obtained, and then the storage capacity adjustment type of the cache server can be determined. Evaluation to achieve the evaluation of the target business. Therefore, while considering storage costs and bandwidth costs, it can also be evaluated for different services.
- FIG. 1 is a flowchart of a method for evaluating a storage capacity based on a CDN application according to an embodiment of the present invention
- FIG. 2 is a schematic structural diagram of a storage capacity evaluation device based on a CDN application according to an embodiment of the present invention.
- An embodiment of the present invention provides a method for evaluating storage capacity based on a CDN application.
- the method can be applied to a cache server of a CDN system.
- the cache server stores files such as video, audio, text, and images, and provides users with various types of services.
- Service a business can rely on multiple cache servers to achieve common, each cache server is independent of each other, you can first evaluate the storage capacity of all cache servers, and then consider the overall evaluation of the storage capacity of the business.
- the cache server may include a processor, a memory, and a transceiver.
- the processor may be used to perform the processing of determining the adjustment type of the storage capacity in the following process.
- the memory may be used to store the data required during the processing and the generated data, and send and receive.
- the device can be used to receive and send relevant data during processing.
- the application scenario of this embodiment may be: selecting a cache server under the target service, determining the old files stored in the cache server, further determining the proportion of old files in all files stored by the cache server, and the popularity of the old files, Determine the type of adjustment of the storage capacity of the cache server according to the proportion of old files and the popularity of the old files. Perform the above processing on all cache servers under the target service, and then determine the target according to the type of adjustment of the storage capacity of all cache servers. The adjustment type of the storage capacity of the service.
- Such an evaluation method is conducive to the overall adjustment of all cache servers from the business level, which is convenient for unified management and operation.
- Step 101 For any cache server corresponding to the target service, the cache server uses the file stored before the preset time node as the old file.
- the manager first selects one of the cache servers corresponding to the target service, and then sets a time node on the cache server, and the file stored on the cache server before the time node is used as the old file. .
- the time node can be set to 7 days, and the files stored on the cache server 7 days ago are old files. It is understandable that in order to unify the evaluation criteria of all cache servers under the same service, for all cache servers under the same service , The preset time node on each server should be the same.
- the processing of the file stored before the preset time node as an old file in the cache server may be specifically as follows: according to the statistical log of the preset query time range of the cache server, obtaining the file generated within the preset query time range Information, file operation information includes file name, file size, file writing time, and last access time; use the file corresponding to the file operation information after the preset time node as the new file; write the file Let the file corresponding to the file operation information before the time node be the old file.
- a statistical log is generated on the cache server.
- the contents of the statistical log record include file operation information and file operation content.
- the operation information specifically includes content information related to file attributes such as file name, file size, file write time, and last access time
- the file operation content includes file read, write, modify, delete and other operation content information.
- the statistics logs are stored in the cache service for a long time, that is, the statistics server stores a long range of statistics logs. For a cache server that has been running for a long time, the number of statistics logs may be relatively large.
- the query time range of the statistics log can be set in advance on the cache server to reduce the amount of file operation information obtained subsequently.
- the query time range can be set to 30 days (1-30 days).
- it can also be set to another time range according to the actual situation of the business or cache server, or it can be set to a specific time range, such as X Month X-Y Month Y.
- it can also be set to get all statistics logs on the cache server.
- the cache server can convert the time node into the actual date in the query time range within the set query time range.
- the cache server After the cache server obtains the file operation information from the statistical log within the preset query time range, according to the file write time in the file operation information, the file corresponding to the file operation information whose file write time is after the preset time node is regarded as a new file.
- the file ID can also be used in the statistics log instead of the file name.
- the current date is March 30, and the preset query time range on the cache server is 30 days, and the preset time node is 7 days. Then, the actual date of the time node within the query time range is March 24 ,
- the cache server can query the statistical logs from March 1 to March 30 in all the stored statistical logs to obtain the corresponding file operation information, and then select the file whose writing time is from March 24 to March 30 Operation information. Based on the file name in these file operation information, the corresponding file is taken as the new file, and the file operation information whose writing time is from March 1 to March 23 is selected. Based on the file name in these file operation information, Use the corresponding file as the old file.
- Step 102 Determine the proportion of old files in all files stored by the cache server and the popularity of the old files within the preset access time range.
- an access time range can be set on the cache server to determine the access popularity of old files within the access time range.
- the access time range can be set to 24 hours, that is, to determine the access popularity of old files in the 24 hours before the current time. Understandably, in order to unify the evaluation criteria of all cache servers under the same service, for all cache servers under the same service , The preset access time range should be the same on each server.
- the process of determining the proportion of old files in all files stored by the cache server may be specifically as follows: determine the total amount of new files stored according to the file size of all new files; The storage capacity of the cache server determines the total storage of the old files; the proportion of the old files is determined based on the total storage of the old files and the storage capacity of the cache server.
- the cache server can obtain the file size of each new file, and then count the file sizes of all the new files to obtain the total amount of new file storage, and then according to the storage capacity of the cache server, After subtracting the total storage of the new file, the remaining part is the total storage of the old file. Finally, the ratio of the total storage of the old file to the storage capacity of the cache server can be used as the proportion of the old file. Understandably, if the total storage of new files is greater than or equal to the storage capacity of the cache server, the calculated total storage of old files is zero or negative, indicating that all files stored on the cache server are within the preset query time range. They are all new files, and even some new files may have been deleted. You should adjust the time node and query time range, for example, extend the lower limit time of the query time range, or reduce the value of the time node.
- the process of determining the access popularity of the old files within the preset access time range may be specifically as follows: determine the old files with the last access time within the preset access time range as the active old files; determine according to the file size of all active old files Total active old file storage; determine the popularity of old file access based on total active old file storage and total old file storage.
- the cache server selects the old files with the last access time within the preset access time range as the active old files from all the old files, and then obtains the file size of each active old file. , Count the file sizes of all active old files to get the total amount of active old file storage, and then based on the previously calculated total amount of old file storage, use the ratio of the total amount of active old file storage to the total amount of old file storage as the old file access heat .
- Step 103 Determine the adjustment type of the storage capacity of the cache server according to the proportion of the old files and the popularity of the old files.
- the adjustment types include at least capacity expansion, capacity reduction, and constant capacity.
- the cache server After the cache server obtains the old file proportion and the old file access popularity, according to the specific results of the old file proportion and the old file access popularity, it is possible to determine at least three types of cache server capacity adjustment types, such as capacity expansion. , To be reduced, the capacity remains unchanged.
- the process of determining the adjustment type of the storage capacity of the cache server according to the proportion of the old files and the popularity of the old files may be specifically as follows: if the proportion of the old files is greater than the preset threshold and the popularity of the old files is less than the preset
- the heat threshold determines whether the adjustment type of the storage capacity of the cache server is to be reduced.
- the adjustment type of the storage capacity of the cache server is determined as To be expanded; if the share of old files is greater than or equal to the preset share threshold and the heat value of the old files is greater than or equal to the preset share threshold, or if the share of the old files is less than the preset share threshold and the share of the old files is less than the preset Heat threshold, which determines that the adjustment type of the storage capacity of the cache server is constant capacity.
- the cache server may compare with the proportion threshold and the heat threshold set by the management personnel in advance. If the proportion of old files is greater than the preset threshold and the heat value of the old files is less than the preset heat threshold, the cache server stores a large number of old files with low repetitive access requirements.
- the cache server relies more on new files to provide services to users.
- the server runs services that rely on back-to-source services (such as live broadcast). Regardless of the storage capacity, the cache server will always maintain a certain back-to-source rate to continuously obtain new files. Even if the storage capacity of the cache server is reduced within a certain range, It will increase the source return rate.
- the adjustment type of the storage capacity of the cache server is capacity reduction. If the proportion of old files is less than the preset threshold and the heat value of the old files is greater than the preset heat threshold, it means that the old files stored by the cache server have high repeated access requirements, and there may be some old files with high repeated access requirements Due to the limitation of storage capacity, these files have been deleted. These old files that have been deleted and have high access requirements may cause the cache server to return to the source. Therefore, you can increase the storage capacity of the cache server to reduce these deleted old files. The number of files reduces the source return rate. Therefore, it is determined that the adjustment type of the storage capacity of the cache server is to be expanded.
- the cache server stores a large number of old files with high repeated access requirements, and the cache server is more dependent on the old files as users Provide services, and the cache server has used large storage capacity to store more old files; if the proportion of old files is less than the preset ratio threshold, and the old file heat value is less than the preset heat threshold, the cache server is more dependent on new files Provide services to users, and the cache server does not waste storage capacity to store more old files with low repetitive access requirements. Therefore, for these two cases, it can be determined that the type of adjustment of the storage capacity of the cache server is constant.
- Step 104 Determine the storage capacity adjustment type of the target service according to the storage capacity adjustment types of all cache servers corresponding to the target service.
- the adjustment type of the storage capacity of each cache server corresponding to the target service may be determined, and then the storage capacity of the target service is further determined Type of regulation.
- the processing in step 104 may be as follows: determine the number of cache servers corresponding to the target adjustment type of each storage capacity under the target service; if the target adjustment The proportion of the cache server corresponding to the type in all cache servers corresponding to the target service is greater than the preset ratio, then the target adjustment type is used as the adjustment type of the storage capacity of the target service; otherwise, the adjustment type of the storage capacity of the target service is set to To be determined.
- the number of cache servers corresponding to each adjustment type can be statistically obtained, and the cache servers corresponding to each adjustment type can be further obtained in the target service. Proportion of all cache servers. If the proportion of cache servers corresponding to a certain adjustment type in all cache servers corresponding to the target service is greater than the ratio preset by the administrator, it indicates that most of all cache servers corresponding to the target service.
- the storage capacity of all cache servers belongs to this type of adjustment. In order to reduce the workload and facilitate unified management and deployment, this adjustment type can be used as the adjustment type of the storage capacity of the target service, so that all cache servers corresponding to the target service can be performed in the future.
- Adjustment if there is no cache server corresponding to a certain adjustment type in the total cache servers corresponding to the target service, the ratio is larger than the ratio preset by the management personnel, indicating that among all cache servers corresponding to the target service, several types of adjustment servers
- the storage server accounts for a similar proportion of all cache servers corresponding to the target service.
- the adjustment type of the storage capacity of the target service can be set to be pending. When the storage capacity of the target service needs to be adjusted in the future, all caches corresponding to the target service can be adjusted.
- the server is split according to the reconciliation type, and then the cache server needed to achieve the target business is reselected.
- the file stored before the preset time node is used as the old file in the cache server; the proportion of the old file among all files stored by the cache server and the preset access are determined.
- the adjustment types include at least capacity expansion, capacity reduction, and constant capacity.
- the adjustment type of the storage capacity of all cache servers determines the adjustment type of the storage capacity of the target service.
- the data access situation of the cache server can be judged by the proportion of the old files and the popularity of the old files, and the storage requirements and bandwidth requirements of the cache server can be obtained, and then the storage capacity adjustment type of the cache server can be determined. Evaluation to achieve the evaluation of the target business. Therefore, while considering storage costs and bandwidth costs, it can also be evaluated for different services.
- an embodiment of the present invention further provides a storage capacity evaluation device based on a CDN application. As shown in FIG. 2, the device includes:
- the classification module 201 is configured to, for any cache server corresponding to a target service, use a file stored before a preset time node as an old file in the cache server.
- the calculation module 202 is configured to determine the proportion of old files among all files stored by the cache server and the access popularity of the old files within a preset access time range.
- An evaluation module 203 configured to determine a storage capacity adjustment type of the cache server according to the old file proportion and the old file access popularity, the adjustment type including at least capacity to be expanded, capacity to be reduced, and capacity unchanged;
- the adjustment type of the storage capacity of the target service is determined according to the adjustment types of the storage capacity of all cache servers corresponding to the target service.
- the classification module 201 is specifically configured to:
- file operation information generated within the preset query time range according to the statistical log within the preset query time range of the cache server, the file operation information including file name, file size, file writing time, and last access time ;
- the file corresponding to the file operation information whose writing time is before the preset time node is used as an old file.
- calculation module 202 is specifically configured to:
- the proportion of the old files is determined according to the total storage of the old files and the storage capacity of the cache server.
- calculation module 202 is further configured to:
- the access popularity of the old file is determined according to the total amount of active old file storage and the total amount of old file storage.
- the evaluation module 203 is specifically configured to:
- a heat threshold is set to determine that the adjustment type of the storage capacity of the cache server is constant capacity.
- the evaluation module 203 is specifically configured to:
- the adjustment type of the storage capacity of the target service is set to be pending.
- an embodiment of the present invention further provides a cache server.
- the cache server may have a large difference due to different configurations or performance, and may include one or more processors and memories.
- the memories may be Is temporary storage or permanent storage.
- the memory may store at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, the at least one program, code set, or instruction set are loaded and executed by the processor to implement the above-mentioned based on Method for evaluating storage capacity of CDN applications.
- the program stored on the memory may include one or more modules, and each module may include a series of instruction operations on the transcoding server.
- an embodiment of the present invention further provides a computer-readable storage medium.
- the storage medium stores at least one instruction, at least one program, code set, or instruction set.
- the at least one instruction, the At least one program, the code set or the instruction set is loaded and executed by a processor to implement the above-mentioned method for evaluating a storage capacity based on a CDN application.
- each embodiment can be implemented by means of software plus a necessary universal hardware platform, and of course, also by hardware.
- the above-mentioned technical solution essentially or part that contributes to the existing technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic Disks, compact discs, and the like include instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or certain parts of the embodiments.
Abstract
本发明公开了一种基于CDN应用的存储容量评估方法和装置，属于内容分发网络技术领域。所述方法包括：对于目标业务对应的任一缓存服务器，在缓存服务器中将预设时间节点之前存储的文件作为旧文件；确定缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度；根据旧文件占比和旧文件访问热度，确定缓存服务器的存储容量的调节类型，调节类型至少包括待扩容、待减容和容量不变；根据目标业务对应的全部缓存服务器的存储容量的调节类型，确定目标业务的存储容量的调节类型。本发明同时考虑到存储成本与带宽成本，又可以针对不同业务进行评估。
Description
本发明涉及内容分发网络技术领域，尤其涉及一种基于CDN应用的存储容量评估方法和装置。
业务运营商通过CDN系统向用户提供点播、直播、下载等业务，这些业务可以依托CDN系统中的多台缓存服务器来实现，缓存服务器可以将业务源站提供的数据资源缓存在本地，当用户需要获取这些数据资源时，无需访问业务源站，直接通过缓存服务器就能获取相应的数据资源。
为了降低运营成本，提高用户访问效率，可以对缓存服务器的存储容量进行评估，通常以存储容量的使用状况作为评估标准，判断存储容量已满或仍有空余，然后再对缓存服务器的存储容量进行扩容或减容。一方面，当前的评估方法只考虑缓存服务器的存储容量，并不区分缓存服务器上运行的业务类型，而不同业务具有不同的缓存需求，例如直播类业务只要求缓存服务器能够在短时间内存储一定数据即可，即使缓存服务器的存储容量已满，也可以不进行扩容处理；而下载类业务对数据的长期缓存需求较高，要求缓存服务器能够长期存储大量数据，所以缓存服务器的存储容量已满时应及时进行扩容，以减少数据被删除后的回源处理。另一方面，增加缓存服务器的存储容量会导致存储成本相应增加，而减少缓存服务器的存储容量会提高回源数据量，从而会导致带宽成本提高，故而，只基于当前的存储容量来对缓存服务器的存储容量进行评估有很大概率会导致总成本大幅增加。因此，目前亟需一种的综合全面的存储容量评估方法，能够针对不同业务，平衡存储成本与带宽成本，降低总体运营成本。
发明内容
为了解决现有技术的问题，本发明实施例提供了一种基于CDN应用的存储 容量评估方法和装置。所述技术方案如下：
第一方面，提供了一种基于CDN应用的存储容量评估方法，包括：
对于目标业务对应的任一缓存服务器，在所述缓存服务器中将预设时间节点之前存储的文件作为旧文件；
确定所述缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度；
根据所述旧文件占比和所述旧文件访问热度，确定所述缓存服务器的存储容量的调节类型，所述调节类型至少包括待扩容、待减容和容量不变；
根据所述目标业务对应的全部缓存服务器的存储容量的调节类型，确定所述目标业务的存储容量的调节类型。
进一步的，在所述缓存服务器中将预设时间节点之前存储的文件作为旧文件，包括：
根据所述缓存服务器预设查询时间范围内的统计日志，获取所述预设查询时间范围内生成的文件操作信息，所述文件操作信息包括文件名称、文件大小、文件写入时间、最后访问时间；
将所述文件写入时间在所述预设时间节点之后的所述文件操作信息对应的文件作为新文件；
将所述文件写入时间在所述预设时间节点之前的所述文件操作信息对应的文件作为旧文件。
进一步的，所述确定所述缓存服务器存储的所有文件中的旧文件占比，包括：
根据全部所述新文件的文件大小确定所述新文件存储总量；
根据所述新文件存储总量和所述缓存服务器存储容量，确定所述旧文件存储总量；
根据所述旧文件存储总量与所述缓存服务器存储容量确定所述旧文件占比。
进一步的，所述确定预设访问时间范围内的旧文件访问热度，包括：
确定所述最后访问时间在所述预设访问时间范围内的所述旧文件作为活跃旧文件；
根据全部所述活跃旧文件的文件大小确定活跃旧文件存储总量；
根据所述活跃旧文件存储总量与所述旧文件存储总量确定所述旧文件访问热度。
进一步的，所述根据所述旧文件占比和所述旧文件访问热度，确定所述缓存服务器的存储容量的调节类型，包括：
如果所述旧文件占比大于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待减容；
如果所述旧文件占比小于预设占比阈值，且旧文件热度值大于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待扩容；
如果所述旧文件占比大于等于预设占比阈值，且旧文件热度值大于等于预设热度阈值，或者，如果所述旧文件占比小于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为容量不变。
进一步的，所述根据所述目标业务对应的全部缓存服务器的存储容量的调节类型，确定所述目标业务的存储容量的调节类型，包括：
确定所述目标业务下每种存储容量的目标调节类型对应的缓存服务器的数量；
如果所述目标调节类型对应的缓存服务器在所述目标业务对应的所有缓存服务器中的占比大于预设比值，则将所述目标调节类型作为所述目标业务的存储容量的调节类型；
否则，将所述目标业务的存储容量的调节类型设置为待定。
第二方面，提供了一种基于CDN应用的存储容量评估装置，所述装置包括：
分类模块，用于对目标业务对应的任一缓存服务器，在所述缓存服务器中将预设时间节点之前存储的文件作为旧文件；
计算模块，用于确定所述缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度；
评估模块，用于根据所述旧文件占比和所述旧文件访问热度，确定所述缓存服务器的存储容量的调节类型，所述调节类型至少包括待扩容、待减容和容量不变；根据所述目标业务对应的全部缓存服务器的存储容量的调节类型，确定所述目标业务的存储容量的调节类型。
进一步的，所述分类模块，具体用于：
根据所述缓存服务器预设查询时间范围内的统计日志，获取所述预设查询时间范围内生成的文件操作信息，所述文件操作信息包括文件名称、文件大小、文件写入时间、最后访问时间；
将所述文件写入时间在所述预设时间节点之后的所述文件操作信息对应的文件作为新文件；
将所述文件写入时间在所述预设时间节点之前的所述文件操作信息对应的文件作为旧文件。
进一步的，所述计算模块，具体用于：
根据全部所述新文件的文件大小确定所述新文件存储总量；
根据所述新文件存储总量和所述缓存服务器存储容量，确定所述旧文件存储总量；
根据所述旧文件存储总量与所述缓存服务器存储容量确定所述旧文件占比。
进一步的，所述计算模块，具体还用于：
确定所述最后访问时间在所述预设访问时间范围内的所述旧文件作为活跃旧文件；
根据全部所述活跃旧文件的文件大小确定活跃旧文件存储总量；
根据所述活跃旧文件存储总量与所述旧文件存储总量确定所述旧文件访问热度。
进一步的，所述评估模块，具体用于：
如果所述旧文件占比大于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待减容；
如果所述旧文件占比小于预设占比阈值，且旧文件热度值大于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待扩容；
如果所述旧文件占比大于等于预设占比阈值，且旧文件热度值大于等于预设热度阈值，或者，如果所述旧文件占比小于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为容量不变。
进一步的，所述评估模块，具体用于：
确定所述目标业务下每种存储容量的目标调节类型对应的缓存服务器的数量；
如果所述目标调节类型对应的缓存服务器在所述目标业务对应的所有缓存服务器中的占比大于预设比值，则将所述目标调节类型作为所述目标业务的存储容量的调节类型；
否则，将所述目标业务的存储容量的调节类型设置为待定。
第三方面，提供了一种缓存服务器，所述缓存服务器包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如第一方面所述的基于CDN应用的存储容量评估方法。
第四方面，提供了一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如第一方面所述的基于CDN应用的存储容量评估方法。
本发明实施例提供的技术方案带来的有益效果是：
本发明实施例中，对于目标业务对应的任一缓存服务器，在缓存服务器中将预设时间节点之前存储的文件作为旧文件；确定缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度；根据旧文件占比和旧文件访问热度，确定缓存服务器的存储容量的调节类型，调节类型至少包括待扩容、待减容和容量不变；根据目标业务对应的全部缓存服务器的存储容量的调节类型，确定目标业务的存储容量的调节类型。这样，可以通过旧文件占比和旧文件访问热度判断缓存服务器的数据访问情况，得到缓存服务器存储需求与带宽需求，进而确定缓存服务器的存储容量调节类型，通过对目标业务的每个缓存服务器进行评估，实现对目标业务的评估。故而，在考虑到存储成本与带宽成本的同时，又可以针对不同业务进行评估。
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明 的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的一种基于CDN应用的存储容量评估方法的流程图；
图2是本发明实施例提供的一种基于CDN应用的存储容量评估装置的结构示意图。
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
本发明实施例提供了一种基于CDN应用的存储容量评估方法，该方法可以应用在CDN系统的缓存服务器中，缓存服务器存储有视频、音频、文字、图像等文件，向用户提供各种类型业务的服务，一项业务可以依托多台缓存服务器来共同实现，每台缓存服务器相互独立，可以先对所有缓存服务器分别进行存储容量评估，之后再整体考虑对该项业务的存储容量进行评估。缓存服务器中可以包括处理器、存储器、收发器，处理器可以用于进行下述流程中的确定存储容量的调节类型的处理，存储器可以用于存储处理过程中需要的数据以及产生的数据，收发器可以用于接收和发送处理过程中的相关数据。本实施例的应用场景可以是：选取目标业务下的一台缓存服务器，确定该缓存服务器中存储的旧文件，进一步确定该缓存服务器存储的全部文件中的旧文件占比以及旧文件访问热度，根据旧文件占比和旧文件访问热度确定该缓存服务器的存储容量的调节类型，对目标业务下的全部缓存服务器均进行上述处理过程，然后，根据全部缓存服务器的存储容量的调节类型，确定目标业务的存储容量的调节类型。这样的评估方法有利于从业务层面对全部缓存服务器进行整体调节，便于统一管理和操作。
下面将结合具体实施方式，对图1所示的一种基于CDN应用的存储容量评估的处理流程进行详细的说明，内容可以如下：
步骤101：对于目标业务对应的任一缓存服务器，在缓存服务器中将预设时间节点之前存储的文件作为旧文件。
在实施中，对于目标业务，管理人员首先从目标业务对应的缓存服务器中 任选一台，然后在该缓存服务器上设置时间节点，将该时间节点之前存储在该缓存服务器上的文件作为旧文件。通常，可以将时间节点设置为7天，7前天存储在该缓存服务器上的文件即为旧文件，可以理解的，为了统一同一业务下全部缓存服务器的评估标准，对于同一业务下的全部缓存服务器，每台服务器上预设的时间节点应当相同。
可选的，在缓存服务器中将预设时间节点之前存储的文件作为旧文件的处理具体可以如下：根据缓存服务器预设查询时间范围内的统计日志，获取预设查询时间范围内生成的文件操作信息，文件操作信息包括文件名称、文件大小、文件写入时间、最后访问时间；将文件写入时间在预设时间节点之后的文件操作信息对应的文件作为新文件；将文件写入时间在预设时间节点之前的文件操作信息对应的文件作为旧文件。
在实施中，当缓存服务器上存储的文件每次发生读写、修改、删除等操作时，缓存服务器上都会相应的生成一条统计日志，统计日志记录的内容包括文件操作信息和文件操作内容，文件操作信息具体包括文件名称、文件大小、文件写入时间、最后访问时间等文件属性相关的内容信息，文件操作内容包括文件读取、写入、修改、删除等操作内容信息。通常，统计日志会长期保存在缓存服务中，即缓存服务器中存储有很长时间范围内的统计日志，对于已经运行了较长时间的缓存服务器，统计日志的数量可能比较多，故而，管理人员可以在缓存服务器上预先设置对统计日志的查询时间范围，以减少后续获取的文件操作信息数量。通常，可以将查询时间范围设置为30天内(1-30天)，当然，也可以根据业务或缓存服务器的实际情况设置为其他时间范围，或者也可以设置成具体的某一段时间范围，例如X月X日-Y月Y日，当然，还可以设置为获取缓存服务器上的全部统计日志。在设置好查询时间范围与时间节点后，缓存服务器可以在设置的查询时间范围内，将该时间节点转换为该查询时间范围内的实际日期。缓存服务器从预设查询时间范围内的统计日志中获取文件操作信息后，根据文件操作信息中的文件写入时间，将文件写入时间在预设时间节点之后的文件操作信息对应的文件作为新文件，将文件写入时间在预设时间节点之前的文件操作信息对应的文件作为旧文件。可以理解的，对于不同的业务，统计日志中也可以使用文件ID来代替文件名称。
例如，当前日期为3月30日，在缓存服务器上预设查询时间范围为30天 内，预设时间节点为7天，那么，该时间节点在该查询时间范围内的实际日期为3月24日，缓存服务器可以在存储的全部统计日志中，查询3月1日至3月30日内的统计日志，获取对应的文件操作信息，然后选取写入时间在3月24日到3月30日内的文件操作信息，通过这些文件操作信息中的文件名称，将对应的文件作为新文件，选取写入时间在3月1日到3月23日内的文件操作信息，通过这些文件操作信息中的文件名称，将对应的文件作为旧文件。
步骤102：确定缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度。
在实施中，缓存服务器在确定旧文件之后，可以进一步的确定这些旧文件在缓存服务器存储的所有文件中的旧文件占比。并且，还可以在缓存服务器上设置访问时间范围，确定在该访问时间范围内旧文件的访问热度。通常，可以将访问时间范围设置为24小时，即确定当前时间之前24小时内旧文件的访问热度，可以理解的，为了统一同一业务下全部缓存服务器的评估标准，对于同一业务下的全部缓存服务器，每台服务器上预设的访问时间范围应当相同。
可选的，由于旧文件的写入时间较早，可能有部分旧文件已经被缓存服务器删除，而新文件的写入时间相对旧文件较晚，通常，只有当所有的旧文件都被删除之后，才可能会删除新文件，故而，确定缓存服务器存储的所有文件中的旧文件占比的处理具体可以如下：根据全部新文件的文件大小确定新文件存储总量；根据新文件存储总量和缓存服务器存储容量，确定旧文件存储总量；根据旧文件存储总量与缓存服务器存储容量确定旧文件占比。
在实施中，缓存服务器在确定新文件与旧文件之后，可以获取每个新文件的文件大小，然后，统计全部新文件的文件大小，得到新文件存储总量，再根据缓存服务器的存储容量，减去新文件的存储总量，剩余部分即为旧文件存储总量，最后，可以将旧文件存储总量与缓存服务器存储容量的比值作为旧文件占比。可以理解的，如果统计得到新文件存储总量大于或等于缓存服务器的存储容量，即计算得到的旧文件存储总量为零或负值，说明预设查询时间范围内缓存服务器上存储的所有文件均是新文件，甚至可能有部分新文件已经被删除，应当调整时间节点和查询时间范围，例如延长查询时间范围的下限时间，或者减小时间节点的数值。
可选的，确定预设访问时间范围内的旧文件访问热度的处理具体可以如下： 确定最后访问时间在预设访问时间范围内的旧文件作为活跃旧文件；根据全部活跃旧文件的文件大小确定活跃旧文件存储总量；根据活跃旧文件存储总量与旧文件存储总量确定旧文件访问热度。
在实施中，缓存服务器在确定新文件与旧文件之后，从全部旧文件中，选取最后访问时间在预设访问时间范围内的旧文件作为活跃旧文件，然后获取每个活跃旧文件的文件大小，统计全部活跃旧文件的文件大小，得到活跃旧文件存储总量，再根据之前计算得到的旧文件存储总量，将活跃旧文件存储总量与旧文件存储总量的比值作为旧文件访问热度。
步骤103：根据旧文件占比和旧文件访问热度，确定缓存服务器的存储容量的调节类型。
其中，调节类型至少包括待扩容、待减容和容量不变。
在实施中，缓存服务器在得到旧文件占比和旧文件访问热度后，根据旧文件占比和旧文件访问热度的具体结果，可以确定至少三种缓存服务器的存储容量的调节类型，例如待扩容、待减容、容量不变。
可选的，根据旧文件占比和旧文件访问热度，确定缓存服务器的存储容量的调节类型的处理具体可以如下：如果旧文件占比大于预设占比阈值，且旧文件热度值小于预设热度阈值，确定缓存服务器的存储容量的调节类型为待减容；如果旧文件占比小于预设占比阈值，且旧文件热度值大于预设热度阈值，确定缓存服务器的存储容量的调节类型为待扩容；如果旧文件占比大于等于预设占比阈值，且旧文件热度值大于等于预设热度阈值，或者，如果旧文件占比小于预设占比阈值，且旧文件热度值小于预设热度阈值，确定缓存服务器的存储容量的调节类型为容量不变。
在实施中，缓存服务器在计算得到的旧文件占比和旧文件访问热度后，可以与管理人员预先设置的占比阈值和热度阈值分别进行比较。如果旧文件占比大于预设占比阈值，且旧文件热度值小于预设热度阈值，说明缓存服务器存储有大量重复访问需求较低的旧文件，缓存服务器更依赖新文件为用户提供服务，缓存服务器运行的是较依赖回源的业务(例如直播)，无论存储容量大小如何，缓存服务器始终会保持一定回源率以不断获得新文件，即使在一定范围内减少缓存服务器的存储容量，也不会增加回源率，故而，可以确定缓存服务器的存储容量的调节类型为待减容。如果旧文件占比小于预设占比阈值，且旧文件热 度值大于预设热度阈值，说明缓存服务器存储的旧文件具有较高的重复访问需求，并且可能有部分重复访问需求较高的旧文件由于存储容量的限制，已经被删除，这些被删除且重复访问需求较高的旧文件可能会导致缓存服务器的回源率升高，因此可以增加缓存服务器的存储容量，以减少这些被删除的旧文件的数量，降低回源率，故而，确定缓存服务器的存储容量的调节类型为待扩容。如果旧文件占比大于等于预设占比阈值，且旧文件热度值大于等于预设热度阈值，说明缓存服务器中存储有大量具有较高重复访问需求的旧文件，缓存服务器更依赖旧文件为用户提供服务，并且缓存服务器已经使用较大的存储容量存储较多的旧文件；如果旧文件占比小于预设占比阈值，且旧文件热度值小于预设热度阈值，说明缓存服务器更依赖新文件为用户提供服务，并且缓存服务器没有浪费存储容量去存储较多重复访问需求低的旧文件，故而，对于这两种情况可以确定缓存服务器的存储容量的调节类型为容量不变。
步骤104：根据目标业务对应的全部缓存服务器的存储容量的调节类型，确定目标业务的存储容量的调节类型。
在实施中，在目标业务对应的全部缓存服务器都按照步骤101到步骤103进行处理后，可以确定目标业务对应的每台缓存服务器的存储容量的调节类型，然后，再进一步确定目标业务的存储容量的调节类型。
可选的，由于目标业务对应有大量的缓存服务器，每台缓存服务器的存储容量以及存储的文件都不相同，所以，每台缓存服务器的存储容量的调节类型也可能不相同，目标业务对应的全部缓存服务器中，可能同时存在几种不同的存储容量的调节类型，故而，步骤104的处理具体可以如下：确定目标业务下每种存储容量的目标调节类型对应的缓存服务器的数量；如果目标调节类型对应的缓存服务器在目标业务对应的所有缓存服务器中的占比大于预设比值，则将目标调节类型作为目标业务的存储容量的调节类型；否则，将目标业务的存储容量的调节类型设置为待定。
在实施中，在确定了目标业务对应的全部缓存服务器的存储容量的调节类型后，可以统计得到各调节类型对应的缓存服务器的数量，并进一步得到各调节类型对应的缓存服务器在目标业务对应的全部缓存服务器中的占比，如果某种调节类型对应的缓存服务器在目标业务对应的全部缓存服务器中的占比大于管理人员预设的比值，说明在目标业务对应的全部缓存服务器中，大部分的缓 存服务器的存储容量均属于这种调节类型，为了减少工作量，便于统一管理调配，可以将该调节类型作为目标业务的存储容量的调节类型，便于以后可以对目标业务对应的全部缓存服务器进行调节；如果没有某种调节类型对应的缓存服务器在目标业务对应的全部缓存服务器中的占比大于管理人员预设的比值，说明在目标业务对应的全部缓存服务器中，几种调节类型对应的缓存服务器在目标业务对应的全部缓存服务器中的占比相近，可以将目标业务的存储容量的调节类型设置为待定，以后需要对目标业务的存储容量进行调节时，可以将目标业务对应的全部缓存服务器根据调节类型拆分，然后重新选择实现目标业务所需要的缓存服务器。
本发明实施例中，对于目标业务对应的任一缓存服务器，在缓存服务器中将预设时间节点之前存储的文件作为旧文件；确定缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度；根据旧文件占比和旧文件访问热度，确定缓存服务器的存储容量的调节类型，调节类型至少包括待扩容、待减容和容量不变；根据目标业务对应的全部缓存服务器的存储容量的调节类型，确定目标业务的存储容量的调节类型。这样，可以通过旧文件占比和旧文件访问热度判断缓存服务器的数据访问情况，得到缓存服务器存储需求与带宽需求，进而确定缓存服务器的存储容量调节类型，通过对目标业务的每个缓存服务器进行评估，实现对目标业务的评估。故而，在考虑到存储成本与带宽成本的同时，又可以针对不同业务进行评估。
基于相同的技术构思，本发明实施例还提供了一种基于CDN应用的存储容量评估装置，如图2所示，所述装置包括：
分类模块201，用于对目标业务对应的任一缓存服务器，在所述缓存服务器中将预设时间节点之前存储的文件作为旧文件。
计算模块202，用于确定所述缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度。
评估模块203，用于根据所述旧文件占比和所述旧文件访问热度，确定所述缓存服务器的存储容量的调节类型，所述调节类型至少包括待扩容、待减容和容量不变；根据所述目标业务对应的全部缓存服务器的存储容量的调节类型，确定所述目标业务的存储容量的调节类型。
可选的，所述分类模块201，具体用于：
根据所述缓存服务器预设查询时间范围内的统计日志，获取所述预设查询时间范围内生成的文件操作信息，所述文件操作信息包括文件名称、文件大小、文件写入时间、最后访问时间；
将所述文件写入时间在所述预设时间节点之后的所述文件操作信息对应的文件作为新文件；
将所述文件写入时间在所述预设时间节点之前的所述文件操作信息对应的文件作为旧文件。
可选的，所述计算模块202，具体用于：
根据全部所述新文件的文件大小确定所述新文件存储总量；
根据所述新文件存储总量和所述缓存服务器存储容量，确定所述旧文件存储总量；
根据所述旧文件存储总量与所述缓存服务器存储容量确定所述旧文件占比。
可选的，所述计算模块202，具体还用于：
确定所述最后访问时间在所述预设访问时间范围内的所述旧文件作为活跃旧文件；
根据全部所述活跃旧文件的文件大小确定活跃旧文件存储总量；
根据所述活跃旧文件存储总量与所述旧文件存储总量确定所述旧文件访问热度。
可选的，所述评估模块203，具体用于：
如果所述旧文件占比大于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待减容；
如果所述旧文件占比小于预设占比阈值，且旧文件热度值大于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待扩容；
如果所述旧文件占比大于等于预设占比阈值，且旧文件热度值大于等于预设热度阈值，或者，如果所述旧文件占比小于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为容量不变。
可选的，所述评估模块203，具体用于：
确定所述目标业务下每种存储容量的目标调节类型对应的缓存服务器的数 量；
如果所述目标调节类型对应的缓存服务器在所述目标业务对应的所有缓存服务器中的占比大于预设比值，则将所述目标调节类型作为所述目标业务的存储容量的调节类型；
否则，将所述目标业务的存储容量的调节类型设置为待定。
基于相同的技术构思，本发明实施例还提供了一种缓存服务器，所述缓存服务器可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上处理器和存储器，其中，存储器可以是短暂存储或永久存储。存储器可以存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现上述的基于CDN应用的存储容量评估方法。存储在存储器上的程序可以包括一个或一个以上模块，每个模块可以包括对转码服务器中的一系列指令操作。
基于相同的技术构思，本发明实施例还提供了一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现上述的基于CDN应用的存储容量评估方法。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务端，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。
Claims (14)
- 一种基于CDN应用的存储容量评估方法，其特征在于，包括：对于目标业务对应的任一缓存服务器，在所述缓存服务器中将预设时间节点之前存储的文件作为旧文件；确定所述缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度；根据所述旧文件占比和所述旧文件访问热度，确定所述缓存服务器的存储容量的调节类型，所述调节类型至少包括待扩容、待减容和容量不变；根据所述目标业务对应的全部缓存服务器的存储容量的调节类型，确定所述目标业务的存储容量的调节类型。
- 根据权利要求1所述的方法，其特征在于，在所述缓存服务器中将预设时间节点之前存储的文件作为旧文件，包括：根据所述缓存服务器预设查询时间范围内的统计日志，获取所述预设查询时间范围内生成的文件操作信息，所述文件操作信息包括文件名称、文件大小、文件写入时间、最后访问时间；将所述文件写入时间在所述预设时间节点之后的所述文件操作信息对应的文件作为新文件；将所述文件写入时间在所述预设时间节点之前的所述文件操作信息对应的文件作为旧文件。
- 根据权利要求2所述的方法，其特征在于，所述确定所述缓存服务器存储的所有文件中的旧文件占比，包括：根据全部所述新文件的文件大小确定所述新文件存储总量；根据所述新文件存储总量和所述缓存服务器存储容量，确定所述旧文件存储总量；根据所述旧文件存储总量与所述缓存服务器存储容量确定所述旧文件占比。
- 根据权利要求3所述的方法，其特征在于，所述确定预设访问时间范围内的旧文件访问热度，包括：确定所述最后访问时间在所述预设访问时间范围内的所述旧文件作为活跃旧文件；根据全部所述活跃旧文件的文件大小确定活跃旧文件存储总量；根据所述活跃旧文件存储总量与所述旧文件存储总量确定所述旧文件访问热度。
- 根据权利要求1所述的方法，其特征在于，所述根据所述旧文件占比和所述旧文件访问热度，确定所述缓存服务器的存储容量的调节类型，包括：如果所述旧文件占比大于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待减容；如果所述旧文件占比小于预设占比阈值，且旧文件热度值大于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待扩容；如果所述旧文件占比大于等于预设占比阈值，且旧文件热度值大于等于预设热度阈值，或者，如果所述旧文件占比小于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为容量不变。
- 根据权利要求1所述的方法，其特征在于，所述根据所述目标业务对应的全部缓存服务器的存储容量的调节类型，确定所述目标业务的存储容量的调节类型，包括：确定所述目标业务下每种存储容量的目标调节类型对应的缓存服务器的数量；如果所述目标调节类型对应的缓存服务器在所述目标业务对应的所有缓存服务器中的占比大于预设比值，则将所述目标调节类型作为所述目标业务的存储容量的调节类型；否则，将所述目标业务的存储容量的调节类型设置为待定。
- 一种基于CDN应用的存储容量评估装置，其特征在于，所述装置包括：分类模块，用于对目标业务对应的任一缓存服务器，在所述缓存服务器中 将预设时间节点之前存储的文件作为旧文件；计算模块，用于确定所述缓存服务器存储的所有文件中的旧文件占比和预设访问时间范围内的旧文件访问热度；评估模块，用于根据所述旧文件占比和所述旧文件访问热度，确定所述缓存服务器的存储容量的调节类型，所述调节类型至少包括待扩容、待减容和容量不变；根据所述目标业务对应的全部缓存服务器的存储容量的调节类型，确定所述目标业务的存储容量的调节类型。
- 根据权利要求7所述的装置，其特征在于，所述分类模块，具体用于：根据所述缓存服务器预设查询时间范围内的统计日志，获取所述预设查询时间范围内生成的文件操作信息，所述文件操作信息包括文件名称、文件大小、文件写入时间、最后访问时间；将所述文件写入时间在所述预设时间节点之后的所述文件操作信息对应的文件作为新文件；将所述文件写入时间在所述预设时间节点之前的所述文件操作信息对应的文件作为旧文件。
- 根据权利要求8所述的装置，其特征在于，所述计算模块，具体用于：根据全部所述新文件的文件大小确定所述新文件存储总量；根据所述新文件存储总量和所述缓存服务器存储容量，确定所述旧文件存储总量；根据所述旧文件存储总量与所述缓存服务器存储容量确定所述旧文件占比。
- 根据权利要求9所述的装置，其特征在于，所述计算模块，具体还用于：确定所述最后访问时间在所述预设访问时间范围内的所述旧文件作为活跃旧文件；根据全部所述活跃旧文件的文件大小确定活跃旧文件存储总量；根据所述活跃旧文件存储总量与所述旧文件存储总量确定所述旧文件访问 热度。
- 根据权利要求7所述的装置，其特征在于，所述评估模块，具体用于：如果所述旧文件占比大于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待减容；如果所述旧文件占比小于预设占比阈值，且旧文件热度值大于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为待扩容；如果所述旧文件占比大于等于预设占比阈值，且旧文件热度值大于等于预设热度阈值，或者，如果所述旧文件占比小于预设占比阈值，且旧文件热度值小于预设热度阈值，确定所述缓存服务器的存储容量的调节类型为容量不变。
- 根据权利要求7所述的装置，其特征在于，所述评估模块，具体用于：确定所述目标业务下每种存储容量的目标调节类型对应的缓存服务器的数量；如果所述目标调节类型对应的缓存服务器在所述目标业务对应的所有缓存服务器中的占比大于预设比值，则将所述目标调节类型作为所述目标业务的存储容量的调节类型；否则，将所述目标业务的存储容量的调节类型设置为待定。
- 一种缓存服务器，其特征在于，所述缓存服务器包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如权利要求1至6任一所述的基于CDN应用的存储容量评估方法。
- 一种计算机可读存储介质，其特征在于，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如权利要求1至6任一所述的基于CDN应用的存储容量评估方法。
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16/465,035 US11005717B2 (en) | 2018-07-16 | 2018-08-21 | Storage capacity evaluation method based on content delivery network application and device thereof |
| EP18900579.6A EP3624398B1 (en) | 2018-07-16 | 2018-08-21 | Storage capacity evaluation method and apparatus based on cdn application |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810779131.2A CN109005056B (zh) | 2018-07-16 | 2018-07-16 | 基于cdn应用的存储容量评估方法和装置 |
| CN201810779131.2 | 2018-07-16 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2020015047A1 true WO2020015047A1 (zh) | 2020-01-23 |
Family
ID=64598432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2018/101516 WO2020015047A1 (zh) | 2018-07-16 | 2018-08-21 | 基于cdn应用的存储容量评估方法和装置 |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US11005717B2 (zh) |
| EP (1) | EP3624398B1 (zh) |
| CN (1) | CN109005056B (zh) |
| WO (1) | WO2020015047A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115052040A (zh) * | 2022-04-26 | 2022-09-13 | 浪潮通信技术有限公司 | Feed流实现方法、系统、电子设备和存储介质 |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112448979B (zh) * | 2019-08-30 | 2022-09-13 | 贵州白山云科技股份有限公司 | 一种缓存信息的更新方法、装置及介质 |
| CN110807009B (zh) * | 2019-11-06 | 2022-04-26 | 湖南快乐阳光互动娱乐传媒有限公司 | 文件处理方法及装置 |
| CN111221824B (zh) * | 2020-01-02 | 2023-05-26 | 广州虎牙科技有限公司 | 存储空间的存储优化方法、装置、设备和介质 |
| CN111669629A (zh) * | 2020-05-19 | 2020-09-15 | 湖南快乐阳光互动娱乐传媒有限公司 | 视频cdn节点即时扩容方法、调度器及cnd存储系统 |
| CN112583667B (zh) * | 2020-12-11 | 2022-02-11 | 上海哔哩哔哩科技有限公司 | 内容分发网络链路评估方法及装置 |
| CN113032142B (zh) * | 2021-03-03 | 2022-12-13 | 广州虎牙科技有限公司 | 资源调整方法、装置、电子设备和计算机可读存储介质 |
| CN115617255A (zh) * | 2021-06-30 | 2023-01-17 | 华为技术有限公司 | 缓存文件的管理方法和管理装置 |
| CN113448950B (zh) * | 2021-07-26 | 2024-03-15 | 北京清博智能科技有限公司 | 一种基于数据量的本地化硬件部署方法 |
| CN115442388B (zh) * | 2022-10-21 | 2023-01-06 | 四川省华存智谷科技有限责任公司 | 用于分布式存储集群利用率100%的扩容方法、装置及系统 |
| CN116708579B (zh) * | 2023-08-04 | 2024-01-12 | 浪潮电子信息产业股份有限公司 | 数据访问方法、装置、电子设备及计算机可读存储介质 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6081840A (en) * | 1997-10-14 | 2000-06-27 | Zhao; Yan | Two-level content distribution system |
| CN103581051A (zh) * | 2012-08-02 | 2014-02-12 | 深圳市腾讯计算机系统有限公司 | 一种文件缓存方法、装置和系统 |
| CN104469392A (zh) * | 2014-12-19 | 2015-03-25 | 北京奇艺世纪科技有限公司 | 一种视频文件存储方法及装置 |
| CN106502576A (zh) * | 2015-09-06 | 2017-03-15 | 中兴通讯股份有限公司 | 迁移策略调整方法、容量变更建议方法及装置 |
| CN106888381A (zh) * | 2015-12-16 | 2017-06-23 | 中兴通讯股份有限公司 | 一种数据资源存储方法及装置 |
Family Cites Families (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5081840A (en) | 1990-01-09 | 1992-01-21 | Landon Hank A | Spool valve and piston power plant |
| US7428540B1 (en) * | 2000-03-03 | 2008-09-23 | Intel Corporation | Network storage system |
| US7340505B2 (en) * | 2001-04-02 | 2008-03-04 | Akamai Technologies, Inc. | Content storage and replication in a managed internet content storage environment |
| CN1926540A (zh) * | 2004-03-12 | 2007-03-07 | 汤姆森许可贸易公司 | 在高速缓存网络环境中调度下载的系统和方法 |
| US8001175B2 (en) * | 2004-10-22 | 2011-08-16 | International Business Machines Corporation | Facilitating server response optimization |
| WO2012152771A2 (en) | 2011-05-12 | 2012-11-15 | Telefonica, S.A | Content server of a service provider's cdn |
| CN102801550A (zh) * | 2011-05-27 | 2012-11-28 | 北京邮电大学 | 一种内容分发网络管理方法及装置 |
| CN102411542A (zh) * | 2011-10-26 | 2012-04-11 | 上海文广互动电视有限公司 | 动态分级存储系统和方法 |
| CN103778071A (zh) * | 2014-01-20 | 2014-05-07 | 华为技术有限公司 | 缓存的空间分配方法及装置 |
| US9628350B2 (en) * | 2014-11-05 | 2017-04-18 | Amazon Technologies, Inc. | Dynamic scaling of storage volumes for storage client file systems |
| CN104881477B (zh) * | 2015-06-03 | 2018-09-21 | 上海新炬网络信息技术股份有限公司 | 一种应用数据空间使用估算方法 |
| CN105515831A (zh) * | 2015-11-27 | 2016-04-20 | 小米科技有限责任公司 | 网络状态信息展示方法及装置 |
| CN106027642A (zh) * | 2016-05-19 | 2016-10-12 | 乐视控股（北京）有限公司 | 用于确定cdn节点磁盘数量的方法及系统 |
| CN106020732A (zh) * | 2016-05-27 | 2016-10-12 | 乐视控股（北京）有限公司 | 节点的磁盘空间确定方法及系统 |
| US10257304B2 (en) * | 2016-06-01 | 2019-04-09 | GM Global Technology Operations LLC | System and method for mobile platform cache using mobile devices |
| WO2018064196A1 (en) * | 2016-09-27 | 2018-04-05 | Level 3 Communications, Llc | System and method for improvements to a content delivery network |
-
2018
- 2018-07-16 CN CN201810779131.2A patent/CN109005056B/zh not_active Expired - Fee Related
- 2018-08-21 US US16/465,035 patent/US11005717B2/en active Active
- 2018-08-21 WO PCT/CN2018/101516 patent/WO2020015047A1/zh unknown
- 2018-08-21 EP EP18900579.6A patent/EP3624398B1/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6081840A (en) * | 1997-10-14 | 2000-06-27 | Zhao; Yan | Two-level content distribution system |
| CN103581051A (zh) * | 2012-08-02 | 2014-02-12 | 深圳市腾讯计算机系统有限公司 | 一种文件缓存方法、装置和系统 |
| CN104469392A (zh) * | 2014-12-19 | 2015-03-25 | 北京奇艺世纪科技有限公司 | 一种视频文件存储方法及装置 |
| CN106502576A (zh) * | 2015-09-06 | 2017-03-15 | 中兴通讯股份有限公司 | 迁移策略调整方法、容量变更建议方法及装置 |
| CN106888381A (zh) * | 2015-12-16 | 2017-06-23 | 中兴通讯股份有限公司 | 一种数据资源存储方法及装置 |
Non-Patent Citations (1)
| Title |
|---|
| See also references of EP3624398A4 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115052040A (zh) * | 2022-04-26 | 2022-09-13 | 浪潮通信技术有限公司 | Feed流实现方法、系统、电子设备和存储介质 |
| CN115052040B (zh) * | 2022-04-26 | 2024-04-19 | 浪潮通信技术有限公司 | Feed流实现方法、系统、电子设备和存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| US11005717B2 (en) | 2021-05-11 |
| CN109005056B (zh) | 2020-06-23 |
| EP3624398B1 (en) | 2021-01-20 |
| US20200344128A1 (en) | 2020-10-29 |
| EP3624398A1 (en) | 2020-03-18 |
| EP3624398A4 (en) | 2020-03-18 |
| CN109005056A (zh) | 2018-12-14 |
| EP3624398A8 (en) | 2020-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020015047A1 (zh) | 基于cdn应用的存储容量评估方法和装置 | |
| US20210144423A1 (en) | Dynamic binding for use in content distribution | |
| US8495021B2 (en) | Distribution data items within geographically distributed databases | |
| CN109547566B (zh) | 一种基于内存分配的多线程上传优化方法 | |
| CN105677250B (zh) | 对象存储系统中的对象数据的更新方法和更新装置 | |
| US11347651B2 (en) | Cache warming: agility for a stateful service | |
| US10956990B2 (en) | Methods and apparatuses for adjusting the distribution of partitioned data | |
| CN106713028B (zh) | 业务降级方法、装置和分布式任务调度系统 | |
| CN109981702B (zh) | 一种文件存储方法及系统 | |
| CN110737388A (zh) | 数据预读方法、客户端、服务器以及文件系统 | |
| US8782285B1 (en) | Lazy transcoding and re-transcoding of media objects in an online video platform | |
| US11494413B1 (en) | Query alerts generation for virtual warehouse | |
| CN112433921A (zh) | 用于动态埋点的方法及设备 | |
| US11055223B2 (en) | Efficient cache warm up based on user requests | |
| CN109325001B (zh) | 基于元数据服务器删除小文件的方法、装置及设备 | |
| CN112241396B (zh) | 基于Spark的对Delta进行小文件合并的方法及系统 | |
| CN114040378B (zh) | 应用的编排方法、装置、计算机设备和存储介质 | |
| JP7477251B2 (ja) | シングル・ページ・アプリケーションのキャッシャビリティの改善方法、システム、プログラム | |
| CN107145303B (zh) | 一种用于在分布式存储系统中执行文件写入的方法与设备 | |
| CN103605795A (zh) | 一种基于元数据的文件存储方法和装置 | |
| CN114020218A (zh) | 混合重复数据删除调度方法及系统 | |
| CN112689166A (zh) | 一种用于cdn热点内容实时弹性增减的方法及系统 | |
| CN109086364A (zh) | 数据管理方法、数据管理设备及存储介质 | |
| CN108762930A (zh) | 一种用户态nfs线程的优化方法、系统及相关组件 | |
| US20230334052A1 (en) | Dynamic Query Allocation to Virtual Warehouses |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| ENP | Entry into the national phase |
Ref document number: 2018900579 Country of ref document: EP Effective date: 20190726 |
|
| ENP | Entry into the national phase |
Ref document number: 2018900579 Country of ref document: EP Effective date: 20190726 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |