WO2019196225A1 - 一种反馈资源文件的方法和装置 - Google Patents

一种反馈资源文件的方法和装置 Download PDF

Info

Publication number
WO2019196225A1
WO2019196225A1 PCT/CN2018/095431 CN2018095431W WO2019196225A1 WO 2019196225 A1 WO2019196225 A1 WO 2019196225A1 CN 2018095431 W CN2018095431 W CN 2018095431W WO 2019196225 A1 WO2019196225 A1 WO 2019196225A1
Authority
WO
WIPO (PCT)
Prior art keywords
resource
target
index value
data
acquisition request
Prior art date
Application number
PCT/CN2018/095431
Other languages
English (en)
French (fr)
Inventor
陈丽娟
王力鹏
Original Assignee
网宿科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 网宿科技股份有限公司 filed Critical 网宿科技股份有限公司
Priority to EP18877297.4A priority Critical patent/EP3579526B1/en
Priority to US16/327,668 priority patent/US11206302B2/en
Publication of WO2019196225A1 publication Critical patent/WO2019196225A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/63—Routing a service request depending on the request content or context
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/957—Browsing optimisation, e.g. caching or content distillation
    • G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/565—Conversion or adaptation of application format or content
    • H04L67/5651—Reducing the amount or size of exchanged application data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/06—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
    • H04L9/0643—Hash functions, e.g. MD5, SHA, HMAC or f9 MAC
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/565—Conversion or adaptation of application format or content

Definitions

  • the present invention relates to the field of data storage technologies, and in particular, to a method and apparatus for feeding back resource files.
  • CDN Content Delivery Network
  • the user may send a resource acquisition request through the terminal, and the resource acquisition request may carry the resource identification information, where the resource identification information may include the request method. Destination IP (Internet Protocol) address, domain name, resource name, URL (Uniform Resource Locator) and other feature information.
  • the CDN service cluster can distribute it to a cache server. The cache server can search whether the data resource is stored locally according to the resource identification information. If not, the data resource can be obtained from the corresponding resource server, and then the data resource is fed back to the terminal, and the data resource can be stored locally. In this way, if the subsequent resource acquisition request is received again, the cache server can directly feed back the data resource stored this time.
  • the resource identification information of different resource servers has different information formats, and in order to improve the security of the data resources, some resource servers often adjust the information format of the resource identification information, so that even if multiple resource acquisition requests point to the same
  • the data resource which carries the resource identification information, is also likely to be different.
  • the cache server needs to obtain and cache multiple data resources from the resource server. Therefore, the cache server may store a large number of duplicate data resources. , cache utilization is low.
  • embodiments of the present invention provide a method and apparatus for feeding back a resource file.
  • the technical solution is as follows:
  • a method of feeding back a resource file comprising:
  • the resource file corresponding to the target feature index value exists locally, the resource file corresponding to the target feature index value is fed back, otherwise the data acquisition request of the specified segment of the target resource is sent to the resource server of the target resource;
  • the resource file corresponding to the target content index value exists locally, the resource file corresponding to the target content index value is fed back, otherwise the resource acquisition request of the target resource is sent to the resource server.
  • the calculating, by using the first preset algorithm, the resource identification information carried by the resource acquisition request to obtain a target feature index value including:
  • the resource identification information carried in the resource acquisition request meets the local pre-stored target information format, extracting feature information in the resource identification information according to the target information format, and the feature is based on the first preset algorithm
  • the information is calculated to obtain a target feature index value
  • the resource identification information carried in the resource acquisition request is calculated according to the first preset algorithm to obtain a target feature index value.
  • the sending by the resource server of the target resource, the data acquisition request of the specified segment of the target resource, including:
  • the calculating, by using the second preset algorithm, the specified segment data to obtain a target content index value including:
  • the resource obtaining request carries a target data segment identifier
  • the resource file corresponding to the target content index value exists locally, the resource file corresponding to the target content index value is fed back, including:
  • the resource file corresponding to the target content index value exists locally, and the resource file includes all resource data pointed to by the target data segment identifier, all the target data segment identifiers in the resource file are fed back Resource data
  • the resource acquisition request of the target resource is sent to the resource server.
  • the method further includes:
  • an apparatus for feeding back a resource file comprising:
  • a first calculation module configured to receive a resource acquisition request of the target resource, and calculate, according to the first preset algorithm, the resource identification information carried by the resource acquisition request to obtain a target feature index value;
  • a first sending module configured to: if a resource file corresponding to the target feature index value exists locally, feed back a resource file corresponding to the target feature index value, and otherwise send the target resource to the resource server of the target resource Fragment data acquisition request;
  • a second calculation module configured to receive the specified segment data of the target resource that is fed back by the resource server, and calculate the target segment index value by using the second preset algorithm to calculate the target content index value;
  • a second sending module configured to: if a resource file corresponding to the target content index value exists locally, feed back a resource file corresponding to the target content index value, and otherwise send a resource acquisition request of the target resource to the resource server.
  • the first calculating module is specifically configured to:
  • the resource identification information carried in the resource acquisition request meets the local pre-stored target information format, extracting feature information in the resource identification information according to the target information format, and the feature is based on the first preset algorithm
  • the information is calculated to obtain a target feature index value
  • the resource identification information carried in the resource acquisition request is calculated based on the first preset algorithm to obtain a target feature index value.
  • the first sending module is specifically configured to:
  • the second calculating module is specifically configured to:
  • the resource obtaining request carries a target data segment identifier
  • the second sending module is specifically configured to:
  • the resource file corresponding to the target content index value exists locally, and the resource file includes all resource data pointed to by the target data segment identifier, all the target data segment identifiers in the resource file are fed back Resource data
  • the resource acquisition request of the target resource is sent to the resource server.
  • the device further includes:
  • a storage module configured to receive the resource file of the target resource that is fed back by the resource server, and store the target feature index value, the target content index value, and the resource file correspondingly locally.
  • a cache server comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or a set of instructions, the at least one instruction, the at least one instruction A method of loading, by the processor, the set of instructions or the set of instructions by the processor to implement a feedback resource file as described in the first aspect.
  • a fourth aspect provides a computer readable storage medium, where the storage medium stores at least one instruction, at least one program, a code set, or a set of instructions, the at least one instruction, the at least one program, and the code
  • a set or set of instructions is loaded by a processor and executed to implement a method of feeding back a resource file as described in the first aspect.
  • the resource acquisition request of the target resource is received, and the resource identification information carried in the resource acquisition request is calculated according to the first preset algorithm to obtain the target feature index value; if the resource file corresponding to the target feature index value exists locally, Feeding the resource file corresponding to the target feature index value, otherwise sending a data acquisition request of the specified segment of the target resource to the resource server of the target resource; receiving the specified segment data of the target resource fed back by the resource server, and specifying the segment data based on the second preset algorithm Performing a calculation to obtain a target content index value; if a resource file corresponding to the target content index value exists locally, the resource file corresponding to the target content index value is fed back, otherwise the resource acquisition request of the target resource is sent to the resource server.
  • the cache server first matches the corresponding resource file by using the feature index value. If the corresponding resource file exists locally, the corresponding resource file can be directly fed back. If not, the content index is based on the content index. The value determines whether the corresponding resource file exists in the local, and the content index value is calculated based on the specified segment data of the target resource. Since the specified segment data is fixed, the resource identification information may be avoided to some extent. The problem of not being able to match the corresponding resources locally, so that the same data resource is returned to the source multiple times and cached, not only reduces the repeated storage rate of the data resources, improves the cache utilization rate, but also improves the speed of resource file feedback.
  • FIG. 1 is a schematic diagram of a scenario frame of a feedback resource file according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a method for feeding back a resource file according to an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of an apparatus for feeding back a resource file according to an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of an apparatus for feeding back a resource file according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a cache server according to an embodiment of the present invention.
  • the embodiment of the present invention provides a method for feeding back a resource file
  • the execution entity of the method may be a cache server
  • the cache server may be a server for storing data resources in a CDN service cluster, and the same geographical location in the entire network scenario
  • Multiple cache servers can be deployed in the area, each cache server is used to store different data resources, and multiple cache servers can be connected to distributors in the CDN service cluster.
  • the distributor may be configured to receive a resource acquisition request sent by the terminal, and distribute the resource acquisition request to the corresponding cache server according to a policy such as resource uniqueness, load balancing, and proximity service.
  • the cache server can also be connected to the resource server. When the resource file of a certain data resource needs to be acquired, the resource file can be downloaded from the corresponding resource server, and the resource file is stored locally.
  • the scenario framework of the CDN service cluster can be as shown in Figure 1.
  • the cache server may include a processor, a memory, and a transceiver.
  • the processor may be used to process a feedback resource file in the following process.
  • the memory may be used to store data and data generated during the following processing, and send and receive data.
  • the device can be used to receive and transmit relevant data in the following processing.
  • Step 201 The cache server receives the resource acquisition request of the target resource, and calculates the target feature index value by calculating the resource identification information carried by the resource acquisition request according to the first preset algorithm.
  • a user when a user wants to acquire a certain data resource (ie, a target resource) through a terminal, if the user wants to watch a video or download a file, the user may send a resource acquisition request of the target resource through the terminal.
  • the distributor of the CDN service cluster can receive the resource acquisition request, and then can determine the target resource pointed to by the resource acquisition request. Afterwards, the distributor may select a cache server according to the attribute information of the terminal and the target resource, such as selecting a service area where the terminal is located, and/or selecting a cache server that establishes a communication connection with the resource server of the target resource, and The get request is sent to the cache server.
  • the cache server can receive the resource acquisition request of the target resource, and then read the resource identification information carried by the resource acquisition request, and the resource identification information can be used to find and identify the target resource.
  • the cache server may calculate the resource identification information based on the first preset algorithm to obtain a unique target feature index value, where the first preset algorithm may be MD5 (Message Digest Algorithm Fifth Edition, Message- Digest Algorithm5) algorithm.
  • MD5 Message Digest Algorithm Fifth Edition, Message- Digest Algorithm5 algorithm.
  • the feature information in the resource identification information may be extracted first, and then the feature index value is calculated based on the feature information.
  • the process in step 201 may be as follows: if the resource identification information carried in the resource acquisition request meets the local pre-stored target information The format, the cache server extracts the feature information in the resource identification information according to the target information format, and calculates the target feature index value based on the first preset algorithm; if the resource identification information carried in the resource acquisition request does not meet the local pre-stored In any information format, the cache server calculates the target feature index value based on the resource identification information carried in the resource acquisition request by using the first preset algorithm.
  • the technician on the cache server side can analyze the resource identification information corresponding to the data resource in advance, and then summarize the unified information format of some resource identification information, and store the information format in the cache server, and the information format can be The similarity is: the total number of bytes is 30-40 bytes, including 4 parts, the first 4 bytes record the request method, then the 8 bytes are the domain name, and the last 10 bytes are the URL.
  • the cache server may determine whether the resource identification information carried in the resource acquisition request conforms to a certain information format pre-stored locally.
  • the cache server may extract the feature information in the resource identification information according to the target information format, and then calculate the feature information by using the first preset algorithm to obtain the target feature index value. If the resource identification information does not meet any of the pre-stored information formats, the cache server may directly calculate the target feature index value by using the first preset algorithm.
  • the data format of the resource resource of each resource server or the resource identification information corresponding to each resource format has a high probability of being consistent, so the technician can base the resource server and Or the resource format groups the data resources, and then sets one or more information formats for each group, so that for the resource acquisition request of the target resource, after obtaining the resource identification information, the cache server may first determine the group to which the target resource belongs. And then determining whether the resource identification information conforms to a certain information format corresponding to the group.
  • the resource identification information may change in format, the feature information included in the resource identification information is relatively fixed. Therefore, by extracting the feature information in the resource identification information, the target feature is calculated based on the feature information. The index value will not change due to the change of the format of the resource identification information, which is more accurate.
  • Step 202 If the resource file corresponding to the target feature index value exists locally, the cache server feeds back the resource file corresponding to the target feature index value, and otherwise sends the data acquisition request of the specified segment of the target resource to the resource server of the target resource.
  • the cache server stores the corresponding feature index value and the content index value when storing the resource file of each data resource, wherein the feature index value can be calculated and generated by the method for calculating the target feature index value;
  • the cache server may obtain the resource data of the specified segment of the resource file, and then calculate the resource data of the specified segment according to the second preset algorithm to generate a content index value of the resource file, so that the cache server may use the feature index value and the content index.
  • the value and the resource file are correspondingly stored.
  • the cache server may establish a resource storage directory, where the corresponding storage record has a feature index value, a content index value, and a storage address of the resource file.
  • the cache server may search the resource storage directory for the local feature index value with the same target feature index value. If yes, the cache server may feed back the terminal to the terminal through the distributor.
  • the resource file corresponding to the feature index value if not present, the cache server may send a data acquisition request of the specified segment of the target resource to the resource server of the target resource.
  • the specified segment can be pre-set by the technician of the CDN service cluster and written into the cache server, the resource file applicable to all data resources, and can be the header fragment of the first N bytes of length of each resource file.
  • the target feature index value is calculated according to the feature information, in some cases, if the resource corresponding to the target feature index value does not exist locally, The file can be considered that the corresponding resource file is not stored locally, and the cache server can directly send the resource acquisition request of the target resource to the resource server of the target resource.
  • the resource server does not need to send a data acquisition request of the specified segment.
  • the processing of step 202 may be as follows: if the data volume of the resource file of the target resource is greater than the data volume of the specified segment.
  • the cache server sends a data acquisition request of the specified segment of the target resource to the resource server of the target resource, and otherwise sends a resource acquisition request of the target resource to the resource server.
  • the cache server may determine the data size of the resource file of the target resource of the current request, if the data volume is greater than the specified segment. The amount of data, the cache server may send a data acquisition request of the specified segment of the target resource to the resource server of the target resource, if the resource file of the target resource is a small file whose data volume is smaller than the specified segment, or part of the breakpoint resume mode. The resource, and the amount of data is smaller than the amount of data of the specified fragment, the cache server can directly send the resource acquisition request of the target resource to the resource server. In this way, for a request with a data amount smaller than a resource of a specified segment, feedback can be directly performed, which improves the feedback speed.
  • Step 203 The cache server receives the specified segment data of the target resource fed back by the resource server, and calculates the target segment index value by calculating the specified segment data based on the second preset algorithm.
  • the resource server of the target resource may feed back the specified segment data of the corresponding target resource to the cache server.
  • the cache server may calculate the target segment index value based on the second preset algorithm to obtain the target content index value.
  • the second preset algorithm may be the same as the first preset algorithm in step 201, or may be different, but the characteristics of the algorithm are the uniqueness of the calculation result, that is, the calculation result of the same input is the same, and the input is different. The calculation results are different.
  • the content index value may be calculated according to the size of the entire resource.
  • the processing of step 203 may be as follows: the cache server obtains the complete resource data size of the target resource, and combines the full resource data size of the target resource, based on the second preset. The algorithm calculates the specified segment data to obtain the target content index value.
  • the full resource data size of the target resource may be recorded.
  • the cache server may first acquire the complete resource data size of the target resource, and then combine the complete resource data size of the target resource to calculate the specified fragment data fed back by the resource server based on the second preset algorithm.
  • the target content index value can be obtained.
  • the target content index value may be calculated by combining the attribute information of other target resources carried in the feedback message. It can be understood that no matter what kind of processing is used, the way of calculating the target content index value here should always be consistent with the calculation of the stored content index value in the cache server.
  • Step 204 If the resource file corresponding to the target content index value exists locally, the cache server feeds back the resource file corresponding to the target content index value, and otherwise sends the resource acquisition request of the target resource to the resource server.
  • the cache server may search the resource storage directory for whether there is a content index value that is the same as the target content index value. If yes, the cache server can feed back the resource file corresponding to the content index value to the terminal through the distributor. If not, the cache server can send the resource acquisition request of the target resource to the resource server of the target resource. Further, the cache server may feed back the resource file of the target resource sent by the resource server to the terminal through the distributor. Further, after receiving the resource file of the target resource fed back by the resource server, the cache server stores the target feature index value, the target content index value, and the resource file correspondingly locally.
  • the user may only request the resource data of the part of the data segment, and the resource acquisition request may carry the target data segment identifier.
  • the processing of step 204 may be as follows: if the resource file corresponding to the target content index value exists locally, The resource file includes all the resource data pointed to by the target data segment identifier, and the cache server feeds back all the resource data pointed to by the target data segment identifier in the resource file; if the resource file corresponding to the target content index value exists locally, and the resource file does not Contains all the resource data pointed to by the target data segment identifier, and the cache server sends a resource acquisition request of the target resource to the resource server.
  • the cache server may obtain the target data segment identifier carried in the resource acquisition request of the target resource. In this way, if the resource file corresponding to the locally stored target content index value is found, the cache server can determine whether the resource file includes all the resource data pointed to by the target data segment identifier. If included, the cache server can feed back all the resource data pointed to by the target data segment identifier. If not, the cache server can send a resource acquisition request of the target resource to the resource server. Further, the resource server may feed back all the resource data pointed to by the target data segment identifier of the target resource to the cache server. Afterwards, the cache server may feed back the corresponding resource data to the terminal, and add the resource data to the resource file corresponding to the target content index value. It should be noted that the foregoing processing is also applicable to the process of feeding back the resource file corresponding to the target feature index value in step 202. The specific steps are basically similar, and are not described in detail in this embodiment.
  • the resource acquisition request of the target resource is received, and the resource identification information carried in the resource acquisition request is calculated according to the first preset algorithm to obtain the target feature index value; if the resource file corresponding to the target feature index value exists locally, Feeding the resource file corresponding to the target feature index value, otherwise sending a data acquisition request of the specified segment of the target resource to the resource server of the target resource; receiving the specified segment data of the target resource fed back by the resource server, and specifying the segment data based on the second preset algorithm Performing a calculation to obtain a target content index value; if a resource file corresponding to the target content index value exists locally, the resource file corresponding to the target content index value is fed back, otherwise the resource acquisition request of the target resource is sent to the resource server.
  • the cache server first matches the corresponding resource file by using the feature index value. If the corresponding resource file exists locally, the corresponding resource file can be directly fed back. If not, the content index is based on the content index. The value determines whether the corresponding resource file exists in the local, and the content index value is calculated based on the specified segment data of the target resource. Since the specified segment data is fixed, the resource identification information may be avoided to some extent. The problem of not being able to match the corresponding resources locally, so that the same data resource is returned to the source multiple times and cached, not only reduces the repeated storage rate of the data resources, improves the cache utilization rate, but also improves the speed of resource file feedback.
  • an embodiment of the present invention further provides a device for feeding back a resource file.
  • the device includes
  • the first calculation module 301 is configured to receive a resource acquisition request of the target resource, and calculate, according to the first preset algorithm, the resource identification information carried by the resource acquisition request to obtain a target feature index value;
  • the first sending module 302 is configured to: if a resource file corresponding to the target feature index value exists locally, feed back a resource file corresponding to the target feature index value, and otherwise send the target resource to a resource server of the target resource. Specify a data acquisition request for the fragment;
  • the second calculating module 303 is configured to receive the specified segment data of the target resource that is fed back by the resource server, and calculate the target segment index value by using the second preset algorithm to calculate the target content index value;
  • the second sending module 304 is configured to: if the resource file corresponding to the target content index value exists locally, feed back the resource file corresponding to the target content index value, and otherwise send the resource acquiring request of the target resource to the resource server .
  • the first calculating module 301 is specifically configured to:
  • the resource identification information carried in the resource acquisition request meets the local pre-stored target information format, extracting feature information in the resource identification information according to the target information format, and the feature is based on the first preset algorithm
  • the information is calculated to obtain a target feature index value
  • the resource identification information carried in the resource acquisition request is calculated according to the first preset algorithm to obtain a target feature index value.
  • the first sending module 302 is specifically configured to:
  • the second calculating module 303 is specifically configured to:
  • the second sending module 304 is specifically configured to:
  • the target resource file with the highest similarity between the feature information and the resource identification information in the plurality of resource files is fed back.
  • the resource obtaining request carries a target data segment identifier
  • the second sending module 304 is specifically configured to:
  • the resource file corresponding to the target content index value exists locally, and the resource file includes all resource data pointed to by the target data segment identifier, all the target data segment identifiers in the resource file are fed back Resource data
  • the resource acquisition request of the target resource is sent to the resource server.
  • the device further includes:
  • the storage module 305 is configured to receive the resource file of the target resource that is fed back by the resource server, and store the target feature index value, the target content index value, and the resource file correspondingly locally.
  • the resource acquisition request of the target resource is received, and the resource identification information carried in the resource acquisition request is calculated according to the first preset algorithm to obtain the target feature index value; if the resource file corresponding to the target feature index value exists locally, Feeding the resource file corresponding to the target feature index value, otherwise sending a data acquisition request of the specified segment of the target resource to the resource server of the target resource; receiving the specified segment data of the target resource fed back by the resource server, and specifying the segment data based on the second preset algorithm Performing a calculation to obtain a target content index value; if a resource file corresponding to the target content index value exists locally, the resource file corresponding to the target content index value is fed back, otherwise the resource acquisition request of the target resource is sent to the resource server.
  • the cache server first matches the corresponding resource file by using the feature index value. If the corresponding resource file exists locally, the corresponding resource file can be directly fed back. If not, the content index is based on the content index. The value determines whether the corresponding resource file exists in the local, and the content index value is calculated based on the specified segment data of the target resource. Since the specified segment data is fixed, the resource identification information may be avoided to some extent. The problem of not being able to match the corresponding resources locally, so that the same data resource is returned to the source multiple times and cached, not only reduces the repeated storage rate of the data resources, improves the cache utilization rate, but also improves the speed of resource file feedback.
  • the device for feeding back the resource file provided by the foregoing embodiment when used for the feedback of the resource file, only the division of each functional module is used for example. In an actual application, the function may be assigned to different functional modules according to requirements. Upon completion, the internal structure of the device is divided into different functional modules to perform all or part of the functions described above.
  • the apparatus for the feedback resource file provided by the foregoing embodiment is the same as the method embodiment of the feedback resource file, and the specific implementation process is described in detail in the method embodiment, and details are not described herein again.
  • FIG. 5 is a schematic structural diagram of a cache server according to an embodiment of the present invention.
  • the cache server 500 can vary considerably depending on configuration or performance, and can include one or more central processors 522 (eg, one or more processors) and memory 532, one or more storage applications 542 or Storage medium 530 of data 544 (eg, one or one storage device in Shanghai).
  • the memory 532 and the storage medium 530 may be short-term storage or persistent storage.
  • Programs stored on storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations in a cache server.
  • central processor 522 can be configured to communicate with storage medium 530, executing a series of instruction operations in storage medium 530 on cache server 500.
  • Cache server 500 may also include one or more power supplies 526, one or more wired or wireless network interfaces 550, one or more input and output interfaces 558, one or more keyboards 556, and/or one or more operating systems. 541, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • the cache server 500 can include a memory, and one or more programs, wherein one or more programs are stored in the memory and configured to execute the one or more programs by one or more processors for performing The above instructions for the feedback resource file.
  • a person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium.
  • the storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Power Engineering (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明公开了一种反馈资源文件的方法和装置，属于数据存储技术领域。所述方法包括：接收目标资源的资源获取请求，基于第一预设算法对资源获取请求携带的资源识别信息进行计算得到目标特征索引值；如果本地存在目标特征索引值对应的资源文件，则反馈目标特征索引值对应的资源文件，否则向目标资源的资源服务器发送目标资源的指定片段的数据获取请求；接收资源服务器反馈的目标资源的指定片段数据，基于第二预设算法对指定片段数据进行计算得到目标内容索引值；如果本地存在目标内容索引值对应的资源文件，则反馈目标内容索引值对应的资源文件，否则向资源服务器发送目标资源的资源获取请求。采用本发明，可以提高缓存利用率。

Description

一种反馈资源文件的方法和装置 技术领域
本发明涉及数据存储技术领域，特别涉及一种反馈资源文件的方法和装置。
背景技术
随着互联网技术的不断进步，CDN(内容分发网络，Content Delivery Network)服务也随之快速发展。CDN服务集群中包括大量用于存储数据资源的缓存服务器，用户可以以较短的时间，就近获取其中存储的数据资源。
当用户想要通过终端获取某个数据资源时，如想要观看视频或者下载文件，用户可以通过终端发送资源获取请求，资源获取请求可以携带有资源识别信息，资源识别信息中可以包含请求方法、目的IP(网络协议，Internet Protocol)地址、域名、资源名、URL(统一资源定位符，Uniform Resource Locator)等特征信息。CDN服务集群在接收到资源获取请求后，可以将其分发至一台缓存服务器。缓存服务器可以根据资源识别信息查找本地是否存储有上述数据资源，如果没有，则可以从相应的资源服务器中获取数据资源，再将数据资源反馈给终端，同时可以将数据资源存储在本地。这样，后续如果再接收到相同的资源获取请求，缓存服务器则可以直接反馈本次存储的数据资源。
在实现本发明的过程中，发明人发现现有技术至少存在以下问题：
对于同一数据资源，不同资源服务器的资源识别信息的信息格式不同，而为了提高数据资源的安全性，部分资源服务器也会经常调整资源识别信息的信息格式，这样，即使多个资源获取请求指向同一数据资源，其中携带的资源识别信息也很有可能不同，而对于该数据资源，缓存服务器则需要从资源服务器处获取并缓存多份数据资源，故而，缓存服务器中可能存储有大量重复的数据资源，缓存利用率较低。
发明内容
为了解决现有技术的问题，本发明实施例提供了一种反馈资源文件的方法 和装置。所述技术方案如下：
第一方面，提供了一种反馈资源文件的方法，所述方法包括：
接收目标资源的资源获取请求，基于第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值；
如果本地存在所述目标特征索引值对应的资源文件，则反馈所述目标特征索引值对应的资源文件，否则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求；
接收所述资源服务器反馈的所述目标资源的指定片段数据，基于第二预设算法对所述指定片段数据进行计算得到目标内容索引值；
如果本地存在所述目标内容索引值对应的资源文件，则反馈所述目标内容索引值对应的资源文件，否则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，所述基于所述第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值，包括：
如果所述资源获取请求携带的资源识别信息符合本地预先存储的目标信息格式，则根据所述目标信息格式提取所述资源识别信息中的特征信息，基于所述第一预设算法对所述特征信息进行计算得到目标特征索引值；
如果所述资源获取请求携带的资源识别信息不符合本地预先存储的任一信息格式，则基于所述第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值。
可选的，所述向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求，包括：
如果目标资源的资源文件的数据量大于指定片段的数据量，则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求，否则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，所述基于所述第二预设算法对所述指定片段数据进行计算得到目标内容索引值，包括：
获取所述目标资源的完整资源数据大小，结合所述目标资源的完整资源数据大小，基于所述第二预设算法对所述指定片段数据进行计算得到目标内容索引值。
可选的，所述资源获取请求携带有目标数据段标识；
所述如果本地存在所述目标内容索引值对应的资源文件，则反馈所述目标内容索引值对应的资源文件，包括：
如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中包含所述目标数据段标识指向的全部资源数据，则反馈所述资源文件中的所述目标数据段标识指向的全部资源数据；
如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中不包含所述目标数据段标识指向的全部资源数据，则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，所述向所述资源服务器发送所述目标资源的资源获取请求之后，还包括：
接收所述资源服务器反馈的所述目标资源的资源文件，将所述目标特征索引值、所述目标内容索引值和所述资源文件对应存储在本地。
第二方面，提供了一种反馈资源文件的装置，所述装置包括：
第一计算模块，用于接收目标资源的资源获取请求，基于第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值；
第一发送模块，用于如果本地存在所述目标特征索引值对应的资源文件，则反馈所述目标特征索引值对应的资源文件，否则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求；
第二计算模块，用于接收所述资源服务器反馈的所述目标资源的指定片段数据，基于第二预设算法对所述指定片段数据进行计算得到目标内容索引值；
第二发送模块，用于如果本地存在所述目标内容索引值对应的资源文件，则反馈所述目标内容索引值对应的资源文件，否则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，所述第一计算模块，具体用于：
如果所述资源获取请求携带的资源识别信息符合本地预先存储的目标信息格式，则根据所述目标信息格式提取所述资源识别信息中的特征信息，基于所述第一预设算法对所述特征信息进行计算得到目标特征索引值；
如果所述资源获取请求携带的资源识别信息不符合本地预先存储的任一信 息格式，则基于所述第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值。
可选的，所述第一发送模块，具体用于：
如果目标资源的资源文件的数据量大于指定片段的数据量，则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求，否则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，所述第二计算模块，具体用于：
获取所述目标资源的完整资源数据大小，结合所述目标资源的完整资源数据大小，基于所述第二预设算法对所述指定片段数据进行计算得到目标内容索引值。
可选的，所述资源获取请求携带有目标数据段标识；
所述第二发送模块，具体用于：
如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中包含所述目标数据段标识指向的全部资源数据，则反馈所述资源文件中的所述目标数据段标识指向的全部资源数据；
如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中不包含所述目标数据段标识指向的全部资源数据，则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，所述装置还包括：
存储模块，用于接收所述资源服务器反馈的所述目标资源的资源文件，将所述目标特征索引值、所述目标内容索引值和所述资源文件对应存储在本地。
第三方面，提供了一种缓存服务器，所述缓存服务器包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如第一方面所述的反馈资源文件的方法。
第四方面，提供了一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如第一方面所述的反 馈资源文件的方法。
本发明实施例提供的技术方案带来的有益效果是：
本发明实施例中，接收目标资源的资源获取请求，基于第一预设算法对资源获取请求携带的资源识别信息进行计算得到目标特征索引值；如果本地存在目标特征索引值对应的资源文件，则反馈目标特征索引值对应的资源文件，否则向目标资源的资源服务器发送目标资源的指定片段的数据获取请求；接收资源服务器反馈的目标资源的指定片段数据，基于第二预设算法对指定片段数据进行计算得到目标内容索引值；如果本地存在目标内容索引值对应的资源文件，则反馈目标内容索引值对应的资源文件，否则向资源服务器发送目标资源的资源获取请求。这样，缓存服务器在接收到资源获取请求后，先通过特征索引值来匹配相应的资源文件，若本地存有相应的资源文件，则可直接反馈相应的资源文件，若不存在，再基于内容索引值判断本地是否存有相应的资源文件，而内容索引值是基于目标资源的指定片段数据计算得出，由于指定片段数据是固定的，由此可在一定程度上避免了由于资源识别信息不同，无法在本地匹配到相应资源，从而对同一数据资源多次回源并缓存的问题，不仅减少了数据资源的重复存储率，提高缓存利用率，而且提升了资源文件反馈的速度。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的一种反馈资源文件的场景框架示意图；
图2是本发明实施例提供的一种反馈资源文件的方法流程图；
图3是本发明实施例提供的一种反馈资源文件的装置结构示意图；
图4是本发明实施例提供的一种反馈资源文件的装置结构示意图；
图5是本发明实施例提供的一种缓存服务器的结构示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
本发明实施例提供了一种反馈资源文件的方法，该方法的执行主体可以是缓存服务器，其中，缓存服务器可以是CDN服务集群中用于存储数据资源的服务器，在整个网络场景中，同一地理区域内可以部署有多个缓存服务器，每个缓存服务器分别用于存储不同的数据资源，多个缓存服务器可以与CDN服务集群中的分发器相连。分发器可以用于接收终端发送的资源获取请求，并基于资源唯一性、负载均衡、就近服务等策略将资源获取请求分发到相应的缓存服务器处。缓存服务器还可以与资源服务器连接，当需要获取某个数据资源的资源文件时，可以从相应的资源服务器处下载该资源文件，并将资源文件存储在本地。
CDN服务集群的场景框架可以如图1所示。上述缓存服务器中可以包括处理器、存储器、收发器，处理器可以用于进行下述流程中的反馈资源文件的处理，存储器可以用于存储下述处理过程中需要的数据以及产生的数据，收发器可以用于接收和发送下述处理过程中的相关数据。
下面将结合具体实施方式，对图2所示的处理流程进行详细的说明，内容可以如下：
步骤201，缓存服务器接收目标资源的资源获取请求，基于第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值。
在实施中，当用户想要通过终端获取某个数据资源(即目标资源)时，如想要观看视频或者下载文件，用户可以通过终端发送目标资源的资源获取请求。CDN服务集群的分发器可以接收到该资源获取请求，然后可以确定该资源获取请求指向的目标资源。之后，分发器可以根据终端和目标资源的属性信息选择出一台缓存服务器，如选择服务于终端所在区域，和/或选择与目标资源的资源服务器建立有通信连接的缓存服务器，并将上述资源获取请求发送给该缓存服务器。这样，缓存服务器可以接收到目标资源的资源获取请求，然后读取该资源获取请求携带的资源识别信息，该资源识别信息可以用于查找并标识目标资源。接下来，缓存服务器可以基于第一预设算法对上述资源识别信息进行计算，从而得到唯一的目标特征索引值，此处的第一预设算法可以是MD5(消息摘要算法第五版，Message-Digest Algorithm5)算法。
可选的，可以先提取资源识别信息中的特征信息，然后基于特征信息计算特征索引值，相应的，步骤201的处理可以如下：如果资源获取请求携带的资源识别信息符合本地预先存储的目标信息格式，缓存服务器则根据目标信息格式提取资源识别信息中的特征信息，基于第一预设算法对特征信息进行计算得到目标特征索引值；如果资源获取请求携带的资源识别信息不符合本地预先存储的任一信息格式，缓存服务器则基于第一预设算法对资源获取请求携带的资源识别信息进行计算得到目标特征索引值。
在实施中，缓存服务器侧的技术人员可以预先对数据资源对应的资源识别信息进行分析，然后归纳整理出一些资源识别信息的统一的信息格式，并将信息格式存储在缓存服务器中，信息格式可以类似为：总字节数为30-40字节，包含为4部分，前4字节记录请求方法，之后8字节为域名，最后10字节为URL。这样，缓存服务器在接收到目标资源的资源获取请求后，可以判断该资源获取请求携带的资源识别信息是否符合本地预先存储的某一信息格式。如果符合目标信息格式，缓存服务器则可以按照目标信息格式提取资源识别信息中的特征信息，然后通过第一预设算法对特征信息进行计算得到目标特征索引值。而如果资源识别信息不符合本地预先存储的任一信息格式，缓存服务器则可以通过第一预设算法，直接对整个资源识别信息进行计算得到目标特征索引值。可以理解，每种资源服务器的数据资源或每种资源格式(如视频、文档、图片)下的数据资源对应的资源识别信息的信息格式有很大概率一致，故而，技术人员可以基于资源服务器和/或资源格式对数据资源分组，再为每组设置一个或多个信息格式，这样，对于目标资源的资源获取请求，缓存服务器在获取了其中的资源识别信息后，可以先确定目标资源所属组，然后判断资源识别信息是否符合该组对应的某一信息格式。可以理解的是，虽然资源识别信息可能会发生格式变化，但其所包含的特征信息较为固定，如此一来，通过对资源识别信息中的特征信息进行提取，再基于特征信息计算出的目标特征索引值则不会因资源识别信息格式的改变而发生变化，较为准确。
步骤202，如果本地存在目标特征索引值对应的资源文件，缓存服务器则反馈目标特征索引值对应的资源文件，否则向目标资源的资源服务器发送目标资源的指定片段的数据获取请求。
在实施中，缓存服务器在存储每个数据资源的资源文件时，都会同时存储 对应的特征索引值和内容索引值，其中，特征索引值可以通过上述计算目标特征索引值的方法计算生成；同时，缓存服务器可以获取该资源文件的指定片段的资源数据，然后按照第二预设算法对指定片段的资源数据进行计算从而生成该资源文件的内容索引值，进而缓存服务器可以将特征索引值、内容索引值和资源文件对应存储，具体的，缓存服务器可以建立资源存储目录，该资源存储目录中对应记录有特征索引值、内容索引值和资源文件的存储地址。这样，在计算得到目标资源的目标特征索引值之后，缓存服务器可以在资源存储目录中查找本地是否存在目标特征索引值相同的特征索引值，如果存在，缓存服务器则可以通过分发器向终端反馈该特征索引值对应的资源文件，如果不存在，缓存服务器则可以向目标资源的资源服务器发送目标资源的指定片段的数据获取请求。此处，指定片段可以由CDN服务集群的技术人员预先设置并写入缓存服务器中，适用于所有数据资源的资源文件，可以是每个资源文件的起始N个字节长度的头部片段。
值得一提的是，基于上述资源识别信息符合预先存储的信息格式的处理，如果目标特征索引值是根据特征信息计算得到的，在某种情况下，如果本地不存在目标特征索引值对应的资源文件，可以认为本地确实未存储有相应的资源文件，缓存服务器则可以直接向目标资源的资源服务器发送目标资源的资源获取请求。
可选的，对于一些数据量较小的资源文件，资源服务器无需发送指定片段的数据获取请求，相应的，步骤202的处理可以如下：如果目标资源的资源文件的数据量大于指定片段的数据量，缓存服务器则向目标资源的资源服务器发送目标资源的指定片段的数据获取请求，否则向资源服务器发送目标资源的资源获取请求。
在实施中，缓存服务器在向目标资源的资源服务器发送目标资源的指定片段的数据获取请求之前，可以先确定本次请求的目标资源的资源文件的数据量大小，如果其数据量大于指定片段的数据量，缓存服务器则可以向目标资源的资源服务器发送目标资源的指定片段的数据获取请求，如果目标资源的资源文件是个数据量小于指定片段的小文件，或者是断点续传模式下的部分资源，且其数据量小于指定片段的数据量，缓存服务器则可以直接向资源服务器发送目标资源的资源获取请求。如此一来，对于数据量小于指定片段的资源的请求， 可直接进行反馈，提升了反馈速度。
步骤203，缓存服务器接收资源服务器反馈的目标资源的指定片段数据，基于第二预设算法对指定片段数据进行计算得到目标内容索引值。
在实施中，目标资源的资源服务器在接收到缓存服务器发送的目标资源的指定片段的数据获取请求后，可以向缓存服务器反馈相应的目标资源的指定片段数据。缓存服务器在接收到该指定片段数据之后，可以基于第二预设算法对指定片段数据进行计算得到目标内容索引值。需要说明的是，第二预设算法可以与步骤201中的第一预设算法相同，也可以不同，但算法的特征都是计算结果的唯一性，即相同输入的计算结果相同，不同输入的计算结果不同。
可选的，可以结合整个资源的大小计算内容索引值，具体的，步骤203的处理可以如下：缓存服务器获取目标资源的完整资源数据大小，结合目标资源的完整资源数据大小，基于第二预设算法对指定片段数据进行计算得到目标内容索引值。
在实施中，目标资源的资源服务器向缓存服务器发送指定片段数据的反馈消息中，可以记录有目标资源的完整资源数据大小。这样，缓存服务器在计算目标内容索引值前，可以先获取目标资源的完整资源数据大小，然后结合目标资源的完整资源数据大小，基于第二预设算法，对资源服务器反馈的指定片段数据进行计算，从而可以得到目标内容索引值。进一步的，除了完整资源数据大小，还可以结合上述反馈消息中携带的其它的目标资源的属性信息，计算目标内容索引值。可以理解，无论采用何种处理，此处计算目标内容索引值的方式应当始终和缓存服务器中已存储的内容索引值的计算方式保持一致。
步骤204，如果本地存在目标内容索引值对应的资源文件，缓存服务器则反馈目标内容索引值对应的资源文件，否则向资源服务器发送目标资源的资源获取请求。
在实施中，在计算得到目标资源的目标内容索引值后，缓存服务器可以在资源存储目录中查找本地是否存在与目标内容索引值相同的内容索引值。如果存在，缓存服务器则可以通过分发器向终端反馈该内容索引值对应的资源文件，如果不存在，缓存服务器则可以向目标资源的资源服务器发送目标资源的资源获取请求。进而，缓存服务器可以将资源服务器发送的目标资源的资源文件，通过分发器反馈给终端。进一步的，缓存服务器在接收资源服务器反馈的目标 资源的资源文件后，将目标特征索引值、目标内容索引值和资源文件对应存储在本地。
可选的，用户可以只请求部分数据段的资源数据，故而资源获取请求中可以携带有目标数据段标识，相应的，步骤204的处理可以如下：如果本地存在目标内容索引值对应的资源文件，且资源文件中包含目标数据段标识指向的全部资源数据，缓存服务器则反馈资源文件中的目标数据段标识指向的全部资源数据；如果本地存在目标内容索引值对应的资源文件，且资源文件中不包含目标数据段标识指向的全部资源数据，缓存服务器则向资源服务器发送目标资源的资源获取请求。
在实施中，缓存服务器在接收到目标资源的资源获取请求中，可以获取其中携带的目标数据段标识。这样，如果查找到了本地存储的目标内容索引值对应的资源文件，缓存服务器可以判断该资源文件中是否包含上述目标数据段标识指向的全部资源数据。如果包含，缓存服务器则可以反馈目标数据段标识指向的全部资源数据，如果不包含，缓存服务器则可以向资源服务器发送目标资源的资源获取请求。进而，资源服务器可以将目标资源的目标数据段标识指向的全部资源数据反馈给缓存服务器。之后，缓存服务器可以向终端反馈相应的资源数据，同时将这些资源数据添加至上述目标内容索引值对应的资源文件中。需要说明的是，上述处理同样适用于步骤202中反馈目标特征索引值对应的资源文件的过程，具体步骤基本相似，本实施例中不再赘述。
本发明实施例中，接收目标资源的资源获取请求，基于第一预设算法对资源获取请求携带的资源识别信息进行计算得到目标特征索引值；如果本地存在目标特征索引值对应的资源文件，则反馈目标特征索引值对应的资源文件，否则向目标资源的资源服务器发送目标资源的指定片段的数据获取请求；接收资源服务器反馈的目标资源的指定片段数据，基于第二预设算法对指定片段数据进行计算得到目标内容索引值；如果本地存在目标内容索引值对应的资源文件，则反馈目标内容索引值对应的资源文件，否则向资源服务器发送目标资源的资源获取请求。这样，缓存服务器在接收到资源获取请求后，先通过特征索引值来匹配相应的资源文件，若本地存有相应的资源文件，则可直接反馈相应的资源文件，若不存在，再基于内容索引值判断本地是否存有相应的资源文件，而内容索引值是基于目标资源的指定片段数据计算得出，由于指定片段数据是固 定的，由此可在一定程度上避免了由于资源识别信息不同，无法在本地匹配到相应资源，从而对同一数据资源多次回源并缓存的问题，不仅减少了数据资源的重复存储率，提高缓存利用率，而且提升了资源文件反馈的速度。
基于相同的技术构思，本发明实施例还提供了一种反馈资源文件的装置，如图3所示，所述装置包括
第一计算模块301，用于接收目标资源的资源获取请求，基于第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值；
第一发送模块302，用于如果本地存在所述目标特征索引值对应的资源文件，则反馈所述目标特征索引值对应的资源文件，否则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求；
第二计算模块303，用于接收所述资源服务器反馈的所述目标资源的指定片段数据，基于第二预设算法对所述指定片段数据进行计算得到目标内容索引值；
第二发送模块304，用于如果本地存在所述目标内容索引值对应的资源文件，则反馈所述目标内容索引值对应的资源文件，否则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，所述第一计算模块301，具体用于：
如果所述资源获取请求携带的资源识别信息符合本地预先存储的目标信息格式，则根据所述目标信息格式提取所述资源识别信息中的特征信息，基于所述第一预设算法对所述特征信息进行计算得到目标特征索引值；
如果所述资源获取请求携带的资源识别信息不符合本地预先存储的任一信息格式，则基于所述第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值。
可选的，所述第一发送模块302，具体用于：
如果目标资源的资源文件的数据量大于指定片段的数据量，则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求，否则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，所述第二计算模块303，具体用于：
获取所述目标资源的完整资源数据大小，结合所述目标资源的完整资源数据大小，基于所述第二预设算法对所述指定片段数据进行计算得到目标内容索 引值。
可选的，所述第二发送模块304，具体用于：
如果本地存在所述目标内容索引值对应的多个资源文件，则反馈所述多个资源文件中特征信息与所述资源识别信息相似度最高的目标资源文件。
可选的，所述资源获取请求携带有目标数据段标识；
所述第二发送模块304，具体用于：
如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中包含所述目标数据段标识指向的全部资源数据，则反馈所述资源文件中的所述目标数据段标识指向的全部资源数据；
如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中不包含所述目标数据段标识指向的全部资源数据，则向所述资源服务器发送所述目标资源的资源获取请求。
可选的，如图4所示，所述装置还包括：
存储模块305，用于接收所述资源服务器反馈的所述目标资源的资源文件，将所述目标特征索引值、所述目标内容索引值和所述资源文件对应存储在本地。
本发明实施例中，接收目标资源的资源获取请求，基于第一预设算法对资源获取请求携带的资源识别信息进行计算得到目标特征索引值；如果本地存在目标特征索引值对应的资源文件，则反馈目标特征索引值对应的资源文件，否则向目标资源的资源服务器发送目标资源的指定片段的数据获取请求；接收资源服务器反馈的目标资源的指定片段数据，基于第二预设算法对指定片段数据进行计算得到目标内容索引值；如果本地存在目标内容索引值对应的资源文件，则反馈目标内容索引值对应的资源文件，否则向资源服务器发送目标资源的资源获取请求。这样，缓存服务器在接收到资源获取请求后，先通过特征索引值来匹配相应的资源文件，若本地存有相应的资源文件，则可直接反馈相应的资源文件，若不存在，再基于内容索引值判断本地是否存有相应的资源文件，而内容索引值是基于目标资源的指定片段数据计算得出，由于指定片段数据是固定的，由此可在一定程度上避免了由于资源识别信息不同，无法在本地匹配到相应资源，从而对同一数据资源多次回源并缓存的问题，不仅减少了数据资源的重复存储率，提高缓存利用率，而且提升了资源文件反馈的速度。
需要说明的是：上述实施例提供的反馈资源文件的装置在反馈资源文件时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的反馈资源文件的装置与反馈资源文件的方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图5是本发明实施例提供的缓存服务器的结构示意图。该缓存服务器500可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上中央处理器522(例如，一个或一个以上处理器)和存储器532，一个或一个以上存储应用程序542或数据544的存储介质530(例如一个或一个以上海量存储设备)。其中，存储器532和存储介质530可以是短暂存储或持久存储。存储在存储介质530的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对缓存服务器中的一系列指令操作。更进一步地，中央处理器522可以设置为与存储介质530通信，在缓存服务器500上执行存储介质530中的一系列指令操作。
缓存服务器500还可以包括一个或一个以上电源526，一个或一个以上有线或无线网络接口550，一个或一个以上输入输出接口558，一个或一个以上键盘556，和/或，一个或一个以上操作系统541，例如Windows ServerTM，Mac OS XTM，UnixTM，LinuxTM，FreeBSDTM等等。
缓存服务器500可以包括有存储器，以及一个或者一个以上的程序，其中一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行所述一个或者一个以上程序包含用于进行上述反馈资源文件的指令。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的 精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (14)

  1. 一种反馈资源文件的方法，其特征在于，所述方法包括：
    接收目标资源的资源获取请求，基于第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值；
    如果本地存在所述目标特征索引值对应的资源文件，则反馈所述目标特征索引值对应的资源文件，否则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求；
    接收所述资源服务器反馈的所述目标资源的指定片段数据，基于第二预设算法对所述指定片段数据进行计算得到目标内容索引值；
    如果本地存在所述目标内容索引值对应的资源文件，则反馈所述目标内容索引值对应的资源文件，否则向所述资源服务器发送所述目标资源的资源获取请求。
  2. 根据权利要求1所述的方法，其特征在于，所述基于第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值，包括：
    如果所述资源获取请求携带的资源识别信息符合本地预先存储的目标信息格式，则根据所述目标信息格式提取所述资源识别信息中的特征信息，基于所述第一预设算法对所述特征信息进行计算得到目标特征索引值；
    如果所述资源获取请求携带的资源识别信息不符合本地预先存储的任一信息格式，则基于所述第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值。
  3. 根据权利要求1所述的方法，其特征在于，所述向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求，包括：
    如果目标资源的资源文件的数据量大于指定片段的数据量，则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求，否则向所述资源服务器发送所述目标资源的资源获取请求。
  4. 根据权利要求1所述的方法，其特征在于，所述基于第二预设算法对所 述指定片段数据进行计算得到目标内容索引值，包括：
    获取所述目标资源的完整资源数据大小，结合所述目标资源的完整资源数据大小，基于所述第二预设算法对所述指定片段数据进行计算得到目标内容索引值。
  5. 根据权利要求1所述的方法，其特征在于，所述资源获取请求携带有目标数据段标识；
    所述如果本地存在所述目标内容索引值对应的资源文件，则反馈所述目标内容索引值对应的资源文件，包括：
    如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中包含所述目标数据段标识指向的全部资源数据，则反馈所述资源文件中的所述目标数据段标识指向的全部资源数据；
    如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中不包含所述目标数据段标识指向的全部资源数据，则向所述资源服务器发送所述目标资源的资源获取请求。
  6. 根据权利要求1所述的方法，其特征在于，所述向所述资源服务器发送所述目标资源的资源获取请求之后，还包括：
    接收所述资源服务器反馈的所述目标资源的资源文件，将所述目标特征索引值、所述目标内容索引值和所述资源文件对应存储在本地。
  7. 一种反馈资源文件的装置，其特征在于，所述装置包括：
    第一计算模块，用于接收目标资源的资源获取请求，基于第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值；
    第一发送模块，用于如果本地存在所述目标特征索引值对应的资源文件，则反馈所述目标特征索引值对应的资源文件，否则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求；
    第二计算模块，用于接收所述资源服务器反馈的所述目标资源的指定片段数据，基于第二预设算法对所述指定片段数据进行计算得到目标内容索引值；
    第二发送模块，用于如果本地存在所述目标内容索引值对应的资源文件， 则反馈所述目标内容索引值对应的资源文件，否则向所述资源服务器发送所述目标资源的资源获取请求。
  8. 根据权利要求7所述的装置，其特征在于，所述第一计算模块，具体用于：
    如果所述资源获取请求携带的资源识别信息符合本地预先存储的目标信息格式，则根据所述目标信息格式提取所述资源识别信息中的特征信息，基于所述第一预设算法对所述特征信息进行计算得到目标特征索引值；
    如果所述资源获取请求携带的资源识别信息不符合本地预先存储的任一信息格式，则基于所述第一预设算法对所述资源获取请求携带的资源识别信息进行计算得到目标特征索引值。
  9. 根据权利要求7所述的装置，其特征在于，所述第一发送模块，具体用于：
    如果目标资源的资源文件的数据量大于指定片段的数据量，则向所述目标资源的资源服务器发送所述目标资源的指定片段的数据获取请求，否则向所述资源服务器发送所述目标资源的资源获取请求。
  10. 根据权利要求7所述的装置，其特征在于，所述第二计算模块，具体用于：
    获取所述目标资源的完整资源数据大小，结合所述目标资源的完整资源数据大小，基于所述第二预设算法对所述指定片段数据进行计算得到目标内容索引值。
  11. 根据权利要求7所述的装置，其特征在于，所述资源获取请求携带有目标数据段标识；
    所述第二发送模块，具体用于：
    如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中包含所述目标数据段标识指向的全部资源数据，则反馈所述资源文件中的所述目标数据段标识指向的全部资源数据；
    如果本地存在所述目标内容索引值对应的资源文件，且所述资源文件中不包含所述目标数据段标识指向的全部资源数据，则向所述资源服务器发送所述目标资源的资源获取请求。
  12. 根据权利要求7所述的装置，其特征在于，所述装置还包括：
    存储模块，用于接收所述资源服务器反馈的所述目标资源的资源文件，将所述目标特征索引值、所述目标内容索引值和所述资源文件对应存储在本地。
  13. 一种缓存服务器，其特征在于，所述缓存服务器包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如权利要求1至6任一所述的反馈资源文件的方法。
  14. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如权利要求1至6任一所述的反馈资源文件的方法。
PCT/CN2018/095431 2018-04-10 2018-07-12 一种反馈资源文件的方法和装置 WO2019196225A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP18877297.4A EP3579526B1 (en) 2018-04-10 2018-07-12 Resource file feedback method and apparatus
US16/327,668 US11206302B2 (en) 2018-04-10 2018-07-12 Method and device for feeding back a resource file

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810317282.6A CN108494875A (zh) 2018-04-10 2018-04-10 一种反馈资源文件的方法和装置
CN201810317282.6 2018-04-10

Publications (1)

Publication Number Publication Date
WO2019196225A1 true WO2019196225A1 (zh) 2019-10-17

Family

ID=63315547

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/095431 WO2019196225A1 (zh) 2018-04-10 2018-07-12 一种反馈资源文件的方法和装置

Country Status (4)

Country Link
US (1) US11206302B2 (zh)
EP (1) EP3579526B1 (zh)
CN (1) CN108494875A (zh)
WO (1) WO2019196225A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110769072A (zh) * 2019-10-31 2020-02-07 北京达佳互联信息技术有限公司 一种多媒体资源获取方法、装置和存储介质
CN112637262B (zh) * 2020-11-18 2022-12-02 网宿科技股份有限公司 一种内容刷新方法和装置
CN115051876A (zh) * 2022-08-12 2022-09-13 中兴通讯股份有限公司 通信方法、xr联动通信系统、运行控制装置及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102111685A (zh) * 2011-02-24 2011-06-29 深信服网络科技（深圳）有限公司 一种网络视频加载的加速方法、设备及系统
CN103248684A (zh) * 2013-04-28 2013-08-14 北京奇虎科技有限公司 一种互联网中资源获取方法和装置
US20140072275A1 (en) * 2012-09-13 2014-03-13 Sony Corporation Display control device, recording control device, and display control method
CN105915930A (zh) * 2016-04-15 2016-08-31 北京网康科技有限公司 一种视频文件发送方法及装置

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8903950B2 (en) * 2000-05-05 2014-12-02 Citrix Systems, Inc. Personalized content delivery using peer-to-peer precaching
EP1154356A1 (en) * 2000-05-09 2001-11-14 Alcatel Caching of files during loading from a distributed file system
US7315884B2 (en) * 2001-04-03 2008-01-01 Hewlett-Packard Development Company, L.P. Reduction of network retrieval latency using cache and digest
US7703006B2 (en) * 2005-06-02 2010-04-20 Lsi Corporation System and method of accelerating document processing
US20090049243A1 (en) 2007-08-13 2009-02-19 Microsoft Corporation Caching Dynamic Content
US8793307B2 (en) 2009-01-28 2014-07-29 Blue Coat Systems, Inc. Content associative caching method for web applications
WO2012100373A1 (en) * 2011-01-28 2012-08-02 Intel Corporation Techniques to request stored data from memory
CN102170479B (zh) * 2011-05-21 2013-12-18 华为数字技术（成都）有限公司 Web缓存的更新方法及Web缓存的更新装置
US10019371B2 (en) * 2012-04-27 2018-07-10 Hewlett Packard Enterprise Development Lp Data caching using local and remote memory
US8938603B2 (en) * 2012-05-31 2015-01-20 Samsung Electronics Co., Ltd. Cache system optimized for cache miss detection
US9407726B1 (en) 2013-03-15 2016-08-02 Juniper Networks, Inc. Caching objects identified by dynamic resource identifiers
US9244937B2 (en) * 2013-03-15 2016-01-26 International Business Machines Corporation Efficient calculation of similarity search values and digest block boundaries for data deduplication
CN103269353B (zh) * 2013-04-19 2016-11-02 网宿科技股份有限公司 Web缓存回源优化方法及Web缓存系统
CN104951480B (zh) * 2014-03-31 2018-05-08 杭州迪普科技股份有限公司 一种cdn系统中资源存储的索引装置及方法
CN106559677B (zh) * 2015-09-30 2020-04-03 华为技术有限公司 终端、缓存服务器及获取视频分片的方法及装置
US10237350B2 (en) * 2016-04-06 2019-03-19 Reniac, Inc. System and method for a database proxy
CN106028154B (zh) * 2016-06-30 2019-05-03 青岛海信电器股份有限公司 一种基于hls协议的节目播放方法和装置
CN107801051B (zh) * 2017-10-27 2019-01-22 广东省南方数字电视无线传播有限公司 虚拟切片信息传输方法和装置、视频服务器
US10997140B2 (en) * 2018-08-31 2021-05-04 Nxp Usa, Inc. Method and apparatus for acceleration of hash-based lookup

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102111685A (zh) * 2011-02-24 2011-06-29 深信服网络科技（深圳）有限公司 一种网络视频加载的加速方法、设备及系统
US20140072275A1 (en) * 2012-09-13 2014-03-13 Sony Corporation Display control device, recording control device, and display control method
CN103248684A (zh) * 2013-04-28 2013-08-14 北京奇虎科技有限公司 一种互联网中资源获取方法和装置
CN105915930A (zh) * 2016-04-15 2016-08-31 北京网康科技有限公司 一种视频文件发送方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3579526A4 *

Also Published As

Publication number Publication date
EP3579526A4 (en) 2019-12-11
CN108494875A (zh) 2018-09-04
EP3579526A1 (en) 2019-12-11
US11206302B2 (en) 2021-12-21
US20210344741A1 (en) 2021-11-04
EP3579526B1 (en) 2021-05-05

Similar Documents

Publication Publication Date Title
US11755371B1 (en) Data intake and query system with distributed data acquisition, indexing and search
US11461402B2 (en) Routing based request correlation
US11194719B2 (en) Cache optimization
US10812358B2 (en) Performance-based content delivery
US11394805B1 (en) Automatic discovery of API information
JP6643760B2 (ja) ショートリンクの処理方法、デバイス、及びサーバ
US20150237113A1 (en) Method and system for file transmission
WO2017092351A1 (zh) 缓存数据的更新方法及装置
US9906595B2 (en) Content source discovery
JP7056893B2 (ja) アプリケーションプログラミングインタフェースａｐｉ要求を伝送するための方法、装置、ａｐｉゲートウェイ、及びプログラム
US9407726B1 (en) Caching objects identified by dynamic resource identifiers
US9386099B2 (en) Local caching device, system and method for providing content caching service
WO2019196225A1 (zh) 一种反馈资源文件的方法和装置
WO2015106692A1 (en) Webpage pushing method, client, server, and system
WO2012139474A1 (zh) 数据的获取方法、设备和系统
CN112513830A (zh) 内容分发网络中的回源方法及相关装置
US20190379709A1 (en) Session Migration - Based Scheduling Method and Server
CN110764688B (zh) 对数据进行处理的方法和装置
JP2017500679A (ja) メディアリソースフィードバック方法、装置、プログラム及び記録媒体
US10021207B1 (en) Architecture for proactively providing bundled content items to client devices
WO2017097027A1 (zh) 基于因特网内容适配协议的数据处理方法、装置和系统
CN108810043B (zh) 一种存储评论信息的方法和装置
WO2020093318A1 (zh) 资源查询处理方法、装置、计算机设备和存储介质

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2018877297

Country of ref document: EP

Effective date: 20190521

NENP Non-entry into the national phase

Ref country code: DE