WO2021104100A1 - Cdn中url刷新方法、装置、设备以及cdn节点 - Google Patents

Cdn中url刷新方法、装置、设备以及cdn节点 Download PDF

Info

Publication number
WO2021104100A1
WO2021104100A1 PCT/CN2020/129230 CN2020129230W WO2021104100A1 WO 2021104100 A1 WO2021104100 A1 WO 2021104100A1 CN 2020129230 W CN2020129230 W CN 2020129230W WO 2021104100 A1 WO2021104100 A1 WO 2021104100A1
Authority
WO
WIPO (PCT)
Prior art keywords
url
refreshed
refresh
cdn
refresh request
Prior art date
Application number
PCT/CN2020/129230
Other languages
English (en)
French (fr)
Inventor
王永强
年彦东
Original Assignee
北京金山云网络技术有限公司
北京金山云科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京金山云网络技术有限公司, 北京金山云科技有限公司 filed Critical 北京金山云网络技术有限公司
Priority to US17/778,102 priority Critical patent/US20230022141A1/en
Priority to JP2022551426A priority patent/JP7392168B2/ja
Publication of WO2021104100A1 publication Critical patent/WO2021104100A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2379—Updates performed during online database operations; commit processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/957—Browsing optimisation, e.g. caching or content distillation
    • G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Definitions

  • This application relates to the field of data processing technology, and in particular to a method, device, device, and CDN node for URL refresh in CDN.
  • the content delivery network (Content Delivery Network, CDN) is an intelligent virtual network built on the basis of the existing network, which can realize uniform resource locator (URL) content storage and URL content distribution.
  • URL uniform resource locator
  • the existing CDN server includes multiple storages. In practical applications, it is often necessary to refresh the URLs stored in these memories to ensure the real-time and correctness of the URLs stored in the CDN.
  • each storage in the CDN needs to perform a delete action on all URLs to be refreshed. Therefore, in the current URL refresh method in the CDN, the workload of each storage in the CDN is relatively large, resulting in low URL refresh efficiency in the CDN in the CDN.
  • the purpose of this application is to provide a URL refresh method, device, device, and CDN node in a CDN to solve the technical problem of low URL refresh efficiency in the CDN.
  • an embodiment of the present application provides a method for refreshing a URL in a CDN, including: receiving a first refresh request, where the first refresh request includes at least one URL to be refreshed; and combining the at least one URL to be refreshed with a local database The stored URLs are compared; the local database stores the URLs corresponding to the locally stored source files; and the refresh operation is performed for the URLs to be refreshed in the database.
  • the URL stored in the database is determined according to a local back-to-source log, and the back-to-source log includes a back-to-source URL.
  • the step of performing a refresh operation for the URL to be refreshed existing in the database includes: deleting URLs that are not in the database among the at least one URL to be refreshed to obtain The URL that exists in the database among the at least one URL to be refreshed; and the refresh operation is performed for the URL that exists in the database among the at least one URL to be refreshed.
  • the first refresh request comes from an auxiliary device
  • the at least one URL to be refreshed from the auxiliary device is obtained by the auxiliary device filtering at least one URL to be refreshed from the load balancing device .
  • the method includes: receiving a second refresh request from a load balancing device, where the second refresh request includes at least one URL to be refreshed from the load balancing device; based on an identifier stored in a local database, for the at least one refresh request from the load balancing device
  • the URL is filtered to obtain at least one URL to be refreshed after filtering; the local database is used to record the identification of the source file stored in the storage device in the same node; and the first refresh request is sent to the storage device in the same node.
  • the first refresh request includes the filtered at least one URL to be refreshed; so that the storage device filters the filtered at least one URL to be refreshed, and performs refresh for the filtered filtered at least one URL to be refreshed operating.
  • the identifier of the source file is a domain name
  • the method further includes: receiving a back-to-source log sent by a storage device in the same node, the back-to-source log including a back-to-source URL; based on the back-to-source The domain name in the URL determines the local database storage.
  • a device for refreshing a URL in a CDN which is characterized by comprising: a receiving module configured to receive a first refresh request, where the first refresh request includes at least one URL to be refreshed; and a comparison module configured to The at least one URL to be refreshed is compared with the URL stored in the local database; the local database stores the URL corresponding to the locally stored source file; the refresh module is set to execute the URL to be refreshed in the database Refresh operation.
  • a device for refreshing a URL in a CDN which is characterized by comprising: a receiving module configured to receive a second refresh request from a load balancing device, the second refresh request including at least one refresh request from the load balancing device URL to be refreshed; a filtering module configured to filter the at least one URL to be refreshed from the load balancing device based on the identifier stored in the local database to obtain at least one URL to be refreshed after filtering; the local database is used to record the same The identifier of the source file stored by the storage device in the node; a sending module configured to send a first refresh request to the storage device in the same node, the first refresh request including the filtered at least one URL to be refreshed; The storage device filters the filtered at least one URL to be refreshed, and performs a refresh operation on the filtered filtered at least one URL to be refreshed.
  • a storage device in a fifth aspect, includes: a memory and a processor, the memory stores a computer program that can run on the processor, and is characterized in that, when the processor executes the computer program, the steps of the method described in the first aspect are implemented .
  • an auxiliary device includes: a memory and a processor, the memory stores a computer program that can run on the processor, and is characterized in that, when the processor executes the computer program, the steps of the method described in the second aspect are implemented .
  • a CDN node includes: a storage device configured to receive a first refresh request, where the first refresh request includes at least one URL to be refreshed; comparing the at least one URL to be refreshed with a URL stored in a local database; the local database stores URL corresponding to the locally stored source file; performing a refresh operation for the URL to be refreshed in the database.
  • it further includes: a load balancing device, configured to send the first refresh request to the storage device.
  • it further includes: an auxiliary device configured to receive a second refresh request from the load balancing device, the second refresh request including at least one URL to be refreshed from the load balancing device; based on local database storage The identification of the at least one URL to be refreshed from the load balancing device is filtered to obtain the filtered at least one URL to be refreshed; the local database is used to record the identification of the source file stored in the storage device in the same node; The storage device in the same node sends a first refresh request, where the first refresh request includes the filtered at least one URL to be refreshed.
  • the embodiments of the present application further provide a computer-readable storage medium, the computer-readable storage medium stores machine-runnable instructions, and when the computer-runnable instructions are called and run by a processor, the The computer-executable instructions cause the processor to execute the method described in the first aspect or the second aspect described above.
  • the method, apparatus and electronic device for URL refreshing in CDN can filter at least one URL to be refreshed by using the URL stored in the database in the storage device, and can realize that only the URL actually stored in the storage device is performed. Refresh reduces the operation of invalid URL refresh, reduces the workload pressure of the storage device, and greatly improves the execution efficiency of URL refresh in the CDN, thereby improving the overall URL refresh efficiency in the CDN.
  • FIG. 1 is a schematic diagram of a CDN node structure provided by an embodiment of the application
  • FIG. 2 is a schematic diagram of another CDN node structure provided by an embodiment of the application.
  • FIG. 3 is a schematic flowchart of a URL refresh method in a CDN provided by an embodiment of the application
  • FIG. 4 is a schematic flowchart of another URL refresh method in CDN provided by an embodiment of the application.
  • FIG. 5 is a schematic structural diagram of a URL refreshing device in a CDN provided by this application.
  • FIG. 6 is a schematic structural diagram of another URL refreshing device in a CDN provided by this application.
  • FIG. 7 is a schematic structural diagram of a storage device provided by this application.
  • FIG. 8 is a schematic structural diagram of an auxiliary device provided by this application.
  • the URL refreshing methods in the CDN provided by the existing CDN include: URL refreshing in the precise CDN, directory refreshing, and filtering and refreshing for URL matching with regular expressions. These refreshing methods are to send all URLs that need to be refreshed to each storage device (also known as CACHE device) in the CDN node. The reason why it is necessary to send all URLs to each CACHE device is because it needs to ensure dirty data It is deleted in every CACHE device.
  • the refresh request is sent to the CACHE of the CDN or other storage, and the CACHE or other storage inside the CDN performs the refresh operation.
  • URL storage uses a consistent hash algorithm to maximize The real file corresponding to a URL is stored on a CACHE device in the distributed CDN cluster. All reverse agents in the CDN cluster will store and forward the URL to the corresponding CACHE for storage, which can be distributed Each CACHE device in the cluster stores different content, which realizes the maximum utilization efficiency of distributed cluster storage space.
  • the CDN cluster may increase or decrease the number of CACHE devices, etc.
  • the same data may also be stored on multiple CACHE devices, and only refreshed by consistent hashing may be dirty.
  • Refreshing needs to ensure that files are deleted first, so in order to ensure safety, the existing logic is to delete all URLs to be refreshed in each CACHE device, but in fact, each CACHE device needs to be stored.
  • the number of refreshed URLs is far less than the total number of all URLs, resulting in a large number of invalid refreshes.
  • the refreshing efficiency is low due to the large amount of tasks. Therefore, the invalid refresh task is heavy, which leads to low refresh efficiency.
  • the CACHE device processes many invalid requests, which makes the workload pressure larger.
  • the embodiment of the present application provides a URL refresh method in the CDN, by which the URL actually stored in the storage can be refreshed, which reduces the invalid operation of refreshing the URL in the storage and reduces the storage load in the CDN.
  • the workload pressure has further improved the URL refresh efficiency in the CDN in the overall CDN.
  • Fig. 1 is a schematic diagram of a CDN node structure provided by an embodiment of the application.
  • the CDN node may include a load balancing device 101 and a storage device 102.
  • the storage device 102 is configured to: receive a first refresh request, the first refresh request includes at least one URL to be refreshed; compare the at least one URL to be refreshed with a URL stored in a local database; the local database stores a locally stored source file Corresponding URL; perform a refresh operation for the URL to be refreshed in the database.
  • the load balancing device 101 is configured to send a first refresh request to the storage device 102.
  • the storage device 102 in the embodiment of the present application may be a CACHE device.
  • CACHE cache Memory
  • the CACHE device parses a URL that is accessed for the first time, it will return to the origin site to obtain the source file.
  • the CACHE device is parsing a URL that is accessed and there is no source file locally, it will obtain the source file from the upper-level CDN node or the source site.
  • the CACHE device pulls the source file from the source station, it will record the log locally. This log can be called the back-to-source log, and the back-to-source log records each time back to the source information.
  • each CACHE device for each CACHE device, a respective back-to-source log is recorded, and the back-to-source log may be recorded in the database 130.
  • the database 130 sinks to each CACHE device in the CDN node. Since each CACHE device records its own return-to-source information, it can reduce leakage, improve real-time performance, and improve the accuracy of URL data recording.
  • the database 130 may also be referred to as the first record table.
  • a database can be established on each CACHE device.
  • the database is used to store the list of URLs that are returned to the source through this CACHE device, that is, which CACHE is used to return to the source, and which one the URL corresponds to is stored in.
  • the database of the CACHE device As an example, every time the CACHE device has a back-to-origin action, it will print a back-to-origin log and store it in the database.
  • the back-to-origin log stored in the database can also be deduplicated based on the URL.
  • the database responsible for recording back to the source log is sinked into each CACHE device in the CDN node, instead of simply being arranged in the log collection device of the CDN node as a whole.
  • Fig. 2 is a schematic diagram of another CDN node structure provided by an embodiment of the application.
  • the CDN node may include a load balancing device 201, a storage device 202, and an auxiliary device 203.
  • the storage device 202 is configured to: receive a first refresh request, the first refresh request includes at least one URL to be refreshed; compare the at least one URL to be refreshed with a URL stored in a local database; the local database stores a locally stored source file Corresponding URL; perform a refresh operation for the URL to be refreshed in the database.
  • the load balancing device 201 is configured to send a second refresh request to the auxiliary device 203.
  • the auxiliary device 203 is configured to receive a second refresh request from the load balancing device 201, where the second refresh request includes at least one URL to be refreshed from the load balancing device 201; One URL to be refreshed is filtered to obtain at least one URL to be refreshed after filtering; the local database is used to record the identification of the source file stored in the storage device in the same node; the first refresh request is sent to the storage device in the same node, and the first refresh The request includes at least one URL to be refreshed after filtering.
  • the auxiliary device 203 and the storage device 202 are double-filtered.
  • the auxiliary device 203 may be a log collection and processing device, and the return-to-source log of each storage device 202 may be reported to the auxiliary device 203 of the same node, and multiple auxiliary devices 203 may be provided in the same node to ensure stability.
  • the auxiliary device 203 has a certain delay in collecting logs in real time compared to the storage device 202, but it can perform most of the refresh filtering.
  • FIG. 3 is a schematic flowchart of a method for URL refresh in a CDN provided by an embodiment of the application. This method can be applied to the memory in the CDN node. As shown in FIG. 3, the method includes:
  • S310 Receive a first refresh request, where the first refresh request includes at least one URL to be refreshed.
  • CDN mainly relies on edge servers deployed in various places, through the load balancing, content distribution, scheduling and other functional modules of the central platform, so that users can obtain the required content nearby, reducing network congestion, improving user access response speed and hit rate, etc. effect.
  • the CDN node to which the embodiment of the present application is applicable may be as shown in FIG. 1 or FIG. 2 described above.
  • the load balancing device in the CDN node may directly send the first URL to be refreshed to all storage devices in the same node.
  • the load balancing device in the CDN node can send the second URL to be refreshed to the auxiliary device in the same node.
  • the first waiting URL is obtained.
  • the URL is refreshed, and the first URL to be refreshed is sent to all storage devices in the same node.
  • the URL stored in the local database is used to record the URL corresponding to the source file stored in the storage device.
  • the source file is recorded locally so that there is no need to obtain it from the source site in the next request.
  • the URL stored in the local database may be in the form of a record table or a database.
  • the URL stored in the local database can filter out the URL that is not stored locally.
  • the URL stored in the local database may be determined based on the local back-to-source log, and the back-to-source log includes the back-to-source URL.
  • the back-to-source log can be understood with reference to the description in the embodiment shown in FIG. 1, and will not be repeated here.
  • the content corresponding to the URL to be refreshed stored locally in the storage device is refreshed. For example, delete the URL to be refreshed and its corresponding source file.
  • At least one URL to be refreshed can be filtered by the URL stored in the database in the storage device, and only the URL actually stored in the storage device can be refreshed, which reduces invalid URL refresh operations and reduces the storage device
  • the workload pressure of the CDN is that the execution efficiency of URL refresh in CDN has been greatly improved, which in turn improves the overall URL refresh efficiency in CDN.
  • the refresh efficiency of the URL refresh method in the CDN may be proportional to the amount of memory in the CDN node. For example, there are a total of m URLs that need to be refreshed, and there are a total of n storage devices, such as n cache (CACHE) devices, in a CDN node. Each storage device in n storage devices can only perform m/n refresh operations. If a piece of URL data only exists in one storage device, only m refresh operations are required in total.
  • CACHE n cache
  • step S330 may be specifically implemented through the following steps:
  • Step 1) deleting at least one URL that does not exist in the database among at least one URL to be refreshed, to obtain a URL that exists in the database among at least one URL to be refreshed;
  • Step 2) Perform a refresh operation for at least one URL that exists in the database among the URLs to be refreshed.
  • the first refresh request comes from the auxiliary device, and the at least one URL to be refreshed from the auxiliary device is obtained by the auxiliary device filtering the at least one URL to be refreshed from the load balancing device.
  • the load balancing device may first send the first refresh request to the auxiliary device, and the auxiliary device filters the first refresh request, and then sends the filtered first refresh request to the storage device.
  • the first refresh request may also be filtered by the load balancing device, and the load balancing device may directly send the filtered first refresh request to the storage device.
  • the auxiliary device or the load balancing device before the storage receives the first refresh request, the auxiliary device or the load balancing device first filters the URLs to be refreshed in the first refresh request; after the storage receives the first refresh request after the first filtering, The storage reuses the URL stored in the local database to filter the URL to be refreshed in the first refresh request after the first filtering for the second time.
  • FIG. 4 is a schematic flowchart of another URL refresh method in a CDN provided by an embodiment of the application. This method can be applied to the memory in the CDN node. As shown in FIG. 3, the method includes:
  • S410 Receive a second refresh request from the load balancing device, where the second refresh request includes at least one URL to be refreshed from the load balancing device.
  • S420 Filter at least one URL to be refreshed from the load balancing device based on the identifier stored in the local database to obtain at least one URL to be refreshed after filtering; the local database is used to record the identifier of the source file stored in the storage device in the same node.
  • S430 Send a first refresh request to a storage device in the same node, where the first refresh request includes at least one filtered URL to be refreshed; so that the storage device can filter the filtered at least one URL to be refreshed, and target the filtered filtering At least one URL to be refreshed after that performs a refresh operation.
  • the influence of the storage device can be reduced.
  • the service domain names are also massive.
  • Each CDN node usually does not serve all customers, and the domain names served by the CDN node have a certain regionality. Therefore, the domain names in the CDN node
  • the auxiliary device can be filtered based on the service domain name, and the domain names that are not served by the CDN node are filtered out first. Based on this, the identification of the source file may be a domain name, and the method may further include the following steps:
  • Step a) receiving the back-to-source log sent by the storage device in the same node.
  • the return-to-origin log includes the return-to-origin URL.
  • composition of the URL includes: protocol, domain name, path, and parameters.
  • Fig. 5 is a schematic structural diagram of a URL refreshing device in a CDN provided by this application.
  • the URL refreshing device 500 in the CDN includes:
  • the receiving module 501 is configured to receive a first refresh request, where the first refresh request includes at least one URL to be refreshed;
  • the comparison module 502 is configured to compare at least one URL to be refreshed with a URL stored in a local database; the local database stores a URL corresponding to a locally stored source file;
  • the refresh module 503 is configured to perform a refresh operation for the URL to be refreshed in the database.
  • the URL stored in the database is determined based on the local back-to-source log, and the back-to-source log includes the back-to-source URL.
  • the refresh module 503 is specifically configured as:
  • the URL refreshing device in the CDN provided by the embodiment of the present application has the same technical features as the URL refreshing method in the CDN provided in the embodiment shown in FIG. 3, so it can also solve the same technical problems and achieve the same technical effects.
  • Fig. 6 is a schematic structural diagram of another URL refreshing device in a CDN provided by this application.
  • the URL refreshing device 600 in the CDN includes:
  • the receiving module 601 is configured to receive a second refresh request from the load balancing device, where the second refresh request includes at least one URL to be refreshed from the load balancing device;
  • the filtering module 602 is configured to filter at least one URL to be refreshed from the load balancing device based on the identifier stored in the local database to obtain at least one URL to be refreshed after filtering; the local database is used to record the source stored by the storage device in the same node The identification of the file;
  • the sending module 603 is configured to send a first refresh request to a storage device in the same node.
  • the first refresh request includes at least one filtered URL to be refreshed; so that the storage device filters the filtered at least one URL to be refreshed, and targets At least one URL to be refreshed after the filtering is filtered to perform a refresh operation.
  • the identifier of the source file is a domain name
  • the apparatus 600 further includes:
  • the receiving module 601 is further configured to receive the back-to-source log sent by the storage device in the same node, where the back-to-source log includes the back-to-source URL;
  • the determination module is set to determine the local database storage based on the domain name in the back-to-source URL.
  • the URL refreshing device in the CDN provided by the embodiment of the present application has the same technical features as the URL refreshing method in the CDN provided in the embodiment shown in FIG. 4, so it can also solve the same technical problems and achieve the same technical effects.
  • a storage device 700 provided by an embodiment of the present application includes a processor 701, a memory 702, and a bus.
  • the memory 702 stores machine-readable instructions executable by the processor 701.
  • the processor 701 communicates with the memory 702 through a bus, and the processor 701 executes the machine-readable instructions to execute the steps of the URL refresh method in the above CDN.
  • the aforementioned memory 702 and the processor 701 can be general-purpose memories and processors, which are not specifically limited here.
  • the processor 701 runs the computer program stored in the memory 702, it can execute the aforementioned URL refresh method in the CDN.
  • an auxiliary device 800 provided by an embodiment of the present application includes a processor 801, a memory 802, and a bus.
  • the memory 802 stores machine-readable instructions executable by the processor 801.
  • the processor 801 communicates with the memory 802 through a bus, and the processor 801 executes the machine-readable instructions to execute the steps of the URL refresh method in the CDN described above.
  • the aforementioned memory 802 and the processor 801 can be general-purpose memories and processors, which are not specifically limited here.
  • the processor 801 runs the computer program stored in the memory 802, it can execute the aforementioned URL refresh method in the CDN.
  • the storage device or auxiliary device may further include: a communication interface, and the processor, the communication interface, and the memory are connected by a bus; the processor is used to execute an executable module stored in the memory, such as a computer program.
  • the memory may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
  • RAM Random Access Memory
  • non-volatile memory such as at least one disk memory.
  • the communication connection between the network element of the system and at least one other network element is realized through at least one communication interface (which may be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used.
  • the bus can be an ISA bus, a PCI bus, or an EISA bus.
  • the bus can be divided into an address bus, a data bus, a control bus, and so on. For ease of presentation, only one bidirectional arrow is used in FIG. 7 or FIG. 8, but it does not mean that there is only one bus or one type of bus.
  • the memory is used to store a program, and the processor executes the program after receiving the execution instruction.
  • the method executed by the process definition apparatus disclosed in any one of the embodiments of the present application can be applied to the processor, or Processor implementation.
  • the processor may be an integrated circuit chip with signal processing capabilities.
  • each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software.
  • the aforementioned processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP for short) , Application Specific Integrated Circuit (ASIC for short), Field-Programmable Gate Array (FPGA for short) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • DSP Central Processing Unit
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
  • an embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores machine-runnable instructions, and the computer-runnable instructions are called by the processor and When running, the computer can run instructions to cause the processor to run the steps of the URL refresh method in the CDN.
  • the URL refreshing device in the CDN provided in the embodiment of the present application may be specific hardware on the device or software or firmware installed on the device.
  • the implementation principles and technical effects of the device provided in the embodiment of the application are the same as those of the foregoing method embodiment.
  • the parts not mentioned in the device embodiment please refer to the corresponding content in the foregoing method embodiment.
  • the specific working process of the aforementioned system, device, and unit can all refer to the corresponding process in the above method embodiment, which will not be repeated here.
  • the disclosed device and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation.
  • multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some communication interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the embodiments provided in this application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the mobility control method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .
  • each block in the flowchart or block diagram may represent a module, program segment, or part of the code, and the module, program segment, or part of the code contains one or more functions for realizing the specified logical function. Executable instructions. It should also be noted that in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings.
  • each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions.
  • the URL stored in the database in the storage device can be used to filter at least one URL to be refreshed, so that only the URL actually stored in the storage device can be refreshed. , Reducing invalid URL refresh operations, reducing the workload pressure of storage devices, making the execution efficiency of URL refreshing in the CDN greatly improved, thereby improving the overall URL refreshing efficiency in the CDN.

Abstract

一种CDN中URL刷新方法、装置、设备以及CDN节点，涉及数据处理技术领域，CDN中的URL刷新效率较低的技术问题。该方法包括：接收第一刷新请求，第一刷新请求包括至少一个待刷新URL(S310)；将至少一个待刷新URL与本地数据库存储的URL进行对比；本地数据库中存储有本地储存的源文件对应的URL(S320)；执行针对存在于数据库中的待刷新URL的刷新操作(S330)。

Description

CDN中URL刷新方法、装置、设备以及CDN节点
本申请要求于2019年11月27日提交中国专利局、申请号为201911197702.2、申请名称为“CDN中URL刷新方法、装置、设备以及CDN节点”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据处理技术领域，尤其是涉及一种CDN中URL刷新方法、装置、设备以及CDN节点。
背景技术
内容分发网络(Content Delivery Network，CDN)是构建在现有网络基础之上的智能虚拟网络，能够实现统一资源定位符(uniform resource locator，URL)的内容存储和URL内容分发。
为了实现URL内容的存储，现有的CDN服务器中包括有多个存储器。在实际应用中，经常需要对这些存储器中储存的URL进行刷新，以保证CDN中储存的URL的实时性和正确性。
为了确保所有的待刷新URL都能够在CDN刷新过程中被删除，CDN中的每个存储器都需要执行对所有的待刷新URL的删除动作。因此，在目前的CDN中URL刷新方法中，CDN中的每个存储器的工作量都较大，导致CDN中的CDN中URL刷新效率较低。
发明内容
本申请的目的在于提供一种CDN中URL刷新方法、装置、设备以及CDN节点，以解决CDN中的CDN中URL刷新效率较低的技术问题。
第一方面，本申请实施例提供了一种CDN中URL刷新方法，包括：接收第一刷新请求，所述第一刷新请求包括至少一个待刷新URL；将所述至少一个待刷新URL与本地数据库存储的URL进行对比；所述本地数据库中存储有本地储存的源文件对应的URL；执行针对存在于所述数据库中的待刷新URL的刷新操作。
在一种可能的实施方式中，所述数据库中存储的所述URL根据本地回源日志确定，所述回源日志包括回源URL。
在一种可能的实施方式中，执行针对存在于所述数据库中的待刷新URL的刷新操作的步骤，包括：将所述至少一个待刷新URL中未存在于所述数据库中的URL删除，得到所述至少一个待刷新URL中存在于所述数据库中的URL；针对所述至少一个待刷新URL中存在于所述数据库中的URL，执行刷新操作。
在一种可能的实施方式中，所述至第一刷新请求来自辅助设备，来自所述辅助设备的至少一个待刷新URL由所述辅助设备对来自负载均衡设备的至少一个待刷新URL进行过滤得到。
第二方面，提供了另一种CDN中URL刷新方法。包括：接收来自负载均衡设备的第二刷新请求，所述第二刷新请求包括来自负载均衡设备的至少一个待刷新URL；基于本地 数据库存储的标识，对所述来自负载均衡设备的至少一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；所述本地数据库用于记录同一节点内存储设备存储的源文件的标识；向所述同一节点内的存储设备发送第一刷新请求，所述第一刷新请求包括所述过滤后的至少一个待刷新URL；以便所述存储设备对所述过滤后的至少一个待刷新URL进行过滤，并针对经过过滤的过滤后的至少一个待刷新URL执行刷新操作。
在一种可能的实施方式中，源文件的标识为域名，所述方法还包括：接收同一节点内的存储设备发送的回源日志，所述回源日志包括回源URL；基于所述回源URL中的域名，确定所述本地数据库存储。
第三方面，提供了一种CDN中URL刷新装置，其特征在于，包括：接收模块，设置为接收第一刷新请求，所述第一刷新请求包括至少一个待刷新URL；对比模块，设置为将所述至少一个待刷新URL与本地数据库存储的URL进行对比；所述本地数据库中存储有本地储存的源文件对应的URL；刷新模块，设置为执行针对存在于所述数据库中的待刷新URL的刷新操作。
第四方面，提供了一种CDN中URL刷新装置，其特征在于，包括：接收模块，设置为接收来自负载均衡设备的第二刷新请求，所述第二刷新请求包括来自负载均衡设备的至少一个待刷新URL；过滤模块，设置为基于本地数据库存储的标识，对所述来自负载均衡设备的至少一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；所述本地数据库用于记录同一节点内存储设备存储的源文件的标识；发送模块，设置为向所述同一节点内的存储设备发送第一刷新请求，所述第一刷新请求包括所述过滤后的至少一个待刷新URL；以便所述存储设备对所述过滤后的至少一个待刷新URL进行过滤，并针对经过过滤的过滤后的至少一个待刷新URL执行刷新操作。
第五方面，提供了一种存储设备。包括：存储器、处理器，所述存储器中存储有可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现上述第一方面所述的方法的步骤。
第六方面，提供了一种辅助设备。包括：存储器、处理器，所述存储器中存储有可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现上述第二方面所述的方法的步骤。
第七方面，提供了一种CDN节点。包括：存储设备，设置为接收第一刷新请求，所述第一刷新请求包括至少一个待刷新URL；将所述至少一个待刷新URL与本地数据库存储的URL进行对比；所述本地数据库中存储有本地储存的源文件对应的URL；执行针对存在于所述数据库中的待刷新URL的刷新操作。
在一种可能的实施方式中，还包括：负载均衡设备，设置为向所述存储设备发送所述第一刷新请求。
在一种可能的实施方式中，还包括：辅助设备，设置为接收来自负载均衡设备的第二刷新请求，所述第二刷新请求包括来自负载均衡设备的至少一个待刷新URL；基于本地数据库存储的标识，对所述来自负载均衡设备的至少一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；所述本地数据库用于记录同一节点内存储设备存储的源文件的标识；向所述同一节点内的存储设备发送第一刷新请求，所述第一刷新请求包括所述过滤后 的至少一个待刷新URL。
第八方面，本申请实施例又提供了一种计算机可读存储介质，所述计算机可读存储介质存储有机器可运行指令，所述计算机可运行指令在被处理器调用和运行时，所述计算机可运行指令促使所述处理器运行上述的第一方面或第二方面所述方法。
本申请实施例带来了以下有益效果：
本申请实施例提供的一种CDN中URL刷新方法、装置以及电子设备，能够利用存储设备中数据库存储的URL，对至少一个待刷新URL进行过滤，可以实现只针对存储设备中实际存储的URL进行刷新，减少了无效刷新URL的操作，减轻了存储设备的工作量压力，使得CDN中URL刷新的执行效率得到了大幅度提高，进而提高了整体CDN中的URL刷新效率。
为使本申请的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本申请具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种CDN节点结构示意图；
图2为本申请实施例提供的另一种CDN节点结构示意图；
图3为本申请实施例提供的一种CDN中URL刷新方法的流程示意图；
图4为本申请实施例提供的另一种CDN中URL刷新方法的流程示意图；
图5为本申请提供的一种CDN中URL刷新装置的结构示意图；
图6为本申请提供的另一种CDN中URL刷新装置的结构示意图；
图7为本申请提供的一种存储设备结构示意图；
图8为本申请提供的一种辅助设备结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合附图对本申请的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例中所提到的术语“包括”和“具有”以及它们的任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括其他没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
目前，现有的CDN提供的CDN中URL刷新方式包括：精确CDN中URL刷新、目录刷新以及针对URL进行正则表达式匹配进行筛选刷新。这些刷新的方式都是将需要刷新的所有URL发送到CDN节点内每一台存储设备(又称CACHE设备)中，之所以需要将所有URL都发送每一台CACHE设备，是因为需要确保脏数据在每台CACHE设备中都被删除。刷新请求被发送到CDN的CACHE中或其他存储器中，由CDN内部的CACHE或其他存储器执行刷新操作。
由于现有的CDN都采用分布式集群进行服务，即反向代理结合CACHE存储的架构，而不是单机的架构，在一个分布式集群中，URL存储通过一致性哈希的算法，尽可能的将一条URL对应的真实文件存储在分布式CDN集群中的某一台CACHE设备上，CDN集群内所有的反向代理都将这条URL的存储转发到对应的CACHE上存储，这样可做到分布式集群内的各个CACHE设备都存储的是不同内容，实现了分布式集群存储空间的最大化利用效率。
但是，由于是分布式存储的结构，CDN集群可能增减CACHE设备的数量等，同一份数据也有可能存储在多个CACHE设备上，只通过一致性哈希刷新，可能存在刷不干净。刷新需要确保文件被删除是第一位的，所以为了确保安全，现有的逻辑都是将所有的待刷新URL，在每台CACHE设备中都会删除一次，但是实际上每台CACHE设备存储的需要刷新的URL数量，远远少于所有的URL总数，从而造成了大量的无效刷新，同时由于任务量较大导致刷新效率低下。因此，无效刷新任务重，导致刷新效率低下，同时CACHE设备处理了很多无效请求，使之工作量压力较大。
为了解决上述问题，本申请实施例提供了一种CDN中URL刷新方法，通过该方法能够针对存储器中实际存储的URL进行刷新，减少了存储器的无效刷新URL的操作，减轻了CDN中的存储器的工作量压力，进而提高了整体CDN中的CDN中URL刷新效率。
图1为本申请实施例提供的一种CDN节点结构示意图。如图1所示，该CDN节点可以包括负载均衡设备101和存储设备102。
其中，存储设备102设置为：接收第一刷新请求，第一刷新请求包括至少一个待刷新URL；将至少一个待刷新URL与本地数据库存储的URL进行对比；本地数据库中存储有本地储存的源文件对应的URL；执行针对存在于数据库中的待刷新URL的刷新操作。
负载均衡设备101设置为：向存储设备102发送第一刷新请求。
示例性的，如图1所示，本申请实施例中的存储设备102可以为CACHE设备。对于CACHE(高速缓冲存储器)设备来说，该CACHE设备在解析某一个第一次被访问的URL时，则会回源站去获取源文件。另外，如果CACHE设备在解析某一个被访问的URL时，本地未存在源文件，则会向上层CDN节点或源站去获取源文件。在CACHE设备向源站拉取源文件时，本地会记录日志，该日志可以称为回源日志，该回源日志记录每次回源信息。
在本申请实施例中，如图1所示，针对每个CACHE设备，均记录各自的回源日志，该回源日志可以记录在数据库130中。该数据库130下沉到CDN节点中的每一台CACHE设备中。由于每一台CACHE设备均记录有各自的回源信息，可以降低泄漏，提高实时性，提高了对URL数据记录的正确性。
该数据库130，也可以称为第一记录表。在实际应用中，可以在每台CACHE设备上建 立一个数据库，该数据库用于存储通过在这台CACHE设备回源的URL列表，即通过哪一台CACHE回源，URL对应的存储在哪一台CACHE设备的数据库中。作为一个示例，CACHE设备每有一次回源动作，都会打印一条回源日志，然后存储在数据库中，数据库存储的回源日志，还可以基于URL进行去重。
在本申请实施例中，通过将负责记录回源日志的数据库下沉到CDN节点中的每一台CACHE设备中，而不是简单的布置在CDN节点整体的日志收集设备中。通过在每台CACHE设备中都建立数据库，不仅能够使数据库中储存的回源日志可以永久保留，还能够直接查询本机的回源日志，提高查询URL的效率。
图2为本申请实施例提供的另一种CDN节点结构示意图。如图2所示，该CDN节点可以包括负载均衡设备201、存储设备202和辅助设备203。
其中，存储设备202设置为：接收第一刷新请求，第一刷新请求包括至少一个待刷新URL；将至少一个待刷新URL与本地数据库存储的URL进行对比；本地数据库中存储有本地储存的源文件对应的URL；执行针对存在于数据库中的待刷新URL的刷新操作。
负载均衡设备201设置为：向辅助设备203发送第二刷新请求。
辅助设备203设置为：接收来自负载均衡设备201的第二刷新请求，第二刷新请求包括来自负载均衡设备201的至少一个待刷新URL；基于本地数据库存储的标识，对来自负载均衡设备201的至少一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；本地数据库用于记录同一节点内存储设备存储的源文件的标识；向同一节点内的存储设备发送第一刷新请求，第一刷新请求包括过滤后的至少一个待刷新URL。
但是由于数据库下沉到CDN节点的存储设备202，查询数据库势必消耗存储设备的计算资源，存储设备202的主要任务是服务客户，计算资源十分宝贵，为了减少查询数据库的计算资源消耗，还可以设置辅助设备203和存储设备202双重过滤的方式。辅助设备203可以为日志收集处理设备，每台存储设备202的回源日志可以会汇报到同节点的辅助设备203，同一节点中辅助设备203可以设置有多台，以确保稳定。辅助设备203收集日志的实时性较存储设备202有一定的延迟，但是可进行大部分的刷新过滤。
图3为本申请实施例提供的一种CDN中URL刷新方法的流程示意图。该方法可以应用于CDN节点中的存储器中，如图3所示，该方法包括：
S310，接收第一刷新请求，第一刷新请求包括至少一个待刷新URL。
其中，CDN主要依靠部署在各地的边缘服务器，通过中心平台的负载均衡、内容分发、调度等功能模块，使用户就近获取所需内容，起到降低网络拥塞、提高用户访问响应速度和命中率等作用。本申请实施例适用的CDN节点可以如前述图1或图2所示。
作为一个示例，CDN节点中的负载均衡设备，在接收到第一待刷新URL后，可以直接将该第一待刷新URL发送给同一节点中的所有存储设备。
作为另一个示例，CDN节点中的负载均衡设备，在接收到第二待刷新URL后，可以将该第二待刷新URL发送给同一节点中的辅助设备，由辅助设备过滤后，得到第一待刷新URL，再将该第一待刷新URL发送给同一节点中的所有存储设备。
S320，将至少一个待刷新URL与本地数据库存储的URL进行对比；本地数据库中存储有本地储存的源文件对应的URL。
其中，本地数据库存储的URL用于记录存储设备储存的源文件对应的URL，通常存储设备在回源后，会将源文件记录在本地，以便下次请求时无需再向源站进行获取。该本地数据库存储的URL可以为记录表或数据库的形式。通过本地数据库存储的URL可以过滤掉本地未存储的URL。
在一些实施例中，该本地数据库存储的URL可以根据本地回源日志确定，该回源日志包括回源URL。该回源日志可以结合前述图1所示的实施例中的描述进行参照理解，此处不再赘述。
S330，执行针对存在于数据库中的待刷新URL的刷新操作。
本步骤中，存储设备本地存储的待刷新URL对应的内容进行刷新。例如，将该待刷新URL及其对应的源文件删除。
通过本申请实施例，能够在存储设备中数据库存储的URL对至少一个待刷新URL进行过滤，可以实现只针对存储设备中实际存储的URL进行刷新，减少了无效刷新URL的操作，减轻了存储设备的工作量压力，是的CDN中URL刷新的执行效率得到了大幅度提高，进而提高了整体CDN中的URL刷新效率。
在实际应用中，本申请实施例提供的CDN中URL刷新方法的刷新效率可以和CDN节点内的存储器数量成正比。例如，需要刷新的URL总共有m条，一个CDN节点中共计有n台存储设备，如n台高速缓冲(CACHE)设备。n台存储器中的每台存储设备只会执行m/n条的刷新操作，如果一条URL数据只存在在一台存储设备中，则总共只需进行m次刷新操作。因此，对于CDN节点或CDN网络来说，URL刷新的执行效率得到了大幅度提高，在不影响整体的CDN中URL刷新进程，保证所有需要刷新的URL数据都得到了刷新的前提下，降低单台存储设备的刷新操作次数，降低了每台存储设备的工作量压力。
在一些实施例中，上述步骤S330具体可以通过如下步骤实现：
步骤1)，将至少一个待刷新URL中未存在于数据库中的URL删除，得到至少一个待刷新URL中存在于数据库中的URL；
步骤2)，针对至少一个待刷新URL中存在于数据库中的URL，执行刷新操作。
对于每一台CACHE设备，先经过基于本机中上述数据库存储的URL的清洗过滤，通将不在本机中储存的URL删除，剩下的即为本机中储存的且需要刷新的URL。通过该方法能够快速、精确的过滤出存储设备中实际需要刷新的URL。
在一些实施例中，第一刷新请求来自辅助设备，来自辅助设备的至少一个待刷新URL由辅助设备对来自负载均衡设备的至少一个待刷新URL进行过滤得到。
作为一个示例，负载均衡设备可以先将第一刷新请求发送至辅助设备，由辅助设备对该第一刷新请求进行过滤后，再将过滤后的第一刷新请求发送至存储设备。
当然，本申请实施例中，也可以是由负载均衡设备对第一刷新请求进行过滤，负载均衡设备可以直接向存储设备发送已经过滤后的第一刷新请求。
本申请实施例中，在存储器接收到第一刷新请求之前，辅助设备或负载均衡设备先对第一刷新请求中的待刷新URL进行过滤；在存储器接收到首次过滤后的第一刷新请求之后，存储器再利用本地数据库存储的URL，对该首次过滤后的第一刷新请求中的待刷新URL进行第二次过滤。通过对待刷新URL的这两次过滤过程，实现了针对待刷新URL的 双重过滤效果。
图4为本申请实施例提供的另一种CDN中URL刷新方法的流程示意图。该方法可以应用于CDN节点中的存储器中，如图3所示，该方法包括：
S410，接收来自负载均衡设备的第二刷新请求，第二刷新请求包括来自负载均衡设备的至少一个待刷新URL。
S420，基于本地数据库存储的标识，对来自负载均衡设备的至少一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；本地数据库用于记录同一节点内存储设备存储的源文件的标识。
S430，向同一节点内的存储设备发送第一刷新请求，第一刷新请求包括过滤后的至少一个待刷新URL；以便存储设备对过滤后的至少一个待刷新URL进行过滤，并针对经过过滤的过滤后的至少一个待刷新URL执行刷新操作。
通过本申请实施例，可以减少存储设备的影响。
在一些实施例中，由于CDN服务于海量客户，所以服务的域名也是海量的，每一个CDN节点通常不会服务所有的客户，而且CDN节点服务的域名有一定的区域性，所以CDN节点中的辅助设备可以基于服务域名进行过滤，将没在此CDN节点服务的域名，先过滤掉。基于此，该源文件的标识可以为域名，该方法还可以包括如下步骤：
步骤a)，接收同一节点内的存储设备发送的回源日志。该回源日志包括回源URL。
步骤b)，基于回源URL中的域名，确定本地数据库存储。
需要说明的是，URL的组成包括：协议、域名、路径以及参数。例如，URL为：https://www.baidu.com/a/b/c11.txt？tn＝48021271_15_hao_pg。
其中，“https”为协议，“www.baidu.com”为域名，“/a/b/c11.txt”为路径，“tn＝48021271_15_hao_pg”为参数。
图5为本申请提供的一种CDN中URL刷新装置的结构示意图。如图5所示，CDN中URL刷新装置500包括：
接收模块501，设置为接收第一刷新请求，第一刷新请求包括至少一个待刷新URL；
对比模块502，设置为将至少一个待刷新URL与本地数据库存储的URL进行对比；本地数据库中存储有本地储存的源文件对应的URL；
刷新模块503，设置为执行针对存在于数据库中的待刷新URL的刷新操作。
在一些实施例中，数据库中存储的URL根据本地回源日志确定，回源日志包括回源URL。
在一些实施例中，刷新模块503具体设置为：
将至少一个待刷新URL中未存在于数据库中的URL删除，得到至少一个待刷新URL中存在于数据库中的URL；
针对至少一个待刷新URL中存在于数据库中的URL，执行刷新操作。
本申请实施例提供的CDN中URL刷新装置，与上述图3所示的实施例提供的CDN中URL刷新方法具有相同的技术特征，所以也能解决相同的技术问题，达到相同的技术效果。
图6为本申请提供的另一种CDN中URL刷新装置的结构示意图。如图6所示，CDN 中URL刷新装置600包括：
接收模块601，设置为接收来自负载均衡设备的第二刷新请求，第二刷新请求包括来自负载均衡设备的至少一个待刷新URL；
过滤模块602，设置为基于本地数据库存储的标识，对来自负载均衡设备的至少一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；本地数据库用于记录同一节点内存储设备存储的源文件的标识；
发送模块603，设置为向同一节点内的存储设备发送第一刷新请求，第一刷新请求包括过滤后的至少一个待刷新URL；以便存储设备对过滤后的至少一个待刷新URL进行过滤，并针对经过过滤的过滤后的至少一个待刷新URL执行刷新操作。
在一些实施例中，源文件的标识为域名，该装置600还包括：
接收模块601还设置为，接收同一节点内的存储设备发送的回源日志，回源日志包括回源URL；
确定模块，设置为基于回源URL中的域名，确定本地数据库存储。
本申请实施例提供的CDN中URL刷新装置，与上述图4所示的实施例提供的CDN中URL刷新方法具有相同的技术特征，所以也能解决相同的技术问题，达到相同的技术效果。
如图7所示，本申请实施例提供的一种存储设备700，包括：处理器701、存储器702和总线，所述存储器702存储有所述处理器701可执行的机器可读指令，当电子设备运行时，所述处理器701与所述存储器702之间通过总线通信，所述处理器701执行所述机器可读指令，以执行如上述CDN中URL刷新方法的步骤。
具体地，上述存储器702和处理器701能够为通用的存储器和处理器，这里不做具体限定，当处理器701运行存储器702存储的计算机程序时，能够执行上述CDN中URL刷新方法。
如图8所示，本申请实施例提供的一种辅助设备800，包括：处理器801、存储器802和总线，所述存储器802存储有所述处理器801可执行的机器可读指令，当电子设备运行时，所述处理器801与所述存储器802之间通过总线通信，所述处理器801执行所述机器可读指令，以执行如上述CDN中URL刷新方法的步骤。
具体地，上述存储器802和处理器801能够为通用的存储器和处理器，这里不做具体限定，当处理器801运行存储器802存储的计算机程序时，能够执行上述CDN中URL刷新方法。
示例性的，存储设备或辅助设备还可以包括：通信接口，处理器、通信接口和存储器通过总线连接；处理器用于执行存储器中存储的可执行模块，例如计算机程序。
其中，存储器可能包含高速随机存取存储器(RAM，Random Access Memory)，也可能还包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。通过至少一个通信接口(可以是有线或者无线)实现该系统网元与至少一个其他网元之间的通信连接，可以使用互联网，广域网，本地网，城域网等。
总线可以是ISA总线、PCI总线或EISA总线等。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示，图7或图8中仅用一个双向箭头表示，但并不表示仅有一 根总线或一种类型的总线。
其中，存储器用于存储程序，所述处理器在接收到执行指令后，执行所述程序，前述本申请任一实施例揭示的过程定义的装置所执行的方法可以应用于处理器中，或者由处理器实现。
处理器可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，简称CPU)、网络处理器(Network Processor，简称NP)等；还可以是数字信号处理器(Digital Signal Processing，简称DSP)、专用集成电路(Application Specific Integrated Circuit，简称ASIC)、现成可编程门阵列(Field-Programmable Gate Array，简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。
对应于上述CDN中URL刷新方法，本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有机器可运行指令，所述计算机可运行指令在被处理器调用和运行时，所述计算机可运行指令促使所述处理器运行上述CDN中URL刷新方法的步骤。
本申请实施例所提供的CDN中URL刷新装置可以为设备上的特定硬件或者安装于设备上的软件或固件等。本申请实施例所提供的装置，其实现原理及产生的技术效果和前述方法实施例相同，为简要描述，装置实施例部分未提及之处，可参考前述方法实施例中相应内容。所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，前述描述的系统、装置和单元的具体工作过程，均可以参考上述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的实施例中，应该理解到，所揭露装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请提供的实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机 软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述移动控制方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释，此外，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，也可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，附图中的流程图和框图显示了根据本申请的多个实施例的装置、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分，所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现方式中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
最后应说明的是：以上所述实施例，仅为本申请的具体实施方式，用以说明本申请的技术方案，而非对其限制，本申请的保护范围并不局限于此，尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本申请实施例技术方案的范围。都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。
工业实用性
基于本公开实施例提供的CDN中URL刷新方法、装置以及电子设备，能够利用存储设备中数据库存储的URL，对至少一个待刷新URL进行过滤，可以实现只针对存储设备中实际存储的URL进行刷新，减少了无效刷新URL的操作，减轻了存储设备的工作量压力，使得CDN中URL刷新的执行效率得到了大幅度提高，进而提高了整体CDN中的URL刷新效率。

Claims (14)

  1. 一种CDN中URL刷新方法，包括：
    接收第一刷新请求，所述第一刷新请求包括至少一个待刷新URL；
    将所述至少一个待刷新URL与本地数据库存储的URL进行对比；所述本地数据库中存储有本地储存的源文件对应的URL；
    执行针对存在于所述数据库中的待刷新URL的刷新操作。
  2. 根据权利要求1所述的方法，所述数据库中存储的所述URL根据本地回源日志确定，所述回源日志包括回源URL。
  3. 根据权利要求1所述的方法，执行针对存在于所述数据库中的待刷新URL的刷新操作的步骤，包括：
    将所述至少一个待刷新URL中未存在于所述数据库中的URL删除，得到所述至少一个待刷新URL中存在于所述数据库中的URL；
    针对所述至少一个待刷新URL中存在于所述数据库中的URL，执行刷新操作。
  4. 根据权利要求1所述的方法，所述第一刷新请求来自辅助设备，来自所述辅助设备的至少一个待刷新URL由所述辅助设备对来自负载均衡设备的至少一个待刷新URL进行过滤得到。
  5. 一种CDN中URL刷新方法，包括：
    接收来自负载均衡设备的第二刷新请求，所述第二刷新请求包括来自负载均衡设备的至少一个待刷新URL；
    基于本地数据库存储的标识，对所述来自负载均衡设备的至少一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；所述本地数据库用于记录同一节点内存储设备存储的源文件的标识；
    向所述同一节点内的存储设备发送第一刷新请求，所述第一刷新请求包括所述过滤后的至少一个待刷新URL；以便所述存储设备对所述过滤后的至少一个待刷新URL进行过滤，并针对经过过滤的过滤后的至少一个待刷新URL执行刷新操作。
  6. 根据权利要求5所述的方法，所述源文件的标识为域名，所述方法还包括：
    接收同一节点内的存储设备发送的回源日志，所述回源日志包括回源URL；
    基于所述回源URL中的域名，确定所述本地数据库存储。
  7. 一种CDN中URL刷新装置，包括：
    接收模块，设置为接收第一刷新请求，所述第一刷新请求包括至少一个待刷新URL；
    对比模块，设置为将所述至少一个待刷新URL与本地数据库存储的URL进行对比；所述本地数据库中存储有本地储存的源文件对应的URL；
    刷新模块，设置为执行针对存在于所述数据库中的待刷新URL的刷新操作。
  8. 一种CDN中URL刷新装置，包括：
    接收模块，设置为接收来自负载均衡设备的第二刷新请求，所述第二刷新请求包括来自负载均衡设备的至少一个待刷新URL；
    过滤模块，设置为基于本地数据库存储的标识，对所述来自负载均衡设备的至少 一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；所述本地数据库用于记录同一节点内存储设备存储的源文件的标识；
    发送模块，设置为向所述同一节点内的存储设备发送第一刷新请求，所述第一刷新请求包括所述过滤后的至少一个待刷新URL；以便所述存储设备对所述过滤后的至少一个待刷新URL进行过滤，并针对经过过滤的过滤后的至少一个待刷新URL执行刷新操作。
  9. 一种存储设备，包括：存储器、处理器，所述存储器中存储有可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现上述权利要求1至4任一项所述的方法的步骤。
  10. 一种辅助设备，包括：存储器、处理器，所述存储器中存储有可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现上述权利要求5或6所述的方法的步骤。
  11. 一种CDN节点，包括：
    存储设备，设置为接收第一刷新请求，所述第一刷新请求包括至少一个待刷新URL；将所述至少一个待刷新URL与本地数据库存储的URL进行对比；所述本地数据库中存储有本地储存的源文件对应的URL；执行针对存在于所述数据库中的待刷新URL的刷新操作。
  12. 根据权利要求11所述的CDN节点，还包括：
    负载均衡设备，设置为向所述存储设备发送所述第一刷新请求。
  13. 根据权利要求11所述的CDN节点，还包括：
    辅助设备，设置为接收来自负载均衡设备的第二刷新请求，所述第二刷新请求包括来自负载均衡设备的至少一个待刷新URL；基于本地数据库存储的标识，对所述来自负载均衡设备的至少一个待刷新URL进行过滤，得到过滤后的至少一个待刷新URL；所述本地数据库用于记录同一节点内存储设备存储的源文件的标识；向所述同一节点内的存储设备发送第一刷新请求，所述第一刷新请求包括所述过滤后的至少一个待刷新URL。
  14. 一种计算机可读存储介质，所述计算机可读存储介质存储有机器可运行指令，所述计算机可运行指令在被处理器调用和运行时，所述计算机可运行指令促使所述处理器运行所述权利要求1至6任一项所述的方法。
PCT/CN2020/129230 2019-11-27 2020-11-17 Cdn中url刷新方法、装置、设备以及cdn节点 WO2021104100A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/778,102 US20230022141A1 (en) 2019-11-27 2020-11-17 Url refreshing method, apparatus and device in cdn, and cdn node
JP2022551426A JP7392168B2 (ja) 2019-11-27 2020-11-17 Ｃｄｎにおけるｕｒｌリフレッシュ方法、装置、機器及びｃｄｎノード

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911197702.2A CN112861031B (zh) 2019-11-27 2019-11-27 Cdn中url刷新方法、装置、设备以及cdn节点
CN201911197702.2 2019-11-27

Publications (1)

Publication Number Publication Date
WO2021104100A1 true WO2021104100A1 (zh) 2021-06-03

Family

ID=75996003

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/129230 WO2021104100A1 (zh) 2019-11-27 2020-11-17 Cdn中url刷新方法、装置、设备以及cdn节点

Country Status (4)

Country Link
US (1) US20230022141A1 (zh)
JP (1) JP7392168B2 (zh)
CN (1) CN112861031B (zh)
WO (1) WO2021104100A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117194827B (zh) * 2023-11-08 2024-02-27 联通在线信息科技有限公司 Cdn目录刷新的方法及电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1996886A (zh) * 2006-12-12 2007-07-11 中兴通讯股份有限公司 Iptv系统冗余数据的清除方法
CN103678337A (zh) * 2012-09-06 2014-03-26 华为技术有限公司 数据清除方法、装置及系统
US20140298445A1 (en) * 2011-12-31 2014-10-02 Huawei Technologies Co., Ltd. Method and Apparatus for Filtering URL
CN110020272A (zh) * 2017-08-14 2019-07-16 中国电信股份有限公司 缓存方法、装置以及计算机存储介质

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU7233700A (en) * 1999-12-23 2001-06-28 Access Devices Limited TV receiving and internet browsing apparatus
US6591266B1 (en) * 2000-07-14 2003-07-08 Nec Corporation System and method for intelligent caching and refresh of dynamically generated and static web content
US7707265B2 (en) * 2004-05-15 2010-04-27 International Business Machines Corporation System, method, and service for interactively presenting a summary of a web site
US8504775B2 (en) * 2007-03-12 2013-08-06 Citrix Systems, Inc Systems and methods of prefreshening cached objects based on user's current web page
US8688837B1 (en) * 2009-03-27 2014-04-01 Amazon Technologies, Inc. Dynamically translating resource identifiers for request routing using popularity information
US20110208850A1 (en) * 2010-02-25 2011-08-25 At&T Intellectual Property I, L.P. Systems for and methods of web privacy protection
US9723047B2 (en) * 2011-12-29 2017-08-01 Koninklijke Kpn N.V. Network-initiated content streaming control
US9141669B2 (en) * 2013-01-22 2015-09-22 Go Daddy Operating Company, LLC Configuring an origin server content delivery using a pulled data list
US20150052584A1 (en) * 2013-08-13 2015-02-19 News UK & Ireland Limited Access Control System
US20160110466A1 (en) * 2014-10-16 2016-04-21 Palo Alto Research Center Incorporated System and method for ranking named-data networking objects in a cache
US9858301B1 (en) * 2015-01-20 2018-01-02 Amazon Technologies, Inc. Selective flushing of a database journal for an asymmetrically-encrypted database
CN106202112B (zh) * 2015-05-06 2019-07-02 阿里巴巴集团控股有限公司 缓存目录刷新方法和装置
CN105847395A (zh) * 2016-04-25 2016-08-10 乐视控股（北京）有限公司 一种缓存文件处理方法及设备
CN109344353B (zh) * 2018-09-12 2021-10-08 福建天泉教育科技有限公司 一种可配置化的本地缓存刷新方法及终端
WO2021007752A1 (zh) * 2019-07-15 2021-01-21 华为技术有限公司 内容分发网络中的回源方法及相关装置
US11128732B1 (en) * 2020-08-04 2021-09-21 Akamai Technologies, Inc. Admission policies for queued website visitors
US20220353081A1 (en) * 2021-04-23 2022-11-03 Meta Platforms, Inc. User authentication techniques across applications on a user device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1996886A (zh) * 2006-12-12 2007-07-11 中兴通讯股份有限公司 Iptv系统冗余数据的清除方法
US20140298445A1 (en) * 2011-12-31 2014-10-02 Huawei Technologies Co., Ltd. Method and Apparatus for Filtering URL
CN103678337A (zh) * 2012-09-06 2014-03-26 华为技术有限公司 数据清除方法、装置及系统
CN110020272A (zh) * 2017-08-14 2019-07-16 中国电信股份有限公司 缓存方法、装置以及计算机存储介质

Also Published As

Publication number Publication date
JP2023500418A (ja) 2023-01-05
CN112861031B (zh) 2024-04-02
CN112861031A (zh) 2021-05-28
US20230022141A1 (en) 2023-01-26
JP7392168B2 (ja) 2023-12-05

Similar Documents

Publication Publication Date Title
CA2881804C (en) High performance real-time relational database system and methods for using same
CN109947668B (zh) 存储数据的方法和装置
US11537572B2 (en) Multidimensional partition of data to calculate aggregation at scale
US20200356277A1 (en) De-duplication of client-side data cache for virtual disks
WO2021077745A1 (zh) 一种分布式存储系统的数据读写方法
WO2019041738A1 (zh) 客户资源获取方法、装置、终端设备及存储介质
CN109167840B (zh) 一种任务推送方法、节点自治服务器及边缘缓存服务器
WO2017092582A1 (zh) 一种数据处理方法和装置
US20120072575A1 (en) Methods and computer program products for aggregating network application performance metrics by process pool
US20220286373A1 (en) Scalable real time metrics management
WO2021073510A1 (zh) 一种数据库的统计方法及装置
CN103207882A (zh) 店铺访问数据处理方法及系统
TW201324211A (zh) 即時資料的獲取方法、裝置及系統
WO2021104100A1 (zh) Cdn中url刷新方法、装置、设备以及cdn节点
WO2017157111A1 (zh) 防止内存数据丢失的的方法、装置和系统
WO2021082936A1 (zh) 一种网页访客数量统计方法及装置
CN113468226A (zh) 一种业务处理方法、装置、电子设备和存储介质
CN108337100B (zh) 一种云平台监测的方法和装置
CN111767481B (zh) 访问处理方法、装置、设备和存储介质
US8717917B1 (en) System and method for managing transaction data in a mobile communication network using selective sampling
CN112860720B (zh) 一种存储容量的更新方法以及装置
US20190228078A1 (en) Methods for automated artifact storage management and devices thereof
US20130282654A1 (en) Query engine communication
CN113411364B (zh) 资源获取方法、装置及服务器
CN113746920A (zh) 一种数据转发方法、装置、电子设备及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20892306

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022551426

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20892306

Country of ref document: EP

Kind code of ref document: A1