CN111131498B

CN111131498B - URL information updating method, cache server, equipment and storage medium

Info

Publication number: CN111131498B
Application number: CN201911411603.XA
Authority: CN
Inventors: 卢畅; 苗辉
Original assignee: Guizhou Baishancloud Technology Co Ltd
Current assignee: Guizhou Baishancloud Technology Co Ltd
Priority date: 2018-03-23
Filing date: 2018-03-23
Publication date: 2023-04-21
Anticipated expiration: 2038-03-23
Also published as: CN110300140A; CN111131498A; CN110300140B

Abstract

The invention discloses a URL information updating method, a cache server, equipment and a storage medium, wherein the method comprises the following steps: receiving an instruction for deleting cache information related to a URL to be deleted, which is sent by a refreshing client of a network node; deleting the cache information related to the URL to be deleted; after a preset time period after receiving an instruction for deleting the cache information related to the URL to be deleted sent by the refreshing client, receiving an instruction for deleting the cache information related to the URL with frequent access sent by the refreshing client; the cached information associated with the frequently accessed URL is deleted. By adopting the scheme of the invention, the problem that the edge node returns to the father node to take the old file can be avoided, and after the refreshing client is on line again, the task which is not executed in the fault period can be continuously completed.

Description

URL information updating method, cache server, equipment and storage medium

The application is a divisional application of China patent application with the name of 'method for updating content in cloud distribution network, refreshing client and network node' which is submitted by China national intellectual property office patent office, application number 201810244982.7 and is filed on 3 months and 23 days in 2018.

Technical Field

The present invention relates to the field of data transmission, and in particular, to a URL information updating method, a cache server, a device, and a storage medium.

Background

In current network environments, content such as pictures, web pages, music, video, etc., typically uses a content delivery network (Content Delivery Network, simply CDN) with caching nodes distributed throughout the world to provide faster content access services to end users. The method is characterized in that a new network architecture is added in the existing Internet, and the content of the website is released to the network 'edge' closest to the user, so that the user can obtain the required content nearby without requesting the content by a source server, and the corresponding speed of the user for accessing the website or downloading files is improved. However, when the provider of the target service upgrades and changes the target service in the service server, the target service stored in the service server is updated, and at this time, the cache node server cached with the target service needs to be updated, so as to ensure that the content resource of the target service acquired by the user terminal through the cache node server is an updated content resource.

When the website updates the resources, all the nodes for caching the partial content also need to be updated (including deletion) in time, otherwise, the user can continue to obtain the old content, for example, once the content published by a news website is wrong or tampered by a hacker, in order to avoid more serious situations, the whole network content needs to be updated in time, and as the CDN often has a large number of nodes distributed in different regions, it is a great challenge to update the cached content on all the nodes in time.

In the traditional CDN cache mechanism, full-network content update is implemented based on a form of message queues, when content is updated, thousands of devices are managed by a central server, the central needs to confirm the states of all the devices and track the push condition of the devices, and when the device update fails, retries are needed. Because the load of the central machine is high, the distance from other nodes is far, and the network is long, when the updating fails, the system can realize the updating of the content in the whole network by means of equipment tracking and failed retransmission. This approach has the following drawbacks:

1. the modes of push messages are different: one is that a father node corresponds a plurality of edge nodes, can carry out the propelling movement of father node earlier in this way, carry out the second grade propelling movement by father node again, propelling movement to edge node. Another is that the order of the machines that the messages arrive on the whole network is not controllable, which results in pushing the content of the edge node first and then the parent node. For the more frequently accessed uniform resource locators (Uniform Resource Locator, URL), after pushing the edge node, the edge node returns to the parent to take the old file, and since the p2p transmission is random, the parent may be pushed first, and the edge may be pushed first, which has a certain probability.

2. After the faulty machine is on line again, the push task submitted before cannot be recorded, so that the URL pushed during the fault cannot be normally pushed.

3. The message queue queues the task only by the time sequence in which the task starts, and the urgency of the task update does not coincide with the time sequence in which the task starts, which results in unfairness of the task update.

4. Conventional use of p2p uses an efficient udp protocol that has a limit on the size of content to be transmitted each time, and when a URL length greater than this limit is encountered, efficient transmission is not possible.

5. Once the network is out of condition, the traditional content updating mode can possibly appear that the server can not acquire the update in the effective time of the task in view of the actual condition of the current network. At this time, the center will retransmit after receiving the tracking feedback, and the final efficiency is greatly compromised. The content update may involve a huge number of URLs at the same time, the time for processing a URL in a message queue is generally in the tens of milliseconds, and a complete update of the whole network content requires several minutes or even tens of minutes, which obviously cannot meet the needs of enterprises.

Accordingly, there is a need for a method of content update in a cloud distribution network that can solve the above-described problems.

Disclosure of Invention

In order to solve the problem in the content updating in the prior art, a method, a refreshing client and a network node for content updating in a cloud distribution network are provided.

According to one aspect of the present invention, there is provided a method for content update in a cloud distribution network, the method comprising:

the refreshing client of the network node acquires URL information to be deleted and instructs a cache server on the network node to delete the cache information related to the URL to be deleted;

and the refreshing client acquires the information of the URL with frequent access, and after a set time period after the cache information related to the URL to be deleted is indicated to be deleted, the cache server is indicated to delete the cache information related to the URL with frequent access again.

Wherein, obtaining the frequent URL information includes:

the refresh client obtains frequent URL information based on the short return code 200 received from the cache server.

Wherein the method further comprises:

when the refreshing client is restarted, recording the restarting moment, and acquiring the moment when the cache information related to the URL to be deleted is successfully deleted before restarting;

and acquiring the lost URL information to be deleted, which is sent to the network node, between the last time of successfully deleting the cache information related to the URL to be deleted and the restarting time, and indicating the cache server to delete the cache information related to the lost URL to be deleted.

The obtaining the time of last successfully deleting the cache information related to the URL to be deleted before restarting comprises the following steps:

the time at which the Squid return code 200 or 404 was last received before the restart is obtained.

Wherein deleting the cache information related to the URL to be deleted further includes:

and acquiring the priority of the URL to be deleted, and indicating the cache server to delete the cache information related to the URL to be deleted according to the order of the priority from high to low.

The refreshing client side obtaining the URL information to be deleted further comprises:

judging whether the acquired URL information to be deleted contains split information or not, wherein the split information comprises a unique identifier for indicating a corresponding URL and a serial number for indicating the position of the URL information to be deleted in the corresponding URL, and if so, merging the URL information to be deleted with the same unique identifier in the split information to determine the URL information to be deleted.

According to another aspect of the present invention, there is provided a refresh client for content update in a cloud distribution network, the refresh client comprising:

the URL acquisition module is used for acquiring URL information to be deleted and acquiring frequent URL information;

and the deleting module is used for indicating the cache server on the network node to delete the cache information related to the URL to be deleted, and after a set time period after the cache information related to the URL to be deleted is indicated to be deleted, indicating the cache server to delete the cache information related to the URL with frequent access again.

The URL obtaining module is further configured to obtain URL information with frequent access based on the short return code 200 received from the cache server.

Wherein the refresh client further comprises: the moment acquisition module is used for recording the restarting moment after the refreshing client is restarted, and acquiring the moment of successfully deleting the cache information related to the URL to be deleted before restarting;

the URL obtaining module is further configured to obtain, between the last time of successfully deleting the cache information related to the URL to be deleted and the restart time, lost URL information to be deleted sent to the network node where the refresh client is located,

the deleting module is further configured to instruct the cache server to delete cache information related to the lost URL to be deleted.

The time acquisition module is further configured to acquire a time when the service return code 200 or 400 is last received before the refresh client is restarted, and take the time as a time when the cache information related to the URL to be deleted is last successfully deleted.

Wherein the refresh client further comprises: the priority acquisition module is used for acquiring the priority of the URL to be deleted;

the deleting module is further configured to instruct the cache server to delete cache information related to the URL to be deleted according to the order of the priority from high to low.

The URL acquisition module is further configured to determine whether the acquired URL information to be deleted includes split information, where the split information includes a unique identifier indicating a corresponding URL and a serial number indicating a position of the URL information to be deleted in the corresponding URL, and if so, the URL information to be deleted having the same unique identifier in the split information is combined to determine the URL information to be deleted.

According to another aspect of the present invention, there is also provided a network node for content update in a cloud distribution network, the network node including the above-mentioned refresh client.

The method and the client for updating the content in the cloud distribution network have the following beneficial effects:

(1) When the edge node is pushed firstly and then the father node is pushed in the network, the problem that the edge node returns to the father node to take an old file can not occur;

(2) After the fault machine is on line again, normal pushing can be carried out on the pushing tasks which are not executed during the fault period;

(3) The system has the capability of considering priority management, and can process urgent tasks preferentially;

(4) The network transmission and response capacity of the system is greatly improved, and the distribution speed of files (particularly large files) is greatly improved;

(5) The pressure of the source server is reduced by means of a P2P mechanism, the original bandwidth is reduced, and the bandwidth cost is reduced;

(6) The whole downloading time is reduced, and the downloading stability is stronger.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention. In the drawings:

FIG. 1 is a flow chart of a method for content update in a cloud distribution network according to the present invention;

FIG. 2 is a schematic diagram of the interaction process between a refresh server and a cache server according to the present invention;

fig. 3 is a block diagram of a refresh client for content update in a cloud distribution network according to the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be arbitrarily combined with each other.

The invention provides a method for updating content in a cloud distribution network, as shown in fig. 1, the method comprises the following steps:

step 101, a refreshing client of a network node acquires URL information to be deleted, and instructs a cache server on the network node to delete cache information related to the URL to be deleted;

step 102, the refreshing client acquires the information of the URL with frequent access, and after a set period of time after the cache information related to the URL to be deleted is indicated to be deleted, the cache server is indicated to delete the cache information related to the URL with frequent access again.

Here, deleting the cache information related to the URL to be deleted on the network node means pushing the URL. And the refreshing client acquires the URL pushing task from the task center through the pushing task acquisition program, namely acquiring URL information to be deleted.

After the refreshing client instructs the cache server to delete the cache information related to the URL for about 2 seconds, the pushing of the URL by the whole network can be completed, namely, the related cache information of the URL is deleted on the network node of the whole network. After the whole-network push is completed, in order to ensure that the node does not retrieve the old file of the URL and collect URL information which is frequently accessed, the cache information is deleted again. After the set period of time for instructing to delete the cached information related to the URL to be deleted in step 101, that is, after the push of the whole network, the set period of time may be set to 5s, for example.

As shown in fig. 2, the refresh client obtains a URL list from the P2P client, sends an instruction to delete the cache to the short cache server, the short cache server performs the deletion operation and gives feedback, the network node performs different operations based on different feedback, for example, the feedback is abnormal, the alarm operation is performed, the feedback is successful, the URL is normally exited, and the feedback return code 200 is pushed again for the frequently accessed URL.

In step 102, obtaining the frequent URL information includes: the refresh client obtains frequent URL information based on the required return code 200 received from the cache server.

Each edge node is provided with an own quick cache server, if the quick cache server receives the deletion instruction of the refreshing client and deletes the information related to the URL, the URL related to the quick return code 200 is considered to be the frequent URL, and the cache server on the node is instructed to delete the related information of the URL which is frequently accessed again. Here, the receiving of the short return code 200 determines whether the corresponding URL is a frequent URL, and in actual operation, it may also be determined whether a URL is a frequent URL in other manners.

The method further comprises the steps of: when the refreshing client is restarted, recording the restarting moment, and acquiring the moment when the cache information related to the URL to be deleted is successfully deleted before restarting; and acquiring the lost URL information to be deleted, which is sent to the network node, between the last time of successfully deleting the cache information related to the URL to be deleted and the restarting time, and indicating the cache server to delete the cache information related to the lost URL to be deleted.

When the refreshing client is in fault shutdown, the URL pushing tasks issued from the center during the fault shutdown cannot be obtained, so that when the refreshing client is restarted, the URL pushing tasks lost during the fault shutdown need to be obtained, and the tasks are re-executed.

The obtaining the time of successfully deleting the cache information related to the URL to be deleted before restarting comprises the following steps: a time is obtained when the late receipt of the Squid return code is 200 or 404 before the restart.

A row of log records is generated in the push log by pushing each URL, and whether the URL is failed or successful is judged by level=error or level=info. If the push is successful, the Squid return code 200 or 404 is received, and if other return codes are received, the push is failed. Therefore, the time for successfully deleting the cache information related to the URL to be deleted before restarting can be obtained by searching the push log.

If the refresh client receives the Squid return code 200 or 404, it considers that the cache information related to the URL to be deleted is successfully deleted, so that the time when the Squid return code 200 or 404 is received last before restarting, that is, the time when the cache information related to the URL to be deleted is successfully deleted last before restarting. The time when the Squid return code 200 or 404 is received is recorded in the push log, and the push log records the push time and the state of the URL, so that when the refreshing client is restarted, the push log is checked.

In step 101, deleting the information related to the URL to be deleted further includes: and acquiring the priority of the URL to be deleted, and indicating the cache server to delete the cache information related to the URL to be deleted according to the order of the priority from high to low.

When pushing a URL, that is, deleting the cached information related to the URL cached on the node, the priority of the URL needs to be deleted. This is because different customers have different sensitivity to time, and some customers, such as e-commerce customers, have a high sensitivity to time, requiring that their URLs be pushed in a short time. This requires that the cached information associated with the URLs be deleted in order of priority from high to low based on the priority of the URLs.

In addition, to ensure the use experience of the time-sensitive client, two push task acquisition programs may be executed in parallel during actual operation. This ensures a use experience for time sensitive clients. Specifically, the refreshing client sets two pushing task acquisition programs, acquires URL pushing tasks from the center through the two pushing task acquisition programs, and allocates different execution speeds for the URL pushing tasks based on the acquired priority of the URL to be deleted.

In step 101, the refreshing client obtaining URL information to be deleted further includes: judging whether the acquired URL information to be deleted contains split information or not, wherein the split information comprises a unique identifier for indicating the corresponding URL and a serial number for indicating the position of the URL information to be deleted in the corresponding URL, and if so, merging the URL information to be deleted with the same unique identifier in the split information to determine the URL information to be deleted.

The protocol used for information transfer between network nodes, such as the efficient UDP protocol, has a limit on the size of the content to be sent each time, normally limits the URL length to 512 bytes, and if the URL information is longer than this limit, it is necessary to split the URL information. Where the URL information includes the URL itself and possibly some information related to the URL. Therefore, URL information whose length exceeds the limit is split at the time of transmission, and split information including a unique identifier indicating the URL and a serial number indicating the position of the portion in the entire URL information is added to each portion resulting from the split. For example, the following URLs are split with lengths exceeding the limit:

URL：http://test.test.com/labcd...2abcd...3abcd

the split portions are shown in the following table:

TABLE 1

Unique identifier	Sequence number	Split part
			ID2001	1	http://test.test.com/labcd...
ID2001	2	2abcd...
			ID2001	3	3abcd...

After receiving the split parts, the refreshing client merges the parts based on split information carried by the parts.

A method for a refresh client of an edge node to perform the content update according to a specific embodiment of the present invention is described in detail below, the method comprising the steps of:

step one, a refreshing client of an edge node acquires a URL to be deleted through real-time monitoring:http:// test.test.com/labcdis a piece of information of (a).

Step two, instruct the cache server on the edge node to delete the URL:http:// test.test.com/labcdrelated cache information.

And step three, the return code returned by the refreshing client side receiving the cache server Squid aiming at the deletion instruction is 200, the URL is put into a URL queue with frequent access, and meanwhile, the moment of receiving the return code 200 is recorded in a push log.

And step four, after the refreshing client instructs the cache server to delete the cache information related to the URL for 5 seconds, the refreshing client instructs the cache server to delete the cache information related to the URL.

And fifthly, after the refreshing client fails to restart, checking the push log to acquire the time when the Squid return code 200 is finally received before restarting.

And step six, the refreshing client acquires the URL information to be deleted lost from the moment when the required return code 200 is finally received before restarting to the restarting moment from the task issuing center, and instructs the cache server to delete the cache information related to the lost URLs to be deleted.

The invention also provides a refreshing client for content update in a cloud distribution network, as shown in fig. 3, the refreshing client comprises:

the URL obtaining module 301 is configured to obtain URL information to be deleted, and obtain URL information with frequent access;

and a deleting module 302, configured to instruct a cache server on the network node to delete the cache information related to the URL to be deleted, and instruct the cache server to delete the cache information related to the URL with frequent access after a set period of time after the cache information related to the URL to be deleted is instructed to be deleted.

The time acquisition module is further configured to acquire a time when the service return code 200 or 404 is last received before the refresh client is restarted, and take the time as a time when the cache information related to the URL to be deleted is last successfully deleted.

the deletion module 302 is further configured to instruct the cache server to delete the cache information related to the URL to be deleted in the order from high to low of the priority.

The URL obtaining module 301 is further configured to determine whether the obtained URL information to be deleted includes split information, where the split information includes a unique identifier indicating a corresponding URL and a serial number indicating a position of the URL information to be deleted in the corresponding URL, and if so, combine URL information to be deleted having the same unique identifier in the split information to determine URL information to be deleted.

The invention also provides a network node for content update in a cloud distribution network, wherein the network node comprises the refreshing client.

It should be noted that the method for updating content, the refreshing client and the network node in the application of the present invention are described in the cloud distribution network environment, and those skilled in the art should appreciate that the method, the refreshing client and the network node can be applied to other network environments as well.

The above description may be implemented alone or in various combinations and these modifications are within the scope of the present invention.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such article or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of additional identical elements in an article or apparatus that comprises the element.

The above embodiments are only for illustrating the technical scheme of the present invention, not for limiting the same, and the present invention is described in detail with reference to the preferred embodiments. It will be understood by those skilled in the art that various modifications and equivalent substitutions may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention, and the present invention is intended to be covered by the scope of the appended claims.

Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

Claims

1. The URL information updating method is applied to a cache server of the network node and is characterized by comprising the following steps:

receiving an instruction for deleting cache information related to the URL to be deleted, which is sent by a refreshing client of the network node;

deleting the cache information related to the URL to be deleted;

after a preset time period after receiving an instruction for deleting the cache information related to the URL to be deleted sent by the refreshing client, receiving an instruction for deleting the cache information related to the URL with frequent access sent by the refreshing client;

and deleting the cache information related to the URL with frequent access again, so that the cache server in the edge node obtains the updated file when the instruction is pushed to the edge node and then pushed to the father node.

2. The URL information updating method as claimed in claim 1, wherein said URL information updating method further comprises:

after deleting the cache information related to the URL to be deleted, giving feedback and sending the feedback to the refreshing client;

wherein the feedback includes anomalies, a Squid return code 404, and a Squid return code 200;

and when the feedback is the Squid return code 200, after a preset time period after receiving an instruction for deleting the cache information related to the URL to be deleted sent by the refreshing client, receiving an instruction for deleting the cache information related to the URL with frequent access sent by the refreshing client.

3. The URL information updating method as claimed in claim 1 or 2, wherein said method further comprises:

receiving an instruction which is sent by the refreshing client and used for deleting the cache information related to the lost URL to be deleted;

deleting the cache information related to the lost URL to be deleted;

the lost URL information to be deleted is URL information to be deleted received by the network node between the moment of last successful deletion of the cache information related to the URL to be deleted before restarting the refreshing client and the restarting moment of the refreshing client.

4. The URL information updating method as claimed in claim 3, wherein the time of last successful deletion of the cached information related to the URL to be deleted before the refresh client is restarted includes:

the time at which the service return code 200 or 404 sent by the cache server was last received before the refresh client was restarted.

5. The URL information updating method as claimed in claim 1, wherein deleting the cache information related to the URL to be deleted further comprises:

receiving an instruction which is sent by the refreshing client and used for deleting the cache information related to the URL to be deleted according to the order of priority from high to low;

and deleting the cache information related to the URL to be deleted according to the order of priority from high to low.

6. A cache server for URL information updating, wherein the cache server is configured to:

deleting the cache information related to the URL to be deleted;

7. The cache server for URL information updates as recited in claim 6, wherein said cache server is further configured to:

8. The cache server for URL information update as claimed in claim 6 or 7, wherein the cache server is further for:

deleting the cache information related to the lost URL to be deleted;

9. The cache server for URL information update as claimed in claim 8, wherein the time of last successful deletion of the cache information related to the URL to be deleted before the refresh client is restarted includes:

10. The cache server for URL information updates as recited in claim 6, wherein said cache server is further configured to:

11. A transmission apparatus, characterized in that the transmission apparatus comprises: a transceiver, a memory, a processor;

the transceiver is used for receiving and transmitting messages;

the memory is used for storing instructions and data;

the processor is configured to read the instructions and data stored in the memory to perform the URL information updating method as set forth in any one of claims 1 to 5.

12. A computer-readable storage medium having stored thereon a computer program, wherein the program when executed by a processor implements the URL information updating method according to any one of claims 1 to 5.