CN109995731B - Method and device for improving cache spitting flow, computing equipment and storage medium - Google Patents

Method and device for improving cache spitting flow, computing equipment and storage medium Download PDF

Info

Publication number
CN109995731B
CN109995731B CN201711491435.0A CN201711491435A CN109995731B CN 109995731 B CN109995731 B CN 109995731B CN 201711491435 A CN201711491435 A CN 201711491435A CN 109995731 B CN109995731 B CN 109995731B
Authority
CN
China
Prior art keywords
host
http
cache
hot spot
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711491435.0A
Other languages
Chinese (zh)
Other versions
CN109995731A (en
Inventor
滕野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
China Mobile Group Liaoning Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
China Mobile Group Liaoning Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, China Mobile Group Liaoning Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN201711491435.0A priority Critical patent/CN109995731B/en
Publication of CN109995731A publication Critical patent/CN109995731A/en
Application granted granted Critical
Publication of CN109995731B publication Critical patent/CN109995731B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00Network arrangements, protocols or services for addressing or naming
    • H04L61/45Network directories; Name-to-address mapping
    • H04L61/4505Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols
    • H04L61/4511Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols using domain name system [DNS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/02Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/1014Server selection for load balancing based on the content of a request
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/568Storing data temporarily at an intermediate stage, e.g. caching
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22Parsing or analysis of headers

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the invention discloses a method, a device, a computing device and a storage medium for improving cache spitting flow, wherein the method comprises the following steps: generating a full-user HTTP ticket based on the HTTP message headers, wherein the full-user HTTP ticket comprises content lengths extracted from the HTTP message headers and a host; based on the total user HTTP ticket, performing summation operation on the content lengths of the same host to obtain the total content length of each host; sorting the total content length by taking the host as granularity, and determining a hot spot flow host; and carrying out cache acceleration processing on the hot spot flow host. According to the scheme, the provincial large-flow domain name can be captured, and the accelerated service of the hotspot service is realized.

Description

Method and device for improving cache spitting flow, computing equipment and storage medium
Technical Field
The invention relates to the technical field of Internet HTTP (hyper text transport protocol), in particular to a method and a device for improving cache spitting flow, computing equipment and a storage medium.
Background
With the development of the internet in recent years, the number of internet users, the application types of the internet, the network bandwidth and the like all show explosive growth, and have great influence on the society and the lives of people. In the next 5 years, the internet traffic in China increases by more than 40% each year, and in contrast, although broadband construction also increases, the service traffic has increased far beyond the speed of bandwidth increase, and the existing bandwidth and the visible increase in the future are far from meeting the phagocytosis of the network bandwidth by the application of large-traffic continuous occupied bandwidth.
In order to respond to the challenge of rapid increase of Internet traffic, mainstream operators around the world start to build a CDN/Cache network, and the purpose of the CDN/Cache network is to add a new network architecture to the existing Internet, and publish the content of a website to the "edge" of the network closest to a user, so that the user can obtain the required content nearby, thereby solving the problem of congestion of the Internet network, improving the response speed of the user accessing the website, and solving the problem of slow response speed of the user accessing the website due to small network bandwidth, large user access amount, uneven website distribution, and the like.
In the Cache network flow guiding, Local DNS forwards domain name query of a user terminal to a Cache network in a DNS forwarding mode, and the Cache network provides service for users and improves response speed. For large network operators such as China Mobile, the types of access services of users are rich, and Cache acceleration needs to be performed on hot spot services. The existing Cache operation method mainly utilizes DNS log data to inquire domain names with a large DNS resolution amount of users, adds the domain names into a Cache network, and provides acceleration service for the service domain names. DNS log data can only provide times for analyzing different domain names by a user DNS, and the method can not accurately capture large-flow domain name services of the user and provide acceleration service for hot spot services of the user when a Cache network is operated.
In the hot spot confirmation, the DNS log data counts the DNS process of each user, that is, the source IP of the initiating user requests a domain name and a resolved address, and the statistics of these data can only calculate the DNS resolution times of each domain name. After initiating the DNS resolution to obtain the server IP, the user interacts with the server through the HTTP protocol to obtain the access content, so the DNS resolution times cannot represent that the service is a hotspot service.
Disclosure of Invention
The embodiment of the invention provides a method, a device, computing equipment and a storage medium for improving cache spitting flow, aiming at the problem that DNS log data only can provide times for analyzing different domain names by a user DNS and cannot accurately capture large-flow domain names of the user.
In a first aspect, an embodiment of the present invention provides a method for improving a cache spitting flow, where the method includes:
generating a full-user HTTP ticket based on the HTTP message headers, wherein the full-user HTTP ticket comprises content lengths and host identification information extracted from the HTTP message headers;
based on the HTTP ticket of the whole user, the content lengths of the same host are summed to obtain the total content length of each host;
sorting the total content length by taking the host as granularity, and determining a hot spot flow host; and
and carrying out cache acceleration processing on the hot spot flow host.
In a second aspect, an embodiment of the present invention provides an apparatus for increasing a cache spitting flow, where the apparatus includes: the device comprises a generating module, a calculating module, a determining module and a processing module.
The generation module can generate a full-user HTTP ticket based on the HTTP message headers, wherein the full-user HTTP ticket comprises the content length extracted from each HTTP message header and the host.
The calculation module can sum the content lengths of the same host based on the full amount of user HTTP tickets to obtain the total content length of each host.
The determining module may rank the total content length with the host as a granularity to determine the hotspot traffic host.
The processing module can perform cache acceleration processing on the hot spot traffic host.
In a third aspect, an embodiment of the present invention provides a computing device, including: at least one processor, at least one memory, and computer program instructions stored in the memory, which when executed by the processor, implement the method of the first aspect of the embodiments described above.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which computer program instructions are stored, which, when executed by a processor, implement the method of the first aspect in the foregoing embodiments.
According to the method, the device, the computing equipment and the storage medium for improving the cache spitting-out flow, provided by the embodiment of the invention, the large-flow domain name can be accurately captured by analyzing the HTTP message, and the cache of the large-flow business can be accurately taken out for acceleration service by combining with the DNS log and the cacheable analysis.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for increasing cache spitting traffic according to an embodiment of the present invention.
Fig. 2 is a schematic diagram illustrating a full-user HTTP ticket according to an embodiment of the present invention.
Fig. 3 shows a schematic diagram of traffic ordering for domain name dimension according to an embodiment of the invention.
Fig. 4 shows a schematic flow diagram of a provincial large flow domain name cache according to an embodiment of the invention.
Fig. 5 is a schematic diagram illustrating an effect of improving cache spitting according to an embodiment of the present invention.
Fig. 6 is a schematic block diagram of an apparatus for increasing a cache spitting traffic according to an embodiment of the present invention.
Fig. 7 is a schematic diagram illustrating a hardware structure of a computing device according to an embodiment of the present invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The proposal utilizes the field in the HTTP protocol to capture the domain name with large flow, and combines the DNS log and the cacheable analysis to perform cache acceleration processing on the host in the cache white list. Fig. 1 is a schematic flow chart of a method for increasing cache spitting traffic according to an embodiment of the present invention.
As shown in fig. 1, in step S100, a full-user HTTP ticket may be generated based on HTTP headers, where the full-user HTTP ticket includes content lengths and host identification information extracted from each HTTP header.
The content length is the length of the content in the message except the header. The HTTP message comprises an uplink message, namely a request message, and a downlink message, namely a response message, the HTTP request message comprises a request method, a request URL, an HTTP protocol and version, a message header and a message body, and the HTTP response message comprises a message protocol and version, a state code and state description, a response header and a response body. The HTTP message header comprises Cache-Control, Content-Type, Content-Length, Content-Charset, Port, Host and the like, and field information such as Content Length-Length, Host and the like can be extracted from the HTTP message header to generate a full-user HTTP ticket, wherein the HTTP ticket is recorded information of HTTP access requests of all users in a certain area. Wherein, the host can be IP, port number or domain name.
According to an embodiment of the present invention, the HTTP message of the user can be obtained by analyzing the deep analysis message data.
The DPI deep analysis message can complete analysis, protocol identification and content identification of the user data message by identifying and analyzing uplink and downlink data packets of the user, and obtain valuable information such as a destination URL (uniform resource locator). And may match control rules based on identified traffic classes or parsed keywords.
According to an embodiment of the invention, the user HTTP message may be identified by the probe device. For example, message recognition may be performed using a probe device TMA (tower mounted amplifier) that is in series on the metro network egress link.
According to an embodiment of the invention, the message header corresponding to the HTTP message successfully analyzed can be imported into a ticket acquisition system to generate a full amount of user HTTP tickets, and the HTTP message unsuccessfully analyzed can be attributed to junk data.
The call ticket collection system can be realized by using C # language and FTP downloading, and can collect port data in real time.
According to an embodiment of the invention, the identified HTTP message can be transmitted to the call ticket collection server through the port mirror image, so that the convergence of the request message and the response message is realized.
The port mirror image is to copy a message of a designated port (source port), vlan (source vlan) or CPU to another port (destination port), the destination port can be connected to the data monitoring device, and the data monitoring device can analyze the message copied to the destination port, so that the request message and the response message can be gathered. Network monitoring and troubleshooting can also be performed, for example, if the B port is a mirror image of the a port, the network data packet of the a port can be acquired on the B port through some common tools, and whether abnormal network traffic exists on the a port can be analyzed through the acquired data packet.
According to an embodiment of the invention, a data table taking the data stream ID of each user HTTP access process as a row and the field extracted from the HTTP message header as a column can be used as a full-user HTTP ticket. The request and response in the HTTP access process, and the data read/write operation on the network can be completed using data streams.
The HTTP ticket is a session level detail record of a signaling process and a service transmission process generated after processing based on the internet full data, and includes all internet access information of a user, which may include quintuple information, HTTP message information, and the like.
Fig. 2 is a schematic diagram illustrating a full-user HTTP ticket according to an embodiment of the present invention. As shown in fig. 2, fields such as a website name, a Host, a PORT, and a total number of URLs may be extracted, data streams generated in one HTTP process initiated by the same user may be named as ID 1, data streams for all users in a certain area are listed in sequence, and a data table with data stream IDs as rows and extracted HTTP fields as columns is generated.
As shown in fig. 1, in step S200, the content lengths of the same host may be summed based on the HTTP tickets of all users, so as to obtain the total content length of each host.
For example, HTTP tickets can be uploaded to a data processing system in an FTP manner with day as granularity, and the total content length of each host can be obtained by performing summation operation on content-lengths of different data stream IDs belonging to the same host field, and can be used as a basis for traffic sequencing.
In step S300, the total content lengths may be sorted by taking the host as a granularity, and the hot spot traffic host is determined. Fig. 3 shows a schematic diagram of traffic ordering for domain name dimension according to an embodiment of the invention.
According to an embodiment of the invention, the total content length can be arranged in a descending order, the first N hosts are used as hot spot flow hosts, and N is the number of cacheable hosts estimated by combining the network service capability of the cache.
For example, according to the network service capability, N may be a ratio of the number of system design connections to the average number of connections per domain name, and topN hosts are selected as the hotspot traffic hosts.
In step S400, cache acceleration processing may be performed on the hotspot traffic host.
According to an embodiment of the present invention, the present province rate of each hot traffic host may be determined in combination with the DNS log, the hot traffic host whose present province rate is smaller than the predetermined threshold is added to the cache white list, the hot traffic host whose present province rate is not smaller than the predetermined threshold is added to the cache black list, and cache acceleration processing is performed on the hosts in the cache white list.
The provincial rate may be a ratio of the number of times that the user DNS resolves to the province to the total number of times that the user DNS resolves.
For example, the attribution of the hot spot traffic host is judged, and if the province rate > is 50%, the host attribution is province; if the province rate is less than 50%, the host belongs to the province. Because the Cache cannot accelerate the in-provincial resources, the Cache blacklist is added if the host is the in-provincial resources, and the Cache whitelist is added if the host is the out-of-provincial resources.
According to an embodiment of the present invention, a port may be extracted from each HTTP packet header, cacheability of a hot traffic host is determined according to port information, and a hot traffic host having a port 80 proportion of 100% is added to a cache white list, and/or a hot traffic host having a port 80 proportion of not 100% is added to a cache black list.
For example, since the Cache network only serves resources with port 80, if the proportion of the port 80 under the host is 100%, the Cache white list is added, otherwise, the Cache black list is added.
Fig. 4 shows a schematic flow diagram of a provincial large flow domain name cache according to an embodiment of the invention. As shown in fig. 4, a DPI is used to capture uplink and downlink messages of a user, a successfully captured paired HTTP message header is imported into a call ticket collection system, and a unsuccessfully captured paired HTTP message header is used as spam data. Extracting fields needing to be reserved in an HTTP message header, wherein the fields comprise content-length/host/cache-control/port and the like, and taking the HTTP message with unsuccessful extraction as junk data. And acquiring a hot spot flow host, and performing summation operation on content-length belonging to the same host to obtain flow sequencing taking the host as granularity, namely accurately capturing the hot spot flow host and not processing non-hot spot flow hosts. And judging the analysis attribution of the host, adding a cache blacklist if the host is an intra-province resource, and adding a cache whitelist if the host is an extraprovince resource. And judging the cacheability of the host, if the ratio of the port under the host to 80 is 100%, adding a cache white list, and if not, adding a cache black list.
Fig. 5 is a schematic diagram illustrating an effect of improving cache spitting according to an embodiment of the present invention. It can be seen that by increasing the cache throughput, the cache rate is increased.
Fig. 6 is a schematic block diagram of an apparatus for increasing a cache spitting traffic according to an embodiment of the present invention. As shown in fig. 6, the apparatus 600 may include: a generation module 610, a calculation module 620, a determination module 630, and a processing module 640.
The generating module 610 may generate a full user HTTP ticket based on the HTTP message headers, where the full user HTTP ticket includes the content length extracted from each HTTP message header and the host.
The calculating module 620 may sum the content lengths of the same host based on the HTTP tickets of all users to obtain the total content length of each host.
The determining module 630 may rank the total content length by host granularity, determining the hotspot traffic host.
The processing module 640 may perform cache acceleration processing on the hotspot traffic host.
According to an embodiment of the present invention, the processing module 640 may include:
the determining unit may determine the provincial rate of each hot spot traffic host in combination with the DNS log.
The first attribution unit is used for adding the hot spot flow host with the province rate smaller than a preset threshold value into a cache white list and adding the hot spot flow host with the province rate not smaller than the preset threshold value into a cache black list;
and the processing unit can perform cache acceleration processing on the hosts in the cache white list.
According to an embodiment of the present invention, the processing module 640 may further include:
and the judging unit is used for judging the cacheability of the hot spot traffic host, adding the hot spot traffic host with the port of 80 being 100% into a cache white list, and/or adding the hot spot traffic host with the port of 80 being not 100% into a cache black list.
According to an embodiment of the present invention, the generating module 610 may include:
and the analysis unit can analyze the deep analysis message data to acquire the HTTP message of the user.
And the importing unit can import the message header corresponding to the successfully analyzed HTTP message into the call ticket acquisition system so as to generate the full-user HTTP call ticket.
The second attributing unit may attribute the HTTP message that is not successfully parsed as the spam data.
The parsing unit may include an identification unit, and may identify the user HTTP message through the probe device.
The importing unit may include a transmission unit, and may transmit the identified HTTP message to the ticket collection server through the port mirror image, so as to implement aggregation of the request message and the response message.
According to an embodiment of the present invention, the determining module 630 may include:
and the sorting unit can perform descending order arrangement on the total content length, the first N hosts are used as hot spot flow hosts, and N is the number of the cacheable hosts evaluated by combining the network service capability of the cache.
In summary, in the technical scheme of the present disclosure, content-length/host/cache-control/port core fields in the HTTP protocol are extracted by the DPI system, and content-length of different resources belonging to the same host are summed, so that a large-traffic domain name can be captured. By combining the DNS log and the cacheable analysis system, the provincial flow domain name and the domain name suitable for the acceleration of the cache system can be accurately captured, and the acceleration service of the hot spot service is realized.
In addition, the method for improving the cache spitting flow according to the embodiment of the present invention described in conjunction with fig. 1 may be implemented by a computing device. Fig. 7 is a schematic diagram illustrating a hardware structure of a computing device according to an embodiment of the present invention.
The computing device may include a processor 701 and a memory 702 storing computer program instructions.
Specifically, the processor 701 may include a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured as one or more Integrated circuits implementing an embodiment of the present invention.
Memory 702 may include a mass storage for data or instructions. By way of example, and not limitation, memory 702 may include a Hard Disk Drive (HDD), a floppy Disk Drive, flash memory, an optical Disk, a magneto-optical Disk, tape, or a Universal Serial Bus (USB) Drive or a combination of two or more of these. Memory 702 may include removable or non-removable (or fixed) media, where appropriate. The memory 702 may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory 702 is non-volatile solid-state memory. In a particular embodiment, the memory 702 includes Read Only Memory (ROM). Where appropriate, the ROM may be mask-programmed ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or flash memory or a combination of two or more of these.
The processor 701 may read and execute the computer program instructions stored in the memory 702 to implement any of the above-described methods for increasing the cache spitting flow.
In one example, the computing device may also include a communication interface 703 and a bus 710. As shown in fig. 7, the processor 701, the memory 702, and the communication interface 703 are connected by a bus 710 to complete mutual communication.
The communication interface 703 is mainly used for implementing communication between modules, apparatuses, units and/or devices in the embodiment of the present invention.
Bus 710 includes hardware, software, or both to couple the components of the computing device to each other. By way of example, and not limitation, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hypertransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus or a combination of two or more of these. Bus 710 may include one or more buses, where appropriate. Although specific buses have been described and shown in the embodiments of the invention, any suitable buses or interconnects are contemplated by the invention.
In addition, in combination with the method for improving the cache spitting flow in the foregoing embodiment, an embodiment of the present invention may provide a computer-readable storage medium to implement the method. The computer readable storage medium having stored thereon computer program instructions; the computer program instructions, when executed by a processor, implement any of the above-described embodiments of a method for increasing cache spitting traffic.
It is to be understood that the invention is not limited to the specific arrangements and instrumentality described above and shown in the drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications and additions or change the order between the steps after comprehending the spirit of the present invention.
The functional blocks shown in the above-described structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, Erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, Radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
It should also be noted that the exemplary embodiments mentioned in this patent describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be performed in an order different from the order in the embodiments, or may be performed simultaneously.
As described above, only the specific embodiments of the present invention are provided, and it can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the module and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. It should be understood that the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present invention, and these modifications or substitutions should be covered within the scope of the present invention.

Claims (11)

1. A method for increasing cache spitting traffic, the method comprising:
generating a full-user HTTP ticket based on HTTP message headers, wherein the full-user HTTP ticket comprises content lengths and server identification information extracted from each HTTP message header;
based on the total user HTTP ticket, performing summation operation on the content lengths belonging to the same host to obtain the total content length of each host;
sorting the total content length by taking the host as granularity, and determining a hot spot flow host;
carrying out cache acceleration processing on the hot spot flow host;
wherein the cache acceleration processing on the hotspot traffic host comprises:
determining the provincial rate of each hot spot flow host by combining the DNS log;
adding the hot spot flow host with the province rate smaller than a preset threshold value into a cache white list;
adding the hot spot flow host with the province rate not less than a preset threshold value into a cache blacklist;
and carrying out Cache acceleration processing on the host in the Cache white list by utilizing a Cache network.
2. The method of claim 1,
the provincial rate is the proportion of the times of the DNS analysis of the user to the province to the total times of the DNS analysis of the user.
3. The method according to claim 1, wherein the full-user HTTP ticket further includes a port extracted from each HTTP packet header, and the step of performing cache acceleration processing on the hot spot traffic host further includes:
and judging the cacheability of the hot spot traffic host, and adding the hot spot traffic host with the port 80 of which the proportion is 100% into a cache white list, and/or adding the hot spot traffic host with the port 80 of which the proportion is not 100% into a cache black list.
4. The method of claim 1, wherein the step of generating a full number of user HTTP tickets based on HTTP headers comprises:
analyzing deep analysis message data to obtain an HTTP message of a user;
leading the message header corresponding to the HTTP message successfully analyzed into a call ticket acquisition system to generate the HTTP call ticket of the full amount of users; and
and attributing the HTTP message which is not successfully analyzed as junk data.
5. The method of claim 4, wherein the step of parsing the deep parsed message data to obtain the HTTP message of the user comprises:
and identifying the user HTTP message through the probe equipment.
6. The method according to claim 4, wherein the step of importing the message header corresponding to the HTTP message successfully analyzed into a call ticket collection system comprises:
and transmitting the identified HTTP message to a call bill acquisition server through a port mirror image so as to realize the convergence of the request message and the response message.
7. The method of claim 1,
the full amount user HTTP ticket is a data table which takes the data flow ID of each user HTTP access process as a row and takes the field extracted from the HTTP message header as a column.
8. The method of claim 1, wherein the step of ordering the total content length with a granularity of hosts to determine hot spot traffic hosts comprises:
sorting the total content length in descending order;
and taking the first N hosts as hot spot flow hosts, wherein N is the number of the cacheable hosts evaluated by combining the network service capability of the cache.
9. An apparatus for increasing cache spit-out traffic, the apparatus comprising:
the generating module is used for generating a full-user HTTP ticket based on the HTTP message headers, and the full-user HTTP ticket comprises the content length extracted from each HTTP message header and a host;
the calculation module is used for summing the content lengths of the same host based on the full-user HTTP ticket to obtain the total content length of each host;
the determining module is used for sequencing the total content length by taking the host as granularity and determining the hot spot flow host; and
the processing module is used for carrying out cache acceleration processing on the hot spot flow host;
wherein, the processing module specifically comprises:
the determining unit is used for determining the provincial rate of each hot spot flow host in combination with the DNS log;
the first attribution unit is used for adding the hot spot traffic host with the province rate smaller than a preset threshold value into a cache white list and adding the hot spot traffic host with the province rate not smaller than the preset threshold value into a cache black list;
and the processing unit is used for performing Cache acceleration processing on the host in the Cache white list by utilizing a Cache network.
10. A computing device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory that, when executed by the processor, implement the method of any of claims 1-8.
11. A computer-readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1-8.
CN201711491435.0A 2017-12-30 2017-12-30 Method and device for improving cache spitting flow, computing equipment and storage medium Active CN109995731B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711491435.0A CN109995731B (en) 2017-12-30 2017-12-30 Method and device for improving cache spitting flow, computing equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711491435.0A CN109995731B (en) 2017-12-30 2017-12-30 Method and device for improving cache spitting flow, computing equipment and storage medium

Publications (2)

Publication Number Publication Date
CN109995731A CN109995731A (en) 2019-07-09
CN109995731B true CN109995731B (en) 2021-11-30

Family

ID=67111039

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711491435.0A Active CN109995731B (en) 2017-12-30 2017-12-30 Method and device for improving cache spitting flow, computing equipment and storage medium

Country Status (1)

Country Link
CN (1) CN109995731B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112272123B (en) * 2020-10-16 2022-04-15 北京锐安科技有限公司 Network traffic analysis method, system, device, electronic equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103338249A (en) * 2013-06-26 2013-10-02 优视科技有限公司 Cache method and device
CN103841045A (en) * 2012-11-22 2014-06-04 中国移动通信集团公司 Internet cache processing method, content detection subsystem and Cache system
CN105975433A (en) * 2016-06-30 2016-09-28 瑞斯康达科技发展股份有限公司 Message processing method and device
CN106302784A (en) * 2016-08-27 2017-01-04 浪潮(北京)电子信息产业有限公司 A kind of method and apparatus improving caching speed
CN106487846A (en) * 2015-08-28 2017-03-08 中国电信股份有限公司 The white list updating method of caching system, acquisition and analysis device and caching system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10951489B2 (en) * 2015-12-29 2021-03-16 Digital River, Inc. SLA compliance determination with real user monitoring

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103841045A (en) * 2012-11-22 2014-06-04 中国移动通信集团公司 Internet cache processing method, content detection subsystem and Cache system
CN103338249A (en) * 2013-06-26 2013-10-02 优视科技有限公司 Cache method and device
CN106487846A (en) * 2015-08-28 2017-03-08 中国电信股份有限公司 The white list updating method of caching system, acquisition and analysis device and caching system
CN105975433A (en) * 2016-06-30 2016-09-28 瑞斯康达科技发展股份有限公司 Message processing method and device
CN106302784A (en) * 2016-08-27 2017-01-04 浪潮(北京)电子信息产业有限公司 A kind of method and apparatus improving caching speed

Also Published As

Publication number Publication date
CN109995731A (en) 2019-07-09

Similar Documents

Publication Publication Date Title
CN109951500B (en) Network attack detection method and device
WO2017107780A1 (en) Method, device and system for recognizing illegitimate proxy for charging fraud
US11399288B2 (en) Method for HTTP-based access point fingerprint and classification using machine learning
US9185093B2 (en) System and method for correlating network information with subscriber information in a mobile network environment
US10263868B1 (en) User-specific policy enforcement based on network traffic fingerprinting
CN107770132B (en) Method and device for detecting algorithmically generated domain name
CN108337652B (en) Method and device for detecting flow fraud
US20170134957A1 (en) System and method for correlating network information with subscriber information in a mobile network environment
US11537751B2 (en) Using machine learning algorithm to ascertain network devices used with anonymous identifiers
US9042863B2 (en) Service classification of web traffic
EP2756432A1 (en) System and method for automated classification of web pages and domains
EP3364601A1 (en) Testing method, device and system
US20160142432A1 (en) Resource classification using resource requests
CN109982068B (en) Method, apparatus, device and medium for evaluating quality of synthesized video
CN108900374A (en) A kind of data processing method and device applied to DPI equipment
CN108322354B (en) Method and device for identifying running-stealing flow account
CN112449371B (en) Performance evaluation method of wireless router and electronic equipment
CN111355817B (en) Domain name resolution method, device, security server and medium
CN111343034B (en) Terminal hotspot opening identification method, device, equipment and storage medium
CN103236940A (en) Method and device for content processing and network equipment
CN109995731B (en) Method and device for improving cache spitting flow, computing equipment and storage medium
Wang et al. Smart devices information extraction in home wi‐fi networks
CN115017502A (en) Flow processing method and protection system
CN109995830B (en) Session management system of LTE network
CN106714160B (en) Method and device for checking validity of short message source, terminal and communication system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant