CN106656666B - Method and device for acquiring first screen time of webpage - Google Patents

Method and device for acquiring first screen time of webpage Download PDF

Info

Publication number
CN106656666B
CN106656666B CN201611148648.9A CN201611148648A CN106656666B CN 106656666 B CN106656666 B CN 106656666B CN 201611148648 A CN201611148648 A CN 201611148648A CN 106656666 B CN106656666 B CN 106656666B
Authority
CN
China
Prior art keywords
time
webpage
file
page element
screen
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611148648.9A
Other languages
Chinese (zh)
Other versions
CN106656666A (en
Inventor
刘永生
王光全
杨艳松
刘畅
何晓峰
王泽林
谢梦楠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China United Network Communications Group Co Ltd
Original Assignee
China United Network Communications Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China United Network Communications Group Co Ltd filed Critical China United Network Communications Group Co Ltd
Priority to CN201611148648.9A priority Critical patent/CN106656666B/en
Publication of CN106656666A publication Critical patent/CN106656666A/en
Application granted granted Critical
Publication of CN106656666B publication Critical patent/CN106656666B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876Network utilisation, e.g. volume of load or congestion level
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters

Abstract

The invention discloses a method and a device for acquiring first screen time of a webpage, relates to the technical field of internet, and can solve the problem that the first screen time of the webpage is difficult to acquire when a user does not want to cooperate with a probe to install. The method for acquiring the first screen time of the webpage comprises the following steps: acquiring a corresponding flow record in a DPI system; determining the request time and the return time of the webpage file, and determining the request time and the return time as the loading time of the webpage file; acquiring at least one first screen page element corresponding to the webpage identifier; determining the request time and the return time of a first screen page element file, and determining the request time and the return time as the loading time of the first screen page element file; determining the difference value between the return time of the webpage file and the request time of the first screen page element file as webpage analysis time; and determining the first screen time of the webpage according to the loading time of the webpage file, the loading time of the first screen page element file and the webpage analysis time. The method and the device are suitable for the process of acquiring the first screen time of the webpage.

Description

Method and device for acquiring first screen time of webpage
Technical Field
The invention relates to the technical field of internet, in particular to a method and a device for acquiring first screen time of a webpage.
Background
The first screen time of the webpage is a key index for evaluating the service quality of the webpage, and the first screen time of the webpage refers to the time from the moment when a user initiates a webpage browsing operation to the moment when the screen of the user equipment completely presents webpage content.
Currently, the time for acquiring the first screen of the webpage is in a probe dialing and testing mode, which requires that special software or hardware equipment is deployed at a user side as a probe and hung on a user home gateway or installed in a user computer. If a large amount of user data needs to be collected, a large amount of probes need to be deployed, and then huge investment is needed; in addition, because the webpage browsing operation of the user is simulated during dial testing, the flow of the user needs to be occupied, the normal internet surfing of the user is influenced, and the cost is borne by the user, so that the user is unwilling to cooperate with probe deployment, and therefore, the great difficulty in acquiring the first screen time of the webpage of the user in a large amount is met in the actual operation process.
Disclosure of Invention
The embodiment of the invention provides a method and a device for acquiring the first screen time of a webpage, which can solve the problem that the first screen time of the webpage is difficult to acquire when a user does not want to cooperate with a probe to install.
In order to achieve the purpose, the embodiment of the invention adopts the following technical scheme:
in a first aspect, the present invention provides a method for obtaining a first screen time of a web page, where the web page includes a web page file and a page element file, and the method includes:
acquiring a corresponding flow record in a Deep Packet Inspection (DPI) system according to a terminal access identifier and a webpage identifier, wherein the flow record comprises data packets generated when a terminal corresponding to the terminal access identifier accesses a webpage corresponding to the webpage identifier and the corresponding sending time or receiving time of each data packet, and the data packets comprise a request data packet and a response data packet;
determining the request time of the webpage file corresponding to the webpage identifier and the return time of the webpage file according to the flow record, and determining the difference value between the return time of the webpage file and the request time of the webpage file as the loading time of the webpage file;
acquiring at least one first screen page element corresponding to the webpage identifier, wherein each first screen page element corresponds to a first screen page element file;
determining the request time of the first screen page element file and the return time of the first screen page element file according to the flow records, and determining the difference value between the return time of the first screen page element file and the request time of the first screen page element file as the loading time of the first screen page element file;
determining the difference value between the return time of the webpage file and the request time of the first screen page element file as webpage analysis time;
and determining the webpage initial screen time of the webpage according to the loading time of the webpage file, the loading time of the initial screen page element file and the webpage analysis time.
In a second aspect, the present invention provides an apparatus for obtaining a first screen time of a web page, where the web page includes a web page file and a page element file, the apparatus including:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a corresponding flow record in a Deep Packet Inspection (DPI) system according to a terminal access identifier and a webpage identifier, the flow record comprises data packets generated when a terminal corresponding to the terminal access identifier accesses a webpage corresponding to the webpage identifier and the corresponding sending time or receiving time of each data packet, and the data packets comprise request data packets and response data packets;
the determining module is used for determining the request time of the webpage file and the return time of the webpage file corresponding to the webpage identifier according to the flow record acquired by the acquiring module, and determining the difference value between the return time of the webpage file and the request time of the webpage file as the loading time of the webpage file;
the acquiring module is further configured to acquire at least one first-screen page element corresponding to the webpage identifier, where each first-screen page element corresponds to a first-screen page element file;
the determining module is further configured to determine, according to the flow record acquired by the acquiring module, a request time of the first-screen page element file and a return time of the first-screen page element file, and determine a difference value between the return time of the first-screen page element file and the request time of the first-screen page element file as a loading time of the first-screen page element file;
the determining module is further configured to determine a difference between the return time of the web page file and the request time of the first-screen page element file as a web page parsing time;
the determining module is further configured to determine the first webpage screen time of the webpage according to the loading time of the webpage file, the loading time of the first screen page element file, and the webpage parsing time.
According to the method and the device for acquiring the first screen time of the webpage, provided by the embodiment of the invention, the loading time of the webpage file, the loading time of the first screen page element file and the webpage analysis time are respectively acquired by extracting the flow record in the DPI system at the network side, and then the first screen time of the webpage is determined according to the loading time of the webpage file, the loading time of the first screen page element file and the webpage analysis time. Compared with the prior art that special software or hardware equipment is deployed at a user side as a probe and hung in a user home gateway or a user computer, the method and the system do not need to deploy the probe at the user side, do not need to occupy the flow of the user, can directly process according to the flow record in the network side DPI system, conveniently obtain the first screen time of the webpage, further evaluate the quality of the internet service according to the first screen time of the webpage, and provide a basis for improving the user perception for operators and internet companies.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a method for obtaining a first screen time of a web page according to an embodiment of the present invention;
fig. 2 is a flowchart of another method for obtaining a first screen time of a web page according to an embodiment of the present invention;
fig. 3 is a structural diagram of an apparatus for acquiring a first screen time of a web page according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a terminal for acquiring a first screen time of a webpage according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With the development and fine operation of data services, operators usually perform Deep Packet Inspection (DPI) on data packets flowing through a network. The DPI technology is a flow detection and control technology based on an application layer, and when an IP (Internet Protocol, a Protocol interconnected between networks) data packet, a TCP (transmission control Protocol), or a UDP (User Datagram Protocol) data stream passes through a bandwidth management System based on the DPI technology, the System recombines application layer information in an OSI (Open System Interconnection ) seven-layer Protocol by deeply reading contents of an IP packet load, thereby obtaining contents of a whole application program, and then performs a shaping operation on a flow according to a management policy defined by the System, thereby forming a series of flow records conforming to the management policy. The embodiment of the invention is based on the flow record obtained by the DPI system for further analysis and processing.
The embodiment of the invention provides a method for acquiring the first screen time of a webpage, which comprises the following steps of:
101. and acquiring a corresponding flow record in the DPI system according to the terminal access identifier and the webpage identifier.
The flow record comprises data packets generated when a terminal corresponding to the terminal access identifier accesses a webpage corresponding to the webpage identifier and the sending time or receiving time corresponding to each data packet; the terminal access identifier may be a terminal IP address and the web page identifier may be an address of a web page.
As a specific embodiment, the method monitors the traffic records in the DPI system, identifies information such as a source IP address, a source port number, a destination IP address, and a destination port number of each traffic record, and associates the source IP address and the destination IP address with a terminal access identifier in an AAA (Authentication, Authorization, Accounting) server, thereby identifying the traffic record corresponding to the terminal access identifier. And then acquiring a data traffic record corresponding to the webpage identifier from the traffic record corresponding to the terminal access identifier.
In addition, in order to quickly find out the data packets required in the flow record, the data packets can be classified in advance, and if the terminal access identification in the data packets is the source IP, which indicates that the data packets are sent by the terminal, the data packets are uplink data packets; if the terminal access identification in the data packet is the destination IP address, which indicates that the data packet is sent to the terminal, the data packet is a downlink data packet.
Then, a data packet of HTTP (HyperText Transfer Protocol) is filtered out, and the filtering method is as follows:
1) TCP packets are chosen because HTTP packets are carried over the TCP protocol.
2) The payload of the TCP message is analyzed, the payload is analyzed into text by using ASCII (American Standard Code for information Interchange) Code, and the text is divided into lines by using a carriage return character and a line feed character ('\ r \ n'). For each line of text, the text is cut into fields using spacers.
3) For the upstream packet, the first field matches "GET", or "POST", or "DELETE", or "HEAD", or "OPTIONS", or "PUT", or "TRACE", the third field matches "HTTP/1.1" or "HTTP/1.0", and if both fields match successfully, the packet is determined to be a request packet of HTTP.
4) For downstream packets, the first field matches "HTTP/1.1" or "HTTP/1.0", the second field matches a three-digit number, e.g., "200", and if both fields match successfully, the packet is determined to be a response packet to HTTP.
102. Determining webpage file request time and webpage file return time corresponding to the webpage identification according to the flow record, and determining the difference value of the webpage return time and the webpage request time as webpage loading time.
Generally, a web page includes a web page file and a number of page element files. The web page file also includes a hyperlink address of a page element needing secondary access, and the hyperlink address corresponds to the page element file. The step of completely presenting the first screen content includes loading the webpage file and loading the content of the first screen page element included in the webpage file, and the loading time of the webpage file in the step only refers to the time from the request of the webpage file to the loading end of the webpage file, and does not include the time for loading the first screen page element file.
As a specific embodiment, in the traffic records acquired in step 101, a request packet for requesting to load the web page file is searched, and the sending time of the request packet may be determined as the request time of the web page file. And searching all response messages corresponding to the request data packet, wherein the time of completely receiving the response messages is the loading completion time of the webpage file, and the time can be determined as the return time of the webpage file.
Specifically, the method for determining the webpage request time comprises the following steps: analyzing the texts above two continuous carriage return symbols and line feed symbols ('\ r \ n \ r \ n') below the first line of the acquired request data packet containing the webpage identifier, and comprising the following steps of:
1) the text of the requested packet is cut into lines using the carriage return character and the line feed character ('\ r \ n'), and each line is cut into fields using spaces.
2) The first line is looked up with "GET" as the line of the first field, and the content of the second field is marked as "part 1". Then, a row with "Host:" as a first field is searched in the data packet, the contents of a second field of the row marked as "part 2", "part 2" and "part 1" are connected to form a requested web page address (i.e. a web page identifier), and the corresponding sending time of the request data packet is the web page request time.
The method for determining the webpage return request time specifically comprises the following steps: and acquiring a response data packet corresponding to the request data packet in the same TCP session with the request data packet, wherein the number of the response data packets is at least one. Analyzing the texts above two continuous carriage return symbols and line feed symbols ('\ r \ n \ r \ n') below the first line of the response data packet, and comprising the following steps:
1) text of the response packet is cut into lines using the carriage return character and the line feed character ('\ r \ n'), and each line is cut into fields using spaces.
2) The first response packet is analyzed (determined according to the sequence number field of the TCP) to find the line starting with "Content-Length:", and the value after the field is the size of the web page.
3) And counting the load of a response data packet corresponding to the request data packet in the TCP session, wherein when the load is equal to the size of the requested webpage, the webpage is completely loaded, and the receiving time corresponding to the last response data packet is the webpage returning time.
Furthermore, if the first response packet does not have a row starting with "Content-Length:", then the response packet may be looked up to include "Connection: close "line, the line information indicating that the web page was transferred. Thus, the term "Connection: the receiving time corresponding to the response data packet of close "is the web page returning time.
If there is neither a line starting with "Content-Length:" nor a line starting with "Connection: in the "close" line, the header of the response packet is searched in the TCP session, and whether the response packet is a file to which the web page responds is determined according to information of a URL (Uniform resource locator) of the header of the packet, and when the header information shows that the response packet is not the web page response file, the last response packet of the packet is the last response packet of the requested web page. And the receiving time corresponding to the last response data packet is the returning time of the webpage file.
And finally, subtracting the webpage request time from the webpage return time to obtain the webpage loading time.
103. And acquiring at least one first screen page element corresponding to the webpage identifier.
Specifically, the web page may further include page elements that need to be accessed twice, such as a picture element, an external script element, an external cascading style sheet element, a frame element, and the like, where the page elements that need to be accessed twice correspond to corresponding page element files in the page element server, respectively, and the first-screen page element refers to a page element that needs to be accessed twice and is displayed in the first-screen page of the terminal.
As a specific embodiment, since the page element requiring the secondary access has a specific format, the page element requiring the secondary access is identified from the request packet according to the specific format. For example: the html (HyperText Markup Language) format corresponding to the page element of the picture type is < img src ═ image. jpg "/>, where src represents the picture file contained; the format of the external script element is < script scr ═ behavior.js > < script >, wherein src represents a script file; the format of the external connection stacking style sheet element is < linkrel ═ style, href ═ style, csss "/>, wherein href represents the stacking style sheet file; the format of the frame element is < iframe src ═ frame.
Due to the fact that the size of the screen of the terminal and the setting of the page scaling proportion by the user are different, the size of the page displayed on the first screen of the terminal by the same webpage is different, the number of the displayed first screen page elements is different, and when the precision required for obtaining the first screen time of the webpage is high, the first screen page elements need to be accurately obtained according to specific conditions.
The user can also set the common screen size of a terminal, the first screen page element which can be displayed in the page with the page scaling being the common value is used as a reference, and when the precision required for acquiring the first screen time of the webpage is not high, the first screen page element in the reference can be directly adopted.
And finally, acquiring the identifier of the first screen page element from the webpage file, and searching the first screen page element file corresponding to the first screen page element.
104. And determining the request time of the first screen page element file and the return time of the first screen page element file according to the flow records, and determining the difference value between the return time of the first screen page element file and the request time of the first screen page element file as the loading time of the first screen page element file.
The loading time of the first screen page element file refers to the time from the beginning of requesting the first loaded first screen page element file to the end of the last loaded first screen page element file.
In the traffic records acquired in step 101, a request data packet corresponding to the identifier for requesting to load the first-screen page element is searched, and the sending time of the request data packet can be determined as the request time of the first-screen page element file.
And searching a request data packet for loading the last first-screen page element, and searching all response messages corresponding to the request data packet, wherein the time for completely receiving the response messages can be determined as the return time of the first-screen page element.
The step 102 may be referred to as a specific method for determining the request time and the return time, which is not repeated herein.
And finally, subtracting the request time of the first screen page element file from the return time of the first screen page element file to obtain the loading time of the first screen page element file.
105. And determining the difference value between the return time of the webpage file and the request time of the first screen page element file as webpage analysis time.
Specifically, the process of loading the web page includes: the terminal requests to load the webpage file, and the webpage server responds to the request and sends the webpage file. The terminal receives the webpage file, after receiving, the terminal begins to analyze the content of the webpage file to obtain a first screen page element file needing secondary access, and then requests to load the first screen page element file. The webpage analysis time is the time from the completion of webpage return to the start of the request of the first screen page element file, and is the webpage analysis time.
106. And determining the first screen time of the webpage according to the webpage loading time, the loading time of the first screen page element and the webpage analysis time.
As a specific embodiment, the webpage loading time, the loading time of the first screen page element and the webpage parsing time are added to obtain the webpage first screen time when the same terminal accesses the same webpage.
According to the method for acquiring the first screen time of the webpage, provided by the embodiment of the invention, the loading time of the webpage file, the loading time of the first screen page element file and the webpage analysis time are respectively acquired by extracting the flow record in the DPI system at the network side, and then the first screen time of the webpage is determined according to the loading time of the webpage file, the loading time of the first screen page element file and the webpage analysis time. Compared with the prior art that special software or hardware equipment is deployed at a user side as a probe and hung in a user home gateway or a user computer, the method and the system do not need to deploy the probe at the user side, do not need to occupy the flow of the user, can directly process according to the flow record in the network side DPI system, conveniently obtain the first screen time of the webpage, further evaluate the quality of the internet service according to the first screen time of the webpage, and provide a basis for improving the user perception for operators and internet companies.
Further, as a refinement and an extension of the method shown in fig. 1, an embodiment of the present invention further provides a method for obtaining a first screen time of a web page, as shown in fig. 2, after step 106, the method further includes:
107. and when the first screen time of the webpage acquired in the step 106 is greater than a preset threshold, determining the average loading time of the webpage file of the webpage accessed by the terminal, the average loading time of the first screen page element file and the average analyzing time of the webpage.
The preset threshold value can be determined as an average value of the first screen time of the webpage when the same terminal accesses the same website for multiple times. The selection of the preset threshold value can also be determined according to the requirement of the user on the first screen time of the webpage, and the specific determination standard is not limited herein.
As a specific embodiment, when the terminal accesses the web page for multiple times, the loading time of the web page files is determined each time, and then the loading times of the web page files are averaged, and the average value is determined as the average loading time of the web page files.
The average loading time of the first-screen page element file and the average parsing time of the web page are determined by a similar method, which is not repeated herein.
108. And comparing the loading time of the webpage file with the average loading time of the webpage file, the loading time of the first screen page element file with the average loading time of the first screen page element file, and the webpage analyzing time and the average webpage analyzing time.
As a specific embodiment, a difference between the loading time of the web page file determined in step 102 and the average loading time of the web page file is calculated, a difference between the loading time of the first-screen page element file determined in step 104 and the average loading time of the first-screen page element file is calculated, a difference between the web page parsing time determined in step 105 and the average parsing time of the web page is calculated, and relative sizes of the three differences are compared.
109. And determining factors influencing the initial screen time of the webpage according to the comparison result.
As a specific embodiment, when the first screen time of the web page determined in step 106 is greater than the preset threshold, it indicates that the first screen time of the web page is too long, and the factors affecting the first screen time of the web page need to be analyzed and determined. The larger the difference between the three parameters determined in step 108 (the difference between the loading time of the web page file and the average loading time of the web page file, the difference between the loading time of the first-screen page element file and the average loading time of the first-screen page element file, and the difference between the web page parsing time and the average parsing time of the web page), the larger the influence on the first-screen time of the web page is. According to the relative size of the difference, the bottleneck factor of the first screen time of the webpage can be positioned.
According to the method for acquiring the first screen time of the webpage, provided by the embodiment of the invention, the average loading time of the webpage file, the average loading time of the first screen page element file and the average analysis time of the webpage are respectively acquired by extracting the flow record in the DPI system at the network side, so that the bottleneck factor of the first screen time of the webpage is determined. Compared with the prior art that only one webpage initial screen time of a single terminal for accessing the webpage is obtained, the method and the device directly calculate the difference conditions of the three parameters and the three average values obtained through actual calculation according to the flow records in the network-side DPI system, locate the factors and the influence degree influencing the webpage initial screen time, and provide the basis for improving the user perception for operators and Internet companies.
In addition, in order to more accurately obtain a bottleneck factor affecting the first screen time of the webpage and provide a basis for improving the first screen time of the webpage, the embodiment of the invention also provides another method for obtaining an average value of three parameters, which comprises the following steps:
another specific method for obtaining the loading time of the average webpage file is as follows:
it should be noted that when the same terminal requests to access the same web page, the file of the web page may be stored in a plurality of web site servers at different locations, and due to the influence of the geographical location of the terminal, the transmission network, and other factors, the web site servers that respond to the request of the terminal each time may be different, and when different web site servers respond, the corresponding web page loading time is different, so the corresponding web page loading time needs to be calculated for different web site servers.
As a specific embodiment, in the request packet obtained in step 101, first, a row with "Host:" as a first field is searched, a second field of the row is an IP address of the web server or a domain name corresponding to the web page, and if the row is the domain name corresponding to the web page, the row can be determined as the IP address of the web server by using a destination IP address of the request packet.
Then, when the same terminal accesses the same website and the same webpage server responds, the webpage request time and the webpage return time of each time are determined, and further the webpage loading time of each time is determined. And averaging the loading time of the web pages, namely reflecting the corresponding average loading time when the web page server responds.
Finally, according to the method, the corresponding average loading time when different webpage servers respond is respectively obtained, a list of the IP addresses of the webpage servers and the corresponding average webpage loading time is established, the average loading time when the different webpage servers respond to the same terminal to access the same webpage can be reflected through the list, the numerical value can reflect different influence conditions of the different website servers on the first screen time of the webpage, the specific response website server can be accurately obtained according to the accurate bottleneck factor of determining the first screen time of the webpage, and the improvement direction and basis can be provided for improving the bottleneck factor.
Another specific method for obtaining the average loading time of the first-screen page element file is as follows:
it should be noted that, when the same terminal requests to access the same first-screen page element of the same web page, the file of the first-screen page element may be stored in page element servers at different positions, and due to the influence of factors such as the geographical position of the terminal and the transmission network, the page element servers that respond to the request of the terminal each time may be different, and when different page element servers respond, the corresponding page elements have different loading times, so that the corresponding page element loading times need to be calculated for different page element servers.
As a specific embodiment, in the request packet obtained in step 101, first, a row using "Host:" as a first field is searched, a second field of the row is an IP address of the page element server or a domain name corresponding to the page element, and if the row is the domain name corresponding to the page element, the row can be determined as the IP address of the page element server by using a destination IP address of the request packet.
Then, when the same terminal accesses the same website and the same page element server responds, the request time and the return time of each page element are determined, and the loading time of each page element is further determined. And averaging the loading time of the page element servers, namely reflecting the average loading time of the corresponding first screen page elements when the first screen page element server responds.
Finally, according to the method, the average loading time when different first screen page element servers respond is respectively obtained, a list of the IP address of the first screen page element server and the average loading time of the corresponding first screen page element file is established, the average loading time when different first screen page element servers respond to the same terminal to access the same first screen page element can be reflected through the list, the numerical value can reflect the influence condition of the different first screen page element servers on the first screen time of the webpage, the first screen page element server which responds to the webpage accurately can be accurately obtained according to the bottleneck factor which accurately determines the first screen time of the webpage, and the improvement direction and basis can be provided for improving the bottleneck factor.
Another specific method for obtaining the average parsing time of the web page is as follows:
it should be noted that, when the same terminal requests to access the same web page, the User Agent information corresponding to the User-Agent field of the terminal is different, and the User Agent information includes a hardware type, a hardware configuration, a software version, and the like, and the web page resolution time is also different.
As a specific embodiment, first, in the request packet obtained in step 101, a row with "User-Agent" as the first field is searched, and the information of the content of the second field is User-Agent information.
Then, when the same terminal accesses the same website and the User-Agent information corresponding to the terminal is the same, determining the webpage analysis time of each access, and then averaging the webpage analysis time to reflect the average analysis time corresponding to the User-Agent information.
Average analysis time of web pages finally, average analysis time of web pages corresponding to different User-agents is respectively obtained according to the method, a list of the average analysis time of the web pages corresponding to the User-Agent information is established, the average analysis time of the same website file analyzed by the different User-agents can be reflected through the list, the numerical value can reflect the influence condition of different User-Agent fields on the first screen time of the web pages, the User-Agent information of a specific terminal can be accurately obtained according to the bottleneck factor accurately determining the first screen time of the web pages, and improvement direction and basis can be provided for improving the bottleneck factor.
In addition, based on the method for obtaining the average value of the three parameters, the following method can be adopted for obtaining the average value of the first screen time of the webpage:
obtaining the minimum value of the average webpage loading time, the average first screen page element loading time and the average analysis time of the webpages according to a formula
Figure BDA0001179357110000141
And calculating to obtain the optimal webpage first screen time. Obtaining the maximum value of the average analysis time of the web page according to a formula
Figure BDA0001179357110000142
And calculating to obtain the worst webpage first screen time.
The condition of the web page quality may be evaluated according to the position condition of the web page first screen time determined in step 106 between the optimal web page first screen time and the worst web page first screen time, or the average web page first screen time may be determined according to the average value of the optimal web page first screen time and the worst web page first screen time, where the average web page first screen time may be used as a preset threshold, and it needs to be explained that only one example of the preset threshold is given here, and the setting of the specific preset threshold is not limited here.
An embodiment of the present invention further provides a device 30 for obtaining a first screen time of a web page, as shown in fig. 3, where the web page includes a web page file and a page element file, and the device includes:
the obtaining module 31 is configured to obtain a corresponding flow record in the deep packet inspection DPI system according to a terminal access identifier and a web page identifier, where the flow record includes data packets generated when a terminal corresponding to the terminal access identifier accesses a web page corresponding to the web page identifier and a sending time or a receiving time corresponding to each data packet, and the data packets include a request data packet and a response data packet.
A determining module 32, configured to determine, according to the traffic record obtained by the obtaining module 31, request time of a web page file and return time of the web page file corresponding to the web page identifier, and determine a difference between the return time of the web page file and the request time of the web page file as loading time of the web page file.
The obtaining module 31 is further configured to obtain at least one first-screen page element corresponding to the web page identifier, where each first-screen page element corresponds to one first-screen page element file.
The determining module 32 is further configured to determine, according to the flow record acquired by the acquiring module 31, the request time of the first-screen page element file and the return time of the first-screen page element file, and determine a difference between the return time of the first-screen page element file and the request time of the first-screen page element file as the loading time of the first-screen page element file.
The determining module 32 is further configured to determine a difference between the return time of the web page file and the request time of the first-screen page element file as a web page parsing time.
The determining module 32 is further configured to determine the first screen time of the webpage according to the loading time of the webpage file, the loading time of the first screen page element file, and the webpage parsing time.
Further, the determining module 32 is further configured to determine, as the request time of the web page file, a sending time corresponding to a request data packet that includes the web page identifier and is used for requesting to load the web page file; searching at least one response data packet which contains the webpage identification and corresponds to the request data packet used for requesting to load the webpage file; and determining the receiving time corresponding to the response data packet received last in the at least one response data packet as the returning time of the webpage file.
The determining module 32 is further configured to determine that the sending time corresponding to the request data packet for requesting to load the first-screen page element file is the request time of the first-screen page element file; searching at least one response data packet corresponding to the request data packet for requesting to load the last first-screen page element file; and determining the receiving time corresponding to the response data packet received last in the at least one response data packet as the returning time of the first screen page element file.
The determining module 32 is further configured to determine a sum of the loading time of the web page file, the loading time of the first screen page element file, and the web page parsing time as the web page first screen time of the web page.
The determining module 32 is further configured to determine, when the first screen time of the web page is greater than a preset threshold, an average loading time of a web page file of the web page accessed by the terminal, an average loading time of a first screen page element file, and an average parsing time of the web page; comparing the loading time of the webpage file with the average loading time of the webpage file, the loading time of the first screen page element file with the average loading time of the first screen page element file, and the webpage analyzing time with the average webpage analyzing time; and determining factors influencing the initial screen time of the webpage according to the comparison result.
According to the device for acquiring the first screen time of the webpage, provided by the embodiment of the invention, the loading time of the webpage file, the loading time of the first screen page element file and the webpage analysis time are respectively acquired by extracting the flow record in the DPI system at the network side, and then the first screen time of the webpage is determined according to the loading time of the webpage file, the loading time of the first screen page element file and the webpage analysis time. Compared with the prior art that special software or hardware equipment is deployed at a user side as a probe and hung in a user home gateway or a user computer, the method and the system do not need to deploy the probe at the user side, do not need to occupy the flow of the user, can directly process according to the flow record in the network side DPI system, conveniently obtain the first screen time of the webpage, further evaluate the quality of the internet service according to the first screen time of the webpage, and provide a basis for improving the user perception for operators and internet companies.
As shown in fig. 4, an embodiment of the present invention provides a terminal for obtaining a first screen time of a webpage, including: a processor 401, a memory 402, a transceiver 403, and a bus 404, the processor 401, the memory 402, and the transceiver 403 communicating with each other through the bus 404. The memory 402 is configured to store a plurality of instructions to implement the dialing method provided by the present invention, and the processor 401 executes the plurality of instructions to implement obtaining a corresponding traffic record in a Deep Packet Inspection (DPI) system according to a terminal access identifier and a web page identifier, where the traffic record includes a data packet generated when a terminal corresponding to the terminal access identifier accesses a web page corresponding to the web page identifier and a sending time or a receiving time corresponding to each data packet, and the data packet includes a request data packet and a response data packet; determining the request time of the webpage file corresponding to the webpage identifier and the return time of the webpage file according to the flow record, and determining the difference value between the return time of the webpage file and the request time of the webpage file as the loading time of the webpage file; acquiring at least one first screen page element corresponding to the webpage identifier, wherein each first screen page element corresponds to a first screen page element file; determining the request time of the first screen page element file and the return time of the first screen page element file according to the flow records, and determining the difference value between the return time of the first screen page element file and the request time of the first screen page element file as the loading time of the first screen page element file; determining the difference value between the return time of the webpage file and the request time of the first screen page element file as webpage analysis time; and determining the webpage initial screen time of the webpage according to the loading time of the webpage file, the loading time of the initial screen page element file and the webpage analysis time.
Further, the processor 401 is further configured to determine, as the request time of the web page file, a sending time corresponding to a request data packet that includes the web page identifier and is used for requesting to load the web page file; searching at least one response data packet which contains the webpage identification and corresponds to the request data packet used for requesting to load the webpage file; and determining the receiving time corresponding to the response data packet received last in the at least one response data packet as the returning time of the webpage file.
Further, the processor 401 is further configured to determine that a sending time corresponding to the request data packet for requesting to load the first screen page element file is a request time of the first screen page element file; searching at least one response data packet corresponding to the request data packet for requesting to load the last first-screen page element file; and determining the receiving time corresponding to the response data packet received last in the at least one response data packet as the returning time of the first screen page element file.
Further, the processor 401 is further configured to determine a sum of the loading time of the webpage file, the loading time of the first screen page element file, and the webpage parsing time as the webpage first screen time of the webpage.
Further, the processor 401 is further configured to determine, when the first screen time of the webpage is greater than a preset threshold, an average loading time of a webpage file of the webpage accessed by the terminal, an average loading time of a first screen page element file, and an average parsing time of the webpage; comparing the loading time of the webpage file with the average loading time of the webpage file, the loading time of the first screen page element file with the average loading time of the first screen page element file, and the webpage analyzing time with the average webpage analyzing time; and determining factors influencing the initial screen time of the webpage according to the comparison result.
The processor 401 according to the embodiment of the present invention may be a single processor, or may be a general term for multiple processing elements. For example, the processor 401 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present invention, such as: one or more microprocessors (digital signal processors, DSP for short), or one or more Field programmable gate arrays (FPGA for short).
The memory 402 may be a single storage device or a combination of storage elements, and is used for storing executable program codes and the like. And the memory 402 may include a Random Access Memory (RAM) or a non-volatile memory (non-volatile memory), such as a magnetic disk memory, a Flash memory (Flash), and the like.
The bus 404 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus 404 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 4, but this does not indicate only one bus or one type of bus.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus embodiment, since it is substantially similar to the method embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiment for relevant points.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (8)

1. A method for obtaining the first screen time of a webpage is characterized in that the webpage comprises a webpage file and a page element file, and the method comprises the following steps:
acquiring a corresponding flow record in a Deep Packet Inspection (DPI) system according to a terminal access identifier and a webpage identifier, wherein the flow record comprises data packets generated when a terminal corresponding to the terminal access identifier accesses a webpage corresponding to the webpage identifier and the corresponding sending time or receiving time of each data packet, and the data packets comprise a request data packet and a response data packet;
determining the request time of the webpage file corresponding to the webpage identifier and the return time of the webpage file according to the flow record, and determining the difference value between the return time of the webpage file and the request time of the webpage file as the loading time of the webpage file;
acquiring at least one first screen page element corresponding to the webpage identifier, wherein each first screen page element corresponds to a first screen page element file;
determining the request time of the first screen page element file and the return time of the first screen page element file according to the flow records, and determining the difference value between the return time of the first screen page element file and the request time of the first screen page element file as the loading time of the first screen page element file;
determining the difference value between the return time of the webpage file and the request time of the first screen page element file as webpage analysis time;
determining the webpage initial screen time of the webpage according to the loading time of the webpage file, the loading time of the initial screen page element file and the webpage analysis time;
when the first screen time of the webpage is larger than a preset threshold value, determining the average loading time of the webpage file of the webpage accessed by the terminal, the average loading time of the element file of the first screen page and the average analyzing time of the webpage;
comparing the loading time of the webpage file with the average loading time of the webpage file, the loading time of the first screen page element file with the average loading time of the first screen page element file, and the webpage analyzing time with the average webpage analyzing time;
and determining factors influencing the initial screen time of the webpage according to the comparison result.
2. The method of claim 1, wherein the determining the request time of the webpage file and the return time of the webpage file corresponding to the webpage identifier according to the traffic record comprises:
determining the sending time corresponding to a request data packet which contains the webpage identifier and is used for requesting to load the webpage file as the request time of the webpage file;
searching at least one response data packet which contains the webpage identification and corresponds to the request data packet used for requesting to load the webpage file;
and determining the receiving time corresponding to the response data packet received last in the at least one response data packet as the returning time of the webpage file.
3. The method of claim 2, wherein determining the request time for the first page element file and the return time for the first page element file from the traffic log comprises:
determining the sending time corresponding to the request data packet for requesting to load the first-screen page element file as the request time of the first-screen page element file;
searching at least one response data packet corresponding to the request data packet for requesting to load the last first-screen page element file;
and determining the receiving time corresponding to the response data packet received last in the at least one response data packet as the returning time of the first screen page element file.
4. The method of claim 1, wherein determining the first screen time of the web page according to the loading time of the web page file, the loading time of the first screen page element file, and the web page parsing time comprises:
and determining the sum of the loading time of the webpage file, the loading time of the first screen page element file and the webpage analysis time as the webpage first screen time of the webpage.
5. An apparatus for obtaining a first screen time of a web page, wherein the web page includes a web page file and a page element file, the apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a corresponding flow record in a Deep Packet Inspection (DPI) system according to a terminal access identifier and a webpage identifier, the flow record comprises data packets generated when a terminal corresponding to the terminal access identifier accesses a webpage corresponding to the webpage identifier and the corresponding sending time or receiving time of each data packet, and the data packets comprise request data packets and response data packets;
the determining module is used for determining the request time of the webpage file and the return time of the webpage file corresponding to the webpage identifier according to the flow record acquired by the acquiring module, and determining the difference value between the return time of the webpage file and the request time of the webpage file as the loading time of the webpage file;
the acquiring module is further configured to acquire at least one first-screen page element corresponding to the webpage identifier, where each first-screen page element corresponds to a first-screen page element file;
the determining module is further configured to determine, according to the flow record acquired by the acquiring module, a request time of the first-screen page element file and a return time of the first-screen page element file, and determine a difference value between the return time of the first-screen page element file and the request time of the first-screen page element file as a loading time of the first-screen page element file;
the determining module is further configured to determine a difference between the return time of the web page file and the request time of the first-screen page element file as a web page parsing time;
the determining module is further configured to determine the first webpage screen time of the webpage according to the loading time of the webpage file, the loading time of the first screen page element file, and the webpage parsing time;
the determining module is further configured to determine, when the first screen time of the webpage is greater than a preset threshold, an average loading time of a webpage file of the webpage accessed by the terminal, an average loading time of a first screen page element file, and an average parsing time of the webpage;
comparing the loading time of the webpage file with the average loading time of the webpage file, the loading time of the first screen page element file with the average loading time of the first screen page element file, and the webpage analyzing time with the average webpage analyzing time;
and determining factors influencing the initial screen time of the webpage according to the comparison result.
6. The apparatus according to claim 5, wherein the determining module is further configured to determine a sending time corresponding to a request packet that includes the webpage identifier and is used for requesting to load the webpage file as the request time of the webpage file;
searching at least one response data packet which contains the webpage identification and corresponds to the request data packet used for requesting to load the webpage file;
and determining the receiving time corresponding to the response data packet received last in the at least one response data packet as the returning time of the webpage file.
7. The apparatus according to claim 6, wherein the determining module is further configured to determine that a sending time corresponding to the request packet for requesting to load the first page element file is a request time of the first page element file;
searching at least one response data packet corresponding to the request data packet for requesting to load the last first-screen page element file;
and determining the receiving time corresponding to the response data packet received last in the at least one response data packet as the returning time of the first screen page element file.
8. The apparatus of claim 5, wherein the determining module is further configured to determine a sum of the loading time of the webpage file, the loading time of the first-screen page element file, and the webpage parsing time as the webpage first-screen time of the webpage.
CN201611148648.9A 2016-12-13 2016-12-13 Method and device for acquiring first screen time of webpage Active CN106656666B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611148648.9A CN106656666B (en) 2016-12-13 2016-12-13 Method and device for acquiring first screen time of webpage

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611148648.9A CN106656666B (en) 2016-12-13 2016-12-13 Method and device for acquiring first screen time of webpage

Publications (2)

Publication Number Publication Date
CN106656666A CN106656666A (en) 2017-05-10
CN106656666B true CN106656666B (en) 2020-05-22

Family

ID=58825073

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611148648.9A Active CN106656666B (en) 2016-12-13 2016-12-13 Method and device for acquiring first screen time of webpage

Country Status (1)

Country Link
CN (1) CN106656666B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108984384A (en) * 2017-06-01 2018-12-11 中国电信股份有限公司 Webpage head screen time-delay measuring method, device and system
CN107992416B (en) * 2017-11-28 2021-02-23 中国联合网络通信集团有限公司 Method and device for determining webpage time delay
CN108671547A (en) * 2018-04-24 2018-10-19 腾讯科技(深圳)有限公司 A kind of method, apparatus and storage medium of the initial screen time obtaining application
CN109039715A (en) * 2018-07-17 2018-12-18 中国联合网络通信集团有限公司 User's web page browsing experience evaluation method and system, network base station configuration method
CN109740093A (en) * 2018-12-26 2019-05-10 无线生活(北京)信息技术有限公司 The determination method and device of initial screen time
CN110717121B (en) * 2019-08-27 2022-07-12 北京奇艺世纪科技有限公司 Method and device for determining first screen time of page
CN110781427A (en) * 2019-10-30 2020-02-11 北京博睿宏远数据科技股份有限公司 Method, device, equipment and storage medium for calculating first screen time
CN111124530B (en) * 2019-11-25 2023-06-02 北京博睿宏远数据科技股份有限公司 Method and device for calculating first rendering time of browser, browser and storage medium
CN112507261A (en) * 2020-12-18 2021-03-16 无线生活(杭州)信息科技有限公司 First screen time calculation method and device

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103324521A (en) * 2013-06-20 2013-09-25 张家港保税区润桐电子技术研发有限公司 Method for measuring initial screen time of webpage
CN103559222A (en) * 2013-10-18 2014-02-05 北京奇虎科技有限公司 Method and device for displaying page loading time in browsers
CN103812713A (en) * 2012-11-06 2014-05-21 腾讯科技(深圳)有限公司 Method and apparatus for testing webpage loading time
CN104412254A (en) * 2012-03-30 2015-03-11 茨特里克斯系统公司 Measuring web browsing quality of experience in real-time at an intermediate network node
CN104685831A (en) * 2012-09-28 2015-06-03 瑞典爱立信有限公司 Measuring web page rendering time
CN104850415A (en) * 2014-02-13 2015-08-19 腾讯科技(深圳)有限公司 Method and apparatus for loading pages
CN104899131A (en) * 2014-03-04 2015-09-09 腾讯科技(深圳)有限公司 Detection method and device of webpage loading in browser, and mobile terminal
CN105045620A (en) * 2015-07-07 2015-11-11 深信服网络科技(深圳)有限公司 Method for measuring webpage first screen loading time and detection equipment
CN105373617A (en) * 2015-11-27 2016-03-02 中国联合网络通信集团有限公司 Web page first screen determination method and apparatus
CN105677678A (en) * 2014-11-20 2016-06-15 腾讯科技(武汉)有限公司 Determination method for first screen positions of pages, webpage information display method and system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101382958A (en) * 2008-10-21 2009-03-11 深圳华为通信技术有限公司 Web page loading method and device
CN103678372B (en) * 2012-09-14 2019-05-07 北京百度网讯科技有限公司 A kind of method and apparatus for obtaining the application performance of the page
US20150149609A1 (en) * 2013-11-22 2015-05-28 Microsoft Corporation Performance monitoring to provide real or near real time remediation feedback

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104412254A (en) * 2012-03-30 2015-03-11 茨特里克斯系统公司 Measuring web browsing quality of experience in real-time at an intermediate network node
US9167021B2 (en) * 2012-03-30 2015-10-20 Citrix Systems, Inc. Measuring web browsing quality of experience in real-time at an intermediate network node
CN104685831A (en) * 2012-09-28 2015-06-03 瑞典爱立信有限公司 Measuring web page rendering time
CN103812713A (en) * 2012-11-06 2014-05-21 腾讯科技(深圳)有限公司 Method and apparatus for testing webpage loading time
CN103324521A (en) * 2013-06-20 2013-09-25 张家港保税区润桐电子技术研发有限公司 Method for measuring initial screen time of webpage
CN103559222A (en) * 2013-10-18 2014-02-05 北京奇虎科技有限公司 Method and device for displaying page loading time in browsers
CN104850415A (en) * 2014-02-13 2015-08-19 腾讯科技(深圳)有限公司 Method and apparatus for loading pages
CN104899131A (en) * 2014-03-04 2015-09-09 腾讯科技(深圳)有限公司 Detection method and device of webpage loading in browser, and mobile terminal
CN105677678A (en) * 2014-11-20 2016-06-15 腾讯科技(武汉)有限公司 Determination method for first screen positions of pages, webpage information display method and system
CN105045620A (en) * 2015-07-07 2015-11-11 深信服网络科技(深圳)有限公司 Method for measuring webpage first screen loading time and detection equipment
CN105373617A (en) * 2015-11-27 2016-03-02 中国联合网络通信集团有限公司 Web page first screen determination method and apparatus

Also Published As

Publication number Publication date
CN106656666A (en) 2017-05-10

Similar Documents

Publication Publication Date Title
CN106656666B (en) Method and device for acquiring first screen time of webpage
CN107624233B (en) VPN transmission tunnel scheduling method and device and VPN client server
CN107341160B (en) Crawler intercepting method and device
CN103179132B (en) A kind of method and device detecting and defend CC attack
CN107341395B (en) Method for intercepting reptiles
EP2831755B1 (en) Measuring web browsing quality of experience in real-time at an intermediate network node
US9426046B2 (en) Web page download time analysis
CN103067214B (en) For testing the method for web site performance, client, server and system
KR101903533B1 (en) Service quality index calculation method and calculation apparatus, and communications system
CN107124430B (en) Page hijacking monitoring method, device, system and storage medium
CN108768921B (en) Malicious webpage discovery method and system based on feature detection
CN108924106B (en) Terminal internet auditing method based on network card packet capturing
CN108206769B (en) Method, apparatus, device and medium for filtering network quality alarms
US10868873B2 (en) Communication session log analysis device, method and recording medium
CN113518077A (en) Malicious web crawler detection method, device, equipment and storage medium
CN105635064A (en) CSRF attack detection method and device
CN114244564A (en) Attack defense method, device, equipment and readable storage medium
CN107992416B (en) Method and device for determining webpage time delay
CN110955544A (en) Method, device and system for detecting usability of web system
US20130268662A1 (en) Hypertext transfer protocol http stream association method and device
CN103067495B (en) A kind of method of pushed information and device
CN109241458A (en) A kind of Ad blocking method and router based on router
CN108259528B (en) Resource caching method and device
CN112202717B (en) HTTP request processing method and device, server and storage medium
CN109039715A (en) User&#39;s web page browsing experience evaluation method and system, network base station configuration method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant