CN105794150B - Method for measuring end-to-end Internet application performance - Google Patents
Method for measuring end-to-end Internet application performance Download PDFInfo
- Publication number
- CN105794150B CN105794150B CN201480065938.XA CN201480065938A CN105794150B CN 105794150 B CN105794150 B CN 105794150B CN 201480065938 A CN201480065938 A CN 201480065938A CN 105794150 B CN105794150 B CN 105794150B
- Authority
- CN
- China
- Prior art keywords
- request
- client
- isp
- classification
- percentage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
- H04L41/5061—Network service management, e.g. ensuring proper service fulfilment according to agreements characterised by the interaction between service providers and their network customers, e.g. customer relationship management
- H04L41/5067—Customer-centric QoS measurements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/54—Store-and-forward switching systems
- H04L12/56—Packet switching systems
- H04L12/5691—Access to open networks; Ingress point selection, e.g. ISP selection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0888—Throughput
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Information Transfer Between Computers (AREA)
- Data Mining & Analysis (AREA)
Abstract
For that can identify the request to media item from least one client in the method and system of Internet application performance characterization.To each request, it can determine that request time, ISP's mark, the position of client, the data volume transmitted and fulfil the request elapsed time.It can identify assessment period, geographical frontier and ISP, and it can determine that matching request, wherein, matching request has the request time within the assessment period, the position in geographical frontier and ISP corresponding with ISP mark.The delivery rate of each matching request can be calculated based on the data volume and elapsed time transmitted.The percentage of the matching request with the delivery rate for meeting transmitting threshold value can then be calculated.
Description
Technical field
It is related to Internet application performances areas in terms of the disclosure, and more particularly, is related to for logical when being delivered
By the method for the end-to-end performance characteristics when crossing specific ISP.
Background technique
On internet, content hosting website or other application allow user to upload, browse and shared digital content.This number
Word content may include audio fragment, vidclip, TV segment and music video, and amateurish content, such as videoblog, short
Original video, picture, photo etc..
However, Internet user experiences difference to internet and these applications Internet-based, depending on their company
Connect rate, the service quality of their ISP (ISP), the ISP to internet remainder interconnection
Degree, the availability of application server and load characteristic, and the quality of even home network.It gives most common current
ISP performance measurement (for example, from the downloading rate for the server for usually geographically close to user and being well interconnected to user), when
When their user experience is unsatisfactory for it is expected caused by a certain access network speed rate promised to undertake as ISP, consumer can be made puzzled.
Be currently available that some measurement standards attempt by by from given ISP can be used for the bandwidth characteristicization of terminal user come
Solve the problems, such as this.However, most common in these measurement standards only measure part path, and therefore cannot sufficiently indicate to appoint
What given user can desired application experience.Another technology for assessing application performance concentrates on measuring specific towards specific
The characteristic of application.However, such measurement standard is difficult to repeat, also, even if which show the collection that can be indicated through ISP
The numerical result of the differential performance of conjunction is generally also difficult to be converted to user experience.
Summary of the invention
In some embodiments, it proposes measurement and passes through the method for the end-to-end Internet application performance of specific ISP.It should
Method includes identifying multiple requests to media item from least one client, and to each of multiple requests, really
Determine request time, ISP's mark, the position of client, in response to requesting transmitted data volume and from sending
It requests to fulfil request elapsed time.This method further includes determining that assessment period, geographical frontier and Internet service mention
Donor, and multiple matching requests are determined from multiple requests, wherein each of multiple matching requests had in the assessment period
Interior request time, the position in geographical frontier and ISP corresponding with ISP mark
Know.In addition, this method includes being calculated in the multiple matching request based on corresponding transmitted data volume and elapsed time
Each of delivery rate, and calculate in multiple matching requests, there is the matching request for the delivery rate for meeting transmitting threshold value
Percentage.
In some embodiments, this method can also include being based on percentage, determine the classification of ISP,
Wherein, classification includes the range of percentage, and classification is supplied to user.Classification can be with maintenance to the media of the first clarity
The minimum delivered rate that project (such as video) is transmitted as a stream is corresponding.
In some embodiments, this method can also include that at least one measurement URL is supplied to client, wherein more
A request is based at least one measurement URL.Client can be multiple client.The data volume transmitted can be downloaded to client
Or it is uploaded from client.
Detailed description of the invention
From the attached drawing of following detailed descriptions provided and the various embodiments of the disclosure, the disclosure will be more fully understood.
Fig. 1 illustrates exemplary system architecture according to one embodiment of the disclosure.
Fig. 2 illustrates performance measurement system according to one embodiment of the disclosure.
Fig. 3 is the flow chart for illustrating one embodiment of the method for measuring internet connection speed.
Fig. 4 illustrates the system for initiatively measurement performance according to one embodiment of the disclosure.
Fig. 5 is the frame that can execute the exemplary computer system of one or more of operation described herein
Figure.
Specific embodiment
The aspect of the disclosure is related to be based on the accessible delivery rate of typical user, by being expected with for Internet application
Family experience characterizes.Can identify the request of media item, wherein it is each request be by internet from client to server
's.To each request, request time, ISP's mark, the position of client, in response to the request can determine that
The data volume transmitted and from issue a request to fulfil the request pass through the time.Can identification assessment the period, geographical frontier and mutually
The Internet services supplier, and can determine that matching request, wherein matching request have request time within the assessment period,
Position in geographical frontier, and ISP corresponding with ISP mark.It can be based on institute
The data volume of transmitting and elapsed time calculate the delivery rate of each matching request.Then, it can calculate to have and meet transmitting
The percentage of the matching request of the delivery rate transmitting of threshold value.
In some embodiments, data related with the request downloaded can be collected.For example, if being done via Web service
HTTP GET data are collected in the request out.To each request, determine transmitted in response to the request byte number (for example, using
Byte).Also determine elapsed time (for example, being completed and counting to fulfiling for the request from the time when issuing the request
According to time when can be used for applying).Goodput or application layer handling capacity can be defined as per unit time, being delivered by network
To the number of the useful information bit of a certain destination.Similarly, time (elapsed time) sample of each byte is passed
The Payload size for the byte passed can indicate goodput sample (goodput sample).Moreover, for each request,
It can determine that the IP address of both server and requesting client, and any other available information of the related request (for example, visitor
User agent, server name, the server location at family end etc.).Client ip address can be used to identify related to each request
The ISP (for example, being mapped using IP to ASN (autonomous system number)) of connection and the geographical location of client.Then, for example, energy
Together with geographical location, ASN/IP, request date and request time, each goodput sample is stored.In other words, energy
Goodput sample is annotated using information related with the calculating of connection speed measurement standard.Sequence can be utilized the data to ask
It asks in order to different types of analysis.As an alternative, data related with the request uploaded can be collected, while being requested from initial p OST
End to the data transmitting from client to server passes through the time come what is determined.Once it is determined that in relation to the data requested,
The measurement standard based on the data can be calculated.Here, can select period (for example, 24 hours or one week) and ISP (or
) and geographical location (for example, city, county, state or country) ASN.It can determine that the goodput sample (example using corresponding informance annotation
Such as, generation and goodput associated with selected ISP (or ASN) within selected period and geographical location
Sample).Each goodput sample can be analyzed to determine whether the transmitting threshold value equal to or higher than pre-determining.It can determine that and be in
Or in the sum of the goodput sample higher than threshold value, and the total sample number that is considered, these goodput samples
Percentage.The percentage can be known as being higher than the goodput (GAT) of threshold value.
Consistency threshold value can indicate the level of GAT required by providing acceptable performance (such as 90% or 95%).If
Identified GAT is higher than consistency threshold value, which can be considered as to the basic service requirement for meeting corresponding transmitting threshold value.Transmitting
Threshold value can indicate the minimum number of the stream transmission of the media item (such as video of a certain clarity) for supporting a certain quality
According to delivery rate.
In embodiment, it is flat in video quality classification (or video of a certain clarity) to match to select transmitting threshold value
Equal bit rate.For example, classification number (such as 3) can be established by selection transmitting number of threshold values (such as 2).In this example, classification energy
Correspond to: (1) high definition (HD) video, wherein the mean bit rate (corresponding to 720p resolution video) of 2.5Mbps is for supporting
It is necessary for the adequate performance of HD video；(2) SD (SD) video, wherein the mean bit rate of 0.7Mbps (corresponds to
360p resolution video) it is necessary for the adequate performance for supporting SD video；And (3) mean bit rate is lower than
Adequate performance cannot be supported in the case where 0.7Mbps.In these examples, definition transmitting threshold value is related to above-mentioned classification to match
The mean bit rate of connection.
In the examples described above, if GAT exceeds HD video class requirement, then it is determined that including the network of corresponding ISP
So as to maintain HD video quality.If GAT exceeds SD video class requirement, but is lower than HD video requirement, then it is determined that packet
The network of corresponding ISP is included, so as to maintain SD video quality.If GAT without departing from be used for the other threshold value of any sort, that
Determine that the network cannot maintain adequate performance for SD or HD video quality.
The aspect of the disclosure provides measurement based on the instrument (for example, byte count and elapsed time) being easily obtained,
User is allowed to understand ISP simply to give the reliability that quality level operates.By network measurement criteria (that is, with position per second
Number is the delivery rate of unit) prospective users experience is converted to (that is, whether user is it can be desirable to SD and/or HD matter can be watched
The content of amount).
Fig. 1 illustrates exemplary system architecture 100 according to one embodiment of the disclosure.System 100 includes passing through network
102 communicate the client/user equipment 105 and data repository 130 of (for example, being coupled to) with content server 110.
Network 102 can be dedicated network (for example, local area network (LAN), wide area network (WAN), Intranet etc.), corporate networks (for example, with
In the private network of the tissue of such as company), broadcasting network, common network (for example, internet), wired network (for example, Ethernet), nothing
Gauze (for example, 802.11 nets) and/or Cellular Networks (such as long term evolution (LTE) net).
Client/user equipment 105 can be any kind of calculating equipment, it may for example comprise processor, computer-readable
The equipment of medium and memory.In some embodiments, client/user equipment 105 can execute browser application or be suitble to
In other application and/or display user interface by internet related protocol (such as TCP/IP and HTTP) communication.Although
Single client/user equipment 105 is illustrated only in Fig. 1, but system 100 can support multiple client/user equipment 105
A large amount of parallel sessions.
Content server 110 may include the calculating equipment with wide-range treatment ability, such as personal computer (PC),
Server computer, personal digital assistant (PDA), smart phone, laptop computer, netbook computer, tablet device
And/or it is able to carry out any machine of the instruction set (continuously or otherwise) of the specified movement that will be taken by the machine.This
Disclosed embodiment can operate in individual server equipment or on multiple server apparatus.
In one embodiment, content server 110 is by media item (for example, digital video, streaming media video, image, number
Word tone pleasure, Streaming Media music, social media information etc.) it is supplied to one or more of client/user equipment 105.For example,
Content server 110 can be a part of content shared platform, and can provide and be asked by user in the active source of channel
The media item (such as streaming media video) asked is used to play back in client/user equipment 105 via network 102.
Data repository 130 can include one or more writeable persistent storage devices, such as memory, tape or disk.
Although each of content server 110 and data repository 130 are portrayed as single different component in Fig. 1, these
Component can be carried out together in one single or be joined in the form of the various combinations of the multiple and different equipment operated together
Net.The example of equipment can include but is not limited to the equipment and class of server, mainframe computer, Net-connected computer, Kernel-based methods
Like the system and equipment of type.
During system 100 operates, user requests media item via client/user equipment 105 by network 102.It is interior
Hold server 110 to receive communication from client/user equipment 105 and correspondingly handle and/or orient these communications.
In embodiment, client/user equipment 105 makes request to content server 110.For example, client/user
Equipment 105 can make request (for example, foradownloaded video) related with the media item from content server 110.Content server
110 receive and respond the request.For example, content server 110 can send requested media item (such as video) to
Client/user equipment 105.Content server 110 may include performance measurement system 135,135 energy of performance measurement system
It records the position of client/user equipment 105 to reach the path at content server 110 by determining that request is propagated and mentions
It is supplied to the ISP of the internet access of client/user equipment 105.Performance measurement system 135 can also record the time of request.Energy
Position, ISP and request time are stored in data repository 130.
When having been completed reception (for example, video content) of requested data, client/user equipment 105 can be to
Content server 110 sends confirmation.Performance measurement system 135 can record content server 110 and confirm received time and institute
The data volume of transmission can store it in data repository 130.
Performance measurement system 135 can continue record and client/user equipment 105 and other client/user equipment institutes
The related information of the additional request made.It is then possible to by performance measurement system 135 using the recorded position of each request,
ISP, request time, acknowledging time and data volume, to analyze the connection speed of each ISP at various locations.
In embodiment, the request from client/user equipment 105 can include the request of upload data, rather than under
Carry the request of data.Here, performance measurement system 135 can record information same as described above, except the amount of data recorded is institute
The data volume of upload, and have recorded data and complete except the time uploaded.
Fig. 2 illustrates performance measurement system 210 according to one embodiment of the disclosure.Performance measurement system 210 can wrap
Include data collection module 201, computing module 202 and reporting modules 203.It can include surveying in performance by more or fewer components
Without loss of generality in amount system 210.In embodiment, performance measurement system 210 is performance measurement system shown in Fig. 1
135 and data repository 250 be data repository 130 shown in Fig. 1.
In embodiment, data collection module 201 is collected related with the request from users/customers equipment to content server
Information (for example, request time, the request path including ISP, request geographical location and request implement confirmation when
Between).Information related with the request is recorded in the request list 251 of data repository 250 by data collection module 201.
In embodiment, the analysis of computing module 202 is in special time period, the connection speed of the specific position of ISPs.It calculates
Module 202 determines geographical frontier, ISP and assessment period.For example, user (for example, system manager) can input expected measurement ginseng
Number, such as the assessment period of the geographical frontier in Los Angeles county, ISP X and 8:00am to 9:00am on January 1.Then, computing module
202 can determine that the matching request in request list 251.For example, computing module 202 is included within Lip river between 8:00am to 9:00am
Owning in the request made in the county Shan Ji by ISP X.
Based on each elapsed time in matching request (that is, to receiving what data were transmitted since request
Elapsed time until confirmation) and data volume, computing module 202 can calculate the delivery rate of matching request.For example, can will pass
Passing rate calculations is data volume divided by elapsed time.
Once delivery rate has been determined for each of request, computing module 202 can determine that whether delivery rate meets
One or more threshold value in each of request.Threshold value can be stored in the request list 251 of database 250.For example, calculating
Module 202 can determine that whether delivery rate meets threshold value for keeping HD video quality or whether delivery rate meets and be used for
Keep the threshold value of SD video quality.
Once computing module 202 has determined whether delivery rate in each of request meets one or more of each response
A threshold value, computing module 202 can determine that in matching request, meet the percentage of the matching request of each threshold value.At this point, can be based on
The percentage of matching request will classify for the connection speed of the period and the ISP in geographical location.
In one embodiment, goodput threshold value can be defined according to the MBPS that will be used in rating scale.For example,
It can be by the average coded bits rate (+20% expense) of various video content resolution ratio for classifying, so that threshold value indicates resolution ratio
Capacity (such as 1080p is 5.0Mbps, 720p 2.5Mbps, 480p 1.12Mbps, 360p are 0.7Mbps and 240p is
0.5Mbps)。
In one embodiment, rating scale can be based on defined threshold value.Exemplary grading mark is shown in Table A
It is quasi-.
In this example, the period, (or selected granularity) can be the just period before the determination of classification,
It generally " scene " is tested so that result provides.For example, the period can be 5 minutes before calculating.In another example, when
Between section can be based on the selected duration, the historical data of selected ISP.For example, energy set period of time, makes score
Class can based on the past 30 days average value (for example, grading based on the request from the past 30 days), be averaged within the past 24 hours
It is worth (for example, based on grading from request in 24 hours before), 7 day average of past (for example, based on from 7 days before
The grading of request), performance hourly etc..
Then, reporting modules 203 can report the classification determined by computing module 202.For example, reporting modules 203 can incite somebody to action
Classification is reported to the user of users/customers' equipment, one or more ISPs or publicly available source (for example, website).Class can be used
Do not determine that the connection speed as provided by the ISP of specific position is appropriate for special-purpose.For example, potential user can use
Classification determines which ISP is best suited for the user, so that the user can transmit HD quality video as a stream in the family of user.
Embodiment provides simple and understandable measurement standard to assess the connection speed of each ISPs, because it
Using objective reality throughput data and avoid any transformation.Across the variable grain degree of this method is (for example, generally instantaneous, small
When, day, week, the moon) and across different data sources (for example, rate test service, video flowing streaming services, file download etc.) be
It is consistent.Rating threshold can also provide non-athletic target for ISPs to take into account when assessment target for improving connection speed.
Fig. 3 is the flow chart for illustrating the embodiment of method 300 of the performance for measuring ISP.It can be by may include
Hardware (for example, circuit, special logic, programmable logic, microcode etc.), software are (for example, operation is on a processing device to execute
The instruction of simulation hardware) or combinations thereof processing logic execute method 300.In one embodiment, by server (for example,
The content server 110 of Fig. 1) execute method 300.
At block 302, the request of media item of the logic identification from client to server is handled.For example, the request can be with
It is the request to video content.
At block 304, processing logic determines the request time for each request, ISP mark, position, the number transmitted
According to amount and elapsed time (for example, elapsed time is completed in transmitting from request time to data).For example, request time can be with
It is the particular moment (for example, the 8am on January 1) in date and date.
At block 306, processing logic determines assessment period, geographical frontier and ISP.For example, the assessment period can be the time
Section (for example, 7:30am to the 8:30am on January 1).In another example, geographical frontier can be postcode, GPS coordinate,
Or the set in city or county boundary.
At block 308, processing logic determines whether respective request is matching request to each request.In other words, locate
Reason logic can determine that position whether in geographical frontier, request time whether assessment the period in and ISP is identified whether and ISP
Matching.
At block 310, the delivery rate of each matching request of logic calculation is handled.For example, delivery rate can be calculated
It is the data volume transmitted divided by elapsed time.
At block 312, processing logic calculation has the hundred of the matching request of the delivery rate for the transmitting threshold value for meeting request
Divide ratio.For example, can be by the quantity of the matching request with the delivery rate for meeting transmitting threshold value divided by the quantity of matching request.
Fig. 4 is one embodiment according to the disclosure, the exemplary system 400 for active measurement performance.In the implementation
In example, the request of pre-determining is can be used in user, is initiatively tested to analyze connection speed, rather than works as request
When, passively collect the information in relation to requesting.System 400 includes having performance server front end 411 (for example, system manager
Console) and performance server rear end 412 Testing Platform.System 400 can also include that there is URL to provide application programming
The content hosting platform 415 of interface (API) 413 and content streaming service device 414.As an alternative, URL provides 413 He of API
Content streaming service device 414 can be the part of different platform, system or physical machine.
In embodiment, it at operation 401, is requested once user, after performance server front end 411 indicates performance server
End 412 is tested to measure the performance of ISP used in content streaming service device 414.At operation 402, performance clothes
Business device rear end 412 sends URL for the request for being used to measure uniform resource locator (URL) and provides API 413.In operation 403
Place, URL provide API 413 and are sent to performance server rear end 412 for URLs is measured.Measurement URL is specified to be passed by content streaming
The position for the media item that defeated server 414 provides.At operation 404, performance server rear end 412 will measure being sent to property of URLs
It can server front end 411.
At operation 405, to each measurement URL, performance server front end 411 can be from content streaming service device 414
Request corresponds to the media item of measurement URL.Media item can be calculated by " range=" query argument checked in measurement URL
Anticipatory data amount.If the complete response of request takes too long (for example, because low goodput), performance server
Front end 411 can cancel the request and the response is reported as partial results.At operation 406, content streaming service
Device 414 can be with requested media item, and can record time of measuring, wherein time of measuring is from when performance server front end
411 when transmitting the request to content streaming service device 414 to when by the being delivered to property of decline of requested media item
Elapsed time when energy server front end 411.
At operation 407, performance server front end 411 will can be used to measure time of measuring and data in each of URL
Amount is supplied to performance server rear end 412.At operation 408, performance server rear end 412 can will be used for each measurement URL's
Time of measuring and data volume, which are sent to, provides the URL of API 413.Time of measuring can be indicated for completing to by performance server
The response elapsed time of request received by front end 411.Data volume can be indicated in response to each request, performance server
The byte number received by the content streaming service device 414 of front end 411.However, if performance server front end 411 because
If having already been through the long time before receiving complete response and eliminating request, data volume can be less than anticipatory data
Amount.In addition, performance server front end 411 can indicate successfully to demonstrate from content streaming service device 414
Response, and for the URL of the request.Then, URL provides API 413 and can be based on for measuring measurement in each of URL
Time and data volume, calculated performance result (i.e. the classification of connection speed).At operation 409, URL provides API 413 and can incite somebody to action
Results of property is sent to performance server rear end 412.At operation 410, performance server rear end 412 can be proposed results of property
Performance server front end 411 is supplied, user (for example, system manager) is given for rendering, so that user can analyze the connection
Whether rate is appropriate.
Fig. 5 illustrates the graphic representation of the machine in the exemplary form of computer system 500, in the computer system
It can be executed in 500 for so that machine executes the collection of the instruction of any one or more of process discussed herein opinion
It closes.In the embodiment of alternative, machine can be connected (for example, being networked) to LAN, Intranet, extranet or internet
In other machines.Machine can operate in client-server network environment as server or client machine, or
Person operates in equity (or distributed) network environment as peer machines.Machine can be personal computer (PC), tablet PC,
Set-top box (STB), personal digital assistant (PDA), cellular phone, web apparatus, server, network router, interchanger or bridge joint
Device or can (continuously or otherwise) execute specifies will be by the set of the instruction for the movement that the machine is taken
Any machine.Further, although only illustrating individual machine, term " machine " should also be taken to include the Arbitrary Sets of machine
It closes, the set (multiple set) that the arbitrary collection of the machine executes instruction alone or in combination is to execute side discussed herein
Any one or more of method opinion.
Exemplary computer system 500 includes the processing equipment (processor) 502 to communicate with one another via bus 530, master
Memory 504 is (for example, read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (such as synchronous mode
DRAM's (SDRAM) or RambusDRAM (RDRAM)) etc.), static memory 506 is (for example, flash memory, static random
Access memory (SRAM) etc.) and data storage device 518.
Processor 502 indicates one or more general purpose processing devices of microprocessor, central processing unit etc..More
Body, processor 502 can be complex instruction set calculation (CISC) microprocessor, reduced instruction set computing (RISC) micro process
Device, very long instruction word (VLIW) microprocessor or implement other instruction set processor or implement instruction set combined place
Manage device.Processor 502 is also possible to one or more dedicated treatment facilities, and such as specific integrated circuit (ASIC), scene can compile
Journey gate array (FPGA), digital signal processor (DSP), network processing unit etc..Processor 502 is configured to execute for executing
The instruction 522 of operation and step discussed in this article.
Computer system 500 may further include network interface device 508.Computer system 500 also may include view
Frequency display 510 (for example, liquid crystal display (LCD) or cathode-ray tube (CRT)), Alphanumeric Entry Device 512 (for example,
Keyboard), cursor control device 514 (for example, mouse) and signal generating device 516 (for example, loudspeaker).
Data storage device 518 may include computer readable storage medium 528, have stored thereon the one of instruction 522
A or multiple set (for example, software), described instruction is by any one of methodology or function described herein or more
A materialization.Instruction 522 can also completely or at least partially reside at master during it is executed by computer system 500
In memory 504 and/or in processor 502, main memory 504 and processor 502 also constitute computer readable storage medium.It can
Further to transmit or receive instruction 522 by network 520 via network interface device 508.
In one embodiment, instruction 522 includes the instruction for performance measurement system 550, can correspond to Fig. 1's
Performance measurement system 135, and/or the software library of the method comprising measuring end-to-end Internet application performance.Although computer can
It reads storage medium 528 and is shown as single medium in the exemplary embodiment, still should take term " computer-readable storage medium
Matter " is with the one or more single media gathered for including store instruction or multiple media (for example, centralization or distributed number
According to library, and/or associated caching and server).Term " computer readable storage medium " should also be taken to include that can deposit
Storage, coding are carried for being executed by machine and machine being made to execute any one or more of disclosed method opinion
Instruction set any medium.Therefore term " computer readable storage medium " should be taken to include but is not limited to solid-state storage
Device, optical medium and magnetic medium.
In description in front, many details are elaborated.However, also can be real in the case where not having these specific details
The disclosure is trampled, this is for being obvious for enjoying those of ordinary skill in the art of disclosure benefit.In some examples
In, well-known structure and equipment are shown in form of a block diagram rather than in detail, to avoid the fuzzy disclosure.
It has been indicated according to the algorithm and symbol of the operation to the data bit in computer storage specific implementation is presented
Some parts in mode.These algorithm descriptions and expression are that those of data processing field technical staff is used to most effectively to
Its action is communicated to others skilled in the art and the means that use.Herein and in general, algorithm is contemplated to be
Cause expected results from consistent series of steps.The step is those of the physical manipulation required to physical quantity.In general,
Although be not necessarily required in this way, this tittle take can by storage, delivering, in conjunction with, compare and otherwise manipulate
The form of the electrical or magnetic signal.It was demonstrated that often for convenience, mainly due to the reason of idiom, by these signals
Referred to as bit, value, element, symbol, character, term, number etc..
However, should keep firmly in mind, all in these and similar terms will be associated with appropriate physical quantity and only
It is the convenient labels for being applied to this tittle.Unless will become apparent from from following discussion specifically stated otherwise, it should be appreciated that whole
In a specification, using such as " determination ", " calculating (computing) ", " calculating (calculating) ", " acquisition ", " know
Not ", the discussion of the terms such as " modification " refers to the movement and process of computer system or similar electronic computing device, the meter
Calculation machine system or similar electronic computing device will be indicated as the physics (example in the register and memory of computer system
Such as, electronics) amount data manipulation and being converted into equally be expressed as computer system memory or register or other as
Other data of physical quantity in information storage, transmission or display equipment.
Present disclosure also relates to apparatuses for performing the operations herein.The device can for required purpose and by
Special configuration or its may include the computer program by being stored in computer to selectively activate or reconfigure
General purpose computer.Such computer program can be stored in computer readable storage medium, such as, but not limited to be appointed
The disk of what type comprising floppy disk, CD, CD-ROM and magneto-optic disk, read-only memory (ROM), random access memory
(RAM), EPROM, EEPROM, magnetic or optical card or any kind of medium suitable for storing e-command.
" one embodiment " or " embodiment " is quoted in entire this specification to mean in relation to specific described in embodiment
Feature, structure or characteristic are included at least one embodiment.Therefore, everywhere in this specification middle appearance it is short
Language " in one embodiment " is not necessarily all referring to identical embodiment " in embodiment ".In addition, term "or" is intended to anticipate
Refer to inclusive "or" and non-exclusive "or".
It should be understood that above description be intended to it is illustrative, and it is and non-limiting.Those skilled in the art read and understand with
After upper description, many other embodiments for them for will be evident.Therefore, the scope of the present disclosure should refer to appended power
Benefit requires the entire scope of the equivalent way assigned together with such claim to determine.
Claims (20)
1. a kind of method implemented by server computing systems, comprising:
Multiple requests to media item from least one client are identified by the server computing systems；
To each of the multiple request, determined by the server computing systems:
Request time,
ISP's mark,
The position of the client,
The data volume transmitted in response to the request, and
From the request is issued to the process time for fulfiling the request；
Assessment period, geographical frontier and ISP are determined by the server computing systems；
Multiple matching requests are determined from the multiple request by the server computing systems, wherein the multiple matching is asked
Each of ask have the request time within the assessment period, the position in the geographical frontier and with the internet
The corresponding ISP's mark of ISP；
By the server computing systems based on the corresponding data amount transmitted and elapsed time, to calculate the multiple matching
Each delivery rate in request；And
In the multiple matching request with the delivery rate for meeting transmitting threshold value is calculated by the server computing systems
Percentage with request.
2. the method as described in claim 1 further comprises:
Based on the percentage, the classification of the ISP is determined, wherein the classification includes the model of percentage
It encloses；And
The classification is supplied to user.
3. method according to claim 2, wherein the classification is passed with the streaming for maintaining the media item of the first clarity
Defeated minimum delivered rate is corresponding.
4. the method as described in claim 1 further comprises:
At least one measurement URL is supplied to the client, wherein the multiple request is based at least one described measurement
URL。
5. the method for claim 1, wherein the client includes multiple client.
6. the data volume the method for claim 1, wherein transmitted is downloaded to the client.
7. the method for claim 1, wherein uploading transmitted data volume from the client.
8. a kind of non-transitory computer-readable storage media for being stored with computer program, when the computer program is by handling
When equipment executes, the processing equipment is caused to execute operation, the operation includes:
Identify multiple requests to media item from least one client；
To each of the multiple request, determine:
Request time,
ISP's mark,
The position of the client,
The data volume transmitted in response to the request, and
From the request is issued to the process time for fulfiling the request；
Determine assessment period, geographical frontier and ISP；
Multiple matching requests are determined from the multiple request, wherein each of the multiple matching request has described
It assesses the request time in the period, the position in the geographical frontier and is taken with the internet of the ISP
The corresponding ISP's mark of business supplier；
Based on the corresponding data amount transmitted and elapsed time, to calculate transmitting speed each in the multiple matching request
Rate；And
Calculate the percentage in the multiple matching request with the matching request for the delivery rate for meeting transmitting threshold value.
9. non-transitory computer-readable storage media as claimed in claim 8, wherein the operation further comprises:
Based on the percentage, the classification of the ISP is determined, wherein the classification includes the model of percentage
It encloses；And
The classification is supplied to user.
10. non-transitory computer-readable storage media as claimed in claim 9, wherein the classification with for maintaining the
The minimum delivered rate of the stream transmission of the media item of one clarity is corresponding.
11. non-transitory computer-readable storage media as claimed in claim 8, wherein the operation further comprises: will
At least one measurement URL is supplied to the client, wherein the multiple request is based at least one measurement URL.
12. non-transitory computer-readable storage media as claimed in claim 8, wherein the client includes multiple visitors
Family end.
13. non-transitory computer-readable storage media as claimed in claim 8, wherein the data volume transmitted is downloaded
To the client.
14. non-transitory computer-readable storage media as claimed in claim 8, wherein passed from the client
The data volume passed.
15. a kind of calculating equipment, comprising:
Memory, and
It is coupled to the processing equipment of the memory, wherein the processing equipment is configured as:
Identify multiple requests to media item from least one client；
To each of the multiple request, determine:
Request time,
ISP's mark,
The position of the client,
The data volume transmitted in response to the request, and
From the request is issued to the process time for fulfiling the request；
Determine assessment period, geographical frontier and ISP；
Multiple matching requests are determined from the multiple request, wherein each of the multiple matching request has described
It assesses the request time in the period, the position in the geographical frontier and is taken with the internet of the ISP
The corresponding ISP's mark of business supplier；
Based on the corresponding data amount transmitted and elapsed time, to calculate transmitting speed each in the multiple matching request
Rate；And
Calculate the percentage in the multiple matching request with the matching request for the delivery rate for meeting transmitting threshold value.
16. calculating equipment as claimed in claim 15, wherein the processing equipment is further configured to:
Based on the percentage, the classification of the ISP is determined, wherein the classification includes the model of percentage
It encloses；And
The classification is supplied to user.
17. calculating equipment as claimed in claim 16, wherein the classification and the media item for maintaining the first clarity
The minimum delivered rate of stream transmission is corresponding.
18. calculating equipment as claimed in claim 15, wherein the processing equipment is further configured to: by least one
Measurement URL is supplied to the client, wherein the multiple request is based at least one measurement URL.
19. calculating equipment as claimed in claim 15, wherein the client includes multiple client.
20. calculating equipment as claimed in claim 15, wherein the data volume transmitted is downloaded to the client.
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201361910732P | 2013-12-02 | 2013-12-02 | |
| US61/910,732 | 2013-12-02 | ||
| US14/153,956 US10284439B2 (en) | 2013-12-02 | 2014-01-13 | Method for measuring end-to-end internet application performance |
| US14/153,956 | 2014-01-13 | ||
| PCT/US2014/068082 WO2015084799A1 (en) | 2013-12-02 | 2014-12-02 | Method for measuring end-to-end internet application performance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105794150A CN105794150A (en) | 2016-07-20 |
| CN105794150B true CN105794150B (en) | 2019-03-15 |
Family
ID=53266242
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201480065938.XA Active CN105794150B (en) | 2013-12-02 | 2014-12-02 | Method for measuring end-to-end Internet application performance |
Country Status (5)
| Country | Link |
|---|---|
| US (3) | US10284439B2 (en) |
| EP (1) | EP3078169B1 (en) |
| CN (1) | CN105794150B (en) |
| AU (1) | AU2014357345B2 (en) |
| WO (1) | WO2015084799A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11258690B1 (en) * | 2017-11-02 | 2022-02-22 | Speedpact Inc. | System and method for testing and/or monitoring broadband internet connectivity |
| US10938699B1 (en) * | 2019-10-01 | 2021-03-02 | Cisco Technology, Inc. | Systems and methods for monitoring and addressing network performance issues |
| KR20220053151A (en) * | 2020-10-22 | 2022-04-29 | 삼성에스디에스 주식회사 | Method and apparatus for controlling peer to peer connection |
| CN115334329B (en) * | 2022-08-09 | 2023-10-24 | 重庆数字涪陵大数据产业发展有限公司 | Big data transmission duration analysis platform and method |
| CN116155618B (en) * | 2023-04-04 | 2023-06-23 | 天云融创数据科技（北京）有限公司 | Data maintenance method and system based on big data and artificial intelligence |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2837649B1 (en) * | 2002-03-21 | 2005-02-18 | Stephane Pau | METHOD FOR OPTIMIZING ACCESS TO A NETWORK, SERVER, TERMINAL AND CORRESPONDING COMPUTER PROGRAM |
| CN102986191A (en) * | 2010-07-09 | 2013-03-20 | 诺基亚公司 | Method and apparatus for calculating a probable throughput for a location based at least in part on a received throughput |
Family Cites Families (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6078956A (en) * | 1997-09-08 | 2000-06-20 | International Business Machines Corporation | World wide web end user response time monitor |
| US6157618A (en) * | 1999-01-26 | 2000-12-05 | Microsoft Corporation | Distributed internet user experience monitoring system |
| US6701342B1 (en) * | 1999-12-21 | 2004-03-02 | Agilent Technologies, Inc. | Method and apparatus for processing quality of service measurement data to assess a degree of compliance of internet services with service level agreements |
| US7006980B1 (en) * | 2000-02-04 | 2006-02-28 | Callvision, Inc. | Method and system for selecting optimal commodities based upon business profile and preferences |
| US20020099816A1 (en) * | 2000-04-20 | 2002-07-25 | Quarterman John S. | Internet performance system |
| US7197557B1 (en) * | 2001-05-29 | 2007-03-27 | Keynote Systems, Inc. | Method and system for evaluating quality of service for streaming audio and video |
| US7246101B2 (en) * | 2002-05-16 | 2007-07-17 | Hewlett-Packard Development Company, L.P. | Knowledge-based system and method for reconstructing client web page accesses from captured network packets |
| US20050050557A1 (en) * | 2003-08-28 | 2005-03-03 | Gabryjelski Henry P. | Adaptive multiple concurrent CD/DVD streaming algorithms |
| US7940686B2 (en) * | 2005-02-09 | 2011-05-10 | Microsoft Corporation | Network performance tuner |
| US7593935B2 (en) * | 2006-10-19 | 2009-09-22 | Paxfire | Methods and systems for node ranking based on DNS session data |
| US8667156B2 (en) * | 2007-05-02 | 2014-03-04 | Elevate Technologies Pty Ltd. | Application-independent service delivery |
| US8521891B1 (en) * | 2007-06-21 | 2013-08-27 | Mcafee, Inc. | Network browser system, method, and computer program product for conditionally loading a portion of data from a network based on a data transfer rate |
| US8150992B2 (en) * | 2008-06-18 | 2012-04-03 | Google Inc. | Dynamic media bit rates based on enterprise data transfer policies |
| MY164504A (en) * | 2008-10-03 | 2017-12-29 | Mimos Berhad | Method to assign traffic priority or bandwidth for application at the end users-device |
| US8280996B2 (en) * | 2009-01-29 | 2012-10-02 | The Nielsen Company (Us), Llc | Methods and apparatus to collect broadband market data |
| US20110119370A1 (en) * | 2009-11-17 | 2011-05-19 | Microsoft Corporation | Measuring network performance for cloud services |
| US20110208668A1 (en) * | 2010-02-22 | 2011-08-25 | Google Inc. | Server Provider Recommendation Engine |
| GB2486002A (en) * | 2010-11-30 | 2012-06-06 | Youview Tv Ltd | Media Content Provision |
| US8863204B2 (en) * | 2010-12-20 | 2014-10-14 | Comcast Cable Communications, Llc | Cache management in a video content distribution network |
| US20130081050A1 (en) * | 2011-09-23 | 2013-03-28 | Elwha LLC, a limited liability company of the State of Delaware | Acquiring and transmitting tasks and subtasks to interface devices |
| US8626910B1 (en) * | 2012-06-19 | 2014-01-07 | Edgecast Networks, Inc. | Systems and methods for performing localized server-side monitoring in a content delivery network |
| US9246965B1 (en) * | 2012-09-05 | 2016-01-26 | Conviva Inc. | Source assignment based on network partitioning |
-
2014
- 2014-01-13 US US14/153,956 patent/US10284439B2/en active Active
- 2014-12-02 AU AU2014357345A patent/AU2014357345B2/en active Active
- 2014-12-02 CN CN201480065938.XA patent/CN105794150B/en active Active
- 2014-12-02 EP EP14827889.8A patent/EP3078169B1/en active Active
- 2014-12-02 WO PCT/US2014/068082 patent/WO2015084799A1/en active Application Filing
-
2019
- 2019-05-06 US US16/404,696 patent/US10771355B2/en active Active
-
2020
- 2020-09-03 US US17/012,030 patent/US11743148B2/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2837649B1 (en) * | 2002-03-21 | 2005-02-18 | Stephane Pau | METHOD FOR OPTIMIZING ACCESS TO A NETWORK, SERVER, TERMINAL AND CORRESPONDING COMPUTER PROGRAM |
| CN102986191A (en) * | 2010-07-09 | 2013-03-20 | 诺基亚公司 | Method and apparatus for calculating a probable throughput for a location based at least in part on a received throughput |
Also Published As
| Publication number | Publication date |
|---|---|
| EP3078169A1 (en) | 2016-10-12 |
| WO2015084799A1 (en) | 2015-06-11 |
| CN105794150A (en) | 2016-07-20 |
| AU2014357345B2 (en) | 2017-06-29 |
| US20150156085A1 (en) | 2015-06-04 |
| US11743148B2 (en) | 2023-08-29 |
| US20210105195A1 (en) | 2021-04-08 |
| US20190260650A1 (en) | 2019-08-22 |
| AU2014357345A1 (en) | 2016-05-26 |
| US10771355B2 (en) | 2020-09-08 |
| EP3078169B1 (en) | 2019-07-31 |
| US10284439B2 (en) | 2019-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105794150B (en) | Method for measuring end-to-end Internet application performance | |
| Hoßfeld et al. | Internet video delivery in YouTube: From traffic measurements to quality of experience | |
| CN102244594B (en) | At the networks simulation technology manually and in automatic testing instrument | |
| Orsolic et al. | A framework for in-network QoE monitoring of encrypted video streaming | |
| US10482477B2 (en) | Stratified sampling applied to A/B tests | |
| CN110268717A (en) | The bit rate of more presentation codes is optimized using statistical information is played | |
| US20140108647A1 (en) | User Feedback in Network and Server Monitoring Environments | |
| CN105794175B (en) | Convey the performance metric of the system of WEB content | |
| Basso et al. | Measuring DASH streaming performance from the end users perspective using neubot | |
| EP3427440A1 (en) | Method and scoring node for estimating a user's quality of experience for a delivered service | |
| Añorga et al. | Analysis of YouTube’s traffic adaptation to dynamic environments | |
| EP3926922B1 (en) | Over-the-top media service testing and qoe issues isolation | |
| Zhang et al. | Qoecenter: A visual platform for qoe evaluation of streaming video services | |
| Ben Letaifa | WBQoEMS: Web browsing QoE monitoring system based on prediction algorithms | |
| Suman et al. | Netflix, Amazon Prime, and YouTube: comparative study of streaming infrastructure and strategy | |
| WO2021009134A1 (en) | System and method for estimating the quality of networks or services through crowdsourcing | |
| Jabbar et al. | Implementing a proposal system for video streaming over MANET using NS3, LXC and VLC | |
| Khokhar | Modeling quality of experience of internet video streaming by controlled experimentation and machine learning | |
| WIT et al. | 5G Experimentation Environment for 3rd Party Media Services D2. 9 Continuous QoS/QoE Monitoring Engine Development-Initial | |
| Ghafil et al. | Video Streaming Forecast Quality of Experience-A survey | |
| CN110324608A (en) | Detection method, device and the computer equipment of video flow quality | |
| Richman | CORONAVIRUS PANDEMIC INFLUENCE ON INTERNET USAGE AND PERFORMANCE IN NORWAY | |
| JP2018097394A (en) | Engagement index value controller, engagement index value controlling method, and program | |
| JP2018097393A (en) | Engagement index value controller, engagement index value controlling method, and program | |
| Wu | Utility Maximization of Machine Learning for bandwidth prediction over DASH |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: American California Applicant after: Google limited liability company Address before: American California Applicant before: Google Inc. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |