WO2020229635A1

WO2020229635A1 - Remotely determining rendering metrics for a web-page

Info

Publication number: WO2020229635A1
Application number: PCT/EP2020/063548
Authority: WO
Inventors: Sam CRAWFORD; Johan Norberg
Original assignee: SamKnows Limited
Priority date: 2019-05-16
Filing date: 2020-05-14
Publication date: 2020-11-19
Also published as: GB201906940D0; GB2584821A

Abstract

A web-browsing test system is provided which implements a method for assessing at a test-center the loading performance of a test web-page as it would be viewed on a web- browser running on user equipment at one or more remote network locations. An off-load server having a web-browser first determines what web-page objects are needed to load the test web-page by relaying requests to test-client platforms located at each of the remote network locations. The off-load server then causes each test-client platform to monitor timing information as it retrieves the web-page objects from data sources. The timing information for each web-object the remote test-client has retrieved is then provided to the off-load server which then loads the web-page using a web-browser running on the off-load server. The web-browser on the off-load server then loads the web-page using the timing information for each required web-page object. Web-page objects may be locally cached in advance by the web-browser on the off-load server or provided with the timing information from the client platform. A video of the web-browser loading the web-page for display is created by the off-load server and perceptual rendering performance metrics for the web-page are derived based on at least the time for web-page content to become perceivably displayed by the web-browser.

Description

REMOTELY DETERMINING RENDERING METRICS FOR A WEB-PAGE

[0001] The present disclosure relates to a system, method, and related aspects for remotely testing a web-browsing experience of a web-page by measuring at a test center how the web-page would be remotely rendered. In particular but not exclusively, a system and related aspects are disclosed for testing at a test-center how a web-page would be rendered if requested at one or more remote network locations without the test causing rendering of the web-page on equipment at the one or more remote location. In particular, but not exclusively, a test system is disclosed which provides a test to measure one or more performance characteristics such as objective and/or perceptual metrics associated with rendering a web-page as it would be viewed on a remote device.

[0002] The disclosed test system seeks to evaluate the speed and content of the web-page as it would be browsed by user, for example, a general member of the public using a device at a remote location. A“remote location” means a network location remote from the network location of the web-browser which loads the web-page under test. Some web-pages will dynamically reference objects when they are loaded by a web-browser for rendering on a display. This creates a problem for developers who wish to test how the web-page will be rendered by viewers/users of the web-page if the web-page references code objects and content which could differ depending on the network location, geographic location and time at which the web-page is loaded. Moreover, the data sources from which a code object or content is retrieved by a web- browser to load a web-page may also differ depending on the location of the web- browser and factors such as network congestion and connection quality etc.

[0003] Several techniques are known in the art to allow developers to test their web pages on user equipment in typical scenarios associated with users such as personal computers and tables or similar computer platforms which may be used to browse the internet using a typical residential data communications network service. Some require a web-browser to run on the end user equipment and report back timing information to a central-test centre. However, such systems require the web-page to be rendered remotely which is not suitable for situations where the web-page is to be tested confidentially. Moreover, the display equipment on which the web-page is being rendered is not available whilst the test web-page is being rendered to capture its performance metrics for any other purposes which may be inconvenient for the user of the equipment. The user of the equipment must also give consent and/or control to the test centre to use that user’s display and accordingly this limits such tests to users at locations who are happy to collaborate with the test centre. [0004] Another type of test avoids the issues with rendering a web-page at a remote location on a user’s network by fetching the web-page and rendering the display at the test center. However, such tests must somehow emulate the timing for loading the web page over at a number of remote network sites and the objects referenced by a web page being tested at a test center are not always the same objects and/or from the same data source as they would be if the web-page was being loaded remotely.

[0005] FIG.1 of the accompanying drawings shows an example of a prior art web page test system. The schematic layout shown provides a central test center 16 which tests how a web-page hosted by a web server 2 would be browsed over the internet 4 in a typical browser network environment such as that of a web-browser 8 running on user equipment 46 (user equipment 46 is not explicitly shown in Fig. 1) within a user network 6. Examples of user networks 6 include small commercial or residential networks, for example, wireless or wired broad-band local area networks.

[0006] The web-browser 8 shown in the prior art system of FIG. 1 runs on customer premises equipment (CPE), such as user-equipment 46, which, for example, could comprise a web-browser on a user’s phone, tablet or personal computer or television or the like (for example, on a domestic appliance). CPE provided in the user network accesses the Internet 4 using a connection 12A over a communications link through the access network 14 provided by an internet service provider (ISP) for the user network 6.

[0007] In Figure 1 , when a web-browser (either the test-center web-browser or the web-browser running on user equipment 8) requests a web-page, the address of the server 2 hosting the requested web-page is first resolved, for example by performing a domain name look-up using a DNS serer 18. The web-browser will then request the web-page code from the host server 2 for that web-page.

[0008] In a prior art system of FIG. 1 , remote testing of how the web-page may be loaded by the web-browser 8 in the user network 6 is performed by the test server 16 in a test center by configuring the web-browser running on the test server 16 to emulate the way the web-page would be loaded by the web-browser running on the user equipment. For example, the test center web-browser may try to adjust the timing of various objects referenced by the web-page based on estimates of the network conditions that might be encountered when the web-page under test is loaded in one or more user networks 6. However, this does not help such a prior art test system to address any issues associated with loading the web-page caused by dynamic objects referenced by the web-page as it is loaded by the test-center web-browser. Such objects may cause issues only when the web-page is being loaded and rendered remotely. Issues caused by dynamically referenced objects include issues caused by different objects being retrieved from those retrieved at the test-center server as well as different network paths, and possible the address of the objects differing. For example, a web-page object may be retrieved from a web server #1 by the test center and from a web server #2 by the web-browser. Different types of objects may include different content and/or different object types (for example, still images instead of video) and different formats and/or file types.

[0009] What dynamically referenced objects are retrieved and where they are retrieved from by a web-browser can differ based on the network address of the device hosting the web- browser, its geographic location and/or proximity to a suitable source for the referenced object, and the local time of day. All of these may also affect how long it takes for the object to be retrieved. Also, for example, what network path is taken to retrieve an object referenced by a web-page can vary according to the particular type of object’s requirements and affect the time it takes to retrieve a requested web-page object. The network path can be affected by the particular service a user has signed up for from an ISP, as well as the particular user network, access network and internet conditions along the path which is taken to retrieve the referenced objects. Examples of network conditions which may affect what path is taken include connection speed, latency, and quality and/or the time-of-day.

[00010] Dynamically referenced web-page objects such as code-objects are often included in a web-page text file containing JavaScript™, or similarly referenced code objects. These dynamically referenced objects can, as the web-page is loaded, themselves (i.e. as the objects are loaded), call other objects (and those other objects may themselves require additional objects to be fetched when they are loaded and so on in a cascade of object requests). In particular, the execution of JavaScript™ in a web-page text file may lead to further requests being generated for retrieving objects that can impact the measured performance metrics. Dynamically referenced objects, especially cascades of referenced objects, accordingly, can impact how a web-page is loaded in a manner which is not detectable until a web-page is loaded at a given location. The effect such objects have on measured performance metrics accordingly is usually not capable of being determined in advance of loading the web-page for rendering on a display using known web-browsing tests run at test-centers.

[00011] To better explain the invention, FIG. 2 of the drawings shows a simple summary of some of the steps which occur when a web- browser application 8 on CPE within user network 6 (for example, as shown in FIG. 1) requests a web-page (step 20) as is known in the art. The web-page request may be generated, for example, as a result of user input, such as when a user activates a link to a web-page URL or types a URL into a web- browser’s navigation bar and/or causes a page to be refreshed in a web-browser. Not shown in detail in FIG. 2 are certain steps which immediately occur after the web-page request is generated to obtain all the data to render the web-page on a display, such as requesting information from a domain name server 18 (as shown in FIG. 1) to resolve the IP address of the web-server 2 hosting the web-page file of the requested web-page which are not shown in FIG. 2. Once the address of the web-server 2 hosting the web-page has been determined, a request is sent to the web- server 2 and the web-page file of the web-page is retrieved (step 22). The web-file is typically a .html (Hyper-Text Markup Language) text file, and typically will be retrieved using a HTTP (Hyper-Text Transfer Protocol).

[00012] Once the response has been received (step 22), the response is parsed (step 24) to identify the resources needed for the web-page text file to be rendered on a display. For example, parsing a HTML type of web-page text file may locate referenced web-page objects such as, for example, files such as image and/or video data, cascading style sheets (CSS), other URLs and JavaScript. The web- browser will then seek to retrieve the referenced objects (step 26), for example, the web-browser 8 may need to request additional CSS/JavaScript resources if none are found to be locally cached from relevant web-server(s)/source. The retrieved object code is then be parsed in turn, and the parsing may locate further objects referenced in the parsed object code which may require additional resources and/or reference other objects, all of which may need to be retrieved before any part of the web-page is capable of being displayed. Not shown explicitly in FIG. 2 (but well- known in the art) are any step(s) to resolve the IP address of the web-server(s)_2 hosting any object(s) referenced by the retrieved web- text file.

[00013] The objects which are required to make a web-page initially fully visible (for example, such as the initial web-page view shown in FIG. 4C) may also be referred to herein as“critical path” objects. A critical path object as referred to herein is an object which would either be rendered visible in the initial view of a web-page prior to any user interaction with the web-page or it is an object required to allow one or more other visible or non-visible objects to be loaded to allow the initial view of the web-page to be rendered on a display prior to any user-interaction. The critical path objects for rendering a web-page on a display can vary depending on the characteristics and/or type of web- browser software and/or hardware. For example, certain capabilities and characteristics of web-browser software applications such as Chrome™, Safari™, Edge™ are different. Examples of hardware characteristics which can affect how a web-page may be rendered include general processing power and speed, graphical processing power and speed, cache memory size and access times, and also display characteristics such as screen size as well as pixel size, and colour resolution etc.

[00014] Returning to FIG. 2, once all objects referenced by the critical path for the requested web-page have been received (step 28), they are processed by the web- browser to build (step 30) the initially visible part of a web-page which is then rendered (step 32) on a screen of a display of the hardware platform supporting the web-browser.

[00015] To see the other parts of a web-page rendered on a screen, some form of user-interaction is needed. For example, a user might scroll or pan across the web-page to see the other parts of the web-page away from the web-page. To display the rest of the web-page accordingly, the web-browser may continue to receive additional objects (step 34) referenced directly by the web-page file (or indirectly by another object referenced by the web-page file). These other non-critical path objects may be retrieved for loading the rest of the web-page concurrently or sequentially with code segments that make up the critical page for the initial web-page to load. Once these additional objects have been parsed (step 36) and any other objects referenced retrieved, the remaining web-page can be build (step 38) and then be suitably rendered (step 40) on the display according to whatever user input is detected.

[00016] United States patent application US2011/0054878 discloses an automated cloud service performance prediction which includes extracting a parental dependency graph for a web-page. The parental dependency graph encapsulates one or more dependency relationships for each web object in the web-page. The prediction further includes determining an original page load time (PLT) and original timing information of a webpage. The prediction also includes simulating a page loading of the webpage based on adjusted timing information of each web object and the PDG to estimate a new page loading time of the webpage. The prediction additionally includes comparing the original PLT of the webpage to the new PLT of the webpage to determine whether the adjusted timing information increased or decreased the new PLT of the webpage.

[00017] For example, in United States Patent US7600014 a method for determining one or more performance metrics for a distributed application is disclosed in which distributed application data is transferred from a first site to a second site over a network. The method disclosed comprises: receiving a request from a user for the distributed application data, the request being transmitted from the second site to the first site over the network; including machine instructions with the distributed application data that define a performance monitoring function, the machine instructions and the distributed application data being a single data file; including a monitor cookie with the distributed application data that indicates that the distributed application data are a monitored object; transmitting, in response to the request, the distributed application data from the first site to the second site over the network; detecting the monitor cookie at the second site; executing, in response to the monitor cookie being detected, the machine instructions at the second site to implement the performance monitoring function used to determine the one or more performance metrics for the distributed application without using the performance monitoring function to request any distributed application data from any site, at least one of the one or more performance metrics being determined in connection with timing of events occurring during the transmission of the distributed application data to the second site; determining a performance metric at the first site; and combining at least one of the one or more performance metrics determined at the second site with the performance metric determined at the first site to determine a correlated performance metric.

SUMMARY

[00018] The following provides a simplified summary of the disclosed technology in order to provide a basic understanding to the reader. This summary is not an extensive overview of the disclosure and it does not identify key/critical elements or delineate the scope of the specification. Its sole purpose is to present a selection of the concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later.

[00019] The preferred aspects and example embodiments of the invention are set out in the accompanying independent and dependent claims.

[00020] Examples of the technology disclosed herein seek to mitigate and/or obviate the problems associated with accurately testing how a web-page would be loaded and rendered at a number of remote locations such as those typically associated with users who are members of the public. The tests disclosed use a web-browser at a test center to render a web-page using the same content and/or code objects from the same data sources as those used to load and render the web-page on a display at one or more remote locations.

[00021] Examples of the technology disclosed herein seek to improve the accuracy of existing rendering metrics. Moreover, examples of the technology disclosed herein do not require the web-page being tested to be displayed at any of the remote locations for which the rendering metrics are being generated. Instead, equipment based the remote network (also referred to as a client network) captures timing information for retrieving various objects the web-page requires in order to be rendered. This timing information is then provided to a remote test center server which uses the timing information to load the web-page and determine rendering metrics for the loading web-page. [00022] Some embodiments of the invention seek to remotely measure the time taken to load and render a web-page comprising a plurality of retrieved web-page objects including at least one dynamically referenced object on a remotely located client-device without requiring a web-browser to be executing on the remote device. The web-browser runs instead on a remote test-center server which first loads the web-page to determine what objects may be dynamically referenced by the web-page. Information on the dynamically reference web-page objects is provided to one or more remote client devices. To ensure that the dynamically referenced web-page objects are the same as those which would be retrieved if the web-browser were running on a client device at the same remote location, a tunnel is established between the test center server and the remote client device. Once address information indicating the identity and data source for any dynamically referenced web-page objects has been determined by the test- center server, the web-server provides the address information to the remote client device which then measures how long it takes for the client device to retrieved the dynamically referenced web-objects. The timing information for the dynamically referenced web-objects is then provided to the remote test-center server which uses the timing information to control when the dynamically referenced web-page objects are loaded by the web-browser running at the test-center. In this way, the web-browser running at the test center can replicate the time to load the same dynamically referenced objects and render the loaded web-page in the same way that the web-page would be loaded at the remote location without requiring a web-browser to render the web-page at the remote location.

[00023] One possible benefit of the preferred embodiments is that timing information to retrieve the web-objects is measured based on the actual data path the web-objects take. Another possible benefit is that it is possible to provide a consistent scalable web page test replicating the rendering of the web-page at a number of different remote. A benefit of a highly scalable test system which is capable of enabling a single developer/test center server to selectively instruct a large number (for example, of the order of thousands to tens of thousands or more) of web-test clients at selected locations all over the world is that the retrieval of web-page code-objects and/or content by the web-test clients can be coordinated to occur simultaneously at a selected plurality of or all of the remote locations an/d/or to occur sequentially, or, for example, to occur at the same local time. This may be beneficial where the web-page is providing live or near-live streamed content for an event for example, where a large number of users may wish to browse the web-page concurrently.

[00024] Some embodiments of the invention provide a distributed web-browsing test system comprising at least a client-side web-test component and a server-side web-test component. In some embodiments, the client-side web-test runs on a customer premises equipment (CPE) hardware platform having a different primary purpose, for example, it may be embedded into a router on the customer premises. Such routers are also commonly referred to as access hubs when they are configured to provide a wireless and/or a wired user network servicing equipment located in the customer premises with access over an Internet Service Provider’s (ISP’s) network to other networks, for example, to the Internet. ISP router devices are often provided as part of the services provided by the ISP, for example, as part of a broadband data communications package and the like. In some embodiments, the client-side web-test software is embedded into the router platform. In other embodiments, the client-side web-test software is provided on a separate hardware platform configured in use to communicate with the server-side of the web-test via the user’s network router.

[00025] Various examples of a web-browsing test system will now be described where the purpose of the web test performed seeks to capture the user experience of browsing by simulating testing inside the user's own network. To get realistic results remote testing is done with a real browser engine. Since many web sites include web resources, for example, code objects, that have no visual impact measuring only transfer times would not be enough. The same goes with JavaScriptinterpretation that could delay the presentation. For that reason, the web-browsing test system measures the visual load progress of a web-page so as to emulate more closely how a real user would perceive the progress of loading the web-page on a display.

[00026] One example of an embodiment of a web-browsing test system uses a client- side web-test agent located on hardware which is available inside the CPE, such as the ISP router equipment or on similar purpose-built web-test hardware. Known router equipment is not powerful enough to run a web-browser (and usually will have no graphics processing capabilities). CPE such as, for example, routers usually will not accordingly be able to cause a web-page to be displayed on a display located in the user network, meaning that known client-side web-test agents which are not running on a user’s laptop/PC cannot realistically emulate the actual browsing experience of on the user’s hardware (e.g. on a laptop/PC).

[00027] To obviate and/or mitigate the problems known in the art associated with web browsing test systems, a distributed web-browsing test system according to various embodiments of the invention provides at least one client-side web-test component comprising a client-side web-test agent hosted on CPE, for example, a router or similar equipment, which is configured to communicate with server-side web-test components comprising at least one or more off-load servers. In some embodiments, a test server is also provided. The test server may be implemented at a hardware level by the same platform that is used to implement on or more of the off-load servers. The client-side and server-side components of the distributed web-browsing test system co-operate to perform testing of a web-page to enable timing information to be obtained which enables a representation of the actual browsing experience a user would perceive on their own devices within a user network to be remotely emulated.

[00028] In some embodiments, each off-load server is able to co-operate with one or more web-test agents running on the client sides of different user networks. Some examples of a client-side web-test agent are hosted by user network router equipment. Other examples of a client-side web-test agent use legacy hardware in other CPE. For example, in some embodiments, the client-side web-test agent may be provided as a software up-date to a legacy hardware measure probe device already installed and operational in a user network on the customer premises.

[00029] An example of a hardware measurement test client-side device known in the art is a SamKnows™ WhiteBox, which may be suitably reconfigured to function as a client-side web-test agent. Examples of client-side web-test agents include functionality to act as a client-side measurement probe for capturing timing information for performing a web-test according to some embodiments of the invention. In some embodiments, the web-test off-load server is configured to communicate an indication to the client-side web-test agent code objects which are referenced dynamically by a web-browser when rendering an initial view of the web-page (as well as all other code-objects which are otherwise referenced by the web-page, for example, which are directly or statically referenced by the web-page’s text file). In such embodiments, the client-side web-test agent captures timing information indicating the amount of time it takes to retrieve all the objects referenced by the web-page for rendering at least the initial view of a web-page. This timing information is then provided back to the off-load server which then uses this to cause a web-browser running on the off-load server to emulate the timing which would be experienced by a web-browser running on the client side when rendering the first initial view of the web-page (also referred to herein as the critical path) view of the web-page.

[00030] The test results obtained by using a web-browsing test system according to some embodiments of the invention are accessible by a test center platform which provides test analytics & test result presentation. In other embodiments of the invention, an adapted web browsing section is provided by the test center which enables page load investigations. [00031] LIST OF FIGURES IN THE ACCOMPANYING DRAWINGS

[00032] Various examples of embodiments of different aspects of the invention will now be described in more detail referring to the accompanying drawings by way of example only, and in which:

FIG. 1 shows a first schematic view of a test environment according to a known web browsing test system;

FIG. 2 shows an example process illustrating how a requested web-page is rendered on a display;

FIG. 3 shows schematically a web-browsing test system for providing performance metrics of a test web-page according to an example of an embodiment of the invention;

FIGS. 4A to 4C shows schematically timing measurements of how a web-page is rendered over time.

Fig. 5A shows schematically different phases of an example embodiment of a web- test according to the invention;

FIG. 5B show steps in an embodiment of a method of testing a web-browser loading performance of a test web-page;

FIGs. 6A to 6D show more detailed steps in an embodiment of a method of testing a web-browser loading performance of a test web-page;

FIG. 7 shows the layered model and linkage and execution context of a web-browser test system according to an embodiment of the invention;

FIGs. 8A to 8E show schematically a message sequence diagram for a test-session of a web- browser test system according to an embodiment of the invention;

FIG. 9A shows example timing measurements for the time to visually complete loading a web- page on a web-test browser according to an embodiment of a web browsing test system for different ISPs; and

FIG. 9B shows example timing measurements for the time to visually complete loading a web- page on different web-test browsers according to an embodiment of a web-browsing test system; and

FIG. 9C shows example timing measurements for the time to visually complete loading different web-pages on a web-test browser according to an embodiment of the invention.

DETAILED DESCRIPTION

[00033] The detailed description provided below along with the accompanying drawings is intended as a description of one or more of the possible embodiments of the invention. It is not intended to represent the only forms in which the embodiments may be constructed or utilized. The description sets forth the structural and functional features of some embodiments of the invention, and includes operational information including one or more examples of possible sequences of some operational steps. Nonetheless, the same or equivalent structural and functional features, and operations, may be accomplished by different embodiments of the invention not described in detail below. Unless specifically excluded, features which are recited in connection with one embodiment may be incorporated additionally or instead of features which are recited in connection with another embodiment, when this would be an apparent possibility to someone of ordinary skill in the art. Any sequence of steps described herein is limited to the specific order indicated only which such a specific order is necessary as a relationship exists between a condition or variable and a particular consequence such that one event or step described in the sequence leads to one or more other events or steps. The steps of the methods described herein may accordingly be carried out in any suitable order, or simultaneously where appropriate. Additionally, individual blocks may be deleted from any of the methods without departing from the spirit and scope of the subject-matter described herein.

[00034] Although some of the presented examples of embodiments of the invention are described and illustrated as comprising a client-side web-test agent being implemented as a software application within a device such as a router, this is only one example of CPE within which the client-side web-test agent can be hosted and the client- side web-test agent may reside in other CPE having access to appropriate communications functionality similar to that provided by a router in other embodiments of the invention. As those skilled in the art will appreciate, the presented examples of embodiments of the invention described are suitable for application in a variety of devices connected to a user network, including a suitably connected stand-alone test- device such as a SamKnows™ White box.

[00035] In the detailed description of the invention referring to the drawings which now follows like elements may be referred to using the same reference numbers. In the drawings, certain features which are known to those of ordinary skill in the art may be omitted for the sake of clarity where their presence in any physical implementation would be an obvious necessity, and their description may also be omitted for the sake of brevity.

[00036] For example, the invention seeks to emulate remotely how a web-page would be rendered using one or more web-browsers running on user equipment with user- networks. The term user-network as used herein refers to any suitable type of communications network within which a web-test client is located which is remote from the communications network within which the web-test server resides. Examples of user-networks include residential and commercial types of wired or wireless local area network, for example, a WiFi network or the like. The term“Internet” is used for any remote network or system of networks interconnectable to such a user network over which a web-page may be retrieved from a web- server or host server. A web-page under test (and /or any code objects and/or content referenced by a web-page under test) may be retrieved using any suitable address format, for example, an Internet Protocol address, such as IPv4 or IPv6.

[00037] Some embodiments of the invention seek to mitigate or obviate any potential network address translation (NAT) issues which might otherwise cause address look-up issues when executing a web-test by configuring the client-side web-test agent to provide an indication to an off-load server of a web-page to be tested, which in some embodiments is achieved by establishing a tunnel between the off-load server and the device in the user network on which the web-test client application is running. In some embodiments, an indication of a web-page to be tested is provided by a remote test- center to a plurality of client-side web-test agents located in different user networks, and each client-side web-test agent then communicates, using a respective tunnel established with an off-load server, timing information and/or code-object and related details of the web-page to be tested via the tunnel to the off-load server.

[00038] The location of any data source for data required to load a web-page is sometimes dependent on where the user equipment hosting web-browser on which the web-page is to be rendered is located and/or what NAT is performed at that user equipment’s location. The web-page tests according to various embodiments disclosed herein account for any NAT performed for the remote user equipment even when there is no explicit reference to NAT in the description of a particular embodiment. Embodiments of the web-test system disclosed herein provide a distributed web-test system where a web-page is rendered using a web-browser located in a test center. The web-page is loaded using web-objects retrieved by the web-browser in the test center which are the same web-objects as those a web-test client agent in a user network would retrieve. This ensures the correct type of objects are used to load the web-page and allows the relationships between the objects referenced by the web-page under test to be maintained. Timing information to retrieve the objects referenced by the web-page (by which is also meant the timing information to retrieve any objects referenced by such objects and so on) is captured by the web-test client agent in the user network and provided to a remote test-center server, or off-load server without causing the web-page under test to be rendered on any equipment within the user network. The test center web-browser uses the timing information for the retrieval of objects referenced by the web-page under test which the remote web-test client agent captures to determine when each object should be loaded for rendering the web-page on a display at the test center. This then allows the test-center to capture various performance metrics associated with how long the web-page under test takes to load.

[00039] Even if a web-test client agent is installed on user-equipment which is being used at the time of the test by a user in the user network, as the web-page under test does not need to be rendered on a display in the user network, the user does not need to be aware the test is taking place. To ensure that the web-test takes into account how all of the objects, especially dynamically referenced objects, which make up a web page and which can affect the visual experience of a user when they view the web-page on a real web-browser, embodiments of a test-system test server (also referred to here as off-load server) according to the invention first determine what objects would be called by ensuring that all requests for object which are sent out from the test center network appear to come from the user network where the test client is located, for example, by tunnelling any such requests for web-page objects that the test-center server generates so they appear to originate from a web-browser running on a device located in a user network 6.

[00040] The term web-page referenced object or“web-object” or“object” is used to refer to a code object including content, and includes objects having a source address which is different from that of the device on which the web-test client agent is running in the user network (which will usually be, but need not be, different from the network address of the web-page test server also referred to herein as an off-load server).

[00041] FIG. 3 of the accompanying drawings shows an example of a distributed web browsing test system according to some embodiments of the invention. Some examples of the distributed web- browsing test system comprise on the client-side one or more user networks 6, each user network 6 including at least one client-side web-test agents 50 (only one user network 6 and client-side web-test agent 50 is shown in FIG. 4) and on the server-side a test center server 54 and one or more off-load servers 52. Although shown separately in Figure 3, in some embodiments of the invention, the test-center server 54 and off-load servers 52 are the same device and/or software platform.

[00042] The distributed web-browsing test system shown in FIG. 3 is configured to emulate using a test-center off-load server 52 the time taken to load a web-page on a display associated with user equipment 46a or 46b for example, by a web-browser 8a or 8b. More than one type of web-browser may run in different or the same user network 6, for example, as shown in FIG. 3, another device 46b has a different web-browser application 8b running on it. This may use the same display and/or another display in practice for rendering a web-page 62 (see FIG. 4C).

[00043] The web-browsing process is emulated by client-side web-test agent 50 working co-operatively with an off-load server 52. In some embodiments, the client-side web-test agent 50 receives information from a test center server 54 about a web-page 62 to be tested and initiates a test session with an off-load server 52 by sending an indication to the off-load server 52 of the web-page to be tested. The off-load server 52 then loads the web-page to find out what code objects are referenced by the web-page and sends a list of the referenced code objects back to the client-side web-test agent so that the client-side-web-test agent can work through list of objects and time how long it takes to retrieve each listed code object associated with the web-page. In other embodiments, the client-side web- test agent 50 may be configured to automatically start to capture timing information for retrieving some types of data required to load the web-page of its own volition so that it is only necessary for the off-load server to determine any objects that the client-side web-test agent would not be otherwise aware of. Examples of such objects include objects which are dynamically referenced by the web-page as it is loaded for rendering, as the client-side web-test agent is configured to only retrieve objects and does not attempt to load the web-page for rendering itself. The client-side web-test agent 50 only sends timing information for the objects it has retrieved (or an indication it was not able to retrieve an object) to the off-load server 52 of the web-browsing test system.

[00044] The off-load sever 52 of the web-browsing test system then uses the timing- information for retrieving code components or objects measured on the client-side by the web test agent 50 to adjust each object’s time of retrieval by a web-browser running on the off-load server 52 so that the web-browser will load the objects of the web-page in a way that replicates the timing of the way the objects would be loaded by a web-browser, for example, such as one of web-browsers 8a, 8b would load the web-page from within the user network 6. The off-load server 6 and/or the test center 54 shown in Fig. 3 can then determine performance metrics for a test web-page related to how the web-page be viewed on the client-side user network 6. The term client-side is used herein to refer to the network environment supported by the user network 6, for example, the network environment provided by a suitable wired or wireless local area network (LAN) 6 hosted by a router 10 on the customer premises according to some examples of embodiments of the invention.

[00045] In the example embodiment shown schematically in FIG. 3, client-side web- test agent 50 is embedded software within an ISP router 10 located in the customer premises. Alternatively, the client-side web-test agent 50 may comprise software hosted within the router 10 or other CPE in the user network 6, for example, on a separate hardware platform communicatively connected to the remote off-load sever 52 via router 10. In some embodiments, the functionality provided by the client-side web-test agent 50 software may be implemented in an existing router and/or separate hardware platform as a remote software installation or as an update to an existing client- side web-test agents.

[00046] In the example of a web-browsing test system as shown in FIG. 3, the CPE local area network (LAN) 6 may be a wired and/or wireless end-user network which may be supported by the router 10. Router 10 also provides access to the internet 4 using a suitable data connection 12A over ISP network 14.

[00047] As shown schematically in FIG. 3 various different client-side devices 46a, b will provide hardware platforms on which one or more web-browser applications 8a, 8b can run which enable a user to browse the internet. The client size devices 8a, 8b may have different types of capabilities including one or more different web-browser applications, differently types of display technology, and screen sizes and resolutions etc., different graphics processing resources, different memory and use different wired and/or wireless communications protocols (e.g. 802.11a, 802.11h, Bluetooth™) etc. when communicating within the user network 6 and over the.

[00048] In some embodiments of the web-browsing test system, the web-browser running on an off-load server 52 may be adapted to emulate individually one or more different types of web-browsers 8a, 8b which may be used on a client-device 46. Examples of user equipment 46a, 46b offering web-browsing functionality include mobile phones, tablets, other hand-held devices having a web-browser display, including Kindles and the like, personal desktop and gaming systems, laptop computers, televisions, and some domestic appliances (for example, smart-refrigerators). Displays may be integrated or separate, and a web-browser may be run on a device such as a mobile phone with the rendered image being mirrored on another larger screen device or only viewed remotely. Accordingly, the same web-page 62 may render very differently for user interaction on each of the client-side devices 46a, 46b located within a single CPE, and also be affected by location and available bandwidth, network congestion and other types of disruption from the way other CPE equipment in other user networks.

[00049] In some embodiments of a distributed web-browsing test system, a web-test server 54 is connected to the Internet 4 using a communications link 12B. Web-test server 54 is configured to use one or a plurality of off-load servers 52, each connected to the Internet using a communications link 12, which are each configured and/or otherwise gathers test-session information from a large number of client-side web-test agents 50. This type of distributed web- browsing test system allows timing information to be obtained from many different types of user networks (only one user network 6 is shown in FIG. 4) and the aggregated test-session results can be statistically processed to obtain more information about how a web-page 62 (see, for example, FIGS. 4A to 4C) would be viewed on client-side web-browsers 8.

[00050] In some embodiments, test-sessions are repeatedly performed by the same client-side web-test agent 52 and the test-result data can then be analysed. Aggregating test-session result data provided using measurements from a large number of web-test agents 52 allows the performance of a web-page 62 to be assessed statistically, for example, over a range of different static and dynamic conditions. This allows insights to be provided about the web-browsing experience of different users requesting a web page 62 which takes into account one or more of : the types of user devices 46 and related displays, where the user-network 10 is located geographically, the type of user networks 6 and what network conditions are being encountered (traffic volumes, delays etc.) at different times. In some embodiments, the different types of client-side web- browsers 8 which may be used by user equipment 46 can also be taken into account by repeating the test using different web-browsers on the off-load servers 52 , in case the use of a particular web-browser affects what objects need to be referenced by a particular web-browser.

[00051] As mentioned above referring to FIG. 2, in the embodiment shown in FIG. 3 of the drawings, when a web-browser 8a, 8b, running on one of client-side device 46a, 46b connected to the user network 6 requests a web-page, the request is forwarded over the ISP network 14 via the router 10.

[00052] The process for rendering the initial view of the requested web-page will follow that shown schematically in FIG. 2., however, not shown in FIG. 2, but often occurring in practice, is any domain name resolution required to resolve the IP address of any code components or objects associated with the requested web-page or NAT translation. Example embodiments of a web-test according to the invention take into account both NAT and DNS look-up times. In some embodiments, the client-side web- test agent 50 initiates requesting the code-components associated with a particular web page which avoids NAT issues which would otherwise cause problems.

[00053] The code-components of a web-page 62 typically comprise a text file for the web-page 62 and a number of objects referenced either by the web-page text file or by another object which itself is referenced by the web-page text-file and so on, until at least all critical path objects for rending an initial view of a web-page 62 have been obtained). The referenced objects may be hosted on one or more different web servers 2 from a web-server 2 which hosts the web-page text file and it may also be necessary to communicate with one or more different domain name servers 18 than the domain server 18 used to resolve the original URL of the web-page 62.

[00054] However, when a web-page is tested using a web-browsing test system according to an embodiment of the invention, such as FIG. 3 shows schematically, the process shown in FIG. 2 is no longer followed as the client-side web-test agent 50 has no ability to actually render the web- page itself. Instead, software runs on the client-side web-test agent 50 when it is participating in a web-page testing session to emulate the data retrieval functionality of a web-browser for retrieving the code-objects referenced by the web-page text file (and the retrieval of the web-page itself). The test-session determines the way the web-page could be loaded when viewed on a web-browser from within the user network 6 by instead, using a web-browser (also referred to herein as a web-browser) running on one of the off-load servers 52 shown in FIG. 3. The off-load servers 52 co-operates with the client-side web-test agent 50 (and in some embodiments, with web-test server platform 54) to implement a test-session for capturing performance metrics for a web-page. The detailed functionality of the off-load servers 52 and their role in the web-browsing test according to various embodiments of the invention is described in more detail herein below. In some embodiments, test server platform 54 is located in the same test center as the off-load servers and in some embodiments (not shown in FIG. 3) the web-browsing test off-load servers 52 are remotely located from the test server platform 54.

[00055] In some embodiments, the test server platform 54 includes data storage facilities for storing the web-loading performance metrics data gathered in web-tests and also test performance files (e.g. for the video test results) which the test system generates however, in some embodiments, the data may be stored elsewhere in the test system and/or in one or more of the off-load servers 52.

[00056] Example embodiments of the web-browsing test system shown in FIG. 3 use off-load servers 52 to emulate the end-user experience of browsing using data gathered from inside the end-user's network 2 by the client-side web-test agent 50, such as timing information data which indicates how long it takes the client-side web-test agent to capture the objects required to load at least the critical path or first initial view of the web-page being tested on at least one display device.

[00057] The term web-browser as used herein refers to a computer application or computer program that renders a web-page step by step as more objects referenced by the web-page are received from one or more remote data sources over the Internet and cause the web-page to be presented on a screen as it receives the data (see also the process shown in FIG. 2). Some embodiments of the web-browsing test system described herein measure what is observable on the screen as a test web-page is loaded. In this way, for example, one or more performance metrics can be provided which indicate the loading performance of the web-page, for example, how much waiting time is needed for a screen to be rendered which shows the web-page and allows the contents of the page to be interacted with. The performance metrics may also distinguish based on web-browser type and/or aspects of the rendering process which depend on the display characteristics used by the web-page loading test (for example, screen size, image resolution (HD, 4K), etc.)

[00058] As web-pages may reference objects which have no visual relevance, for example, an object may be invisible to the user (e.g. an advertising beacon), while some objects have less impact on the visuals, the embodiments of the invention capture a “visual loading progress” which is a useful perceptual metric as it indicates the extent to which a page is being presented in a manner visible to an end-user throughout the time, usually a few seconds, that a page is loading.

[00059] One example of a performance metric which is captured by embodiments of the invention is the is the duration of the overall page load process and its visual implications, however, other examples of performance metrics comprise object-specific performance details. Object-specific statistics can indicate, for example, why a page load is broken or slow (e.g. a slow-to-download JavaScript file may slow down the entire page load, but a slow-to-download GIF may not). Many websites require access to multiple web servers hosted with different providers in different physical locations and a slow-down or issue with just one of these could significantly impact page loading performance. Some objects (e.g. HTML, CSS or JavaScript) have other objects as dependencies which further increases the page load time if they are the bottleneck. The web-browsing test system of the example embodiments enables where delays are originating to be determined. By identifying where delays are happening, for example at a server and/or within a network, as a web-page is being loaded, it is possible to improve the web-browsing experience of the end-user by removing the cause of delay and/or selectively improving the network performance and/or server performance.

[00060] In some embodiments, the client-side web-test agent 50 is implemented in a broadband router 10 as shown in FIG. 3, for example, it may be implemented in an embedded Linux device. The client-side web-test agents 50 hosted on such devices have no access to sufficient processing power or processing resources to run a full blown web browser, nor can they access a screen to render the web-page being tested on.

[00061] As shown in FIG. 3, the web-test offload server 52 allows a remote testing of the browsing experience based on how it would be if tested from within the user's own network 6. The offload server 52 is configured to run an actual browser whereas the client-side web-test agent 50 performs time-based performance tests to measure timing between the user’s home network 6 and various of the web servers 2 as the code components (the web-page text file and/or any static referenced objects and/or dynamically referenced objects indicated by the off-load server) required to render at least the critical path view of the web-page are retrieved. Examples of timing measurements include but are not limited to domain name server 18 (DNS) resolution time, transport connection protocol (TCP) connection time, secure socket layer/transport layer service (SSL/TLS) handshake time, object transfer time. These timings are sent to the offload server 54 which then uses the same measured timings to load the same web page with all its objects exactly according to the timings measured from the user's network for use in emulating the web-browsing experience at the CPE which is described in more detail later herein below.

[00062] The use of off-load servers 52 enables a large-scale web-test to be provided that captures the visual experience of remote users. In at least one embodiment of the web-browsing test system according to the invention , the test environment uses only the hardware of each user’s ISP router 10 installed on the end-user’s premises into which the test-system router software development kit (SDK) is integrated. It is only possible to emulate an actual user’s experience on a web- browser running on a device having a particular IP address if the connections used by that device can be emulated as they originated from the client-side. As some user’s networks may have significant bandwidth limitations in the uplink direction, a shared cache is used in some embodiments of the web-browsing test system to keep the test duration as short as possible.

[00063] In the example web-page test system shown in FIG. 3, several off-load servers 52 are illustrated, however, each client- side web-test client 50 connects to just one of the off-load servers 52. In some embodiments, which off-load server 52 a client- side web-test agent 50 connects to is predetermined but in some embodiments, an off load server 52 is dynamically allocated to a client-side web-test agent. For example, the off-load server 52 used may be randomly selected for load balancing by the central test system server 54 and/or be a failover server if the off-load server 52 which would have been the normal server selected is not available at that point in time (for example, if it is overloaded with requests and/or is unreachable due to a network connectivity issue).

[00064] As shown later in FIGs. 6A to 6D , in some embodiments of a web-browsing test system according to the invention, the web-browsing test system uses communications between the client-side web-test agent 50 and the off-load server 52 which are based on a proprietary Web-Test Control Protocol (WTCP), in which a single TCP connection is used to send and receive messages with multiplexing multiple socket connections being used to relay the browser’s socket on the off-load server to the real Web-Server via the Test Client Platform. Compression is used in some embodiments to reduce the bandwidth used over the socket connections. More details of the communications established are provided later herein below.

[00065] As web servers 2 may produce content differently depending on the geographical location of the user, in various embodiments of the invention, the offload server 52 never talks directly to any web server (e.g. to decide which objects to measure). Instead a tunnel is established by the client- side web-test agent 50 between the router 10 located in the user network 6 and the offload server 52 so that all traffic originates from the user network 6.

[00066] In some embodiments, to emulate the timings experienced by a user browsing on a user network 6, the offload server 52 uses an instrumented web browser that fetches its content from an artificially delayed local cache (not shown in FIG. 3). The data fetches are delayed according to the timings obtained from the client-side web-test agent 50 running in router 10 in the user network 6). The result is a web browsing measurement that gives results that depict exactly what a real user would see if they loaded the same web page 62 on their broadband connection 8 in that user network 6.

[00067] In some embodiments, the web-browsing test system shown in FIG. 3 is scaled-up to include thousands of end-user networks 6 enabling each test’s timing measurements captured by the respective client-side web-test agent 50 to generate a set of results which can be statistically analysed to obtain insights for a web-page. This allows statistical results to be obtained for a web-page which represent, for example, the average browsing experience over a selected group of user-networks, for example, by geographical area and/or by an ISP, as the client-side web-test agent software may be run on a number of different broadband routers and broadband connections/networks. In some embodiments of the web-browsing test system, web tests for a web-page 62 are performed periodically and/or at designated times, by a large number of user networks (>1000) so that any difference in web-browsing performance for a particular web-page can be monitored over a duration of time. [00068] Some embodiments of the invention provide a web-page test system for remotely testing how a web-page loads in at least one user network 6 which comprises at least one client-side web-test agent 50 located in each user network 6 and at least one off-load server 52. In some embodiments of the web- browsing test system further comprise at least one test- server 54 which sends out an indication of the web-page 62 to be tested to each of the client-side web-test agents 50, and each client-side web-test agent 50 in turn notifies an off-load server 52 of the web-page to be tested. Each client- side web-test agent 50 is configured to capture timing information associated with retrieving one or more code objects associated with the web-page text-file to be tested from one or more data sources (typically web-servers for the web-page, or servers hosting data objects referenced by the web-page text file or by a data object within the web-page text file). In some embodiments, each of the off-load servers 52 is configured to receive an indication of the web- page to be tested from each client-side web-test agent. The off-load server 52 comprises a web- browser and is configured to receive and process requests from a client-side web-test agent to perform a performance test for loading a web-page. The web-browser performs an initial load of the web-page text file which identifies one or more code objects referenced by the web-page and one or more sources for the identified one or more code objects. This process is described in more detail below. Some web-pages will include at least one referenced code-object which is only dynamically referenced by the web-page as the web-page is loaded by the web-browser running on the off-load server. For example, the web-page may include a JavaScript™ code-object or a cascaded style sheet (CSS) code object. The off-load server 52 then provides an indication of the web-page text file and at least each of the dynamically referenced code-object back to the client-side web-test agent. In some embodiments a corresponding source address from which the dynamically referenced code-object can be retrieved is also provided to the client- side web-test agent. Alternatively, in some embodiments, the client-side web-test agent 50 is configure to look-up a source address for a code object indicated by off-load server 52. In some embodiments, the client-side web-test agent 50 may retrieve a web-page text file prior to or in parallel with providing an indication of an associated web-page to be tested to the off-load server 52. This web-page text may be stored in a local memory so that the off load server 52 does not have to provide an indication of any statically referenced objects or data sources in the web-text file.

[00069] The indication of the web-page sent by the client-side web-test agent 50 to the off-load server 52 may comprise a request for the off-load server 52 to execute a web-page test, which causes the off-load server 52 to perform an initial load of the web- page. The off-load web-browser then loads the web-page and identifies all of the code objects referenced by the web-page text-file, including one or more code objects which are dynamically referenced directed by the web-page text-file or indirectly by an object referenced (dynamically or statically) by the web-page text file. An indication of the code-components (the web-page text file and the referenced code objects) are then returned to the client-side web-test agent in some embodiments of the invention, although, as mentioned above, in some embodiments the off-load server 52 may only return an indication of code-objects which are dynamically referenced as the actual web page is loaded.

[00070] The client-side web-test agent then seeks to retrieve all the code-components associated with the web-page, including the web-page text-file and the code-objects statically and dynamically referenced by the web-page and captures timing information as each code component is retrieved from its source. The timing information captured for each retrieved code-component associated with the web-page is then sent to the off load server 52. The off-load server 52 is configured to receive the timing information captured by the client-side web-test agent 52 for each referenced code component, including timing information for code objects which are only dynamically referenced by web-page, and uses this timing information to time how it re-loads the web-page, so that each code-component is rendered using the timing information captured by the web-test agent when it sought to retrieve that code-component from its source. In this way, the off-load server web-browser is able to take into account timing information for all code components associated with a web-page, both statically referenced code-components and dynamically referenced code-objects. This allows the off-load server to more accurately emulate the rendering process of the web-page as it would be performed by a web-browser running within the user network, even though no client-side web-browser is used in the test.

[00071] Some embodiments of the invention provide a web-browsing test system for remotely replicating the rendering process of a web-page as it would be presented on a display in a user network 6, where the web-browsing test system comprises at least one client-side web-test agent 50 located in a user-network 6, at least one server-side off load server 52 remotely located from the user- network 6, and a test- server 54 remotely located from the user-network. The client-side web-test agent is configured to receive an indication of a web-page, capture at least data source and related data retrieval timing information for retrieving code objects, also referred to herein as code components, required to render an initial view of the indicated web-page on a display located in the user-network, and send at least the timing and source information for the indicated web- page to one of the at least one off-load servers. The off-load server 52 is configured to receive the at least data source and related data retrieval timing information for the code components required to render the web-page from the at least one client-side web-test agent 50, retrieve the code components from the data sources indicated by the received data source information for the web-page, locally cache the received code components, build the web-page for display by retrieving the locally cached code-components using the received retrieval timing information; and render the web-page on a display (not shown in the drawings, any suitable display may be used). The code components for the indicated web-page comprise at least a web- page text file and one or more code components or objects referenced by the web-page file. The timing information comprises time-stamps for when each code-object was requested and received, including DNS look-up times for each object. The one or more objects can comprise, for example, JavaScript™ objects, CSS objects, image file objects; video file objects; and URL objects for other web-pages/data sources. The timing information also includes timing information for each object referenced directly or indirectly in the indicated web page test file in the form of a time-stamp associated with when the object was requested and received. Some embodiments capture timing information for objects requested and/or received in parallel.

[00072] The web-browsing test system can capture different types of performance metrics as results for each execution of the web-test or web-test session. In some example embodiments, the web-test outputs three different types of data results when testing a web-page. One type of data output from a web-test according to some embodiments of the invention comprises test-result data, which is suitably stored, for example, in a database, to allows queries to be performed on the test result data. This data provides both page-level information and per-object information for the web- page. Another type of data output from a web-test according to some embodiments of the invention comprises a video file capturing the page load process of the web-page as it is rendered on a screen. Another type of data output from a web-test according to some embodiments of the invention comprises a HTTP Archive format (or HAR) type or similar JSON-formatted archive file, which captures the web-browser’s inaction with the individual objects of the web-page, and provides timing information etc. for how individual objects are loaded which allows objects which cause bottle-necks and/or errors when loading the web-page to be identified.

[00073] For each individual test session, the web-test results for a web-page include the universal resource locator (URL) for the web-page being tested, the time-stamp information for each measurement taken of a code-component associated with a web- page. One example of another web-test result includes the total number of unique DNS requests. Each lookup introduces latency that may affect several objects. This also indicates the number of web-servers required to provide the code-components used by the web-page. One example of another web-test result includes the total number of objects transferred (successful and failed objects recorded separately). In some embodiments one or more of the objects may have no direct visual relevance but must be processed to render the objects that do have visual relevance. One example of another web-test result includes the total transfer size of the data forming the web page. The total download size is usually indicated in Bytes. This indicates the amount of data to be transferred which may cause time-outs and other issued if a slow connection is being used by a web-browser located in a user network.

[00074] One example of another web-test result includes the time to first visual. This is the elapsed time from the time-stamp captured when a web-browser first requests a web-page to when the web- browser displays the first visual difference from its default background display (as shown in Fig. 4A the web-browser default background display is just a blank or white screen). The timing information shown in Fig. 4A is indicated in microseconds. This is an important performance metric as if waiting time for any visual element to appear is too long, a user may lose interest, and navigate away from the site and/or question if the site is running/being maintained/genuine etc. and generally may get a bad impression of the organization hosting the site.

[00075] One example of another web-test result includes the time to visually complete the initial view of the web-page. In some embodiments, this is the elapsed time from the time-stamp captured when the web-page was requested by the web-browser time to last visual update representing the fully displayed web page. Unit: microseconds. In some embodiments, the video recording is automatically terminated after a period of time if the web page doesn't continue to load within a cut-off time to avoid overly large video files from being generated.

[00076] One example of another web-test result includes timing information for the page load progress. This groups timing information into an array for the page load process at various stages.

[00077] Examples of the stages of progress for a web-page to completely load may be used such as, for example: [0-10%, 10-20%, 20-30%, 30-40%, 40-50%, 50-60%, 60- 70%, 70-80%, 80-90%, 90-100%] of the visual elements of the finally loaded web-page. Figures 4A to 4C show an example such as might be associated with an array with 10- percentage ranges with elapsed time from start of test, where the timing information unit includes microseconds. This metric helps assess the 'time to visually useful', metric, also shown in Figure 4C. This metric is web-page dependent, but typically around 60-70% of the page load progress.

[00078] One example of another web-test result comprises a link to retrieve a page load video, for example, a URL. This provides a link to retrieve a video file which shows graphically how the page is loaded in the browser with measured network latencies. Includes page load metrics such as 'time to first visual', 'time to visually useful', 'time to visually complete' as well as page load progress in percentage, such as FIGs. 4A to 4C show. One example of another web-test result comprises a link to retrieve a page load HAR type file, for example, a URL to retrieve a HAR file which contains information about how individual objects directly or indirectly reference by the web- page text file is loaded. Viewing the HAR file in a web-browser allows the web-page object timing information to be displayed in the form of a waterfall diagram. In some embodiments, the HAR file uses the W3C HTTP Archive standard format.

[00079] Some embodiments of the invention aggregate the above web-test result data generated for each individual test sessions.

[00080] Examples of the first type of web test output data from the web-test application running on the off- load server 52 which are forwarded to the test server 54 for storage and subsequent data analysis include at least the timestamp for the web-test to complete, the URL fetched, and one or more indications of the total number of DNS requests made, the total number of objects transferred, the total download and upload volume of data (for example, in Bytes) , the number of successful objects loaded; the number of failed objects, the peak number of concurrent HTTP requests, the average DNS lookup time, the distribution of DNS resolution time (per web-test session), the average TCP connection time (per web-test session), the distribution of TCP connection time, the average object transfer time, the distribution of object transfer time, the time to first element being visible, the time to browser engine reported it was finished, the time to the page being visually complete; and the distribution of the page loading time.

[00081] The second type of web-test output data are determined when the Web Test captures information on a per-object basis during a test session. Given that a web page can often reference dozens or even hundreds of objects, this means that a very large amount of data is collected and the following information is captured on a per-object basis: timestamp, method (e.g. GET, POST); URL, data transfer size; transfer speed; HTTP version, HTTP response code, source server IP, request start time, request end time; DNS lookup time; and TCP connect time.

a. Examples of the third type of test result determined from a web-test session run by the off-load server 52 comprise a video recorded by the off-load server 52 of its browser’s rendering of the web-page (with a statistics bar overlaid onto it such as FIGS. 4A-4C show schematically) and a HTTP Archive file (HAR) showing how the objects of the web-page were interacted with by the web-browser 8.

[00082] FIGs. 4A to 4C show an example of the third type of test result obtained from running a web-test according to an embodiment of the invention. In FIGs. 4A to 4C, page loading metrics have been captured by analysing a video of a web-page being rendered on a display, and page lading performance metrics are being displayed and updated as the video of the web-page being loaded is replayed. In Figure 4A, the web page loading has yet to start. In Figure 4B, graphic detail is now displayed but the web page has not yet completely loaded. In Figure 4C, the web-page has completely loaded.

[00083] Some embodiments of the web-page test video show how different web-page objects are loaded. The way each web-page is rendered by a web-browser by loading different web-page objects is measured by capturing a video of the web-page being rendered on a display and analysing the video to determine what proportion of objects are being visually presented at any time. In some embodiments, as not all objects will have any direct visual presence, it is helpful to capture the video and when the critical path objects have all been rendered, to store the video and replay the video captured by the web-test off-load server with an indication of what visually relevant objects and/or what total objects have been loaded at any given point in time whilst the video is replayed.

[00084] The process used to generate the one or more displayed performance metrics for a web-page using the rendered video is described in more detail later herein below. The performance metrics are suitably displayed with the video as it is replayed (for example, as shown in FIGs 4A to 4C in a bar which is overlaid along the top of the webpage. In some embodiments, the video of the web-page can be replaced with a customisable set of page loading statistics also displayed.

[00085] In FIG. 4A, a screen shot of the web-test video being replayed shows the web-page 62 has not yet begun visually loading any objects on a display 64. The performance metrics 60a,b,c,d,e which are displayed with the video and in FIG. 4A these all indicate that no time has yet elapsed, and as such emulate the state of the browser prior to the web-page being loaded. In practice, if the web-page was being loaded as a result of a request generated when another web-page was being shown, display 64 would not be blank as shown in Figure 4A.

[00086] In FIG. 4B, the screenshot is taken later during the playback of the video of the web-page 62 as it is being loaded on display 64 by a web-browser running on the off-load server 52. In FIG. 4B, some but not all of the objects used by the web-page 62 have been retrieved and are being displayed, but the web-page 62 has not yet completely loaded. Performance metrics such as the elapsed time 60a now indicate a time measurement value since the web-browser generated the request to load the web page, and the performance metric 60b indicating the time to the first visual element being rendered on the display has also updated. Finally, in FIG. 4C, the screenshot is taken after the web-page has completely loaded, and the performance metrics for the elapsed time to a visual useful page element being rendered 60c, and the elapsed time to completely render the critical path of the web-page and provide the first visible part on the display 62 is indicated by the done performance metric (60d). Also shown in FIGS. 4A to 4C as a performance metric is progress indicator 60e which indicates how many objects have been loaded, here provided as a percentage of the total number of critical path objects (both visual and non-visual) which are collectively required to render the first full screen of the web-page on display 64 as shown in FIG. 4C.

[00087] Each of the screenshots shown in FIGs. 4A to 4C indicates the different values of the loading performance metrics 60a,..,60e which indicate the visual progress made by that point. In FIG 4B, the screenshot shows the visual state of the loading web page 0003:846 seconds after the request to load the web-page was generated, by which time 24% of the objects required to fully render the web-page have had loaded. The first visual object loaded was presented 00:03:136 seconds later on the screen being used to browse the web-page, but no visually useful object the user could interact with had yet been presented. FIG 4C shows the web-page was fully loaded (done, or 100% progress) at 00:05:647 seconds after it had been requested, with the same first visual object loading time, and FIG 4. C also shows it took 4 seconds and 472 milliseconds from when this example web-page was first requested by the web-browser for it to be visually useful. In at least one embodiment of the test system, the web test web-browsing measurements are freshly loaded by the web-browser running on the off load server using objects held in a local cache.

[00088] In at least one embodiment of the web-browsing test system, the web browsing test system additionally captures the second type of test results in the form of a file, for example, a HyperTextTransferProtocol (HTTP) Archive (HAR) file, which indicates how individual objects referenced by the web-page are loaded. This enables bottlenecks in the data delivery to be identified and/or loading or delivery errors to be identified at the level of each code object reference in the critical path of a web-page being tested. In at least one embodiment, the timing results for each code object are presented as a waterfall diagram which distinguishes for each object‘first visual’ and ‘visually complete’ where the referenced object is a rendered object (not all referenced objects may be rendered, some may influence how another object is rendered for example).

[00089] At least one embodiment of a distributed web-browsing test system according to the invention provides one or more web browsing performance measurements to be made from within the client-side of the test system which enable various tests to be performed by the off-load server 52 which emulate the experience of browsing a web page from within a user’s home network. This enables a truly representative view of the web browsing performance of a browsed web- page 62 to be obtained that takes into account factors such as the ISP used, what content distribution network (CDN) relationships between the ISP and the CDN exist which could affect the browsing experience of the end-user, and dozens of other factors. In embodiments where the web-browsing test system is configured to run its client-side web-test agent 50 using a client platform such as an existing ISP router 10 on the user network 6, or another existing user network device, the client-side of the web-test is able to run on extremely low power hardware, which reduces the impact of running the test at the user’s premises. This also allows for power efficient scalability as it is possible to scale the number of tests over a larger range of end-users and correspondingly a wider range of browsing platforms and network environments. Also, as mentioned above, in at least one embodiment of the invention, individual test session results are aggregated across many user networks 6 where a user has access to the internet for web- browsing purposes to provide a statistically representative performance metrics for the web-page loading performance. By minimising the power used by each test-session, collectively there is very likely to be little or no impact on power resources in a particular region even when running thousands of concurrent test-sessions on different user-networks. The term user-networks, as mentioned herein, refers to any suitable network environment where a user of a web-browser may browse a web-page, including home and commercial network environments, and may, in some embodiments, including intranets. The generated video and HTTP Archive files also significantly aide in troubleshooting and shortening the time to resolve issues by allowing the impact of individual objects referenced by the web-page as it loads to be determined, which enables web-pages with JavaScript and other dynamic links and object calls to also be assessed by the test.

[00090] An example of the stages of a web-browsing test implemented by an embodiment of a web- browsing test system according to the invention is shown in FIG. 5A of the accompanying drawings. Figure. 5A shows an overview of example implementation phases of a web-test session comprising: a first relay phase 70 during which data is relayed between the off-load servers 52 and the client-side web-test agent 50, followed by a transfer measurement phase 72 during which the client-side web-test agent transfers at least timing information for retrieving the web-page and any referenced objects to the off-load server 52. After this the web-test replay phase 74 can start in which the off-load server 52 replays how the web-page would have been presented using a browser 8 in the user network 6 built by emulating how rendering would have been performed on the client-side using the timing information received from the client-side web- test agent 50. The off-load server 52 causes a video recording of how the web-page is rendered to be captured. This recording is then analysed to capture performance metrics and a second video is then created which shows a dynamically updated indication of how the performance metrics 60 change as the web page is rendered. The performance metrics 60 are updated as the video is replayed to indicate various items of timing information and/or other performance metric values for the web-page as it is rendered on a display 64. The analysis and generation of the second video occur in an image analysis and video generation phase 76.

[00091] Next, a suitable results file is created for storing the web-test results 78, in a suitable file format. In one embodiment, an indication of how the web-browser of the off load server 52 has interacted with the web-page at an object level is captured and saved in a suitable file format which provides information at the level of each object references by the web-page being tested, for example, by the off-load server 52 entering a HAR File Generation Phase 78. Next the off-load server 52 stores the results, which in some embodiments are uploaded to a remote data storage facility in a file storage phase 80 (also referred to herein as a file upload phase (80)) to provide results in a suitable form (for example, in a web-browser, or spreadsheet format) in a results generation phase 82. In some embodiments, stored data is aggregated and collated, for example, in a spreadsheet, pivot table or the like, to provide statistical insights for a plurality of web- test sessions. In some embodiments, the final stage of a web-test session generate the results in the form of one or more CSV files, for example, by implementing a CSV Result Generation Phase.

[00092] An example embodiment of the preliminary phases of a web-test session implemented by a web-browsing test system such as that shown in FIG. 3 will now be described in more detail by referring to FIG. 5B of the drawings in which test center 54 provides an indication of the web-page 62 to be tested to at least one client -side web- test agent 50 and/or to an off-load server 52. If communicated to off-load server(s) 54, then each off-load server 54 which receives an indication of the web-page 62 to be tested (step 90) informs one or more client-side web-test agents 50 of the web-page 62 to be tested. If the test center 54 informs one or more client-side web-test agents 50 first, then each of the web-test agents 50 initiates a test session by communicating and indication of the web-page to be tested to one of the off-load servers 52. In some embodiments, the test center 54 configures a web-test to be conducted as a plurality of web-test sessions all run in parallel, as one-off tests and/or as repeated or periodic tests. Each off-load server 52 determines data sources for all of the directly referenced and indirectly referenced code components (also referred to herein as code objects) associated with the indicated web-page 62, including any dynamically referenced code objects, (step 92) and identifies one or more data sources (for example, web servers 2) for each code component (step 94) and informs each client- side web-test agent 50 in to request the code components and capture measuring information(step 96).

[00093] In this embodiment, steps 92 to 96 accordingly comprise an example of the relay phase 70 of the example web-test. When the client-side web- test agent 50 receives a request to capturing timing information for retrieving the indicated code components/objects, the client-side web-test agent is configured to generate requests for the indicated code components and to measure timing information for the time which elapses after requesting each code component from its indicated data source until the code component is received. In some embodiments, the timing information will include any time taken to perform any necessary address look-up operation(s) to determine a source address for each code-component/object indicated as well as the time to retrieve the actual code components/objects indicated from their sources. At least the retrieval timing information and an indication of the code component for which it relates to are then sent to back to the web- test off-load server 52 in the transfer measurement phase 72 of the example web-test. In order to perform the replay phase 74 of the test, the off load server 52 uses the received retrieval timing information to time when its browser loads the web-page code components for rendering the web-page 62 (step 100). The off-load serer 52 then generates a video recording and captures rendering performance related metrics by analysing the video rendering in combination with the objects which are being rendered and/or other objects required to be able to render the web-page at various points in the video (step 102). Once the web-page 62 is fully loaded (step 104) the video recording is stopped (this can be automatically triggered in some embodiments when all visible critical file objects have been rendered as the image will normally no longer update). At this point, additional performance metrics 60 can be determined derived from the number of objects and the elapsed time to display the fully rendered critical path web-page 62 and the image analysis and video generation phase 76 shown in FIG. 5A is complete.

[00094] More detailed example embodiments of the various phases of an embodiment of a web-test will now be described.

[00095] An example embodiment of a relay phase 70 of the web-test is implemented by the off-load server 52 of the web-browsing test system determining which objects are associated with the web-page 62 whose browsing performance is being measured by the test client 50. During the relay phase 70, the off-load sever 52 and client-side test agent 50 cooperate to identify what objects will need to be fetched (and later measured) by the client.

[00096] Since many websites dynamically reference objects, especially objects which generate content both on the client side (JavaScript) as well as on the server side, the relay phase 70 is implemented by using a web-browser (also referred to herein as a web-browser) 160 (see Figures 8A to 8E and FIG. 9 for example) running on the off-load server 52 in order to determine the dynamically and non-dynamically referenced code components. Examples of such code components include, for example, the web-page text file and any code objects referenced by the web-page text file and/or objects referenced indirectly by the web-page text file (for example, which are referenced by another object, or which are only referenced dynamically as the web- page is loaded by a browser for rendering). The browser engine 160 loads the complete page including all the critical path code components which when loaded provide the visible web-page prior to any further user interaction, and creates a list of the code components required to provide to the client-side web-test agent 50 of the web-browsing test system. This web loading phase loads the page at the off-load server 52 by communicating with web servers 2 and the HTTP requests which have been relayed via the client-side web-test agent 50.

[00097] In order for the browser 160 running on the off-load server 52 to set up a connection to a particular web server 2 from which a code component must be obtained it must first lookup the server address for the web-server’s domain name by consulting the domain name server (DNS) for that web-server 2. In some embodiments, this functionality is wrapped or relayed via the client-side device by the off-load server 52 asking the client-side web-test agent 50 to lookup the IP address in the form of a relayed DNS request. The DNS lookup time for that code component which the client-side web- test agent experiences when fulfilling this request is stored locally on the client side in a suitable memory or local cache and then returned to the off-load server 52 and stored until it is needed later in the test by the off-load server 52.

[00098] In some embodiments, at this point it is possible for the off-load server 52 to establish a socket connection to the web server 2. This is also wrapped from the off-load server 52 via the client-side web-test client agent 50 so that instead of passing a stream of data directly from the off-load server’s own kernel to the relevant web-server 2 the IP data is streamed in a relay via the client-side web-test agent 50. In some embodiments, the client side web-test agent 50 then establishes a real TCP connection to the remote web server (for example, a web-server 2 as shown in the embodiment of the web browsing test system illustrated schematic ally in FIG. 3) and sends the streamed IP data originating from the off-load sever 52 to the remote web-server 2. When the Web server 2 responds, the client-side web-test agent 50 takes the incoming stream of IP data and forwards this on, back over the same bi-directional TCP socket connection to the offload server 52. The offload server 52 in turn will feed the IP data back to the browser's wrapped socket connection with the client-side web-test agent 50.

[00099] In some embodiments, the initial stream data to/from the web server 2 and the client-side web- test agent 50 comprises a secure socket layer (SSL) negotiation. Later a Hyper Text Transfer Protocol (HTTP) request header is sent and the web-server 2 replies with HTTP response header and body. The bi-directional TCP stream relay between the client-side web-test agent 50 and the off-load server 52 forwards any data that is needed to load the web-page 62 received from a web-server 2 over the bidirectional TCP connection. The same is done for all parallel relay connections that the browser application 160 on the off-load server 52 needs to establish. Web-browsers often use multiple connections over the Internet when using HTTP 1.1 to reduce transfer latencies, and the traffic between the client-side web-test agent 50 and offload server 52 will accordingly need to emulate these multiple connections with the same level of complexity. To reduce socket usage on the server side and to have less connections on the client side a single TCP connection is used in some embodiments of the invention and the browser connections are tunnelled over this using a suitable multiplexing protocol such as the WTCP (Web Test Control Protocol). In some embodiments, WTCP is also used internally on the server side as a common protocol for communicating between processes.

[000100] In the preliminary relay phase 70, the off-load server’s web browser application 160 will load the entire web page. This allows the relevant code components (the web-page text file and any directly or indirectly referenced objects) to be loaded by the web-page 62 to be determined by the browser application 160. While objects are being loaded the complete HTTP request/response header/body pairs are stored on the off-load server in a local cache as they are needed later in the web-test process by the off-load server 52. This is done by wrapping the HTTP layer and storing everything in a data cache (not shown) located in the main offload server process and is, in some embodiments, a local cache accessible by the off-load server 52. Due to the increased latency (especially for connections with slow uplink bandwidth) the relay phase 70 may take a considerate amount of time. This puts a limit on how many tests a single off-load server 52 can handle. For this reason, in some embodiments of the invention, the offload server 52 uses the caching fields in the HTTP response header so that if data from previous test is available it can be re-used, and the associated object then does not need to be fetched over the tunnel between the web-server 2 and the client-side device hosting the client-side web-test agent 50.

[000101] In some embodiments, many (if not the majority) of cacheable objects will have been previously cached by the off-load server 52 prior to receiving timing information from the client-side web-test agent 50 about when to load them. In such embodiments, the off-load server fetches only non- cacheable objects (for example, objects which may include content which might be dynamic, such as a link to an image on a news web-site for example) and relays the DNS requests (not needed for relay but the measurements by the client-side device will be needed in later phases). In some such embodiments, even if data is taken from the cache as a part of producing a list of needed objects, they will later always be fully measured on the client side.

[000102] In some embodiments of the transfer measurement phase 72, when the web page 62 is fully loaded by web-browser engine 160 running on the off-load server 52, a complete list of URLs that the page is composed of is created. Data providing the list of URLs together with their associated saved HTTP request headers is then sent to the client-side web-test agent 50 so that the transfer measurements can be carried out by the web-test agent 50 from within the user network 6. In some embodiments, the client- side web-test agent 50 imitate the behaviour of a browser processing a web-page request by carrying out a plurality of object transfers in parallel. Detailed measurements about TCP connection time, SSL negotiation time, waiting time and transfer time can then be saved on a per-object basis.

[000103] In some embodiments of replay phase 74, when the client-side web-test agent 50 has finished recording its measurements, it asks the offload server 52 to replay the web site loading experience on the client-side web-test agent 50 using the same DNS lookup timing and transfer measurements made by the web-test agent50. The off load server 52 then starts a new web browser session in replay mode. In replay mode, however, the off-load server does not fetch data from the internet or make any external communications and the HTTP layer is wrapped and for every object that is requested the data is taken from the data cache. [000104] In some embodiments of the replay phase 74, due to the multi-process design of modern browsers, some or all of the web-page code component data is not fetched dynamically (“on the fly”) as part of the main web-page loading process performed by an offload server 52, , as this would introduce additional lookup latencies. Instead, when a top-level object referenced by a web- page 62 is requested, the previously generated URL list is used to fetch all needed objects to be put in a process local data cache which has a minimal lookup overhead. In some embodiments, a web-page being tested may comprise a large (for example, hundreds or even thousands) of objects. In such embodiments, all referenced objects required for performing the web-page test (for example, those required by the critical path) are first stored in a hash table so that they can be loaded in accordance with the page load measurements. Each object is given to the off-load server web-browser 160 is based on its individual measured latency from within the user network 6 by the client-side web-test agent 50. A similar process is performed for the DNS lookup time the first time a fully qualified domain name (FQDN) is accessed.

[000105] In some embodiments, whilst the web-page 62 is being loaded and rendered by the browser on a display during the replay phase 74, periodic screenshots of the browser window are taken, for example, every 200ms which are suitably stored in memory. When the web-page 62 is fully loaded the screenshots are encoded in a suitable file format (for example, a lossless data compression file format may be used such as a PNG file format) and stored in a suitable file system. As a final step, a replay report is generated for the entire session as well as replay timing measurements for each object.

[000106] In some embodiments of the image generation and video generation phase 76 which takes place after the replay phase 74 has ended, a video is generated of the web-page load process. Image analysis then takes place in which screenshots of the video are analysed to determine the rate of web-page load progress based on the determined elapsed times of certain web-page performance metrics 60 such as 'first initial', 'visually useful' and 'visually complete' are identified as well as the overall page load progress expressed as a percentage. With the available information a video is generated where screenshots are mapped to a high frame-rate video timeline. An on screen display of ticking timestamp, page load progress and measurement timestamps is included in some embodiments such as are shown schematically in FIGs. 4A to 4C. The video gives a visual feel of how fast the page is loaded.

[000107] In some embodiments of file generation phase 78, a results file is generated which has a HTTP Archive (HAR) file format and which includes data generated by combining some or all available measurements provided by the web-test agent 50 and the off-load server 52. Examples of available measurements may include network characteristics from the user’s network, the HTTP request and response header/body pair from the data cache, and the replay report. In addition, insights and conclusions from the image analysis is in some embodiments used to produce the HAR file and allow detailed inspection of the timing and complete content of the page load operation.

[000108] In some embodiments of the invention, the distributed web-browsing test system produces a HAR file for each referenced object web-browsing test measurement recorded by the web-browser of the off-load server conducted. A HAR file is a W3C standard file format used to record a complete web browsing transaction, and tools known in the art are available that allow deep analysis and/or playback of a web browsing session given a HAR file as input. By capturing a HAR file for each web browsing test, developers and operational staff can perform deeper analysis/troubleshooting of an individual web browsing session from the perspective of the client (thus capturing all ISP and other effects) without requiring a user at home to do anything. Accordingly, whilst the visual analysis provided by the web-browsing test system is useful as an indication of how the user experience of browsing a web-page on a web-browser is impacted by a web-page’s loading performance, it is only by pairing that visual analysis alongside the object-level detail provided within the HAR file that it is possible to identity what objects have caused the visual impact.

[000109] Whilst known web-browsers such as Chrome™ and Firefox™ can generate HAR files, these require a user to invoke them manually. Embodiments of the invention provide a distributed system which captures object-level loading performance information, such as timing information and related per object information, of a web browsing session operated remotely on a device in a user's home without user interaction, for example, by recording HAR files of web browsing sessions operated remotely on a device in a user's home without user interaction.

[000110] In some embodiments, to store the results file in file storage phase 80, it may be uploaded to a remote data storage facility. For example, if the video file and HAR file will be too large to just be stored in a database for later retrieval. It is also likely that they will become less relevant over time and may be purged after a period of time. In some embodiments, the video and/or HAR resource files are instead uploaded using unique test session prefix to a file storage repository. In some embodiments, each resource file generated by a test-session is able to be accessed via a public URL from the file storage repository.

[000111] In some embodiments, the results generation phase 82 provides the final test output in the form of a data file indicating the web-session results obtained from one or more client-side web-test agents 50. In one embodiment of the invention, the test results are provided by the CSV data on the client side which consists of overall test results as well as per-object statistics which are later stored in a columnar database for data analysis at the test center 54.

[000112] An example embodiment of a test sequence will now be described in more detailed referring to FIGs. 6A to 6E, which may refer in turn to some of the components of the distributed web-browsing test system software shown in FIG. 7

[000113]

n the example embodiment of a web-browsing test sequence shown in FIG. 6A, the client-side web test agent 50 is invoked on the command line with one or more suitable options (step 110). For example, a URL may be given for the web-page 62 to be tested, e.g. www.abc.co.uk. The invocation is done remotely, and the command may be generated from the web-test server 54 shown in FIG. 3 or from one of the off-load servers 52 also shown in Fig. 3.

[000114] In the embodiment shown in FIG. 6A, the client-side web-test agent 50 then sets up a TCP connection to the offload server 52 in step 112. This channel is used for all types of tasks involving the off-load server 52. The TCP connection helps avoid issues which might otherwise occur due to the presence of NAT. Each message is multiplexed using the WTCP protocol (Web Test Control Protocol). An initial request is sent requesting a test list of needed web resources to the off-load server 52 in step 114. The offload server 52 receives the command and invokes its browser engine 160 (implemented using Web Kit in some example embodiments) which will try to connect to the web server 2 in step 116. A socket wrapper is connected underneath the library which will not forward each socket command to the kernel. Instead the off-load server 52 TCP stream data is relayed back to the client-side web test agent 50 over WTCP in step 118. When HTTPS is used the SSL library routines ssl_write(), ssl_read() and similar each in the end call the socket wrapper functions. When the socket client receives relayed data, it will set up its own TCP connection(s) to the web server(s) 2 (step 120).

[000115] Continuing with the method shown in FIG. 6A, in FIG 6B, for each new web server 2 the client- side web-test agent 50 measures DNS lookup times (step 6124) (DNS lookup is also wrapped over WTCP) and then the relayed data from the browser engine 160 running on the off-load server is then sent to the web server 2 and all data that comes back is redirected back to the offload server 52 over WTCP (step 126).

[000116] The offload server 52 then feeds the relay data back to the socket wrapper and the web browsing engine (in step 128). In practice this may be implemented using another process by the offload server. During the page load operation all HTTP headers that are sent/received is captured on the libsoup HTTP library level and sent to a data cache in the offload server process (step 130).

[000117] When the page has been fully loaded, a list of needed web resources is sent back to the client- side web agent 50 in step 132. This list includes the HTTP request header/body data for each URL. The client-side web-test agent 50 sets up new connection(s) to the web server(s) (step 134).

[000118] The process is continued in FIG. 6C, where the client-side web-test agent 50 measures HTTP/HTTPS transfer timings using the same HTTP header/body as when the browser engine was communicating to the server (e.g. session cookie) in step 136. The client-side web-test agent 50 then sends a replay measurement request to the offload server 52, including DNS and transfer measurements, in step 138.

[000119] The offload server 52 tries to load the web page in replay mode in step 140. When the top-level HTTP/HTTPS request is sent to the libsoup level, the web Engine downloads all relevant entries for the test session and stores it in its local data cache. When everything is ready for replay, a shared memory segment indicates when the test actually starts. This triggers time measurement in the GTK web Engine process together with periodic screenshots (step 142 in FIG 6C).

[000120] Next, instead of communicating over network, the data is fed to the web- browser from a local data cache on the off-load server into the asynchronous library functions for reading HTTP response header and body, using the same timings as measured on the client side in step 144. The web objects that are loaded will be in the order requested by the browser (step 146).

[000121] The replay function also introduce DNS lookup latencies according to measurements. In order to lookup a requested URL in the data cache, fuzzy matching is needed in some cases and timestamps and other random parts of the URL are dynamically generated by the JavaScript engine. For that reason, an external tool is invoked to calculate the edit-distance between all URLs using the same FQDN to find the most suitable entry in the data cache. When the browser engine indicates that the web page is complete the test waits for a period of time (to fully allow JavaScript to execute). Then it stops periodic screenshots and informs the offload server process which will then start the snapshot analyser process (step 148).

[000122] FIG 6D shows a brief overview of the steps performed by a web-browsing test snapshot analyser20 component 162 (see also FIG. 7) according to some embodiments of the invention. In FIG. 6D, the web-browsing test snapshot analyser 162 first filters the image list and removes duplicate images (images with no visual difference) in step 150. Then the snapshot analyser module calculates the pixel delta between each screenshot which is expressed as a bounding rectangle in step 152. The pixel delta bounding rectangles together with timing information captured about the elapsed time (expressed in the file name) are used to determine one or more web-page loading performance metrics such as the timing of the first visual element of the web page and a“visually done” metric in step 154 . Only after a visual done screenshot has been recognized by the web-browsing test system, does the loading of a web-page stop. This“visually completely loaded” screenshot then acts as reference for the amount of visual progress for all screenshots leading to that final image. This is done with a tool that can compute structural image similarities between two images (dssim) as described earlier.

[000123] Some embodiments of the invention generate information about page load timings, such as the actual screenshots and show visual progress using a PNG image sequence. A scalable vector graphics library, for example, Cairo, is used in some embodiments where a transparent progress bar on top of the screen is added with a ticking timestamp and indicators of time to first visual time to visually done and visual progress. This layer is scaled to fit on top of a pixel buffer containing the screenshots. A web browsing timeline produces screenshots dynamically so that each screenshot is introduced in real-time. The PNG image sequence that is generated is mapped to a video timeline with a certain frame rate. In the end the FFmpeg tool is used to encode an x264 video file of the page load process. An animated GIF of the same video with lower FPS is also generated.

[000124] Not shown in FIG. 6D, in some embodiments, the video file incorporating the dynamic performance metrics for the web-page is uploaded to a test center file server, which will be accessible using a certain URL. The snapshot analyser 162 in the software stack of the web- browsing test system causes all results to be sent via the off-load server 52 to the client-side web-test agent 50, which are then sent on to the test server 54 which invoked the test. In some embodiments, test server 54 aggregates received test results with other web-test result information received from at least one other client- side web-test agent 50 running in a different user network. The test server 54 then suitably stores aggregated test result data in a suitable file format (for example, in a CSV format).

[000125] FIGURE 7 of the accompanying drawing provides an example of a web- browser software stack 172 which may be used to implement some example embodiments of the web-browsing test system. The software stack 172 comprises software for the client-side web-test agent 50, off- load server 52, a web-test web-engine 160 which runs on the off-load server 52, a fuzzy matching tool 164 (SAM - please explain what this is used for briefly), the web-test snapshot analyser and video generator 162, and a common library 170 which supports dynamic linkage of code elements running on the web-test server. Both client and server link against the library 162 with common components. In at least one embodiment of the invention, the web-browser software stack 172 allows the use different web-test engines 160 (e.g. Web-Kit, Firefox or Chromium) by having source code in the bottom layers of the stack shown in FIG. 7 by a web-test offload server 52 which can communicate with the kernel of the off-load serer 52 using sockets. The sockets must be wrapped to relay TCP stream data to another host (for example, a stand-alone client-side web- test agent 50) which will create real sockets that connect to web-servers 2 on the Internet 4. In some embodiments of the web-browser 160 located at the off-load server 52, source code is available for the HTTP message layer, where request/response pairs are monitored as well as replayed.

[000126] As shown schematically in FIG 7 of the accompanying drawings, the web browsing test system according to some embodiments of the invention uses wrapping and logic inside external libraries, and the components shown represent a layered web- test software stack which allows for dynamic linkage and execution context. For example, FIG. 7 shows schematically how several processes are used on the server side in at least one embodiment of the invention. Network communication including socket wrapper and HTTP message layer is performed in a process called WebKitNetworkProcess while the graphical GTK window is managed by the GTK application process. A DBus protocol is internally used to relay information and to address any potential issues arising as the instantiated grand-children of the main web engine process will not be aware about their purpose when it comes to each web-test, they are configured to connect to the main offload server 52 and query their working context for each test session.

[000127] In some embodiments, the software stack shown supports the main test server process and acts as a communication server which connects everything together. It maintains a data cache and the test session state for each test session. In some embodiments, the test server 54 is responsible for load distribution and overload protection, in other embodiments, each off-load server 52 is responsible. In some embodiments, an off-load server 52 can rejecting a test-session request from a client- side web-test agent 50 when the off-load server 52 has reached its maximum capacity. In some embodiments, the off-load server 52 will then direct the client-side web-test agent 50 to another offload server 52 that it has in its list. [000128] FIG.s 8A to 8E shows an example message sequence diagram for an example test session implemented by an example a web-browsing test system according to an embodiment of the invention.

[000129] In the example message sequence shown in Figure 8A, test server 54 triggers the start of the test by requesting a web-page 62 (indicated as https://www.abc.com) is loaded in step 800 which is sent directly to a client-side web- test agent 50. The client-side web-test agent 50 requests a list of the web-page’s code components to measure timing information for to offload server 52 in step 802. The off load server 52 sends a start-web-engine message to its web-browser 160 (see also FIG. 7) in step 804. When the browser engine is loaded, it will call a socket layer to connect to the external network environment (steps 806, 810). The socket layer established (in steps 806, 810) is then wrapped with the client-side web-test 50 in step 810 in order start a relay mode 70 and obtain the timing information the off-load web-browser 160 needs to emulate the timing of the load process of web-page 62 as it would be loaded by a client-side web-browser 8 (for example, such as web-browser 8 shown in FIG. 3). Next the relay mode 72 is begins with a DNS request 810 for the web-page 62 which is sent over the socket layer from the web-browser 160 running on offload server 52 to the client-side web-test agent 50. When the client-side web-test agent 50 processes the domain-name resolution request 810 for the web-page 50 URL: https://www.abc.com, it generates its own DNS request for https://www.abc.com in step 812 and sending this request to a DNS server 18 (as shown in FIG. 3) in step 812 also triggers the start of a timing measurement. Web-test agent 50 is configured to automatically stop the timing measurement for the DNS look-up when it receives a DNS query response from the DNS server 18 (for example, https://www.abc.com may translate to server with an IP address 91.198.174.192) in step 814. The DNS query response is then relayed back to the browser 160 running on off- load server 52 by the client-side web-test agent 50 in step 816. Next a library call is made in step 818 to the libsoup layer of the web-browser software stack which is sent when the first HTTPS get command is generated by the web-browser 160. This starts the test session by requesting off-load server 52 generate a test session connection request by sending a Test_Session_CTX_Request (pld) in step 820.

[000130] Continuing in Figure 8B, the off-load server 52 responds with a Test_Session_CTX_response to the web-browser in step 822. Only after the response is received in step 822 does the web-browser 160 allow any indication of the web-page related code-components to be retrieved by the web-browser 160 (in step 824) to be relayed over the socket layer to the client-side web-test agent 50 via the off-load server 52 (in step 826) using a TCP_Stream_Relay (for example, by using an lnitial_HTTP_GET command). The TCP_Stream_Relay is established in some example embodiments of step 824 using suitable code elements such as socketWrapper..socket(), socketWrapper.. connect () and then socketWrapper..send(). Client-side web-test agent 50 receiving the TCP_Stream_Relay performs the corresponding socket(), connect(), and send() and sends the relayed HTTPS request to the web-server 2 having the web-address indicated by the DNS server as hosting the requested web-page 62. The web-server 2 then sends an HTTPS response message in step 832 back to the client-side web-test agent 50 which will usually include at least a web-page text file for the web-page layout. The client-side web-test agent 50 then relays the web-page layout text file using a TCP_Stream_Relay back to the web-browser 160 running on offload server 52 in step 834. Web-browser 160 then processes the received request and returns the socket layer to the browser layer (for example, by using a socketWrapper_recv() - non-blocking socket command) in step 836. The Web-Browser 160 then commands the off-load server 52 using a Test_Session_Http_Req_Resp_Pair in step 838 to store a complete record of the HTTP request message header and body and the HTTP response message header and body in a local cache (not shown in the Figures) in step 840.

[000131] In most, if not all, examples of a web-page 62, the web-page test file will refer to code components such as objects which determine the layout and content of the web page when it is rendered on screen. Some of the objects will in turn refer to other code components and so on. Thus as FIG. 8B indicates schematically, the name resolution steps, TCP relay steps, record HTTP Req/Res pair steps and (not shown explicitly in FIG. 8B) the build URL and Test List steps are repeated as necessary to determine all relevant code components which are needed to build the initially visible (critical path) part of the web-page 62 with the relevant timing information for their retrieval provided by the client-side web-test agent 50.

[000132] After all required code components have been retrieved and loaded by the web-browser 160 running on the off-load server 52 to allow the web-page to be rendered by the web-browser 160 (although the actual rendering on a display need not yet occur in some embodiments), in order to obtain the same timing information which would be associated with the load process by a web- browser 8 running on the client- side, the example test-sequence message flow then continues as shown in FIG. 8C. In FIG. 8C, the browser engine 160 reports that all the web-page 160 has been loaded (and all browser related processes then exit), by sending a Test_Session_Web_Page_Loaded message in step 842 to the off-load server 52. The off-load server 52 then send a test-list response message in step 844 to the client-side web-test agent 50. The test-list response message provides a list of web-resources (for example code-components such as web-page URLs and any referenced objects) used by the web-page 62, and requests that the data transfer times are measured for each resource requested. In the embodiment shown, the process uses pre-populated local DNS cache since the name lookup has already been measured. The client-side web-test agent 50 then generates a https request 848 and receives a https response in steps (848, and 850) for each web-resource indicated in the test-list, and captures the timing information, which is provided back to the off-load server in step 852 in a replay measurement request. When the off-load server 52 receives the replay_measurement_request from the web-test agent 50 in step 852, it generates a command for browser engine to start loading the web-page using the received timing engine in step 854.

[000133] FIG. 8D shows the test message sequence which occurs after the off-load server 52 has sent the start web-engine command in step 854 to the browser engine 160. First the browser engine 160 starts the loading part of the test by establishing a connection request in step 856 using the received timing information for the client-side web-test agent 50. The web-server then sends a response at a subsequent time aligned to the same time-gap between the connection request and response measured by the web-test agent 50. Next the test enters the replay mode, where the browser engine 160 first requests each code component for the web-page 62 to be retrieved and stored in a local data cache using data cache requests (step 860) and responses confirming the web-page components have been cached in step 862. This ensures that no browser HTTP requests are handled until all data is available in the local cache which is to be used for loading the web-page by the web-engine 160 so that the timing of the web page loading on the client side can be properly emulated. The http request and response steps 864 and 866 are then repeatedly performed for all URLs to populate the local data cache in the browser process running on web- browser 160. When all data has been stored, the browser engine 160 signals in step 868 that page load measurement can start, for example, by signalling over a shared memory segment. This signal triggers the image analysis and video generation phase 76 shown in FIG. 5A. On receiving this signal, the browser application starts recording the web-page loading process and capturing screenshots periodically (or at other time intervals). The loading process replays the web-traffic (e.g. the amount of time after a data request for a component was sent on the client- side, to the time the component is received on the client-side). Inside the Libsoup software component 170 in FIG. 7, the browser engine 160 feeds the HTTP response header and body back to the browser layer with respect to the measured name look-up and transfer speeds. The fuzzy matching tool 164 (shown in FIG. 7) performs fuzzy matching for all URLs for web-resources used by the web-page that fail exact matching. As the loading process is emulated using local cached data, the tests run on the web-engine 160 do not generate any network traffic. In some embodiments, a suitable error message may be generated (not shown) for URLs that are dynamically generated if these differ from those requested during the first web page load. Finally, the web-browser 160 running on the off-load server reports that the test-session web-page has been loaded using the timing information measured on the client-send in step 870 and all browser related processes exit in step 872.

[000134] Referring now to FIG. 8E, next the snap-shot analyser software component 162 (see FIG. 7) starts analysing in step 874 the video recording of the loading process. In some embodiments of the test-message sequence, the next step 876 comprises the analyser component 162 determining the differences between screenshots or still images from the video, for example, in the form of screen-shot pixel delta information, and generates from time-stamp information the time which has elapsed from the start of the page loading process at which certain predetermined loading performance metrics 60 occur, for example, performance metrics such as time to first visual, time to visually done, time to web-engine reporting done, The visual progress can be calculated using the time-stamp information associated with different screenshots using structural image analysis.

[000135] A page load video of the loading process together with the performance metric timing information which dynamically updates as the video is played back is then generated. A measurement response message is then relayed back to test agent 50 in step 878 and the web test results may then be forwarded on to the test server 54 via the client-side web-test agent 50 in step 880.

[000136] In some embodiments, the server-side OS may be hosted in a cloud environment, which enables regular, for example, hourly test sessions to be scheduled by the test server 54 on thousands of client-side web-test agents 50. In one embodiment, each client-side web-test agent 50 has an ordered list of offload servers that they will contact. If a server is unreachable or have reached max capacity it will continue with the next.

[000137] To recap, the purpose of a web test implemented by an example web browsing test system embodiment of the invention is to capture the user experience of browsing by testing inside the user’s own network. Since a browser application is a graphical program that renders a web-page step-by-step as more objects referenced by a web-page are received, the web-test provides measurements related to what is observable on a screen displaying the web-page. The goal of the test is to highlight how much waiting time is needed for a human to be able to start accessing the contents of the web-page, it does not necessarily mean that all objects referenced by a web-page which are retrieved have a visual relevance. Examples of objects which are not related to the rendered web-page (and so are invisible to a viewer) are objects causing updating of web- page statistics on a server. Other objects may have an indirect impact on the visual objects rendered. A useful performance metric for a web-page relates to the rate of visual page load progress against a screen displaying the fully rendered page. This is a useful performance metric which can be used to define when a page has enough objects visible to bring value to the user. Some embodiments of the web-tests provide (in addition to information indicating a time frame for a web-page to load and its visual implications) performance metrics which are object-specific. Object-specific metrics may highlight what objects are loaded and which are not and allow insights about why a web page doesn't load fast enough or as expected and/or why a part of the page may be broken.

[000138] As many web sites require access to multiple web servers with different physical locations, when a web-page does not load or takes too long to load, this may result from a bottleneck in just a small part of a network. As mentioned above, some objects (e.g. HTML, CSS or JavaScript) have other objects as dependencies. If one of these objects is causing an issue, this may increase the page load time even if the parent object does not experience any network congestion or bottleneck. Embodiments of the invention which provide object-specific timing information allow which object(s) are causing delay/where time is being spent to be identified. This allows for better decisions to be made when improving the network or increasing server performance. Whilst using real user equipment could provide the more accurate measurements, this would require the cooperation of users and/or the web-browser application providers to modify their browser software. In the absence of such co-operation, it is possible to use the local area network router (commonly referred to as a broadband router) for user networks. Such routers are commonly found in residential and office networks.

[000139] Embodiments of the web-browsing test system according to the invention use an offload server to test one or more browsing experiences of a web-page from one or more users’ networks. In some embodiments, a web-test server 54 runs a browser application while in each user network 6, a web-test client agent 46 hosted in a broadband router 10 performs time measurements between its CPE user network 2 and all needed web servers 2 (needed in the sense that they provide an object referenced by the web-page). Embodiments of the web-test for a web-page measure DNS lookups and object transfers for the web-page. The offload server 52 is configured to load the web page with all its objects according to the measured timings from the user's network 6.

[000140] In some embodiments, web servers 2 produce content differently depending on the geographical location of the user (and NAT may also affect which web-servers 2 are used). To avoid this resulting in different code objects being retrieved by browsers 8 within a user network 6 from those retrieved by the web-browser 160 running on the off load server 52, in some embodiments, the offload server 52 is configured to never communicate directly with any of the web servers 2 which provide the object referenced directly or indirectly by the web-page 62 under test. Instead a tunnel is established by the router between the user's router 10 and the offload server 52 so that all traffic is originated from the user's own network 6. The loading operation performed by the web- test server 54 involves the web-browser hosted by the web-test server 54 using a server-side data cache of HTTP data and timing information provided by the web-test client. The web- browser and web-test server using the HTTP cached data and timing information for obtaining that data as it would have been obtained by a web-browser running in the user network to control of the timing of the web-page loading process being replicated at the web-test center. The result of each web-page rendering process performed by the test server 53 will be a replication of the process as it would have been performed by a web-browser running in the user network and the performance metrics accordingly provide user specific test results.

[000141] Advantageously, in some embodiments, the tests are scaled up to cover the web-browsing performance of a web-page as it would be perceived on user equipment on thousands of user networks. This provides a good overall picture of browsing experience over a large geographical area. As some embodiments of the web-test run on client-side web-test agents embedded in broadband routers, this allows far more user network locations to be tested than would be otherwise possible. Some embodiments of the invention provide for periodically testing a large number of user networks the difference in performance can be seen over time, which can also give insights into network congestion levels and other network conditions.

[000142] Performing an example of the Web Test on hundreds of broadband connections, it is possible to compare page loading time. By analysing collectively the web-test data obtained by aggregating web-test sessions run on client-side web-test agents, it is also possible to understand other factors which might affect how a web page is browsed in a large number of user-networks under different network conditions, such as time-of-day, and also how frequently it may be necessary to repeat a web-test in the course of a single day for example.

[000143] FIG. 9A shows, as an example, how users loading the same real web-page had very different browsing experiences depending on which ISP A, B, or C was used. FIG. 9A shows the time for the same web-page to visually complete being downloaded for browsing on a user’s web-browser (y-axis) averaged over several hundred end-users over a period of several days (x-axis).

[000144] FIG. 9B shows time to fully load a web-page (y-axis) over a period of several days (x-axis) which illustrates how a group of end-users with the same ISP but different service products (for example, packages which have different maximum contention ratios/average download connection speeds) will have differing web-browsing experiences of the same web-page 62. In Figure 11 B, three different maximum download speed broadband service subscriptions are compared to illustrate how (as would be expected) the higher the broadband connection, the quicker the same web page 62 is to load.

[000145] FIG. 9C shows how a page loading metric such as the time to the first visual of a web-page can differ depending on the nature of the web-site and the content media supported. In FIG. 9C, the % of web-page objects which are loaded (y-axis) against time to load (x-axis) is illustrated. In FIG. 3C, three web-pages are compared, web-page A is a well-known social media web-site which supports a mix of multi-media, text and video content (and takes longest to fully load), web-page B is a well-known on-line crowd- source encyclopaedia (which loads fastest), and web-page C the last is a social web-site often used in a mobile context, where text is frequently updated several times a day and text is predominantly used as the preferred medium. Fast loading is usually a 15 priority for such web-sites, but the graph shows that whilst the web-site starts to load the most rapidly of all three, after about 5% of objects have loaded the rate at which the web-site loads falls away, which may indicate one or more objects are causing a small bottleneck.

[000146] In one embodiment of an implementation of the web-browsing test system, the web-browser engine used is WebKit which has available source code and underlying dependencies which are also built as a part of the build system (i.e. they are not simply assumed to be installed in the browser system). Such a browser engine enables wrapping requests and responses to be more simply implemented, for example, for HTTP monitoring and replay, an http client/server library such as libsoup may be wrapped by the off-load server to enable communication to the client. Similarly, for DNS lookup and TCP stream wrapping, a library software stack toolkit such as GLib may be wrapped. [000147] By using wrapping to access external libraries the web-browser build system can be simplified. In one preferred embodiment, to reduce complexity, the wrapping is done in the lowest level of a Web Kit web-browser, before calling the external libraries. In other embodiments of the web- browsing test system however something similar would still be needed to support the necessary message exchange between the client and the Offload Server.

[000148] In some embodiments, state data is stored in hash tables for performance reasons. Some tables need multiple keys (e.g. one key to identify the client connection, another to identify a socket wrapper process). For this reason, a multi-key hash table has been implemented, which essentially is one hash table per key pointing to a shared record with a life cycle as long as the last remaining key. The data structures have been implemented specifically for this project to reduce dependencies on the embedded client side.

[000149] Each test run is described as a test session and all needed state data is stored in such a table. HTTP related data is stored in the data cache, which allows partial re-use of previous test runs when allowed by the HTTP caching rules. The data is used during both relay and replay. The tables separate HTTP request headers, HTTP response headers and HTTP response bodies.

[000150] Due to dynamic web content and techniques to disable caching when loading a web page, URLs used in previous browser sessions may have slight variations. This results in some table lookups failing since the exact URL is not present in the table. To solve this problem fuzzy matching is used as a fall back when an exact lookup fails. If the URL is close enough a representative object can be fetched from the cache. Typically, the difference would be a different timestamp in the URL. This is implemented by computing Levenshtein distances, which is an edit-distance algorithm to calculate string similarity.

[000151] During the replay phase periodic screenshots are recorded. They are later analysed to compute various metrics (such as the time to first visual). The PNG files need to be compressed in order to reduce the storage space used and making it possible to identify duplicates. The images are first filtered with respect to uniqueness, which is decided based upon their size (a single byte difference represents a unique screenshot). Duplicate files are ignored, which is a common situation where the web engine hasn't rendered anything new after several repeated 200ms iterations.

[000152] The images that remain can be analysed more deeply in some embodiments of the invention as part of the web-test performed for a web-page. For example, a combination of pixel accurate delta calculations and structured image dissimilarity (DSSIM) techniques may be used where the quality of the picture is measured against a perfect uncompressed and distortion-free reference. This allows the measurements taken by the web-browsing test system to focus on the overall structure in the image, which is similar to how a human detects objects in a picture. In some embodiments of the web-browsing test system, the reference picture is another screenshot and is a measurement of the difference in load progress. Different embodiments of the web browsing test system may use different DSSIM implementations. Examples of suitable DSSIM include OpenCV, which is a highly efficient library commonly used for computer vision projects. Alternatively, the GDK library can be used for image file access and manipulation. Some embodiments of the web-browsing test system are implemented using image delta, as this also supports using a pixel threshold for ignoring minor pixel differences where the delta is represented with bounding rectangles.

[000153] During the browser's page rendering process, the web-engine may shift web content down graphically as it receives more data which shows more of the web-page (this is shown in FIG. 4B and FIG. 4C of the accompanying drawings). Progress calculations are compared against the final image previously obtained prior to shifting. If shifting has occurred, then the DSSIM calculation can detect a large structural difference, even if for a human observer there is almost no visual difference.

[000154] Some embodiments of the web-page test system corrected for image shift by performing image shift compensation. The shift is first detected by repeatedly comparing scanlines between both images. For each scanline a forward scan is performed to determine the shift length. By building a histogram of the shift lengths it is possible to come to a conclusion if the shift is significant and the vertical range of all the shifts. Then a new image is generated where the affected area is shift compensated so that when comparing against the reference screenshot the image structure is aligned between the two images. The DSSIM calculation is now representative of how a human perceives the image difference.

[000155] As mentioned, in some embodiments, the image analysis outputs is the time to 'first visual', time to 'visually done' and all intermediate visual progress steps between the screenshots. It is then used as input for the video generation.

[000156] The screenshot frequency is much lower than the video playback rate. The screenshots are mapped to the video timeline. The implementation is flexible and can present playback of any frame rate, but currently 30fps is used. The reason for such a high frame rate is the smoothness of the ticking timestamp, which has millisecond resolution. A very small randomness is added to the timestamp to look more representative, which otherwise would be increments of the same identical millisecond number.

[000157] In some embodiments of the invention, a vector graphics header is included in each frame where timestamps and load progress is presented. Then as time elapse screenshots are added/updated to represent how the user would have seen the web page being loaded. For example, PNG image generation for the video frames is performed with the help of libCairo. The final video file may conform to the x264 video standard or similar, for example, FFmpeg is used in some embodiments to encode an x264 video file based on the generated image frames.

[000158] In some embodiments, websites which use animations that never end are managed by limiting the maximum load time of the test to, for example, 30 seconds. Alternatively, in some embodiments, animated web-page performance metrics are measured by injecting modified animated GIFs to prevent as much image movement as in the original web-page but to still enable a representative measurement of the page load time to be provided by ensuring the same data size for the images loaded are used.

[000159] Further detailed example embodiments are now described in which a web browsing test system provides information indicating how a web-page is browsed based on test results from a large cluster of client-side web-test agents 50 running in different user networks 50. In some embodiments, the web-browsing test system captures data enabling analysis by geographical region or filtered by ISP. This at least the following performance metrics for a test web-page to be determined: tope level test results including one or more graphical page load progress measurements (time to first visual, time to visually done, time to visually useful) as well as a more detailed set of results for example, broken down to show timing information for DNS look-ups, connection establishment, transfer times or other top-level statistics.

[000160] Some embodiments of the web-test are configured to be implemented by ISPs to respond to a customer request to investigate a slow speed for a web-age to load, by allowing tests to be performed to indicate at a per-object level, the timing information to load a web-page.

[000161] Some embodiments of the invention are configured to allow other third-party web portals also support offering normal end-users the ability to check their web page access from their own network (believing their ISP is 'filtering' web sites) to be reached. The Web Test web-browsing test system could also give a hint of expected performance when comparing different ISPs and their product range. Web page load times could be measured with respect to the price tag of advertised bandwidth and latency of a connection. It could also possibly be an indication that some ISPs need to adjust their access to external networks or that the content providers need to have a local server in a particular geographical region or need to reduce the web page complexity.

[000162] In some embodiments, some results such as video files are uploaded to separate server which are then referenced as a URL. Below is what is stored in the database. All time measurements are indicated by a suitable unit of time, usually in microsecond units, except for Epoch time.

[000163] Examples of a top-level web-browsing test result data for a test web-page include on or more of the following results information elements: an indication of the result type, an indication of the start timestamp, for example in epoch time, the status of the test result, for example, OK / FAIL, any error codes and error strings which provide more detailed error messages, a test-session identifier for example, which may also identify which off-load server 52 was used to generate the web-page performance metrics), the web-page URL (the top-level web address), the type of browser engine, the client-side web-test agent version, the offload server version, the resolution of the display screen on which the web-page is rendered by the browser engine 160 of the offload server 52, for example, in a format such as: '1366x768', the total number of DNS requests required to load the web-page, the total number of objects referenced by the web-page, the total upload size, for example, the size in bytes of any payload data, excluding headers, the total download size (for example in bytes, including payload but excluding any payload headers, the number of successful objects loaded, the number of failed objects not loaded, the peak number of concurrent requests, the average DNS lookup time, DNS time percentiles (pet): for example, [min, 5th pet, 10th pet, 25th pet, 50th pet, 75th pet, 90th pet, 95th pet, max], the average TCP connection time, the TCP time percentiles, for example, [min, 5th pet, 10th pet, 25th pet, 50th pet, 75th pet, 90th pet, 95th pet, max], the average object transfer time, the object transfer time percentiles, for example, [min, 5th pet, 10th pet, 25th pet, 50th pet, 75th pet, 90th pet, 95th pet, max], the time to first visual, the time to when the browser engine reports it is done with processed page code components for rendering (this may be less than the visually complete time of the web-page displayed on the screen), the time to visually complete the display of the web-page on the screen, an example of a visual progress group, such as the time to achieve a certain percentage of visual progress: UINT64_LIST - [0-10%, 10-20%, 20-30%, 30-40%, 40-50%, 50-60%, 60-70%, 70-80%, 80-90%, 90-100%].

[000164] Other information results in some examples of the top-level test include a page load video, which allows the composite video of the video recording and dynamic performance metrics of the web- page to be retrieved later, and also for the web- resources/code-components used when loading the web-page, a page load waterfall diagram.

[000165] A lower level set of results in some embodiments provides a set of per-object results which are also recorded for every object that is referenced by a web page, for example, a result type, a start timestamp (in epoch time), a Test Session ID to identify offload server and test number, a Method to indicate the HTTP method used, the URL for the web-page being tested, the size of the objects, for example, in bytes an indication may be given of the payload size, excluding headers, of each object retrieved, the load speed of the object (in bytes/s), the HTTP version, the HTTP response code, the Server IP address, the request start time for the respective object (for example, since the replay started), the request end time (for example, since replay started), the DNS lookup time, which may be zero if not applicable, and the TCP connect time (which may be zero if existing TCP session is re-used)

[000166] In some embodiments, the client-side web-test agent 50 can be suitably invoked, for example, on the command line. It can potentially have many options to allow configuration of test setup. For example, the address of the web-page in the form of a URL may be provided, along with the domain name of the offload server, which may be repeated to allow dynamic reassignment to another offload server should the initial offload server not be available (for example, if there is no connectivity or its maximum loading has been reached). The client-side web-test agent 50 is expected to output nothing but test results in a suitable format, for example, as comma-separated values (CSV) results data when invoked. In some embodiments of the invention, the client-side web-test agent 50 is configured to perform data transfers for loading a web page and all its resources twice and test specific communication with the server in a low-power mode.

[000167] In some embodiments, the server side of the web-browsing test system is invoked during a web- test session by the web-test client 50 via a command line. In some embodiments, the off-load server 52 enables the relay mode via a TCP stream relay function using multiplexed connections over single TCP connection due to presence of NAT. The off-load server 52 is also configured to enable inspection of HTTP header/body inspection in relay mode of both request and responses and store relevant information in a data cache. The request header/body is also sent to client side for proper transfer measurements (including HTTP session cookie and similar). The off-load server 52 also enables HTTP replay using data cache, where each web engine instance has its own local data cache for instant lookup. Each HTTP response lookup allows fuzzy string-matching fall back in case the java script engine has dynamically generated a string to ensure that no regular caching can be performed of the URL. During replay the server will measure elapsed time using monotonic clock and take periodic screenshots of the replay. It then analyses the images and calculate image delta so that a conclusion about first visual and visually done can be reached.

[000168] In some embodiments, to manage web sites that provide never ending animations (image carousels or videos) the off-load server 52 must modify the JavaScript replay content so that these timers are extended beyond the measurement window to enable the video analysis and any other visually based measurement representative and accurate. The off-load server 52 must also periodically ensure that web resources (stored in data cache or similar) that have not been referenced for a long time (i.e. referenced for a time over a predetermined threshold for example) are cleared so as to free up memory. In some embodiments, the off-load server 52 is configured to share storage of HTTP response bodies between several concurrent test sessions to reduce memory.

[000169] In some embodiments, the off-load server 52 reduces data transfers in relay mode 70 by relying on HTTP 1.0/1.1 cache fields in the response headers. HTTP response data is taken from local data cache whenever possible. This reduces overall relay time and for clients with slow connections it is essential to reduce risk of relay timeout (for example, after 90s). In some embodiments, the server side of the web browsing test system is able to identify and report errors/faults to the client side: for example, server at max capacity, server unable to produce URL list, server unable to replay or expired test session.

[000170] Some statements reciting further examples of embodiments of the invention follow:

[000171] Clause 1 : A distributed web-page testing system for remotely testing how a web-page loads in a user network, the system comprising: a client-side web-test agent located in the user network, wherein the client-side web-test agent is configured to capture timing information associated with retrieving one or more code objects from one or more sources; and an off-load server having a web-browser configured to: perform an initial load of the web-page code to identify one or more code objects referenced by the web-page and one or more sources for the identified one or more code objects, wherein at least one of the code-objects comprises a code-object which is dynamically referenced by the web-page as the web-page is loaded by the web-browser running on the off-load server; provide an indication of at least each dynamically referenced code object to the client-side web-test agent; receive timing information captured by the client- side web-test agent associated with retrieving each indicated dynamically referenced code object; and load the web-page using timing information captured by the web-test agent retrieving each of the indicated dynamically referenced code-objects for emulating the rendering of the web-page as it would be performed by a web-browser running within the user network, wherein the initial load of the web-page by the off-load server web-browser is performed via a tunnel established between the off-load server to the client-side web-test agent, whereby the objects requested by the off- load server in the initial load are requested via the tunnel from the user-network.

[000172] Clause 2. A distributed web-page testing system for remotely determining perceivable rendering metrics to test how a web-page loads in a user network, the system comprising: a client-side web-test agent located at a location in the user network, wherein the client-side web-test agent is configured to capture timing information associated with retrieving one or more code objects from one or more sources; and an off-load server located at another location different from the location of the client-side web-test agent, the off-load server having a web-browser configured to: perform an initial load of the web-page code to identify one or more code objects referenced by the web-page and address information for one or more code objects sources from which the identified one or more code objects are retrievable, wherein the initial load of the web-page by the off-load server web-browser is performed via a tunnel established between the off-load server and the client-side web-test agent, wherein the one or more code objects are identified and retrieved based on the location of the client-side web-test agent, wherein at least one of the code-objects comprises a code-object which is dynamically referenced by the web-page as the web-page is loaded by the web-browser running on the off load server, and wherein the web-browser of the off-load server is further configured to: provide an indication of at least the identity of each dynamically referenced code-object to the client-side web-test agent; receive timing information captured by the client-side web-test agent associated with the client-side web-test agent retrieving each indicated dynamically referenced code object; and load the web-page using the received timing information to emulate the rendering of the web-page as it would be performed by a web-browser running on a device within the user network.

[000173] Clause 3: A distributed web-page testing system according to either clause 1 or 2, wherein the timing information associated with retrieving the code objects is captured by the client-side web-test agent without causing rendering of the web-page at the remote location.

[000174] Clause 4: A distributed web-page testing system according to any of clauses 1 to 3, wherein a device unable to render a web-page hosts the client-side web-test agent located in the user-network.

[000175] Clause 5: A web-page testing system according to any previous clause, wherein the timing information provided by the client-side web-test agent includes timing information for the client to perform an address look-up operation to obtain an indication of a source address of at least one code component associated with the web-page, in addition to the timing information for retrieving the code component from the indicated source address.

[000176] Clause 6: A distributed web-page testing system as claimed in any previous clause, configured to test a web-page on a plurality of client-side web-test agents, each client-side web-test agent located in a different user-network, wherein the off-load server is remotely located from the plurality of user-networks.

[000177] Clause 7: A distributed web-page testing system as claimed in clause 6, wherein the web-page testing system further comprises: a test-server configured to provide an indication of a web-page to be tested to each of the client-side web-test agents.

[000178] Clause 8: A distributed web-page testing system as claimed in any previous clause, wherein each said client-side web-test agent is configured to: receive an indication of a web-page from the test- server; request a said remote off-load server to provide at least an indication of a plurality of code-objects required to load the web page; and for each code-object retrieved, capture timing information indicating at least an amount of time for the client-side web-test agent to retrieve each code object from a data source, and send at least the timing information for each code object the indicated by the remote off-load server back to at least one off- load server.

[000179] Clause 9: A distributed web-page testing system as claimed in any previous clause; wherein a off-load server comprises a web-browser which is configured to: receive for each code component an indication of a suitable data source and related data retrieval timing information from the client-side web-test agent; retrieve the code components from the data sources indicated by the received data source information for the web-page; cache the received code components; build the web-page for display by retrieving the cached code- components using the received retrieval timing information from the client-side web-test agent; and render the web-page on a display using each cached code-components timing information.

[000180] Clause 10. A distributed web-page testing system as claimed in any previous clause, wherein the initial load of the web-page by the off-load server web-browser is performed via a tunnel established between the off-load server to the client-side web- test agent, whereby the objects requested by the off-load server in the initial load are requested via the tunnel from the user-network.

[000181] Clause 11. A distributed web-browsing test system according to any previous clause, further comprising at least one server configured to perform: recording the rendering of the web-page on the display using the timing information from the client- side web-test agent in a video file; terminating recording the video file when it is determined the rendering on the display has been completed for at least the initially visible part of the web-page; analysing one or more still images from the video file video file; and determining one or more performance metrics for the web-page by associating the loading of referenced objects from the web-page with content displayed in the still images.

[000182] Clause 12. A distributed web-browsing test system according to clause 11 , further comprising at least one server configured to perform: generating a composite video of the rendering of the web-page with dynamically updating timing information for one or more performance metrics of the web-page rendering process.

[000183] Clause 13. A distributed web-browsing test system according to clause 11 or 12, wherein the performance metrics include one or more of: a time to first visual; a time to visually complete; and a web-page load progress indicator.

[000184] Clause 14. A distributed web-browsing test system as claimed in any preceding clause, where the client-side web-test agent measures the time to perform DNS look-ups for one of more of said code components and provides the off-load server with the DNS look-up times.

[000185] Clause 15. A distributed web-browsing test system according to any previous clause, wherein the system creates a results file for storing data indicating how the web- browser of the off-load server interacted with at least each object referenced in the critical path of the web-page being tested. In some embodiments, the results file provided is a HAR format file.

[000186] Clause 16. A method for remotely assessing the loading performance of a test web-page as it would be rendered by a web-browser in a remote user network, wherein the method comprises: receiving an indication of a test web-page; determining, using a web-browser running on an offload server, a plurality of data objects referenced by the test web-page; identifying data sources for retrieving the data objects referenced by the web-page or objects of the web-page references by the web-page as it loads in a web- browser, wherein the initial load of the web-page by the off-load server web-browser is performed via a tunnel established between the off-load server to a web-test agent in the remote user network, whereby the objects requested by the off-load server in the initial load are requested via the tunnel from the user-network; causing the web-test agent running in the remote user network to render the web-page to: retrieve the data components required to load the web-page in the web-browser, wherein the web-test agent monitors timing information as it retrieves the data components from the identified data sources; and provide the timing information measured by the web-test agent as it retrieves each data component required to the off-load server; and storing the provided timing information in association with each data component used by the web-page or an object called by the web-page.

[000187] Clause 17. A method according to clause 16, further comprising the web- browser running on the off-load server loading the web-page using the timing information for each required data component measured by the web-test agent.

[000188] Clause 18. A method according to clause 17, further comprising creating a results file for storing data indicating how the web-browser of the off-load server interacted with at least each object referenced in the critical path of the web-page being tested.

[000189] Clause 19. A method according to clause 17 or 18, wherein when the web- browser running on the off-load server loads the web-page for rendering on a display, a video recording of the loading progress on a display is started and the detected completion of the web-page loading automatically causes stopping of the recording .

[000190] Clause 20. A method according to clause 19, further comprising determining, using at least the time to complete loading of the web-page from the recorded video, a number of perceptual web-page loading performance metrics.

[000191] Clause 21. A method according to clause 19 or 20, further comprising capturing at set intervals of time, screen-shots of a display on which the web-page is being loaded using the video showing the process of loading the web-page.

[000192] Clause 22. A method according to clause 19 or 20, wherein the web-page loading performance metrics determined from the video recorded comprise one or more of: time to first visual of the loading web-page; time to the browser engine determining loading the web-page is visually complete; and time to visually complete the loading web-page.

[000193] Clause 23. A distributed web-page testing system for remotely assessing the loading performance of a test web-page on a web-browser running on a device in a remote user network, the system comprising: means for receiving an indication of a test web-page; means for determining, using a web-browser running on an offload server, a plurality of objects which are called by the test web-page; means for identifying data sources for retrieving the objects referenced by the web-page as it loads in a web- browser, wherein the objects requested by the web-browser running in the off-load server are requested via a tunnel established with a web-test agent in the user-network; means for causing test client running in the remote network to: retrieve the data components including data objects referenced by the test web- page required to load the web-page in the web-browser, wherein the test client monitors timing information as it retrieves the data components from the identified data sources; and provide the timing information measured by the test client as it retrieves each data component required to the off-load server; and means for storing the provided timing information in association with each data component used by the web-page or an object called by the web-page.

[000194] Clause 24. A distributed web-page testing system according to clause 23, wherein the web- browser running on the off-load server loads the web-page using the indicated timing information for each required data component and stores data indicating how the web-browser of the off-load server interacted with each object referenced in at least the critical path of the web-page as it is loaded.

[000195] Clause 25. A distributed web-page testing system according to clause 24, wherein the web-browser loading the web-page triggers starting a video recording of the loading progress on a display and the completion of the web-page loading causes stopping of the recording.

[000196] Clause 26. A distributed web-page testing system method according to clause 25, wherein the time to complete loading of the web-page recorded video is used to determine a number of perceptual web-page loading performance metrics.

[000197] Clause 27. A distributed web-page testing system according to clause 26, wherein the web-page loading performance metrics determined from the video recorded comprise one or more of: time to first visual; time to browser engine says loading the web-page is done; and time to visually complete.

[000198] Clause 28. A distributed web-page testing system according to clause 27, wherein the time used to determine the performance metrics is the elapsed time from a detected or determined triggering event, for example, a triggering event associated with requesting or loading the web-page under test.

[000199] Clause 29. A computer-software product which when executed on a computer-system implements a method for remotely assessing a loading performance of a web-page on a web- browser running at a remote location, comprising: determining one or more network conditions which would be experienced by the remote web- browser when loading the web-page at the remote location; simulating the remote web- browser loading the web-page by replicating the network conditions when loading the web-page on a test web-browser by establishing a tunnel with the remote network via which objects referenced by the web-page can be identified; and determining at least one perceptual metric associated with a visual presentation performance parameter for the web-page as it is loaded by the test web-browser. [000200] Clause 30. A computer-software product according to clause 29, further comprising: starting a video recording when simulating the remote web-browser loading the web-page and stopping the video when the web-page has been completely loaded, wherein the at least one perceptual metric is determined using an initial image in the video and a final image in the video, wherein the final image is taken when the web page has completely loaded.

[000201] Clause 31. A computer-software product according to clauses 27 to 30, wherein a said perceptual metric comprises a measurement of one of: a time to first visual of the web-page; a time to the browser engine determining loading the web-page is visually complete; and a time to visually complete the loading of the web-page.

[000202] Clause 32. A web-test client for use in a system according to clauses 1 to 15 or 23 to 28, the web-test client comprising at least: means to receive data from an off load server identifying at least a plurality of remote web- server sources for code objects or components referenced by a web-page; means to monitor timings of connections established to retrieve web-page data objects or components from the plurality of remote web-servers; and means to provide the monitored timing information associated with processing each identified code-object and retrieving each identified web-page data object- associated with loading the web-page which is retrieved to the off-load server.

[000203] Clause 33. A distributed web-browsing test system which captures object- level loading performance information of a web-page in a web-browsing session operated remotely on a device in a remote data network without user interaction.

[000204] Clause 34. A distributed web-browsing test system according to clause 33, wherein the object- level loading performance information of the web-page in the web browsing session operated remotely on a device in a remote data network without user interaction is captured by recording HAR files of web browsing sessions operated remotely on a device in a user's home without user interaction.

[000205] Clause 35. A distributed web-browsing test system according to any of clauses 33 to 34, wherein the object-level loading performance information includes at least timing information for retrieving and loading each object referenced by the web page.

[000206] Clause 36. A method comprising capturing object-level loading performance information of a web-page in a web-browsing session operated remotely on a device in a remote data network without user interaction.

[000207] Clause 37. A method according to clause 36, wherein the object-level loading performance information of the web-page in the web-browsing session operated remotely on a device in a remote data network without user interaction is captured by recording HAR files of web browsing sessions operated remotely on a device in a user's home without user interaction.

[000208] Clause 38. A method according to clause 37, wherein the object-level loading performance information includes at least timing information for retrieving and loading each object referenced by the web-page.

[000209] Clause 39. The system of clause 33, wherein the system is a system according to any of clauses 1 to 15 or 23 to 28.

[000210] Clause 39. The method of any of clauses 35 to 27 further comprising steps according to any of clauses 16 to 22.

[000211] Clause 40. A server for use in a server system according to any previous system clause.

[000212] Clause 41. A server for determining metrics associated with displaying a web page on a remotely located display device, the server being configured to establish communications with at least one client-side web-test agent device connected to a communications network, wherein the server is remotely located from the web-test agent, the server comprises a web-browser configured to: load code for displaying a web-page to identify one or more code objects referenced by the web-page during the loading of the web-page; and determine one or more sources for the identified one or more code objects, wherein at least one of the one or more code-objects comprises a code-object which is dynamically referenced by the web-page as the web-page is loaded by the web-browser running on the off-load server, wherein the server is further configured to: establish communications with the client-side web-test agent device and provide an indication of at least the dynamically referenced code-object to the client-side web-test agent device; receive timing information captured by the remote client-side web-test agent device as the client-side web-test agent retrieves each indicated code object identified by the off-load server; cause, using the web-browser, the web-page to be loaded in dependence on the received timing information; and determine one or more metrics associated with displaying loading web-page, wherein the timing information associated with retrieving the code objects is captured without causing rendering of the web-page at the remote location.

[000213] Clause 42. A server according to clause 41 , configured for use in a server system according to any previous system clause.

[000214] Clause 43. A server according to any of clauses 41 to 42 configured as an off load server for use in a method according to any of clauses 16 to 22, 35 to 27, or 39.Aspects of any of the examples of the embodiments described herein may be combined with aspects of any of the other examples described to form further examples of embodiments of the invention without losing the effect sought and without requiring explicit pointers if implicit pointers based on the explicit disclosures herein would be apparent to someone of ordinary skill in the art.

[000215] Clause 44. A distributed web-page testing system for remotely testing how a web-page loads in a user network, the system comprising: a client-side web-test agent located in the user network, wherein the client-side web-test agent is configured to capture timing information associated with retrieving one or more code objects from one or more sources; and an off-load server having a web-browser configured to: perform an initial load of the web-page code to identify one or more code objects referenced by the web-page and one or more sources for the identified one or more code objects, wherein at least one of the code-objects comprises a code-object which is dynamically referenced by the web-page as the web-page is loaded by the web-browser running on the off-load server; provide an indication of at least each dynamically referenced code object to the client-side web-test agent; receive timing information captured by the client- side web-test agent associated with retrieving each indicated dynamically referenced code object; and load the web-page using timing information captured by the web-test agent retrieving each of the indicated dynamically referenced code-objects for emulating the rendering of the web-page as it would be performed by a web-browser running within the user network.

[000216] Clause 45. A distributed web-page testing system according to clause 44, wherein a device unable to render a web-page hosts the client-side web-test agent located in the user-network.

[000217] Clause 46. A web-page testing system according to clause 44, wherein the timing information provided by the client-side web-test agent includes timing information for the client to perform an address look-up operation to obtain an indication of a source address of at least one code component associated with the web-page, in addition to the timing information for retrieving the code component from the indicated source address.

[000218] Clause 47. A distributed web-page testing system according to any of clauses 44-46, configured to test a web-page on a plurality of client-side web-test agents, each client-side web-test agent located in a different user-network, wherein the off-load server is remotely located from the plurality of user-networks, wherein the web-page testing system further comprises: a test-centre server configured to provide an indication of a web-page to be tested to each of the client-side web-test agents, and wherein each said client-side web-test agent is configured to: receive an indication of a web-page from the test-centre server; request a said remote off-load server to provide at least an indication of a plurality of code-objects required to load the web-page; and for each code-object retrieved, capture timing information indicating at least an amount of time for the client- side web-test agent to retrieve each code object from a data source, send at least the timing information for each code object the indicated by the remote off-load server back to the off-load servers; wherein a said off-load server comprises a web-browser engine which is configured to: receive for each code component an indication of a suitable data source and related data retrieval timing information from the client-side web-test agent; retrieve the code components from the data sources indicated by the received data source information for the web-page; cache the received code components; build the web-page for display by retrieving the cached code-components using the received retrieval timing information from the client-side web-test agent; and render the web page on a display using each cached code-components timing information.

[000219] Clause 48. A distributed web-test system according to clause 47, further comprising: recording the rendering of the web-page on the display using the timing information from the client-side web-test agent in a video file; terminating recording the video file when it is determined the rendering on the display has been completed for at least the initially visible part of the web-page; analysing one or more still images from the video file to determine one or more performance metrics for the web-page.

[000220] Clause 49. A distributed web-test system according to clause 48, further comprising: generating a composite video of the rendering of the web-page with dynamically updating timing information for one or more performance metrics of the web-page rendering process.

[000221] Clause 50. A distributed web-test system according to clause 48 or 49, wherein the performance metrics include one or more of: a time to first visual; a time to first usual; and a web-page load progress indicator.

[000222] Clause 51. A distributed web-test system according to any of clauses 44-50, where the client-side web-test agent measures the time to perform DNS look-ups for one of more of said code components and provides the off-load server with the DNS look-up times.

[000223] Clause 52. A method for remotely assessing the loading performance of a test web-page as it would be rendered by a web-browser in a remote user network, the method comprises: receiving an indication of a test web-page; determining, using a web- browser running on an offload server, a plurality of data objects referenced by the test web-page; identifying data sources for retrieving the data objects referenced by the web- page or objects of the web-page references by the web-page as it loads in a web- browser; causing a web-test agent running in the remote user network on a device without any resources to render the web-page to: retrieve the data components required to load the web-page in the web-browser, wherein the web-test agent monitors timing information as it retrieves the data components from the identified data sources; and provide the timing information measured by the web-test agent as it retrieves each data component required to the off-load server; and storing the provided timing information in association with each data component used by the web-page or an object called by the web-page.

[000224] Clause 53. A method according to clause 52, further comprising the web- browser running on the off-load server loading the web-page using the timing information for each required data component measured by the web-test agent.

[000225] Clause 54. A method according to clause 53, wherein when the web-browser running on the off-load server loads the web-page for rendering on a display, a video recording of the loading progress on a display is started and the detected completion of the web-page loading automatically causes stopping of the recording.

[000226] Clause 55. A method according to clause 54, further comprising determining, using at least the time to complete loading of the web-page from the recorded video, a number of perceptual web-page loading performance metrics.

[000227] Clause 56. A method according to clause 54 or 55, further comprising capturing at set intervals of time, screen-shots of a display on which the web-page is being loaded using the video showing the process of loading the web-page.

[000228] Clause 57. A method according to clause 54 or 56, wherein the web-page loading performance metrics determined from the video recorded comprise one or more of: time to first visual of the loading web-page; time to the browser engine determining loading the web-page is visually complete; and time to visually complete the loading web-page.

[000229] Clause 58. A distributed web-page testing system for remotely assessing the loading performance of a test web-page on a web-browser running on a device in a remote user network, the system comprising: means for receiving an indication of a test web-page; means for determining, using a web-browser running on an offload server, a plurality of objects which are called by the test web-page; means for identifying data sources for retrieving the objects referenced by the web-page as it loads in a web- browser; means for causing test client running in the remote network on a device without any resources to render the web-page to: retrieve the data components required to load the web-page in the web-browser, wherein the test client monitors timing information as it retrieves the data components from the identified data sources; and provide the timing information measured by the test client as it retrieves each data component required to the off-load server; and means for storing the provided timing information in association with each data component used by the web-page or an object called by the web-page.

[000230] Clause 59. A distributed web-page testing system according to clause 58, wherein the web-browser running on the off-load server loads the web-page using the indicated timing information for each required data component.

[000231] Clause 60. A distributed web-page testing system according to clause 59, wherein the web-browser loading the web-page triggers starting a video recording of the loading progress on a display and the completion of the web-page loading causes stopping of the recording.

[000232] Clause 61. A distributed web-page testing system method according to clause 60, wherein the time to complete loading of the web-page recorded video is used to determine a number of perceptual web-page loading performance metrics.

[000233] Clause 62. A distributed web-page testing system according to clause 61 , wherein periodic snapshots of the video showing progress of loading a web-page are associated with a number of said measured perceptual web-page loading performance metrics.

[000234] Clause 63. A distributed web-page testing system according to clause 61 or 62, wherein the web-page loading performance metrics determined from the video recorded comprise one or more of: time to first visual; time to browser engine says loading the web-page is done (which may be less than the time for the web-page to be perceived to be visually complete); and time to visually complete.

[000235] Clause 64. A computer-software product which when executed on a computer-system implements a method for remotely assessing a loading performance of a web-page on a web-browser running at a remote location, comprising: determining one or more network conditions which would be experienced by the remote web- browser when loading the web-page at the remote location; simulating the remote web- browser loading the web-page by replicating the network conditions when loading the web-page on a test web-browser; and determining at least one perceptual metric associated with a visual presentation performance parameter for the web-page as it is loaded by the test web-browser.

[000236] Clause 65. A computer-software product according to clause 64, further comprising: starting a video recording when simulating the remote web-browser loading the web-page and stopping the video when the web-page has been completely loaded. [000237] Clause 66. A computer-software product according to clause 64 or 65, wherein the at least one perceptual metric is determined using an initial image in the video and a final image in the video, wherein the final image is taken when the web page has completely loaded.

[000238] Clause 67. A computer-software product according to any of clauses 64 to 66, wherein a said perceptual metric comprises a measurement of one of: a time to first visual of the web-page; a time to the browser engine determining loading the web-page is visually complete; and a time to visually complete the loading of the web-page.

[000239] Clause 68. A web-test client for use in a system according to any one of clauses 44 to 5 or 58 to 63, the web-test client comprising at least: means to receive data from an off-load server identifying at least a plurality of remote web-server sources for code objects or components referenced by a web-page; means to monitor timings of connections established to retrieve web-page data objects or components from the plurality of remote web-servers; and means to provide the monitored timing information associated with processing each identified code-object and retrieving each identified web-page data object- associated with loading the web-page which is retrieved to the off-load server.

[000240] Features of any of the aspects of any of the examples of the embodiments described herein may be combined with features of any of the aspects of any of the other examples described to form further examples of embodiments of the invention without losing the effect sought and without requiring explicit pointers if implicit pointers based on the explicit disclosures herein would be apparent to someone of ordinary skill in the art.

[000241] The methods described hereinabove may be performed in software in machine-readable form on one or more tangible storage mediums, including tangible mediums which are distributed over a range of physical locations. Such software, whether implemented in a distributed manner or not, may comprise one or more computer programs, each comprising computer program code which provides means adapted to perform one or more or all of the steps of the methods described herein when the program is run on one or more computers and where the computer program is embodied on one or more computer-readable mediums.

[000242] Examples of tangible storage media include computer storage devices comprising computer-readable media such as memory components, as well as disks, drives, etc. Some or all of the computer-code which is required to implement the invention may be delivered to such tangible storage media from a remote source in the form of a propagated signal in some embodiments of the invention. Those skilled in the art will appreciate that storage devices may be distributed across one or more networks, and a remote computer/server may store software which if executed on the client-side implements part or all of an embodiment of a web-browsing test system or web browsing test system process or method according to an example of an embodiment of the invention. A local CPE such as a router may then download components of the software needed to run the web-test on the client-side by accessing the remote computer/server for the code required and then download the software.

[000243] The terms software or computer program(s) as used herein include software and computer program(s) comprising one or more a commercial products. In some embodiments, software which implements a web-browsing test system according to an embodiment of the invention runs or control standard computer hardware to carry out the desired functionality of the web-browsing test system, and may be hosted either locally on such standard computer hardware and/or remotely cause such standard computer hardware to implement one or more steps in a web-test conducted by the web-browsing test system.

[000244] The terms software or computer program(s) also refers to software and computer program(s) which define the configuration of hardware, such as HDL (hardware description language) software for configuring programmable chips to carry out desired functions. The software may be suitable for execution on a parallel processor or a serial processor such that the method steps may be carried out in any suitable order, or simultaneously. Those of ordinary skill in the art will also realise that in some embodiments, some or all of the software instructions may be carried out by a dedicated circuit and/or by using programmable logic or the like.

[000245] The example embodiments of an off-load server described herein may be implemented using any suitable computer system/hardware. For example, it may take the form of a general-purpose computing device having one or more processing units, a memory, one or more input devices, one or more output devices (including devices suitable for outputting to a local or remote screen on which a web-page can be rendered by a browser engine running on the off-load serer), one or more data storage component(s), and one or more data communication components. The processing unit(s) may be real and/or virtual processor(s), each processor being capable of performing a various processes in accordance with program instructions stored in memory. Memory may be volatile memory (e.g. registers, cache, random-access memory (RAM) or non-volatile memory (e.g. read-only memory (ROM), flash memory, or the like). The data storage may be removable or non-removable, local or remote, and may include network or cloud-based data storage. The data storage may take any suitable form of machine-readable data storage medium for storing information in a suitable form which can be accessed by the computing system of the off-load server, for example, magnetic disks, optical disks, flash drive either locally or remotely, including from a remote cloud-based storage system which such data storage may be distributed.

[000246] The example embodiments of an off-load server described herein also comprises means to write and read from such data-storage and memory, including, for example, suitable drivers and/or interface components. The off-load server also includes a suitable operating system, as well as one or more applications and/or program modules/components and data for implementing the web-browsing test system. The off load server’s memory may include at least one program component of software configured to implement an embodiment of the web-browsing test system according to the invention comprising a set of program modules or code components which are configured to carry out the functionality of the various embodiments of the web-test off load server described herein. The off-load server includes communication components enabling the off-load server to communicate as required by the embodiments described herein, for example, a using a suitable network card or the like.

[000247] Program code for carrying out the methods described herein may be written in any combination of one or more programming languages. The program code executes on one or more components of the web-browsing test system such as a client-side web- test agent and/or off-load server and/or the test-. Similarly, the client-side web-test agent will comprise suitable hardware and software components for implementing the client- side of a web-browsing test system according to an embodiment of the invention, including but not limited to: a device having one or more processing units, a memory, one or more data interfaces, one or more data storage component(s), and one or more data communication components. The processing unit(s) may be real and/or virtual processor(s), each processor being capable of performing a various processes in accordance with program instructions stored in memory. Memory may be volatile memory (e.g. registers, cache, random- access memory (RAM) or non-volatile memory (e.g. read-only memory (ROM), flash memory, or the like). The data storage may be removable or non-removable, local or remote, and may include network or cloud-based data storage. The data storage may take any suitable form of machine-readable data storage medium for storing information in a suitable form which can be accessed by the computing system of the off-load server, for example, magnetic disks, optical disks, flash drive either locally or remotely, including from a remote cloud-based storage system which such data storage may be distributed. The communications components allow communications with at least the web-test off-load server and test-center but may comprise a suitable interface to communications components hosted on a different device, such as a router, on the CPE.

[000248] Embodiments of the web-browser/ web-browsing engine of the off-load server referred to herein are capable of generating files such as HAR files or equivalent file- types which store data capturing at a per object level how the web-browser/web- browsing engine interact with objects referenced by a web-page as the web-page is being loaded.

[000249] Whilst methods and operations may be described above as being implemented in a particular order, especially when referring to the drawings, such operations may not be required to be performed in the particular order shown or even sequentially in some embodiments where multitasking and/or parallel processing is implemented, to achieve the same results. Whilst specific implementation details have been provided for particular examples of embodiments of the invention, the invention should not be construed as limited to the scope of the subject matter described herein, as such descriptions are of features that may be specific only to particular embodiments. In some embodiments, certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable sub combination.

Claims

1. A distributed web-page testing system for remotely determining perceivable rendering metrics to test how a web-page loads in a user network, the system comprising:

a client-side web-test agent located at a location in the user network, wherein the client-side web-test agent is configured to capture timing information associated with retrieving one or more code objects from one or more sources; and

an off-load server located at another location different from the location of the client- side web-test agent, the off-load server having a web-browser configured to:

perform an initial load of the web-page code to identify one or more code objects referenced by the web-page and address information for one or more code-objects sources from which the identified one or more code objects are retrievable, wherein the initial load of the web page by the off-load server web-browser is performed via a tunnel established between the off load server and the client-side web-test agent, wherein the one or more code objects are identified and retrieved based on the location of the client-side web-test agent, wherein at least one of the code-objects comprises a code-object which is dynamically referenced by the web-page as the web-page is loaded by the web-browser running on the off-load server, and wherein the web- browser of the off-load server is further configured to:

provide an indication of at least the identity of each dynamically referenced code object to the client-side web-test agent;

receive timing information captured by the client-side web-test agent associated with the client-side web-test agent retrieving each indicated dynamically referenced code object; and load the web-page using the received timing information to emulate the rendering of the web-page as it would be performed by a web-browser running on a device within the user network.

2. A system as claimed in claim 1 , wherein the timing information is captured using a device which hosts the client-side web-test agent located in the user-network without causing any local display of the web-page.

3. A system as claimed in claim 1 or 2, wherein the timing information provided by the client- side web-test agent includes timing information captured by the client-side web-test agent associated with performing an address look-up operation to obtain an indication of a source address of at least one code object associated with the web-page, in addition to the timing information for retrieving the code object from the indicated source address.

4. A system as claimed in any previous claim, configured to test a web-page on a plurality of client-side web-test agents, each client-side web-test agent located in a different user-network, wherein the off-load server is remotely located from each of the plurality of user-networks, and wherein the web-page testing system further comprises:

a test-center server configured to provide an indication of a web-page to be tested to each of the client-side web-test agents, wherein each said client-side web-test agent is configured to:

receive an indication of a web-page from the test-center server;

request a said remote off-load server to provide at least an indication of a plurality of code-objects required to load the web-page; and

for each code-object retrieved, capture timing information indicating at least an amount of time for the client-side web-test agent to retrieve each code object from a data source, send at least the timing information for each code object the indicated by the remote off-load server back to the off-load servers.

5. A system as claimed in any of claims 1 to 4, wherein, in addition to receiving timing information from the client-side web-test agent for a code-object, the off-load server also receives the address of each data source from which a code-object was retrieved by the client-side web-test agent, wherein the web-browser running on the off-load server is further configured to:

retrieve each code object from the same address at which it was retrieved by the client-side web-test agent;

cache the received code objects;

build the web-page for display by retrieving the cached code-objects using the received retrieval timing information from the client-side web-test agent; and

render the web-page on a display.

6. A system as claimed in claim 5, wherein the off-load server is further configured to:

measure the amount of perceivable content displayed as the web-page is rendered on a display.

7. A system as claimed in claim 6, wherein the off-load server measures the amount of perceivable content displayed as the web-page is rendered by:

recording the rendering of the web-page on the display using the timing information from the client-side web-test agent in a video file;

terminating recording the video file when it is determined the rendering on the display has been completed for at least the initially visible part of the web-page; analysing one or more still images from the video file; and

determining one or more performance metrics for the web-page by associating the loading of referenced objects from the web-page with content displayed in the still images.

8. A system as claimed in claim 7, further comprising: generating a composite video which displays the rendering of the web-page with timing information for one or more perceivable rendering performance metrics of the web-page rendering process.

9. A system as claimed in any one of claims 5 to 8, wherein the perceivable rendering performance metrics include one or more of: a time to first visual; a time to visually complete; and a web-page load progress indicator.

10. A system as claimed in any preceding claim, where the client-side web-test agent measures the time to perform DNS look-ups for one of more of said code components and provides the off-load server with the DNS look-up times.

11. A system as claimed in any previous claim, wherein the system creates a results file for storing data indicating how the web-browser of the off-load server interacted with at least each object referenced in the critical path of the web-page being tested.

12. A distributed web-page testing method for remotely assessing the loading performance of a test web-page on a web-browser running on a device in a remote user network, the method comprising:

receiving an indication of a test web-page;

determining, using a web-browser running on an offload server, a plurality of objects which are called by the test web-page;

identifying data sources for retrieving the objects referenced by the web-page as it loads in a web-browser, wherein the objects requested by the web-browser running in the off-load server are requested via a tunnel established with a web-test agent in the user- network;

causing the web-test client running in the remote network to retrieve the data components including data objects referenced by the test web-page required to load the web-page in the web-browser, wherein the test client: monitors timing information as it retrieves the data components from the identified data sources;

provides the timing information measured by the test client as it retrieves each data component required to the off-load server; and

the method further comprises

storing the provided timing information in association with each data component used by the web-page or an object called by the web-page.

13. A computer-software product which when executed on a computer-system implements the method of claim 12.

14. A web-test client for use in a system as claimed in any one of claims 1 to 11 , the web-test client agent comprising at least:

means to receive data from an off-load server identifying at least a plurality of remote web- server sources for code objects or components referenced by a web-page;

means to monitor timings of connections established to retrieve web-page data objects or components from the plurality of remote web-servers; and

means to provide the monitored timing information associated with processing each identified code-object and retrieving each identified web-page data object- associated with loading the web-page which is retrieved to the off-load server.

15. A server for determining metrics associated with displaying a web-page on a remotely located display device, the server being configured to establish communications with at least one client-side web-test agent device connected to a communications network, wherein the server is remotely located from the web-test agent, the server comprises a web- browser configured to:

load code for displaying a web-page to identify one or more code objects referenced by the web-page during the loading of the web-page; and

determine one or more sources for the identified one or more code objects, wherein at least one of the one or more code-objects comprises a code-object which is dynamically referenced by the web-page as the web-page is loaded by the web-browser running on the off-load server,

wherein the server is further configured to: establish communications with the client-side web-test agent device and provide an indication of at least the dynamically referenced code-object to the client-side web-test agent device;

receive timing information captured by the remote client-side web-test agent device as the client-side web-test agent retrieves each indicated code object identified by the off-load server;

cause, using the web-browser, the web-page to be loaded in dependence on the received timing information; and

determine one or more metrics associated with displaying loading web-page, wherein the timing information associated with retrieving the code objects is captured without causing rendering of the web-page at the remote location.

16. A distributed web-page testing system for remotely testing how a web-page loads in a user network, the system comprising: a client-side web-test agent located in the user network, wherein the client-side web-test agent is configured to capture timing information associated with retrieving one or more code objects from one or more sources; and an off-load server having a web-browser configured to: perform an initial load of the web-page code to identify one or more code objects referenced by the web-page and one or more sources for the identified one or more code objects, wherein at least one of the code-objects comprises a code-object which is dynamically referenced by the web page as the web-page is loaded by the web-browser running on the off-load server; provide an indication of at least each dynamically referenced code-object to the client-side web-test agent; receive timing information captured by the client-side web-test agent associated with retrieving each indicated dynamically referenced code object; and load the web-page using timing information captured by the web-test agent retrieving each of the indicated dynamically referenced code-objects for emulating the rendering of the web-page as it would be performed by a web-browser running within the user network.

17. A distributed web-page testing system as claimed in claim 16, wherein a device unable to render a web-page hosts the client-side web-test agent located in the user-network.

18. A method for remotely assessing the loading performance of a test web-page as it would be rendered by a web-browser in a remote user network, the method comprises:

receiving an indication of a test web-page;

determining, using a web-browser running on an offload server, a plurality of data objects referenced by the test web-page;

identifying data sources for retrieving the data objects referenced by the web-page or objects of the web-page references by the web-page as it loads in a web-browser;

causing a web-test agent running in the remote user network on a device without any resources to render the web-page to:

retrieve the data components required to load the web-page in the web-browser, wherein the web-test agent monitors timing information as it retrieves the data components from the identified data sources; and

provide the timing information measured by the web-test agent as it retrieves each data component required to the off-load server; and

19. A distributed web-page testing system for remotely assessing the loading performance of a test web-page on a web-browser running on a device in a remote user network, the system comprising:

means for receiving an indication of a test web-page;

means for determining, using a web-browser running on an offload server, a plurality of objects which are called by the test web-page;

means for identifying data sources for retrieving the objects referenced by the web-page as it loads in a web-browser;

means for causing test client running in the remote network on a device without any resources to render the web-page to:

retrieve the data components required to load the web-page in the web-browser, wherein the test client monitors timing information as it retrieves the data components from the identified data sources; and

provide the timing information measured by the test client as it retrieves each data component required to the off-load server; and

means for storing the provided timing information in association with each data component used by the web-page or an object called by the web-page.

20. A computer-software product which when executed on a computer-system implements a method for remotely assessing a loading performance of a web-page on a web-browser running at a remote location, comprising:

determining one or more network conditions which would be experienced by the remote web-browser when loading the web-page at the remote location;

simulating the remote web-browser loading the web-page by replicating the network conditions when loading the web-page on a test web-browser; and

determining at least one perceptual metric associated with a visual presentation performance parameter for the web-page as it is loaded by the test web-browser.