BR112017013811B1 - DIGITAL CONTENT VIRTUAL BROADCAST METHOD AND SERVER - Google Patents

Abstract

METHOD AND SYSTEM OF VIRTUAL BROADCAST OF DIGITAL CONTENT. The virtual broadcast system ('virtual broadcast system') according to the present invention optimizes the routing of digital content between nodes over overlapping networks that are dynamically reconfigured based on predictions of frequently changing congestion levels of network interconnections. components within an underlying network. In the context of providing Internet video streaming to large numbers of simultaneous users, the present invention makes efficient use of the limited capacity of congested ASN peer points by employing deep learning methods to predict congestion levels between such peer points. of ASN and, based on these predictions, optimize the routing of video content over dynamically reconfigured overlay networks. The virtual broadcast system handles scheduled and unscheduled events, live streaming and pre-recorded events and streams these events in real time with minimal delay, in a highly scalable manner that maintains consistent QoE across large numbers of simultaneous observers.

Description

Field of Technique

[001] The present invention relates, generally, to an overlay network architecture for providing digital content between nodes of an underlying network and, more specifically, to a virtual broadcast system (“virtual broadcast system”) that optimizes the routing of digital content between nodes along overlay networks that are dynamically reconfigured based on predictions of frequently changing congestion levels at component interconnections within the underlying network.

Description of the Technique Field

[002] Network congestion:

[003] As wired and wireless network traffic continues to expand exponentially, the finite capacity of shared links or interconnections between components of a network is becoming an increasingly relevant and disruptive problem. Furthermore, as the level of congestion on these shared links is dynamic and subject to much volatility as network traffic ebbs and flows, measuring this congestion is difficult at any given time and is particularly difficult to predict, even in the short term. term.

[004] This problem is somewhat analogous to that of traffic congestion at shared highway and motorway intersection junctions in increasingly populated areas. Although existing GPS navigation and traffic control systems measure current congestion at these junctions and calculate optimal routes to reroute individual drivers around that congestion, their ability to predict the optimal route in advance for each specific driver is impeded by the volatile nature of the traffic congestion.

[005] When, alone, a company like Netflix represents more than a third of peak Internet traffic, companies that simultaneously provide digital information via the Internet (particularly large amounts of linear data) need to somehow address the nature increasingly volatile Internet congestion. Similarly, as mobile voice and data usage soars, the limited availability of regulated RF spectrum is of particular concern to companies developing high-bandwidth mobile applications.

[006] Although a specific application of the present invention is described herein in the context of providing streaming video via the Internet to large numbers of simultaneous users, the principles of the present invention apply equally to numerous other contexts, in which limited capacity of shared links between network components restricts the routing of any type of information that can be converted into digital format (e.g. audio, images, 3D models, etc.). Other potential applications of the present invention include, for example, VoIP, corporate video conferencing, virtual reality, multiplayer gaming, and a number of other bandwidth-intensive applications (with respect to the level of congestion of shared links in a network underlying a anytime).

[007] As will be discussed in more detail below, the present invention does not “cure” the problem of limited capacity or “network congestion” on component links in an underlying network such as the Internet, but rather makes efficient use of that limited capacity by monitoring and analyzing network traffic across these links to optimize the routing of digital content between nodes in overlay networks that are dynamically reconfigured based on predictions of congestion levels on these links.

[008] Video streaming events:

[009] Since the advent of the Internet and IP-based routing, many approaches to streaming video via the Internet have emerged. Before discussing their relative advantages and disadvantages, it is useful to look back and consider the problem being addressed. To distribute video content via the Internet, it first needs to be captured and digitized. Video content can be characterized as an “event” that is captured “live” (or digitally generated) and distributed via the Internet. References herein to video events include the capture or generation of video and audio, as well as any associated metadata.

[0010] Video events can be scheduled or unscheduled. The “Super Bowl”, for example, is a scheduled event, so that the moment of its occurrence is known in advance, while other events (for example, natural disasters, a baby's first steps or even video on demand - “VOD ”) are unscheduled and may occur with little or no notice.

[0011] Video content can be fully captured to generate a digitized video file before being distributed via the Internet as any other type of file is transferred (for example, via “FTP”, or file transfer protocol). This “file transfer” approach, however, imposes a delay on the receiver watching (playing) the video content - that is, the receiver needs to wait until the entire file has been transferred before watching the video content. Considering the relatively large sizes of digitized video files, this delay can be significant.

[0012] Video content is therefore often streamed to users so that they can continuously receive and view the content while it is still being sent. Essentially, video content is divided into an ordered linear stream of small files or “video segments” (e.g., one to ten seconds long), which are served to users who can begin watching them as they are being played. received. To watch a continuous stream of video content without lag or jitter, each video segment needs to play at regular intervals - for example, thirty frames per second. Note, however, that video segments do not need to be received at regular intervals, as long as each segment is received before the completion of execution of the previous segment.

[0013] Whether the event is scheduled or not, it can be streamed “live” (i.e., at the same time as the event occurs) or “pre-recorded” for streaming at any time after the event occurs. event. The Super Bowl, for example, could be captured and broadcast live as it happens or pre-recorded for later broadcast.

[0014] Finally, whether the event is scheduled or unscheduled and whether it is pre-recorded or broadcast live as it occurs, it can be streamed in “real time” (i.e., with a delay that is often imperceptible from the sender to the recipient) or “delayed” with a transit time of seconds or even minutes. Viewers, for example, of a television program (e.g. a baseball game) that is broadcast via the Internet but not in real time, may watch the streamed event at different times from each other or at different times from viewers who watch the same program broadcast on cable or satellite. These delays (particularly if they are more than a few seconds) can reduce the user's “quality of experience” (QoE) – that is, the application-level or user-centric view of quality, as opposed to “quality of service” (QoS), which is a measure of performance based on network-centric measures (e.g., packet delay, packet loss, or jitter caused by routers or other network resources).

[0015] Social interaction between viewers, for example, may be restricted (completely independent of jitter or other video artifacts) because viewers watch the same event at different times. This is particularly problematic today, when so many events (scheduled and unscheduled) are communicated in real time in so many different ways - from broadcast radio or television to social networking and other Internet services, accessible through cell phones and desktop computers and laptops, as well as an ever-evolving array of consumer electronic devices.

[0016] It is therefore desirable for a video streaming system to handle scheduled and unscheduled events, transmit live and pre-recorded events, and stream these events in real-time with minimal delay in order to provide consistent QoE to spectators. Furthermore, as the number of concurrent viewers of a streaming video event increases, maintaining consistent QoE becomes a formidable problem. For this reason, scaling is a fundamental design objective of any system of this type.

[0017] Despite recent advances in video streaming technology, the historical ad hoc evolution of the Internet infrastructure still presents significant obstacles to Internet-based video streaming and one of them is inconsistent QoS, which generates network congestion at times and places across the Internet that are difficult to predict. Although a fundamental objective of the present invention is to maintain consistent QoE for viewers of video streaming events, this objective is constrained by network congestion throughout the Internet, which ultimately cannot be eliminated.

[0018] Underlying Internet Architecture:

[0019] Since ARPANET (the first packet-switching network to implement the Internet Protocol suite, or TCP/IP) and later NSFNET, the “structure” of the Internet has been designed as a “network of networks.” (i.e., the Internet) that provided reliability or “resilience” by decentralizing control and providing alternative communication paths (routing) for information to reach its desired destination. Still, with packets taking different paths between routers and other shared network resources, maintaining consistent QoS or QoE across the Internet remains an extremely difficult problem.

[0020] As the structure of the Internet evolved and was privatized, redundancy and overlap developed between the networks of the traditional structure and those owned by long-distance telephone providers. For the purposes of this descriptive report, a distinction is made between large “public” networks, which provide data to consumers, directly or through other smaller “Internet service providers” (ISP) networks, and large “internet service providers” (ISP) networks. private companies” that conduct data only between themselves or serve as conduits between large ISPs, but do not provide data directly to consumers. In either case, these large public and private networks are typically implemented as “fiber rings” interconnected with fiber optic trunk lines – that is, multiple fiber optic cables bundled together to increase network capacity.

[0021] For routing purposes, the larger network providers that carry the heaviest network traffic (e.g., large ISPs and private backbone networks) are assigned blocks of IP routing prefixes known as “autonomous systems” (AS). , each of which is assigned an “autonomous system number” (ASN). Each of these companies' large fiber rings is called an ASN. The number of ASNs has grown dramatically in recent years, from around 5000 ASNs fifteen years ago to more than 50,000 ASNs worldwide today. As indicated above, many large network providers also have structural fiber ring networks (i.e., private ASNs) that do not serve customers, but can be connected to their own “public ASNs” or those owned by third parties.

[0022] As different companies have ASNs, they establish agreements with each other to facilitate the routing of Internet traffic through the ASNs and throughout the global Internet. Each ASN uses a bank of routers often called a “peer point” to control access to another ASN, employing a routing protocol known as a “border gateway protocol,” or BGP. Any given ASN can employ several peer points to connect to several different ASNs. Interconnected ASNs can be geographically adjacent or they can be distant, connected via long fiber trunks that span large distances (e.g., across countries or even oceans). Public ASNs can also be interconnected through “private ASNs” or structural networks.

[0023] Monitoring QoS in and across ASNs is extremely difficult. Large network providers keep much of the routing and performance information within their ASNs (including dynamic congestion measures) confidential. Although the “Open Message Format” (from current BPG, version 4) provides a “data store” for certain information when a TCP/IP connection is established to a BGP router, this mechanism is not extremely useful in practice. Many BGP routers do not support Open Message Format, while others simply turn it off. Additionally, information is typically five minutes late, which is a relatively long time considering how frequently congestion levels change on the Internet.

[0024] Because this large amount of Internet traffic flows through relatively high-bandwidth peer points that interconnect ASNs, these peer points are often key “bottlenecks” or sources of much of the congestion on the Internet at any given time. , in addition to the “last mile” problem in an ASN (i.e., congestion across wired and wireless connections with relatively lower bandwidth between end users and their “gateway” ISPs).

[0025] As the traffic load increases through an ASN peer point, for example, the routers in the ASNs on each side of the peer point become congested. In other words, these routers experience high rates of RAM usage, CPU and other shared resources with limited capacity. Increased demand on these resources reduces performance (i.e. bit speeds) across these peer points and can eventually lead to data packet loss. Because network traffic across the Internet has no central control, it is difficult to predict frequently changing levels of “peer congestion” across the Internet at any given time.

[0026] If consistent QoS cannot be guaranteed in and across ASNs, it becomes very difficult to maintain consistent QoE for viewers of video streaming events. Any system that transmits video over the Internet is subject to unreliability and constantly changing congestion levels from shared routers, particularly at ASN peer points through which so much Internet traffic flows. This problem is exacerbated when streaming video to large numbers of simultaneous viewers on the Internet, particularly through these ASN peer points.

[0027] Existing video streaming approaches:

[0028] Various approaches to streaming video over the Internet have evolved over the past few decades, with a wide variety of terminology being employed to characterize and differentiate different methods of generating overlay network topologies (over the Internet) and delivering video content between nodes across these overlapping networks. When comparing different approaches, it is useful to quickly return to the GPS navigation analogy and consider the factors that affect the time required to travel between any two points or nodes - that is, distance, speed and congestion (typically addressed by redirection to the along a different route).

[0029] In the context of routing packets via the Internet, distance (or geographic proximity) is not directly relevant, as packets travel close to the speed of light. Speed, however, is affected by the number of stops or blockages encountered along a route or, in this context, the number of “hops” found in intermediate routers between two nodes. In this way, it can be said that two nodes are “close” to each other (in “network proximity”) if they are only relatively few hops apart, regardless of their geographic proximity. Congestion at intermediate nodes along the path between two nodes affects overall travel time and can be addressed by dynamic redirection of traffic - that is, dynamic reconfiguration of the overlapping networks that determine the path between two nodes. As will be discussed below, these factors serve to illustrate key distinctions between different approaches to streaming video via the Internet.

[0030] The most common method of delivering video outside of the Internet is to “broadcast” a video stream (e.g., a television program) from a “point of origin” to all destination viewers simultaneously - e.g. through dedicated cable or satellite infrastructure. Although network centers can be employed in a LAN to transmit information to all nodes on the network, broadcasting packets of video segments through switches and routers via the Internet would be largely impractical and inefficient. Most network users would not be interested in seeing any data “channels” of video content, and significant congestion would occur near the point of origin, so routers transmitting video segments to other routers would quickly become congested. A broadcasting solution is simply not viable for delivering a channel of video content via the Internet from a single point of origin to a large number of simultaneous viewers who can watch the channel at any time.

[0031] An alternative “multicast” approach involves simultaneously streaming each video segment from a point of origin to previously defined groups of nodes across the Internet. Similarly, this approach is impractical for large-scale video distribution via the Internet. Furthermore, it requires specialized infrastructure, such as dedicated routers with multicast functionality, which is also impractical and prohibitively expensive for large-scale commercial use.

[0032] Unlike broadcast and multicast methods, a “unicast” approach to video streaming involves sending video segments from a source point to a single destination node (e.g., establishing TCP/IP connection with a defined destination node IP address). Providing a large number of unicast packets simultaneously to each receiving node, however, would also quickly congest routers at or near the point of origin and fail to achieve consistent QoS for many of the reasons noted above, not to mention the enormous cost of providing sufficient bandwidth to handle this large number of simultaneous transmissions.

[0033] Some VOD companies (such as Netflix and YouTube) have employed variations of this unicast approach that generally rely on expensive “terminal server” infrastructure. This approach (sometimes called a “content delivery network” or CDN) involves deploying many physical servers across the Internet and distributing copies of each video channel's content to each server. As a result, observation nodes can receive desired video content from a nearby server (in network proximity - just relatively few hops away from an observation node).

[0034] Each terminal server typically has significant computing capabilities and bandwidth and essentially constitutes a separate video content source from which nearby observing nodes can obtain any channel with video content at any time (“on demand”). . This approach of adding physical infrastructure is somewhat similar to building additional freeways and exit ramps to allow more people to get to popular destinations more quickly (with fewer turns and less time spent on slower highways). slow).

[0035] Although different users typically want to watch different video channels at any given time, VOD systems occasionally face periods of “peak” demand, during which a specific video event must be streamed to a large number of users. simultaneous viewers (e.g., a final episode of a popular television series), which can congest even the largest video streaming company's infrastructure, or at least result in inefficient “worst-case” deployment of infrastructure expensive that is often underutilized (i.e. during more common off-peak demand periods). Alternative VOD solutions have been tested to avoid the need for expensive terminal server infrastructure by playing and distributing video content between network nodes themselves (as discussed, for example, in published U.S. Patent No. 2008 /0059631).

[0036] With or without expensive terminal server infrastructure, none of these VOD solutions address the QoS problem for unscheduled video events, as they all rely on “pre-seeding” terminal servers or observation nodes throughout the Internet with content known in advance, to ensure a close source of video content. Live unscheduled event streaming would require simultaneous real-time delivery of video content to all of these terminal servers or observation nodes, an issue not addressed by any of these VOD systems.

[0037] More recently, certain unicast-based video streaming standards (e.g., “WebRTC”) have evolved to enable “peer-to-peer” streaming of video between mobile and desktop Web browsers without the need for plugins. Many existing smartphones, as well as desktop and laptop computers, include WebRTC libraries that support cross-browser video streaming, as well as “adaptive streaming” libraries that allow a peering node to sense its bandwidth and CPU capacity in real time and automatically request higher or lower “bitrate” to adapt to changes in these measurements.

[0038] Adaptive streaming implementations include Apple's “HTTP Live Streaming” (HLS), Microsoft's “Smooth Streaming”, and the “MPEG-Dash” ISO standard, among others. In a typical peer-to-peer video streaming scenario, a receiving node periodically requests an HTTP server for “manifest files,” which include the locations of each available bitrate version of subsequent video segments (e.g., the next eight). Each video segment may, for example, be available in 1080p, 720p and 480p versions, reflecting different “video resolutions” that require different streaming bit rates (bandwidth) to ensure that each video segment is delivered. in essentially the same period of time, regardless of its resolution.

[0039] Standard HTML5 video devices (in Web browsers that support WebRTC) typically buffer three video segments before they begin playing video content. They use the current manifest file to send HTTP request to an HTTP server for each video segment. The sending node then “pushes” each video segment (in small “sets”) to the receiving node according to WebRTC standards for execution in the receiving node's web browser. If the receiving node supports adaptive streaming implementations and determines that the time required to receive recent video segments increases or decreases significantly, it automatically begins requesting lower or higher bitrate video segments from the selections in the manifest file . In other words, it “adapts” to your actual bandwidth over time by varying the resolution of the video segments it requests.

[0040] The “resolution” of a video frame is the measurement of its width x length (e.g., 1920 x 1080 or 1080p) or number of pixels in a frame, while its “bit rate” designates the number of “bits per second” (bps) that are transmitted from the sender to the receiver. If 30 frames of a 1080p video are delivered per second (30 “frames per second”) and each color pixel contains 24 bits (24 “bits per pixel”), the bit rate would be equal to almost 1.5 Tbps (1,492,992,000 bps, i.e. 1,492,992,000 = (1920 x 1080 pixels per frame) x (24 bits per pixel) x (30 frames per second).

[0041] Standard video codecs employ compression (e.g., MPEG2 compression) and other video encoding methods to generate lower effective bit rates (e.g., 3 Mbps). In view of the above, “bitrate” and “resolution” are highly correlated as they can increase or reduce the effective bitrate by providing higher or lower resolution video frames. We therefore use these terms somewhat interchangeably at present.

[0042] WebRTC and adaptive streaming standards allow virtually any smartphone user to capture and stream live video events and also allow those users to link to a video streaming channel originating from another point of origin on the Internet, which It ranges from other individual smartphone users to large companies that host a body of video content. These standards are, however, designed for peer-to-peer video streaming and do not address the “video delivery” problem of streaming a video channel to large numbers of simultaneous viewers via the Internet.

[0043] To address this problem, some video streaming companies (e.g., StreamRoot) have adopted an approach that typically involves “peer to peer” (P2P) or interlaced network topology, in which video content is relayed from one node observation to another (sometimes called “peercasting”). In a video streaming context, these terms can be used interchangeably to designate overlay networks configured on the Internet that allow observation nodes to relay streaming video content to each other in a distributed manner. The provision of video streaming to large numbers of simultaneous viewers should be distinguished, however, from uses other than P2P or interlaced network topology streaming, for example, for file transfer or file sharing applications.

[0044] P2P video streaming systems provide a channel of video content from a single point of origin to large numbers of simultaneous users via the Internet. These systems tend to be resilient and scalable, as their distributed nature makes it possible to recover from individual points of failure (e.g., by redirecting content through other nodes) and their reliability and performance actually improves as more nodes are added to the network (i.e. , as more and better routing relay options become available).

[0045] When new nodes connect to a video channel or existing nodes leave (stop watching) the channel, P2P video streaming systems need, to some extent, to dynamically reconfigure the overlay network topology - i.e. modify at least some of the routing paths between network nodes to accommodate the new nodes. When adding a new node, for example, its geographic location may be considered in an effort to select existing nodes nearby to receive (and retransmit) video content.

[0046] But, if “peer” nodes are selected merely based on their geographic proximity, they may still be relatively “distant” from each other (and not in network proximity) - for example, if they are on different ASNs. As a result, traffic between them may cross one or more potentially congested peer points. The actual latency between two nodes in good geographic proximity may, for example, exceed the sum of the latencies between each of these nodes and a geographically distant node. This phenomenon is sometimes called “triangle inequality violation” (TIV), which illustrates the disadvantages of relying on BGP routing to deliver digital content between nodes in an overlay network through ASN peer points.

[0047] One reason for this problem with existing P2P video streaming systems is the fact that they are not built to be “compatible” with the underlying architecture of the Internet. Any overlay network topology built on top of the Internet is still subject to many points of disruption or failure (in addition to new or disappearing nodes), such as the series of QoS problems noted above. By not addressing the underlying QoS volatility of the Internet, particularly at ASN peer points, these systems face significant obstacles in providing their users with consistent QoE.

[0048] Existing P2P video streaming systems (such as GPS navigation systems) therefore rely on geographic proximity (rather than network proximity) to select peer relay nodes and redirect traffic only “after the fact ”, when encountering congestion. Furthermore, real-time streaming of linear data to simultaneous users imposes additional constraints not found in GPS navigation systems - content needs to arrive “simultaneously” at each node. Terminal servers and other physical infrastructure approaches (similar to building freeways and exit ramps to provide higher speed lanes) are expensive and also fail to adequately address the problems of unscheduled events and high-concurrency usage of any specific event.

[0049] There is therefore a need for a digital content delivery system that addresses the deficiencies discussed above and takes into account the underlying architecture of the Internet (particularly at ASN peer points through which much Internet traffic flows) to Dynamically generate and reconfigure overlay networks to provide client nodes with consistent QoE.

Brief Description of the Invention

[0050] According to the present invention, various embodiments of innovative architectures and methods are described with respect to a digital content delivery system that provides users of nodes in an underlying network (e.g., the Internet) with consistent QoE through: (1) maintaining a map of shared links that interconnect components of the underlying network (e.g., ASNs and the peer points that interconnect them), including a location of each node in one of the components (e.g., within an ASN); (2) generating measurements by monitoring network traffic between nodes that cross these shared links (ASN peer points) along one or more overlay networks built on top of the underlying network (Internet); (3) analysis of measurements and map over time to predict congestion levels that reflect changing capacity of shared links (ASN peer points) over time; and (4) dynamically reconfiguring the topology of the overlay networks, based on predicted congestion levels, to generate optimal digital content pathways between client nodes across the overlay networks.

[0051] Specific embodiments of the “virtual broadcast” system in accordance with the present invention are described herein in the context of providing consistent QoE among viewers of one or more channels of video content, each of which is streamed. in real time from a single point of origin to potentially large numbers of simultaneous observers who may access the channel at different times (while avoiding the need for terminal servers or other expensive physical infrastructure). As will be explained in more detail below, we use the term “virtual broadcast” in the context of unicast streaming of linear content to simultaneous users. From the users' point of view, content is transmitted to them so that they receive the content “simultaneously”, even though unicast streaming is used to direct the content. Other embodiments of the present invention will be apparent to those skilled in the art in numerous other contexts in which the limited capacity of shared links between network components restricts the routing of any type of information that can be converted into digital form.

[0052] For the purposes of this specification, a single node that receives several different channels simultaneously can be considered a distinct node on separate overlay networks, each defined by a single channel. In a VOD context, each separate “airing” of a specific program can be considered a separate channel that has its own network of watch nodes.

[0053] The system according to the present invention is capable of handling scheduled and unscheduled events, streaming live as well as pre-recorded events and streaming these events in real time with minimal delay in a highly scalable manner that maintains QoE consistent across large numbers of concurrent users - despite being implemented in overlay networks built on top of the Internet that are subject to Internet QoS volatility. System performance and user QoE actually increase as the number of concurrent users increases (particularly on any given ASN).

[0054] Client and server architecture is employed to centralize server-side routing decisions. Distributed streaming delivery of video content is accomplished through dynamically reconfigurable P2P overlay networks that allow video content to be relayed (i.e., “pushed”) between the watch nodes of each video channel. Client nodes may employ standard HTML5 video devices (built into most desktop and mobile web browsers) to watch or play the video and rely on specific embedded code (such as Javascript) to implement additional functionality, e.g. receiving video segments and retransmitting those video segments to other nodes, as well as monitoring various performance measures. In other embodiments, this functionality, in whole or in part, may be integrated into a specific mobile or application.

[0055] The system according to the present invention enables “peer-to-peer” video streaming between mobile and desktop web browsers and adapts to changes in node bandwidth by automatically requesting lower bitrate video segments or higher. In one embodiment, the virtual broadcast system employs unicast standards, including WebRTC and adaptive streaming standards (such as HLS, MPEG-Dash, or Smooth Streaming) to enable video streaming without the need for Web browser plugins and allow users to we detect your performance capabilities, such as CPU capacity and bandwidth.

[0056] Each event is provided to a “Virtual Broadcast Server” for providing the “point of origin” of each channel to simultaneous users over several dynamically reconfigurable overlay networks. Events can be obtained from virtually any source (including CDN), whether they are transferred as complete files or streamed live to the Virtual Broadcast Server. In deployments that utilize WebRTC, any user with a smartphone that implements WebRTC can load pre-recorded video events or capture live events and upload them to the Virtual Broadcast Server (as well as watch other channels streamed by the Virtual Broadcast Server ) for onward delivery to users via the overlay networks.

[0057] The Virtual Broadcast Server includes, in one embodiment, a "POI Content Server", which serves as the point of origin for each channel from which video segments are delivered through the dynamically reconfigurable overlay networks built over the Internet. Video segments are typically fixed in length (for example, one to ten seconds), as determined by the original publisher of the video event. Video segments are observed by client nodes and “pushed” (i.e., retransmitted in the form of individual “chunks” with a fixed size according to the WebRTC standard) from one node to another along paths defined by the overlay networks. In one embodiment, each video segment is divided into 64 kB chunks to match the size of a UDP datagram “packet” for maximum efficiency when transmitted via the MPEG2 transport protocol.

[0058] Although video segments are effectively pushed to each client node in most cases, a client node may, in one embodiment, detect that all chunks of a video segment have arrived late and may utilize the current manifest file to request the video segment from the POI Content Server (i.e., as a “fallback” feed location).

[0059] As each node seeks to connect to a channel made available by the POI Content Server, the node determines (with assistance from the Virtual Broadcast Server, in another embodiment) the specific ASN in which that node resides. The Virtual Broadcast Server uses this “ASN location” information in conjunction with an “ASN Interconnection Map” of the Internet (including ASNs and their various peer interconnections) and various monitored performance measures to optimize routing of channel content between overlapping networks that are dynamically reconfigured based on predictions of congestion levels at these ASN peer points. In another embodiment, the Virtual Broadcast Server also uses the geographic location of each node, in addition to its ASN location, to assist in this process.

[0060] In one embodiment, the topology of the overlay networks defines the routing paths of video segments between observation nodes and is dynamically reconfigured (in whole or in part) for each video segment of a channel. In another embodiment, they are dynamically reconfigured (in whole or in part) for each piece of video segment. In this way, the Internet architecture (as well as predicted congestion levels at ASN peer points) is taken into consideration to determine optimal routing paths for each video segment of a video channel. In another embodiment, if some or all of the paths along the overlay networks are capable of delivering video segments in time (even if those paths are not ideal), those paths are not reconfigured until a predefined congestion threshold is reached. or until another sufficiently significant problem is identified.

[0061] In one embodiment, the client nodes monitor performance issues relating to, for example, arrival issues and Internet QoS issues (including congestion at ASN peer points), as well as congestion resulting from the number of concurrent viewers of one or more channels of the virtual broadcast system itself. They monitor the time required to contact designated locations on the Internet and in ASNs, as well as the time required to relay video segments to other nodes. Measurements monitored by clients are communicated to the Virtual Broadcast Server for use in making dynamic routing decisions. In one embodiment, the Virtual Broadcast Server includes a “Signaling Server” for communicating with client nodes via standard WebSocket protocols.

[0062] Client nodes optionally include an “Uploader” that allows users to capture a video event and upload it to the Virtual Broadcast Server in real time. As the path from any client node to the Virtual Broadcast Server may cross several ASNs, a standard “shower” protocol is employed to enable streaming of the video event and avoid delay or blocking of packets at intermediate routers. In one embodiment, client nodes may also search and observe “trending” events (hereinafter referred to as “splashes”) through a “Splash Extractor” search engine on the Virtual Broadcast Server that identifies splashes and, based on user surveys, provides users with the ability to stream and observe trending events from across the Internet that are not otherwise available through the POI Content Server.

[0063] Upon request to join a channel, nodes are ranked by the Virtual Broadcast Server based on their retransmission capabilities (i.e., their reliable “upstream” bandwidth, which is deduced from several factors, including your connection type (e.g. 3G or 4G cellular, WiFi, LAN, etc.), as well as your CPU, operating system, browser and memory settings, as well as other fixed and variable performance measures monitored over time. In one embodiment, nodes are classified into three levels based on their corresponding relay capability. The lowest-level nodes (“C” nodes) can observe video segments, but cannot relay them to other nodes. Intermediate-level nodes (“B” nodes) can observe and relay video segments within an ASN. Higher-level nodes (“A” nodes) can observe and relay video segments to other A nodes, both within and across ASNs. .

[0064] In another embodiment, node classifications may be dynamically changed, for example, based on monitored performance measures and current system needs for more or fewer relay nodes of a given classification. Additionally, if there are enough A nodes in an ASN to retransmit video segments, node A may be referred to as node “B:A”, which indicates that it will be treated as node B, but may be elevated to node A if necessary. (e.g. if existing A nodes leave the channel). In one embodiment, if an individual node exhibits a significant change in performance (up or down), the node may be reclassified (e.g., from node B to node C or vice versa), and if and when the problem is resolved , be restored to its initial rank.

[0065] In another embodiment, client nodes are assigned various "slots" (based, for example, on their capabilities and client performance measures) to allow them to receive and retransmit chunks of video segments to and from several other nodes. In this embodiment, client nodes receive a video segment from only one “feed” node, but can “feed” or retransmit that video segment to several other client nodes. Nodes are allocated up to eight relay slots, four for relaying to nodes A in the same ASN and four for relaying to nodes A in other ASNs - that is, through an ASN peer. Nodes B:A and B are allocated up to eight slots for relaying to other customer nodes (i.e., other nodes B:A, B, and C) within their ASN. In another embodiment, a client node may be “powered” by several other client nodes (e.g., by alternating chunks across several input slots). This technique can be employed for high bitrate video streaming (e.g. 4K) where higher performance is required.

[0066] In another embodiment, certain nodes (based, for example, on their capabilities and client performance measures) may receive multiple resolutions of a video segment from a single feeder node (or, in an alternative embodiment, receive different resolutions from different power nodes). If the bandwidth of these nodes upstream is sufficient, they may be considered “polycast” nodes and may, as necessary, also retransmit or feed these multiple resolutions of a video segment to one or more designated nodes.

[0067] To facilitate dynamic reconfiguration of relay networks, the Virtual Broadcast Server employs a “Deep Mapper” deep learning engine that continuously analyzes performance measurements to predict the level of congestion along ASN peer points - that is, predicting the congestion level of an ASN peer point at a short time (e.g., one minute) in the future. In one embodiment, a predicted “congestion value” is generated for each potential inter-ASN path between nodes A - for example, from a node A in one ASN to a node A in another ASN. In another embodiment, the congestion value reflects the predicted level of congestion for the ideal route between each pair of nodes A.

[0068] In one embodiment, the Virtual Broadcast Server employs an “Overlay Network Creator” to dynamically generate and reconfigure (in whole or in part) inter-ASN and intra-ASN overlay networks - determining, for example, optimal path for segments of video to be transmitted from one node to another within and across ASNs. In this realization, the Overlay Network Creator considers the number of available slots that each node can use, as well as the number of resolutions that each node can receive or retransmit.

[0069] The Overlay Network Builder dynamically generates and reconfigures (with assistance from the Deep Mapper) a “Virtual Data Trunk” overlay network between ASNs, which represents the topology of A nodes. In other words, it represents A nodes and links or routing paths that the video segment will follow between nodes A in and (particularly) through ASNs - that is, through potentially congested ASN peer points.

[0070] The Virtual Data Trunk identifies the set of nodes A that will be instructed to request each video segment from the nearest POI Content Server (using, for example, the current manifest file), as well as the set of nodes A to which each of them will transmit that video segment and so on (both within and across the ASNs). As a result, that video segment will be spread across every ASN that contains an observation node. To reach each observation node, the segment can also travel through private structural ASNs without observation nodes.

[0071] The Overlay Network Creator also generates one or more intra-ASN “crowd” overlay networks to relay a video segment from nodes A in an ASN to nodes B:A, B, and C in that ASN. These crowd-overlay networks can be dynamically reconfigured (in whole or in part) for each video segment (or for each piece of video segment in an alternative embodiment). In one embodiment, each crowd overlay network in an ASN represents a hierarchical topology (with respect to a node A within the ASN) of nodes B:A, B, and C that receive, observe, and relay (with the exception of nodes C) the video segment between the nodes of that crowd hierarchy.

[0072] The virtual broadcast methods and system in accordance with the present invention make efficient use of limited capacity at ASN peer points and other major points of congestion by monitoring and analyzing network traffic to optimize network routing. digital content between nodes of Crowd Overlay and Virtual Data Trunk networks that are dynamically reconfigured based on predictions of congestion levels at these key congestion points, in order to maintain consistent QoE between system users.

Brief Description of Figures

[0073] Fig. 1 is a graph illustrating an embodiment of overlay networks in accordance with the present invention dynamically configured over the Internet.

[0074] Fig. 2 is a block diagram illustrating an embodiment of important client-side components of a client video streaming device in accordance with the present invention.

[0075] Fig. 3 is a block diagram illustrating an embodiment of important server-side components of a virtual broadcast server in accordance with the present invention.

[0076] Fig. 4 is a flowchart illustrating an embodiment of a dynamic video streaming process in accordance with the present invention.

Detailed Description of the Invention

[0077] Detailed realizations of the systems and methods according to the present invention are illustrated in the attached Figures and described below. It should be noted at the outset that the present invention is not limited to the specific embodiments discussed below with reference to the Figures.

[0078] As indicated above, although a specific application of the present invention is described in the context of providing streaming video via the Internet to large numbers of simultaneous users, the principles of the present invention apply equally in numerous other contexts in which the ability Limited number of shared links between network components restricts the routing of any type of digital content.

[0079] Even within the context of providing video streaming via the Internet, the allocation of functionality between client nodes and server components described herein is the result of design tradeoffs and much of this functionality may be reallocated between client-side components and server-side without abandoning the spirit of the present invention. Similarly, client-side functionality may be allocated to a single modular component or spread across several different components and may be implemented as one or more standalone applications or mobile applications or as a combination of standalone applications or applications and Javascript or other languages. programming or scripting. Furthermore, server-side components can be implemented on a single hardware server or on several different servers. This functionality may also be integrated into a single software module or allocated to different software modules spread across one or more hardware servers.

[0080] Finally, in embodiments where standard protocols and libraries are used (e.g., HTTP, WebSocket, WebRTC, STUN, and various adaptive streaming standards), the functionality provided by some or all of these standard protocols and libraries may be replaced by other standard or particular embodiments, without abandoning the spirit of the present invention.

[0081] Overlapping networks:

[0082] Fig. 1 is a graph illustrating an embodiment of Overlay Networks 100 in accordance with the present invention mapped over the Internet. Although the Internet itself can be illustrated in a number of different forms, Fig. 1 illustrates the Internet as a set of rings of ASN fibers 110, interconnected through Peer Points 120. The individual customer nodes that watching a specific video channel at any time are illustrated within each ASN 110. Although not noted in Fig. 1, several channels and therefore several sets of Overlay Networks 100 may (in one embodiment) be simultaneously active.

[0083] As noted above, a Virtual Data Trunk overlay network represents interconnections 175 between A nodes 130, both within an ASN 110 (directly connected) and across ASNs 110 (i.e., via Peer Points 120). The main chain connector 195 illustrates the interconnection of A nodes between two ASNs 110, through a private ASN (not shown) that does not include any commercial nodes, but simply interconnects two public ASNs 110. The main chain connector 195 is shown , for example, connecting a node A 130 in ASN 110-f with a node A 130 in ASN 110-e. In this scenario, traffic between two A nodes 130 may travel over several “private” Peer Points 120 (or other proprietary connections with private ASNs).

[0084] As indicated above, in one embodiment, the performance of these connections may be monitored only at the endpoints (i.e., the two A nodes 130), as is the case with connections 175 between the A nodes 130 in two public ASNs different 110 (i.e. through Peer Point 120). Traffic along a connection 175 between two A nodes 130 on the same ASN 110 will likely be relatively faster than traffic along ASNs 110 because it does not traverse a potentially congested Peer Point 120. Although the bottom connector 195 and connections 175 to/from nodes 130 are illustrated as one-way arrows, they only reflect current unidirectional routing paths, despite the fact that bidirectional connectivity is sustained between all illustrated client nodes. in Fig. 1.

[0085] It should be noted that all traffic between any two client nodes in accordance with the present invention traverses the public Internet and therefore passes through a number of intermediate routers (not shown) that affect QoS. The system monitors the effects of QoS in an ASN 110 and across ASNs 110 (and therefore one or more Peer Points 120). In one embodiment, this inter-ASN and intra-ASN traffic is monitored by each client node (at the direction of the Virtual Broadcast Server) and provided to the Virtual Broadcast Server for dynamic reconfiguration of the nodes and routing paths represented by Overlay Networks 100 (including the Virtual Data Trunk overlay network between the A nodes 130 and the crowd overlay networks from each A node 130 in an ASN 110 to the B (and B:A) nodes 140 and C nodes 150 in that ASN 110).

[0086] Fig. 1 illustrates the various routing paths followed by a video segment between client nodes, considering a “current state” of these Overlay Networks 100. In other words, it illustrates a current topology of these Overlay Networks 100 that , in one embodiment, may be dynamically reconfigured for each video segment (and, in an alternative embodiment, for each chunk of a video segment). It should be noted that for any specific video segment, Overlay Networks 100 may or may not be reconfigured (in whole or in part), as this decision will depend, at least in part, on performance measures gathered over time. .

[0087] ASN 110-c illustrates a scenario in which the POI Content Server (not shown) resides on or near an ASN 110-c (e.g., across one or two different ASNs 110) and responds to a request of HTTP to provide the current video segment to node A 130-a in order to begin streaming video segments on a channel over the Overlay Networks 100. As will be discussed in more detail below, the POI Content Server typically will provide each video segment to a number of requesting A nodes 130 on or near the same ASN 110 and these A nodes 130 will, in turn, transmit the video segment to a number of other nodes along the Overlay Networks 100, resulting in “redistribution” of multiple simultaneous copies of chunks of video segments that are provided to and retransmitted to us from clients at any time.

[0088] In this scenario, node A 130-a retransmits the video segment to two other nodes A 130 - one on ASN 110-c and the other via a Peer Point 120 to ASN 110-a. As indicated above, the Virtual Data Trunk overlay network represents the routing paths that a segment will follow as it is relayed between and across ASNs 110 between A nodes 130. In this scenario, therefore, the video segment is relayed not only between several A nodes 130 in ASN 110-c, but also from ASN 110-a over several Peer Points 120 to several directly interconnected ASNs (i.e., 110- a, 110-d, 110-f and 110-g), of which it is further relayed over several hops of the Virtual Data Trunk overlay network to other ASNs 110.

[0089] As will be explained in more detail below, the number of A 130 nodes required in an ASN 110 will depend on several factors, such as the number of other customer observation nodes in that ASN 110, as well as their relative capabilities (as determined by its rating, number of open slots and performance measures monitored over time). ASNs 110-b, 110-f, 110-i, and 110-j, for example, are illustrated with just one node A 130 each, even though they have different numbers of other client nodes to feed (compare the other isolated node in ASN 110-f with the various other nodes in ASN 110-i).

[0090] While the bandwidth upstream of a node's monitored flow is a key factor in determining how many nodes it will feed directly (i.e., how many output slots it will utilize), it is important to recognize that the length of the “chain” of nodes in an ASN 110 (which retransmits a video segment from one to the next, and so on) is largely irrelevant considering how quickly these retransmissions are performed (typically well under 1 ms). Isolated node A in ASN 110-i, for example, which directly feeds two A nodes in external ASNs 110 (ASN 110-g and ASN 110-j), as well as two B nodes 130 in ASN 110-l, uses four output (which reflect relatively high bandwidth amplitude above in the monitored flow in this embodiment). Furthermore, the long chain of nodes B 140 and nodes C 150 that are fed indirectly from the isolated node A in ASN 110-i is not reflective of its bandwidth upstream.

[0091] In each ASN 110, one or more crowd overlay networks are generated (dynamically reconfigured for each video segment in this embodiment) to retransmit the video segment in that ASN 110 from each node A (i.e., the “root” node ” of a crowd overlap network) to the various B (and B:A) nodes 140 and C nodes 150 in that crowd overlap network. Although only one crowd overlap network is illustrated in ASN 110-c (compared to two crowd overlap networks illustrated in ASN 110-h), the number of crowd overlap networks generated in each ASN 110 (and internal topology of each network overlapping in crowd) will depend on several factors, such as the number of customer observation nodes on that ASN 110, as well as historical and current performance measures, number of open slots, etc.

[0092] As indicated above, a client node, such as node A 130-b in ASN 110-b, may receive a video segment from several other client nodes (in this case, from two other nodes A 130 at different ASNs (110-a and 110-d)). In one embodiment, these two other alternate feeder nodes send chunks of the video segment to node A 130-b for performance reasons - for example, because these chunks intersect Peer Points 120, whose congestion levels are continuously monitored. as will be explained in more detail. In other embodiments, this may be accomplished for redundancy reasons - for example, because the reliability of power nodes may be questionable based on historical performance measures (independently of or in addition to Peer Point 120 congestion.

[0093] The methods by which performance measures are monitored, video segments are retransmitted, and Overlay Networks 100 are dynamically reconfigured are explored in more detail below with respect to Fig. 4, following a discussion of important side components. client (Fig. 2) and server-side (Fig. 3) that implement these methods.

[0094] Client video streaming device:

[0095] Returning to Fig. 2, Client Device 200 illustrates an embodiment of important components of a video streaming device in accordance with the present invention. The Client Device 200 may be implemented in the form of a desktop or laptop computer, as well as a smartphone or other mobile device, or virtually any other consumer electronic device capable of handling streaming content, such as streaming video. Client Device 200 includes certain standard hardware and software computing components and related peripherals 210, including a CPU 212, memory 214, operating system 216, network adapter 217, display 218, and camera 219, which are well known in the art. Client Device 200 utilizes these components and peripherals 210, in conjunction with certain Standard Libraries 220, to become a network node and receive, display, and retransmit streaming video content between other network nodes of the virtual broadcast system. according to the present invention.

[0096] The present invention leverages certain Standard Libraries 220 (also found on most smartphones as well as many other computing devices) that implement network protocols and other functionality that can be employed to facilitate the streaming of video content between devices . Video content can, for example, be streamed between two smartphone users and displayed in their mobile web browsers without the need for plugins. The standard libraries 220 include WebRTC APIs 222 (which enable communication between browsers for streaming video content), various adaptive streaming implementations 224, such as HLS, MPEG-Dash and Smooth Streaming, among others (which allow tuning automatic streaming bitrates to “adapt” to real-time detection of changes in client bandwidth and CPU capacity), the WebSocket 226 protocol (which enables fast bidirectional client-server communications via a single TCP/IP connection) and HTTP 228 (for less frequent standard communications between web servers and client web browsers).

[0097] The client device 200 also includes a standard player 232 (in one embodiment, a standard video player integrated with a standard HTML5 web browser 230) for viewing or playing streaming digital content. In other embodiments, the standard player 232 is integrated into a standalone desktop application or smartphone application. An advantage of leveraging the standard HTML5 Web browser 230 is the fact that many of the standard libraries 220 are designed to work with Web browsers and therefore do not require plugins or other specific functionality that a desktop application or application would require. of independent smartphone.

[0098] Additionally, Web browsers also support client-side scripting languages, such as Javascript, which is often used to supplement the functionality of standard Web browsers (provided, for example, by a standard Web server as part of of a web page, without requiring any client browser plugin). In one embodiment, the important non-standard components of the client device 200 (which include the communicator 270, performance monitor 240, receiver 250, relay 260, and loader 280) are implemented in Javascript and the content sets 255 are generated and maintained by that code in Javascript. It should be noted, however, that some or all of these components may be implemented in other programming languages and in standalone desktop applications or smartphone applications without departing from the spirit of the present invention.

[0099] Standard libraries 220 facilitate generic point-to-point (unicast) streaming of content, including video content. The important non-standard components of the client device 200 address the client-side aspects of the digital content delivery architecture implemented by the virtual broadcast system in accordance with the present invention. In one embodiment, a streaming protocol is built on top of WebRTC 222 in which content routing is centralized via client-server architecture and the content itself is streamed in a distributed manner (pushed from one node to another). through dynamically reconfigurable P2P overlay networks.

[00100] A user of client device 200 may first encounter one or more content channels in several different ways - for example, through links in an email or web page or from a standalone desktop application or smartphone application. In one embodiment, the virtual broadcast server 300 (discussed in more detail below with respect to Fig. 3) provides a standard HTML5 web page with a selection of channels to the HTML5 web browser 230. This “channel web page” includes patented Javascript code that is interpreted by an HTML5 web browser 230 to implement the functionality of the non-standard components of the device client 200, which includes communicating with the signaling server 330 as well as other client nodes (using e.g. , WebRTC libraries 222 and adaptive streaming 224 ), as well as receiving, processing, and retransmitting chunks of video segments to and from these nodes.

[00101] By clicking on a channel link on the channel's web page, the user generates a link request to a specific channel of video content that is currently streaming or, in another embodiment, will begin streaming via streaming at a previously defined later time (a “call request”). The signaling server 330 of the virtual broadcast server 300 responds to the connection request by attempting to establish connection between the WebSocket 226 and the client device 200 via the communicator 270. As will be discussed in more detail below with respect to Fig. 3, the virtual broadcast server 300 employs the “STUN” protocol 322 to discover the public IP address of client device 200 (e.g., behind a NAT firewall), so that client device 200 can establish a WebSocket connection 226 with virtual broadcast server 300 and WebRTC connections 222 to other client devices 200 to receive and retransmit video content.

[00102] In the embodiments discussed herein, the client device 200 connects to only one video channel at any given time. In other embodiments, client device 200 may connect to multiple channels simultaneously without departing from the spirit of the present invention.

[00103] Client device 200 uses communicator 270 for bidirectional communications with signaling server 330 to facilitate rapid message exchanges while maintaining a single TCP/IP connection open. As will be discussed in more detail below, these communications are employed for a variety of purposes, including (i) providing the virtual broadcast server 300 with initial information regarding the capabilities of the client device 200 (e.g., operating system, web browser, and type connection - 3G, 4G, WiFi, LAN etc.); (ii) enabling virtual broadcast server 300 to verify connectivity of client nodes for inter-node streaming of subsequent WebRTC video segments 222 via overlay networks 100; and (iii) exchanging real-time dynamic monitoring information (obtained via performance monitor 240, as discussed below) with virtual broadcast server 300.

[00104] In one embodiment, this Javascript code contained in the channel web page also analyzes the capabilities of the client device 200 to determine whether it is a C node (which receives video segments, but does not retransmit them to other client nodes). and provides this information to the signaling server 330. In other embodiments, certain capabilities of the client device 200 are sent to the virtual broadcast server 300, which determines whether the client device 200 is a C node.

[00105] This Javascript code also enables communications with the POI Content Server 380 to manage the receipt of video segments by the receiver 250 for execution by the standard player 232. This process is, in effect, an extension of the point video streaming scenario to standard point, which leverages standard adaptive streaming 224 and WebRTC 222 functionality.

[00106] In one embodiment, the standard web browser 230 interprets the particular Javascript code of the channel web page to request manifest files periodically, as described above. These standard HTTP requests are directed to the POI 380 content server, which serves the manifest files. The standard web browser 230 also leverages the standard adaptive streaming libraries 224 to request its own video segments from the locations specified in the manifest file, including higher or lower bitrate versions of those video segments, as discussed above ( for example, when detecting a change in bandwidth).

[00107] These requests for video segments are intercepted by the particular Javascript code of the channel's Web page, i.e., because each video segment is pushed to the client device 200 from another node (feeder) of overlay networks 100 ( eliminating the need for the client device 200 to initiate an HTTP pull request). In one embodiment (discussed in more detail below), the virtual broadcast server 300 adds the client device 200 to the overlay networks 100 (and thus the channel) shortly after receiving the connection request, such that one or more Initial video segments are pushed to the client device 200 to allow it to begin playing the video content as soon as possible.

[00108] As the receiver 250 receives chunks of each video segment, it generates sets of content 255 to facilitate the receipt and execution of the video segments, as well as the retransmission of the video segments (if the client device 200 does not have a designated C node) to other client nodes. The receiver 250 generates a receiver set 256 to compile the chunks into a complete video segment, which is fed to the three-segment buffer maintained by the standard player 232. If, upon intercepting the HTTP request for a video segment, the receiver 250 determines that the complete video segment is not already in the receiver set 256, the video segment will be requested from an alternative location (or “fallback”) specified in the manifest file (i.e., POI content server 380). From the perspective of the standard player 232 , it receives video segments in response to standard HTTP requests and is unaware that the video segments are actually being pushed to the client device 200 via overlay networks 100 .

[00109] Additionally, in one embodiment, receiver 250 also leverages adaptive streaming libraries 224 to communicate to signaling server 330 (via communicator 270) the bit rate that client device 200 can handle (regardless of whether the default player 232 performs this request in the normal way via the manifest file). If the client device 200 experiences a significant temporary drop in its bandwidth (and as a result a video segment does not reach the receiver set 256 before it is needed), for example, it may request a video segment (fallback). to the POI content server 380 and then receive subsequent lower resolution video segments via overlay networks 100. When its bitrate returns to normal, it can then receive higher resolution video segments as before of the problem occurring.

[00110] As indicated above, in one embodiment, virtual broadcast server 300 dynamically reconfigures overlay networks 100 for each video segment, including virtual data trunk overlay networks (between nodes A in and across ASNs) and overlay networks in crowd (from each node A in an ASN to other nodes in that ASN). Unless the client device 200 is classified as a C node (which receives video segments but does not retransmit them to other client nodes), the relay 260 will receive instructions from the virtual broadcast server 300 (with respect to each video segment of the video channel, if connected) with respect to the node(s) to which it will retransmit that video segment. As discussed above with reference to Fig. 1, if the client device 200 is a node A, B:A or B, it may be requested to retransmit the video segment to several other client nodes.

[00111] The length of the video segments (e.g., one to ten seconds) is defined by the video content provider in accordance with adaptive streaming standards 224. Relay 260 will retransmit the video segment to each video client node. designated destination pushing chunks according to the “RTCData Channel” component of the WebRTC 222 standard (which does not define a signaling protocol).

[00112] In one embodiment, each video segment is divided into 64 kB chunks to match the size of a UDP datagram (“packet”) for maximum efficiency when transmitted via the MPEG2 transport protocol. The client device 200 sends and receives UDP “packets” one piece at a time (falling back to TCP when necessary, in accordance with the WebRTC standard 222). A one-second video segment, for example, would contain about 625 chunks (assuming a 1080p H.264 encoder, which outputs about 5000 kbps).

[00113] As receiver 250 receives chunks of each video segment, it causes receiver assembly 256 to compile those chunks and construct complete video segments. Relay 260 generates relay assembly 257 to compile these chunks for the purpose of sending (relaying) them to designated target client nodes. In this way, the relay assembly 257 acts as a buffer for received and sent pieces of a video segment. As will be discussed below, performance monitor 240 tracks the time required to stream the complete video segment to each designated target client node and reports this measurement back to virtual broadcast server 300 (for subsequent use in reconfiguration). dynamics of overlapping networks 100).

[00114] In one embodiment, a receiving client node receives a video segment from a single feeder node, such as client device 200. In another embodiment, several potential feeder nodes are selected by the virtual broadcast server 300 and communicate with each other to negotiate the “top two” candidates (for example, based on current bandwidth or other monitored performance measures) and then send alternating chunks to the designated receiving client node.

[00115] In another embodiment, several different resolutions (e.g., 1080p, 720p, and 480p) of each video segment are pushed between nodes A and the virtual broadcast server 300 informs node A at the root of each overlapping crowd network which of these resolutions must be transmitted to the other nodes in that overlay crowd network (e.g., based on the capabilities of those other nodes, as discussed in more detail below).

[00116] During the time that receiver 250 is receiving chunks of a video segment for execution and relay 260 is streaming those chunks to other designated client nodes, performance monitor 240 gathers various dynamic performance measurements static and real-time measurements as reported by virtual broadcast server 300, and continuously provides these measurements back to virtual broadcast server 300 via signaling server 330.

[00117] As indicated above, these measurements are used by the virtual broadcast server 300 to dynamically reconfigure the overlay networks 100 in order to optimize the routing of the next video segment. In particular, performance measures are used to classify and reclassify client nodes, allocate and deallocate slots for retransmitting video segments to other client nodes, determine what resolutions of video segments can be received and retransmitted to other client nodes, and , finally, modify a subset of the routing paths between customer nodes when the overlay networks 100 are dynamically reconfigured. The precise way in which these performance measures are utilized by the virtual broadcast server 300 will be discussed in more detail below with respect to Fig. 3.

[00118] Static performance measurements such as operating system type, browser, and connection (e.g., 3G or 4G cellular, WiFi, LAN, etc.) are not prone to frequent changes and are typically reported to signaling server 330 only upon initial connection request by the client device 200 (although they will be reported in the event of a change - for example, changing the cellular connection from 3G to 4G).

[00119] Although dynamic information can be collected and reported on an ongoing basis (i.e., as it is gathered), several trade-offs are taken into account in a realization to ensure that the “headline” (frequency of monitoring and reporting these dynamic measures to the signaling server 330) does not affect the payload or performance of the delivery of the video itself (i.e., the streaming of chunks to and from the client device 200). In one embodiment, these measurements are used solely for the next video segment, while in other embodiments, changes may be made for the next chunk (or multiple chunks) while delivering the current video segment.

[00120] In one embodiment, two types of dynamic performance monitoring are performed. The first involves “ping” times (or other similar measurements) to known locations on the Internet (e.g., to a Yahoo Web server, virtual broadcast server, etc.), both within and across the ASN in which the device resides. client 200. Individually, these measurements provide indications of the performance of the client device 200, while collectively they provide additional indications of QoS, within the ASN in which the client device 200 resides and across the ASNs through specific peer points. Although the virtual data trunk overlay network (between nodes A) is of relatively greater concern (due to congestion at peer points), congestion in ASN is also relevant (it may, for example, require dynamic reconfiguration of at least part of one or more of the crowd overlapping networks within the ASN).

[00121] The other type of dynamic performance monitoring involves the total time required to retransmit a video segment from one client node to another. In one embodiment, each node (other than C nodes) records the “start” time when it sent the first chunk of a video segment to a designated destination client node, as well as the “stop” time after receiving the last chunk of that video segment (for example, because the WebRTC 222 standard provides checks for each packet). Performance monitor 240 sends this total time (for each video segment it sends) to signaling server 330. This measurement can also provide indications regarding not only the individual performance of client device 200, but also the level of congestion in its ASN and across ASNs (e.g., if the client device 200 is a node A that feeds another node A through an ASN peer point).

[00122] In one embodiment, the user of client device 200 may also be the source of the video content. In most cases, this scenario results from the ever-increasing quality of smartphone cameras (such as camera 219), which allow users to capture video events “anywhere, anytime.” It is also possible, however, for users of desktop or laptop computers, as well as smartphones, to also obtain pre-recorded video events from other sources.

[00123] The problem is that the client device 200 must somehow stream its video content via the Internet to the virtual broadcast server 300, which may be many hops away over several ASNs. Loader 280 addresses this problem through a particular “shower” protocol designed to avoid delaying or blocking UDP packets at intermediate routers. In one embodiment, the loader 280 is implemented via a dedicated smartphone application on the client device 200, rather than relying on more limited client-side Javascript functionality.

[00124] To implement this shower protocol, loader 280 establishes a TCP/IP connection with virtual broadcast server 300 and employs UDP “pulses” to provide the largest available IP packet sizes (“maximum transmission unit”). or MTU). Additionally, continuous UDP streams (whether sent through an isolated router port or distributed across multiple router ports) will often be detected by intermediate routers as a “denial of service” (DOS) attack and therefore blocked. . Additionally, these UDP streams can overwhelm the router's allocated memory (for example, a FIFO queue), because routers typically allocate memory for UDP packets (as opposed to the more common TCP packets) only while they are being received.

[00125] To address these obstacles, loader 280 not only distributes UDP packets among multiple ports (e.g., six ports in one embodiment), but also slows down packets sent on any individual port to avoid detection as a DOS attack. In one embodiment, the delay on each port is long enough to avoid detection as a DOS attack and long enough to allow routers to allocate enough memory, but short enough to avoid being perceived as the end of a UDP stream (the which would cause the router to suspend memory allocation for UDP packets and essentially “throw them away”).

[00126] As loader 280 delivers each video segment to virtual broadcast server 300 in this manner, virtual broadcast server 300 then generates a channel to redistribute this video content across overlapping networks 100, as if it had been received from a more traditional CDN. In another embodiment, the virtual broadcast server 300 employs this particular shower protocol in the relatively infrequently used scenarios where it is the fallback point source of a video segment to a client node whose current video segment is not arrived in time over 100 overlapping networks.

[00127] Virtual broadcast server:

[00128] Fig. 3 illustrates an embodiment of key server-side components of a virtual broadcast server 300 in accordance with the present invention. As indicated above, although the components of the virtual broadcast server 300 are illustrated on an isolated physical hardware server, the functionality of these components can be reallocated among several different physical hardware devices and different software modules without departing from the spirit of the present invention.

[00129] The virtual broadcast server 300 includes certain standard features, such as the standard HW/SW 310, found in most hardware servers - e.g., CPU 312, memory 314, operating system 316, network adapter 317 and display 318. In certain embodiments, the virtual broadcast server 300 also leverages standard libraries 320, which may include, for example, (i) the STUN 322 (Session Traversal Utilities for NAT) protocol, which enables discovery of public IP addresses of client devices 200 behind a NAT firewall, so that the client nodes can send and receive video to and from other client nodes, in addition to establishing connections to the virtual broadcast server 300; (ii) the WebSocket 326 protocol, which enables fast bidirectional client-server communications through a single TCP/IP connection; and (iii) HTTP 328, which is employed for less frequent standard communications with client web browsers, such as the standard HTML5 web browser 230.

[00130] The virtual broadcast server 300 does not need to support WebRTC 222 and adaptive streaming standards 224 as it is not a client node in the overlay networks 100, even though it continually analyzes performance measurements obtained from client nodes and dynamically reconfigures the routing paths for the video content channels distributed among client nodes over the overlay networks 100.

[00131] The virtual broadcast server 300 serves as the channel source point for the overlay networks 100, particularly the virtual data trunk overlay network. In one embodiment, the POI content server 380 designates one or more nearby A nodes (preferably in their ASN, if possible) to issue HTTP requests for video segments. These A nodes effectively serve as the root of the virtual data trunk overlay network and push each video segment to other A nodes within and across ASNs, and ultimately to other nodes across the crowd overlay networks within each ASN.

[00132] As will be described in more detail below with reference to the POI 380 content server, this “channel source” functionality does not require the use of the standard adaptive streaming 224 and WebRTC 222 libraries that are targeted at browser video streaming. for browser. As indicated above, the POI content server 380 also serves as an occasional fallback source of video segments for client nodes that do not receive the current video segment in time over the overlay networks 100. These client nodes issue requests to which the POI 380 content server responds by sending the requested video segment.

[00133] As also indicated above, the POI content server serves as the point of origin for all video channels (in one embodiment), whether the video content is obtained from a client device 200 via loader 280 or from a More traditional CDN (either streamed to virtual broadcast server 300 in real time or provided in advance for later streaming).

[00134] The channel administrator 385 is responsible for configuring and maintaining each channel, while the POI content server 380 prepares the content itself for streaming as a channel to client nodes. In one embodiment, the channel administrator 385 generates and maintains the channel Web page for delivery by the POI content server 380 via the Internet and use by the signaling server 330 when responding to connection requests from client devices 200 seeking connection to a channel. specific.

[00135] For support purposes, “viewer support console” is established and maintained by channel administrator 385 to support individual observers whose client devices 200 are experiencing issues, as well as an “execution center” for live monitoring of all video channels so that channel-specific and region-specific issues can be addressed (for example, as support calls from a specific geographic region accumulate). Real-time “channel analytics” monitoring is also maintained by channel administrator 385 to provide useful data to these support functions as well as to producers of video content (e.g., in a CDN). Analyzes include, for example, real-time measurements relating to the current state of each video channel and network nodes across the 100 overlay networks, as well as on-arrival and other issues relating to video bit rates, congestion points, node latency etc.

[00136] Finally, “channel administration” functionality is provided to manage the video channels and interface with the signaling server 330 so that it has current information necessary to facilitate its communications with client devices 200 ( for example, with respect to binding to a channel, providing client-monitored performance measurements, obtaining routing and resolution or bitrate changes for retransmission targets, etc.).

[00137] The remaining server-side functionality illustrated in Fig. 3, with the exception of splash extractor 390, will be described, for simplicity, in the context of a single channel of video content. Note, however, that this functionality is carried out simultaneously, in one implementation, for several channels at any given time and for a series of digital content.

[00138] Before client nodes have access to a video channel, the video content is transcoded to create multiple lower resolution streams of video segments. In one embodiment, the POI content server 380 is implemented as an HTTP server 228 that can communicate with standard HTML5 web browsers 230 on client devices 200. Unlike signaling server 330, which establishes WebSocket connections with devices of client 200 for frequent bidirectional communications (e.g., exchanging routing changes, performance data, etc.), the POI content server 380 responds to HTTP requests 228 from clients with relatively low frequency of standard HTML5 web browsers 230 for files of manifesto, occasional video segments that didn't arrive in time via overlay networks 100, etc.

[00139] As indicated above, the POI content server 380 also relies on the HTTP protocol 228 to implement its higher bandwidth channel source functionality - i.e., responding to HTTP requests for video segments from nearby A nodes (at the root of the virtual data trunk overlay network, typically in the same ASN as the POI 380 content server or within one or two hops). In other embodiments, these video segments are pushed to these nodes A in accordance with adaptive streaming standards 224 and WebRTC 222 or through other video streaming methods (including the shower protocol utilized by loader 280, as discussed above ).

[00140] In one embodiment, the POI content server 380 transcodes video content into three different resolutions (1080p, 720p, and 480p), while various other higher and lower resolutions are supported in other embodiments (e.g., 4K, 360VR, 180VR , 240p, etc.), including a single fixed resolution for all video content. If the original source video is provided in a lower resolution (e.g. 720p), only 720p and 480p resolutions can be supported for that video channel. This functionality enables adaptive bitrate streaming, whether initiated by client nodes (as discussed above) or by virtual broadcast server 300 based on analysis of client performance measurements.

[00141] In one embodiment, the POI content server 380 initiates a channel responding to an HTTP request to provide all available versions (e.g., three different resolutions) of each video segment to one or more nearby nodes (typically, nodes A) that begin pushing each video segment along the 100 overlapping networks. In another embodiment, these nodes relay all versions to nodes B (and nodes B:A) and ultimately to nodes C, from so that each client node can leverage adaptive streaming capabilities 224. Nodes that relay various resolutions to other nodes “polycast” these various versions of a video segment to other client nodes via the overlay networks 100, as explained with more details below.

[00142] Note that while the POI content server 380 initiates a channel by providing video segments to one or more nearby nodes (in response to HTTP requests), all client observation nodes actually receive and observe each segment. simultaneously - that is, they are all synchronized as long as each video segment traverses the overlapping networks 100 before the previous video segment finishes playing. Because the client devices 200 buffer at least three video segments in this embodiment, this buffer provides some “margin of error” if a video segment occasionally experiences some delay. Furthermore, in another embodiment, the start of a channel may be delayed to provide additional buffering when the POI content server 380 first begins transmitting the channel. When a client device 200 issues a video segment request directly from the fallback POI content server 380 (e.g., because the video segment did not arrive in time via the overlay networks 100), this buffer may be necessary, e.g. , if that video segment crosses one or more ASNs.

[00143] As indicated above, the POI content server 380 also provides periodic manifest files in response to requests from the client device 200. Although these manifest files are provided via standard HTTP 328 protocols, they are relatively small and much less time critical than video segments. In one embodiment, each manifest file identifies the location of the following eight video segments at various available bit rates. In this embodiment, the locations are the fallback locations of the POI content server 380 as the video segments are pushed to each client device 200 via the overlay networks 100.

[00144] After preparing a channel of video content for streaming (starting with POI content server 380), signaling server 330 waits for connection requests from client devices 200. Upon receiving a connection request for that channel of a client device 200, the signaling server 330 relies on the STUN protocol 322 to ensure that it can establish WebSocket connection 326 through any NAT firewall that may be present on that client device 200. Additionally, by identifying the IP address public IP address of that client device 200 , it may provide that public IP address to other client nodes (e.g., to relay a video segment to that client device 200 ).

[00145] Upon establishment of a WebSocket connection 326, the client device 200 provides the signaling server 330 with information regarding its capabilities (e.g., operating system, web browser, and connection type - 3G, 4G, WiFi, LAN etc.), including, in one embodiment, whether the client device 200 is a C node (considered, for example, for cellular connections in this embodiment. The client device 200 also provides its ASN location to the signaling server 330, which will later be used to add the client device 200 to the overlay networks 100.

[00146] In one embodiment, signaling server 330 prioritizes providing one or more initial video segments to client device 200 (via overlay networks 100) so that it can begin playing video content from the channel as soon as possible. To begin this process, it transfers control to the overlay network builder 350, which adds the client device 200 to a crowd overlay network in its ASN (directing, for example, a node B of that ASN to retransmit video segments for client device 200). Note that client device 200 has not yet been classified and will not retransmit any video segments to other client nodes. But, by being part of overlay networks 100, the client device 200 can begin receiving video segments and playing the channel's video content, as well as collecting client performance measurements, which will enable its classification.

[00147] The signaling server 330 therefore obtains (via its WebSocket connection 326) the bandwidth up and down the stream from the client device 200. Note that this measurement is not terribly useful, as the connection may cross multiple ASNs (even though the signaling server 330 knows the ASN location of the client device 200). A more relevant measure will concern communications between the client device 200 and other client nodes in its own ASN.

[00148] Upon receipt of client performance information (collected by performance monitor 240 on client device 200) from client device 200 (and other client nodes), signaling server 330 forwards this information to the client device 200. performance 340 for initial analysis and subsequent use by the overlay network builder 350 and the deep mapper 360 in dynamically reclassifying client nodes and reconfiguring overlay networks 100 for the next video segment, as explained below. Performance tracker 340 monitors the performance of each client node and determines whether the client node is still “alive.” If the client device 200, for example, has closed the connection and left the channel or does not respond to a “ping” within a timeout period, it will be considered to have left the channel (either intentionally or as a result of hardware or software failure). Performance tracker 340 also converts customer performance measures into appropriate format for storage in performance history database 345 and use by overlay network builder 350 and deep mapper 360.

[00149] In one embodiment, the overlay network builder 350 is also responsible, with assistance from the deep mapper 360, for the ongoing process of evaluating current and historical customer performance measures (maintained in the historical performance database 345 ) and, dynamically, for each video segment (i) reclassifying client nodes and (ii) optimizing routing paths, generating and reconfiguring the overlay networks 100, including the virtual data trunk overlay network (to retransmit the segment between nodes A, within and across ASNs) and crowd overlay networks (to relay the video segment from each node A in an ASN to some other nodes B:A, B and C in that ASN). The topology of the overlay networks 100 is maintained in the overlay network database 375 for use by the overlay network creator 350 and the deep mapper 360.

[00150] With respect to performance measurements received from the newly added client device 200, the overlay network builder 350 uses these measurements to initially classify the client device 200. In one embodiment, this process is also used to potentially reclassify the client nodes for each video segment (not simply when they join the channel). Although client nodes are not typically reclassified very frequently, clients may experience temporary bandwidth drop (for example, from a household microwave or other interference). Additionally, as more A nodes are needed (for example, due to redundancy or due to client nodes leaving a channel), B:A nodes can be upgraded to A nodes. Other issues detected in an ASN or across multiple ASNs can may also require the reclassification of certain nodes.

[00151] The overlay network builder 350 allocates to the client device 200 input and output slots (i.e., network ports) so that it can receive chunks of video segments (via input slots) pushed by other client nodes and be able to relay (push) these chunks of video segments (through output slots) to other client nodes. Although the WebRTC standard 224 supports 256 input and output ports (slots), only one isolated input slot is allocated in an embodiment (to maximize the quality of video content that can be played on the client device 200) and, in the maximum, eight output slots are allocated (to maximize throughput over overlapping networks 100) and support wide variety of client devices 200 and limited bandwidth connections). As indicated above, A nodes are allocated four output slots for relaying video segments to other A nodes through ASN peer points and four output slots for relaying video segments to other A nodes within their ASN. As will be explained below, not all allocated slots will necessarily be employed at any given time.

[00152] The overlay network builder 350 analyzes the bandwidth above and below the stream from the client device 200 to facilitate the classification process. As indicated above, if the client device 200 connects via a cellular connection (3G, 4G, or even LTE), it is automatically considered too unreliable to retransmit video segments and is therefore classified as a C node. In other embodiments, this automatic classification may be limited to certain cellular connections (e.g., 3G) or eliminated entirely.

[00153] In one embodiment, the overlay network builder 350 employs typical upstream/downstream bandwidth categories (in Mbps) to enable further classification, including: (1) LAN connections (e.g., 100/100 ), (2) fiber connections (100/50), (3) ADSL connections (100/20), cable connections (100/10) and WiFi connections, which vary greatly). In this embodiment, if client device 200 is no longer considered node C and has bandwidth upstream of at least 50 Mbps, it is initially classified as node A (or as node B:A if the 360 deep mapper indicates that no additional A nodes are required in your ASN). Otherwise, it will be classified as node B.

[00154] As will be discussed below, the overlay network designer 350 further analyzes the bandwidth above the stream from the client device 200 (in one embodiment) to calculate the number of available output slots it can utilize before determining the bandwidth. which (if any) shall dynamically reconfigure the overlay networks 100. It also determines the extent to which the client device 200 is capable of receiving and/or polycasting various resolutions.

[00155] In one embodiment, the bandwidth below the total stream of a client node is used for its isolated input slot, while only 1/3 of its bandwidth above the stream is used to retransmit video segments between the your output slots. Its bandwidth above the full stream is not utilized, as retransmission of video segments may interfere with TCP/IP and other connections used by client device 200 for other applications.

[00156] Overlay network builder 350 analyzes the bandwidth downstream of client device 200 (even if classified as node C) to determine the number of resolutions it can sustain through its isolated input slot. If 1080p requires, for example, a bit rate of 3 Mbps and 720p requires a bit rate of 1.5 Mbps and 480p requires a bit rate of 500 kbps, the client device 200 would require bandwidth below in the stream of at least 5 Mbps to sustain all three resolutions, at least 4.5 Mbps to sustain 1080p and 720p, at least 3 Mbps to sustain 1080p only, at least 2 Mbps to sustain 720p and 480p, at least 1.5 Mbps to sustain only 720p and at least 500 kbps to sustain only 480p. In one embodiment, bit speeds of less than 500 kbps will not be sustained. In other embodiments, lower resolutions may be sustained and other methods (such as greater compression, different video formats, etc.) may be employed to reduce bandwidth requirements.

[00157] As indicated above, in one embodiment, nodes A, B:A and B can also be considered polycast nodes that can relay various resolutions to other nodes through one or more of their output slots. In this particular, the overlay network designer 350 analyzes the bandwidth above the stream from the client device 200 to determine the number of resolutions it can relay to other client nodes.

[00158] Since a client node can utilize only 1/3 of its upstream bandwidth in this embodiment, client device 200 would require upstream bandwidth of at least 15 Mbps (per output slot) to polycast all three resolutions, at least 13.5 Mbps (per output slot) to polycast 1080p and 720p, at least 9 Mbps (per output slot) to send 1080p only, at least 6 Mbps (per slot output) to polycast 720p and 480p, at least 4.5 Mbps (per output slot) to retransmit only 720p, and at least 1.5 Mbps (per output slot) to retransmit only 480p.

[00159] Client device 200 cannot retransmit a resolution that it does not receive. Furthermore, the polycast capabilities of the client device 200 are considered in conjunction with the ability of other client nodes to receive various resolutions, as explained below. But, as indicated above, client device 200 employs adaptive streaming implementations 224 to request lower or higher resolution versions of video segments as it experiences significant changes in its bandwidth. If it receives several different resolutions of a video segment, it will simply play the highest resolution received.

[00160] Considering that the client device 200 is not a C node, the overlay network designer 350 calculates the number of available output slots that it can utilize by analyzing its bandwidth upstream and also considering the extent to which it can perform polycast of different resolutions. If the client device 200 is classified as node A with a LAN connection that has bandwidth above 100 Mbps stream, for example, it can use only about six output slots to polycast video segments, both on its ASN and through ASNs. In this embodiment, the overlay network creator 350 would allocate four slots to polycast to other A nodes across ASNs (giving priority to these slots across ASNs), leaving two remaining slots to polycast to other A nodes within its ASN. In other embodiments, these allocations may naturally vary without departing from the spirit of the present invention.

[00161] Similarly, if the client device 200 is classified as a B:A or B node with a cable connection that has bandwidth above 10 Mbps flow, it may use only one output slot to perform polycast of resolutions of 720p and 480p, or send only 1080p. In one embodiment, priority is given to higher quality resolutions (to the extent that nodes can receive that resolution) and therefore a slot would be allocated only for 1080p. Here too, these allocations may vary without abandoning the spirit of the present invention.

[00162] After classifying the client device 200 and determining the number of slots that can be used (including multi-resolution polycast), the overlay network builder 350 determines the extent to which it will dynamically reconfigure the overlay networks 100 to optimize paths of routing. If the client device 200 is a node A, the overlay network builder 350 will first obtain from the deep mapper 360 the congestion levels for each inter-ASN path between nodes A (as discussed in more detail below), and then It will then dynamically reconfigure at least part of the virtual data trunk overlay network to incorporate the client device 200.

[00163] Considering a set of weighted paths (where each path has “congestion level” weighting), the overlay network builder 350 employs standard path search methods to determine the optimal path to distribute a video segment among the A nodes (analogous to, for example, GPS navigation routing). Note, however, that this process is slightly complicated by the use of multiple relay slots - for example, four output slots for nodes A that relay to nodes A in an ASN and four output slots for nodes A that relay to nodes A through an ASN peer. Still, this is just a slight variation of the simplest case where node A has only one output slot. In other words, the overlay network creator 350 tracks the number of open (unused) slots during the generation or reconfiguration of the virtual data trunk overlay network and fails to assign a specific node A as a retransmission source when it has no more slots. open unused.

[00164] If the client device 200 is a B:A or B node, the overlay network builder 350 dynamically reconfigures, in whole or in part, the intra-ASN crowd overlay networks in the ASN in which the client device resides. 200. Note that, if there are several A nodes in that ASN, their paths between them will be determined as part of the virtual data trunk overlay network. In one embodiment, only one node A will be used to create a crowd overlay network (if sufficient slots are available), while in other embodiments, the other nodes may be allocated equally among the various A nodes or distributed based on breadth above. in relative flow or other measurements.

[00165] With respect to any specific A nodes and the remaining B, B:A, and C nodes in an ASN, these nodes are evaluated first based on their classification (i.e., B:A, then B, then C ) and then based on its corresponding bandwidth (i.e. the number of available slots that can be used, as described above). Note that the crowd overlay network is a hierarchy in this realization, considering that each node has only a single feeder node. Similar methods can be employed for non-hierarchical “grid” crowds in other realizations.

[00166] In this hierarchical crowd realization, the process starts with the root node A, which will have a certain number of output slots that can be used (for example, two output slots). These slots will be directed to the next level of the hierarchy - for example, the two B:A nodes with the highest amount of available slots that can be used. After determining these paths, the available egress slots from these nodes will be routed to the remaining B:A nodes with the highest number of available slots. This process continues down the hierarchy (through B nodes and finally C nodes) until all paths have been determined.

[00167] Note that the chain length below any one client node (e.g., one hundred client nodes, each with a single output slot) is of relatively little concern considering the relatively high speed (well below 1 ms) of relay between nodes in an ASN. Considering a one-second video segment, chains of hundreds of nodes could still be accommodated (although they would be rare, considering that many nodes in an ASN would likely support multiple output slots). If not all nodes could be included in a crowd (for example, if C nodes and B nodes with zero available slots remained unrepresented), there would be a need for additional nodes with open slots in that ASN, which would be allocated as they become available . Meanwhile, these nodes would be directed to requesting video segments from the POI 380 content server.

[00168] Before turning to the 360 Deep Mapper, which predicts and quantifies congestion levels along ASN peer points (e.g., for the next minute), it is helpful to understand the limitations of BGP routing protocols to appreciate the meaning of ASN peer peer congestion. BGP routers determine congestion at “routing time” and do not have prediction capabilities. They only know their own routers and the latency “one hop away” along an ASN peer point. They do not know the number of hops or latency to any final destination, which may be several hops away across multiple ASN peer points. Given a selection of multiple ASN peer points, they essentially select the one with the highest bandwidth currently available (i.e., the one with an open slot and the lowest latency one hop away).

[00169] On the other hand, the 360 deep mapper leverages its knowledge of the underlying architecture of the Internet. In one embodiment, the 360 deep mapper maintains an ASN interconnection map of the Internet (including ASNs and their various peer interconnections), as generally illustrated in Fig. 1. This map does not change frequently, although it is monitored, in one realization, every five minutes to capture these irregular changes.

[00170] The overlay networks 100 built on top of these ASNs are, however, analyzed frequently (e.g., through client-side monitoring, as discussed above) and potentially reconfigured every video segment (e.g., every second in one embodiment) by virtual broadcast server 300. In practice, however, overlay networks 100 are actually modified only when warranted - for example, not only when new nodes join or leave the channel, but also when sufficient problems are detected (with based on historical and current information maintained in the DB 345 performance history).

[00171] Various internal “congestion limits” are employed, for example, in one realization. Upon initial detection of a specific relatively low congestion threshold of a specific client device 200 or in an ASN, the overlay network builder 350 simply “marks” the client device 200 or ASN and waits to see if the problem reappears (e.g. example, in the following video segment). If this occurs, it can decrease the resolution (and therefore the bitrate) of the next video segment retransmitted to that client node (or all client nodes in that “problem” ASN). Eventually, if the problem becomes worse (e.g., exceeding a higher congestion threshold), a portion of the overlapping networks 100 (e.g., a subset IP range in an ASN) may be dynamically reconfigured. Finally, a complete ASN or perhaps the virtual data trunk overlay network itself may require dynamic reconfiguration.

[00172] In any eventuality, the purpose of these congestion limits is to proactively identify and correct problems before they degenerate into more significant problems, causing the loss of video segments, or even causing nodes to fail. of clients resort to obtaining video segments from the fallback location of the POI 380 content server.

[00173] By maintaining awareness of the Internet ASN interconnect map and the ASN location of nodes in the overlay networks 100 and monitoring in real-time the historical and current performance of these nodes, the 360 deep mapper minimizes the likelihood that any client node unnecessarily retransmit a video segment to a distant client node (for example, many hops away via multiple ASN peer points). As initial matter in one embodiment, for example, the virtual data trunk overlay network will tend to direct video segments (whenever possible) from one node A to another node A in the same or a nearby ASN, through a single ASN peer point.

[00174] Not all isolated jumps are, however, created equally. The deep mapper 360, for example, may “learn” over time (based on performance measures maintained in the historical performance database 345) that a peer point between “ASN 1” and “ASN 2” is getting worse. congested and can “predict” that a two-hop path from “ASN 1” to “ASN 3” to “ASN 2” is actually faster than the current one-hop path (or will be faster in the very future). next, based on recent and historical trends). By quantifying peer point congestion based on the current and historical performance of A nodes across peer points, the 360 Deep Mapper can enable dynamic reconfiguration of the virtual data trunk overlay network topology - potentially for each segment or at least when peer point congestion necessitates such changes (based on internal limits).

[00175] In one embodiment, the 360 deep mapper quantifies congestion with respect to each pair of nodes A (whether they reside in the same ASN or different ASNs), employing a scale of 1 to 10, where 1 is the most low predicted short-term congestion and 10 is the highest. As indicated above, the overlay network designer 350 uses this congestion level “assessment” to compare different potential paths between nodes A and determine the most efficient path (i.e., the path with the lowest “weighted hop”). As a result, the A nodes that are furthest away (in weighted hops) from the POI 380 content server will minimize the amount of time required for a video segment to traverse the virtual data trunk overlay network to those server A nodes. of POI 380 content.

[00176] In one embodiment, for each pair of nodes A, the deep mapper 360 generates an assessment of the predicted congestion level for each lane from one node A to the other and then selects the lowest congestion level assessment to be applied to that pair of nodes A, which returns to the overlay network 350. In other embodiments, the deep mapper 360 generates a function different from those predicted congestion level assessments (for each lane, from one node A to the other), such as mean, median etc.

[00177] Deep mapper 360 is, in one embodiment, a deep learning engine that continuously analyzes performance measurements maintained in historical performance database 345 and predicts (e.g., one minute in the future) the level of congestion along the ASN peer points. It should be noted that, like any deep learning engine, the 360 deep mapper employs several nonlinear transformations to model the behavior of ASN peer points, with respect to traffic between A nodes along those peer points.

[00178] As indicated above, it cannot monitor the volume of Internet traffic crossing these peer points, but only the effect over time that this traffic has on inter-ASN hops between nodes A through these peer points . The more performance measurements you obtain, the better you can predict the time required for these inter-ASN hops, which is then quantified as the relative congestion level (e.g., compared to intra-ASN hops which are typically much less congested, although also monitored in this realization).

[00179] As the congestion level of peer points is very dynamic, these predictions can only be accurate for a short period of time. But considering that this analysis is performed continuously and may change for the next one-second video segment, it is not critical that the prediction is accurate over a long period of time.

[00180] In one embodiment, the 360 deep mapper initially quantifies ASN peer points based on very general information (i.e., before large amounts of client performance measurements are obtained). If an ASN has 1000 peer points, for example, it can be considered to be a main chain that is likely to be much faster than another ASN with six peer points. The more client performance measurements you obtain, the more accurate these peer congestion levels become. In another embodiment, several “learning nodes” are deployed to “jump and start” a new channel. These learning nodes are send-only nodes that do not watch the video, but are simply deployed to provide information about client performance quickly so that the 360 deep mapper can start making more accurate predictions sooner than would otherwise be the case. other way.

[00181] Additionally, in one embodiment, the 360 deep mapper also considers intra-ASN congestion, as this may suggest the need for, for example, additional A nodes in an ASN and thus the creation of additional crowd overlay networks. . If many client nodes in an ASN are gradually taking longer to obtain video segments over time, for example, the deep mapper 360 marks the ASN to indicate that additional A nodes are needed and the overlay network builder 350 can “ promote” one or more B:A nodes to B nodes, resulting in partial reconfiguration of the virtual data trunk overlay network and ultimately necessitating new crowd overlay networks in the ASN. In another embodiment, the deep mapper 360 applies deep learning methods on each ASN and assists the overlay network builder 350 in generating intra-ASN crowd overlay networks.

[00182] The overlay network builder 350 and the deep mapper 360 work together to establish pathways between customer nodes (through the overlay networks 100) that are based on the underlying Internet architecture (ASN interconnection map) and the location of ASN of client nodes overlaid on top of that architecture in order to minimize retransmissions of video segments over unnecessarily distant paths (i.e., across multiple ASN peer points). Additionally, the overlay network builder 350 and the deep mapper 360 also work together to continuously analyze real-time client performance measurements obtained by client devices 200 and dynamically reconfigure the overlay networks 100 if these measurements reveal significant problems (often due to congestion at ASN peer points). As a result, Internet QoS volatility can be monitored and the effects on client nodes of congestion (particularly at ASN peer points) can be minimized through dynamic rerouting around these problems “before they occur” (with based on congestion levels predicted and generated by the deep mapper 360).

[00183] In one embodiment, the virtual broadcast server 300 includes a splash extractor search engine 390 for the purpose of identifying trending video events (“splashes”) and allowing users to search across the domain of these events and stream immediately streaming the desired splash result in the form of a video channel from the POI content server 380 (where such channel was not otherwise available on the virtual broadcast server 300).

[00184] In one embodiment, splash extractor 390 continuously collects data from multiple news sources (e.g., via APIs for Twitter, RSS feeds, Reddit, and tens of thousands of online magazines). On average, thousands of distinct “current events” are revealed in these sources per hour. The 390 Splash Extractor employs innovative automatic methods to identify these trending events (splashes) and locate and extract related videos that can be obtained and streamed via the POI 380 content server.

[00185] The splash extractor 390 identifies “deviations from the norm” in order to detect splashes. For example, a baseline is developed (without the need for normalized data) employing, for example, a standard Levenshtein comparison algorithm across the domain of news sources. On average, no more than a few sources will discuss the same “topic” (i.e., a set of keywords) in any short period of time, unless and until a specific topic is, in fact, trending. At this time (for example, when fifteen or more sources discuss the same topic in a short space of time), that topic is identified as a diversion and therefore a splash.

[00186] Splash extractor 390 then extracts the “most important” keywords from those sources (e.g., forty keywords in one realization) - in one realization, employing standard neural network methods to learn and predict the words different keywords from “splash-related” articles. These keywords are then classified (e.g. like news, sports, etc.) and evaluated by frequency.

[00187] Splash extractor 390 then uses these keywords to search social media for videos relating to each splash and indexes the text related and associated with these potential splash video channels. Users can search through that index or simply browse the video event categories in Splash. When selecting a result (either through search or browsing), the user can immediately receive the desired video by streaming. In one embodiment, the user is simply linked to the current source of the video, while in another embodiment, the video is obtained via the virtual broadcast server 300 and streamed via the POI content server 380 (useful, for example, if large numbers of simultaneous users request the same video channel in splash).

[00188] Dynamic video streaming process:

[00189] After discussing the main client-side and server-side components of the virtual broadcast system in accordance with the present invention, flowchart 400 of Fig. 4 illustrates how these components interact dynamically. In other words, flowchart 400 illustrates a dynamic streaming process embodiment in accordance with the present invention implemented by these components. It should be noted that because much of this process is event-driven and non-linear, flowchart 400 illustrates steps from the standpoint of the interaction between client-side and server-side functionality.

[00190] Step 401 illustrates the process performed by loader 280 (and described above), in which a video event is captured by a client node (e.g., a smartphone camera 219 on client device 200), digitally generated or obtained from an external source. In any eventuality, the client (e.g., client device 200) streams video segments of that video event (whether captured live or pre-recorded) to the virtual broadcast server 300.

[00191] Whether the video events are obtained from clients or from the more traditional CDN (and whether they are pre-recorded or broadcast live), the virtual broadcast server 300, at step 410, prepares each video channel for live streaming of the POI 380 content server, as discussed above. At this point, in one realization, a channel web page is generated, which is eventually found by a lead node. When a user of a client device 200 clicks on a desired channel, a connection request is sent to the signaling server 330, along with the capabilities of the client (such as the type of operating system, browser, connection, etc.). Alternatively, the user of client device 200 may find a trending splash video event (as discussed above) and select that video event (at step 410) for streaming in the form of a POI content server video channel. 380.

[00192] In step 412, signaling server 330 checks the client's connectivity to the channel (employing, for example, the STUN protocol 322 to identify the client's public IP address) and then establishes a connection via WebSocket 326 through any NAT firewall that may be present on the client, and subsequently provides that public IP address to other client nodes in order to relay a video segment to that client. The signaling server 330 then hands over control to the overlay creator 350, who adds the client (not yet classified) in node form onto the overlay networks 100, from which initial video segments will be pushed to the client (in step 414), so that the user can immediately start watching the video channel, in step 415.

[00193] Then, the signaling server 330, in step 416, classifies the client device 200 as node A, B:A, B, or C, and in step 430, employs the overlay network builder 350 and the deep mapper 360 to dynamically reconfigure inter-ASN (virtual data trunk) and intra-ASN (crowd) overlay networks 100 to incorporate client device 200 into the network topology. The signaling server 330 then provides the relevant routing information for other client nodes to begin relaying video segments to the client device 200.

[00194] The POI content server 380, in step 435, next responds to HTTP requests from nearby nodes (typically, nodes A) to stream video segments to those nodes as the origin point of the video channel to the over current (reconfigured) overlay networks 100, where each video segment is relayed from one node to another until it is relayed to and observed by the client device 200.

[00195] While client device 200 is receiving chunks and compiling them, at step 450, to watch each video segment of the channel (and potentially also retransmit chunks to other designated client nodes, at step 440), it is also monitoring its performance at step 425, as discussed above with respect to performance monitor 240, and providing client performance measurements to signaling server 330. Additionally, as each video segment is requested, these requests are intercepted at step 455 (by client Javascript code at receiver 250, in one embodiment), as the video segments are being pushed to the client device 200 along the overlay networks 100, as discussed above. The arrow from step 455 to step 425 simply indicates that the monitoring process of step 425 is continuous, simultaneous with receiving, observing, and retransmitting chunks of video segments.

[00196] Also as indicated above, client device 200 periodically initiates HTTP requests for manifest files (e.g., containing the locations of the following eight video segments) from POI content server 380, even though video segments are being pushed to the 200 client device from other client nodes. Occasionally, if a video segment does not arrive in time, the client device 200 will request that video segment directly from the POI content server 380 as a fallback location. Additionally, from time to time, in accordance with adaptive streaming standards 224, the client device 200 may also contact the POI content server 380 to request modified bitrate (e.g., upon detection of changes in its performance levels). ) for subsequent video segments. As indicated above, however, the receiver 250 may well detect this need earlier and contact the visual broadcast server 300 to effect these changes via the overlay networks 100, directing a feeder client node to push video segments with lower or higher resolution to the client device 200 automatically (i.e., not in response to your request).

[00197] In step 452, the POI content server 380 responds to these HTTP requests and provides the requested manifest files and fallback video segments to the client device 200. As indicated above, changes to bit rates are addressed by through the overlay networks 100 (and at step 430), which results in higher or lower resolution video segments being pushed to the client device 200.

[00198] Step 454 encompasses the continuous process (performed for each video segment, in one embodiment, and described in detail above) performed by the performance tracker 340, overlay network creator 350, and deep mapper 360. In this step 454, the Client performance information is continually updated and, if necessary, at step 430 (as indicated by the arrow from step 454 to step 430), the overlay networks 100 are dynamically reconfigured and new routing information is provided to relevant relay nodes by via the signaling server 330.

[00199] Finally, in step 460, splash extractor 390 continually identifies trending splash video events, which users of client devices 200 can search or browse and then stream to watch immediately as per discussed above.

[00200] The present invention has been described here with reference to specific embodiments, as illustrated in the attached drawings. It should be understood that, in light of the present invention, additional embodiments of the concepts described herein may be devised and implemented within the scope of the present invention by those skilled in the art.

Claims (8)

1. A virtual broadcast server (300) adapted to determine routing of each of a plurality of digital content segments to a plurality of destination client nodes (200) of an underlying network (110), the virtual broadcast server ( 300) characterized by the fact that it comprises: (a) a memory adapted to store an Autonomous System Number (ASN) interconnection map of the underlying network (110) and the peer points that interconnect them and an ASN location of each target client node (200); (b) an overlay network database (375) adapted to store an overlay network topology representing a current state of an overlay network (100) built on the underlying network (110), wherein the overlay network topology includes: (i) each of the plurality of target client nodes (200), each target client node being both a node of the overlay network (100) and the underlying network (110); and (ii) a set of routing paths that each segment traverses as it is relayed between the plurality of destination client nodes (200) via shared links that interconnect components of the underlying network (110); (c) a performance tracker (340) adapted to monitor measures of network traffic between the plurality of target customer nodes (200) along the overlay network (100), wherein said measures include congestion at points of ASN peer; wherein the virtual broadcast server (300) further comprises: (d) a deep learning engine (360) adapted to analyze the ASN interconnect measurements and map of the underlying network (110) over time to predict levels congestion indicators that reflect the changing capacity of ASN peer points over time; and (e) an overlay network builder (350) adapted to: (1) determine whether one or more of the predicted congestion levels meet a predefined congestion threshold and, if so, (ii) dynamically reconfigure the overlay network topology by modifying one or more of the routing paths. 2. The virtual broadcast server (300) of claim 1, wherein the underlying network (110) is the Internet and the digital content segments are ordered segments of video content streamed to the plurality of target client nodes (200). 3. The virtual broadcast server (300) of claim 1, wherein the reconfiguration of the overlay network topology (100) is based, in part, on the ASN locations of one or more of the plurality of target client nodes (200). 4. Virtual broadcast server (300), according to claim 1, characterized by the fact that the reconfiguration of the overlay network topology (100) is based on current and historical trends detected from analysis of measurements throughout the time. 5. Virtual broadcast method for routing each of a plurality of digital content segments originating on a server or node to a plurality of destination client nodes (200) of an underlying network (110), the method being performed by a virtual broadcast server (300) and characterized by the fact that it comprises the following steps: (a) maintaining an Autonomous System Number (ASN) interconnection map of the underlying network (110) and the peer points that interconnect them and an ASN location of each target customer node (200); (b) generating an overlay network topology and storing via an overlay network database (375) an overlay network topology representing a current state of an overlay network (100) built on the underlying network (110), wherein the overlay network topology includes: (i) each of the plurality of target client nodes (200), each target client node being both a node of the overlay network (100) and the underlying network (110); and (ii) a set of routing paths that each segment traverses as it is relayed between the plurality of destination client nodes (200) via shared links that interconnect components of the underlying network (110); (c) monitoring via a performance tracker (340) measurements of network traffic between the plurality of target client nodes (200) along the overlay network (100), wherein said measurements include congestion at peer points from ASN; wherein the method further comprises the following steps: (d) analyzing the ASN interconnect measurements and map of the underlying network (110) over time to predict via a deep learning engine (360) the congestion levels that reflect the changing capacity of ASN peer points over time; (e) determine via an overlay builder (350) whether one or more of the predicted congestion levels meet a predefined congestion threshold and, if so, (f) dynamically reconfigure via the overlay builder (350) the network topology overridden by modifying one or more of the routing paths. 6. The method of claim 5, wherein the underlying network (110) is the Internet and the digital content segments are ordered segments of video content streamed to the plurality of target client nodes (200). 7. The method of claim 5, wherein the reconfiguration of the overlay network topology (100) is based, in part, on the ASN locations of one or more of the plurality of target customer nodes (200). 8. Method, according to claim 5, characterized by the fact that the reconfiguration of the overlay network topology (100) is based on current and historical trends detected from analysis of measurements over time.