TW201001177A - Apparatus for recursive recombination of streaming interactive video - Google Patents

Abstract

An apparatus is described comprising a Power-over-Ethernet interface, a High-Definition Media Interface (HDMI), a Bluetooth interface, a means for decompressing low-latency streaming interactive video and audio from an Internet-based hosting service and outputting the decompressed audio and video through HDMI, and a means for transmitting to the Internet-based hosting service user control information and statistics from received packets. Further, the apparatus is able to receive two interleaved streams of low-latency streaming interactive video.

Description

201001177 VI. Description of the Invention: [Technical Field of the Invention] The present disclosure is generally in the field of a data processing system that improves the ability of a user to manipulate and access audio and video media. This application is a part of the continuation (CIP) application No. 10/3 15,460, entitled "Apparatus and Method for Wireless Video Gaming", filed on December 10, 2002. The case has been given to the assignee of this CIP application. [Prior Art] Since the days of Thomas Edison, recorded audio and film media have become one aspect of society. In the early 20th century, there were widespread releases of recorded audio media (magnetic columns and records) and movie media (coin-operated jukeboxes and movies), but the two technologies are still in their infancy. In the late 1920s, films and audio were combined on the basis of bestsellers, followed by color films and audio. Radio broadcasts have evolved into broadcast-selling audio media that largely support the form of advertising. When the television (TV) broadcast standard was established in the mid-1940s, television and radio were joined in the form of broadcast-selling media to bring previously recorded or live-lived movies to the home. By the middle of the 20th century, most American households had a phonograph record player for playing recorded audio media, a radio for receiving live broadcasts of broadcast audio, and a broadcast audio for live broadcasts. / Video (A/V) media TV. Of course, these three "media players" (recorders, radios and TVs) group 139848.doc 201001177 S are turned into a "media in the media 2" in a cabinet that shares a common speaker. Although media choices are limited for consumers, the media “ecology=system” is very embarrassing. Most consumers know how to use "media playback" and enjoy the full range of their abilities. At the same time, media publishers φ % ~ and f see jobs and 'and music companies' can distribute their media to both cinemas and families without extensive piracy or "secondary sales:" (ie, Resale of media has been used). Often, publishers do not receive sales, and therefore, secondary sales reduce the revenue that publishers receive for buyers who originally sold their own media. Do = exist in the use of the albums sold during the mid-20th century, but the == will have a big impact on the record publisher, because unlike the movie or view 2 (which is usually viewed by adults - once or only a few times), Music tracks can be listened to hundreds or even thousands of times. Therefore, the music media is far more than the film / : wide: the body "lasting" (that is, for the adult consumer, it has consumers who may like the right consumer to the music, then two: the middle of the century to the present' media ecology The system has experienced a series of fundamental changes for consumers and publishers, as well as a wide range of (especially cassettes with high-quality stereo). In the case of the month, there is indeed a high degree of consumption. The convenience is now. (4) The media practice - the beginning of piracy. Indeed, ^ = 1 is for convenience. The students in the dorms who are going to access each other's album collection will be pirated. In addition, consumers will record J39848.doc 201001177 via radio, instead of buying the album or tape from the publisher. More consumer convenience, because the emblem can now be set to record TV programs, which can be viewed at a later time, and the VCR also led to the establishment of the video rental industry, including movies and TV festivals. Designs can be accessed on a "on-demand" basis. The rapid development of popular home media devices since the mid-1980s has led to unprecedented choice and convenience for consumers, and has led to a rapid expansion of the media publishing market. Today, consumers are faced with multimedia choices and multimedia devices, many of which are tied to a specific form of «or a specific publisher. Congratulations: the body consumer may connect the stack device to each room in the house. A computer that causes a "snake" cable to one or more televisions and/or personal computers (PCs) and a group of remote controls. (In the context of this application, the term "personal computer" is a "PC" Refers to any type of computer suitable for use in a home or office, including a desktop computer, Macintosh® or "his non-Windows computer, WindWinws compatible device, χηχ variant laptop, etc." These devices may include video game consoles, VCRs, DVD players, audio surround sound processors/amplifiers, satellite set-top boxes, cable TV set-top boxes, etc. For enthusiastic consumers, there may be multiple devices with similar functions due to compatibility issues. For example, 'Shafei may have both HD-DVD and Blu-ray DVD players or Mlcrosoft Xbox® Sony Playstation® video game system, 'first two. In fact, due to the incompatibility of some games across several versions of the game console' consumers may have both XBox and later versions (such as Xbox 360®) Frequently, 'consumers are confused about which video 139848.doc 201001177 input and which remote end is used. Even when placing the disc on the correct player (for example, DVD, HD-DVD, Blu-ray, Xbox or Playstation), Consumers still face technical challenges after selecting video and audio inputs for their devices and finding the right remote controls. For example, in the case of a wide-screen DVD, the user may need to first determine the correct aspect ratio (eg, 4:3, full, zoom, wide magnification, cinema wide, etc.) and then on their TV or monitor screen. Set the correct aspect ratio. Similarly, the user may need to first determine the correct audio surround sound system format (e.g., AC-3, Dolby Digital, DTS, etc.) and then set the correct audio surround sound system format. Often, consumers are unaware that they may not have access to the media content of their television or audio system (for example, watching a movie that is squeezed in the wrong aspect ratio, or listening to stereo audio instead of surround sound) . Increasingly, Internet-based media devices have been added to the stack of devices. Audio devices like the Sonos® digital music system stream audio directly from the Internet. Similarly, devices like music players record video and stream it through the home network or via the Internet, where the video can be viewed remotely on the PC. And Internet Protocol Television (IPTV) services provide services similar to eMOT over digital subscriber line (DSL) or other home Internet connections. There have also been recent efforts to integrate multiple media functions into a single device, such as the Moxi® Media Center and the PC running the Windows XP Media Center version. While each of these devices provides a little convenience for the functions they perform, each lacks universal and simple access to most media. In addition, these devices are often costing hundreds of dollars to manufacture, often due to the need for expensive processing and/or local storage. In addition, such modern consumer electronic devices typically consume large amounts of power and consume large amounts of power even when interspersed, which means that they are expensive and waste energy over time. For example, if a consumer forgets to cut a device or switch it to a different video input, the device may continue to operate. In addition, because none of these devices is a complete solution, it must be integrated with other devices in the home, leaving the user with a mouse-line and many remote controls. In addition, when many newer Internet-based devices work properly, they typically provide media in a more general form (compared to the form that would otherwise be available). For example, 'video-based, Internet-based streaming devices often only stream video material, and can't often accompany Qing's interactive "extra billing project" streaming, such as video production, games, or navigation. ; Comments. This is due to the fact that interactive materials are often intended to be used: in the specific format of the particular device in which this μ handles interactivity. For example, each of HDVD, hD-DVD, and Blu-ray Disc has its own special interactive format. Any home media private or local computer (which may be developed to support all popular formats) will be too expensive and complicated to suffer from the singularity (s〇phlstlcation) and flexibility. In the case of Hew's fee operation, this problem is aggravated. If it is later introduced in the future, it may not be more supportive of the new format than the "Beijben" piece, and this will mean that you need to purchase the upgraded local media. Device. The 'end device' may not have the computing power to decode the video, or /, can be ''introduced to the resolution of the video or the stereoscopic view (eg 'every-eye-view 139848.doc 201001177 stream') Month b does not have hardware for outputting the video in a new format (for example, false-twisting achieves stereoscopic vision by 120 fps video synchronized with shuttered glasses), where 60 fps is delivered to each eye, if Consumer video hardware can only support 60 fps video, this option will not be available in the absence of upgraded hardware purchases).

When it comes to cutting-edge interactive media (especially video games), the problem of media device obsolescence and complexity is a serious problem.现代 , , Hyundai video game applications are basically divided into four major non-portable hardware platforms. Sony Playstation® i, 2 and 3 (PS1, PS2, and PS3);

Microsoft Xb〇x (g^Xb〇x 36〇8; and Nintend〇 8 and

Wll Μ, and Pc-based games. Each of these platforms is different from the others, so that games written to be executed on one platform are typically not executed on another platform. There may also be compatibility issues between a generation of devices and next-generation devices. Even though most software game developers build software games designed to be platform-independent, in order to execute specific games on a particular platform, a proprietary software layer (often referred to as the "game development engine") is needed to adapt the game to Used on specific platforms. Each platform is sold to the consumer in the form of a "console" (i.e., a separate box attached to the TV or monitor/speaker) or itself as a pc. Generally, video games are sold on optical media such as Blu-ray DVD, DVD_R〇M4CD_R〇M, which has a video game embodied as a sophisticated instant software application. As family broadband speeds increase, video games are increasingly becoming available for download. 139848.doc 201001177 Due to the immediacy and high computing requirements of high-end video games, the special requirements for achieving platform compatibility with video game software are extremely demanding. For example, 'we may expect full game compatibility from a generation of video games to next-generation video games (eg 'from XBox to XBox 360' or from playstati〇n 2 ("ps2") to Playstation 3 ("PS3")) Sex, as there is a general compatibility of productivity applications (eg, Microsoft Word) from a PC to another PC with a faster processing unit or core. However, this is not the case for video games. Because when a video game is released, video game manufacturers often seek the highest possible performance for a given price point, so the system is often dynamically architected so that many games written for the previous generation are in the next generation. Not working on the system. For example, XBox is based on the x86 family of processors, while the χΒ〇χ 36 is based on the PowerPC family. Techniques can be used to simulate the previous architecture, but it is often impractical to assume the same behavior in the simulation, assuming that the video game is an instant application. This is a loss to consumers, video game console manufacturers and publishers. For the consumer, it means to keep the old = Chengshang console and the new __ generation video game console connected to η in order to be able to play all the games. For the console manufacturer, it means the cost associated with the simulation and slower adoption of the new console: and the business means that it may have to issue multiple versions of the new game to cover all potential consumption. _ is not only released for video games such as ΧΒ ° Χ, Ρ 1 _ version, and often issued; , . . . version of each version of the trademark (for example, PS2 and PS3). Xing 139848.doc -10- 201001177 For example, develop a separate version of the electronic art "Crazy Rugby 08" for XBox, XBox 36〇, ps2, pS3, ~__, and platform and other platforms.

Portable devices such as cellular phones and portable media players also challenge game developers. Increasingly, these devices are connected to the wireless data network and are capable of downloading video games. However, there are a variety of cellular phones and media devices in the market that have many different display resolutions and computing capabilities. Also, because such devices typically have power consumption, cost, and weight constraints, they typically lack high-level graphics acceleration hardware like a graphics processing unit ("GPU") (such as those manufactured by CAiSanta clarai NviDiA). Therefore, game software developers typically develop a given game title that is used for many different types of portable devices. The user can find that: The given game title is not available for its special nested phone or portable media player. In the case of a home game console, hardware platform manufacturers typically ask software game developers for a version of the tax on their ability to publish games on their platform. Honeycomb telephony wireless communication companies often also ask game publishers for royalties to download games to cellular phones. In the case of pc games, there is no royalties paid for publishing games, but due to the high consumer service burden for supporting multiple PC configurations and possible installation problems, game developers often face high costs. . Oh, pcs often have fewer barriers to piracy of game software because they can be easily reprogrammed by technically savvy users and games can be more easily pirated and more easily distributed (9) as 'via the Internet') . Therefore, for software game developers 139848.doc 201001177 and s, there are costs and disadvantages found in game consoles, cellular phones and PCs. For game publishers of consoles and PC software, the cost is not limited to this. In order to distribute the game via the retail channel, the publisher asks the retailer for a wholesale price that is lower than the sale price to give the retailer a profit margin. Publishers must also pay for the cost of manufacturing and distributing physical media for the game. Retailers often ask publishers for price protection fees to cover possible incidental expenses (such as games not being sold, or the price of the game is reduced, or the retailer must refund some or all of the wholesale price and/or recover from the purchaser) game). In addition, retailers often ask publishers for help in selling games in advertising flyers. In addition, retailers are increasingly purchasing back games from users who have completed the game, and then the games are sold as used games, and the revenue of the used games is typically not shared with the game publisher. The following facts add to the cost burden imposed on game publishers: the game is constantly being hacked, and the Internet is pirated and distributed for users to download and make free copies. The Internet broadband speed is increasing and broadband connectivity is becoming more widespread in the United States and around the world (specifically, to the home and to the "Internet cafes" that lease PCs connected to the Internet), games are increasingly being Download and assign to a PC or console. Moreover, the wideband connection is increasingly used to play multiplayer and big eve online games (both of which are referred to in this disclosure by the acronym 3 "MMOG"). These changes mitigate some of the costs and issues associated with retail distribution. Downloading online games solves some of the disadvantages of game publishers' because distribution costs are usually small and there are fewer or no 139848.doc 12· 201001177 There is a cost of unsold media. But the downloaded games are still subject to piracy, and large (often a lot of billions of bytes) and sent: very long time to download. In addition, a plurality of games may be filled with::: a drive such as a portable computer or a video game console. However, insofar as the game or MMQG requires an online connection to make the game playable, the piracy problem is alleviated because the user is often required to have a valid user account. It is different from linear video (such as video and music) that can be recorded by the camera or by recording the microphone from the speaker. Each video tour is simple and cannot be used for simple video/ Audio recording to copy. MOG is also protected from piracy and therefore supports commerce in areas where copyright law is not enforced and piracy is promoted. For example, it has been: deploying Vivendi SA's "World of Warcraft" view (10), and: the whole world ilb is not '= gold version. And many online or MM〇G games (such as Linden II's "Second Life" MM〇G) through the economic model built into the game = income of the game operator, where assets can be sold with online tools with A And even established. -τ/± Build ® This allows you to pay for the use of online games using a mechanism other than the customary stomach or ting. :: Due to the nature of online or MMOG, piracy is often mitigated, and every operator is still facing the remaining challenges. Many games require a large amount of two, ie, “in-home” processing resources for proper work on the line or at the job (10). A user with a low-performance local computer (for example, no coffee: a computer such as a low-end laptop) may not be able to play the game. In addition, as the game console ages, it is far behind the current Technical status 139848.doc -13 201001177 State and may not be able to handle higher order games. Even assuming that the user's local PC can handle the calculation of the game ^... Seeking 'there is often an ampoule complexity.肊 There is a drive incompatibility (for example, ^ ^ ^ ^ , right download new game, then it may shape the drive, which makes it dependent on the old version of the graphics driver... the game is not operational h with the download more games, : system. May use up the local disk space. When it is found that the game has been modified (for example, if it is too difficult to play or too easy to play), the level game developer of the complex game, ^/Current game receives the download. ~ Move away from the new.../ : (patch). These patches require not all users to complete the space consumption problem under all patches. The red version introduces other compatibility or disk two: play During playback, 'large data download may be required to provide graphics or 贝 for Beixun to the local console. For example, if you use one of the rooms in the _ ', and the user is not available on the local machine U line has the information to download the scene or character 〃 'm character' must be... Beca. If the Internet connection is not fast, this can lead to real f delay during game play. In addition 1 Increase more than the local PC Or the console's storage space or computing power = it can be generated in the game's situation - the situation: the user can not: play the performance, or must continue with the quality of the graphics. Therefore, l often limit its storage and / Or computational complexity requirements: Second, 'it often limits the amount of data transfer during the game. MOG games can also narrow the market for users of playable games., 139848.doc • 14- 201001177 Increasingly reverse the engineering of the game's original copy and modify the game to make it cheating. Cheating may be more like repeating button presses that are faster than humans (for example, so that you can shoot quickly) Simple. In games that support asset trading in games^, cheating can lead to fraudulent transactions involving fraudulent transactions involving real estates. When online or MMOG economic models are based on such asset transactions, this can lead to the game. The actual harmful consequences of the operator. ^ The cost of starting a new game with PC and console can make more sophisticated "plays (for example, more aggressive The growth of graphics (such as instant ray tracing) and more realistic behaviors (such as real-time physics simulation). In the early days of the video game industry, video game development was very similar to the development of application software; Most development costs are in the development of software (as opposed to the development of θ汛 and behavioral elements or “assets”), such as: 1 open for their software development for films with a wide range of special effects. Today, many cutting-edge video game development efforts are more like software development than software development. For example, 'many video games = a total of 3 simulations, and produce increasingly real (that is, seemingly The characters, props and environment of the camera-like live action image-like computer graphics. The most challenging aspect of photo-realistic game development is the creation of a computer that cannot be distinguished from the human action. Human face. ^ Obtained techniques (such as 'developed by CA^an ^ 〇 〇: ~ authenticity capture system) capture the precise geometric shape of the performer's face with high resolution (four) 精确 precise geometry when the performer is in motion shape. This technique allows the 3D face in the % or game console, I39848.doc -15· 201001177 3 D face to be virtually indistinguishable from the captured live action face. Accurately capturing and reproducing "photo-realistic" faces is useful in several ways. First, 'highly recognizable celebrities or athletes are often used in video games (often employed at high cost), and imperfections may be obvious to the user', making the viewing experience distracting or unpleasant. Often, high levels of detail are required to achieve a high degree of photorealistic sensibility - potentially the need to reproduce a large number of polygons and high resolution textures (in the case where polygons and/or textures change with the movement of the face on the basis of the frame) . When a high polygon count scene with a fine texture is rapidly changing, the game-enabled pc or game console may not have enough RAM to store enough polygons for the desired number of frames generated in the game segment and Texture data. In addition, a single disc drive or a single-disc drive on a game board or a game console is usually much slower than a ruler, and usually cannot keep up with the maximum data rate that the GPU can accept in reproducing polygons and textures. The pre-game usually carries most of the polygons and textures into the ram, which is largely limited by the amount of RAM and the amount of time given. In the case of, for example, a face-to-face book, this may limit the PC or the game console to no realism, and may suspend the iL in the game or limit it to only one, ', more The polygons and textures of the frame (and 1 of his stock) were preceded by a limited number of eves in the eight. The number of frames is made into a realistic face when the PC or console displays something like "Loading progress bar across the screen slowly moving by the current complex 丄/心 when the viewer is recognized as an inherent disadvantage. Next - Game The use of the % view from the disk (unless there is another article 139848.doc -16 - 201001177 pieces) otherwise in this article "Hone, disk" heart non-volatile optical media or magnetic media, and such as the semi-guided " The delay in loading of a new conductor can be a thousand or even a few minutes. This is a waste of time and can make the game players quite frustrated. As discussed earlier, a large or all delay may be due to the loading time of the 边 之 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 纹理 ' ' ' ' ' ' ' ' ' ' ' ' ' ' / or when the coffee is used in the scene, it takes a part of the loading time. For example, a 5'-style soccer game will be allowed to watch the game. The game allows players to choose among a large number of players, small =, sports field and weather conditions #. Therefore, depending on the choice of the Heteron combination, it may be necessary to use different polygons, textures and other materials (collectively referred to as "objects") for the scene (for example, , and with the pro, the ancients have different color centers and patterns on their uniforms) . It is possible to enumerate many of the various permutations of the pre-computed objects and to compose the objects on the disk for storing the game. Counting the county, the secret is 'there is a large storage, and there are objects. ί: Material 1 can be too large to be installed on the disk (or too unrealistic to download). Therefore, the complexity and playback duration of the existing scene and control 1 are constrained and the =2 scene suffers from long load times. The prior art video game system and application software are limited in that they are used with increased benefits (for example, 2D objects) = (such as polygons and textures), and such large capitals: "Processing." As mentioned above, ; = stored in =, the „„ long time (four) '.', m library is at the remote location and via _ (four) to save 139848.d〇 (201001177 take, then load The time is usually much more severe. In this case, the library may take a few minutes, hours or even days. In addition, it often generates a lot of expenses (four), used for games, movies or historical records. High 3D model of sailing ship) and intended for sale to: end: end user. However, once the database is downloaded to the local end, there is a risk of being pirated. In many cases, the user only wants to estimate the suitability of the user for the user's needs (for example, when the ^ is performed to perform a specific move, whether the milk position for the game character has a thoughtful appearance or appearance) For the purpose and hope to download the database. Long-term man time can be a hindrance to users who estimate the 3D database before deciding to make a purchase. ~ Similar issues arise in MMOG (specifically, games that allow users to take advantage of increasingly customized characters). For a pc that displays a character or a game console, it needs to be able to access a library of 30 geometries (polygons, textures, etc.) and the behavior of the character (eg, if the character has a shield, then the shield is strong enough to deflect the spear). Usually, when mm〇g*—the first time the user plays, the large database for the character is already in the initial copy of the game, 'two available, the initial copy of the game is available on the game disc or downloaded to the magnetic on the local end. dish. However, as the game progresses, if the user encounters a person or object that the database is not available at the local end (for example, if another user has created a custom character), then the user or object must be downloaded before the person or object can be displayed. database. This can result in a substantial delay in the game. Given the sophistication and complexity of video games, another challenge for video game developers and publishers in the case of prior art video game consoles is at 139848.doc •18· 201001177 at: Developing video games often costs 2 year? 3玍, 士士士 Α The owner of the cattle for 3 years, the cost is tens of millions of dollars. Assume that the new video game console platform is introduced at a rate almost every five years. 'The game developers need to start the development of their games in the years before the new game console is released, to the debts of the media> π Make video games available at the same time when a new platform is released. Several consoles from the competitive ping-pong 1 & _ 1 ^ double-five I& quotient are sometimes issued at about the same time (for example, within - or two years of each other), but the system yet to be distinguished The popularity (for example, which console will produce the largest video game software sales). For example, + Niu 1 and 5, in the recent console loop, Microsoft XBox 360, s〇ny (4) Chan (10) 3 and coffee _ are scheduled to be introduced in about the same general time box. But in the years leading up to these introductions, game developers must essentially “press” which console platforms will be more successful than others and invest their development resources accordingly. The film production company must also be different from the Japanese door basin before the film is released. The long-term squad will share its limited production consultations based on its estimated possible success. As the degree of tribute increases, game production becomes more and more similar to film production, and game production companies routinely invest in their production resources based on their assessment of the future success of a particular video game. However, unlike the film company, this bet is not based solely on the success of the production itself; it is based on the success of the game console on which the game is intended to execute. Simultaneously distributing games on multiple consoles can reduce risk', but this extra effort increases costs and often delays the actual release of the game. The process of threading and cutting on the pc is becoming more computationally intensive, dynamic and interactive, not only making it more visually appealing to the user, but also making it more useful and intuitive. For example, the new I-%«乍 139848.doc •19· 201001177 system and the successive versions of the Macintosh® operating system have both visual effects. High-end graphical tools, such as MayaTM from Auto desk, provide very sophisticated 3D rendering and animation capabilities that push the limits of current state of the art CPU and GPU. However, the calculation requirements for these new tools create many practical problems for users and software developers of such products. Because the visual display of the operating system (OS) must work on a variety of computers (including those that are no longer sold but can still be upgraded with the new OS), the OS graphics requirements are largely influenced by the computer that os intends to use ( It usually includes the least common point limitation of computers that do not include GPUs. This severely limits the graphics capabilities of the OS. In addition, battery-powered portable computers (e.g., laptops) limit visual display capabilities because of the high computational activity in the CPU or GPU that typically results in lesser power; less power consumption and shorter battery life. Portable computers typically include software that automatically reduces processor activity to reduce power consumption when the processor is not being utilized. In some computer models, the user can manually reduce processor activity. For example, Sony's VGN-SZ280P laptop has a side labeled "Stamina" on one side (for low performance, longer battery life) and the other side is labeled "Speed" (for high performance, Switch for shorter battery life). An OS executing on a portable computer must be able to function even when the computer is executed at one of its peak performance capabilities. As a result, OS graphics performance often remains available at much lower computing power than current state of the art. High-end, computationally intensive applications such as Maya are often sold, and it is expected that such applications will be used on high-performance PCs. This usually results in much higher efficiency, and is more expensive and less portable, with the least commonality required. As a result, these applications have a much lower target audience than Universal OS (or Universal Productivity Applications, similar to Microsoft 〇ffice) and are typically sold in much lower quantities than generic OS software or general purpose application software. Potential audiences are further limited because it is often difficult for prospective users to try out = insidious applications in advance. For example, suppose students want to know how to use Maya or know that potential buyers of such apps are buying

I hope to try aya before investing (this may involve buying a high-end computer that can execute Maya). When a student or potential purchaser can download a demo version or receive a physical media copy of the Maya demo version, if it lacks a computer capable of performing Maya to its full potential (for example, handling complex buckle scenarios), it will not be able to make the product. Comprehensive evaluation. This essentially limits the audience for these high-end applications. #Also makes the selling price higher, as development costs are usually paid off through purchases that are much smaller than the number of purchases by the generic application. High-priced apps also generate more incentives for individuals and businesses that use pirated copies of the application software. As a result, 'high-end application software has been pirated.' Although publishers of the software have made a lot of efforts to mitigate the piracy. But even when using pirated high-end applications, it is impossible to rule out the need to invest in expensive versions of the current state of the art. Therefore, although the user can obtain the software application using the actual retail price of the software application, the user of the golden software still needs to purchase or obtain an expensive PC in order to fully utilize the application. B9848.doc -21 - 201001177 This is also true for users of high-performance pirated video games. Although pirates can get the game at a fraction of the actual price of the game, they still need to purchase the expensive computing hardware needed to play the game properly (for example, GPU_Enhanced PC' or a high-end video game console like ΧΒox 360 ^ Assume that video games are typically entertainment for consumers, and the extra cost for high-end video game systems can be prohibitively expensive. This situation is worse in countries where the average annual income of current workers is relatively low (relative to the average annual income of current workers in the United States) (for example, China). As a result, a much smaller percentage of the population has high-end video game systems or high-end PCs. In these countries, it is quite common for users to pay for the use of computers connected to the Internet. Often, such cyberspaces have older models or low-end PCs that do not have high performance features, such as GPUs that would otherwise enable players to play computationally intensive video games. This is at the low end! ^ Key factors for the success of the game on execution. For example, Vivendi's "World of Warcraft" is highly successful in China and is usually "playing in China's network". It is even less likely that the games that are intensive (such as "Second Life") are less likely to be played on PCs installed in Chinese Internet cafes. These games do not actually reach users who are only able to access the low performance % of Internet cafes. ― For those who consider purchasing a video game and first want to try out the demo version of the game by downloading it to their home via the Internet (4). Video game demos are often a versatile version of the game, where - some are disabled or impose restrictions on the amount of game play. This may involve a long process (possibly several hours) of loading billions of 70 sets of data before the gameable women's on a PC or console and on a % or console. In Pc 139848.doc -22- 201001177 • / brother, it may also involve calculating which special drivers the game needs (for example, this (10) X or 〇PenGL driver), download the correct version, install the correct 'definite version' And then it is determined whether the PC can play the game. The latter step may involve determining if the PC has sufficient processing (cpu and GPU) capabilities, sufficient RAM, and compatible 旳 (for example, some games are executed on Windows XP and not on Vista). lL ,, _ 凡 仃) Therefore, after trying to perform the video game demonstration process, you can use _ I w to find that the video game demo can not be C; can play (given the user's PC configuration). Worse, once the user has downloaded a new drive to try the demo, the version of the drive may be incompatible with other games or applications that the user is accustomed to using on the PC, so the installation of the demo may result in the previous The game or application being operated cannot be operated. These obstacles not only frustrated users, but also created obstacles to the sale of video game software publishers and video game developers. Another problem that leads to non-economic benefits is related to the fact that a given pc or game console is usually designed to fit the application and/or game (a certain degree of performance... for example, one (four) has more or less The RAM, the slower or faster cpu and the slower or faster Gp are called if they have 2 coffee. Some games or applications take advantage of the full computing power of a given PC or console, while some games or applications do not take advantage of the full computing power of a given PC or console. If the user's game or application selection does not meet the peak performance capabilities of the local PC or console, the user may be wasting money on the PC or console due to unused features. In the case of the console, the console manufacturer may pay 139848.doc -23· 201001177 for the cost of the console. It is the other part of the video game and enjoys the other. The problem involves the user purchasing the game. Previously allowed users to watch others play games. There are several prior art methods for replaying at a later time. °, National Patent No. 5,558,339 teaches that during the "playback period, the game status information (including "playing user's electric target action" is recorded in the video game can be used at a later time to be owned by Γ). This status information console) is on the basis of this view; ^game client computer (for example, 在于 lies in: for viewing the recorded game = this = significant disadvantages can play the game's view ^ user must have r ... *Playing the client computer and having to have the video game application in the power of the 7th, in order to make the game play the game, etc. _. The play that has been recorded has to be recorded in the _ / / In addition to this, the video game application - the difference in the way of writing is written in this way. It is worthy of the fact that the large-scale graphics system on the basis of the frame is based on the frame. For many games, depending on the θ foot Other delays (such as = special: complex or whether there is a mitigation line, so that the CPU from the game application to take the process = the process may be able to spend more than a gentleman's clothing 'clothes) 'game logic sometimes available - The cost is shorter than the frame time or the figure displayed in a frame below the frame time. In this game, the second H is a little less time (for example, when several cpu days ^ frame) The value box will eventually appear. When using the cypress door "The time limit for calculating the same - the scene, may accommodate two or two:: state information and then cp 亀 cycle time (for example, if the inner = two box time more than a u bus is slightly external with 139848.doc -24- 201001177 The dram bus is out of phase, and there is no other process from the game processing to take a few seconds of CPU time.

The y spoon is too late, which also introduces a delay of several CPU cycles. Therefore, when the game is held, the frame becomes calculated in two frame times instead of a single one. Some behaviors are based on the frequency at which the game calculates new frames (for example, when the game samples from the input of the game controller). When playing a game, this deviation in the time reference for different behaviors does not affect the game play, but it can lead to the playback of the game.

The play produces different results. For example, t, ϊ+,> 士° The right A-ball track is calculated at a steady 60 fps rate, but the game controller input is sampled based on the calculated frame rate, then when the game is recorded At the time, the calculated frame rate may be 53 fps'. When the game is played back, the calculated frame rate may be W, which may cause the basketball to be blocked from entering the difference in t, resulting in a difference. result. Therefore, #using the game state to record video games requires a very cautious game software design, 'M隹 uses the same game state information playback to produce exactly the same results. ^ The technical method for recording video is to record only the video output of the PC or video game system (for example, to the VCR' dvd recorder, or to the video capture board p on the PC, then the video can be rewinded and replayed. Or, alternatively, uploading the recorded video to the Internet (usually after video is pressed, afterwards). The disadvantage of this method is that the user is limited to self-examination only when replaying the game sequence (sequence (4) The sequence is viewed from its record. In other words, the user cannot change the view point of the scene. In addition, the compressed video of the recorded game sequence played on the home PC or game console via the Internet is other When the user can use 139848.doc -25· 201001177, even if the video system is compressed immediately, the compressed video can be uploaded to the Internet in real time. The reason for the field is because the world is connected to the Internet. Many homes in the network have a high (four) called wideband connection (for example, cable and cable modems typically have a much lower downlink bandwidth than the upload bandwidth). Compressed high resolution video sequence Often having a bandwidth that is higher than the upload bandwidth of the network' makes it impossible to upload instantly... after playing the game sequence (possibly a few minutes or even hours), another user on the Internet can view the There will be a significant delay before the game. Although this delay can be tolerated in certain situations (such as 'watching the outcome of a gamer who appeared at a previous time), it eliminates the live broadcast of the watch game (eg, a basketball tournament played by a winning player) Ability to play or "live replay" when playing a game live. Another technology method allows TV viewers to watch live video games, but only under the control of TV producers. Some of the national television channels provide visual inspection video channels, in which TV H can view specific video game users on the video game channel (for example, participate in the top players of the tournament). This will be through the video game system (PC and / or console) video output feed to the video segment for the TV channel: and processing device to complete. For example, when a TV channel broadcasts a live basketball game, several cameras provide on-the-spot money feeds from different angles around the basketball court. The TV channel can then use its video/audio processing and effects devices to manipulate various video game systems. For example, the TV channel can overwrite the text indicating the status of different players on the video from the video game (as it can be overwritten during the live broadcast of the basket 139848.doc -26- 201001177 during the ball game). It can be discussed that in the game sn video, you can add the audio from the commentator (the action that appeared during the game). In addition, the game output fish Ji Zanyou> & 、, outside, you can view the video... The visual response of the player to the game is a combination. Machine (for example, 'The exhibitor goes there - the problem is: must immediately make the news feed as a mine; S phase, bitter 4, a 4 ' see the live broadcast of the eye I The video distribution and processing equipment can be used with the right side of the scene, with the use of 'to make it video tour # However, as discussed, when

The system is from the time of family execution (especially when it comes to video from the real world of the game player, the broadcast of the video broadcast), which is often the scene of the camera. You can modify the game and make a description of it. For these reasons, the video broadcast on the TV channel is often skillful, = in the public location (for example, in the TV studio or in the competition) player and video game system, where TV production The device can be used as a video game system and a potential live broadcast camera: these prior art video game TV channels can provide a stimuli for television viewing (which is similar to live sports events (eg, The experience of video game players in the same way as sports games is based not only on their actions in the video game world, but also on their actions in the real world.] But these video game systems are often limited to players. The entities are very close to each other's (four). In addition, because the television channel is broadcast, each broadcast channel can only display a video stream selected by the producer of the television channel. Due to these restrictions and broadcast time, production equipment and production staff 139848.doc -27. 201001177 South Cost' player. These TV channels usually only showcase the top level of the top-level tournaments. In addition, the TV channels that broadcast the full-screen video game to the TV viewers will only display one video game at a time. This severely limits the choice of TV viewers. For example, a TV viewer may not be interested in a game at a given time. Another viewer may only be interested in watching a particular player's game play that is not shown by the video channel at a given time. In other situations, the viewer may only be interested in seeing how the player is dealing with a particular level of the game. Other viewers may wish to control the view point (from which the video game is viewed), which is different from the view point selected by the production team or the like. In short, 'TV viewers may have 4 innumerable preferences in watching video games (even if several different TV channels are available, the specific broadcast of the TV network should not be equal to the bias. For all of the above reasons, the prior art video game TV channel There are significant limitations in presenting video games to TV viewers. Another of the four systems of video technology and application software systems is that it is complex and often suffers from errors, crashes, and/or unwanted needs. Behavior (collectively referred to as "defects"). Although games and applications are in the test: they usually go through the process of debugging and tuning (often referred to as "software, beta brother" or SQA), but they are almost not a.疋· Once the game or application 3

When it is released to the vast majority of the field, the defects will suddenly appear. It is difficult for software developers to identify and track many missing vehicles after the release of the car. It may be difficult for human developers to be confused. Even if it understands that there is a lack of τ ', there may be only information that can be used to scream and scream what is wrong with J 39848.doc -28- 201001177. For example, the fee-based service hotline and the user can call the game developer to cancel the message. The message states that when the game is played, the screen starts to flash, and then the micro-j_, the connection, the solid blue and the PC freeze. It provides very little information for the SQA team k to be useful in tracking defective towels. One of the online connections Some games or applications are available in the 胪~ 埂 之 在 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时 有时For example, sometimes you can monitor the game or application by using the "A... Watchdog" process. The watchdog process collects statistics about the game or application when the game or application crashes, 'the privacy of the game (for example, the memory stack 2 =, the extent to which the game or application progresses, etc.), and the Internet and uploading his information to the game or application, 兮次二+ [in the noisy ^bei 5 hole can be used for a very long time to decrypt, 乂 准确 准确 准确 准确 准确 准确 准确 准确 准确 准确 准确 准确 准确 准确What to order. Although it is impossible to determine what sequence of events caused the crash. The problem with fc and the game console is that it suffers from the problem of inconvenience. Service issues are also bully or game control makers' because they usually need to send special boxes to safely ship damaged PCs or consoles, and cut islands A, and thus incur the cost of repairs (if PC or control. Under warranty period). The game or application software publisher may also be affected by the loss of sales (or usage) caused by the PC and/or the system in the repair state. Figure 1 illustrates such as s〇ny Play Can. Η8 3, Micr〇s〇ft xb〇x 3 60®, N her nd0 WiiTM, Lai (10)~ based PC or Apple MaeintGsh's prior art video game system. Each of these systems includes a central processing unit (cpu) for executing code (usually 139848.doc -29-201001177 graphics processing unit (GPlJ) for performing high-level graphics operations), and for Multiple forms of input/output (1/〇) for external devices and user communication. For the sake of simplicity, these components are shown as being combined into a single unit 100. The prior art video game system of FIG. 1 is also shown to include an optical media drive 104 (eg, a DVD_ROM drive); a hard drive 103 for storing video game code and data; for playing a multiplayer game, for downloading a game, Patch, demo or other media network connection 1; random access memory for storing code currently being executed by CPU/GPU 100 (RANDUH; used to receive input commands from the user during game play) Game controller 106; and display device 1〇2 (eg, sdtv/HDTV or computer monitor). The prior art system shown in Figure 1 suffers from several limitations. First, compared to the access speed of RAM 101, Has a much slower access speed. When due to RAM 1 01 且 且 古 古 “ “ 光 光 光 光 光 光 及 及 及 及

The number of polygons per second that can be much more per second. But only a limited amount

Ram provides these prior art systems ten (the female group). Therefore, it is often necessary to "loading..." sequence is periodically filled in for video games.

(for example, 256-5 12 dead bits) sequence, where RAM 1〇1 data for a scene. Load and play with the game, dry, six, and happy, but for $ (for example, if along the road The geometry of the building I39848.doc •30- 201001177 is approaching the road. For complex and/or fast scene changes, this type of overlap usually does not work. For example, when the user is in the campaign In the case where the RAM 101 is completely filled with the information indicating the object in the view of the time, if the user quickly moves the view to the left to view the object not currently loaded in the RAM 101, the action will be caused. Discontinuity, because there is not enough time to load new objects from the hard disk drive 1 〇 3 or the optical drive 1 〇 4 into the RAM 101.

Another problem with the system of Figure 1 is due to the limitations of the storage capacity of the hard disk drive 1 and optical media 1. Although a disk storage device can be fabricated to have a relatively large storage capacity (eg, 50 billion bytes or more than 500 million bytes), it does not provide the specifics encountered in current video games. Sufficient storage capacity for the situation. For example, as previously described, a British football video game may allow a user to select among many groups, players, and sports fields around the world. For each group, each player, and each stadium, a large amount of texture mapping and environment mapping is required to characterize the world's 3D surface (eg, a small group with a single jersey, each requiring a unique texture mapping). The problem of the stupid person - the technical department: for the game, the user selects the texture and environment mapping' to calculate the texture in advance: this can involve many calculations on the π π π 映射 映射 ... & & & & & & & & & & View ^ v 0± , ^ mouth this, § video game to perform this sweating different day f, there is a delay for the user u, Ding. To reduce this delay, the old law is in principle: the initial development of the sneak sneak peeks to play all of these groups, the player roster and the sports field $ main ΐ -· - including the release version of the small techno-H game thus I39848.doc • 31 - 201001177 will include all of this pre-processed material stored on optical media 1〇4 or on one or more servers on the Internet, when the user chooses: will only be used for the given via the Internet The selected pre-processed materials selected by the team, the player roster, and the playing field are downloaded to the hard disk drive.每 And as a practical matter, the data of each of the possible permutations in the game play may easily be the data of the terabyte, which is the capacity of the current optical media device. In addition, the information used for a given roster or sports field selection may easily be hundreds of millions of bytes or hundreds of millions of bytes. In the case of a home network connection (: column 2 Mbps), downloading this data via the Internet connection 1〇5 will take longer than the data. Therefore, the prior art game architecture not shown in the ® 1 allows the user to experience significant delays between large scene transitions of the game. Another problem with prior art methods such as the prior art method shown in Figure 1 is that video games tend to become higher order and more (10) coffee processing capabilities over the years. Therefore, even with unlimited, the video game hardware requirements exceed the peak level of processing power available in these systems. Therefore, it is necessary for the user to upgrade the game hardware several years to keep it in sync (or play newer games with a lower level of clarity and a lower mouth-to-mouth level). The trend of higher-order video games than before - the result is that the game for video games for home use | § usually not mother, thick 4 2 /,, work efficiency, because its cost is usually It can be judged by the requirements of the highest performance game that can be supported. For example, it is possible to use 36〇 to play a similar "War Machine Coffee". The game requires high performance CPu, Gpu and hundreds of millions, I39848.doc -32· 201001177. People may use xb〇x 360 to play Pacman. (PacMan), which is a game from the age of 2, which requires only a few thousand bytes of ram and a very inefficient cpu. In fact, ΧΒοχ 36〇 has enough computing power to host many of the same “Pacman” games at the same time. In most hours of the week, the video game console is usually cut off. According to a study of active gamers aged 13 and over in July, on average, active players spend 14 hours a week or only 12% of all hours of the week to play consoles. Video games] means that the average viewing game console is interspersed 88% of the time, which is an inefficient use of expensive resources. It is particularly meaningful to assume that the video game console is often funded by the manufacturer to lower the purchase price (which is expected to be earned by royalties from future video game software purchases). The video game console also incurs a link to almost any consumer electronics device. For example, the electronics and mechanisms of the system need to be housed in a foreign language. Manufacturers need to provide service guarantees. The sellers who sell the system are required to receive sales of the system and/or about the video game software: the profit from the sale. All of these factors add to the cost of the video game console, and the cost must be funded by the manufacturer, passed to the consumer, or funded by both _ and consumers. In addition: piracy is a big problem in the video game industry. In fact, the security agencies used on each of the larger video game systems have been "broken" over the years, resulting in unauthorized copying of video games. For example, the 360 security system ruptured in July of the next six years and users can now download X illegal copies online. Downloadable games (for example, games for PC or ]Viac) I39848.doc -33· 201001177 Don't be vulnerable to piracy. In a specific region of the world (where piracy is not strong), there is no viable market for independent video game software on Beibei, because users can easily buy pirated copies at a fraction of the cost with legitimate copies. Moreover, in many parts of the world, the cost of the game console is a high percentage of revenue, so that even if piracy is controlled, few people can afford the game system of the current state of the art. In addition, the market for used games has reduced the revenue of the video game industry. When the user becomes bored with the game, they can send the game out to the store where the game is resold to other users. This unauthorized but widespread practice significantly reduces the revenue of game publishers. Similarly, when there is a platform change every few years, typically about 50% of sales are reduced. This is due to the fact that when the user knows that a newer version will be released (4), the user stops buying the game for the older platform (for example, when the playstati〇n 3 is about to be released, the user stops purchasing the Playstation 2 game). In combination, the loss of sales and the increased development costs associated with the new platform can have a significant adverse effect on the profitability of game developers. The new game console is also very expensive. Xb〇x 36〇, Nintend〇 and –PlayStat_ 3 are all retailed for hundreds of dollars. A high-capacity PC can cost up to $8000. This represents a significant investment by the user, and the rumor 'considered the fact that the hardware became obsolete in a few years and many (4) were purchased for the child. The above-mentioned method is an online game in which the game code and the machine are hosted on the server and delivered to the heart machine as required, and the compressed video and audio are streamed via the digital broadband network. . Some 139848.doc -34- 201001177 divisions (such as G-Cluster of FhUand, which is now a subsidiary of Sakamoto's SOFTBANK Broadmedia) currently offer such services online. Similar gaming services become available in local networks such as hotels and their networks provided by cable TV providers. The larger drawback of such systems is the problem of latency, i.e., the time it takes for the signal to travel to the server and • travel from the game server, which is typically located at the "head end" of the operator. A fast action video game (also known as a "twitch" video game) requires very low material between the time the user makes the game (4) and the time when the user displays the screen to show the result of the user's action. . When f asks for money, it will satisfy the user at different delay intervals by making money feel that the game responds "immediately" to the type of visual game and the user's proficiency. For example, for slow: : Official game (similar to backgammon) or slow action role-playing game: ' _ millisecond delay may be tolerable, but in fast action game 'more than 7 milliseconds or 8G milliseconds The delay can cause the user to behave in the game = 1 and therefore is unacceptable. For example, in a game that requires fast response... when the delay increases from 5〇 milliseconds to (10) the sharp drop in accuracy. There is a controlled network environment when the game or application server is installed in the vicinity or the network path is predictable and/or can tolerate the peak of the bandwidth. In the case of large delays and delays, the control delay Easy: 夕 (example wide) so the user observes from the digital view via the network

^ meal, scheduled exercise). The degree of control can be reached between the following heads of the network to Fanpeng π Ganer, , see TV, between TV family, or from DSL center 139848.doc -35- 201001177 to DSL users. Or in a commercial office from a feeding device or user, in a local area network (LAN) environment. Alas, it is possible to obtain a point-to-point private connection between the businesses with a bandwidth that is paid to the security and a specific rating of 14 o'clock. However, in the game or application system in which the game console is hosted in a feeder center connected to the universal Internet and then the video is streamed to the user via a broadband connection, many factors cause delays. This has led to serious limitations in the deployment of prior art systems. In a typical broadband wideband family, the latter can have (4) DSL or electrical data modems for broadband services. These broadband services typically result in up to _ second round trips between the home of the ^ and the general Internet (with eves). In addition, there is a round trip delay due to the routing of resources through the Internet. When the L is based on the Internet, the route based on the material and the data are delayed due to the delay in the data being routed. In addition to the routing delay, it also causes the fine travel (4) due to the speed of the light passing through the domain of most of the Internet. For example, for every 1000 miles, the speed of the trip, the speed of the first pass through the fiber, and other round trip delays of about 22 milliseconds. The additional delay can be sold by the data rate of the data streamed via the Internet. The user will likely get at most less than 5 Mbps under the line and will likely periodically See the degradation of the connection due to factors such as congestion at the peak load n, (10) LAM). A similar problem can arise if (4) ^ : the local shared coaxial cable is wide/ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Thus, the data rate of the cable modem used for the connection sold by the "6 Mbps Cable Data Server Service" is reduced to much less than the data rate. If the data packet at a stable rate of 4 Mbps is streamed unidirectionally from the server center via the connection in a User Datagram Protocol (UDp) format, if the data is properly operated, the data packet will pass. There is no additional delay, but if there is congestion (or other obstruction) and only 3.5 Mbps can be used to stream data to the user, in a typical situation, the packet will be discarded, resulting in lost data, or the packet will be in a crowded point. The queue is queued until it can be sent, thereby introducing an extra = hour. *The same __ has a different poem to save the late packet into the queue capacity 'so in some cases' immediately discards the packet that cannot be successfully resolved. In other cases, millions of bits of data are queued: they are eventually sent. But in almost all cases, the queue at the point of congestion has a capacity limit' and once it exceeds them, the packet will be ". Therefore, in order to avoid incurring extra delays == I am missing, I must avoid exceeding the data rate capacity from the game or application server to the user. Second, due to the time required to compress the video in the server and decompress the video in the client device, the delay D # is performed on the server. The video game being executed is calculating the next frame to be displayed, which further incurs delay. The video compression algorithm currently used by I suffers from high data rates or high latency. For example, the motion PEG is a compression algorithm with only a loss in the frame, which is characterized by a low delay. The #: frame is compressed independently of each other frame of the video. Although the client device receives the frame of the motion video that has been reduced, its I39848.doc -37-201001177 can immediately decompress the frame and display the frame, resulting in very low latency. However, because each frame is compressed separately, the algorithm cannot take advantage of the similarity between successive frames, and therefore only the in-frame video compression algorithm suffers from very high data rates. For example, 6〇 fps (frames per second) 64〇χ480 motion JPEG video may require more than 4 Mbps (megabits per second) or 40 Mbps (megabits per second). These high data rates for such low resolution video windows will be too expensive in many broadband applications (and indeed for most consumer Internet-based applications). In addition, because each frame is independently compressed, artifacts in the frame that may be generated due to lossy compression may appear at different locations in successive frames. This can result in a visual artifact that appears to the viewer as moving when the video is decompressed. Save their changes as: r B" frame (the other party's frame (for changes before the I frame) and

Other compression variants (such as MPEG2 'H.264 or VC9 from Microsoft Corporation) can achieve high compression ratios when used in prior art configurations, but at the expense of high latency. The algorithms utilize inter-frame compression and intra-frame C, and sink periodicity, and the algorithms perform only in-frame compression of the frame. This type of frame is called a key frame (commonly called a "〗" frame). Then, this derivation usually compares the I frame with the previous frame and the subsequent frame. Instead of compressing the previous frame and the successive frames independently, the algorithm determines how the image changes from the I frame to the previous frame and the subsequent frame, and then transmits it at a given data rate, 139848.doc -38· 201001177 It takes a long time in proportion. Consider, for example, a situation where the I frame is 10 times the size of the B frame and the p frame 'and for each single 1 frame, there is a ^ frame + 3 (^ solid? frame = 59 Between the frames, or for each "frame group" (G〇p) a total of 6 frames. Therefore 'at 60 fPs, there is one 60 frame GOP per second. Suppose the transmission channel has a maximum of 2 Mbps. Data rate. In order to achieve the most south quality video in the channel, the compression algorithm will generate a 2 Mbps data stream, given the above ratio 'this will result in 2 million bits per frame (Mb) / (59 + 10) = 30, 394 bits and 303, 935 bits per I frame. When receiving a compressed video stream by a decompression algorithm 'in order to stably play video, it needs to be decompressed and displayed at regular intervals (eg '60 fps) In order to achieve this result, 'If any frame is subjected to transmission delay, then all frames need to be delayed by at least one delay' so the worst case frame delay will define the delay for each video frame. I frame is the largest, so the I frame introduces the longest transmission delay' and the entire 1 frame will have to Decompress and display the I frame (or between any frames depending on the 1 frame) before receiving. Assuming the channel data rate is 2 Mbps' then transmitting an I frame will cost 3〇3,935/2 Mb=145 milliseconds. The inter-frame video compression system (as described above), which is a large percentage of the bandwidth of the channel, will be delayed by the length of the frame due to the large size of the frame relative to the average size of the frame. Or, in other words, when the prior art is compressed between frames When the algorithm achieves a lower average data rate per frame than the in-frame compression algorithm (eg, 2 MbPs vs. 40 Mbps), it still suffers from high peak per frame data rate due to the large I frame (eg '303,935* 60 = 18.2 Mbps). However, it has been lived. The above analysis assumes that both the p-frame and the B-frame are much smaller than the I-frame. 139848.doc -39- 201001177 Although this is generally true 'but for the previous frame In the case of high motion or scene changes, the frame of high image complexity is irrelevant. In these cases, the 'P frame or B frame can become larger than the I frame (if P frame or B picture) The box becomes larger than the I frame, then the tip compression algorithm will usually be "strong" I frame and replace the P or B frame with an I frame. Therefore, the peak data rate of the I frame size can appear in the digital video stream at any time. Therefore, for compressed video, when When the average video data rate is close to the data rate capacity of the transmission channel (often for this situation, given the high data rate requirements for video), the high peak data rate from the I frame or the large P or B frame results in rl Frame delay. Of course, the above discussion only characterizes the compression algorithm delay generated by the large B frame, p frame or I frame in G〇p. If B frame is used, the delay will be higher. The reason for this is because all B frames and I frames after the B frame must be received before the B frame can be displayed. Therefore, in a picture group (GOP) sequence such as BBBBBIPPPPPBBBBBIPPPPP, there are 5 B frames in front of each I frame, which can only be displayed by the video decompressor after receiving the subsequent 8 frames and I frames. The first B frame. Therefore, if the video is streamed at 60 fps (ie, 16.67 milliseconds/frame), before the first B frame can be decompressed, no matter how fast the channel bandwidth is, five B frames and frames are received. It will take 16.67 * 6 = 1 〇〇 milliseconds ' and this is only 5 5 frames. A compressed video sequence with 3 frames of B is quite common. In addition, at a low channel bandwidth such as 2 Mbps, the delay effect due to the size of the j frame is largely added to the delay caused by waiting for the beta frame to arrive. So on the 2 Mbps channel, in a lot of]6 139848.doc -40- 201001177

In the case of frames, it is quite easy to use a prior art video compression technique with a delay of more than 5 milliseconds or more than 500 milliseconds. If the b-frame is not used (for the given quality level, at the expense of a lower compression ratio), the 8 frame delay is not incurred, but the delay due to the peak frame size described above is still incurred. The problem is exacerbated by the nature of many video games. The video compression algorithm utilizing the G〇P structure described above is largely optimized for use in conjunction with live video or movie material intended for passive viewing. Usually, the camera (real camera, or the virtual camera generated by the computer, the situation is relatively virtual) and the scene is relatively stable, because if the camera: the scene moves too far and bumpy, then the video or movie material (4) is usually viewed: Unpleasant, and (b) if it is being watched, when the camera suddenly squats back and forth, 'the viewer is usually unable to follow the action closely (4), if the camera is disturbed when shooting a child who is smouldering on the oyster cake And suddenly bumping back and forth between the cakes, the viewers usually focus on the children and the cake, ^ ignore the brief interruption when the camera suddenly moves). In the case of video conferencing or video conferencing, the camera can be held in a fixed position and moved by 1 at all, resulting in very few data spikes. However, the high-definition action video game is characterized by the definite movement (for example, considering the deduction race, the entire frame is in rapid motion during the duration of the competition, or considering the first-person shooter game, where the virtual camera is strangely Back and forth: These video games can produce a sequence of frames with large and frequent peaks, where the user may want to hear the sound, and see what happens during their sudden movements. In the 3D high-motion video game, the compression artifact I39848.doc •41 - 201001177 2 ί' the video of many video games _ due to its nature, the 玍 has a non-existent and frequent 介一 ^ only the peak of the compression Video streaming. False 疋 fast motion video game degree, and given all the above delay reasons sub-time delay has a small tolerance network streaming on the server " see. Hole in, .. 罔冰 4 + generation s The limitations of the video game. Another network: ... interactive application is hosted on the universal Internet:: and the video is ", then the users of these applications are: === network configuration required ,its Hosting the feeder directly: placed under (10) the condition of the broadband (in the case of broadband) or the central subscriber line (4), or in the commercial context = private connection of the rating, in order to control from the user device to ::: route and distance to minimize delay and adapt to peak without incurring delay Γ f (usually at (10) Mbps_1 Gbps) and lease with sufficient bandwidth _ one capacity = 71 one-value bandwidth for right special adaptation The peak feng feng feng _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ For example, on an electrical environment τν system, a dedicated bandwidth can be given for digital video traffic. The dedicated bandwidth can be processed such as large! The peak of the frame. In addition, on a DSL system, a higher speed off-data processor (considering high peaks) can be supplied or a specially graded connection that can handle higher data rates can be supplied. However, conventional electronic data processors and DSL infrastructure attached to the universal Internet are far from tolerant for peak bandwidth requirements for compressed video. Therefore, the online service (hosting the video game or application host in the server center long distance from the user device, and 139848.doc -42- 201001177 is connected via the conventional residential broadband connection video output Streaming) suffers from frequent delays and peaks and turns to reduce the peak bandwidth requirement of those who need to be compressed - especially for shooting games and other multi-users, interactive μ/ (eg 'di-person, person, User interactive action game, or "application of time" is required. [Embodiment] The present disclosure will be more completely understood from the following [embodiment] and the accompanying drawings, and 'it should not be used to disclose The subject matter is limited to the examples shown, and is only for the purposes of illustration and understanding. The specific details (such as device type, system group, communication method, etc.) are described in the following description in order to provide the pin-to-, ', The apostle provides a thorough understanding of the disclosure. However, those skilled in the art should understand that the specific examples described in the practice may not require such specific details. Λ Figures 2a to 2b provide two embodiments. The high-end architecture in which the video game and software applications are hosted by the colocation service 21 and hosted by the user site 211 under the subscription service (note that the "user site" means where the user is located. The location, if the mobile device is used, includes the outdoor device device 205 is accessed via the Internet 2〇6 (or other public or private network). The client device 2〇5 may be a general-purpose computer having an internal or external display device (2) with a wired or wireless connection to the Internet (such as a Microsoft Windows 4 Unux-based pc: Apple's Macint〇sh computer)' or It can be a dedicated client device (such as a wired or wireless connection to the Internet) that outputs video and audio to a monitor or television 222, such as a set-top box, or it can be speculated to be 139848.doc -43-201001177 A mobile device with a wireless connection to the Internet. Any of these devices may have its own user wheeling device (eg, keyboard, button, touch screen, track pad or ininei-sensing wand, video capture camera and / or motion tracking camera, etc., or it can use an external input device 221 that is connected by wire or wirelessly (eg, keyboard, mouse, game controller, inertial sensor, video capture camera, and/or motion tracking) Camera, etc.). As described in more detail below, the colocation service 210 includes servers of various performance levels (including those with Agilent CPU/GPU processing capabilities). During playback of the game or use of the application on the colocation service 210, the home or office client device 205 receives keyboard and/or controller input from the user, and then transmits the controller input to the host via the internet. The escrow service 210, the colocation service 21, in response to this, executes the game code and generates successive frames for the video output (video image sequence) of the game or application software (eg, if the user presses, the guide will be directed) The button on the screen that moves to the right, the game program will then produce a sequence of video images showing the person moving to (four). The purely delayed video compressor then compresses the video image sequence, and the colocation service 21 then transmits the low latency video stream over the Internet 206. The home or office client device decodes the compressed video stream and reproduces the decompressed video image on the dish or TV. Therefore, the calculation and graphics hardware requirements of the client device 2〇5 are significantly reduced. The client 205 only needs to have the processing capability for forwarding the keyboard/controller input to the Internet 206 and decoding and decompressing the compressed video stream received from the Internet 2〇6, actually 139848 .doc -44 - 201001177 Any personal computer can do this in software on its CPU (for example, Intel & Dual Core cpu running at about 2 GHz can decompress compressors using H.264 and Windows Media να 72〇ρ Η〇τν). In addition, in the case of any client device, the dedicated chip can also be executed for these standards in real time at a much lower cost than the general purpose CPU and much less power consumption than a general purpose CPU, such as is required for modern PCs. Video decompression. Notably 'in order to perform the function of transferring the controller to input and decompress the video', the home client device 2G5 does not require any special graphics processing unit (GPU), CD player or hard disk drive (such as shown in Figure i). Demonstrated prior art video game system). As games and software become more complex and more realistic, they will require higher performance CPUs, GPUs, more RAM, and larger and faster disks. The computing power of the A-tube service is constantly upgrading, but the end-user will not need to upgrade the home or office=client platform 205' because the home or office client platform will be enabled by a given video decompression algorithm. The processing requirements of 205 remain constant for display resolution and frame rate. Therefore, there are no hardware limitations and compatibility issues seen today in the systems illustrated in Figures 2a through 2b. In addition, because the game and application software is only executed in the server in the colocation service (10), in the user's home or office (unless otherwise qualified, the "office" as used in this article will include There is never a copy of the game or application rights in the background of the house, including (for example) classrooms (in the form of optical media) or for downloaded software, which significantly reduces the illegal copying (piracy) of games or applications. It may be 139848.doc 201001177 sex, and the possibility of mitigating the piracy of the valuable sub-library that can be used by the game or the application software. It is very expensive for the server to be specialized. Responsible, large or noisy equipment) to play games or application software that is not feasible for 豕j or office use, even if you get the pirate copy of the application software, it will not be in the court or office. operating. In the case of Λ , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 〇 - To enable the waiter to design a game that can be hosted on the host. These tools allow developers to take advantage of the features of colocation services (these features are usually not available in a standalone PC or game console) (for example! Idle speed access to complex geometries is very large, otherwise otherwise " Geometry ^ ^ (unless the 3D dataset's polygons, lines and / generations - and parameters)). So ... Ming '仃 and other components under this architecture 'different business models are possible. Under a model, the host generation The service 21 is from the terminal 佶田土... and the user charges the subscription fee and pays the royalty to the developer 2 2 ' as shown in Figure 2. In the alternative, the developer 220 directly smashes the field to spear 10... The basic principle of charging the user for the subscription fee and paying for the host to host the game or the application is not limited to the use of the general, edge (or online game or application host generation). Any specific business model of g. Compressed video features 139848.doc -46 - 201001177=First: One of the major problems with video games or application software services on the n-line is the delay. 7 Λ -^ ^ 毛心-8 〇 The delay of seconds (when the input benefit is actuated by the user until the response is displayed on the display device, W is the upper limit for games and applications that require fast response time. However, 'due to a large number of actual and physical constraints This is very difficult to achieve in the case of the diagram and the structure shown in Figure (4). In the case of Figure 3, when the user subscribes to the Internet service, the connection is usually rated to the user's home or office. The nominal maximum data rate is 〇1. Depending on the provider's policy and routing device capabilities, the maximum data rate can be strictly or _ executed, but usually the actual available data is made by one of many different reasons. The rate is low. For example, there may be too many network traffic at the DSL central office or on the local electrical slow circuit loop' or there may be noise on the electrical material, causing the discarded packet, or providing It is possible to establish the maximum number of digits per month for each (four). The current maximum data rate for power mitigation and DSL services is usually in the range of hundreds of kilobits per second (Kbps) to 3 inches. Honeycomb services are typically limited to hundreds of Kbps of data. However, the speed of broadband service and the number of users subscribed to broadband services will increase dramatically over time. When Xiao's analysts estimate 33% of the US Broadband User 1 has a data rate of 2 MbPd2 Mbpsa. For example, some analysts predict that: to 2 (more than 85% of US broadband users will have 2 Mbps or more than 2 Mbps) Rate." As indicated in Figure 3, the actual available maximum data rate of 3〇2 can fluctuate over time. Therefore, 'in low latency, online games or application software ^ 139848.doc -47· 201001177 Sometimes difficult to predict The actual available resources for a particular video stream are for a specific amount of median > ^ - in ~ complexity and motion in a given number of frames per second ★ PS to give full resolution (for example, 64Gx48G @ 60 fps The data rate required to maintain a given quality level 3 () 3 rises above the actual maximum available; = as indicated by the peak in Figure 3), there may be several cases of cattle, and some Internet The road service will only discard the packet, thus guiding The lost data on the user's video glory and the distorted, my lost image. Other services will temporarily buffer (ie, rank) additional packets and provide the packet to user# at the data rate, resulting in an increase in latency: unacceptable results for many video games and applications. After taking it, some Internet service providers regard the increase in data rate as a malicious attack (such as 'No-destination attack (by computer hacker money network)

The well-known technology for disconnecting the connection is a) and will disconnect the user's internet connection during a specific time period. _, the embodiments described herein seek to ensure that the required data rate for video games does not exceed the maximum available data rate. Host Managed Service Architecture Figure 4a illustrates the architecture of a hosted service 2 〇 according to an embodiment. The host agency service 210 can be located in a single server center or can be distributed across multiple feeder centers (providing a low latency connection to users with a lower latency (four) to a particular server center than others) Provides load balancing between users and provides redundancy in the event of - or multiple server center failures. The colocation service 21〇 can eventually include tens of thousands or even millions of feeders 4〇2, so that it can be used very much Ϊ 39848.doc •48· 201001177

USer base. The colocation service control system provides overall control of the main hosting service 21' and directs the router, the server in one embodiment, the service control system 401 such as the colocation compression system, the billing and accounting system, etc. Linux-based decentralized processing system^, which is bundled into a squad for storing user information, server information, and system statistics, unless attributed to other features Otherwise hosted by the host

The various actions implemented are initiated and controlled by the host hosted service control system 〇1. The colocation service 21 includes a number of servers 4, such as those currently available from Intel, IBM, and Hewiett Packard, among others. Alternatively, the server 402 can be assembled into a custom component configuration, or the server 402 can ultimately be integrated to implement the entire server as a single wafer. This figure shows a few servers 4〇2 for illustrative purposes, but in actual deployment, there may be as few as one server 4〇2 or as many as millions or millions of servers 402. server. Servers 4〇2 can be configured in the same way (as some examples of configuration parameters, with the same cpu type and performance; with or without GPU, and with GPU, with the same GPU type and performance; with the same number Cpu&Gpu; RAM with the same amount and same type/speed; and with the same rAM configuration), or various subsets of servos 402 can have the same configuration (for example, 25% of the servers can be configured in a specific way) 50% of the servers are configured in a different way, and 25% of the servers are configured in a re-method, or each server 4〇2 can be different. 139848.doc -49- 201001177 In the embodiment, the word is free of magnetic disks, that is, not the local bulk storage device with the basin itself (which is an optical or magnetic storage device, or a semiconductor-based storage). A large-capacity storage device such as a flash memory or a feed-like function, each of which accesses a shared mass storage via a quick backplane or network connection. In one embodiment, the quick connect is connected to a separate redundant disk array (10) series of local area network (SAN) 4〇3, with connections between cryings implemented using ultra-high speed Ethernet. . As is known to those skilled in the art, (10) can be used to combine many RAID arrays 4〇5, resulting in extremely high frequencies - close to or possibly exceeding the RAM available for use in current game consoles and % The resulting bandwidth. Moreover, while RAID arrays based on rotating media such as magnetic media often have significant seek time access delays, the (4) column of the US conductor storage can be implemented to have a much lower latency. In another configuration, some or all of the servers 402 provide some or all of their own A-capacity storage at the local end. For example, the feeding device 4〇2 can store frequently accessed information (such as a copy of its operating system and video game or application %) in a low-latency local flash memory based storage. However, it can utilize a SAN to access a large library of rotating media based iR arrays 4〇5 with higher seek latency to access geometry or game state information less frequently. Additionally, in an embodiment, the colocation service 210 uses the low latency video compression logic 4〇4 described in detail below. The video compression logic 4〇4 can be implemented in soft peptides, hardware, or any combination thereof (the specific embodiments thereof are described below). Video compression logic 404 includes logic for compressing audio and visual material. 139848.doc -50-201001177. In the process of 'playing a video game via a keyboard, mouse, game controller or other input device 421 or using an application at the user's premises m, the control signal logic on the client 415 will indicate that the user is The control signal of the button (and other types of user input) is ab (usually in the form of a (4) packet) transmitted to the colocation service 21〇. In the future, the control signal from the user will be routed to the appropriate feeder (or if multiple feeds! § 'Respond to the use of the singular „„ As illustrated in Figure 4a, the control signal 406a can be routed to the server 4〇2 by the SAN. Alternatively or additionally, the control signal can be routed directly to the feeder 402 via a colocation service network (e.g., an Ethernet network based on the Ethernet). Regardless of how the control signals 4〇6a-b are transmitted, servers such as Hi or βH are executed in response to the control signal to execute the game or application software. Although not illustrated in Figure 4a, various network connection components (such as firewalls and/or gateways) can handle the hosted services at the edge of the network (e.g., 'hosting services 21' and the Internet. Incoming and outgoing traffic at the edge of the user's premises 211 (between the Internet 41〇 and the home or office user's mouth). The graphics and sounds of the executed game or application software That is, the DFL output (i.e., the new video image sequence) is provided to the low latency video compression logic 404, which compresses the low latency video compression technology (such as the techniques described herein). Video video sequence and compressed video stream (typically with compressed or uncompressed audio) via the Internet 410 (or, as described below, 'Best high speed network service bypassing the universal Internet) Transferring back 139848.doc 201001177 to the client 415. The low latency video decompression logic 4 12 on the client 415 decompresses the video and audio streams and reproduces the decompressed video stream, and The decompressed audio stream is often played on the display device 422. Alternatively, the audio can be played on a separate speaker from the display device 42 or the audio is not played at all. Note that although the input device 421 and the display device 422 are in Figure 2a and Shown in Figure 2b as a stand-alone device, but it can be integrated into a client device such as a portable computer or mobile device. Home or office client 415 (previously described as a home or office client in Figures 2a and 2b) 205) may be a very low cost and low capability device with very limited computational or graphical performance and may have very limited local large capacity storage or no local mass storage. In contrast, coupled to Each server 4〇2 of the SAN 403 and the plurality of RAIDs 405 can be an extraordinarily high-performance computing system, and in fact, if a plurality of servers are cooperatively used in a parallel processing configuration, there is almost no affordable The calculation amount and the limitation of the graphics processing capability. In addition, due to the low-latency video compression 4〇4 and the low-latency video decompression 412 (perceived by the user), the server 402 is The computing power is provided to the user. When the user presses the button on the input device, the image on the display benefit 422 is updated in response to the button press (with no meaningful delay in perception), as if the game or application soft system Execution at this point. Therefore, for a very low-performance computer or a home or office client 415 that only implements low-latency video decompression and control (4) logic 413 low-cost chips, it seems to be available at the local end. The location provides the user with any computing power. This gives the user an application for playing the self-contained, intensive (usually new) video game and the highest performance 139848.doc •52- 201001177 Ability. Figure 4c shows a very low-cost, low-cost home or office client device 465. This device is an embodiment of the home or office client from Figure 4a and the servant. It is about 2 inches long. It has an Ethernet jack 462 that interfaces with an Ethernet cable (Ethernet-powered (P〇E)) that receives its power from the Ethernet jack 4 6 2 and its access to the Internet. Network connectivity. The device is capable of performing NAT operations in a network that supports network address translation (NA Ding). Many new Ethernet switches have P〇E and bring P〇E directly to Ethernet jack in the office. In this case, the required Ethernet power from the wall jack to the client 465 is required. If the available Ethernet connection does not carry power (for example, if you have a 胤 or package, 'see data machine but no home), then there is a cheap wall-type "brick" available (ie, power supply) ), it will accept an etherless Ethernet cable and output an Ethernet network with P〇E. The client 465 contains control signal logic coupled to the Bluetooth wireless interface (Fig. 4a). The supervisory wireless interface interfaces with a Bluetooth input device 479 such as a keyboard, mouse, game console and/or microphone and/or earphone. Moreover, in the case of coupling with the display device 468, the video can be output at 120^, the display device can support 120 fps video and signal a pair of shading glasses 466 (usually via infrared) for each-succession The frame alternately obscures one eye and the other eye. The user should feel that i should be - jumped out to display the stereoscopic image of the screen. One of the operations is supported. The alpha haiming piece 468 is s_ung mow. Because the eye-flow system for each eye is separate, the frame is interleaved in time by an embodiment in which two independent video streams are compressed by the host 139848.doc -53* 201001177 official service 210. And the frame is decompressed in the client 465 by two independent decompression processes. The client 465 also includes low-latency video decompression logic 412, which decompresses incoming video and audio and outputs via HDMI (High Definition Multimedia Interface), connector 463, HDMI (High Definition Multimedia Interface), connector 463 It is inserted in SDTV (standard definition television) or hdtv (high definition television) 468 to provide video and audio to the TV, or to be inserted into the monitoring 468 supporting HDMI. If the user's monitor 468 does not support HDMI, HDMI to DVI (Digital Visual Interface) can be used, but the audio will be lost. Under the HDMI standard, display capabilities (eg, supported resolution, frame rate) 464 are communicated from display device 468, and then this information is passed back to the colocation service 21 via Internet connection 462, thus The colocation service 210 can compress the compressed video in a format suitable for the display device. Figure 4d shows the home or office client device 4, which is identical to the home or office client device 465 shown in Figure 4c, except that the client device 475 has more external interfaces. Again, the client 475 can accept p〇E to power, or it can occupy an external power adapter (not shown) that is plugged into the wall. The video camera 477 provides the compressed video to the client 475 using the client 475 USB input, and the compressed video is uploaded by the client 475 to the host representative service 210 for use as described below. The low latency compressor will be built into camera 477 using the compression techniques described below. In addition to having an Ethernet connector for its Internet connection, the client 475 also has an 8 〇 211 g wireless interface to the Internet. Both interfaces 139848.doc -54- 201001177 are capable of using NAT within a network that supports NAT. In addition to the HDMI connector for outputting video and audio, the client 475 also has a dual-link DVI-I connector, and the dual-link DVI-I connector includes an analog output (and has a standard that will provide VGA output). Adapter cable). It also has an analog output for composite video and S video. For audio, the client 475 has a left/right analog stereo RCA jack and has a TOSLINK output for digital audio output. In addition to the Bluetooth wireless interface to input device 479, it also has a USB jack for interfacing to the input device. Figure 4e shows an embodiment of the internal architecture of the client 465. All or some of the devices shown in this figure can be implemented in field programmable logic arrays, custom ASICs, or several discrete devices (custom design or off-the-shelf). Ethernet 497 with PoE To the Ethernet interface 48 1. Power 499 is derived from Ethernet 497 with PoE and is connected to the remaining devices in client 465. Bus 480 is a common bus for communication between devices. The control CPU 483 that executes the small client control application from flash memory 476 (almost any small CPU is suitable, such as the MIPS R4000 series CPU at 100 MHz with embedded RAM) is implemented for the network (ie The protocol of the Ethernet interface is also stacked and also communicates with the colocation service 2 1 0 and configures all devices in the client 465. It also processes the interface with the input device 469 and sends the packet (along with the user controller data subject to the forward error correction protection) back to the colocation service 2 1 0. 139848.doc -55- 201001177 Also, 'controls C P U 4 8 3 & sees packet traffic (for example, whether the packet is lost or delayed, and the timestamp of its arrival). This information is sent back to the colocation service 210 so that it can constantly monitor the network connection and adjust the content it sends accordingly. The flash memory 476 is initially loaded with a control program for controlling the CPU 483 and a serial number unique to the particular client 465 unit at the time of manufacture. This sequence number allows the colocation service 21 to uniquely identify the user 465 unit. The Bluetooth interface 484 wirelessly communicates to the input device 469 via its antenna (within the client 465). Video decompressor 486 is a low latency video decompressor configured to implement the video decoupling described herein. A large number of video solutions exist, either as off-the-shelf products or as intellectual property (IP) with a design that can be integrated into an fpga or custom ASIC. - provided for H.264 decoding

Australia^Ocean Logic of Manly 〇 The advantage of using I: is that the compression technique used in this article is not the same as the compression standard: Some standard decompressors are flexible enough to be configured to accommodate the compression techniques in this article, but some standard decompressors may not. However, in the case of ip, the decompressor is redesigned as needed: full flexibility. The round output of the video decompressor is coupled to the video output subsystem 487, and the output subsystem 487 combines the video to the video device of the HDMI interface, or the device is implemented. Implemented as a standard available for Ip' or can be implemented in audio decompression (for example) 139848.doc •56- 201001177

The audio decompression is implemented in the control processor 483 of the Vorbis audio decompressor. The device that implements the audio decompression is coupled to an audio output subsystem 489 that couples the audio to the audio output of the HDMI interface 490. Figure 4f shows an embodiment of the internal architecture of the client 475. As can be seen, in addition to the additional interface and optional external DC power from the power adapter plugged into the wall (and if so, the optional external DC power replacement will be from the Ethernet PoE 497), The architecture is the same as that of the client 465. The functionality common to the client 465 will not be repeated below, but the additional functionality will be described below. The CPU 483 communicates with additional devices and configures additional devices.

The WiFi subsystem 482 provides wireless internet access via its antenna as an alternative to Ethernet 497. The WiFi subsystem is available from a number of manufacturers, including Atheros Communications of Santa Clara, CA. USB subsystem 485 provides an alternative to Bluetooth communication for wired USB input device 479. The USB subsystem is fairly standard and can be easily used in FPGAs and ASICs, and is often built into existing devices that perform other functions such as video decompression. The video output subsystem 487 produces a wider range of video outputs than the video output in the client 465. In addition to providing HDMI 490 video output, it provides DVI-I 491, S-Video 492 and Composite Video 493. Also, when the DVI-I 49 1 interface is used for digital video, the display capability 464 is passed back from the display device to the control CPU 483 so that it can notify the host escrow service 210 of the capabilities of the display device 478. The 139848.doc -57-201001177 interface provided by the video output subsystem 487 is a fairly standard interface and is readily available in many forms. Audio input*, system 489 via digital interface 494 (S/PDIF and/or

Toshnk) outputs audio digitally and outputs an analog analog video via stereo analog interface milk. Round trip delay analysis Of course, in order to achieve the benefits of the front-end, the user's action of using the input device 42: and the effect of the back-and-forth between the action on the display device (4) should be no more than 7 milliseconds _8 milliseconds . This delay must account for all factors in the path from the input device 421 in the user premises 211 to the colocation service (10) and back to the user premises 211 to display device 422 again. Figure 4b illustrates the various components and networks (the signals must pass through and travel, and the components above the network and the timetable above the network are the timetables, which list the actual real important path routes. Below j: __ to show only the phase The data of the other characteristics of the money - this double arrow (such as 'arrow 453') indicates the round trip delay and the head of the single head (for example, arrow 457) indicates a one-way extension and " Dan-Blrfc ~" indicates an approximate y It should be noted that there will be real world conditions that cannot be reached by the listed delays: but in a large number of situations, in the United States, use to the user field = muscle and electricity (four) machine connection, these delays can be achieved in the following situations. Also, note that although the Internet to the second = connection will indeed be on the screen & gas,,,, v spring nesting f #/ ... IH # A f number of current American bee (EVDG) incurs very High latency and will not be the same: the delay shown in the fine. However, these basic moons are implemented on the future honeycomb technology with a delay of this level. 139848.doc •58- 201001177 • Self-user The input device 421 at the location 211 begins, as soon as The user actuates the input device 4 21, 対, amine #田土,, and sends a user control signal to the user terminal. '4_15 (which may be a stand-alone device such as a set top box, or it may be in a pc or The software or hardware implemented in the other device of the mobile device, and packetized (in the embodiment - in bold format) and given the destination address for the packet to reach the colocation service 2;! 〇. It will also contain a message indicating that the control signal is from that user. The control signal packet is forwarded to the vertical plane 442 via the firewall/router/NAT (Network Address Translation) device 443. The WAN interface 442 The interface device provided by the user's lsp (internet service provider) to the user's premises. The WAN interface 442 can be a cable or DSL data machine, a WiMax transceiver, a fiber optic transceiver, a cellular data interface, and a power line network. Network protocol interface (Internet pr〇t〇c〇i_ 〇Ver-P〇werline interface), or any other interface in many interfaces of the Internet. In addition, firewall / router / ΝΑτ device 443 (and ( Possibly) WAN The interface 442) is integrated into the client 415. One such instance would be a mobile phone that includes software for implementing the functionality of the home or office client 415, as well as for passing a certain standard (eg, 802.11) g) wirelessly routing and connecting to the components of the Internet. The WAN interface 442 then routes the control signals to the "presence point" 441 referred to herein as the Internet service provider (lsp) for the user. The WAN interface 442 is a facility that provides an interface between the WAN transporter connected to the user premises and the general Internet or private network. The nature of the point of existence will vary depending on the nature of the Internet service provided. For Dsl, the system will be the central office of the telephone company where the DSLAM is located. For the 139848.doc -59- 201001177 cable modem, it will typically be the head of the electric power, 4 multi-system operator (MSO). For the honeycomb system, Litong is riding & ... ', f is the control associated with the honeycomb tower. However, regardless of the nature of the point, it will then route the control signal packet to the universal Internet. The splicer routes the control signal packet to the lion interface 444 to the host escrow service 2U via the interface that is most likely to be the fiber-optic interface. WAN 444 will then route the control signal packets to the logical (which can be implemented in many different ways, including the Ethernet parent exchange and routing server), routing logic to fully estimate the user's address and will control The signals are routed to the correct server for a given user - the server 402 then treats the control signals as inputs to the game or application software executing on the server 4〇2 and uses the control signals to Handle the game or application under a frame. Once the next frame is generated, the view and status are rotated from the servo state 402 to the video compressor 4〇4. The video and audio can be output from the server to the compressor 4〇4 via various components. First, the compressor 404 can be built in the server 4〇2, so compression can be performed at the local end in the server 4〇2. Alternatively, it may be connected via a network to the network (either a private network between the feeder 402 and the video compressor 4〇4, or a network via a shared network such as the SAN 403) (such as , Ethernet connection) output video and / or audio in a packetized form. Alternatively, the video can be output from the servo hack 402 via a video output connector (such as a DV: ut VGA connector) and then captured by the video compressor 4 〇 4 . Alternatively, the tone can be output as a digital audio (e.g., via a t〇SLINk or S/PDIF connection state) or analog audio, and the analog audio is digitized and encoded by the audio compression logic within the video compressor. 139848.doc -60· 201001177

Once the video compression Is 404 has captured the video frame from the server 4〇2 and the audio generated during the frame time, the video compressor will use the technique described in the text to reduce the video and audio. Once the video and audio are compressed, they are packetized by an address to send it back to the user's client 415 and routed to the WAN interface 444, which then routes via the universal Internet 410. For video and audio packets, the universal Internet 410 then routes the video and audio packets to the user's isp location 441 'Existence 44' to route the video and audio packets to the WAN interface 442 'WAN interface 442 at the user's premises. Routing the video and audio packets to the firewall/router/NAT device 443 'Firewall/Router/Return to Device 3 and then route the video and audio packets to the client 4丨5. The client 415 decompresses the video and audio, and then displays the video on the display device 422 (or the built-in display device on the user side) and sends the audio to the display 422 or to a separate amplifier/speaker or to the built-in The amplifier/speaker in the client. /Users _ _ The entire process described is sensible without lag - the lingering delay needs to be less than 70 milliseconds or 80 milliseconds. Some of the delay delays described in the return path are subject to the colocation service 2U) and/or = ^ ' while other delay delays are not controlled by the colocation service 210 and/or the user. ^ "So" is based on a large number of real world situation analysis and heart type, the following is an approximate measurement. j unidirectional transmission time of the 19-signal signal... usually less than 1 millisecond, the red-user location of the round-trip route M2 is usually attached The easy-to-obtain consumption goes to you', E-Tai, . Lulu-level firewall/router/ΝΑΤ exchanger and at about 1 139848.doc -61 · 201001177. The user ISP changes its back and forth extensively. The delay is delayed, but in the case of DSL and cable modem providers, it is usually seen between milliseconds and 25 milliseconds. The round-trip delay on the universal Internet 41 is how the traffic is routed and whether it exists on the route. Any failure (and such Q is described below) varies greatly, but typically the universal Internet provides a fairly optimal route and the delay is largely determined by the speed at which light passes through the fiber (10) to the destination. The distance is determined as follows. As discussed further below, it has been confirmed that 1GGG is the farthest distance that is expected to place the colocation service away from the user's location. At 1 mile (round trip) 200 ancient 0 miles), the actual transmission time for signals via the Internet is about 223⁄4 & to the host interface service 21q wan interface (10) is usually a commercial-grade fiber-optic high-speed interface with a decisive delay. Therefore, the universal network The network delay 4 5 4 is usually between 1 millisecond and * seconds. The delay of the early route 455 via the colocation service can be reached in less than 1 millisecond. The servo will always be less than - frame time (its Calculate a new frame for the game or the heart-shaped type in 6〇 as follows, so 16 milliseconds is the maximum one-way delay 456 that will make ^. The video compression described in this article. In the best hardware implementation of the algorithm, compression 457 can be completed in ! milliseconds. In the next best version, 懕+ less desirable version can take longer to gate up to 6 milliseconds (of course, 'A Jia Longer days • But these implementations will affect the round trip: 'delay and will require other delays to be shorter (for example, to reduce the allowable distance via GM/Internet) to maintain 7g milliseconds (10) milliseconds == Care network = 454, _453 and user site K Ding private I & Decompression for video 458 extension 139848.doc • 62· 201001177 曰 曰 ΐ αΐ1 decompression 458 delay depends on whether the video decompression 458 is implemented in dedicated hardware or implemented in the client device 415 (such as pc or mobile device) In the upper body, the size of the visual display and the efficiency of the decompression change. Usually, decompression 458 takes between 1 millisecond and 8 milliseconds. Therefore, 'all the worst conditions that can be seen in practice. The delay adds up to the worst-case back-and-forth delay that the user of the system shown in Figure 4a can expect to experience. It is: +25+22+ι + ΐ6+6. In addition, in practice, in practice (with the prevention of the solution discussed below), this is roughly the use of the prototype version of the system shown in Figure 4a (using the off-the-shelf Wind〇ws pc as a user) The back-and-forth delay seen by the end device and the home DSL and the electrical modem connection). Of course, situations that are better than the worst can lead to much shorter delays, but you can't rely on them to develop widely used business services. In order to achieve the delay listed in the servant via the universal Internet, the video M-capitalizer in the clientization and the video decompressor 412 (from Figure 4a) are required to generate a packet stream having (4) specific characteristics to The sequence of packets generated via the entire path from the hosted service 21 to the display device 422 is not subject to delay or excessive packet loss' and, in detail, consistently falls over the Internet connection via the user (via _ Interface 442 and firewalls within the bandwidth of the /router/NAT 433)w user. Back to the other, the video reducer must generate a sufficiently robust packet stream to cause the inevitable packet loss and packet reordering that occurs in the normal Internet and network transport. Low-latency video compression 139848.doc •63· 201001177 In order to accomplish the above objectives, an embodiment employs a new video compression method that reduces the delay and peak bandwidth requirements for transmitting video. Prior to describing such embodiments, an analysis of the current video rendering techniques will be provided with respect to Figures 5 and 6a-6b. Of course, if the user has sufficient bandwidth to handle the data rate required by such technologies, then these techniques can be used according to the basic principles. The main method does not solve the audio pressure and shrink, but the audio compression system is stated. Video compression is implemented simultaneously and simultaneously. Prior art audio compression techniques that meet the requirements for this system exist. Figure 4 illustrates a particular prior art for compressing video, in which a compression algorithm 520 compresses each individual video frame 5〇1-503 using a particular compression algorithm to produce a series of reduced frames 511- 513. One embodiment of this technique is "Motion JPEG" in which each frame is reduced based on a discrete cosine transform (DCT) according to a Joint Photographic Experts Group (JPEG) compression algorithm. Various different types of compression algorithms can be used, and these basic principles are still adhered to (eg 'wavelet-based compression algorithms such as jpEG_ 2000). One of the problems of cheek compression is in ', w _ heart page i| ^ but it does not take advantage of the similarity between successive frames to reduce the total video stream 4 rate. For example, as illustrated in Figure 5, assuming 640 χ 480 χ 2 ^ / pixel = 640 * * * 24 / 8 / 1 〇 24 = _ kilobytes / frame (Μ, a box) frame rate, then Recognized u, ,,, ° 疋〇〇 之 之 , 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动 运动In the 6 frames/sec, this will require 90 KB 8 bits 兀 60 frames / sec = 42.2 Mbp ', I see frequently 'for the United States today, a few rich ploughs from a, letter to see all the 豕Internet I| 139848.doc -64- 201001177 ' The word is the bandwidth of the extreme return, and for many office Internet connections • ° is the bandwidth of the past. Strictly, assuming that it is in the case of such a high frequency width = = constant data stream 'and it will only be gamma-users, it will consume 10 Mbps Ethernet even in the case of the brothers [ ΑΝ 频 的 的 的 的 的 的 的 的 的 的 。 。 。 。 。 。 。 。 。 。 。 。 。 。 This is in contrast to other compression techniques, such as those described below, which are less efficient than compression of motion video. In addition, the use of lossy Li (★ 寅 法 之 之 之 ( ( ( ( ( ( ( ( ( ( ( ( : : : : : : : : : ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( The false shadows in the leaves may not appear as false shadows, because the eyes are not green, and the dense leaves should be presented.) However, once the scene is in the middle of the rice, it can be highlighted because the eye is detected. The frame changes to the frame: the artifact, although the artifact is in the area of the scene (in this area, the artifacts are not noticeable in the image). This leads to the moon in the sequence of frames. The mouth and mouth analogy of "Northern &a; A ^ 3 Hai Background Noise"

The "+ L (J see's) noise is similar during TV reception. Of course, this type of compression can still be used in the specific embodiment described in Φ>, +· > k Λ + However, in general, in order to avoid the background noise of the hall, the high data rate (that is, the low compression ratio) is required for a given perceived quality. ', the compression of the award (such as Η.264, Or Windows Media VC9, MPEG2, and MPEG4) are more efficient in compressing video streams because they exploit the similarities between phase and frame. These techniques all rely on the same general techniques used to compress video. Although the Η·]” standard will be described, the same general principles apply to various other compression algorithms. A large number of Η·〕"pressure 13984S.doc -65· 201001177 Reducers and decompressors are available, including for compression. 264 x264 open source software library and FFmpeg open source software library for decompressing 264.264. Figures 6a and 6b illustrate an exemplary prior art compression technique in which a series of uncompressed video frames are compressed by compression logic 620. 5〇1_5〇3, 559_561 pressed into a series I frame "611, 671; "p frame" 612-613; and "B frame" 670. The vertical axis in Figure 6a generally represents the resulting size of each of the encoded frames (although such The frame is not drawn to scale.) As described above, those skilled in the art have a good understanding of the video writing code using the I frame, the B frame, and the P frame. In short, the I frame 611 is completely absent. DCT-based compression of compressed frame 501 (similar to compressed JPEG images as described above). The size of p frame 6 12-61 3 is typically significantly smaller than the size of j frame 6 11 'because its use The data in the previous I frame or p frame; that is, 'which contains data indicating the change between the previous I frame or p frame. In addition to the B frame, use the frame in the subsequent reference frame and (possibly In addition to the frame in the frame, B frame 670 is similar to the P frame. For the following discussion, it will be assumed that the desired frame rate is 6 frames/sec, and each I frame is about 160 Kb. 'The average P frame and B frame are 16 Kb and a new I frame is generated every second. Under this group of parameters, the average data rate will be: 160 Kb+16 Kb*59=1, 1 Mbps. This data rate falls appropriately within the maximum data rate for many current broadband Internet connections to homes and offices. This technology also tends to avoid coming from only the box. The background noise problem of the code, because the difference between the P frame and the B frame tracking frame, the compression artifact tends not to appear and disappear from the frame to the frame, thereby reducing the background described above The problem of noise. 139848.doc -66· 201001177 One of the problems with the above-mentioned type of contraction is that although the average data rate is relatively low (for example, 1.1 Mbps), it is single! Frames can take several frame times to transfer. For example, using the prior art, a 2 2 Mbps network connection (eg, a DSL or cable modem with a 2.2 Mbps peak from the largest available data rate of FIG. 3 & 3 〇 2) would normally be sufficient to cause the video to be 1; l Mbps serial machine, 160 Kbps I frame per 60 frames. This will be done by having the decompressor discharge the leap second video into the queue before decompressing the video. Within a working second, 1.1 Mb of data will be transmitted, which will be easily accommodated by the maximum available data rate of 2·2 Mbps, even if it is assumed that the available data rate may periodically drop by as much as 5 〇 /〇. Unfortunately, this prior art method would result in a 1-second delay in video due to the 1 second video buffer at the reception. This kind of delay is enough for the tactical application (for example, the playback of linear video). The fast action for the delay of more than 70 milliseconds _8 milliseconds is extremely long. Delay. If an attempt is made to eliminate the i-second video buffer, it will still not cause enough delay for the fast-motion video game. For example, the use of the U-frame as previously described will require reception! All b frames and I frames before the frame. If it is assumed that approximately 59 non-frames are split between the P and B frames, there will be at least 29_frames and the I frames received before any b frames can be displayed. Inb, especially phase,,, u, the bandwidth available for this is not required, all need 29 + 1 = 30 frames each go / 6 ^ 丨, 4 * mother 1 / 60 seconds duration Delay, or a delay of 500 milliseconds. Obviously, his time is extremely long. 0 Λ The H method will eliminate the B frame and only use the I frame and the p frame. (There is a consequence: for a given quality level, the data rate will increase by 139848.doc -67· 201001177 plus, but for consistency in this example, continue to assume that the size of each ^^ frame is 160 Kb and the average P frame size is 16 Kb, and therefore the data rate is still 1.1 Mbps). This method eliminates the unavoidable delay introduced by the 8-frame because each of the P-frame decoding relies only on the previously received frame. The problem with this method is that the frame is much larger than the average p-frame, so that on the low-band wide channel (as is typical in most homes and many office=), the transmission of the I-frame adds substantial delay. This is illustrated in Figure 6b. The video stream data rate 624 is lower than the available maximum data rate 621 (except for the I frame), where the peak data rate 623 required for the I frame is well above the maximum available data rate 622 (and even exceeds the nominal maximum data rate 621). . The data rate required for the P frame is less than the maximum data rate available. Even if the peak maximum data rate peak at 2.2 Mbps is steadily maintained at its 22 Mbps peak rate, it will take 16 〇Kb/22 Mb = 7l milliseconds to transmit the work frame, and if the maximum available data rate 622 drops by 〇% ( 11 , it will take 142 milliseconds to transmit the frame. Therefore, the delay in the transmission 1 frame will be somewhere between 71 milliseconds and 142 milliseconds. This delay is added to the delay identified in the map servant (these The delay is a total of 7 〇 milliseconds in the worst case, so this will result in a total round trip delay of 141 _222 milliseconds from the moment the user actuates the input device 421 until the image master is on the display device 422. It is extremely good, and if the maximum data rate can be dropped below 22 Mbps, the delay will increase further. Also / think that 'before the peak data rate 623 of the available data rate 622 clogging the ISP' usually has serious consequences. Different ISp The devices in the system will behave differently, but when the packet is received at a data rate 13984S.doc •68, 201001177 that is much higher than the available data rate 622, the following behavior is quite common among the DSL and cable modem isp. a) delaying the packet (introduction delay) by placing the packet in the queue, () tea 2 or all packets, (c) deactivating the connection for a period of time (most likely for the ISP to be a malicious attack, Such as "denial service" attack L port This transmission of packet flow at full vegetative rate (having characteristics such as the one in Figure 6b) is not a viable option. It can be used in the host generation & spear 2 1G The peak (3) row is listed and sent at a data rate lower than the maximum data rate available, thereby introducing an unacceptable delay described by the former. The video stream data shown in Figure 6b is "_(-" The rate of the data rate is expected to be generated by the video from the end of the video sequence. The video sequence does not change, the two are very large and have very little motion (for example, as in the video) In the electro-meeting session, the 普 , 兮 兮 Λ罨 Λ罨 , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Objects in Di Di) show less movement). The video streaming asset shown in Figure 6c is a typical (4) video that has a lot of video (such as may be seen in a movie or video application). In addition to the peak 633 of the frame, there is also a phase out of 1 M. , which is in the phase of the large field and exceeds the peak P of the maximum available material rate, G·35 and 636). Although the peak value of this P-frame is not as aim as the frame, it is still quite large, but it is still so large that it is carried at full data rate, and as such, the peak of the frame must be slow, _ ΰ Busy peak one-bay, friend-k transmission (by increasing the delay). 139848.doc -69- 201001177 On a high-bandwidth channel (for example, 100 Mbps lAN, • private connection), the network will be able to tolerate large peaks such as _ == frame peak 636, but in principle or 15 曰# μ j, with low latency. The Internet is often shared among many users (for example, I: to the brothers of % i), and the data of "Hui Feng" will affect the performance of the second party, especially when the network traffic is routed to private use. In the case of sharing "heart to office". First, remember this material: • An example of a relatively low-resolution video stream of pixels. The X-V stream is easily accessible by modern computers and monitors. The 256GX144G resolution display is increasingly available (for example, Apple's 30" display). Using H 264 compression, the following 192 〇 X1080 high motion video sequence may require 4.5 M—the ergonomic quality level. If assumed! The frame ♦ value is the nominal data rate of the Cong, which will produce 45 Mbp_values and smaller but still quite large p-frame peaks. Several users are sharing the same - 1 Mbps network (for example, office Receiving the video stream on the private network connection to the poor center, it is easy to see how the peak of the video stream from several users can be aligned, thus drowning the bandwidth of the network and possibly flooding the network. The bandwidth of the backplane that supports the user's switch. Even in the case of ultra-high-speed Ethernet, if enough users have enough peaks for simultaneous alignment, they can flood the network = network switch. In addition, once 256 〇χ 144 〇 resolution video becomes more common, the average video stream data rate may be 9.5, which may result in a 95 Mbps peak data rate. Needless to say, the 100 Mbps connection between the data center and the office (which is now an extraordinarily fast connection) will be completely defeated by the early-user peak traffic of 139848.doc -70-201001177. Therefore, even if lan and private network connections are more tolerant to peak-stream video, streaming video with high peaks is not required and may require special padding and adaptation D in the office of the office. Tian ', , ; , , for standard linear video applications, these problems are not a problem, the mouth rate is "smoothed" at the transmission point and the data for each frame is below the maximum available data rate 622, And before decompressing the frame 'p frame' and the frame sequence, the buffer of the client_stores the I round frame, the frame and the frame sequence. Because &, the data rate on the network remains close to the average data rate of the video stream. Unfortunately, this introduction delay, even if the B-frame is not used, is unacceptable for low-profile applications such as video games and applications that require fast response times. Prior art solutions for mitigating video streams with high peaks are often referred to as "constant bit rate" (CBR) coding techniques. CBR seems to imply that all frames are compressed to have the same bit rate (also #, size), but it is often referred to as the compression range W, which allows for a specific number of frames in the shrinking example. (in my case, 1 frame) the maximum bit rate. For example, in the case of the map &, if a CBR constraint is applied to the code that limits the bit rate to, for example, 70% of the nominal maximum data rate 621, the compression algorithm will limit the frames. Each of the frames is compressed such that any frame that would normally be compressed using more than 7% of the nominal maximum data rate 621 will be compressed with fewer bits. The result of this is that it will generally require more bits to maintain the "extremely lacking" bit of the frame of a given quality level and the image quality of those frames will be higher than the required ratio of 139848.doc 201001177 The quality of the 7% of the catch rate of 62i is bad. This method is for a specific type of dust-defining video: two less motion or scene change and (b) user accessibility (sexuality expectation can produce acceptable results. The main meaning of the application suitable for (10) is because there is a comparison Less peaks, and below the quality 'below (eg 'If the camera is swept' resulting in significant scene motion and university values' then there may not be enough during the glance ^

: like, it will lead to degraded image quality), most make SC legacy (four), coffee is not well suited for scenes with high complexity or big movements and / or need people ' Λ ^ to be the quality of the constant Many other applications of level 0 The low latency compression logic 404 used in one embodiment uses several different techniques to solve many of the problems of streaming low latency compressed video while maintaining high quality. First, the low-latency compression logic 4〇4 only produces a frame and a 15 frame' thereby relieving the need to wait for several frame times to decode each frame. In addition, as illustrated in Figure 7a, in an embodiment, the low latency compression logic 404 divides each uncompressed frame 7 "6 into a series of "tiles" and will each - The image blocks are individually encoded as a ^ frame or a p frame. In this paper, the compressed j frame and p frame are referred to as "r frame" 711-770. In the particular example shown in Figure 7a, each uncompressed frame is subdivided into 16 image blocks of a 4x4 matrix. However, these basic principles are not limited to any particular sub-dividing mechanism. In the example, the low-latency compression logic 404 divides the video frame into a plurality of image blocks, and encodes one of the image blocks from each frame (ie, 139848.doc • 72-201001177 compression) into an i-frame ( That is, the image block is compressed as if it were a single video frame of 1/16 of the size of the full image, and the compression for the "mini" frame is I frame compression) and the remaining image blocks are encoded. It is a p-frame (that is, the compression for each "mini" 1/16 frame is p-frame compression). The image blocks compressed into I frames and the image blocks compressed into p frames will be referred to as "I image blocks" and "P image blocks", respectively. The image block to be encoded as an I image block is changed with each successive video frame. Therefore, in a given frame time, only one of the image blocks in the video frame is an image block, and the remaining ones of the image blocks are p-image blocks. For example, in the figure, the image block of the uncompressed frame 701 is encoded as an image block and the remaining 1-15 image blocks are encoded as p image blocks (ΡιιΡΐ5) to generate an r frame 711. In the next uncompressed video frame 7〇2, the image block of the uncompressed frame 701 is encoded as the image block 1 and the remaining image blocks are encoded into 15 pixels as p image blocks (p 〇, and I to p〗 5) to generate the R frame η]. Therefore, the I image block and the p image block for the image block are gradually interleaved in time on successive frames. The process continues until R is generated. Image block 77〇 (the last image block in the matrix is encoded as a Γ image block (ie, so far. The process then restarts, resulting in, for example, frame 711 (ie, for image block 0, coded I image block), etc. The other - R frame. Although not illustrated in the figure, in the consistent embodiment, the first R frame of the video sequence of the R frame contains only the image block (that is, so that the subsequent p frame has Reference image data (from which motion is calculated)). Or, in an embodiment, the start sequence is used. Normally the same I image block type, but does not include p image blocks for their image blocks that have not been encoded together with the image block. In other words, before the arrival of the first image 139848.doc -73 - 201001177 block Any data is encoded together with a particular image block, thereby avoiding the start peak in the video stream data rate 934 of Figure 9a, which is described in further detail below. Further, as described below, various different sizes and shapes can be used for the image blocks simultaneously. The basic principles are still adhered to. The video decompression logic 412 executing on the client 415 decompresses each image block as if it were a separate video sequence of the small I frame and the P frame, and then reproduces each image block. The frame buffer for driving the display device 422. For example, the Up block from the R frames 711 to 77 is used to decompress and reproduce the image block of the video image. Similarly, use the frame from the R frame 71 to "Ο之丨 to reconstruct the image block, etc. ^ As mentioned above, the decompression of the frame and the p frame is well known in the art, and] the decompression of the image block and the p image block can be borrowed This is done by having multiple execution entities of the video decompressor in use. Although the multiplication process appears to increase the singular burden on the client 415 'but it does not actually increase the computational burden on the client 415' because of the image The blocks themselves are proportionally smaller (relative to the number of additional processing). The number of pixels displayed by this is the same, as if there is a processing and using the full-size frame and p-frame of the conventional. The frame technique significantly mitigates the bandwidth peaks typically associated with the (four) box (illustrated in the figure and in the figure because any given frame is mainly made up of a frame that is usually smaller than 1 frame). For example, suppose the code (1) The frame is 1 60 Kb, and the j image block of the mother of the frame in the frame of the 6th month of the month of Figure 7a will be roughly 丨/16戎丨10 Kb for this amount. Similarly, assuming that the typical P frame is 16 Kb, the P frame used for each of the image blocks of Figure 5a may be approximately 1 Kb. In the end, the gentleman wants the helmet to take, and the mouth is about 10 Kb + 15 * l Kb = 25 to the rule 139848.doc -74- 201001177 box. Therefore, each 60-frame sequence will be Μ Kb*6〇 = i 5 MbW. Therefore, at 60 frames per second, this would require a frequency that is capable of maintaining a bandwidth of 1.5 Mbps, but because the I-picture block is dispersed throughout the 6-frame interval, it has a much lower peak.

Note that in the previous example, the average data rate was M Mbps with the same assumed rate of the frame and the p-frame. This is because in the previous example, only a new I frame is introduced every 60 frame time, and in this case, the 16 image blocks constituting the I frame are cycled in 16 frame times, and thus A J is introduced every 16 frame times resulting in a slightly higher average data rate. Despite this, the introduction of more frequent I-frames does not linearly increase the f-rate. This is due to the fact that the P-frame (or p-image block) mainly encodes the difference from the previous frame to the next frame. Therefore, if the previous frame is similar to the next frame, the P frame will be very small. If the previous frame is quite different from the next frame, the P frame will be very large. However, because the p-frame is largely derived from the previous frame instead of being derived from the actual frame, the resulting coded frame may contain more errors than a Z-frame with a sufficient number of bits (9). Visual artifacts). In addition, when the -p frame follows the other -p frame, an error accumulation can occur (when there is a long P-frame sequence, it gets worse). Now, the cutting-edge video retractor will detect the fact that the quality of the image is degraded after the sequence P frame, and if necessary, it assigns more bits to the subsequent P frame to the quality of the temple. Straight or Wu Shi, the second right to take a valid action process, replace the P frame with a frame. Therefore, when using a long P-frame sequence (for example, 59 p-frames) as in the "example" above, it is specific when the scene has a large number of complex 139848.doc -75 - 201001177 noise and / or motion, usually For the p-frame, more bits are needed (because it becomes farther away from the I-frame). Or 'seeing the P-frame from the relative view point, the P-frame that closely follows the I-frame tends to require The bit of the p frame is farther away from the frame. Therefore, in the example shown in Figure 7a, the p-free frame is farther away from the I frame by 15 frames (before the I frame) In the previous example, the p-frame can be separated from the I frame by 59 frames. Therefore, in the case of more frequent frames, the p-frame is smaller. It changes based on the nature of the video stream, but in the example of Figure 7a, if the image block is 10 Kb, the size of the P block can be only 0.75 kb on average, resulting in 1〇Kb+15*0.75 Kb=21.25 Kb. , or at 60 frames per second, the data rate will be 21.25 Kb*60 = l.3 Mbps, or a frame with a frame of 1 p 1 frame followed by a stream of 59 p frames The rate is about 16% higher. Again, the relative result between these two methods for video compression will change depending on the video sequence. 'But usually, we have empirically found that for a given quality level, use the R frame. It takes about 20% more bits than using the I/p frame sequence. However, of course, the R frame sharply reduces the peak, which makes the video sequence available at a much slower delay than the I/p frame sequence. The R frame is configured in a number of different ways depending on the nature of the sequence, the reliability of the channel, and the available data rate. In an alternative embodiment, a number of image blocks other than 16 are used in the 4x4 configuration. Use 2 image blocks in 2χΐ 4x1 or 1x4 configurations or 1x2 configurations, 6 in 4x2 6X1 or 1 x6 configurations, 4 image blocks in 3x2, 2x3, image blocks or 4χ2 (8 image blocks are used in Figure 7b. Note, shown), 2M, 8xl or lx8 configuration image blocks do not need to be square, video frame 139848.doc -76- 201001177 The frame does not have to be square, or even rectangular The image block can be decomposed into the best suitable video for use. Regardless of the shape of the application, in another embodiment, the loop of the I image block and the P image block is not locked to the number of the image block. For example, in the 8 image block 4x2 configuration, it is still available. The 16-cycle sequence is used as described in 7b. The sequence of uncompressed frames: each of m3 is divided into 8 image blocks ", and each block is individually compressed. From R frame 73l, only image blocks〇 Compressed into ^ image blocks, and the remaining image blocks are compressed into p image blocks. For subsequent frame 732, all 8 image blocks are compressed into p-image blocks, and then for frame 733, image block i is compressed and the other image blocks are compressed into P-blocks. In addition, this continues for the 16 frames, only one every - frame! The image block is therefore generated at the end of the ... fixed frame time (not shown in Figure 7b) and during the 16th frame time for the last of the image block 7! Image block (compress r frame 780 using all p image blocks). Then, the sequence begins again with the image block () being compressed into a phosphor block and the other image block being compressed into a P block. As previously implemented, the first frame of the entire video sequence will typically be & image blocks to provide a reference for P blocks that are forward from the other point. ! The loop of the image block and (9) the block does not even need to be an even multiple of the number of image blocks. For example, in the case of 8 image blocks, before using another Z image block, each frame having a called image block may be followed by two frames of all (9) image blocks. In yet another embodiment, for example, it is known that the area of the screen material has more motion (requiring more frequent I image blocks) and other areas are more static (eg, the score K of the game is less frequently required! Image blocks) ), and other image blocks 139848.doc -77- 201001177 more often than τ sort the specific image block together with the work image block. In addition, in the figure 7a-7b, each frame has a single - The taste block, = can encode multiple I image blocks in a single frame (depending on the transmission: 迢 frequency is). Conversely, a particular frame or frame sequence can be transmitted without a B-picture block (ie, only a P-picture block). The reason why the method works properly is that although the 1 image block that does not have a span across the parent frame seems to cause a large peak, the behavior of the system is not so simple. Because each image block is compressed separately from other image blocks, each image block: encoding can become less effective when the image block becomes smaller, because the compressor of a given image block cannot utilize other image blocks. Similar to image features and similar sports. Therefore, when the disc divides the screen into 8 image blocks, dividing the screen into "image blocks will usually result in less efficient coding. However, if the screen is divided into 8 image blocks and it causes every 8 blocks. The introduction of the frame (rather than every 16 frames) - the data of the complete 1 frame, which results in a much higher data rate overall. Therefore, by every 16 frames (rather than every 8 frames) The introduction of a full I ® box 'reduces the total data rate. Again' by using 8 larger image blocks instead of 16 smaller image blocks, the total data rate is reduced, which will also be The peak of the data caused by the larger image block is reduced to some extent. In another embodiment, the low-latency video I of FIG. 7a and the figure are based on the known characteristics of the video sequence to be shrunk. Automatically control the allocation of bits to each image block in the R frame based on pre-configured or based on ongoing analysis of the image quality in each image block. For example, in some competitive video games, The player's car (which is relatively no in the scene 139848.doc -78- 201001177) The upper part of the movement is on the second half of the screen: according to: the majority of the lower half of the screen, and it is almost always in motion += near roads, buildings and scenery ' eight ^ right compression logic 404 will be equal to the number The bit ^ ^ mother / shirt image block, then the image block (image block 4 · 7) on the lower half of the uncompressed frame 721 in Figure 7b will generally be compared to that in Figure 7b, : the quality between the image blocks (image blocks 〇-3) in the upper half of the screen in block 721, and the end n ^ n , θ is known to have such characteristics for this particular scene of the particular game or game. Main drag

The operator of the colocation service 210 can configure the compression logic 404 to allocate more bits to the image block at the top of the camp (compared to the bits of the image block at the bottom of the screen). ). Alternatively, compression logic 404 may estimate the compression quality of the image block after compressing the frame (using _ or more of a number of compression quality metrics, such as peak signal to noise ratio ("PSNR)) m at a particular time f, specific As the image block consistently produces better quality results, it gradually assigns more bits to the image blocks that produce lower quality results until the various image blocks reach a similar level of quality. In an alternate embodiment, compressor logic 404 allocates bits to achieve higher quality in a particular image block or image block group. For example, it can provide a better overall perceived appearance to have a higher quality at the center of the screen than at the edges. In one embodiment, in order to improve the resolution of a particular region of the video stream, video compression logic 404 uses a smaller image block to encode a region of the video stream that has relatively more scene complexity and/or motion (with video stream). Relatively small compared to the complexity of the scene and/or the area of motion). For example, as illustrated in Figure 8, an R frame 811 (may be followed by the same image block large 139848.doc -79- 201001177 糸R R frame (not shown is Λ /, l &quot ;> e| seconds to use the lx small image block around the character. Then, when the character 8G5 moves to a new area of the image, a smaller image block is used around the new area in the other -R frame 812. As noted above, various sizes and shapes can be used as "image blocks" while still complying with these basic principles. Although the cyclic I/p image blocks described above substantially reduce the data rate of the video stream. Value, but it does not completely eliminate ♦ values, especially in the case of fast-changing or highly complex video images (such as in movies, video games, and an application software). For example, during a sudden scene transition A complex frame may be followed by another complex frame that is completely different. Even if several image blocks can lead the scene transition only a few frame times' it does not help in this case because the new frame The material and the previous I image block are not In this case (and in other situations where a large number of images change even if not everything changes), the video compressor 4〇4 will determine that many, if not all, P-picture blocks are coded more efficiently as Image block, and resulting in a very large peak in the data rate of the frame. As discussed previously, it is only for most consumer-grade Internet connections (and many office connections), It is only not possible to "block" data that exceeds the maximum data rate available at 622 and the maximum data rate of 62 1 shown in Figure 6c. Note that the rated maximum data rate of 62 丨 (for example, "6 Mbps DSL") is essentially for consideration of purchase. The number of users of the Internet connection is sold, but it is not guaranteed to be a performance level. For the purposes of this application, it is irrelevant because we only focus on the maximum data rate available when streaming video over a connection 622 Therefore, in Figure 9a and Figure 9%, when the solution to the peak problem is described in 139848.doc -80-201001177, the rated maximum data rate is omitted from the graph and only The maximum data rate 922 is available. The video stream data rate must not exceed the maximum data rate available. 92 2. To solve this problem, the first thing the video compressor 4〇4 performs is to determine the peak data rate 941, which is stable for the channel. The rate of data processed by the ground. This rate can be judged by a number of techniques. One of the techniques is to gradually increase the data rate test stream from the hosted service 21 to the user terminal 415 (K4a and Fuji). And causing the UE to provide feedback on the level of packet loss and delay to the colocation service. When the packet is lost and/or the delay begins to show a sharp increase, it is an indication that the maximum data rate 922 is available. After that, the host The escrow service (4) can gradually reduce the data rate of the test stream until the client 415 reports that the test stream has been received within a reasonable period of time (packet loss and the acceptable level of delay is near minimum). This determines the peak maximum data rate state, which will then be used as the peak data rate for streaming video for a time lapse, and the peak data rate _ will fluctuate (eg, 1 family _ the other - the user begins to seriously use the Internet Connected), and the client 415 will need to constantly monitor the peak data rate 941 to see if the packet loss or delay is ( Λ ( (private unavailable maximum rate 922 drops below the previously determined peak data rate = (4) Value data rate 941. Class (4), if over time: the user knows that 415 finds that the packet is lost and the delay remains at the optimal level, then "the video compressor can be requested to slowly increase whether the f data rate is increased (for example, If home 7 can ~ T < another use! P position to the serious use of Internet connection), and Dan - people look until the packet is lost 139848.doc -81 201001177 Loss and / or higher delay indication has exceeded The maximum data rate can be used, and the lower level of the material data rate 941 can be found again, but the lower level may be higher than the level before the increased data rate of the test. The peak data rate 94 can be found by using this technique (and other techniques similar thereto) and periodically adjusted as needed. The peak data rate 9 can be used by the video compressor 404 to stream the video to use. The maximum data rate. The logic used to determine the rate of data capture can be implemented at the user premises 2U and/or on the colocation service 21. At the user premises 2U, the client device 415 performs the calculation. Determining the peak data rate and transmitting this information back to the colocation service 21; at the colocation service 210, the courier service at the colocation service 々 "service benefit 402" performs calculations based on the self-use: end 415 The peak data rate is determined by statistics (eg, peak loss, delay, maximum data rate, etc.) Figure 9a shows an example video stream with substantial scene complexity and/or motion

The data rate 934, which was generated using the previously described L-field, illustrated in Figures 7a, 7b, and 8 and illustrated in Figures 7a, 7b, and 8, is generated by the loop Ι/p shadow stupid sneakers. Video _ is configured to output at an average data rate lower than the peak f-rate 941 = contraction: video' and note that most of the time, the video stream data rate is below the peak data rate 941. Data rate 934 and Figure & The video stream data rate 634 (which uses a Ι/ρ/Β^/ρ frame to show a much smoother yield than the cyclic slab block compression) is 2 times the peak value of the frame 952 (which is close to 2 times the peak data rate) However, the value is corrected to be close to 4 times the peak data rate 944) τ, and the value of the data rate 941' is unacceptable. In practice 139848.doc -82· 201001177 The rate of 94! from the fast-changing video game is also less than ° action video, the peak value of the peak data rate 942 is exceeded; The peak of 2 times the value of the capital is difficult to silver, and more than 3 times the peak data rate 943 Han Han 辔, # 田其真贝 appear (for example, in the scene image. 贞枓 rate must produce good quality video solution One way of this problem is to hurt the zinc π „ 巧 巧 ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” ” The resulting video wheel has a bad mouthpiece, because the compression algorithm is extremely lacking in "bits." The resulting result is a dog-like transition or a fast motion 4 with compression artifacts, and in time, the user begins to realize that when there is a sudden False shadows always appear suddenly when changing or moving fast, and they can be quite annoying. Although the human visual system is quite sensitive to visual artifacts that occur during sudden changes or rapid movements, but in such situations The decrease in frame rate is not very sensitive. In fact, when these (4) changes occur, it seems that the human visual system is focused on tracking these changes, and if the frame rate temporarily drops from 60 fps to 3〇fps and then immediately return to 6〇, then the human visual system will not notice, in addition, in the case of a very sharp transition (such as a sudden scene change), if the frame rate drops to 2〇fps or Even Η fps and then immediately return to 60 fps, the human visual system will not notice. As long as the frame rate reduction occurs only occasionally, it seems to the human observer that the video system is continuously executed at 60 fps. The special 139848.doc-83-201001177 nature of the human visual system is utilized by the technique illustrated in Figure 9b. The servo state 402 (from Figures 4a and 4b) is used to stabilize the frame rate (in one embodiment, An uncompressed video output stream is generated at 60 fps. Each frame output is displayed every 1/60 second at time two. Starting from frame 961, each uncompressed video frame is rotated to a low delay. Video compressor 404, low latency video signal The device 4〇4 is compacted in a time less than a frame time. The frame is generated to produce a compressed frame for the first frame 1 981. Finely generated for the compressed frame i 981 The data may be larger or smaller depending on many of the factors previously described. If the data is small enough that the peak data rate 941 can be transmitted to the client 415 at frame time (W6 leap seconds) or less than one frame time. , then transmit it during the transmission time (xmit time) 991 (the length of the arrow indicating the duration of the transmission time). In the next frame time, the server 4〇2 produces an uncompressed frame 2 962, which will It is reduced to a compressed frame 2 982 and transmitted to the client terminal 4 at a peak data rate 941 during a transmission time 992 of less than one frame time. Next, in the next frame time, the server 4〇2 produces an uncompressed frame 3 963. When the uncompressed frame 3 963 is dusted by the video frame, the resulting compressed frame 3 983 is more data than can be transmitted in the frame time at the peak data rate 941. Therefore, at the =transmission time (2x peak) 993, it will occupy the portion of the frame time and the next frame time. Now, the server 402 generates another during the next frame time. The compressed frame 4 964 is output to the video compressor 404, but the data is ignored and illustrated by 974. This is because the video compressor 404 is configured to ignore the frame that was previously transmitted before it was still transmitted. The other uncompressed video maps that arrive at the time are not. Of course, the video decompressor of the client\39S4S.doc -84- 201001177 41 5 will not receive the frame 4, but it simply continues on the display device 422. Frame 3 is displayed for 2 frame times (ie, the frame rate is temporarily reduced from 60 fps to 30 fpS).

For the next frame 5, the server 4〇2 outputs an uncompressed frame: 965, which is compressed into a compressed frame 5 985 and transmitted in a frame during the transmission time. The video decompressor of client 415 decompresses frame 5 and displays it on display device 422. The server then outputs the uncompressed frame 6 966, which is compressed by the video compressor 4〇4 into a compressed frame 6 986, but the data obtained at this time is very large. The compressed frame is transmitted at the peak data rate 941 during the transmission time (4 times peak) 996, but it takes almost 4 frame times to transmit the frame. In the next three frame time periods r 曰 1, the video compressor 404 ignores & 3 frames from the server 4 〇 2, and the decompressor of the client 415 keeps the frame 6 stably displayed. The device 422 lasts for 4 frame times (i.e., temporarily reduces the frame rate from 6 至 to 15 fps). Finally, the server 4〇2 outputs the frame, and the video compressor 404 compresses it into a compressed frame 1〇987, and transmits it during the transmission time 997, and decompresses the client 415. The device decompresses the frame H) and displays it on the display device 422 and again - the secondary video (4) if it is restarted. Note that although the video compression is crying 4Γ)4·ί·畚·? +Difficult D° 4U4 discards the video frame from the video stream generated by the server 402, and makes a pottery | $1 will not discard the audio data (regardless of the form of the audio system), and杳 4 4 θ > When Shi Tian abandons the video frame, the video compressor 404 continues to compress the audio data and transmits it to the client 415, and the user terminal continues to decompress the audio data and provide the audio to the user for playback. Sound 139848.doc •85- 201001177 2 No matter what device. Therefore, during the period of discarding the frame, the audio continues, the male does not decrease (4) and the compressed, reduced video phase & the compressed audio consumes 100% of the frequency, and therefore does not The rate has a big impact. Although not illustrated in any of the data rate maps, there is always a data rate capacity reserved for the compressed audio stream within peak = material rate 941. 'Select the example just described in Figure 9b to illustrate how the frame rate drops during peak data rates, but not stated, when using the previously described ι/p image block technique, these data rate peaks And the consequent frame of Μ is rarely 'even in high scene complexity/high action sequences (such as those appearing in games, movies, and some-application software). Therefore, the reduced frame rate is rare and temporary, and the human visual system does not detect them. Depending on the frame rate reduction mechanism that will be ruined:: view: stream data rate, then in the video stream;: value (6), and 4 times peak 955 has been, salt / / reduced to flatten 2x ridges are flattened by a 4x peak 955, and 941 "rate 934 remains at or below the peak data rate. Therefore, using the techniques described above, consumer-grade Internet connectivity = Internet And in addition, on the LAN (for example, 1〇〇Mh 'last pass to action video stream. 1 〇〇Mbs Ethernet or 8n way) or private network (for example, data office ~ line network

Mbps connection) on the office ~ do &to; between H) 0 ... in 'can be transmitted without peaks 139848.doc • 86- 201001177 motion video stream to make multiple users (for example, to 4 • 5μ_Transfer 6〇 plus 192 _) can be connected using LAN or shared private data without flooding the network or network switch backplane. Data Rate Adjustment π In the embodiment, the colocation service 210 initially evaluates the channel's available maximum data rate 622 and delay to determine the appropriate data rate for the video stream and then dynamically adjusts the data rate in response thereto. In order to adjust the data

The rate, colocation service 210 can, for example, modify the image resolution and/or the number of frames per second of the video stream to be sent to the client 415. In addition, the colocation service can adjust the quality level of the compressed video. When changing the resolution of the video stream (e.g., from 1280 720 resolution to 64 〇 >< 36 〇), the video decompression logic 412 on the client 415 can scale the image to maintain the same on the display glory. Image size. In one embodiment, the colocation service 2) suspends the game in the event that the channel is completely logged out. In the case of a multiplayer game, the colocation service reports to other users that the user has quit the game and/or paused the game for other users.丢弃 A discarded or delayed packet. In one embodiment, if the data is lost due to packet loss between the video compressor 404 and the client 415 in FIG. 4a or the servant, or cannot be decompressed due to being too late to arrive. And the packets satisfying the delay requirement of the decompressed frame are lost without being received in order, and the video decompression logic 412 can reduce the visual artifacts. In the streaming I/P frame implementation, if there is a lost/delayed = packet, the entire screen is affected, which may cause the screen to completely collapse. 139848.doc •87- 201001177 - Time period or other screen wide viewing The lost/delayed packet causes the I frame to be 'v. "For the column, if you lose the front, the solution will lack the missing p frame for receiving the new 1 frame with pf, then it will affect the reference following the P frame. If there is a frame before it appears For how long, this will have a screen frame of the screen, . ', with a car owner or a shorter visual shadow. Using the interlaced 的' package shown in Figures 7a and 7b is unlikely to affect the entire screen. : Image block contained in the block loss/delay packet. If 匕= it only affects the affected meat brigade, the data of the shirt image block is in the individual packet/transport', then if the packet is lost, it only affects - Image block. The duration of the image will depend on whether the image block is lost or not, and how many graphs will be spent before the 1 block appears in the case of a ghost loss, but 'assuming different images on the screen The block is updated by the J frame very frequently (perhaps every frame), and even if the image block on the screen is affected, other image blocks may not be affected. In addition, if an event causes the f-packet to be lost at the same time (for example, a temporary interruption in the power of the adjacent DSL line), some image blocks will be more affected than their image blocks, but because - Some image blocks will be updated quickly by the new image block so they are only temporarily affected. In addition, in the case of the implementation of the stream/p frame, not only the 1 frame is the most critical frame, but the J frame is extremely large, so if there is an event causing the discarded/delayed packet, then a much smaller t image is used. The block has a higher probability of being affected than the I frame (that is, if any part of the I frame is lost, it is impossible to decompress the J frame). For all of these reasons, the use of Ι/p blocks causes a much smaller visual artifact when the packet is discarded/delayed compared to the Ι/P frame. 139848.doc -88 · 201001177 An embodiment attempts to reduce the effects of lost packets by intelligently encapsulating compressed video blocks into TCP (Transmission Control Protocol) packets or UDP (User Datagram Protocol) packets. For example, in one embodiment, the image block is aligned with the packet boundary whenever possible. Figure 10a illustrates how an image block can be encapsulated within a series of packets 1001_1005 without implementing this feature. Specifically, in Figure 1a, the image block crosses the packet boundary and is inefficiently packaged such that the loss of a single packet results in the loss of multiple frames. For example, if the packet 1003 or 1004 is lost, three image blocks are lost, resulting in visual artifacts. In contrast, Figure 10b illustrates image block packaging logic 1010 for intelligently encapsulating image blocks within a packet to reduce the effects of packet loss. First, image block encapsulation logic 1010 aligns the image block with the packet boundary. Therefore, the image blocks ΤΙ, Τ3, Τ4, Τ7ΑΤ2 are aligned with the boundary of the packet just .5. Image block encapsulation logic also attempts to combine image blocks into packets in a potentially more efficient manner, while leaving the packet boundaries. Based on the size of each of the image blocks, the image blocks are combined in a package 1〇〇1; the Τ3 and Τ5 are combined in a package 1〇〇2; the image blocks 4 and D8 are combined. In a packet 1003; the image block is added to the packet 1〇〇4; and the image block τ2 is added to the packet 丨005. Therefore, under this scheme, the loss of a single packet will result in the loss of 2 image blocks (instead of the loss of the 3 image blocks as illustrated in Figure 〇a. & one of the embodiments shown in Figure 10b) An additional benefit is that the image blocks are transmitted in a different order in which they are displayed within the image. If the neighboring packets are due to the same event that interferes with the transmission (which would affect the areas on the screen that are not close to each other 139848.doc -89 - 201001177 domain) Loss of shadow. This method produces a less noticeable error on the display using forward error correction (FEC) technology to protect the video stream from being affected by channel errors. Knowledge 1 such as Reed _ Solomon and v Secrets of the coffee technology generated error correction information and attached to the information transmitted via the communication channel. If the error occurs in the basic information (for example, the old box), then Can be used to correct (4) errors. The FEC code increases the f rate of the transmission, so ideally it is only used when it is most needed. If the data is being sent 'and it will not cause very noticeable = visual artifacts' Do not use FEC code to protect (4) Bu: In other words, the p image block immediately before the missing I image block will produce a visual artifact of _ seconds (that is, the image block on the screen is new). It can hardly be detected by the human eye. As the p image block is further backward from the I image block, the loss of the P image block becomes more and more noticeable. For example, if the image block cycle pattern is available again in the image block If the previous I image block is followed by 15 P image blocks, if the image block immediately after the τ image block is lost, it will cause the image block to display the incorrect image for 15 frame time (at 60 fps). Next, he will be 25 〇 milliseconds. The human eye will easily detect the interruption of the flow of 250 milliseconds. Therefore, the ρ image block is farther backwards from the new I image block (that is, the closer the P image block follows the image block) ), the more attention is paid to the artifacts. As discussed earlier, in general, the closer the p image block follows the I image block, the smaller the data used for the ρ image block. Therefore, the I image block is followed. The P image block is not only protected against loss 139848.doc -90- 2010 01177 is more critical and smaller in size. Furthermore, in general, the smaller the data to be protected, the smaller the FEC code required to protect it. Thus, as illustrated in Figure 11a, in one embodiment, Due to the importance of the j image block in the video stream, only the image block has the FEC code. Therefore, the FEC 1101 contains the error correction code for the I picture block 1100 and 17] £: 11 〇 4 contains the I picture block. Error correction code of 1103. In this embodiment, no FEC is generated for the p-image block. In one embodiment illustrated in Figure 11b, the p-image block that causes visual artifacts at the most lucrative b when lost is also The coffee code is generated. In this embodiment, the FEC 1105 provides an error correction code for the first 3 P-picture blocks but not for the following p-picture blocks. In another embodiment, FEC Ma is generated for the p-image block with the smallest data size (JL will recognize & pe丄V, liver mussel toward the P-image block that appears first after the j image block, It is the most critical for protection). In another embodiment, instead of sending the coffee code along with the image block, the image block is transmitted twice, each time in a different packet. If one packet is lost/delayed, another packet is used. In one example of the sinus rhodium palladium shown in Fig. 11c, FEC codes 1111 and 111 for respectively transmitting the enthalpy packets (1) 〇 and (1) 2 from the host escrow service simultaneously with the video are generated. 3. Maintaining video stream neutrality. ^ ^, the integrity of the news is particularly important, because the distortion of the θ (for example, ticking *. X new voice) will lead to a particularly inconvenient user experience. The FEC code helps ensure that the reproduction is reproduced without distortion. In another embodiment, the intranet is not sent, but the audio data is transmitted twice, and each == lean-transmitter is transmitted in a different packet. i3984S.doc •91 - 201001177 If one packet is lost/delayed, another packet is used. In addition, in the embodiment illustrated in the figure, the coffee code (1) is used for (10) for the user input command (for example, press button M) 112G for uplink transmission from the client terminal 415 to the colocation service (10). 1122. This is important because 'missing press or mouse movements in video games or applications can lead to an undesirable user experience. In another embodiment, the FEC code is not transported together with the user input command data, and the user input command data is transmitted twice, each time in a different packet. If the packet is lost/delayed, an additional packet is used. ▲ In the case of 贝 〜 , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , What part of the application is Qing. The "quality" of the evaluation frequency may include functions such as estimating packet loss, delay, etc. as described above. If the channel is particularly unreliable, the colocation service 210 can apply FEC to all 1 video blocks, P video blocks, audio and user commands. In contrast, the 'right channel reliable' % colocation service 2 〇 can only request C for audio and user commands, or may not apply to audio or video, or may not use FEC at all. Various other arrangements of FEC applications can be used: At the same time, the broad principles are still adhered to. In an embodiment, the colocation service 2 1 0 constantly keeps the video channel strong, and changes the FEC policy accordingly. In another embodiment, 'refer to FIG. 4a and FIG., when the packet is lost/delayed', resulting in loss of image block data, or if the FEC cannot correct the missing image block data due to the particularly bad packet loss, Use 139848.doc •92·201001177 terminal 415 to evaluate how many frames remain before receiving a new 1 image block and compare it with the round trip delay from the user terminal 415 to the colocation service 2ig. (4) Less than the number of frames before the new I image block should arrive. Then the client 415 sends a message to the colocation service 21 to request a new I block. This message is routed to the Video Recipient _, and it does not generate a P-Video block for the image block in which the data has been lost, but instead generates an If; image block. Assuming that the systems shown in SJ4a and Figure 4b are designed to provide a round trip delay of typically less than 80 milliseconds, this results in the image block being corrected to within 8 milliseconds (at 60fPs, the frame has a duration of 6.67 milliseconds, So in the full frame time, the 80 millisecond delay will result in a corrected image block in 83 to 33 milliseconds, and 83.33 milliseconds as 5 frame time, which is an interesting interruption, but far less than (for example) for 15 Frame 25 〇 millisecond interrupt reference: Note). When the compressor 404 is out of its normal cyclic order to generate such a block of image blocks, if the I block will cause the bandwidth of the frame to exceed the available bandwidth, the compressor 404 will delay the loop of the other blocks, so that The other image blocks receive the P image block during the frame time (even if the image block will be an I image block during the frame) and then the normal loop will continue from the next frame, and will usually An image block that has received 1 image block in the previous frame will receive an I image block. Although this action temporarily delays the stage of the ruler loop', it will usually be visually unobtrusive. Video and Audio Compressor/Decompressor Implementation Figure 12 illustrates a particular embodiment in which multiple cores and/or multiple processors 1200 are used to compress eight image blocks in parallel. In one embodiment, a dual-core processor, a quad-core Xe〇n CPU computer 139848.doc-93·201001177 system, executed at 2.66 GHz or higher, is used as an independent process to implement open source x264 h.264 compression. . However, various other hardware/software configurations can be used while still complying with these basic principles. For example, each of the Cj>u cores can be replaced by an H,264 compressor implemented in an FPGA. In the example shown in Figure 12, the cores 1201 - 1 208 are used to process the I and Q blocks simultaneously as eight separate threads. As is well known in the art, current multi-core and multi-processor computer systems are inherently integrated with systems such as Microsoft Windows XP Professional (64-bit or 32-bit) and Linux. Ability to perform multi-thread processing. In the embodiment illustrated in Figure 12, since each of the eight cores is only responsible for one image block, it operates largely independently of the other cores, each performing a separate instantiation of X264. Use ϋνι capture cards based on PCI Express ( (such as from NetheHands

Microtronix of 〇osterh〇ut's Sender video imaging development board) captures uncompressed video at 640x480, 800><600 or 128〇><.72〇 resolution, and the FPGA on the card Direct memory access (DMA) is used to transfer the captured video to the system ram via the DVm stream. The image blocks are arranged in a 4χ2 configuration 12G5 (although this is illustrated as a square image block, in this embodiment it has (10) X24G resolution). (10) Each—the real (four) is configured to compress one of the eight 160><240 image blocks, and they are synchronized so that at the beginning! After the image block is compressed, each core enters a second cycle, and each frame is out of phase with another frame, and the small U phase is compressed - the I image block is followed by seven P image blocks as shown in FIG. The resulting compressed 139848.doc -94 - 201001177 reduced image blocks are combined into a packet stream at each frame time using the techniques described above and the compressed image blocks are then transmitted to the destination client terminal 41 5 . Although not illustrated in Figure 12, but if the data rate of the combined 8 image blocks exceeds the specified peak data rate 941, then all 8 χ264 processes will temporarily suspend the time frame up to the necessary frame time until the 8 for the combination has been transmitted. The data of the image block. In one embodiment, the client 415 is implemented as a software on a PC that implements eight instantiations of FFrnpeg. The receiving process receives eight image blocks and routes each of the tiles to FFmpeg. The FFmpeg instantiates the decompressed image block and reproduces it to the appropriate image block location on display device 422. The client 41 5 receives the keyboard, mouse or game controller input from the input device driver of the PC and transmits it to the server. The server 402 then receives the input device data received by the application and applies it to the game or application executed on the server 4〇2, and the server 4〇2 executes the wind〇WS2pC using the Inty 216 GHz dual-core CPU. The server 4〇2 then generates a new picture C, and the second picture is output by its DVI output from the motherboard-based graphics system or via the DVI output of the NVIDIA 8800GTX PCI Express card.

At the same time, Servo 402 outputs audio generated by the game or application via its digital audio output (eg, S/PDIF), which is coupled to the digital audio of the dual quad-core Xeon-based PC that implements video compression. Input. The v〇rbis open source audio compressor is used to compress audio at the same time as video, using whatever core can be used to process the thread. In the case of knowing, the core of compressing the image block is first to perform the audio pressure 139848.doc •95· 201001177, the ubiquitous, the compressed video is transmitted together with the compressed video, and the Yorbls audio is decompressed on the 5 To decompress the compressed audio. The colocation service server center distributes light passing through a glass (such as an 'optical fiber) at a certain rate of light at a speed in the vacuum and thus can determine the exact propagation speed of the light at the fiber. However, in the real &, considering the use of routing delay, transmission inefficiency and other consumption, I observed that the optimal delay on the Internet is closer to 50/ of the speed of light. Transmission speed. Therefore, the optimal mile round trip delay is 々22 ,, and the optimal 3000 mile round trip delay is about 64 milliseconds. Therefore, a single server on the US coast will be too far apart to serve the client on the other coast (which may be up to 3000 miles) with the desired delay. However, as illustrated in Figure 13a, if the colocation service 2i server center 1300 is located at the center of the United States (eg, old (10), Nebraska, etc.), the distance to any point in the continental United States is approximately The i5 mile or less 1500 mile 'round trip internet latency can be as low as 32 milliseconds. Referring to Figure 4b, note that although the worst case delay allowed by the user ISP 453 is 25 milliseconds, it is generally observed that in the case of DSL and cable modem systems, a delay of approximately 1 〇 _ 丨 5 milliseconds is observed. Further, Fig. 4b assumes that the maximum distance from the user premises 211 to the colocation center 210 is 1 mile. Thus 'in the case of the typical 15 millisecond user lsp round trip delay used and the maximum internet distance of 1500 miles for a 32 millisecond round trip delay, 'from the moment the user actuates the input device 42 1 The total round trip delay for seeing a response on display device 422 is 139848.doc • 96· 201001177 l + l + 15 + 32 + l + 16 + 6 + 8 = 80 milliseconds. As a result, an 80-millisecond response time is typically achieved over an Internet distance of 15 miles. This will allow any user location in the US mainland with a sufficiently short user ISP delay of 453 to be accessed in a centrally located single server center. In another embodiment illustrated in Figure 13b, the colocation service 21" server centers HS1-HS6 are strategically located around the United States (or other geographic area), a particular larger colocation service server center Positioned close to high population centers (eg, HS2 and HS5). In one embodiment, server centers HS1-HS6 exchange information via network 1301, which may be an internet or private network or a combination of both. In the case of multiple server centers, users with high user ISP delays 453 can be served at lower latency. The distance on the Internet is indeed a factor that affects the latency of travel back and forth via the Internet, but other factors that are largely unrelated to latency also work. Sometimes the packet flow is routed through the Internet to a distance back and back again, resulting in a delay from the long loop. Sometimes there are routing devices that are not operating properly on the path, resulting in delays in transmission. There are sometimes traffic that overloads the path, which introduces delays. In addition, sometimes, there is a failure to prevent the user's ISP from routing to a given destination. Therefore, although the universal Internet usually provides connections from one point to another with fairly reliable and optimal routing and latency, this fairly reliable and optimal route and delay is largely determined by distance (especially It is the case where the V is routed to the long-distance connection outside the user's local area.) X The sin and delay are not guaranteed and often cannot be used 139848.doc •97- 201001177 User's place to general Achieved by a given destination on the Internet. In an embodiment, when the user client 415 is initially connected to the colocation service 210 to play a video game or use an application, the client is in startup with the available colocation service server center Hs丨_HS6. Each of them communicates (eg, using the techniques described above). If the delay is low enough for a particular connection, use the connection. In an embodiment, the client communicates with all of the copier service server centers or a subset of the colocation service server center, selecting the colocation service server with the lowest latency connection. The client can select the service center with the lowest latency connection, or the server center can identify the server center with the lowest latency connection and provide this information (for example, in the form of an internet address) to the client. If the specific colocation service is overloaded with the server center and/or the user's game or application can tolerate the delay to another, less loaded colocation server server center, the client 415 is redirected to Another—host hosting service server center. In this case, the game or application being executed by (4) will be temporarily placed on the server at the user's overloaded server center and the game or application status data will be transferred to another hosted service server. The server 402 at the center of the device. Then you will restart the game or the app. In an embodiment, the colocation service 210 will wait until the game or application reaches the natural pause point (10), between the levels in the game 'or after the user starts "protection:::" in the application. In a further embodiment, the host escrow service waits until the user activity stops for a specified period of time (eg, α 139848.doc -98 - 201001177) and will then initiate transmission at that time. In one embodiment, the colocation service 21 subscribes to the Internet bypass service 440 of Figure 14 in an attempt to provide a guaranteed delay to its client. As used herein, the Internet is orbiting The service system provides services for private network routes with guaranteed characteristics (eg, delay, data rate, etc.) from one point to another on the road. For example, if the host is on behalf of The service provider is using the AT&T DSL service provided by San Francisc〇, which is based on a large number of users (not routed to the central office based on AT&T San Francisco). 210 will Hiring from a service provider (possibly AT&T itself or another provider) between one or more of the central office based on San Frandsc〇 and the server center for colocation service 210 The capacity is privately connected to the bait. Then, from all the colocation service server centers HS1-HS6 via the universal Internet to the users of the DSL who use DSL in the buckle (3), the material can be changed to Use private data connections. Although private data connections are usually more expensive than routes via the universal Internet, as long as they maintain a colocation service (10) - a small percentage is connected to the user, the total cost impact is low and the user Experience a more complete service experience. In the case of a power failure, the server often has two backup powers: the first layer is usually from the battery (or from the ready-to-use source of the alternative, such as keep running and attached The backup power connected to the generator's flywheel, which provides power immediately when the mains fails, and keeps the server center running. If the power failure is temporary and the mains are fast Back (example 139848.doc •99· 201001177 eg, within one minute), turn. However, if the power failure lasts twice, the system maintains the center of the feeding machine (for example, the diesel engine power supply ^% cycle, then Usually the power generation can be started. These generator poles:: batteries and generators as long as they have a fuel server center usually self-respecting P-because because they must be able to generate as much power as possible from the mains. In the embodiment, Φ her, the shared (four) service HSl-HS5 each of them can be in the ongoing tour (four) ... with power failure, the application state data is suspended from the i program 'and then the game or heart The server is 4〇2, and then: the service 2 is transferred to the other feeding device α ^ ^ .. The test will inform the user of the user 415, then you will use the new to feed The server is 4〇2. Fake $ such occasions may be transferred to a hosted service server that does not provide the best delay (also gp users will only have to tolerate higher latency time lags: continuation time) may be acceptable, it will Allows the option to transfer the range of wide payouts. For example, given the difference between the United States and the United States, users on the East Coast may be sleeping at 11:30 PM, while users in the West Sea are starting to peak in video games at 8:3 〇pM. On the other hand, there is a power failure in the colocation service server center on the West Coast. There may not be enough West Coast servers 4〇2 for handling all users at the other colocation service server center. In this case, some users may be transferred to the colocation service server center with the available server 402 on the East Coast, and the only consequence for the user is a higher latency. Once the user is transferred from the server center where the power is lost, the server center can then start its server and set up the 139848.doc 100-201001177 orderly cut-off, so that it can be used in the life/bucket Or cut off all equipment immediately before power backup. In this way, the cost of the motor is exhausted. In order to avoid the use of the Provider Center, in the embodiment, during the serious manned period of the colocation service 2ig (or due to the peak user loading θ device center failure), based on the user is using # The delay in the game or application being used by the servo user, one or the other, is required to transfer the user to another center. Therefore, the use of ρ for games that require low latency will be preferred for the low-latency server connections that are available for the line. Host Escort Service Features Figure 15 illustrates an embodiment of the components of the feeder center for the colocation service 210 utilized in the following description of the features. As with the colocation service 210 illustrated in Figure 2a, the components of the servo write center are controlled and coordinated by the colocation service 21〇 control system 4〇 unless otherwise conditional. The person from the user client 415 埠 Internet traffic measurement guide 皋 route 1502. Usually, 'incoming Internet traffic 15〇1 will enter the feeding center through the high-speed fiber connection to the network, but any network connection components with sufficient bandwidth, reliability and low latency will be sufficient. of. The incoming and outgoing route 15〇2 is a network (which can be implemented as an Ethernet, fiber channel: way, or via any other transport component) switch and supports such traffic. The routing servo system's takes the arriving packets and routes each packet to the appropriate application/game server l52i_i 525. In one embodiment, the packet delivered to the particular application/game server represents a subset of the data received by the client 139848.doc -101 - 201001177 and/or may be other components within the data center (eg , network connection components, such as gateways and routers, to translate/change, and in some cases, for example, if a game or application is executing concurrently on multiple servers, routing packets to one at a time The above server 152]-1525. The RAID arrays 1511_1512 are connected to the inbound routing network 1502 such that the application/game server 1521 array is added to the " RA core (10) 12. = fetch: array 1515 (which can be implemented as multiple RAID arrays) is also connected to珲 Route 1502, and the data from the RAID array 1515 can be read from the application/game server 152W525. The incoming route (10) can be used in a variety of prior art network architectures (including tree structure switches, Internet access) Ϊ́5〇ι implemented in its roots; implemented in a mesh structure interconnecting all kinds of devices; or as a sub-network sequence of interconnects (concentrated traffic among interworking devices and centralized traffic isolation among other devices) One type of network configuration is S AN, which, although commonly used for storage devices, can also be used for general high-speed data transfer between devices. Moreover, applications can have as many routes as they want (10) Network Connections ^ For example, the feeder may have a network connection to a subnet attached to the RAm array 1511-1512 and another network connection to a subnetwork attached to other devices. The hoppers 1521-1525 can be configured identically, somewhat or all differently, as previously described with respect to the server 402 in the example illustrated in Figure 4a. In one embodiment, each The use of the host agency service is usually at least - application / game 139848.doc -102 - 201001177 device 1 521 -1525. For the sake of simplicity of the description, it will be assumed that the given user is using the application / game servo The device 1521, but the plurality of servers can be used by a user, and the plurality of users can share a single application/game servo „ 1521-1525. The user sent from the user terminal 41 5 (as described previously) The control input is received as an incoming Internet service 15〇1 and routed to the application/game server 1521 via the incoming route 1502. The application/game server 1521 uses the user's control input as the on-server The control input of the game or application executed on the device, and calculate the video and

The next frame of the audio associated with it. The m/game (4) H (10) then outputs the uncompressed video/audio 1529 to the shared video compression 1530. The application/game server can output uncompressed video via any component (including - or ^ ultra-high speed Ethernet connection), but in a implementation of the video system output via DVI connection, and audio and other The compression and communication channel status information is output via a universal serial bus (u called connection. Open video compression -, - service 1 :> 2 1 1525 uncompressed video and audio. This compression can be completely Hardware or hardware implementation of the software. There may be a dedicated repeater for each application/game server 1521_1525, or if the I shrinker is fast enough, it can be compressed using line shrinkage from - More than one video/audio of m21/525. For example, at 6〇, the video frame time is 16.67 milliseconds. If the compressor can compress 1 frame in milliseconds, then the compressor can be used. By taking the wheel from the server, the dust is from up to 16 applications/games 139848.doc •103- 201001177. fl / θ汛, the compressor saves each video/audio compression State of the process and when it is Switching the background from the loop in the video/audio stream of the server. This results in a real cost savings for the compressed hardware. Since different feeders will be at different times, so in the embodiment, the compressor resources are in a common pool 153 for storing a common storage member (for example, RAM, flash memory) in the state of each dust reduction process, and when the server 1521-1525 is intact and ready to be compressed, the control member It is judged that the compressed resource is available, and the compressed resource is provided with the state of the shrinking process of the feeding device and the uncompressed video/audio frame to be compressed. Note that the compression is used for each server. One of the states of the process includes information about the compression itself, such as the frame buffer data of the previous frame (which can be used as a reference for the image block), the resolution of the video output, the quality of the compression, and the image quality. Block structure; allocation of bits per image block, compact quality, audio format (eg stereo, surround sound, Dolby® AC-3). However, the compression process status also includes the following Channel channel status information: peak data rate 941, and whether the previous frame (as illustrated in the figure) is currently being output (and therefore the current frame should be ignored), and potentially whether there should be considerations in compression (such as , excessive packet loss) affects the channel characteristics of the compression decision (eg, in terms of the frequency of the image block, etc.) because the peak data rate 941 or other channel characteristics change over time, such as by monitoring each user to monitor the user The application/game server of the data sent by terminal 415 is determined by 1 5 2 1 -1 5 2 5, so the application/game server 1 5 2 1 -1 5 2 5 sends the relevant information to the shared hardware. Compressed 139848.doc -104- 201001177 1530. - Using hardware pressure & 1530 also uses the components of the components such as those previously described to packetize the video/audio, and, where appropriate, apply the FEc code to 'copy specific material, or take other steps so that The ability to ensure that the video/audio data stream is received by the client 415 and decompressed with feasibility is substantially ensured. Some applications (such as the applications described below) require the video/audio output of a given application/game stitcher 1521-1525 to be available at the same time (or in multiple formats). If the application/game servo state 1521-1525 thus informs the shared hardware to compress 153 resources, the uncompressed video tone 1529 of the application/game feeder 1521_1525 will be in different formats, different resolutions and , or in the same packet / error = structure in the same day "shrink. In some cases, some of the shrinking resources can be shared in the multiple compression process of the video 7 audio (for example, in the Xu Ding...& The step of adjusting the shadow to multiple sizes before applying compression. If you need to output images of different sizes, = this step can be used to simultaneously servo several compression processes.) In other cases, separate compression resources will be required for each format. In any case, the compressed video will be used for all the various resolutions and formats required for a given application/game server 1521_1525 (the same)... Output to the outgoing route i 54. In one embodiment, the output of the compressed audio 1539 is in UDP format, so it is a one-way packet routing network 1540 comprising a series of routing servers and switches, 139848.doc -105- 201001177 System] Routing Known Benefits and Parental Converters will compress each compressed video/audio stream via the Internet Protocol 599 interface (which will typically connect to the Internet's fiber) Interface) to the desired user or other destination and/or back to the delay buffer 1515, and/or back to the routing route 15〇2, and/or output via a private network (not shown) For video distribution. Note (as described below). Routing 154 can output a given video/audio stream to a destination at the same time. In the example, this uses the Internet Protocol (IP). Multicasting, where the broadcast is intended to simultaneously stream to a given UDP stream of multiple destinations, and the broadcast is repeated by the routing server and switching 3 in the outgoing routing 154〇. Through the Internet to multiple users The client 415, via the routing route 15〇2, to the plurality of application/game servers 1521_1525, and/or to the one or more delay buffers 1515. Thus, compressing the output of the given server 1521_1522 into one or more Format and direct each compressed stream to one or more destinations. Additionally, in another embodiment, if multiple application/game servers 1521-1 525 are simultaneously used by a user (eg, 'at For generating a parallel processing configuration with a complex field of 3D output) and each server produces a portion of the resulting image, the view of the plurality of servers 1 52 1 I 525 can be output by the shared hardware compression 153 The composite frame is assembled and processed from the other side as described above as if it were from a single application/game server 1521-1525. In the embodiment, a copy of all the video generated by the application/game server i 52 i _ 1525 (at least the resolution of the video viewed by the user or higher resolution) is recorded in the delay buffer 1515 for a duration. To 139848.doc -106 - 201001177 Less than a certain number of minutes (15 minutes in one embodiment) This allows each user to "rewind" the video from each session to check previous work or performance (in the game) In this case, in the embodiment, each compressed video/audio output (four) stream routed to the user client 415 is also multicast to the delay buffer 1515. When the video/audio is stored in the delay buffer When the device is on the delay, the directory on the delay buffer 1515 provides the network address of the application/game server (10), which is the source of the delayed video/audio, and the delay buffer can be found on the stomach 1515. According to the position between Jingxun and the reference. The live, instantly viewable, instantly playable game application/game server n_1525 can be used not only to execute a given application or video game of the user, but also to be used to support hosting via the host. The user interface application of the hosted service 210 of the navigation and other features of the service 210. The screen shot of the user interface application is shown in Figure 16 ("Game Viewfinder" screen). This particular user interface screen allows the user to view 15 games played (or delayed) by other users on the spot. Each of the "thumbnail" video windows (e.g., deleted) is a live video window in motion that displays video from the user's game. The view shown in the thumbnail may be the same view that the user is looking at or it may be a delayed view (eg, if the user is playing a wrestling game) the user may not want other users to see where they are hidden and The basin can optionally hold the spleen and choose to delay any view of its game play for a period of time (eg, 10 minutes). The view can also be a camera view of the game that is different from the view of any user. The user can select the game to be simultaneously viewed via the 139848.doc .107·201001177 menu selection (not shown in this description) based on a variety of criteria. As a small sample of exemplary choices, the user may select a random selection of games (such as those shown in Figure 16), all-category games (both played by different players), only top players of the game, games A given level of play, or a lower level player (eg, if the player is learning the basics), a player who is a "partner" (or a competitor), a game with the most number of viewers, and the like. Note that in general, each user will decide whether the view from their game or application can be viewed by others, and if so, determine which other people can view and when they can be viewed by others, and decide whether the video can only be View with delay. An application/game that produces the user interface screen shown in FIG. 16: the server 1521-1525 is requesting each user (the application/game server xian-1525 is requesting a game from the user) The application/game server 1521-1525 sends a message to retrieve the 15 video/audio feeds. The message is sent via the incoming route 15〇2 or another network to include the size and format of the requested video/audio and will identify the user of the user interface screen. You can choose to choose the "Piracy" mode and not allow (4) other users to view the game's view of the game ("from its view point or from another view point), or as described in the previous paragraph, the user can choose Allows viewing of video/audio from its games for delayed viewing of video/audio. User application/game word processor 1521·: Calling it to accept and accept the request to allow its video/audio to be viewed, then this will be confirmed to the requester and it will also notify the shared hardware compression 153. Ϊ 3984S .doc 201001177 Generates an additional compressed video stream of the requested format or screen size (assuming the format and screen size are different from the format and screen size already generated), and it will also indicate the destination of the compressed video (ie, requesting the servo) Device). If the requested video/audio is only delayed, the requesting application/game server 1521-1525 will be notified as such, and it will look for the location of the video/audio in the directory on the delay buffer 1515 as a delay. The video/audio source of the application/game server 1521_1525 network address is obtained from the delay buffer 1 5 1 5 to obtain delayed video/audio. Once all such requests have been generated and processed, up to 5 live broadcast thumbnail-sized video streams are routed from the outgoing route 1540 to the incoming route 1502' to the application/game server that generates the user interface screen. 1521_1525, and will be decompressed and displayed by the servo. The delayed video/audio stream may be oversized, and if so, the application/game server i52i_ 1 525 will decompress the streams and scale the video stream down to the thumbnail size. In one embodiment, the request for audio/video is sent to a central "management" service (not shown in Figure 5) similar to the host's proxy service control system (and managed by the central "management" service) The central "management" service then redirects the requests to the appropriate application/game server 1521-1525. Moreover, in an embodiment, a request may not be required because the thumbnails are "pushed" to the user of the user who is allowed to. All of the simultaneous audio from 15 games can produce harsh sounds. The user can choose to mix all the sounds in this way (perhaps to get the "flaps" from all the actions being viewed), or the user can choose to listen to only one game at a time. & Single Tour 139848.doc -109· 201001177 The choice of play is by moving the yellow selection box _ to the given game to 1 (page color frame can be moved by using the keyboard as e #丄<月] The shell key, the bad is done by moving the slider, by moving the joystick or by pushing a direction button on another device such as a mobile phone. Once selected a single: the audio of the game. Also, the exhibition does not play - Beixun 1602. In this game, for example, 'published trademarks ("EA") and the game labeled "Need for Fly. Valley" and the color strips indicate the number of people who play or view the game at a particular time under relative conditions. (In this case, many, the game is "hot"). In addition, "can t _ heart," 'heart does not exist 14 5 players are actively playing the Need for Speed game 8 different instantiations (that is, the game can be played by individual player games or multiplayer games) ), and there are 68 viewers (one of the users). Note that these statistics (and 1 other statistics) are collected and stored by the hosted service control system: on the arrays 1511-1512 for the purpose of maintaining the colocation service 21 〇 operation and for appropriate Billing the user and paying the fee to the publisher who provided the content. Since the action by the service control system 401 is recorded, the statistics are collected by the individual application/game server. 1521-1525 is reported to the service control system 4〇1. For example, when the game is being viewed (and when the game is stopped viewing), the application/game (4) 152 1525 executing the game viewfinder application sends a message to the colocation service control system 40i to enable The colocation service control system 4〇1 can update how many games are in the view of the statistics. - These statistics can be used by user interface applications such as the 'game viewfinder application.' 139848.doc -110- 201001177 If the user clicks on the start button on his input, he will see the thumbnail in the yellow frame and the thumbnail video will be live broadcast to full screen size. This effect is shown in Figure 17. Note that the video window mou is slightly increased in order to implement this effect. The application/game feed server 1521-1525 requests the application/game server 152H525 from the selected game to have the full screen size of the game routed to it. A copy of the video stream (at the resolution of the user's display device 422). The game application/fourth device (4) 1521_1525 informs the shared hardware compressor (10) that the thumbnail size of the game is no longer needed (unless another application/game feeder 1521-1525 requires such a thumbnail), and # The guide shared hardware compressor 153 sends a copy of the full screen size of the video to the application/game pad 1521] 525 of the enlarged video. The user playing the game may or may not have a display device 422 having the same resolution as the display device of the user who zooms in on the game. D, other viewers of the game may or may not have the same display crying 422 as the user who zoomed in on the game (and may have different audio playback components, such as stereo or surround sound). Because A, the shared hardware M 153 determines whether a suitable compressed video/audio stream has been generated that satisfies the requirements of the user requesting the video/audio stream, and if a suitable compressed video/audio stream does exist, Then, the shared hardware compressor 153 notifies the bee route 154. Routing the duplicate of the stream to the application/game server ΐ52ι_ΐ 525 that amplifies the video, and if the appropriate compressed video/audio stream does not exist, then the compressed video is adapted to the user's other copy and the material routing will (4) Send back to the entrance road ^ 〇 2 and enlarge the video application / game feeding device (1) Bu ^ 5. 139848.doc ΊΗ - 201001177 This server that now receives the full screen version of the selected video will decompress the full screen version and gradually scale it up to full size. Figure 18 illustrates how the screen looks after the game is fully enlarged to full screen and the game is displayed at full resolution of the user's display device 422 (as indicated by the image pointed to by arrow 1800). The application/game server 1521_1525 executing the game viewfinder application sends a message to the other application/game server 1521_1525 that provides the thumbnail to indicate the thumbnails. The figure no longer needs and hosts the service control server 4 to the host. 〇丨 Send a message to indicate that no other games will be viewed. At this point, the only display that appears is the top 1 801 of the screen top 4, which provides information and menu controls to the user. Note that as the game progressed, the audience grew to 2, 5, 3 viewers. In the case of so many viewers, there must be a number of viewers with display devices 422 that have the same or close resolution (each application/game vocabulary 1521_1525 has a proportionally adjusted video to The ability to adjust the fit). The game for the show is a multiplayer game, so the user can decide to join the game at the moment. Due to the multiple, ', 7 u Wang Yu Dai Guan Service 2 1 0 may or may not allow users to join the game. Lift, dry n叩. The user may have to pay for the cost of playing the game and choose to support, and the 1T user may not have enough (i) level of the game (for example, for other players, it will not compete) The formula has enough low latency to allow the user to play (for example, to view the game's delay constraint; u 1FG j is remotely focused without delay (actually, h condition) Under the contempt of playing on the other continents), but for the game I39848.doc -Π2- 201001177 The game played ^Word' delay must be low enough for the user (4) to enjoy the game, and (b) is at a higher Low-latency connections are equal to other players). If the user is not allowed to play, the application/game server 1521_1525 providing the user interface of the game viewfinder user will request the host hosting service control server 401 to start (ie, locate and start) the appropriate group. The game/play server ΐ52ι_ 1525 for playing a specific game to carry the game from the RAID array 15n_1512, and then the colocation service control server 401 will guide the routing 15〇2 to control from the user The signal is sent to the application/game server of the current hosted game and now the hosted game application/game server will direct the shared hardware compression 1 530 self-compressing application from the hosted game viewfinder application / Game server video/audio switch to compress video/audio from the application/game server of the current hosted game. The vertical synchronization of the game viewfinder application/game service with the new application/game server of the colocation game is not synchronized, and thus there may be a time lag between the two synchronizations. Since the shared video compression hardware will start compressing the video after the application/game server 1521_1525 completes the video frame, the first frame from the new server can be completed earlier than the full frame time of the old server. The first frame of the server may be completed before the transmission of the frame by the first compressed frame (for example, consider the transmission time 992 of Figure 9b: if the uncompressed frame 3 963 is completed one and a half times the frame time) 'Then it will impact transmission time 992). In this case, the shared video compression hardware 1530 will ignore the first frame from the new server (eg, ignore (974) frame 4 964), and the client 41 5 will come from the old servo 139848.doc - 113 · 201001177 The last frame of the device maintains an extra frame time, and the shared video compression hardware 1530 will begin to compress the next frame time video from the new application/game server of the hosted game. For the user, the transition from one application/game server to another application/game server will be visually seamless. The colocation service control server 〇1 then switches the application/game server i52i_i525, which informs the courier game viewfinder, to the idle state until it is needed again. The user can then play the game. In addition, the exceptions and games will be played instantly on the sensation (because the game has been loaded onto the application heart game server i52i_i525 from the _Array 1511-512 at a rate of one billion bits per second) and will be borrowed From the ideal drive n, the register configuration (in the case), the game is loaded with the exact system from the L for the operating system of the far-fashioned game - to the server that is exactly suitable for the game, and no Other applications that may compete with the operation of the game are executed on the server. — Again, as the user progresses through the game, each of the segments of the game will be at a rate of one billion bits per second (ie, '8 seconds of people's 10th place') loaded from the RAID array 1511_1512 Into the benefits of Ning and due to the huge storage capacity of the RAID array 5U-1512 (because of the 'shared resources of the multi-users of the horse', it may be very large, but still have, Zhong Zengji / j certificate), so that the juice can be pre- - · Shape setting or other game pieces D Λττ, ^ \ door only and set it on the array 15 11-15 12 and very fascinating ^. In addition, since the parent-application/game server 1521_1525 " ', the force P 4 hard-arm configuration and calculation capabilities are known, the pixel and vertex coloring can be pre-computed. So the game can be launched almost instantly, 1 , and will be executed in an ideal environment, 139848.doc • 114· 201001177 and the subsequent clips will be loaded almost instantly. However, in addition to these advantages and disadvantages (via the previously described game taker ^ to determine the play β not poke, and other components), and both are not interesting, clever. In addition, if you use j, you can learn from others by watching others... Instantly demonstrate the game without waiting for a large download and/or installation, and make the search based on a trial of a smaller fee:: : Play the game instantly (possibly or on a longer term basis). In addition, c will be able to play the game on a s to::-, on a television, at home, while traveling, and even on a mobile phone, by a wireless connection of sufficiently low latency. Bu, this can be done without having a copy of the game. 4 The limbs are not described above, the user can decide, do not allow his play to be viewed by others, allow his game to be viewed after the delay, allowing his game to be viewed by the selected user' or allowed Its game can be viewed by all users. How to 'in the embodiment' store the video/audio in the delay buffer cry 1515 for 15 minutes, and the user will be able to "rewind" and view his previous game play 'and pause the game, slow the game Playback, fast forward the game, etc., as it can be done while watching a digital video recorder (DVR) UV. m tube In this example, the user is playing the game, but if the user is using the application, the same "DVR" capability is available. This may be useful in verifying previous work and in other applications as detailed below. In addition, if the game is designed to have the ability to fall back based on the use of game state information, so that the camera view can be changed, the "3D DVR" capability will also be supported, but it will need to design the game to support "Defense 139848." Doc -115 . 201001177 Two L capabilities. The "DVR" 35 force using the delay buffer 1515 will work together with the game or application & (of course, limited to the video generated by the game or application =) However, in the case of a player who has the ability to make a request, the user can control the 3D "travel" of the previously played clip and cause the delay buffer n 1515 to record the video and the game state. Therefore, the spleen sputum ~* slice 1 and ^ will be known as "tooth" recorded as compressed video, but because the game state will also be recorded, different traversal will likely be at a later date in the same segment of the game. As described below, the users on the colocation service 21G will each have a 〃 page 纟 亥 使用者 user page, the user can publish information about himself and other information. The user will be able to announce things - For video & playback from its saved games. For example, if the user has overcome the particular difficulty of the game, the user can just "rewind" to achieve great results in the game. Before the location, and then the host hosting service 210 saves a video clip of a duration (eg, 3 seconds) on the user page of the user for other (4) viewing. To implement this, the only thing the user is using is the app/game pad bum that will be stored on the user page of the delay buffer 15 R and then the video segment:= . If the game has the ability to have a 3D DVR, as described above, the user can also record the game state information required by the 3D DVR and make it available to the user's user page. In the case where the game is designed to have a "bystander" in addition to an active player, 139848.doc -116· 201001177 (that is, a user who can travel in the 3D world without observing and observes the action) The game viewfinder app will enable the user to join the game as a bystander and player. From the point of view of the implementation of the self-examination, there is no difference between the user and the active player. The game will be loaded on the 152W525 and the user will control the game (eg, to control the virtual camera of the world). The only difference will be the user's gaming experience. Multi-user collaboration The co-hosting service 2U) is a feature that allows multiple users to collaborate while viewing live video (even with very different devices for viewing). = Useful when playing games and when using the app. bow

... many pcs and mobile phones are equipped with video cameras and have the ability to perform instant video compression (especially when the image is small). Also, it is not difficult to implement a compact M camera that can be attached to a television and is implemented in software or in many hardware compression devices for compressing video. Also, many % and mobile phones have a microphone and the headset is available with a microphone. A camera and/or microphone that combines native video/audio compression capabilities (specifically, using the low-latency video compression techniques described herein) to enable the user to combine video and/or audio with the control information 'Self, the user premises 211 is transferred to the colocation service 21〇. When using these techniques, the capabilities described in Figure 19 can be achieved. The user can make the Voice & SoundTM appear in another user's game or application: see the screen. This example is a multiplayer game, #中队友 cooperates in the car. The video/audio of 139848.doc •117· 201001177 can only be selectively viewed/heard by their teammates. Moreover, because there is effectively no delay in using the techniques described above, the player will be able to talk to each other or exercise at once without an appreciable delay. This video/audio integration is accomplished by causing compressed video and/or audio from the user's camera/microphone to arrive as an access network. Next, the router 15G2 routes the video and/or audio to the application/game server 1521-1525 that is permitted to view/hear the video and/or audio. Then, the user who selects the respective application/game server 1521·1525 of the video and/or audio device decompresses the video and/or tone and integrates it as needed to appear in the game or the program. As explained by 1900. The example in Figure 19 shows how to use this collaboration in a game, but this collaboration can be an extremely powerful tool for applications. Consider - Scenario: One of the major buildings is being designed by New York City for the New York-based premises in the New York-based premises for the construction of the New York-based premises in Chili π 耳 && ^ & Financial investors who are on the move and happen to be at Miami Airport, and need to make decisions about specific design elements of the building (how they relate to the buildings in the vicinity) to satisfy both the technology and the real estate developer. Suppose the construction company has a high-resolution monitor with a camera attached to the PC in Chicago, the real estate developer has a laptop with a camera in New York, and the investor has a camera-initiated phone call in Miami. Construction companies can use colocation services 2 10 . It is a powerful architectural design application that can be highly realistic 3D reproduction and it can take advantage of the large database of New York City buildings and the library of buildings being designed. The architectural design application will execute on the application / 139848.doc -118 - 201001177 game server 1521-1525 (or on several if it requires a lot of computing power). Each of the three users at the disparate location will be connected to the colocation service 21, and each will have a simultaneous view of the video output of the architectural design application, but it will be targeted for each - The user has a given device and network connection characteristics and is appropriately sized by the shared hardware compression (10) (for example, the construction company can continue to use the Internet connection to see 256 〇 xl 44 〇 6 〇 plus display, New York Real estate developers can see (four) fpS images via a 6-muscle connection on their laptops, and cast 250, and the honeycomb data link sees 32〇x(10)60 fps images). Each party will hear the other party's voice (which will be used by the application/games (5) Bu (9) in many of the widely available conference beer calling software packages: the conference call) and via the user input device By the way - the user will be able to use the local camera to make the video appear. As the meeting progresses, 'the architect will be able to reproduce the display by making the building a freak, and the piece is just like the real thing.” Like what, and all parties will be in 1 built-in tooth 越 铱 &; Λ 均 均 将 均 均 均 均 均 均 均 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方 各方Anyone can use this realism to say that downloading or even storing and reproducing the New York City Xitian is not asking for Jing, not to mention the seat. έ # , ,..., the huge resources required for the buildings around the city, despite the 俜Into the "seeing", although far away, and the second series of different local devices, but it will simply have a seamless experience under the difficulty of private. In addition, when the true-sense conveys its emotional state, it can do so. In addition, if the real estate 139848.doc • J19- 201001177 developer or investor wants to control the building program and use its own input device (which is a keyboard, mouse, keypad or touch screen), then it can be, and it can There is no perceived delay to respond (assuming that its network connection does not have an unreasonable delay). For example, in the case of a mobile phone, if the mobile phone is connected to the WiFi network of the airport, it will have a very low latency. However, if it uses a cellular data network available in the United States today, it is likely to suffer a noticeable lag. But even for most of the purposes of the conference (where the intruder is watching an architect controlling a building crossing or talking about a video conference call), even a hive delay should be acceptable. Finally, at the end of the cooperative conference call, real estate developers and investors will enter their comments and stop the colocation service, and the construction company will be able to "rewind" the video of the conference that has been recorded on the delay buffer 1515. And reviewing comments, facial expressions, and/or actions applied to the model of the building during the meeting. If there are specific segments that they wish to save, they can be moved from the delay buffer 丨 5 i 5 to the RAID array 1511-1512 for archival storage and later playback. From a point of view, if an architect only needs to use the New York City's ability to measure and a large database for a 15-minute conference call, then it only needs to pay for the time when the resources are used, without having to have high capabilities. The workstation does not have to purchase an expensive copy of the large database. Video-rich community services Hosting services 21 are an unprecedented opportunity to build video-based community services on the Internet. Figure - , Figure 20 shows the use of the game player on the colocation service 2] to the user page. As with the game viewfinder; 139848.doc • 120- 201001177 耘 The user page is the application executed on the application/game server i 52 i _ i 525 t. All thumbnails and video windows on this page show a constantly moving video (if the clip is short, it loops). "Using a video camera or by uploading a video, the user (whose user name is ILLHAZARD) can publish his own video 2 (other users can view the video). The video is stored on the RAID array 1511_15丨2. Also, when other users come to the KILLHAZARD2 user page, if KILLHAZARD is using the colocation service 21() at this time, it will display the video of the live broadcast that is being ordered. It allows for the inspection, and the user of the eight user page views it). This will be hosted by the host page application application/game server i52i_i525 self-service control system, 4 401 π seeking KILLHAZARD is active (and so, then β is seeking the application/game server being used Ϊ́52ι_ΐ525) to complete. The compressed video stream of the appropriate resolution and format is then sent to the application/game server 1521-1525 executing the user page application and displayed using the same method used by the game viewfinder application. The right user selects the window with the live broadcast of KILLHAZARD and then clicks on its input device appropriately, the window will be enlarged (the same method as the game viewfinder application), and the live video will be The screen is populated by viewing the resolution of the display device 422 of the user (suitable for viewing the characteristics of the user's internet connection). This is a key advantage over prior art methods. The user viewing the user page can see the live broadcast of the game that the user does not own, and may not have a local computer or game console capable of playing the game. It is 139848.doc -121 - 201001177 The user has an excellent opportunity to see the user page game, and it is an opportunity for the I:::Make: the user to play a better game. The user may wish to try or come from KILLHAZARD's 2栌n?+ 4 video clips also displayed on the user °: The recorded or uploaded visual instructions, and below each video clip are two = play The text of the game (for example, one is playing the game, the rider (Erag〇n)” and the MrSn ^ menu item (not shown... video clip = video (4) series has not been recorded or uploaded (4) with the official service The game on 2lG is broadcast live video of the content being carried out in its game. Because it is a game viewfinder for group building. If you choose a partner game, you will hit the game early. To the full screen, and the user will be able to watch the game broadcast live on the full screen. The user of the double-player game does not own the copy of the game, nor does it have the local end for playing the game. Calculate/game console resources. The game view is valid instantaneous. / / As described earlier, when the user plays the game on the colocation service 210, the user can "rewind" the game and find the hope Save the view hole ^^ and then will The video clip is saved to its user page. These are referred to as "Brag CUP." Video clips 2003 are self-reward clips (9) saved by KILLHAZARD from previous games played by them. How many times the self-reviewed clip has been viewed, and when the self-reward clip is viewed, the user has an opportunity to rate it, and the number of icons of the orange keyhole shape 2005 indicates how high the rating is. When the user views 139848 .doc -122- 201001177 User page leaf, self-reward clip 2003, along with the rest of the video on the page, continually follows %. If the user selects and clicks on one of the self-review clips, it zooms in to the king. Enjoy clip 2〇〇3, and DVR control that allows playback, pause, rewind, fast forward, step, etc. The clip is loaded by the application/game server ι521_ 1525. The self-recommended clips are stored in the array and are decompressed and played back for playback. "The self-reward clip 2003 can also be a 3D DVR from a game that supports DVR-capable games." Video Fragment (ie, a sequence of game states from a game that can be played back and allows the user to change the camera view point). In this case, in addition to the particular "crossing" compressed video recording that the user made while recording the game segment In addition, the game status information is also stored. When the user page is being viewed and all thumbnails and video windows are constantly cycled, the 3D 自 自 自 2003 2003 2003 will allow the user to record the "crossing" of the game segment. The self-reward clip 2003 recorded as compressed video is constantly looped. However, when the user selects the 3D DVR self-reward clip 2003 and clicks 3D 〇¥11 self-reward clip 2〇〇3, in addition to allowing playback of the compressed video from In addition to the DVR control of the clip, the user will be able to click on the button that gives their 3D DVR capabilities for the game clip. It will be able to independently control the camera "crossing" during the game segment, and if it wishes (and the user of the user page so allows), it will be able to record the alternate self-reward clip in the form of compressed video. The alternative self-reward clip "crossing" will then be available to the user page I for other viewers (immediately, or after the owner of the user page has 139848.doc -123 - 201001177 has the opportunity to verify the clip ). This 3D DVR Self-Camp Clip 2003 capability is enabled by launching a game that will replay the recorded game state information on another application/game server Hl521_1525. Because the game can be launched almost instantaneously (as previously described), so it is activated (its playback is limited to the game state recorded by the self-reward clip) and then the user is allowed to record the compressed video to the delay buffer 1515. It is not difficult to use the camera to "cross" at the same time. Once the user completes the "crossing", the game is deactivated. "From the user's point of view, launching the 3D DVR self-reward clip 2003 crossover" is not as difficult as controlling the linear self-reward clip 2〇〇3 d vr control. It can't be a tour of the sea ^ or even know how to play the game. It is only a virtual camera operator staring at the 3D world during the game segment recorded by another operator. The user will also be able to add his or her own audio to the self-reward clip (or from the eve of the game, or use the self-reward clip to use the characters and actions from the game to generate the brakes) This dynamic production technique is often referred to as "game movie (-a)." As the user progresses through the game, they will reach different skill levels. The played game reports the results to the service control system, and this The skill level will also be displayed on the user's page. Interactive animation ads, SQ have changed from text to still images, video, and now to the second move ^ fragment, usually using the likes of Ad〇be Flash The reason for using 'animated thin client' is that user 139848.doc -124· 201001177 is usually delayed and more patient than being peddled by selling products. Also, the thin client is not... Tm sounds prosperous and ancient ^ & performs on non-hanging inefficient PCs, and therefore ^ 商商 can have "degree of confidence: interactive advertising will work properly. Unfortunately, such as Adobe Flash + + + this 々 , aSh's simplistic client is limited in the degree of interaction and the duration of the experience (to reduce download time). 2: Mingyi interactive advertising, * users will be in the car in the showroom = select The exterior and interior color of the car, while ray tracking

How does the car look. The face avatar is then used to drive the car, and the pick-up user can use the car for driving on a race track or through a country other than Monaco. The user can select a larger engine or a helium tire' and then see how the changed configuration affects the ability to maintain vapor stability. Of course, the ads are effectively for the cutting-edge 3] video game. But for such ads that can be played on a PC or video console, it will require 10^ MB of downloads, and in the case of a PC, it may need to install a special driver to steal 'and may not have enough CPU on the PC or GPU computing power is not at all 亍 此 ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' 亥 ' 亥In the colocation service 210, the advertisements are placed almost immediately, and are preferably executed regardless of the capabilities of the user's client 415. As a result, the eight-figure streamlined user-side interactive ad is delivered more quickly, with a richer experience and a higher level of reliability. Streaming geometry RAII) during instant animation. The array 151 M12 and the incoming route 1502 can provide such a fast data rate with such low latency that the design is likely to depend on the 13984B.doc -125-201001177 RMD array 1511·1512 And the routing of the rods to directly deliver geometric video games and applications in the middle of the game play or in the application during the live action (for example, with the traversal of complex billiards).

In the prior art (four) system (such as the video game system shown in Figure 1) :: a large number of available storage devices (especially in practical home devices) 爰 k 牝 牝 串 串 串 串 串 串 串 串 串 串 串 串 串(except for situations where the required geometry is slightly predictable). For example, in a driving game where a given road exists, the geometry of the building for entering the field of view can be reasonably predicted and the mass storage device can search for the location where the upcoming geometry is located in advance. But in complex scenes with unpredictable changes (for example, in a battle scene with complex characters around), if on a PC or video game system

Ram is completely filled with the geometry of the object currently in view, and then after the user suddenly turns his character to view his character, if the geometry is not preloaded into the RAM, the geometry may be visible There is a delay before the shape. In the colocation service 210, the RAID array 1511_1512 can exceed the speed data of the ultra-high-speed Ethernet speed, and under the SAN network, there are "Τ π achieve better than 10 ten-bit bits. The network is 10 billion bits per second faster than other network technologies. 10 billion bits per second will carry 1 billion bytes of data in less than - seconds. At approximately 60 fps frame time (16.67 milliseconds), approximately 170 million bits (21 MB) of data can be loaded. Of course, even in a RAID configuration, 'rotating media will still incur a delay greater than one frame time' but the flash memory-based rAID storage will eventually be rotated with 139848.doc -126- 201001177 media RAID arrays in general Large and will not incur this high latency. In an implementation, a cache with a large amount of RAM writes is used to provide very low latency access. Therefore, at a high enough network speed and a large amount of low enough latency, the CPU and/or GPU can process 3D data to quickly stream geometry to the application/game server.丨 525. Thus, in the previously given example, where the user suddenly turns his character and looks backwards, all the characters behind him can be loaded before the character completes the rotation (geometry, and thus 'for the user, It would seem that it is in a world that is as realistic as a lively photo-realistic photo. As discussed earlier, one of the last boundaries of a photo-realistic computer is the human face, and because of the human eye For incomplete sensitivity, the slightest error from a photo-realistic face can lead to negative reactions from the viewer. Figure 22 shows the use of ContourTM authenticity capture technology (the following is the subject of the application in the application: 2004) "Apparatus and method for capturing the motion of a performer", No. 10/942, 609, September 15th, 2004; "Apparatus and method for capturing the expression of a performer" ;Approving the application of improving marker identification within a motio on February 25, 2005 n capture system"; "Apparatus and method for performing motion capture using shutter synchronization", March 10, 2005, pp. 11/255, 854, "Apparatus and 139848.doc" -127-201001177 method for performing motion capture using a random pattern on capture surfaces"; June 1, 2006 曰 Application No. 1 1/449, 131 "System and method for performing motion capture using phosphor application techniques"; June 2006 System and method for performing motion capture by strobing a fluorescent lamp, No. 11/449,043, filed on the 7th, and 1/449, 127, "System and method for three dimensional capture of stop-motion animated Characters, each of these applications has allowed the live broadcast performance captured by the assignee of this CIP application to result in a very smooth capture surface', thus achieving a high polygon count tracking surface (ie 'Polygon motion accurately follows the movement of the face ). Finally, a photo-realistic result is produced when the video of the live broadcast of the performance is mapped onto the tracking surface to produce a textured surface. Although current GPU technology can reproduce tracking surfaces and textures

All available RAM in a modern PC or video game console is consumed.

The integration of linear content and interactive features 'continues the geometry 1521-1525 _ so that it can be used to create a video game with almost no face becomes 139848.doc -128- 201001177 TV and audio materials (collectively referred to as "linear content") A wide range of (3) forms can be used for home and office use. Linear content is available on physical media such as HD DVD and Blu-ray media. It is also recorded by DVRs from satellites and Φ^ ΤΛ7 Home 4 Tv broadcasts. In addition, it is available as Pay-per-View (PPV) content via satellite and TV, and Video on Demand (VOD) via TV. Six "springs" [the inner valley can be downloaded via the Internet and connected to the present, does not exist today - can experience the position of the linear media related body. For example, 'DVD and other video optics Media comment: 'There are interactive features that are not available at other locations (such as the director's snippet, button, etc.). The online music site has cover art and song information that is usually not on the CD, but not all CDs. Available online: Websites associated with TV shows often have additional features, blogs, and sometimes comments from actors or creators. Even in the case of many movies or sports events, there are often often Release (in the case of a movie) or (in the state of the sport (4) play., the event of the event (for example, the transaction of the game): The agency service 210 is very suitable for delivering linearity when the content of the disparate form is started. Indeed, delivering a movie movie is less challenging than a mobile video game, and the colocation service 210 can deliver each device to a variety of devices in the home or office, or to a Z°° piece. 3 Demonstrate a 21G interface page for colocation services that demonstrates the choice of linear content. Using 139848.doc -129- 201001177 However, unlike most linear content delivery systems, colocation services 2 10 can also deliver relevant Interactive components (for example, menus and features on DVDs, interactive overlays on HD-DVDs, and Adobe Flash animations on the website (as explained below)). Therefore, the client device 4丨5 limit is no longer Introducing restrictions on which features are available. In addition, the colocation system 21 can dynamically and instantly link linear content to video game content. For example, if a user is watching a Quidditch game in a Harry Potter movie And decide that he is willing to try to play Qiudditch, then it can just click the button and the movie will pause and it will be immediately sent to the Quidditch segment of the Harry Potter video game.

After the Quidditch game, another click on the button and the movie will start again. In the case of film-like graphics and production technology, the video obtained by the camera cannot be distinguished from the live action characters, when the user enters the video from the Quidditch game in the live action movie to the hosted service. When the Quidditch game in the game changes (as described in this article), the two scenes can't actually be distinguished. This is a brand new creation for directors of both linear content and inter-form (eg, video game) content, because the lines between the two worlds become indistinguishable. Using the colocation service architecture shown in Figure 14, J can provide virtual machine control of the 3D movie to the viewer. For example, in the event of: a scene towel in the car will likely allow the viewer to control the camera while the story is progressing. This assumes that all 3D objects (r" in the train and the ability to reproduce the scenes and the original movie 139848.doc -130-201001177 are available. And even for the non-computer generated ^ Fu City exists can provide very exciting; Temple sign. For example, the 2〇〇5 film "many of the beautifully decorated old British buildings; the scene, the user "squatting in the special building" will be seen as a temporary and then control the possible surrounding area of the camera. In order to implement this, Xia Yi has carried a camera with a fisheye lens. When it tracks its position, A, is very similar to the implementation of the prior art.

Apple's QuickTime vr. The various map expansion frames will then be converted so that the image is not distorted' and then it is stored on the dice 12 along with the movie and is played back when the user chooses to continue the virtual tour. In the case of a sporting event, the traffic can be streamed through the colocation service 2H to see a sporting event (such as a basketball game) for the user to watch (as it would for a typical TV). After the user views a particular play, the video game of the game (finally the basketball player looks as realistic as the real player) can catch up with the player starting in the same position, and the user (possibly controlling one player each) can replay To see if it can do better than these players. The colocation service described in this article is extremely suitable for supporting this future world because it can withstand the computing power and large-capacity storage resources that are impractical to be installed in the home or in most office settings, and Computing resources are always up-to-date (in the case of the latest computing hardware available), but in the home background, there will always be families with older generations of PCs and games. In addition, in the colocation service 2, the use of 139848.doc -131 - 201001177 is concealed by all this computational complexity, _, even with the use of very sophisticated 糸 」 匕 匕 匕 , , , , From the point of view of the user, it is also as simple as changing the TV. In addition, the user will be able to access all of the experience that the singular and singular capabilities will bring from any of the clients 415. - month multiplayer game = the degree of multiplayer game, it will be able to communicate not only to the road, but also to the application/game server 1521-1525 and - by the network, the bridge to the absence The servo executed in the colocation service 210 is the internet of the game machine (not shown). When playing the multiplayer game with the Thunder of You &#^ on the universal network, the application/game server U21 1525 will have the advantage of extremely fast access to the Internet. Compared to the implementation, but it will be limited by the ability of other computers to play games, and potentially also by the game server on the Internet to adapt to the least commonality (the game servo is The factual limit will be for a relatively slow consumer Internet connection on a home computer). However, when playing a multiplayer game entirely in the colocation service 210 server center, a great difference can be achieved. Hosting the game for the user's game - the application/game server 1521-1525 will be connected to other applications/games $2 1 1 5 2 5 and with extremely high speed, very low latency connectivity and The huge, very fast storage array host hosts any server interconnects that are centrally controlled by multiplayer games. For example, if a hyper-speed Ethernet is used to route the 1502 network, the application/game servers 1521-1525 will be communicated to each other and communicated to potentials at a gigabit per second rate. Only i 139848.doc -132- 201001177 milliseconds or less than 1 millisecond delay under the host to host any server for the central control of multiplayer games. In addition, RAID arrays 1511-1512 will be able to respond very quickly and then transfer data at a gigabit per second rate. As an example, if the user customizes the character in terms of appearance and clothing so that the character has a large number of geometric shapes and behaviors that are unique to the character, the previous is limited to the game client that is executed on the pc or the game console in the home. Under the technical system, if a character enters the field of view of another user, the user will have to wait until a long slow download is completed in order to load all geometry and behavioral data into his computer. Within 21 主机 of the colocation service, the same download can be superior to the ultra-high-speed Ethernet that is served from the RAID array 1511-512 at a rate of one billion bits per second. Even if the home user has an Internet connection of 8 m (which is extremely fast according to current standards), the ultra-fast Ethernet network is 100 times faster. Therefore, it takes less than a second to spend a minute on a fast internet connection on a super-fast Ethernet. Top player grouping and tournaments Hosting services 2 1 0 is perfect for tournaments. Since no game is executed in the local client, there is no chance for the user to cheat. Again, due to the ability of the output route 1540 to multicast UDP streams, the colocation service 210 can simultaneously broadcast larger tournaments to thousands of people in the audience. In fact, when there are so popular that thousands of users are receiving the same stream of specific video streams (for example, showing a larger tournament view), the video stream is sent to a content delivery network (CDN) (such as Akamai or

Limelight) may be more efficient for distribution to a large number of client devices 415. 139848.doc -133 - 201001177 When using CDN to show the top viewer grouped game viewfinder page, you get a similar level of efficiency. - For larger tournaments, live celebrity commentators can be used to provide comments during the special event. Although a large number of users will be watching the (4) mark, and a relatively small number will be played in the tournament. The audio from the celebrity commentator can be routed to the host to host the game in the tournament and the host will be the singer; ^ any of the bystander mode replicas of the game in the game / game feeding device 1521_1525, and The audio is added to the game audio to cover the video of the celebrity speaker on the fun scene (or just on the bystander view). Web page loading accelerates the World Wide Web's primary shipping agreement, hypertext transfer protocol (h is conceived and defined in the era of commercial high-speed Internet connectivity, and online consumers using dial-up modems or ISDN) The 黄金 黄金 咩 咩 咩 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金 黄金

Mbps data rate (ie, equal data rate in both directions). , · Today's situation is completely different. In a large number of developed worlds, the average home connection speed via a muscle or electrograph computer has a much higher down= tribute rate than the τι line. In fact, in some parts of the world, fiber-to-the-curb is introducing high rates of A/ru bamboo violations of 50 Mbpsi 〇〇 Mbps into the home. Unfortunately, HTTP has not been architected (and not implemented) to make effective use of these dramas to improve speed. The website is a collection of files on the remote server. _ Speaking alone, HTTP requests the first-construction, waiting to download the project, and pick up; 139848.doc • 134· 201001177 request the second file, waiting to download the file. In fact, HTTp allows more than one "open connection" (ie, requests more than one standard at a time), but only a very small number of open connections are allowed due to agreed standards (and the desire to prevent the web server from being overloaded). . In addition, because the web page is constructed in a 'cold' browser, it is often not aware of multiple simultaneous pages that can be used for immediate downloading (ie, it becomes apparent only after parsing a page: a new file such as an image needs to be downloaded) . Therefore, the files on the website are essentially one by one.

Load. In addition, due to the request and response protocol used by HTTP, there is a delay of 10 milliseconds associated with the parent-slot case (a typical network server accessing the United States). In the case of a relatively low speed connection, this does not introduce a lot of problems, because the download time for the slot itself determines the waiting time of the web page. However, as the connection speed increases (especially in the case of complex web pages), it starts to cause problems. In the example shown in Figure 24, a typical commercial website is shown (this particular station is from the Che Yi Da Sneakers trademark). There are 54 slots on the website. The $ includes HTML, CSS, JPEG, PHp, and Heart (4) f ' and includes video content. Before the live web page (that is, the user can click on it and start using it), the total defamation tuple must be loaded. There are many reasons for the large number of slots. First, it is a complex and sophisticated network, and secondly it is a web page based on the combination of accessing the page (eg, which country the user came from, what: S, whether the user made a purchase before) Etc.) and download different files based on all of these factors. However, it is still a very typical commercial web page. 139S48.doc -135- 201001177 Figure 24 shows the amount of time that elapsed before the live web page was broadcast as the connection speed increased. At 1.5 Mbps connection speed 2401, using a conventional web server with a conventional network continuation browser, it takes 13.5 seconds before the live web page is live. At a 12 Mbps connection speed of 24 〇 2, the load time is reduced to 65 seconds, or approximately doubled. However, at a connection speed of 2403 at 96 Mbps, the load time is only reduced by v to about 5.5 seconds. The reason for this is because at this high download speed, the time to download the file itself is the smallest, but the delay of approximately 1 00 milliseconds per file remains, resulting in a delay of 54 files * 1 〇〇 milliseconds = 5.4 seconds . Therefore, no matter how fast the connection to the home, the site will spend at least 5.4 seconds before the live broadcast. Another factor is that the server side is queued up; each π 求 system is added after the j-nine column, so this will have a significant impact on the busy feeder, because for each to be obtained from the web server For small projects, HTTP requests need to wait for them to return. One way to solve these problems is to discard or redefine. Alternatively, it may be possible for the website owner to better merge their files into a single file (eg, in Ad®be Flash format). However, as a practical matter, this company and many others have invested heavily in their website architecture. In addition, some families have 12_1 Mbps connections, but most families still have the privilege and slow speed, and HTTP does work well at slow speeds. An alternative is to host the web browser host on the application/game server 1521-1525, and the host for the web server is on the RAID array! 5 !] _] 5] 2 (or potentially hosted on the host escrow network Navigator application / game feeder 1521_1525 on the MM or on the local storage). Due to the route through the port 丨5〇2 (or to the local storage H9848.doc -136- 201001177

The very fast interconnects are not in the case of using HTTP. Each slot has a delay of 〇〇 milliseconds. Instead, there is a minimum delay per file under HTTP. Then, instead of allowing the user in the home to access the webpage via the UI, the user can access the webpage via the client 4155. Then, even under a 15 Mbps connection (because this page does not require a lot of bandwidth for its video), the web page will be live broadcast in less than 1 second per line 2400. In essence, there will be no delay before the webcaster executing the application/game server 1521_1525 displays the live broadcast page, and there will be no detectable before the client 415 displays the video output from the web browser. The measured delay. When the user uses the mouse to search the webpage and/or type a word on the webpage, 'send the user's input information to the web browser executed on the application/game server 1521-1525, and the web browser A response will be responded accordingly. The $W of the service is ‘If the duster is transmitting the video message :: it uses the frequency 1 ’ even if the page becomes static. This can be remedied by grouping to transmit data only when (and if) the web page changes and then only transfer the beaker to the page portion where the change occurred. When there are some webpages with a slogan, such as a flashing slogan, these web pages tend to be annoying, and unless there is a reason to move something (for example, a video clip), the webpage is usually static. of. For these pages, the following may be the case. . Using the host agency service 210 will transfer less data (with the custom network feed) because only the actual displayed image will be transmitted, no compact user can execute the code, and there is no large object that may never be viewed ( Such as moving the image). r 139848.doc -137- 201001177 Therefore, 'using the colocation service 21〇 to host the old version of the webpage, the webpage loading time can be reduced to the extent that the webpage is similarly changed to the channel on the TV: effectively and instantly Live the page. Promoting the Debugging of Games and Applications As mentioned earlier, video games and applications with the same (4) shape are very complex applications and often have defects when they are released into the field. Although the software developer will get feedback on the defect from the user and it may have some way to force the machine state back after the crash, the phase identification identifies what caused the game or instant application to crash or improperly execute. very difficult. When the game or application is executed in the colocation service 210, the video/audio output of the game or application is constantly recorded on the delay buffer 15 15 . In addition, the watchdog process executes each application/game word benefit 1521_1525. The watchdog process will periodically report to the colocation service control system 〇1 that the application/game server 1521_1525 is performing smoothly. If the watchdog process fails to report, the feeder control system 4〇 will attempt to communicate with the application/game server 152丨_1 525, and if successful, will collect whatever machine status is available. Regardless of what information is available, it is sent to the developer along with the video/audio recorded by the delay buffer 1515. Therefore, when a game or application software developer gets a notification of a crash from the colocation service, it gets a record of the frame of the frame that caused the crash. This information is extremely valuable in tracking down defects and fixing them.八1贝139848.doc • 138 - 201001177 Also note that when the application/game server 1521-1525 crashes, the server is restarted at the most recent restartable time and the message is provided to the user, thus Apologize for the difficulty. Resource Sharing and Cost Savings The systems shown in Figures 4a and 4b provide multiple benefits for both end users and game and application developers. For example, in general, home and office client systems (e.g., PCs or game consoles) are only in use for a small percentage of the week. According to Nielsen Entertainment "Active Player Benchmark Study" (http://www.prnewswire.com/cgi_bin/stories.pl?ACCT=104&STORYWwww/story/l〇-〇5-20〇6/ 〇〇 〇4446115&EDATE=) 2006 January 5th, the active gamer spends an average of 14 hours a week playing on the video game console and playing on the handheld device for about 17 hours a week. The report also stated that for all game play activities (including consoles, handheld devices, and ... game play), active players averaged 13 hours per week. Considering the higher number of console video game play time, there are _ weeks for 168 hours. In the home of the gamer, the video control is only used in the week of 17/win ι〇%. Or '9〇% of the time, video game "糸 idle" given the fact that the video game console is high to help these devices, this A 曰 primary, β 夂 I k 商贝士,, this is mussel - Beyin's very inefficient high-end applications (such as Δ ,, especially (AUTOdesk Maya) often desktop PCs. 栲 ^ 而 而 而 尽 尽 尽 尽 一些 一些 一些 一些 一些 一些 一些 一些 一些 一些 一些 一些 一些 一些 一些 一些 一些And holiday pcs (for example, portable PCs with $ back and some for security at night) are available at 139848.doc -J39-201001177 for all hours and holidays, but most commercial activities tend to Focusing on approximately 9 AM to 5 pM from Monday to Friday in a given business day area, less false and disconnected time (such as lunch), and because most PCs are used actively by users in PCs It appears, so it follows: desktop PC utilization tends to follow these hours of operation. If it is assumed that the five days of the week from 9am to 5PM constantly use pc, then he will imply that pc is 4 weeks in a week. /168=24% of the hours are utilized. High-performance desktop K: for commercial use Responsible investment, and this reflects very low utilization. On the table: the school of teaching can use electricity in one week or even less, and change the number of hours of teaching as it is, but Most of the teaching takes place during the hours between the days of ..^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Weeks - during the hours of the Friday, during the hours of the Shang Zhang and & Shicha, working in Zizi, so these people usually do not play video games in this small = and therefore when they do play video games When it is during other hours (such as evenings, weekends, and false alarms). Given the configuration of the colocation service shown in Figure 4a, the usage patterns described in the two paragraphs above result in very efficient resources. Utilization. It is obvious that there is a real-time response to the number of servos that can be served by the host escrow service 210, especially when the user needs to use the complex application video game. Sexual love In the case, however, the video console of the video game or the % used by the merchant #

The time is idle, and the server can be reused by different users. For example, high-performance dual (four) and dual GPU 139848.doc -140· 201001177 and a large amount of RAM high-performance server 4〇2 can be used by businesses and schools on non-holiday 9 AM to 5 PM, but by playing cutting-edge video Game players are used at night, on weekends and on holidays. Similarly, low-performance applications can be utilized by commercial and learning boards during low-performance servers with Ceieron CPUs, GPU-free (or very low-end GPUs), and limited RAM during business hours and are inefficient. The low performance server 4〇2 is utilized during non-commercial hours. In addition, in the case of the colocation service configuration described herein, the lean source is effectively shared among thousands (if not millions) of users. In general, online services only have a small percentage of their total user base to use the service on a given day. If you consider the Nie;isen video game listed above, it is easy to understand why. If the active player plays the console game only 17 hours a week, and if the value of the game's ♦ value is assumed to be at night (5-12 AM, 7*5 days = 35 hours/week) and weekend (8 am_12 AM, During a typical non-working, non-commercial hour of 16*2=32 hours/week, there is 35+32 = 65 peak hours for a 17-hour game. It is difficult to estimate the exact peak user load on the system for a number of reasons: some users will play during the extra-peak time, there may be a cluster peak of the user during a specific daytime, and the peak time may be affected by the game being played. Types (for example, children's games will probably play at an earlier time in the evening) and other effects. But 'assuming that when the player is likely to play the game, the average number of hours played by the player is much less than the number of hours in the day, then only the hosted service = 21Gt number of users will use the colocation service at a given time. 21〇. For this analysis, 'the peak load m will be assumed so only 12.5% of the calculation, compression and bandwidth resources are used at 139848.doc 201001177 at a given time, resulting in only 1.5% hard due to resource reuse. Body cost to support a given level of performance for a given user. In addition, it is assumed that some games and applications require more than others to be able to dynamically allocate resources based on the game being played by the user or by the application executed by the user. Therefore, users who choose low-performance games or applications will be assigned low-performance (lower-cost) servers 4〇2, and users who choose high-performance games or applications will be assigned high-performance (more loyal) clothes. 402. In fact, a given game or application may have a lower performance and higher performance area of the game or application, and the user can switch from one server 402 to another during the game or application area. 402, to maintain the user's execution on the lowest cost server 402 that satisfies the needs of the game or application. Note that RA is much faster than a single disk. [d array 405 will be used even for low performance servers 4〇2, which has a faster disk transfer rate. Therefore, the average cost per server 4/2 across all games played or used is much less than the cost of most expensive servers 402 playing the highest performance game or application private, however, even low performance Server 402 will also derive disk performance benefits from RAID array 405. In addition, the server 402 in the colocation service 210 may only be non-compliant. - PC board with the same 4; face (different from the network interface), and just happens to be integrated into a single positive film with a fast network interface to s AN 403. The AID array 405 with a large τ% may be shared among those who are far smaller than the case where the disk is present, so the cost of the disk for each active user will be 'a disk drive'. All of this equipment will likely reside in a rack in an environment that is controlled by the environment 139848.d〇c • 142 - 201001177. If the server 402 fails, it can be easily repaired or replaced at the colocation service 21〇. In contrast, a PC or game console in a home or office must be sturdy, and must be able to survive a reasonable wear and tear to prevent a single appliance from being hit or dropped, requiring a housing with at least one drive, must It is spared from adverse environmental conditions (for example, being forced into a superheated Av cabinet with other appliances), requires a service guarantee, must be packaged and shipped, and is sold by a retailer who may receive retail profits. In addition, the PC or game console must be configured to meet the peak performance of the most computationally intensive expected game or application that will be used at some point in the future, even for lower performance games or applications (or games or applications). District) may also play most of the time. In addition, if a PC or console fails, it can be a costly and time consuming process (which adversely affects manufacturers, users, and software developers). Therefore, it is assumed that the system shown in FIG. 4a provides the user with an experience equivalent to the experience of the local computing resource for the user to experience the computing power of the location in the home, office or school, via FIG. 4a. The architecture shown in the middle provides a much lower computational power. Eliminate the need for upgrades In addition, users no longer have to worry about upgrading their PCs and/or consoles to play new games or handle new applications with higher performance. Any game or application on the Hosting Service 21 (regardless of the type of server 402 that the game or application requires) can be used by the user and all games and applications are executed in close proximity (ie 'Quickly loaded from the RAID array 405 or the local storage on the server 402) and suitably with the latest updates and defects 139848.doc -143 . 201001177 ...that is, the 'software developer will be able to choose to execute the given The ideal server configuration for the game or application's feeding machine, and then configure the optimal service (4), and then over time, the developer will be able to update, fix, etc. at the same time All copies of the game or application provided to the host hosting service 210). In fact, after the user starts using the colocation service 21(), the (4) person may find that the game and the application continue to provide a better experience (eg, via update and/or bug fix) and may be the following: user one Years later, it was discovered that new games or shore applications could be used on services 210 that utilized computing technology (eg, higher performance GPUs) (which: not even a year ago), so it would be impossible for users to purchase After years of playing games or executing applications - the skills of the year ^ because the game or the computing resources of the execution should be visible to the user (that is, from the user's point of view, the user only chooses + near immediately The game or application that starts to execute is more like the user changing the channel on the TV. The user's hardware will be "upgraded" even if the user does not even know the upgrade. " Eliminate the need for backup • Another major problem for users in business, schools, and homes is backup. If the disk fails, or if there is any inadvertent erasure, the information stored in the local PC or video game console (eg, in the case of the console, the user's game results and ratings) may be lost. There are many available applications that provide automatic backup for i. The game control=status can be uploaded to the online server for backup, but the local backup is usually restored and stored in a female and organized organization. Another local disk somewhere (or 139848.doc -144- 201001177 other non-volatile storage devices), and because of the slowdown upstream speed available via a typical low-cost Internet Backups are often limited. In the hosted service 21 of Figure 4a, the data stored in the raid array can be configured to be used by the prior art RAm configuration techniques well known to those skilled in the art. Not losing the data will notify the technician at the server center that contains the failed disk, and then the technology will replace the disk, which will then be automatically updated to cause the RAID array to tolerate the failure again. In addition, since all the drives are close to each other with a fast local network via the SAN, all the disk systems are configured to be periodically backed up to the secondary material n in the server center (the publication (4) It is not difficult to take a $ towel or relocate it easily. From the point of view of the hosted service user, the user's information is always completely secure and it never has to be considered for backup. Access to the demo User, you want to try the game or application before buying a game or application. As previously (4) 'existence of demonstrations (the "verb" form means the trial version, the demo version is also called "demo", but as a noun) the prior technical components of games and applications, but each of them Subject to restrictions and/or inconvenience. With the colocation service 21, it is easy and convenient for the user to try out the demo. In effect, the user performs the presentation and trials the presentation via a user interface, such as the user interface described below. The presentation will load almost instantly onto the server 402 suitable for the presentation, and it will execute exactly like any other game or application. Regardless of whether the demo requires a very efficient server 139848.doc -145- 201001177 is still a low-performance health tool, and regardless of the type of home or office client 415 used by the user, from the user's point of view, the presentation Will work. Software publishers of game demos or application demos will be able to control the user's trials and trials for a long time^', and the demoizable users will be able to provide the full version of the game or application. The user interface element of the access opportunity. Because the presentation may be below cost or free of charge, some users may try to use repeated presentations (especially repeating playable game demos that may be interesting). The colocation service 210 can use various techniques to limit the presentation usage for a given user. The most straightforward method is to establish a user ID for each user and limit the number of times the given user (1) can play the presentation. However, the user can set a plurality of users m, especially if they are free. One technique for addressing this problem is to limit the number of times a given client 415 can play a presentation. If the client is a standalone device, the device will have a - sequence number and the colocation service 21 can limit the number of times the presentation can be accessed by the client with the serial number. If the client 415 is executing on software on or on other devices, the host escrow service 210 can assign a serial number and store the serial number on the PC and use the serial number to limit the presentation usage, but assume that the PC can be reprogrammed by the user. And the serial number is erased or changed, then the remote host colocation service 2 1 0 maintains the pc network adapter media access control (MAC) address (and / or other machine specific identifier 'such as hard Record the number of the disc player, etc.) and limit the use of the demo to the MAC address. It is assumed that the MAC address of the network adapter can be changed. However, this is not an extremely simple method. Another method is to limit the number of times a presentation can be played to a given IP address. 139848.doc -146 - 201001177 Although the IP address can be reassigned periodically by cable modems and DSL providers, it does not occur very frequently in practice and can be determined (eg by contacting an ISP) The IP system is in a block of IP addresses for residential DSL or cable modem access, and typically a small number of presentations for a given home can be established. Also, there may be multiple devices in the home after a NAT router sharing the same IP address, but typically in a residential setting, there will be a limited number of such devices. If the IP address is in the servo commercial block, a larger number of presentations for the business can be established. However, in the end, all combinations of the previously described methods are the easiest way to limit the number of presentations on the PC. There may be no such thing as a very simple way for a determined and technically skilled user to be limited in the number of replayed presentations. 'But creating a large number of obstacles can create enough obstacles to make most Pc users not worth the fee. God went to abuse the demonstration Li Yigan + 廿立一π~丁,, and, on the contrary, used the demo when he wanted to try out new games and applications. Benefits to schools, businesses and other institutions

The cost of a PC in an office or school environment. PCs usually only have a one-week hour to make the cost of positioning the performance much higher than in the server.

In the case of a 1T department setting server, such as a 'big university', such a computer that maintains access via a LAN level and remote access 139848.doc • J47-201001177 can be practical. There are many solutions for remote access to a computer via a LAN or via a private high frequency wide connection between offices. For example, 'by Micr〇s〇fti wind〇wy$ server, or by virtual network computing application (such as VNC from RealVNC Ltd.) or by streamlined users from gun Micr〇systems The end component allows the user to gain remote access to the PC or server, with a range of qualities in graphical response time and user experience. In addition, these self-managed server centers are typically dedicated to a single business or school and therefore cannot be utilized when disparate applications (eg, entertainment and business applications) utilize the same computing resources at different times of the week. The overlap of possible uses. As a result, many businesses and schools lack the scale, resources, or expertise to independently set up a network of aircraft connected to each user's LAN. In contrast, a large percentage of schools and businesses have the same Internet connections as homes (eg, DSL, cable modems). However, such organizations may still have the need for very high performance calculations (either periodically or periodically). For example, Xiaojianhao may have only a small number of architects, and when it comes to design work, it has a relatively modest computing need, but hey, this periodically requires very efficient 3D. When built # μ e ^, fixed for 3D traversal of the new architectural design of the user side). The decoration shown in Figure 4a, which is suitable for such tissue. These organizations only need to provide the same kind of network to the family.

Muscle, electricity, «machines" and usually very cheap. It can be (invalid ^ as user-end, or not at all (four) can also use low-cost PC signal logic 413 and low-latency, the purpose of the use of only low-cost __ contraction (1) low-cost dedicated device 139848.doc - 148-201001177 :.: Other features are particularly attractive for schools that may have plagiarism of PCs or problems with the defamation of dedicated groups within π. This configuration solves many of the advantages of using these organizations. Shared for home users for general purpose computing). ^ 夕 = equal operating costs (which ultimately must be y, y, too, and the heart is passed back to the user to have a quotable business) may be much lower because (a) calculate Φ a _ * π shell source is shared with other applications that have different peak usage times in a week. (9) These groups & can be obtained only when needed (and incur costs) for high-performance computing resources = organization does not have to provide for backup Or otherwise maintain resources for efficient monthly sources. The elimination of piracy is another thing: games, applications, interactive movies, etc. may no longer be as current. ^^^° The system is executed at the center of the device. So use ^: access to the basic code. There is piracy. Even if the source code is copied, the user cannot execute the code on the standard game console or the court computer. This has opened up the world where standard video games are not available (such as 'China, Zhanguyi is also impossible to sell, the city's W re-sales for game developers, as in today's situation, there are fewer markets. Continuity. With the new generation of technology forcing users and developers to upgrade and game developers depending on the current situation of timely delivery of the hardware platform, the main = total service 210 can be gradually updated as the game requirements change over time. Example Streaming Interactive Video 139848.doc -149- 201001177 The above description provides low-latency streaming interactive video based on the Universal Internet (which also implicitly includes audio along with video, as described in this article A variety of applications that enable the use of novel basic concepts. Prior art systems that provide streaming video over the Internet have only applications that can be enabled by high latency interactions. For example, basic playback controls (e.g., pause, rewind, fast forward) for linear video work properly at high latency, and it is possible to select among the online video feeds. In addition, as stated in the previous month, the nature of some video games allows them to be played at high latency. However, the high latency (or low compression rate) of prior art methods for streaming video severely limits the potential application of streaming video or narrows its deployment to a specialized '.周路私境, and even In such environments, prior art also introduces a substantial burden on the network. The technology described in this article opens up a large 1 ' 疋 疋 由 肖 肖 肖 肖 肖 在 在 经由 经由 经由 经由 经由 经由 经由 经由 经由 经由 经由 经由 经由 经由 经由 经由 经由 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖 肖And enable them to apply. "actually" is between the user device that is generally small compared to the user's shirt in Figure 4c's enough to use any amount of effective T # this force, the amount of fast library and the powerful server The extremely fast road connection provides an enhanced user of the moon, which enables a new era of computing. In addition, because of the bandwidth requirements, it does not follow the trend, and the first difference is "growth and grow (that is, because the bandwidth is about display resolution, quality, and frame, F network far food, i 疋 sheep), so once the broadband Internet lining connectivity is in (10) & wireless coverage, reliable and right-handed, it has a low latency and is sufficient for all the components. The 422 needs a sufficiently high bandwidth, and the spoofing will be typical consumers and 139848.doc -150· 201001177 Business applications are necessary for heavy clients (such as executing __,

LinUX, 〇SX, etc. pc or mobile phone) is even a thin client (such as 'Adobe Flash or Java'). The emergence of streaming interactive video has led to a reconsideration of assumptions about the structure of the scoring architecture. One of the examples is a colocation service that is not shown in _ 圑 1 3 210 server center embodiment. The video path for the feedback channel of the Tieyi Hengyi G/ and/or the group video 1550, the multicast streaming video output of the eight-T application/game server 1521 1525 via the path "52" or via the path The 1551 is fed back to the application/game server 1521 Ο 25 after the optional delay. This enables a variety of practical applications (9) such as, for example, by prior art servers or local computing architectures that are not possible or feasible. 16. The applications described in Figures 17 and 20). However, as a more general architectural feature, feedback loop 1550 provides recursion for streaming interactive video levels because video is required in the application. The video loops infinitely. This enables a variety of application possibilities that have never been available before. Another key architectural feature is that the video stream is unidirectionally streamed. This effectively results in any degree of interactive video streaming. Multicast (1), the bidirectional flow such as TCP/IP flow will increase more and more traffic stagnation on the network of incoming 2 letters as the number of users increases. Broadcasting Γ Γ Γ 重要 重要 重要 因为 因为 因为 因为 因为 because it allows the system to respond to the growing needs of Internet users (and 'the world's population of the world') Communication. Again, an example of the use of both hunger-interactive video recursion and multicast (such as Figure 339848.doc 201001177 16) discussed in this article is only "the tip of a very large iceberg with possibilities." , the various functional modules and phases described by _ contain specific hard-wired logic for performing these steps... 'Xiaoshu Application Integrated Circuit ("ASIC")) or by computerized components Any combination of hardware components is implemented. In one embodiment, a child can be applied to a programmable digital signal processor ("Dsp") such as a Tex device, such as a Tex device, such as a gamma (10) coffee or a side-by-side search. on. A variety of different DSPs can be used while still following these basic principles. The examples can include various steps as set forth above. These steps can be implemented in a machine executable that causes a general purpose or special purpose processor to perform certain steps. Various components (such as computers) that have nothing to do with these basic principles

Memory, hard drive, and wheeled devices are omitted from the figure to avoid mixing the relevant aspects. Elements of the subject matter disclosed by P may also be provided as a machine readable medium for storing machine executable instructions. A machine-readable medium can include, but is not limited to, a flash memory, a compact disc, a CD_R〇M, a dvd r〇m, a ram, an EPROM, an EEPROM, a magnetic or optical card, a broadcast medium, or other type suitable for storing electronic instructions. Machine readable medium. For example, the present invention can be downloaded as a computer program. The computer program can be transmitted from a remote computer via a communication key (eg, a data machine or a network connection) by means of a signal embodied on a carrier wave or other transmission medium. For example, the 'feeder' is transferred to the requesting computer (for example, the client). It should also be understood that the elements of the subject matter that can not be revealed can also be provided as a computer program. 39948.doc -152- 201001177 The product is installed. The computer program product is loaded with u*. These instructions can be used to know the machine. ^ ^ ^ ^ 1 ^ (for example, processing 芎 or other electronic devices) to perform - at ^ by the combination of hardware and software (4) known as wood. The machine readable medium can be limited to soft disk, CD, CD_R〇M, and — RAM, EPR〇M, EEPRQM, magnetic card 〇M, ^ card or card, media or suitable for storing electronic instructions. Other types of media/machine readable media. For example, the object of the subject matter disclosed by Lu's ^^ element can be downloaded as a computer program product, where the program can be embodied in the carrier via a communication link (eg _ u J Go data machine or network connection) Or other transmission media is delivered from the remote device or electronic device to the request process. It is to be understood that the subject matter of the present invention is to be construed as a Accordingly, the specification and drawings are to be regarded as illustrative rather BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 illustrates the architecture of a prior art video game system. 2a-2b illustrate a high order system architecture in accordance with an embodiment. Figure 3 illustrates the actual, rated, and required data rate for communication between the client and the server. Figure 4a illustrates a colocation service and a client used in accordance with an embodiment. Figure 4b illustrates an exemplary delay associated with communication between the client and the colocation service. Figure 4 c #儿明 According to the consistent application of the user-side crying. 139848.doc - 153 - 201001177 Figure 4d illustrates a client device in accordance with another embodiment. Figure 4e illustrates an example block diagram of the client device of Figure 4c. Figure 4f illustrates an example block diagram of the client device of Figure 4d. Figure 5 illustrates an example of video compression that may be used in accordance with an embodiment. Figure 6a illustrates an example of video compression that can be used in another embodiment. Figure 6b illustrates the learned values in the data rate associated with transmitting low complexity, low motion video sequences. Figure 6c illustrates the peak in the data rate associated with transmitting a high complexity, high motion video sequence. Figures 7a through 7b illustrate an example video compression technique used in an embodiment. Figure 8 illustrates an additional example video compression technique used in an embodiment. Figures 9a through 9c illustrate example techniques used in consistent applications to mitigate data rate spikes. Figure 1 〇a to Figure 1 Ob illustrates a consistent example of effectively encapsulating an image block in a package. Figures 11a through 11d illustrate an embodiment using a forward error correction technique. Figure 12 illustrates an embodiment of compression using a multi-core processing unit. Figures 1 3a through 1 3b illustrate geolocation and communication between colocation services in accordance with various embodiments. Figure 14 illustrates an exemplary delay associated with communication between the client and the colocation service. 139848.doc -154 · 201001177 Figure 15 illustrates an example host hosting service server central architecture. Figure 16 illustrates an example screen shot of an embodiment of a user interface including a plurality of live video windows. Figure 17 illustrates the user interface of Figure 6 after selecting a particular video window. Figure 18 illustrates the image after zooming in on a particular screen size to full screen size! User interface of 7. Figure 19 illustrates example collaborative user video material overlaid on a multiplayer game screen. Figure 20 illustrates an example user page for a game player on a colocation service. Figure 21 illustrates an example 3D interactive advertisement. Figure 22 illustrates an example sequence of steps for surface capture from a live broadcast performance to produce a photorealistic image with a textured surface. Figure 23 illustrates an example user interface page that allows selection of linear media content. Fig. 24 is a graph showing the amount of time elapsed and the speed of flail before the live web page is live. [Main component symbol description] 100 101 102 103 104 105

CPU/GPU random access memory (RAM) display device (SDTV/HDTV or computer monitor) hard disk drive optical media drive / optical media / CD player network connection 139848.doc • 155- 201001177 106 205 206 210 211 220 221 222 301 302 303 401 402 403 404 405 406 406a 406b 409 410 412 413 Game Controller Client Device / Client / Client Platform Internet Host Hosting Service / Hosting System User Location Game or Application Software Developer Input Device Display Device/Monitor or TV Nominal Maximum Data Rate Data Rate Required for Maximum Data Rate Hosting Service Control System/Hosting Service Control Server Server Storage Area Network (SAN) Video Compressed Logic/Video Compressor Independent Redundant Array (RAID) Control Signal Control Signal Control Signal Routing Logic Internet Low Latency Video Decompression Logic Control Signal Logic 139848.doc -156- 201001177 415 Client/Client Device/ Client computer 420 display device 421 input device 422 display device / display 440 internet winding Service 441 Presence 442 WAN Interface 443 Firewall/Router/NAT (Network Address Translation) Device 444 WAN Interface 451 One-way transmission time for transmitting control signals 452 Round-trip routing via user premises/user location routing 453 Arrow/round trip delay/user ISP 454 universal internet delay 455 one-way routing 456 maximum one-way delay 457 arrow/compression 458 video decompression 462 Ethernet jack/internet connection 463 HDMI (high definition Multimedia interface), connector 464 display capability 465 client device/user terminal 466 opaque glasses 468 display device / SDTV (standard definition television) or HDTV 139848.doc -157- 201001177 469 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 (high definition TV) / monitor input device client device flash memory video camera display device Bluetooth input device / wired USB input device bus bar Ethernet interface WiFi subsystem control CPU / Control Processor Bluetooth Interface USB Subsystem Video Decompressor Video Output Subsystem Audio Decompression Audio output subsystem interface system HDMI

491 DVI-I 492 S-Video 493 Composite Video 494 Digital Interface 495 Stereo Analog Interface 497 Ethernet 139848.doc -158- 201001177 499 Power 501 Uncompressed Video Frame 502 Uncompressed Video Frame 503 The compressed video frame 511 undergoes a reduced frame 512. The compressed frame 513 is subjected to a dust reduction frame 520. Compression logic 559-561 uncompressed video frame 611 I frame 612-613 P frame 620 compression Logic 621 Rated Maximum Data Rate 622 Available Maximum Data Rate / Available Data Rate 623 I Required Peak Data Rate / Peak Data Rate 624 Video Stream Data Rate / Video Stream Data Rate Sequence 633 I Frame Peak 634 Video Stream Data Rate Sequence/video stream data rate 635 P frame peak 636 P frame peak 670 B frame 671 I frame 701-760 uncompressed frame 711-770 R frame 139848.doc -159- 201001177 721-770 not Compressed frame 731-780 R frame 805 moving character 811 R frame 812 R frame 922 available maximum data rate 934 video stream brood rate 941 peak data rate / peak maximum Material rate 942 2 times peak data rate 943 3 times peak data rate 944 4 times ridge data rate 952 frame 2 times peak 953 flattened 2 times peak 954 frame 4 times bee value 955 flattened 4 times peak 961 Compressed frame 1 962 Uncompressed frame 2 963 Uncompressed frame 3 964 Uncompressed frame 4 965 Uncompressed frame 5 966 Uncompressed frame 6 967 Uncompressed Box 7 968 Uncompressed frame 8 969 Uncompressed frame 9 139848.doc * 160 - 201001177

970 uncompressed frame 10 981 compressed frame 1 982 compressed frame 2 983 compressed frame 3 985 compressed frame 5 986 compressed frame 6 987 compressed frame 10 991 Transmission time (xmit time) 992 Transmission time 993 Transmission time (2 times peak value) 995 Transmission time 996 Transmission time (4 times peak value) 997 Transmission time 1001-1005 Packet 1010 Image block package package logic 1100 I image block 1101 Forward error Correction Code (FEC) 1103 I Image Block 1104 Forward Error Correction Code (FEC) 1105 Forward Error Correction Code (FEC) 1110 Audio Packet 1111 FEC Code 1112 Audio Packet 1113 FEC Code 139848.doc • 161 - 201001177 1120 1121 1122 1123 1200 1201- 1205 1300 1301 1501 1502 1511 1512 1515 1521- 1529 1530 1539 1540 1550 1551 1552 1599 139848.doc User input command FEC code User input command FEC code multi-core and / or multi-processor 1208 Core 4 X 2 configuration / Uncompressed Frame Server Center/Host Hosting Service Network Incoming Internet Traffic Incoming Routing/Incoming Routing Network RAID Array RAID Array Column RAID Array / Delay Buffer 15 25 Application / Game Server Uncompressed Video / Audio Shared Video Compression / Shared Pool / Shared Hardware Compression / Shared Hardware Compressor / Shared Video Compression Hardware Compressed Video / Audio output routing/outbound routing network delay buffer and/or group video/feedback thin path path out internet communication-162- 201001177 1600 1601 1602 1700 1800 1801 1900 2000 2001 2002 2003 2004 2005 2400 2401 2402 2403 HS1-"Thumbnail" video window yellow selection box game information video window arrow overlay video and audio user's own video live broadcast video partner video clip / self-reward clip / 3D DVR self-review clip digital graphic line 1.5 Mbps connection speed 1 2 Mbps connection speed 96 Mbps connection speed HS6 colocation service server center 139848.doc -163 -

Claims (1)

201001177 VII. Application for Patent Range·· 1. A device comprising: a rear double tamping, and a camouflage to the video compression device, the video compression device transmits a cross-talk interactive video to a plurality of users And a feedback loop for the streaming interactive video to be recursively combined in the video round of the plurality of servers. 139848.doc