CN101918956A - System and method for compressing video by adjusting tile size based on detected intraframe motion or scene complexity - Google Patents

System and method for compressing video by adjusting tile size based on detected intraframe motion or scene complexity Download PDF


Publication number
CN101918956A CN 200880119436 CN200880119436A CN101918956A CN 101918956 A CN101918956 A CN 101918956A CN 200880119436 CN200880119436 CN 200880119436 CN 200880119436 A CN200880119436 A CN 200880119436A CN 101918956 A CN101918956 A CN 101918956A
Prior art keywords
Prior art date
Application number
CN 200880119436
Other languages
Chinese (zh)
Original Assignee
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US99965707A priority Critical
Priority to US11/999,657 priority
Application filed by 生命力有限公司 filed Critical 生命力有限公司
Priority to PCT/US2008/085600 priority patent/WO2009073825A1/en
Publication of CN101918956A publication Critical patent/CN101918956A/en



    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server
    • H04N21/6587Control parameters, e.g. trick play commands, viewpoint selection
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/137Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/14Coding unit complexity, e.g. amount of activity or edge presence estimation
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2383Channel coding or modulation of digital bit-stream, e.g. QPSK modulation
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/478Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4781Games


A system and method are described below for encoding interactive low-latency video using interframe coding. For example, one embodiment of a computer-implemented method for performing video compression comprises: detecting motion or high scene complexity within a sequence of images occurring at different regions within the sequence of images; logically subdividing each of the sequence of images into a plurality of tiles, each tile having a size selected based on the amount of motion detected in a region in which the tile is positioned; and encoding one or more of the tiles within each image of the sequence of images using a first compression format and encoding the remainder of the tiles within each image of the sequence of images using a second compression format.


用于通过基于所检测的帧内运动或场景复杂度调节图像块大小来压缩视频的系统及方法 A system and method for compressing video images by the complexity of adjusting the block size based on the detected scene or frame motion

[0001] 相关申请 [0001] RELATED APPLICATIONS

[0002] 本申请为2002年12月10日提交的、题为“用于无线视频游戏的装置及方法(Apparatus and Method for Wireless Video Gaming) ” 的、第10/315,460 号的部分继续(CIP)申请,该申请已转让给本CIP申请的受让人。 [0002] This application is December 10, 2002, filed, entitled "Device and method for wireless video game (Apparatus and Method for Wireless Video Gaming)", Part No. 10 / 315,460 to continue ( CIP) application, which application is assigned to the assignee of the present CIP application.

技术领域 FIELD

[0003] 本公开总体上涉及改善用户操作及存取音频和视频媒体的能力的数据处理系统的领域。 Relates to the field of data processing systems improve the ability of a user operation and access audio and video media on the [0003] present disclosure generally.

背景技术 Background technique

[0004] 自从托马斯·爱迪生(Thomas Edison)时代以来,记录的音频及电影媒体已成为社会的一方面。 [0004] Since Thomas Edison (Thomas Edison) era, recording audio and film media has an aspect of society. 在20世纪初期,记录的音频媒体(磁柱及唱片)及电影媒体(投币式自动点唱机及电影)广泛发行,但两种技术仍处于其起步阶段。 In the early 20th century, recorded audio media (cylinders and records) and motion picture media (coin-operated jukebox and movie) wide distribution, but both technologies were still in their infancy. 在20世纪20年代后期,在大众市场的基础上将电影与音频组合,之后将彩色电影与音频组合。 In the late 1920s, on the basis of motion pictures with audio in the mass market, followed by color motion pictures with audio. 无线电广播逐渐演变成很大程度上支持广告的形式的广播大众市场音频媒体。 Radio broadcasting gradually evolved into a largely supported by advertising in the form of mass-market audio broadcast media. 当在20世纪40年代中期建立电视(TV)广播标准时,电视与无线电以广播大众市场媒体的形式接合,从而将先前记录的或现场直播的电影带入家庭中。 When creating a television (TV) broadcast standard in the mid-1940s, television and radio broadcasting in the form of joining the mass media market, bringing previously recorded or live movie into the home.

[0005] 至20世纪中期为止,大部分美国家庭已具有用于播放记录的音频媒体的唱片播放器(phonograph record player)、用于接收现场直播的广播音频的无线电设备,及用于播放现场直播的广播音频/视频(A/V)媒体的电视机。 [0005] until the mid-20th century, most American homes had for playing recorded audio media disc player (phonograph record player), for receiving the live radio broadcast audio, and for playing live broadcast audio / video (a / V) media of television. 常常将所述3个“媒体播放器”(唱片播放器、无线电设备及TV)组合到共有公共扬声器的橱柜中,变成家庭的“媒体中心”。 Often the three "Media Player" (record player, radio and TV) were combined into one cabinet sharing common speakers, the family becomes a "media center." 尽管对于消费者而言媒体选择有限,但媒体“生态系统”非常稳定。 Despite the limited media choices for consumers, but the media "ecosystem" is very stable. 大多数消费者知道如何使用“媒体播放器”且能够享受其能力的全部范围。 Most consumers know how to use the "Media Player" and be able to enjoy the full range of their capabilities. 同时,媒体的出版商(大多为电影和电视工作室,及音乐公司)能够将其媒体分配给电影院与家庭两者,而不遭受广泛的盗版或“二次销售”(也即,已使用媒体的重新销售)。 Meanwhile, media publishers (mostly for film and television studios, and music labels) can be assigned to both the media and the cinema home, without suffering from widespread piracy or "secondary sales" (ie, the media has been used re-sales). 通常,出版商不会从二次销售得到收入,且因此, 二次销售减少了出版商对于新的销售从原本可自己使用媒体的购买者得到的收入。 In general, the publisher will not get revenue from secondary sales, and as such, it reduces revenue for publishers new sales derived from the original purchaser can use their own media. 尽管在20世纪中期期间的确存有已使用的唱片出售,但所述销售不会对唱片出版商有大影响,因为不同于电影或视频节目(其通常被成年人观看一次或仅数次),音乐曲目可被收听数百次或甚至数千次。 Although there certainly were used records sold during the mid-20th century, but the sale will not have a big impact on record publishers because, unlike a movie or a video (which is usually an adult viewing only once or several times), Music tracks can be listened to hundreds or even thousands of times. 因此,音乐媒体远比电影/视频媒体“经久”(也即,对于成年消费者而言,其具有持久价值)。 Therefore, the music media than film / video media "durable" (ie, an adult consumer, it has lasting value). 一旦购买了唱片,若消费者喜欢该音乐,则消费者可能将其保持很长时间。 Once purchased the records, if consumers like the music, then the consumer may keep it for a long time.

[0006] 自20世纪中期到现在,媒体生态系统对于消费者与出版商的利益及损失来说都经历了一系列根本改变。 [0006] Since the mid-20th century to the present, the media ecosystem for the benefit and the detriment of consumers and publishers who have experienced a series of radical changes. 在音频录音机(尤其是具有高质量立体声的盒式磁带)的广泛引入的情况下,的确有较高程度的消费者便利。 In the case of an audio recorder (especially those with high-quality stereo cassette) is widely introduced, there is indeed a higher degree of consumer convenience. 但其也标志着现在广泛的消费者媒体实践_盗版的开始。 But now it also marks the beginning of a wide range of consumer media practice _ piracy. 的确,许多消费者纯粹出于便利起见而使用盒式磁带来录制其自己的唱片,但日益增加的消费者(例如,宿舍中准备存取彼此的唱片收集的学生)将进行盗版复制。 Indeed, many consumers are used purely for convenience cassette tape to record their own records, but increasingly consumers (for example, a dormitory ready access to each record collections) would make pirated copies. 而且,消费者将录制经由无线电播放的音乐,而非从出版商购买唱片或磁带。 Moreover, consumers will be recorded music played over the radio, rather than buying a record or tape from the publisher.

[0007] 消费者VCR的出现导致更多的消费者便利,因为现在VCR被设置为记录TV节目, 其可在稍后时间观看,且VCR也导致视频租赁业的建立,其中电影以及TV节目设计可在“点播”基础上进行存取。 The [0007] consumer VCR led to even more consumer convenience, because now the VCR is set to record TV programs, which can be viewed at a later time, and the VCR also led to the creation of the video rental industry, where movies and TV programming can be accessed "on demand" basis. 自20世纪80年代中期以来的大众市场家庭媒体设备的快速开发已导致消费者的空前的选择及便利程度,且也导致媒体出版市场的快速扩张。 Rapid development since the mid-1980s mass-market home media devices has led to an unprecedented choice and convenience for consumers, and also led to the rapid expansion of media publishing market.

[0008] 现今,消费者面对过多媒体选择以及过多媒体设备,其中许多被绑定到特定形式的媒体或特定出版商。 [0008] Today, consumers are faced with choices as well as multimedia multimedia devices, many of which are tied to particular forms of media or particular publishers. 热衷的媒体消费者可能将一堆设备连接到房屋各房间中的TV及计算机,造成至一或多个电视机和/或个人计算机(PC)的“鼠窝式”电缆以及一群远程控制。 Avid consumer of media devices may be connected to a bunch of houses in each room of TV and computer, to cause one or more television and / or a personal computer (PC) of "rat's nest" and a group of cable remote control. (在本申请的上下文中,术语“个人计算机”或“PC”指代适合于在家庭或办公室中使用的任何种类的计算机),包括台式计算机、Macintosh麦金托什机器)⑧或其他非Windows (视窗)计算机、与Windows相容的设备、Unix变体、笔记本计算机等)。 (In the context of this application, the term "personal computer" or "PC" refers to any kind of computer is suitable for use in the home or office), including desktop computers, Macintosh Macintosh machines) ⑧ or other non-Windows (Windows) computer, the Windows-compatible devices, Unix variant, notebook computers, etc.). 所述设备可包括视频游戏控制台、VCR、DVD播放器、音频环绕音效处理器/放大器、卫星机顶盒、电缆TV机顶盒等。 The apparatus may include a video game console, VCR, DVD player, audio surround-sound processor / amplifier, satellite set-top box, cable TV set-top boxes. 此外,对于热衷的消费者,可能由于相容性问题而存在多个类似功能的设备。 In addition, for an avid consumer, may exist device more like functions because of compatibility issues. 举例而言,消费者可能拥有HD-DVD与蓝光(Blu-ray)DVD播放器两者,或MicrosoftXbox (微软家用游戏机)®与Sony Playstation (索尼游戏站)®视频游戏系统两者。 For example, consumers may have HD-DVD and Blu-ray (Blu-ray) both the DVD player or MicrosoftXbox (Microsoft's home console) ® with both the Sony Playstation (Sony PlayStation) ® video game system. 实际上,由于一些跨游戏控制台版本的的游戏的不相容性,消费者可能拥有XBox与稍后的版本(诸如,Xbox 360®)两者。 In fact, due to the incompatibility of some games across versions of game consoles, the consumer may own both an XBox and a later version (such as, Xbox 360®). 经常地,消费者对于使用哪个视频输入端及哪个远端感到迷惑。 Often, consumers which video input and which remote to use confused. 甚至在将光碟置放于正确的播放器(例如,DVD、HD_DVD、蓝光、Xbox或Playstation)中、选择用于该设备的视频及音频输入端且发现正确远程控制之后,消费者仍面临技术挑战。 Even after the disc is placed into the correct player (eg, DVD, HD_DVD, Blu-ray, Xbox or Playstation), select the device for video and audio input terminal and find the right remote control, consumers still face technical challenges . 举例而言,在宽屏DVD的状况下,用户可能需要首先确定正确的纵横比(例如,4 : 3、完全、放大、宽放大、 电影院宽等)且接着在其TV或监视器屏幕上设定正确的纵横比。 For example, in the case of wide-screen DVD, the user may need to first determine the correct aspect ratio (e.g., 4: 3, Full, Zoom, Wide Zoom, Cinema Wide, etc.) and then set on his TV or monitor screen the correct aspect ratio. 类似地,用户可能需要首先确定正确的音频环绕音效系统格式(例如,AC-3、杜比数字、DTS等)且接着设定正确的音频环绕音效系统格式。 Similarly, the user may need to first determine the correct audio surround sound system format (e.g., AC-3, Dolby Digital, DTS, etc.) and then set the correct audio surround sound system format. 时常,消费者未意识到其可能未享受到其电视或音频系统的全部能力下的媒体内容(例如,观看以错误纵横比挤压的电影,或收听立体声的音频而非环绕音效的音频)。 Often, consumers may not realize that their media content to the full capacity did not enjoy their TV or audio system (for example, watch the wrong aspect ratio of the extruded film, or listen to stereo audio rather than surround sound audio).

[0009] 日益增加地,已将以基于互联网的媒体设备添加到设备的堆栈中。 [0009] Increasingly, has been added to the stack will be the device's Internet-based media devices. 类似Sonos (索罗斯)®数字音乐系统的音频设备使音频直接从互联网流动(stream)。 Similarly Sonos (R) Digital Music system ® audio device audio directly from the Internet (stream). 同样地,类似Slingbox™(视灵宝™)娱乐播放器的设备记录视频且使其经由家庭网络流动或经由互联网流动而出,其中可在PC上远程观看该视频。 Likewise, similar Slingbox ™ (depending on Lingbao ™) entertainment player, video recording equipment and make it through the home network or out through the Internet, which can remotely view the video on a PC. 且互联网协议电视(IPTV)服务经由数字用户线(DSL)或其他家庭互联网连接而提供类似电缆TV的服务。 And Internet Protocol Television (IPTV) services services via a digital subscriber line (DSL) or other home Internet connection and provide cable TV-like. 近来还努力将多个媒体功能整合到单个设备(诸如,Moxi (摩西)⑧媒体中心及执行Windows XP媒体中心版本的PC) 中。 Recently also working to integrate multiple media functions into a single device (such as, Moxi (Moses) ⑧ Media Center running Windows XP Media Center Edition PC) in. 尽管所述设备中的每个设备对其执行的功能提供一点便利,但每个设备缺乏对大多数媒体的普遍且简单的存取。 Although the devices with each device providing convenience for the functions performed, each lacks most universal and simple media access. 另外,常常由于昂贵的处理和/或本地储存的需要而使得所述设备经常花费数百美元来制造。 Further, since the often expensive processing and / or local storage needs such devices frequently cost hundreds of dollars to manufacture. 另外,所述现代的消费者电子设备通常消耗大量电力,甚至当闲置时也消耗大量电力,这意谓着其随着时间而更加昂贵且浪费能源。 In addition, the modern consumer electronic devices typically consume large amounts of power even when idle also consume large amounts of electricity, which means they are more expensive over time and a waste of energy. 举例而言,若消费者忘记将设备切断或将其切换到不同视频输入端,则该设备可能继续操作。 For example, if the consumer neglects to turn it off or switches to a different video input terminal, then the apparatus may continue to operate. 此外,因为所述设备当中没有一个设备为完全的解决方案,所以必须将其与家庭中的其他设备的堆栈整合在一起,这仍对用户留下鼠窝式线及许多远程控制。 Furthermore, because none of the devices is a complete solution, it must be integrated with the other stack devices in the home, which still leaves the user with a rat's nest of wires and a remote control. [0010] 此外,当许多较新的以互联网为基础的设备适当地工作时,其通常提供更一般形式(与其原本可能可用的形式相比)的媒体。 [0010] Furthermore, when many newer Internet-based devices do work properly, they typically offer a more general form (as compared to its original form may be available) media. 举例而言,使视频经由互联网流动的设备常常仅使视频材料流动,而不能使常常伴随DVD的互动式“额外项目”流动,如视频的“制作”、 游戏或导演评论。 For example, the video equipment via the Internet often stream just the video material, rather than allow the flow of interactive "extras" are often accompanied by a DVD, such as "making of" video, games or director's commentary. 这是由于以下事实:互动式材料经常是以特定格式制作,该特定格式意欲用于在本地处理互动性的特定设备。 This is due to the fact that: interactive materials are often produced in a particular format, the specific format intended for specific equipment processing interactive locally. 举例而言,DVD、HD-DVD及蓝光光碟中每一者具有其自身的特定互动格式。 For example, DVD, HD-DVD and Blu-ray discs, each having its own specific interactive format. 任何家庭媒体设备或本地计算机(其可能经开发以支持所有流行格式)将需要一定程度的尖端性(sophistication)及灵活性,其将可能对于消费者操作而言过于昂贵及复杂。 Any home media devices or local computer (which may be developed to support all popular formats) will require a certain level of sophistication (sophistication) and flexibility, it will probably be too expensive and complicated for the consumer to operate.

[0011] 使该问题加重,若稍后在将来引入新格式,则本地设备可能不具有支持新格式的硬件能力,这将意味着消费者将必须购买升级的本地媒体设备。 [0011] Adding to the problem, if the later introduction of a new format in the future, the local device may not have the hardware capability to support the new format, which would mean that consumers would have to purchase equipment to upgrade the local media. 举例而言,若在稍后的日期引入较高分辨率的视频或立体视频(例如,每一只眼一个视频流),则本地设备可能不具有解码该视频的计算能力,或其可能不具有用于以新格式输出该视频的硬件(例如,假定通过与遮光眼镜(shuttered glassess)同步的120fps视频来实现立体视觉,其中将60fps 传送到每一只眼,若消费者的视频硬件仅可支持60fps视频,则该选项在缺乏升级的硬件购买的情况下将不可用)。 For example, if the introduction of a stereoscopic video or higher resolution video at a later date (e.g., one video stream for each eye), the local device may not have the computational capability to decode the video, or it may not have hardware for outputting the video in the new format (e.g., assuming stereoscopy is achieved through 120fps video synchronized with the shutter eyeglasses (shuttered glassess), with 60fps delivered to each eye, if the consumer video hardware can only support 60fps video, this option will not be available in the absence of upgraded hardware purchase).

[0012] 当谈及尖端的互动式媒体(尤其是视频游戏)时,媒体设备废弃及复杂度的问题为严重问题。 [0012] When it comes to sophisticated interactive media (especially video games), issue of media device obsolescence and complexity is a serious problem.

[0013] 现代视频游戏应用基本上划分成四个主要非便携式式硬件平台: SonyPlayStation ® 1、2 及3(PS1、PS2,及PS3) ;Microsoft Xbox ® 及Xbox 360 ®;及Nintendo Gamecube (任天堂方糖)⑧及Wi i™ ;以及以PC为基础的游戏。 [0013] Modern video game applications are largely divided into four major non-portable hardware platforms: SonyPlayStation ® 1,2 and 3 (PS1, PS2, and PS3); Microsoft Xbox ® and Xbox 360 ®; and Nintendo Gamecube (square Nintendo sugar) ⑧, and Wi i ™; and PC-based games. 所述平台中的每一者不同于其他者,使得被编写以在一个平台上执行的游戏通常不会在另一平台上执行。 Each of said platforms is different than the others so that games written to execute on one platform usually do not run on another platform. 也可能存在一代设备与下一代设备的相容性问题。 There may be compatibility issues generation equipment and next-generation devices. 即使大多数软件游戏开发者建立独立于特定平台而设计的软件游戏,为了在特定平台上执行特定游戏,也需要专有软件层(其经常被称为“游戏开发引擎」”)来调适游戏以在特定平台上使用。每一个平台以“控制台”(也即,附接到TV或监视器/扬声器的脱机盒(standalone box))的形式出售给消费者或其本身为PC。通常,视频游戏在诸如蓝光DVD、DVD_R0M或⑶-ROM的光学媒体上出售,该光学媒体含有体现为尖端的实时软件应用程序的视频游戏。随着家庭宽带速度增加,视频游戏正日益变得可用于下载。 Even though most software game developers to create software game independent of a particular platform designed to run a particular game on a particular platform, also needs to proprietary software layer (which is often called the "game development engine" ") to adapt the game for use on a specific platform. each platform "console" (i.e., attached to the offline cartridge TV or monitor / speakers (standalone box)) is sold to the consumer or to itself as a PC. generally, video games sold on optical media such as Blu-ray DVD, DVD_R0M or ⑶-ROM, which contains optical media embodied as a sophisticated real-time software applications and video games. with the increase in household broadband speeds, video games are becoming increasingly available for download .

[0014] 由于高级视频游戏的实时性及高计算要求而使得实现与视频游戏软件的平台相容性的特殊性要求极端苛刻。 [0014] Due to the real-time nature and high computational requirements of advanced video games and makes the realization of the special requirements of the compatibility of video game software platform for extremely demanding. 举例而言,一个人可能期望从一代视频游戏到下一代视频游戏(例如,自XBox 至XBox 360,或自Playstation 2(“PS2”)至Playstation 3(“PS3”)) 的完全游戏相容性,正如存在从一个PC到具有较快处理单元或核心的另一PC的生产力应用程序(例如,MicrosoftWord(微软文字处理软件))的普遍相容性。 For example, one might expect from one generation to the next video game video game (e.g., from XBox to XBox 360, or from Playstation 2 ( "PS2") to Playstation 3 ( "PS3")) the complete game compatibility , just as there is from one PC to another processing unit with a faster core or a PC productivity applications (eg, MicrosoftWord (Microsoft's word processing software)) universal compatibility. 然而,对于视频游戏并非是这种状况。 However, for video games is not the situation. 因为当发行一代视频游戏时,视频游戏制造商通常寻求对于给定价格点的最高可能性能,所以经常对系统进行动态架构改变,以使得被编写以用于前代系统的许多游戏在稍后一代系统上不能工作。 Because when issuing generation video game, video game manufacturers often seek the highest possible performance for a given price point, so often the dynamic architecture of the system changed so that was written for many games the previous generation system at a later generation It does not work on your system. 举例而言,XBox基于x86系列处理器,而XBox 360基于PowerPC系列。 For example, XBox based on x86 family of processors, XBox 360 and PowerPC-based series.

[0015] 可利用技术来模仿先前架构,但假定视频游戏为实时应用程序,则在模仿中达成完全相同的行为常常不切实际。 [0015] can use technology to mimic the previous architecture, but the behavior is assumed for real-time video game application, to achieve exactly the same in imitation often impractical. 这是对消费者、视频游戏控制台制造商及视频游戏软件出 This is a consumer, manufacturer of video game consoles and video game software out

5版商的损失。 Version 5's loss. 对于消费者而言,这意味着将旧的一代视频游戏控制台与新的一代视频游戏控制台两者保持接通到TV以便能够玩所有游戏的必要性。 For consumers, this means that the old generation of video game consoles and two new generation video game console to the TV to remain switched on all games can play. 对于控制台制造商而言,这意味着与新控制台的模仿及较慢采用相关的成本。 For console manufacturers, this means that the new console and associated costs imitate slower adoption. 且对于出版商而言,这意味着可能必须发行新游戏的多个版本以便涵盖所有潜在的消费者_不仅发行用于视频游戏的每个商标(例如,XBohPlaystation)的版本,而且常常发行用于给定商标的每个版本(例如,PS2及PS3) 的版本。 And for the publisher, which means you may have to release multiple versions of new games to cover all potential consumers of each trademark issue _ not only for a video game (for example, XBohPlaystation) versions, and are often issued for given to each version of the trademark (eg, PS2 and PS3) version. 举例而言,开发艺电有限公司(ElectronicArts)的“疯狂橄榄球08”的单独版本以用于XBox、XBox 360、PS2、PS3、Gamecube、Wii及PC平台以及其他平台。 For example, the development of Electronic Arts Inc. (ElectronicArts) of "Madden NFL 08" separate version for XBox, XBox 360, PS2, PS3, Gamecube, Wii and PC platforms and other platforms.

[0016] 便携式设备(诸如,移动电话及便携式媒体播放器)也对游戏开发商提出挑战。 [0016] portable devices (such as mobile phones and portable media players) also present challenges to game developers. 日益增加地,所述设备连接到无线数据网络且能够下载视频游戏。 Increasingly, the device connected to wireless data networks and are able to download video games. 但是,市场中存在具有多种不同显示分辨率及计算能力的多种移动电话及媒体设备。 However, the market there are many mobile phones and media devices having a plurality of different display resolutions and computing capabilities. 而且,因为所述设备通常具有电力消耗、成本及重量约束,所以其通常缺乏类似于图形处理单元(“GPU”)的高级图形加速硬件(诸如,由美国加州的圣克拉拉的NVIDIA(英伟达公司)制造的设备)。 Also, because such devices typically have power consumption, cost and weight constraints, they typically lack similar graphics processing unit ( "GPU") advanced graphics acceleration hardware (such as by the NVIDIA of Santa Clara, California (NVIDIA ) equipment manufacturers). 因此,游戏软件开发商通常开发同时用于许多不同类型的便携式设备的给定游戏标题。 Therefore, the game software developers typically develop simultaneously for many different types of portable devices of a given game title. 用户可发现:给定游戏标题不可用于其特定移动电话或便携式媒体播放器。 User may find: a given game title is not available for his particular cell phone or portable media player.

[0017] 在家庭游戏控制台的状况下,硬件平台制造商通常向软件游戏开发商收取用于在其平台上发布游戏的能力的版税。 [0017] In the home game console status, hardware platform manufacturers typically collect royalties for the ability to publish games on its platform to the software game developers. 移动电话无线通信公司通常也向游戏出版商收取用于将游戏下载到移动电话中的版税。 Mobile phone companies often charge a wireless communications to the game publisher for the game downloaded to the mobile phone in royalties. 在PC游戏的状况下,不存在用于发布游戏所支付的版税, 但由于用于支持多种PC配置及可能引起的安装问题的较高消费者服务负担而使得游戏开发商通常面临高成本。 In the case of PC games, there is no game for Royalty publish paid, but because of higher consumer services to support the burden of multiple PC configuration and installation issues that may arise while making the game developers typically face high costs. 而且,PC通常较少阻碍游戏软件的盗版,因为其可很容易地由精通技术的用户重新编程且游戏可更容易地被盗版且更容易地被分配(例如,经由互联网)。 Furthermore, PC is usually less hindered pirated game software, because it can be easily re-programmed by tech-savvy users and the game can be more easily pirated and more easily distributed (for example, via the Internet). 因此,对于软件游戏开发商而言,在游戏控制台、移动电话及PC上发行具有成本及不利之处。 Therefore, for game software developers, on the game console, mobile phone and PC issuance costs and disadvantages.

[0018] 对于控制台及PC软件的游戏出版商而言,成本不止于此。 [0018] For game consoles and PC software publishers, the cost more than that. 为了经由零售通道分配游戏,出版商向零售商收取低于出售价格的批发价格以使零售商具有利润率。 In order to distribute games through retail channels, publishers charge to retailers to sell below the price of the wholesale price to retailers with a profit margin. 出版商通常也必须支付制造及分配保存游戏的物理媒体的成本。 Publishers usually have to pay the cost of manufacturing and distribution of physical media to save the game. 零售商经常也向出版商收取“价格保护费”以涵盖可能的意外费用(诸如,游戏售不出,或游戏的价格降低,或零售商必须退还部分或所有批发价格和/或从购买者收回游戏)。 Retailers also often charged to the publishers' price protection fee "to cover possible contingencies (such as a game not for sale, or lower the price of the game, or retailer must refund part or all of the wholesale price and / or recovered from a purchaser game). 另外,零售商通常也向出版商收取用于帮助在广告传单中销售游戏的费用。 In addition, retailers often charge fees to help sell the game in advertising flyers to the publisher. 此外,零售商日益增加地从已玩完游戏的用户购买回游戏, 且接着将所述游戏以使用过的游戏出售,通常不与游戏出版商分享使用过的游戏的收入。 In addition, retailers are increasingly buying from users who have finished the game back to the game, and then the game to sell used games, do not usually share the revenue used games and game publishers. 以下事实增加了施加给游戏出版商的成本负担:游戏常常被经由互联网盗版及分配以供用户下载及进行免费复制。 The fact that increases the cost burden imposed upon game publishers: games are often pirated and distributed via the Internet for users to download and make free copies.

[0019] 随着互联网宽带速度增加且宽带连接性在美国及全世界变得更广泛(更具体地, 到家庭和到租赁连接互联网的PC的“网吧”),游戏被更多地经由下载而分配到PC或控制台。 [0019] As Internet broadband speeds increase and broadband connectivity becomes more widespread (more specifically, to families and to lease a PC connected to the Internet "Internet cafes") in the United States and around the world, the game is more via downloads assigned to the PC or console. 而且,宽带连接更多地用于玩多人及大型多人在线游戏(该两者在本公开中由首字母缩写词“MM0G”来指代)。 Moreover, broadband connections are increasingly used for playing multiplayer and massively multiplayer online games (both in this disclosure, the abbreviation "MM0G" to refer to by the initials). 这些改变减轻了与零售分配相关的一些成本及问题。 These changes alleviate some of the problems and costs associated with retail distribution. 下载在线游戏解决了游戏出版商的一些不利之处,因为分配成本通常较小且存在较少或不存在未出售媒体的成本。 Download online game publishers to solve some of the disadvantages of the game, because the distribution costs are typically smaller and there is less cost or unsold media does not exist. 但已下载的游戏仍被盗版,且由于其大小(大小常常为几十亿字节)而使得其可能花费非常长的时间来下载。 However, the downloaded game is still piracy, and because of its size (size often billions of bytes) so that it can take a very long time to download. 另外,多个游戏可装满小磁盘驱动器,例如连同便携式计算机一起或连同视频游戏控制台一起出售的那些磁盘驱动器。 In addition, multiple games can be filled with small disk drives, for example, together with portable computers or those disk drives sold with video game consoles. 然而,就游戏或MMOG需要在线连接以使得游戏可玩的程度而言,盗版问题得以减轻,因为通常需要用户具有有效的用 However, the game, or MMOG require an online connection to make the game playable degree, the piracy problem is mitigated, because usually require the user to have a valid use

6户帐户。 6 account. 不同于可由相机拍摄显示屏幕的视频或由麦克风记录来自扬声器的音频来复制的线性媒体(例如,视频及音乐),每个视频游戏体验是唯一的,且不可使用简单的视频/音频记录来复制。 May be different from the video camera display screen or a microphone to record audio from the speakers to replicate linear media (eg, video and music), each of the video game experience is unique and can not use a simple video / audio recorder to copy . 因此,甚至在未强力执行版权法且盗版猖獗的区域中,也可保护MMOG免于被盗版,从而可支持商业。 Therefore, even in areas not strongly enforced copyright laws and rampant piracy, but also protects the MMOG from being pirated, which can support business. 举例而言,已成功地部署Vivendi SA(维旺迪公司)的“魔兽世界”MM0G,而在全世界未遭受盗版。 For example, it has been successfully deployed Vivendi SA (Vivendi) "World of Warcraft" MM0G, but all over the world without suffering from piracy. 且许多在线或MMOG游戏(诸如,Linden Lab (林登实验室)的“第二人生”MM0G)通过建在游戏中的经济模型而产生游戏运营商的收入,其中资产可使用在线工具而带来、出售且甚至建立。 And many online or MMOG games (such as, Linden Lab (Linden Lab) of "Second Life" MM0G) to produce the game operator revenue through economic models built in the game, in which the assets can be brought to use online tools , sold, and even established. 因此,可使用除传统游戏软件购买或订阅之外的机制来为在线游戏的使用付费。 Therefore, the mechanism can be used in addition to the traditional game software purchases or subscriptions to pay for the use of online games.

[0020] 尽管由于在线或MMOG的性质而使得常常可减轻盗版,但在线游戏运营商仍面临其余挑战。 [0020] Although the nature of online or MMOG and often mitigated piracy, but online gaming operators still face remaining challenges. 许多游戏需要大量的本地(也即,家庭内)处理资源以供在线或MMOG适当地工作。 Many games require a lot of local (ie, home) processing resources for online or MMOG to work properly. 若用户具有低性能的本地计算机(例如,不具有GPU的计算机,诸如低端笔记本计算机),则其可能不能够玩该游戏。 If the user has a low performance of the local machine (for example, the computer does not have a GPU, such as low-end notebook computers), it may not be able to play this game. 另外,随着游戏控制台老化,其远落后于目前技术状态且可能不能够处理更高级的游戏。 Additionally, as game consoles age, far behind the current state of the art and it may not be able to handle more advanced games. 即使假定用户的本地PC能够处理游戏的计算要求,常常也存在安装复杂度。 Even assuming the user's local PC is able to handle the computational requirements game, there are often installation complexities. 可能存在驱动器不相容性(例如,若下载新游戏,则可能安装新版本的图形驱动器,其致使依赖于旧版本图形驱动器的先前已安装的游戏不可操作)。 There may be driver incompatibilities (for example, when downloading a new game, you may install a new version of the graphics driver that causes relied on the old version of the graphics driver has been previously installed games inoperable). 随着下载更多游戏,控制台可能用完本地磁盘空间。 With download more games, console may run out of local disk space. 当发现缺陷并修复时或若对游戏进行了修改(例如,若游戏开发商发现游戏的级别太难玩或太容易玩),复杂游戏通常随着时间推移而从游戏开发商接收下载的补丁(patch)。 When bugs are found and fixed, or if the game has been modified (for example, if the game developers found that levels of game play too hard or too easy to play), complex games usually receive downloaded patches from game developers with the passage of time ( patch). 该补丁需要新的下载。 The patch requires a new download. 但有时并非所有用户完成所有补丁的下载。 But sometimes not all users finish downloading all the patches. 在其他时候,下载的补丁引入其他相容性或磁盘空间消耗问题。 At other times, download the patch introduce other compatibility issues or disk space consumption.

[0021] 而且,在游戏播放期间,可能需要大数据下载以将图形或行为信息提供到本地PC 或控制台。 [0021] Also, during game play, you may need to download large data graphics or behavioral information to the local PC or console. 举例而言,若用户进入MMOG中的一个房间中,且遇到由图形数据组成或具有在用户的本地机器上不可用的行为的场景或人物,则必须下载那个场景或人物的数据。 For example, if the user enters MMOG in a room, and encounters a scene or a character from the graphic data or with unavailable on the user's local machine behavior, you must download the data that scene or character. 若互联网连接不够快,则此可导致玩游戏期间的实质延迟。 If the Internet connection is not fast enough, this can result in substantial delays during game play. 此外,若所遇到的场景或人物需要超过本地PC或控制台的储存空间或计算能力的储存空间或计算能力,则其可产生下述情形: 其中用户不能在游戏中继续,或必须以质量降低的图形继续。 In addition, if encountered scenes or characters need more than storage space or storage space or computing power computing power of the local PC or console, it can create a situation: where the user can not continue in the game, or must be based on quality reduce graphics continue. 因此,在线或MMOG游戏常常限制其储存和/或计算复杂度要求。 Thus, online or MMOG games often limit their storage and / or computational complexity requirements. 另外,其常常限制游戏期间的数据传送的量。 Additionally, they often limit the amount of data transferred during a game. 在线或MMOG游戏也可使可玩游戏的用户的市场变窄。 Online or MMOG games playable game also allows users to narrow the market.

[0022] 此外,精通技术的用户越来越多地对游戏的本地复本进行反向工程且修改游戏以使得他们可以作弊。 [0022] In addition, users are increasingly tech-savvy local copy of the game to reverse engineer and modify the game so that they can cheat. 作弊可能与进行比用人力可能的速度快的重复按钮按压(例如,为了非常快速地射击)一样简单。 Cheating may be more than is humanly possible speed repeated button presses (for example, to shoot very quickly) just as simple. 在支持游戏中资产交易的游戏中,作弊可达到导致欺骗性交易涉及具有实际经济价值的资产的欺诈程度。 In support of asset transactions enabled games, cheating can lead to achieve the degree of fraud fraudulent transactions involving assets have real economic value. 当在线或MMOG经济模型基于所述资产交易时,这可导致对游戏运营商的实质有害后果。 When online or MMOG economic model based on the asset transaction, which can lead to substantial detrimental consequences for game operators.

[0023] 开发新游戏的成本随着PC及控制台能够制作越来越尖端的游戏(例如,具有更逼真的图形(诸如,实时光线追踪),及更逼真的行为(诸如,实时物理学仿真))而增长。 [0023] The cost of developing new games with the PC and consoles are able to produce increasingly sophisticated games (for example, have a more realistic graphics (such as real-time ray tracing), and more realistic behavior (such as real-time physics simulation )) increases. 在视频游戏业的早期,视频游戏开发是应用程序软件开发非常类似的过程;也即,大多数开发成本在软件的开发中(与图形、音频及行为要素或“资产”的开发相对比),诸如可被开发以用于具有广泛特殊效果的电影的那些软件开发。 Early in the video game industry, video game development is very similar to application software development process; that is, most of the development costs of software development (development and graphics, audio and behavioral elements or "assets" relative ratio), such as those that may be developed in order to have a wide range of software for movie special effects development. 现今,许多尖端的视频游戏开发成果比软件开发更类似于富有特效的电影开发。 Today, many cutting-edge video game development efforts more akin to a movie full of special effects than the development of software development. 举例而言,许多视频游戏提供3-D世界的仿真,且产生更加真实(也即,看似与摄影拍摄的实景(live action)图像一样逼真的计算机图形) For example, many video games provide a 3-D simulation of the world, and produces more realistic (ie, seemingly shooting and photography of real (live action) as realistic computer graphics images)

7的人物、道具及环境。 7 characters, props and environments. 照片一样逼真的游戏开发的最具挑战方面中的一者为创建不能区别于实景人脸的计算机产生的人脸。 People photo-realistic of the most challenging aspects of game development for the creation of one indistinguishable from a live human face computer-generated face. 面部捕获技术(诸如,由加州的圣弗朗西斯科的Mova开发的Contour™(轮廓™)真实性捕获系统)捕获表演者的面部的精确几何形状并在表演者处于运动中时以高分辨率追踪表演者的面部的精确几何形状。 Facial capture technologies (such as (outline ™) Reality Capture developed by Mova of San Francisco, California Contour ™) to capture the precise geometry of a performer's face and high resolution is in motion to track the performances of performers the precise geometry of the face of the person. 此技术允许在PC或游戏控制台上再现3D面部,该3D面部实际上不能区别于所捕获的实景面部。 This technology allows the reproduction of the 3D face on a PC or a game console, the 3D face virtually indistinguishable from a real face captured. 精确地捕获及再现“照片一样逼真的”人脸在多个方面是有用的。 Accurately capture and reproduce "photo-realistic" human face is useful in several ways. 首先,高度可识别的名人或运动员常常用于视频游戏中(常常被高成本雇用),且不完美性可能对于用户而言显而易见,从而使观看体验(viewing experience)分心或令人不愉快。 First, the highly recognizable celebrity or athlete is often used in video games (often high cost of hiring), and imperfections may be apparent to the user, so that the viewing experience (viewing experience) or unpleasant distraction. 经常地,需要高度细节来实现高度的照片一样的逼真感_潜在地需要再现大量多边形及高分辨率纹理(在多边形和/或纹理在帧接帧的基础上随着面部移动而改变的情况下)。 Realism often, it is necessary to achieve high degree of detail of the photo-reproduction _ potentially require a large number of polygons and high-resolution textures (in and / or texture polygonal frame-by-frame basis as the face moves changed ).

[0024] 当具有详细纹理的高多边形计数场景快速改变时,支持游戏的PC或游戏控制台可能不具有足够的RAM来储存用于游戏片段中所产生的所需数目的动画帧的足够多边形及纹理数据。 [0024] When high polygon-count scenes with detailed textures change rapidly, the PC or game supports game console may not have sufficient RAM to store enough polygon game segment for the required number of animation frames generated in and texture data. 另外,通常可用于PC或游戏控制台上的单个光学驱动器或单个磁盘驱动器通常比RAM缓慢得多,且通常不能跟上GPU在再现多边形及纹理中可接受的最大数据速率。 Further, typically the single optical drive or single disk drive on a PC or game console is usually much slower than the RAM, and typically can not keep up with the GPU rendering polygons and textures acceptable maximum data rate. 当前游戏通常将大多数多边形及纹理载入到RAM中,这意味着给定场景在复杂度及持续时间上很大程度上受RAM的容量限制。 Current games are usually the most polygons and textures are loaded into RAM, which means that a given scene to a large extent by the capacity of the RAM limit on the complexity and duration. 在例如面部动画制作的状况下,这可能将PC或游戏控制台限制于并无真实感的低分辨率面部,或限制于仅可在游戏暂停且载入用于更多帧的多边形及纹理(及其他数据)之前在有限数目的帧中制作成动画的真实感面部。 In a situation such as facial animation, which may be a PC or a game console is not limited to low resolution realistic facial, or limited to only pause in the game and loads polygons and textures more frames ( and other data) prior to a limited number of animated frames in realistic facial.

[0025] 当PC或控制台显示类似于“正在载入...”的消息时,观看进程条跨屏幕缓慢地移动被现今的复杂视频游戏的用户公认为是内在缺点。 [0025] When the PC or console display similar to the "Loading ..." message when watching a progress bar move slowly across the screen are today's complex video games users recognized the inherent drawbacks. 下一个场景从磁盘(除非另外有条件,否则本文中的“磁盘”指非易失性光学媒体或磁性媒体,以及诸如半导体“闪存”存储器的非磁盘媒体)载入时的延迟可花费若干秒或甚至若干分钟。 Next scene loads from the disk (unless otherwise qualified herein, "disk" refers to non-volatile optical disk medium or magnetic media, such as semiconductor and "flash" memory) of delay can take several seconds or even several minutes. 这浪费时间且可能使游戏玩家相当沮丧。 This is a waste of time and may make gamers very frustrating. 如先前所述,大量或所有延迟可能是由于来自磁盘的多边形、纹理或其他数据的载入时间,但也可能是以下状况:当PC或控制台中的处理器和/或GPU准备用于场景的数据时,花费一部分载入时间。 As previously discussed, much or all of the delay may be due to the load time for polygon, textures or other data from a disk, but it may be the case: if the PC or console processor and / or GPU prepared for the scene data, takes part load time. 举例而言,英式足球视频游戏可允许玩家在大量玩家、小组、运动场及天气条件当中选择。 For example, a soccer video game may allow players to choose among a large number of players, teams, stadiums and weather conditions. 因此,取决于选择什么特定组合,可能需要用于场景的不同多边形、纹理及其他数据(统称“对象”)(例如,不同小组在其制服上具有不同色彩及图案)。 Thus, depending on what particular combination is chosen, it may be required for different polygons, textures and other data for the scene (collectively "objects") (e.g., different teams have different colors and patterns on their uniforms). 可能要列举各种排列中的许多或所有排列且提前预先计算对象中的许多或所有对象并将对象储存在用于储存游戏的磁盘上。 May be a variety of permutations include many or all permutations and pre-compute many or all of the object and store the objects in objects on the disk used to store the game. 但是,若排列的数目很大,则所有对象所需的储存量可能过大以致不能安装在磁盘上(或太不切实际以致不能下载)。 However, if the number of permutations, the storage capacity required for all of the objects may be too large to fit on the disk (or too impractical to download). 因此,现有的PC及控制台系统通常在给定场景的复杂度与播放持续时间两者上受约束且对于复杂场景遭受长的载入时间。 Therefore, the existing PC and console systems are usually on both the complexity of a given scene and play duration to be bound for complex scenes suffer from long load times.

[0026] 先前技术的视频游戏系统及应用程序软件系统的另一显著限制在于:其越来越多地使用例如3D对象的大数据库(诸如,多边形及纹理),所述大数据库需要被载入到PC或游戏控制台中以用于处理。 [0026] [0043] Another video game systems and application software systems that significantly limits the art: for example, it is increasingly a large database of 3D objects (such as polygons and textures), a large database needs to be loaded to a PC or a game console for processing. 如上所述,当将所述数据库在本地储存于磁盘上时,所述数据库可花费长时间来载入。 As described above, when the database is stored locally on disk, the database may take a long time to load. 然而,若数据库系储存于远程位置且经由互联网来存取,则载入时间通常严重得多。 However, if the database is stored a remote location and is accessed through the Internet, the load time is typically much more serious. 在此种情形下,下载大数据库可能花费几分钟、几小时或甚至几天。 In such cases, downloading a large database may take minutes, hours or even days. 另外, 所述数据库常常产生大量费用(例如,用于游戏、电影或历史记录片中的详细的高的有桅帆船的3D模型)且意欲用于销售给本地终端用户。 Further, such databases are often created a great expense (e.g., for a game, movie, or historical record of a detailed tall-masted sailing 3D model) and are intended for sale to the local end-user. 然而,一旦数据库被下载至本地用户,其就有被盗版的风险。 However, once the database is downloaded to the local user, that there is the risk of piracy. 在许多状况下,用户仅为了评估数据库来观看其是否适合用户的需要(例如,当用户执行特定移动时,用于游戏人物的3D服装是否具有满意的外观或外表) 的目的而希望下载数据库。 In many cases, only the user database to assess their suitability for viewing the user's needs (for example, when a user performs a specific move for 3D game character has a satisfactory appearance or fashion or appearance) purposes want to download the database. 对于在决定进行购买之前评估3D数据库的用户而言,长载入时间可能是阻碍。 Before deciding to purchase 3D assess user database, the long load times may be hindered.

[0027] 类似问题在MMOG (更具体地,如允许用户利用更加定制化的人物的游戏)中出现。 [0027] Similar problems in MMOG (more specifically, such as allowing the user to more customized character of the game) appear. 对于显示人物的PC或游戏控制台,其需要能够存取具有3D几何形状(多边形、纹理等)以及所述人物的行为(例如,若人物具有盾牌,则盾牌是否足够强以使矛偏转)的数据库。 Character display for PC or game console, which needs to have access 3D geometry (polygons, textures, etc.), and behavior of the person (e.g., if the character has a shield, whether the shield is strong enough to deflect a spear) of database. 通常,当MMOG由用户初次玩时,用于人物的大量数据库在游戏的初始复本下已经可用,游戏的初始复本在本地在游戏光盘上可用或被下载到磁盘。 Typically, when a user first play MMOG, extensive database for the characters already available in the initial copy of the game, the game's initial replica is available locally or downloaded to a disk on the game disc. 但是,随着游戏进展,若用户遇到数据库在本地不可用的人物或对象(例如,若另一用户已建立一定制人物),则在可显示该人物或对象之前,必须下载其数据库。 However, as the game progresses, if the user encounters a character or object in the database are not available locally (for example, if another user has created a customized character), then can be displayed before the character or object, you must download its database. 这可导致游戏的实质延迟。 This can result in a substantial delay of game.

[0028] 给定视频游戏的尖端性及复杂度,则在先前技术视频游戏控制台情况下对视频游戏开发商及出版商的另一挑战在于:开发视频游戏经常花费2年到3年,成本在数千万美元。 [0028] given video games of sophistication and complexity, is another challenge for video game developers and publishers is that in the case of prior art video game console: the development of video games often take two to three years, costs in millions of dollars. 假定新视频游戏控制台平台以大致每隔五年一次的速率引入,则游戏开发商需要在新游戏控制台发行之前的数年开始那些游戏的开发工作,以便在发行新平台时使视频游戏同时可用。 Given that new video game console platforms are generally introduced at a rate of once every five years, game developers need to work to develop the game in those years before the start of a new game console release, so that the video game at the same time at the launch of the new platform available. 来自竞争性制造商的若干个控制台有时大约同时发行(例如,彼此在一年或两年内),但尚待分晓的是每个控制台的流行性(例如,哪个控制台将产生最大的视频游戏软件销售)。 Competition from a number of manufacturers sometimes released around the same console (for example, in another year or two years), but it remains to be seen is the popularity of each console (for example, which console will have the greatest video game software sales). 举例而言,在最近的控制台周期中,Microsoft XBox 360、SonyPlaystation 3及Nintendo Wii计划在大约相同的大体时段引进。 For example, in a recent console cycle, Microsoft XBox 360, SonyPlaystation 3 and Nintendo Wii scheduled to be introduced around the same general time period. 但在所述引进之前的数年中,游戏开发商实质上必须“压注(place bets)”哪些控制台平台将比其他者更成功,且相应地投入其开发资源。 But in a few years prior to the introduction, the game developers essentially had to "pressure injection (place bets)" which console platforms are more successful than others, and devote their development resources. 电影制作公司也必须在电影发行之前很长时间基于其估计可能成功的电影而分摊其有限的制作资源。 Film production companies also have a long movie based on their estimated chance of success and share its limited production resources before the movie release. 给定视频游戏所需的投资的增长程度,则游戏制作越加变得类似电影制作,且游戏制作公司常规上基于其对特定视频游戏的将来成功的估计而投入其制作资源。 Given the extent of growth in video game investment required, the game production is increasingly becoming like film production, and production resources invested in its success based on its estimate of the future on a particular video game regular game production company. 但是,不同于电影公司,此压注并非仅基于制作本身的成功;而是,其依据于游戏要在其上执行的游戏控制台的成功。 However, unlike the film company, this bet is not only based on the successful production itself; rather, it is based on the game to be successful in the game console on which to perform. 同时在多个控制台上发行游戏可减轻风险,但此额外努力增加成本,且经常延迟游戏的实际发行。 Releasing the game on multiple consoles can reduce the risk, but this additional efforts to increase the cost, and often delayed the actual release of the game.

[0029] PC上的应用程序软件及用户环境正变得更为计算上密集、动态及互动,不仅使其在视觉上更吸引用户,而且使其更有用及直观。 [0029] application software and user environment on the PC is becoming more computationally intensive, dynamic and interactive, not only to make it more appealing to the user visually, and make it more useful and intuitive. 举例而言,新Windows Vista(视窗远景) ™操作系统与Macintosh®操作系统的后续版本两者并入了视觉动画效应。 For example, both the follow-up version of the new Windows Vista (Windows Vista) ™ operating system and Macintosh® operating systems incorporate visual animation effect. 高级图形工具(诸如,来自Autodesk(欧特克)公司的Maya™(玛雅™))提供非常尖端的3D再现及动画制作能力(其推动了目前技术状态的CPU及GPU的限制)。 Advanced graphics tools (such as from Autodesk (Autodesk) company Maya ™ (Maya ™)) provide very sophisticated 3D rendering and animation capabilities (which contributed to the current state of the CPU and the technical limitations of GPU). 然而,这些新工具的计算要求对于所述产品的用户及软件开发商而言产生了许多实际问题。 These calculations are new tools, however, required to produce a number of practical problems for the product users and software developers.

[0030] 因为操作系统(OS)的视觉显示必须在多种计算机(包括不再出售但仍可随着新OS而升级的前代计算机)上工作,OS图形要求在很大程度上受OS要用于的计算机(其通常包括不包括GPU的计算机)的最少共同点限制。 [0030] Because the operating system (OS) visual display must be (including no longer sold, but still with the new OS upgrade previous generation computers) working on a variety of computer, OS graphical requirements to be affected to a large extent OS a computer (which typically includes a computer does not include a GPU) a least common denominator. 这严重地限制OS的图形能力。 This severely limits the graphics capabilities of the OS. 此外,电池供电的便携式计算机(例如,笔记本计算机)限制视觉显示能力,因为CPU或GPU中的高计算活动通常导致较高电力消耗及较短电池寿命。 Furthermore, battery-powered portable computers (e.g., laptops) limit the visual display capability since high computational activity in a CPU or GPU typically results in higher power consumption and shorter battery life. 便携式计算机通常包括在不利用处理器时自动地减低处理器活动性以降低电力消耗的软件。 Portable computers typically include software that automatically lowers processor activity when the processor is not utilized in the software to reduce power consumption. 在一些计算机型号中,用户可手动地减低处理器活动性。 In some computer models, users can reduce the activity of the processor manually. 举例而言,Sony的VGN-SZ280P笔记本计算机包括在一侧上标记为"Stamina (持久性)”(用于低性能,更长电池寿命)且另一侧上标记为“Speed (速度)”(用于高性能,较短电池寿命)的交换器。 For example, Sony's VGN-SZ280P laptop computer includes indicia on one side "Stamina (Persistent)" (for low performance, more battery life) and the indicia on the other side "Speed ​​(speed)" ( for high performance, less battery life) switch. 在便携式计算机上执行的OS必须能够即使在计算机以其峰值性能能力的一小部分执行的情况下也可用地起作用。 An OS running on a portable computer must be able to function usably even in the case where the computer executes a fraction of its peak performance capability. 因此,OS图形性能常常保持为远低于目前技术状态的可用计算能力。 Thus, OS graphics performance often remains far below the state of the art available computational capability.

[0031] 经常出售高端的计算上密集的应用程序(如Maya),期望所述应用程序将用于高性能PC上。 [0031] Applications are frequently sold (e.g., Maya) high-end computationally intensive, it is desirable for the high performance application PC. 此通常产生高得多的性能,及更昂贵且便携性较差、最少共同点的要求。 This typically establishes a much higher performance, and more expensive and less portable, least common denominator requirement. 因此, 所述应用程序具有比通用OS(或通用生产力应用程序,类似Microsoft Office)有限得多的目标受众且通常以比通用OS软件或通用应用程序软件低得多的量出售。 Therefore, the application than the general-purpose OS (or general productivity applications, like Microsoft Office) is much more limited target audience and typically sell in volume than general purpose OS software or general purpose application software much lower. 潜在的受众进一步受限制,因为预期的用户时常难以提前试用所述计算上密集的应用程序。 Potential audience is further limited because the intended user is often difficult to advance the trial computationally intensive applications. 举例而言,假设学生希望了解如何使用Maya或已经知道所述应用程序的潜在购买者在购买中希望在进行投资之前试用Maya (此可能涉及也购买能够执行Maya的高端计算机)。 For example, assume that students want to learn how to use Maya or been aware of the potential buyers want to try the application before making an investment Maya (this may involve the purchase of high-end computer capable of running Maya) in the purchase. 当学生或潜在购买者可下载Maya的演示版本或得到Maya演示版本的物理媒体复本时,若其缺乏能够执行Maya的全部潜能(例如,处理复杂3D场景)的计算机,则其将不能够进行产品的全方位评估。 When a student or potential buyers can download a demo version of Maya Maya demo version or get a copy of physical media, if its full potential is capable of performing the lack of Maya (for example, dealing with complex 3D scenes) computer, it will not be able to assess the full range of products. 此实质上限制所述高端应用程序的受众。 This substantially limits the audience of end applications. 这也使出售价格变高,因为开发成本通常经过比通用应用程序的购买次数小得多的购买次数而分摊。 It also enables a high selling price, because development costs are usually much smaller than the number of purchases through the general application of the number of purchases and share.

[0032] 高价应用程序也对使用应用程序软件的盗版复本的个体及商业产生更多刺激。 [0032] high-priced applications also create more incentive for individual and business use of pirated copies of the application software. 因此,高端应用程序软件遭受猖獗盗版,尽管该软件的出版商进行了大量努力来通过各种技术减轻该盗版。 Therefore, the high-end application software suffers from rampant piracy, although the publisher of the software made considerable efforts to reduce the piracy by various techniques. 但是,甚至当使用盗版的高端应用程序时,用户也不可能排除投资昂贵的目前技术状态的PC来执行盗版复本的需要。 However, even when using high-end applications pirated, the user can not rule out investment in expensive state of the art PC to perform the required pirated copies. 因此,尽管用户可以用软件应用程序的实际零售价格的一小部分获得软件应用程序的使用,但盗版软件的用户仍需要购买或获得昂贵的PC,以便完全利用该应用程序。 Therefore, although the user can obtain the use of a software application using a fraction of the actual retail price of software applications, but users still need to buy pirated software or acquire expensive PC, in order to fully utilize the application.

[0033] 此对于高性能盗版视频游戏的用户同样成立。 [0033] This is also true for high-performance users of pirated video games. 尽管盗版者可以用游戏的实际价格的一小部分得到游戏,但其仍需要购买适当地玩游戏所需的昂贵计算硬件(例如,GPU-增强型PC,或类似XBox 360的高端视频游戏控制台)。 Although the pirates can get a game with a fraction of the actual price of the game, but still need to purchase needed to properly play the game expensive computing hardware (eg, GPU- enhanced PC, XBox 360 or a similar high-end video game console ). 假定视频游戏通常是消费者的娱乐, 则用于高端视频游戏系统的额外成本可能是过于昂贵的。 Given that video games are usually consumers of entertainment, the additional cost for high-end video game system may be too expensive. 该情形在当前工人的平均年收入相当低(相对于美国的当前工人平均年收入)的国家(例如,中国)中更糟。 The current situation in the country the average annual income of workers is quite low (relative to the US average annual income of current workers) (eg, China) in worse. 这样,小得多的百分比的人口拥有高端视频游戏系统或高端PC。 In this way, a much smaller percentage of the population with high-end video game system or a high-end PC. 在这些国家中,用户可支付费用以使用连接到互联网的计算机的“网吧”相当普遍。 In these countries, users pay a fee to use a computer connected to the Internet, "Internet cafes" are quite common. 经常地,所述网吧具有不具有高性能特征(诸如,原本可使玩家能够玩计算上密集的视频游戏的GPU)的较旧型号或低端PC。 Often, the Internet has (GPU such as, otherwise enable players to play computationally-intensive video games) do not have the high performance characteristics of older models or low-end PC. 这是在低端PC上执行的游戏成功的关键因素(诸如,Vivendi的“魔兽世界”,其在中国高度成功,且通常是在中国的网吧中玩)。 This is a key factor in the success of the game executed on the low-end PC (such as, Vivendi's "World of Warcraft" which is highly successful in China, and often play in Internet cafes in China). 相比之下,计算上密集的游戏(如“第二人生”)更不可能在安装于中国网吧中的PC上玩。 In contrast, a computationally-intensive games (such as "Second Life") is more likely to be playable on the PC installed in Internet cafes in China. 所述游戏实际上对于仅能够存取网吧中的低性能PC的用户来说是不可访问的。 Such games are virtually only able to access the Internet for a user in the low-performance PC is inaccessible.

[0034] 对于考虑购买视频游戏且首先愿意通过经由互联网将演示下载到其家庭而试用游戏的示范版本的用户也存在障碍。 [0034] For considering purchasing a video game and would first like to download to their families via the Internet and try the demo version of the game's demonstration users obstacles. 视频游戏演示常常为游戏的全能版本,其中一些特征停用,或对游戏播放的量施加限制。 Video game demo version of the game's all-around often, with some features disabled, or impose restrictions on the amount of game play. 此可能涉及在可将游戏安装于PC或控制台上且在PC或控制台上执行之前下载数十亿字节的数据的长过程(可能几个小时)。 This may involve in the game can be installed on a PC or console, and the long process of downloading several gigabytes of data (perhaps hours) before executing on a PC or console. 在PC的状况下,其也可能涉及算出游戏需要哪些特殊驱动器(例如,DirectX或OpenGL驱动器),下载正确的版本,安装正确的版本,及接着确定PC是否能够播放该游戏。 In the case of the PC, it may also involve figuring out which particular driver (for example, DirectX or OpenGL drivers) need to download the correct version, install the correct version, and then determine whether the PC is capable of playing the game. 后者步骤可能涉及确定PC是 The latter step may involve determining a PC

10否具有足够的处理(CPU及GPU)能力、足够的RAM及相容的OS (例如,一些游戏在Windows XP上执行而不在Vista上执行)。 No 10 has sufficient processing (CPU and GPU) capability, sufficient RAM, and compatible OS (for example, some games run on Windows XP without performing on Vista). 因此,在试图执行视频游戏演示的长过程之后,用户可能发现在给定用户的PC配置的情况下视频游戏演示不可能玩。 Thus, after a long process of trying to perform a video game demo, the user may find that in the case of a given user's profile to the PC game demo video can not play. 更糟地,一旦用户已下载新驱动器以用于尝试该演示,这些驱动器版本就可能与用户在PC上习惯使用的其他游戏或应用程序不相容,因此,演示的安装可致使先前可操作的游戏或应用程序不能操作。 To make matters worse, once the user has downloaded the new drive to try for the demo versions of these drives may not be compatible with other games or applications on the PC users accustomed to using, and therefore, the presentation of the installation may render previously operable game or application can not operate. 这些障碍不仅使用户沮丧,而且其也对视频游戏软件出版商及视频游戏开发商销售其游戏产生障碍。 These barriers not only frustrating for the user, but it can also create an obstacle for video game software publishers and video game developers sell their games.

[0035] 导致不具经济效益的另一问题与以下事实有关:给定PC或游戏控制台通常被设计以适应对应用程序和/或游戏的特定程度的性能要求。 [0035] leads to uneconomic another problem related to the fact: given PC or game console are typically designed to accommodate a certain level of performance requirements for the application and / or games. 举例而言,一些PC具有或多或少的RAM、较慢或较快的CPU及较慢或较快的GPU (若其具有GPU)。 For example, some PC have more or less RAM, slower or faster slower or faster CPU and the GPU (if it has the GPU). 一些游戏或应用程序利用给定PC或控制台的全计算能力,而一些游戏或应用程序却不利用给定PC或控制台的全计算能力。 Some games or applications take advantage of the full computing power of a given PC or console, and some games or applications do not take advantage of the full computing power of a given PC or console. 若用户的游戏或应用程序的选择未达到本地PC或控制台的峰值性能能力,则用户可能由于未利用的特征而在PC或控制台上浪费了财力。 If you select a game or application user did not reach peak performance capabilities of the local PC or console, users may not use due to the characteristics of wasted money on a PC or console. 在控制台的状况下,控制台制造商可能支付比资助控制台成本所要的多的成本。 In the case of a console, the console manufacturers may pay more than the cost of the console costs to be funded.

[0036] 存在于视频游戏的销售及享受中的另一问题涉及在用户实施购买游戏之前允许用户观看他人玩游戏。 [0036] exists in video game sales and other issues involved in the enjoyment of others allows users to view before the user buy the game to play the game. 存在用于记录视频游戏以在稍后时间重放的若干先前技术方法。 Exist for recording a video game in a number of prior art methods playback at a later time. 举例而言,美国专利第5,558,339号教导了在“游戏播放”期间将游戏状态信息(包括游戏控制器动作)记录在视频游戏客户端计算机(由同一个或不同用户拥有)中。 For example, U.S. Patent No. 5,558,339 teaches during "game play" the game state information (including game controller actions) recorded in the video game client computer (owned by the same or different user). 此状态信息可在稍后时间使用以在视频游戏客户端计算机(例如,PC或控制台)上重放一些或所有游戏动作。 This status information can be used at a later time to replay some or all of the game action on the video game client computer (eg, PC or console). 该方法的显著缺点在于:对于观看已记录的游戏的用户,用户必须具有能够播放该游戏的视频游戏客户端计算机且必须具有在该计算机上执行的视频游戏应用程序,以使得当重放被记录的游戏状态时游戏播放是完全相同的。 A significant disadvantage of this method is that: for user viewing the recorded game, the user must possess a video game client computer capable of playing the game and must have the video game application executing on the computer, so that when the reproduction is recorded when the game state of the game play is exactly the same. 除此之外,视频游戏应用程序必须是以在被记录的游戏与经回放的游戏之间不存在可能的执行差异的方式编写。 In addition, the application must be based on a video game is not possible ways to perform a differential exists between games and recorded by the replay games written.

[0037] 举例而言,游戏图形大体在帧接帧基础上计算。 [0037] For example, game graphics are generally based on a frame-by-frame calculation. 对于许多游戏,取决于场景是否特别复杂或是否存在减缓执行的其他延迟(例如,在PC上,另一过程可能正在执行,以致从游戏应用程序夺走CPU周期),游戏逻辑有时可能花费比一帧时间短或比一帧时间长的时间来计算为下一个帧而显示的图形。 For many games, depending on whether the scene is particularly complex or whether there are other delays slow down the implementation of (for example, on a PC, another process may be executing, so that CPU cycles away from the game application), game logic can sometimes take more than a shorter or longer than one frame time to calculate time graphic display of the next frame and the frame time. 在此种游戏中,以比一帧时间稍少的时间(例如,少几个CPU时钟周期)计算的“临限值”巾贞最终可出现。 In such a game, to slightly less than one frame time period (e.g., a few CPU clock cycles) to calculate a "threshold" Zhen towel can eventually occur. 当使用完全相同的游戏状态信息再次计算该同一场景时,可能容易花费比一帧时间多几个CPU时钟周期的时间(例如,若内部CPU总线稍微与外部DRAM总线不同相,且即使不存在来自从游戏处理夺走数毫秒CPU时间的另一过程的大延迟,其也引入几个CPU周期时间的延迟)。 When using the exact same game state information same scene is computed again, it could easily take more time than a few CPU clock cycles of time, e.g., if an internal CPU bus is slightly out of phase with the an external DRAM bus (and even absent from game processing away from the large delay of milliseconds of CPU time to another process, it introduces a few CPU cycle times of delay). 因此,当回放游戏时,帧变成以两个帧时间计算而非以单个帧时间计算。 Thus, when the game is played back the frame gets calculated in two frame times rather than a single frame time is calculated. 一些行为基于游戏计算新帧的频率(例如,当游戏取样来自游戏控制器的输入时)。 Some behaviors are based frequency game calculates a new frame (e.g., when the game samples the input from the game controllers). 当播放游戏时,用于不同行为的时间参考中的该偏差不会影响游戏播放,但其可导致所回放的游戏产生不同结果。 When the game is played, the time reference for different behaviors of the deviation does not impact game play, but it can result in the played game producing a different result. 举例而言,若篮球的轨道是以稳定的60fps速率来计算,但游戏控制器输入是基于计算的帧的速率来取样,则当记录游戏时, 计算的帧的速率可能为53fps,而当重放游戏时,计算的帧的速率可能为52fps,此可使得篮球是否被阻止进入篮中存在差异,从而导致不同结果。 For example, if a basketball track at a steady rate of 60fps to calculate, but the game controller input is based on rate of computed frames to sample, when the game was recorded, the rate of computed frames may be 53 fps, and when the weight when put the game, the frame rate may be calculated 52fps, so that this can be prevented from entering the basketball basket whether there are differences, leading to different results. 因此,使用游戏状态记录视频游戏需要非常谨慎的游戏软件设计,以确保使用同一游戏状态信息重放产生完全相同的结果。 Therefore, the game state recorded video games need to be very cautious game software designed to ensure that the use of the same game status information reproducing produce exactly the same results.

[0038] 用于记录视频游戏的另一先前技术方法是仅记录PC或视频游戏系统的视频输出(例如,到VCR、DVD记录器,或到PC上的视频捕获板)。 [0038] for recording of another video game prior art method is to simply record the video game system or a PC video output (e.g., to a VCR, DVD recorder, or to a video capture board on a PC). 接着可将视频回倒及重放,或替代地,将记录的视频上传到互联网(通常在将视频压缩之后)。 The video can then be rewound and replayed, or alternatively, the video recording uploaded to the Internet (usually after the video compression). 该方法的不利之处在于:当回放3D游戏序列时,用户限于仅从观看点(序列从该观看点被记录)来观看序列。 A disadvantage to this method is that: when a 3D game sequence is played back, the user is limited only from the point of view (the sequence is recorded from the point of view) to view sequence. 换言之, 用户不可改变场景的观看点。 In other words, the user can not change the viewing point of the scene.

[0039] 另外,当经由互联网而使在家庭PC或游戏控制台上播放的记录的游戏序列的经压缩的视频为其他用户可用时,即使视频是实时压缩,也不可能实时地将经压缩的视频上传到互联网。 [0039] In addition, when the compressed video game sequence played via the Internet on a home PC or a game console records available for other users, even if the video is compressed in real time, and can not be compressed in real time video uploaded to the Internet. 其原因是因为世界上连接到互联网的许多家庭具有高度不对称的宽带连接(例如,DSL及电缆调制解调器通常具有比上流带宽高得多的下流带宽)。 The reason for this is because many households connected to the Internet has a highly asymmetric broadband connections in the world (for example, DSL and cable modems typically have much higher upstream bandwidth than downstream bandwidth). 被压缩的高分辨率视频序列常常具有比网络的上传带宽容量高的带宽,使得其不可能实时上传。 Compressed high resolution video sequences often have higher bandwidths than the upload bandwidth capacity of the network, making them impossible to upload in realtime. 因此,在播放游戏序列之后(可能几分钟或甚至几小时),在互联网上的另一用户能够观看该游戏之前,将存在显著延迟。 Therefore, after playing the game sequence (may take several minutes or even hours), another user on the Internet can be viewed before the game, there will be a significant delay. 尽管该延迟在特定情形下(例如,观看在先前时间出现的游戏玩家的成果)可容忍,但其消除了观看游戏现场直播(例如,由优胜玩家玩的篮球锦标赛)的能力或现场直播地播放游戏时的“即刻重放”能力。 Despite the delay in certain circumstances (for example, watch the previous time appears gamers outcome) can be tolerated, it eliminates the ability to watch a game live (for example, played by champion players Basketball Championship) capability or played live "instant replay" capability of the game.

[0040] 另一先前技术方法允许具有电视接收器的观看者观看视频游戏现场直播,但仅在电视制作人员的控制下。 [0040] Another prior art method allows the viewer to watch a television receiver of the video game live, but only under the control of the television production crew. 美国与其他国家中的一些电视频道提供视频游戏观看频道,其中电视观众能够在视频游戏频道上观看特定的视频游戏用户(例如,参加锦标赛烦人顶级玩家)。 US video games to watch channels with other countries in some of the TV channels, including TV viewers can watch a particular video game user on the video game channel (eg, the tournament top-rated players). 这通过将视频游戏系统(PC和/或控制台)的视频输出馈送至用于电视频道的视频分配及处理设备中来完成。 This is accomplished by the video game system (PC and / or consoles) fed into the video output of the video distribution and processing equipment for the television channel. 这正如电视频道广播现场直播的篮球比赛时的情况,其中若干个相机从篮球场周围的不同角度提供现场直播的馈送。 It is like the case when the television channel broadcasting a live basketball game, in which several cameras provide live feeds from different angles around the basketball court. 电视频道接着能够利用其视频/音频处理及效应设备来操作来自各种视频游戏系统的输出。 TV channels can then use of their video / audio processing and effects equipment to manipulate the output from the various video game systems. 举例而言,电视频道可在来自视频游戏的视频之上叠加指示不同玩家的状态的文字(正如其可在现场直播的篮球比赛期间叠加文字),且电视频道可加录来自评论员(其可论述在比赛期间出现的动作)的音频。 For example, the TV channels can be superimposed text indicate the status of the different players in the top of the video from the video game (as it can overlay text during a live basketball game), and a television channel can record from commentators (which may discuss the action occurring during the game) audio. 另外,可将视频游戏输出与记录游戏的实际玩家的视频的相机(例如,显示玩家对游戏的情绪反应)组合。 Further, the video of the actual players of the video game output of the camera recording the game (e.g., display of a game player emotional response) in combination.

[0041] 该方法的一个问题在于:必须实时地使所述现场直播的视频馈送为电视频道的视频分配及处理设备可用,以便使其具有现场直播的广播的刺激性。 [0041] One problem with this method is that: the real time must be a live video feed to video distribution and processing equipment in the television channel is available, so that it has a stimulating broadcast live. 然而,如先前所述,当视频游戏系统从家庭执行时(尤其是当广播的一部分包括来自正捕获游戏玩家的真实世界视频的相机的现场直播的视频时),这常常不可能。 However, as previously described, when the video game system from execution when the family (especially when broadcast part includes live video being captured from the real world players of the game when the video camera), which is often impossible. 另外,在锦标赛情形下,所关注的是家庭中游戏者可修改游戏及作弊,如先前所述。 In addition, in a tournament situation, concern is families player can modify the game and cheating, as previously described. 由于这些原因,电视频道上的所述视频游戏广播常常配置有聚集于公共位置处(例如,在电视演播室处或在竞技场中)的播放器及视频游戏系统,其中电视制作设备可接受来自多个视频游戏系统及潜在的现场直播的相机的视频馈送。 For these reasons, such video game broadcasts on television channels are often arranged aggregated at a common location (e.g., at a television studio or in an arena) players and video game systems, where the television production equipment can accept from multiple video game systems and the potential of the camera's live video feed.

[0042] 尽管所述先前技术视频游戏电视频道可为电视观众提供非常刺激的演出(这是与现场直播的运动事件同类(例如,与以“运动员”呈现的视频游戏玩家同类)的体验,不仅根据其在视频游戏世界中的动作,而且根据其在真实世界中的动作),但这些视频游戏系统常常限于玩家彼此身体上极接近的情形。 [0042] Although the prior art video game television channels can provide a very exciting show to the television audience (which is akin to a live sporting event (for example, the video game player to "athlete" presented similar) experience, not only according to its actions in the video game world, but also according to their actions in the real world), but these are often limited to a video game system on player's body in close proximity to each other's situation. 此外,因为电视频道被广播,所以每个被广播的频道仅可显示由电视频道的制作人员选择的一个视频流。 Further, since television channels are broadcasted, each broadcasted channel can only show one video stream produced by a person selected television channel. 由于这些限制及广播时间、制作设备及制作人员的高成本,所述电视频道通常仅显示参加顶级锦标赛的顶级玩家。 Because of these limitations and the high cost of airtime, production equipment and production staff of the television channels often show only the top players to participate in top-level tournament.

[0043] 另外,向全部电视观众广播视频游戏的全屏幕图像的给定电视频道每次仅显示一 [0043] In addition, each time only to display full screen image of all television viewers broadcast video game given a television channel

12个视频游戏。 12 video games. 这严重地限制电视观看者的选择。 This severely limits a television viewer. 举例而言,电视观看者可能对给定时间显示的游戏不感兴趣。 For example, television viewers may not be interested in the game at a given time is displayed. 另一观看者可能仅对观看并非由电视频道在给定时间放映的特定玩家的游戏播放感兴趣。 Another viewer may only be interested in watching the game is not played by the television channel at a given time show a particular player. 在其他状况下,观看者可能仅对观看内行玩家如何处理游戏中的特定级别感兴趣。 In other cases, viewers could only watch expert players how to handle a certain level of interest in the game. 其他观看者可能希望控制观看点(视频游戏从该观看点来看),该观看点不同于由制作小组等选择的观看点。 Other viewers may wish to control the viewing point (a video game from the view point of view), which is different from the view point from the point of view of production team and other options. 简言之,电视观看者在观看视频游戏中可能具有无数的偏好(即使若干个不同电视频道可用,电视网络的特定广播也不适应所述偏好)。 In short, TV viewers may have a myriad of preferences (even if several different television channels available, specific broadcast television networks do not meet the preference) watching video games. 由于所有上述原因,使得先前技术视频游戏电视频道在向电视观看者呈现视频游戏中具有显著限制。 For all these reasons, prior art video game television channels have significant limitations in presenting to television viewers in a video game.

[0044] 先前技术视频游戏系统及应用程序软件系统的另一缺点在于:他们很复杂,且通常遭受错误、崩溃和/或无意识且不需要的行为(统称“缺陷”)。 [0044] Another drawback of prior art video gaming system and application software systems is that: they are complex, and often suffer from errors, crashes, and / or unintended and unwanted behavior (collectively, the "defects"). 尽管游戏及应用程序在发行之前通常经历除错及调谐过程(经常称为“软件质量保证”或SQA),但几乎不变的是: 一旦游戏或应用程序被发行到领域中的广大受众,缺陷就会突然出现。 Although games and applications typically go through debugging and tuning process (often referred to as "Software Quality Assurance" or SQA), but almost invariably is: once the game or application is released into the field of a wide audience, defect it will suddenly appear. 遗憾的是,软件开发商难以在发行之后识别及追踪到许多缺陷。 Unfortunately, the software developers is difficult to identify and track down many bugs after release. 软件开发商可能难以意识到缺陷。 Software developers may be difficult to realize defects. 即使当其了解缺陷时,也可能仅存在其可用于识别是什么引起该缺陷的有限量的信息。 Defects even when it is understood, that there may only be used to identify what caused the bug is a limited amount of information. 举例而言,用户可打电话给游戏开发商的消费者服务热线且留下消息,该消息陈述:当玩游戏时,屏幕开始闪烁,接着变成固体蓝(solid blue)且PC冻结。 For example, the user can call up a game developer's customer service line and leave a message, which states: When playing the game, the screen began to flicker, then become solid blue (solid blue) and the PC freezes. 其为SQA小组提供了在追踪缺陷中有用的非常少的信息。 Which provides defect tracking very little useful information to the SQA team. 在线连接的一些游戏或应用程序在特定状况下有时可提供更多信息。 Some games or applications online connection can sometimes provide more information under certain conditions. 举例而言,有时可使用”看门狗”过程来监视游戏或应用程序是否“崩溃”。 For example, sometimes use a "watchdog" to monitor the course of a game or application is "collapse." 看门狗过程可收集游戏或应用程序崩溃时关于游戏或应用程序过程的状态(例如,存储器堆栈使用状态、 游戏或应用程序进展到的程度等)的统计,且接着经由互联网而将所述信息上传至SQA小组。 The watchdog process can gather game or applications about the status of the game or application process (e.g., stack memory usage state, the game or application to the degree of progress) statistics, and then the information via the Internet uploaded to the SQA team. 但在复杂游戏或应用程序中,该信息可花费非常长的时间来解密,以便准确地确定在崩溃时用户正在进行什么。 But in a complex game or application, this information can take a very long time to decipher in order to accurately determine what the user is doing at the time of the crash. 尽管如此,也不可能确定什么事件序列导致崩溃。 Nevertheless, it is impossible to determine what the sequence of events leading to a crash.

[0045] 与PC及游戏控制台相关联的又一问题在于:其经受使消费者极不便利的服务问题。 [0045] A further problem with the PC and game console associated with that: it is subject to the consumer extremely convenient service issues. 服务问题也影响PC或游戏控制台的制造商,因为其通常需要发送特殊盒子以安全地装运破损的PC或控制台,且因而招致修理的成本(若PC或控制台处于保修期内)。 Service issues also affect PC or game console manufacturers, as it usually needs to send a special box to safely ship the broken PC or console, and thus incur the cost of repair (if the PC or console is in warranty). 游戏或应用程序软件出版商也可受处于修理状态中的PC和/或控制台引起的销售损失(或在线服务使用)影响。 Game or application software publishers can also be affected by the loss of sales in the PC and / or console due to repair state (or use the online service) influence.

[0046] 图1 说明诸如Sony Playstation ® 3、Microsoft Xbox 360 ®、NintendoWii™、以Windows为基础的个人计算机或Apple Macintosh的先前技术视频游戏系统。 [0046] Figure 1 illustrates such as Sony Playstation ® 3, Microsoft Xbox 360 ®, NintendoWii ™, Windows-based personal computer or Apple Macintosh prior art video game system. 所述系统中的每一者包括用于执行程序码的中央处理单元(CPU)(通常为用于执行高级图形操作的图形处理单元(GPU)),及用于与外部设备及用户通信的多个形式的输入/输出(1/0)。 A central processing unit (CPU) each of the system comprises means for executing a program code (typically a graphics processing unit for performing advanced graphical operations (GPU)), and multiple external devices and users to communicate with a form of input / output (1/0). 为简单起见,将所述组件显示为组合在一起为单个单元100。 For simplicity, the assembly is combined into a single display unit 100. 图1的先前技术视频游戏系统也显示为包括光学媒体驱动器104 (例如,DVD-ROM驱动器);用于储存视频游戏程序代码及数据的硬盘驱动器103 ;用于播放多人游戏、用于下载游戏、补丁、演示或其他媒体的网络连接105 ;用于储存当前正由CPU/GPU 100执行的程序码的随机存取存储器(RAM)IOl ;用于在游戏播放期间接收来自用户的输入命令的游戏控制器106 ;及显示设备102(例如,SDTV/HDTV 或计算机监视器)。 FIG prior art video game system 1 is also shown as including an optical media drive 104 (e.g., DVD-ROM drive); download games for playing multiplayer games, for; hard disk drive for storing video game program code and data of 103 , patches, demos or other media network connection 105; for storing program code currently being executed by the CPU / GPU 100 random access memory (RAM) IOl; for receiving input commands from the user during game play games The controller 106; and a display device 102 (e.g., SDTV / HDTV or a computer monitor).

[0047] 图1中所显示的先前技术系统受到若干限制。 Prior art systems [0047] Figure 1 suffers from several limitations. 首先,与RAM 101的存取速度相比较,光学驱动器104及硬碟机103往往具有慢得多的存取速度。 First, compared with access speed of the RAM 101, optical drives 104 and hard drives 103 tend to have much slower access speed. 当直接通过RAM 101工作时,由于RAM 101通常具有高得多的带宽且不会受到磁盘机构相对长的搜寻延迟的事实, When working directly through RAM 101, due to the RAM 101 typically has a much higher bandwidth, and disk mechanism will not be the fact that a relatively long search delay

13CPU/GPU 100在实践中可处理比直接从硬盘驱动器103或光学驱动器104读出程序代码及数据时可能的每秒多边形数多得多的每秒多边形数。 13CPU / GPU 100 can handle much more polygons per second than is possible number of polygons per second readout program codes and data directly from the hard drive 103 or optical drive 104 in practice. 但仅有限量的RAM提供于这些先前技术系统中(例如,256-512兆字节)。 But only a limited amount of RAM is provided in these prior art systems (e.g., 256-512Mbytes). 因此,常常需要“正在载入...”序列,其中RAM 101被周期性地填充有用于视频游戏的下一个场景的数据。 Therefore, often a "Loading ..." sequence, which RAM 101 is periodically filled for the next scene of the video game data.

[0048] 一些系统试图同时地重迭程序代码的载入与游戏播放,但这仅可在存在已知序列的事件时进行(例如,若正沿道路驾驶车,则可在驾驶车的同时载入路旁的正接近的建筑物的几何形状)。 [0048] Some systems attempt to overlap the program code is loaded and the game play, but this can only be carried out (for example, if the car driving down a road, you can load the car while driving in the event there is a known sequence the roadside approaching the geometric shape of the building). 对于复杂和/或快速场景改变,此类型的重迭通常不起作用。 For complex variations and / or rapid scene, this type of overlapping usually does not work. 举例而言, 在用户处于战役进行之中且在那时刻的视图内RAM 101完全被填满表示对象的数据的状况下,若用户将视图快速地向左移动以观看当前未载入在RAM 101中的对象,则将导致动作的不连续性,因为不存在足够的时间来将新对象自硬盘驱动器103或光学媒体104载入到RAM 101 中。 Under example, in the midst of a battle and the user at the time the view that RAM 101 is completely filled with data representing an object of the situation, if the user moves the view rapidly to the left to view is not currently loaded in RAM 101 objects will cause discontinuity in the action, since there is no time enough to load the new objects from hard drive 103 or optical media 104 into the RAM 101.

[0049] 图1的系统的另一问题是由于硬盘驱动器103及光学媒体104的储存容量的限制引起。 Another problem with the system [0049] FIG. 1 is a hard disk drive 103 due to the limitation and the storage capacity of optical media 104.. 尽管磁盘储存设备可被制造成有相对较大的储存容量(例如,500亿字节或500亿字节以上),但其仍不提供用于在当前视频游戏中所遇到的特定情况的足够储存容量。 Although disk storage devices can be manufactured with a relatively large storage capacity (e.g., more than 50 billion bytes or 50 billion bytes), but still do not provide for the specific case in current video games encountered sufficient storage capacity. 举例而言,如先前所述,英式足球视频游戏可允许用户在全世界的许多小组、玩家及运动场当中选择。 For example, as previously described, soccer video games can allow users to choose among dozens of teams around the world, players and stadiums. 对于每个小组、每个玩家及每个运动场,需要大量纹理映射及环境映射来特征化世界上的3D表面(例如,每个小组具有唯一运动衫,每一者需要唯一纹理映射)。 For each team, each player and each stadium, requires a lot of texture mapping and environment mapping to 3D surface characterization of the world (for example, each team has a unique jersey, each requiring a unique texture mapping).

[0050] 用于解决上述后者问题的一个技术是:对于游戏,一旦用户选择了纹理及环境映射,就预先计算纹理及环境映射。 A technique [0050] for solving the latter problem is: for the game, once the user selects a texture and environment maps, to pre-compute texture and environment maps. 此可涉及许多计算上密集的过程,包括解压缩图像、3D映射、加阴影、组织数据结构等。 This may involve a number of computationally-intensive processes, including decompressing images, 3D mapping, shading, organizing data structures, etc. 因此,当视频游戏执行这些计算时,对于用户可能存在延迟。 Thus, when the video game is performing these calculations, there may be a delay for the user. 减少此延迟的一个方法原则上为:最初开发游戏时执行所有这些计算_包括小组、玩家名册及运动场的每个排列。 One way to reduce the delay of this principle is: perform all these calculations _ each permutation includes team, player roster and stadium when the game was originally developed. 游戏的发行版本因而将包括储存在光学媒体104上或互联网上的一个或多个服务器上的所有所述经预先处理的数据,当用户作出选择时,仅经由互联网将用于给定小组、玩家名册、运动场选择的选定的预先处理的数据下载到硬盘驱动器103。 Release version of the game would then include a store or on the Internet or all of the pre-processed data on a plurality of servers on optical media 104, when the user makes a selection, only via the Internet for a given team, player roster, pre-processed data stadium selection downloaded to the hard disk drive 103. 然而,作为实际问题,游戏播放中可能的每个排列的该预先载入的数据可能轻易地为几兆兆字节(terabyte)的数据,其远超过现今的光学媒体设备的容量。 However, as a practical matter, the game play may be pre-loaded for each of the data arrangement could easily be terabytes (Terabyte) data, which is far more than the capacity of today's optical media devices. 此外,用于给定小组、玩家名册、运动场选择的数据可能轻易地为几亿字节的数据或几亿字节以上的数据。 In addition, for a given team, player roster, stadium selection may be hundreds of megabytes of data or more easily be hundreds of megabytes of data. 在家庭网络连接的情况下(例如,10Mbps),经由网络连接105下载该数据将比在本地计算数据花费更长时间。 With a home network connection (e.g., 10Mbps), 105 via a network connection to download the data takes longer than in the local computing data.

[0051] 因此,图1中所显示的先前技术游戏架构使用户在复杂游戏的较大场景转变之间经受显著延迟。 [0051] Thus, in Figure 1 shown prior art game architecture allows the user to significant delays between major scene transitions of complex games.

[0052] 诸如图1中所显示的先前技术方法的先前技术方法的另一问题在于:这些年来, 视频游戏倾向于变得更高级且需要更多CPU/GPU处理能力。 [0052] Another problem with prior art methods is that the technical methods such as those shown in Figure 1: Over the years, video games tend to become more sophisticated and require more CPU / GPU processing power. 因此,即使采用无限量的RAM, 视频游戏硬件要求也超过所述系统中可用的处理能力的峰值水平。 Thus, even assuming an unlimited amount of RAM, video games hardware requirements go beyond the peak level of the system's available processing power. 因此,需要用户每隔几年升级游戏硬件以保持同步(或以较低质量水准玩较新游戏)。 Therefore, users need to upgrade gaming hardware every few years to keep pace (or at a lower quality level playing newer games). 比以往更高级的视频游戏的趋势的后果为:用于家庭用途的玩视频游戏的机器通常不具经济效益,因为其成本通常由其可支持的最高性能游戏的要求来确定。 Consequence of the trend to ever more advanced video games are: Play video game machines for home use are typically economically inefficient, because the cost is usually its most demanding performance games can support is determined. 举例而言,可能使用XBox 360来玩类似“战争机器(Gears of War) ”的游戏,该游戏要求高性能的CPU、GPU及几亿字节的RAM,或者可能使用XBox 360来玩“吃豆(Pac Man) ”,其为来自20世纪70年代的游戏,其仅需要几千字节的RAM及非常低性能的CPU。 For example, using the XBox 360 might play a game like "Gears (Gears of War)" of the game that demands a high performance CPU, GPU, and hundreds of megabytes of RAM, or the XBox 360 might use to play "Pac (Pac Man) ", which is from the 20th century, the game of the 1970s, it requires only a few kilobytes of RAM and very low performance CPU. 实际上,XBox 360具有同时主机代管许多同时的“吃豆”游戏的足够计算能力。 In fact, XBox 360 has enough computing power to simultaneously host a number of "Pac" game while in escrow.

[0053] 在一周的大多数小时中,通常关闭视频游戏机。 [0053] In most hours a week, the normally closed video game consoles. 根据2006年7月Nielsen(尼尔森)娱乐对13岁及13岁以上的活跃游戏者的研究,平均起来,活跃游戏者一周中花费十四个小时或一周中的全部小时的仅12%来玩控制台视频游戏。 According to July 2006, more than 13 years of research and entertainment and 13-year-old active gamer Nielsen (Nielsen), on average, spend fourteen hours active all hours of the week or the week only player to play 12% of control video game. 这意味着平均视频游戏控制台在88%的时间内闲置,这是昂贵资源的无效率使用。 This means that the average video game consoles in the 88 percent idle time, which is an inefficient use of expensive resources. 假定视频游戏控制台常常是由制造商来资助以降低购买价格(期望该资助将通过来自未来视频游戏软件购买的版税来赚回), 则这特别有意义。 Video game consoles are often assumed to be financed by the manufacturer to reduce the purchase price (expect this funding will come from the future by video game software purchases to earn royalties), which is especially meaningful.

[0054] 视频游戏控制台也造成与几乎任何消费者电子设备相关的成本。 [0054] Video game consoles also causes related to almost any consumer electronics device cost. 举例而言,需要将系统的电子设备及机构容纳于外壳中。 For instance, the electronics and mechanisms of the systems need to be housed in an enclosure. 制造商需要提供服务保证。 Manufacturers need to provide service guarantees. 出售该系统的零售商需要收取关于系统的销售和/或关于视频游戏软件的销售的利润。 Retailers need to sell the system to receive profits on the sale of systems and / or sales of video game software. 所有这些因素添加视频游戏控制台的成本,该成本必须由制造商来资助、传递至消费者,或者由制造商与消费者两者来资助。 All of these factors add cost video game console, the costs must be funded by the manufacturer, delivered to the consumer, or be funded by both manufacturers and consumers.

[0055] 另外,盗版是视频游戏工业的较大问题。 [0055] In addition, piracy is a major problem video game industry. 实际上每个较大视频游戏系统上所利用的安全机构这些年来已“破裂”,导致视频游戏的未经授权的复制。 On virtually every major video game systems used by the security agencies over the years has been "broken", leading video game of unauthorized copying. 举例而言,Xbox 360安全系统在2006年7月破裂且用户现在能够在线下载非法复本。 For example, Xbox 360 security system ruptured in July 2006, and users can now download illegal copies online. 可下载的游戏(例如,用于PC或Mac的游戏)特别容易经受盗版。 Downloadable games (for example, for PC or Mac games) particularly vulnerable to piracy. 在世界的特定区域(其中盗版管制不强)中,实质上不存在独立视频游戏软件的可行市场,因为用户可与合法复本一般容易地以成本的非常小一部分购买盗版复本。 In certain regions of the world (where piracy control is not strong), the substantial absence of a viable independent video game software market, because users can easily buy legal copies of pirated copies of a very small part of the cost. 而且,在世界的许多地方,游戏控制台的成本占收入的高百分比, 以致即使盗版受控制,也很少有人可买得起目前技术状态的游戏系统。 Moreover, in many parts of the world, the cost of game consoles accounted for a high percentage of income, so that even if piracy were controlled, few people can afford the current state of the art gaming system.

[0056] 另外,已使用的游戏的市场减少了视频游戏业的收入。 [0056] In addition, the used game market reduces revenue for the video game industry. 当用户变得对游戏厌倦时, 其可将游戏出售给将游戏转售给其他用户的店铺。 When you become tired of a game, a game that can be sold to other users of the game resale shop. 这种未经授权但普遍的实践显著减少了游戏出版商的收入。 This widespread practice of unauthorized but a significant reduction in the revenue game publishers. 类似地,当每隔几年存在平台转变时,通常出现大约50%的销售减少。 Similarly, when there is a platform change every few years, usually about 50% of the sales decrease. 这是因为:当用户知道即将发行较新版本的平台时,用户停止购买用于较旧平台的游戏(例如,当即将发行Playstation 3时,用户停止购买Playstation 2游戏)。 This is because: when the user knows the more forthcoming when a new version of the platform, users stop buying games for older platforms (for example, when forthcoming 3:00 Playstation, users stop buying Playstation 2 games). 组合起来,销售的损失及与新平台相关的增加的开发成本可对游戏开发商的收益性有非常显著的不利影响。 Combined loss of sales related to the new platform and increase the cost of development can have a very significant negative impact on the profitability of game developers.

[0057]新游戏控制台也非常昂贵。 [0057] The new game consoles are also very expensive. Xbox 360、Nintendo Wii 及Sony Playstation 3 均以数百美元零售。 Xbox 360, Nintendo Wii and Sony Playstation 3 are hundreds of dollars retail. 高能力的个人计算机游戏系统可花费高达$8000。 High-capacity personal computer gaming systems can cost up to $ 8000. 这表示用户的显著投资,具体来说,考虑到硬件在几年后变陈旧及许多系统是为孩子而购买的事实。 This represents a significant investment of users, in particular, taking into account the hardware in a few years become obsolete and the fact that many systems are purchased for children.

[0058] 上述问题的一个方法是在线游戏,其中将游戏程序代码及数据主机代管于服务器上且按要求将其传送至客户端机器,经压缩的视频及音频经由数字宽带网络而流动。 A method of [0058] the above-described problems is online gaming in which the gaming program code and data are hosted on a server and transfer it to the requirements of the client machine, the compressed video and audio streamed over a digital broadband network. 一些公司(诸如,芬兰的G-ClusteHG-群集公司),其现在为日本的SOFTBANK Broadmedia(软银宽媒)的子公司)当前在线提供所述服务。 Some companies (such as G-ClusteHG- cluster Finland companies), which is now a subsidiary of SOFTBANK Broadmedia (SOFTBANK media width) in Japan) currently provide these services online. 类似游戏服务变得在本地网络(诸如,旅馆内及由DSL及电缆电视提供者提供的那些网络)中可用。 Similar games services become (such as those provided in the hotel and by the network to provide DSL and cable television) are available in the local network. 这些系统的较大缺点是延时的问题,也即,信号行进到游戏服务器及从游戏服务器行进所花费的时间,游戏服务器通常定位在运营商的“前端”中。 A major drawback of these systems is the problem of latency, ie, the signal travels to the game server and game server from the time it takes to travel, which is typically located in the operator's "front end" in. 快速动作视频游戏(也称为“极速(twitch)”视频游戏)在用户通过游戏控制器执行动作的时间与更新显示屏幕以显示用户动作的结果的时间之间需要非常低的延时。 Fast action video game (also known as "speed (twitch)" video game) the user through the game controller executes the action of time and update the display screen to display a very low latency time required between the results of user actions. 需要低延时,以使得用户感觉到游戏“即刻地”响应。 Low latency is needed to enable the user to feel the game "swiftly" in response. 可视游戏的类型及用户的熟练程度而以不同延时间隔来满足用户。 Types of user proficiency visual game with different latency intervals to satisfy users. 举例而言,对于缓慢的非正式游戏(类似西洋双陆棋)或慢动作角色扮演游戏而言,100毫秒的延时可能是可容忍的,但在快动作游戏中,超过70毫秒或80毫秒的延时可引起用户在游戏中更拙劣地表现,且因此不可接受。 For example, for a slow casual game (like backgammon) or a slow-action role playing game, 100ms of latency may be tolerable, but in a fast action game in excess of 70 or 80ms the delay may cause the user to perform more poorly in the game, and thus is unacceptable. 举例而言,在需要快反应时间的游戏中,当延时自50毫秒增加至100毫秒时,存在准确度的锐降。 For example, in a game that requires fast reaction time, when the delay increases from 50 milliseconds to 100 milliseconds, sharp declines in the presence of accuracy.

[0059] 当游戏或应用服务器安装在附近的受控网络环境或至用户的网络路径可预测和/ 或可容忍带宽峰值的网络环境中时,在最大延时以及延时的一致性方面,控制延时容易得多(例如,因此用户经由网络观察到来自数字视频流动的稳定运动)。 [0059] When the controlled network environment in the game or application server installed near the user or the network path to the predictable and / or can tolerate bandwidth peaks network environment, consistency in delay and maximum delay control much delay (e.g., via a network so the user observes steady motion from digital video streaming) easily. 该程度的控制可在以下达成:在电缆TV网络前端到电缆TV用户的家庭之间,或自DSL中央办公室至DSL用户的家庭,或在来自服务器或用户的商业办公室区域网络(LAN)环境中。 Such level of control can be achieved in the following: a cable TV network head-end to a cable TV subscriber home, or from a DSL central office to DSL subscriber's home, or in a commercial office LAN from a server or a user (LAN) environment . 而且,有可能获得商业之间的具有得到保证的带宽及延时的特定分级的点到点私用连接。 Further, it is possible to obtain specially-graded with guaranteed bandwidth and latency between the commercial point private connections. 但在将游戏主机代管于连接到通用互联网的服务器中心中且接着经由宽带连接而使经压缩的视频流动(stream) 到用户的游戏或应用系统中,许多因素造成延时,导致先前技术系统的部署中的严重限制。 But in a game server hosting center to be connected to the general Internet and then through a broadband connection to the user's game or application system, a number of factors delay the flow of the compressed video (Stream), results in prior art systems severely limited deployment.

[0060] 在典型的连接宽带的家庭中,用户可具有用于宽带服务的DSL或电缆调制解调器。 [0060] In a typical broadband-connected home, a user may have a DSL or cable modem for broadband service. 所述宽带服务通常造成用户的家庭与通用互联网之间的多达25毫秒的来回行程延时(且有时更多)。 Such broadband services commonly incur as much as 25 ms round-trip latency between the home and the general Internet users (and sometimes more). 另外,存在由于经由互联网将数据路由到服务器中心而造成的来回行程延时。 Further, since the presence of routing data through the Internet to a server center caused roundtrip latency. 经由互联网的延时基于给出数据的路线及数据被路由时数据所造成的延迟而改变。 It varies based on the delay time caused by the routing data and the route data given data via the Internet latency. 除路由延迟之外,还由于光穿过使大多数互联网互连的光纤的速度而造成来回行程延时。 In addition to routing delays, also because the speed of light traveling through the optical fiber caused by most Internet interconnected round-trip latency. 举例而言,对于每1000英里,由于光穿过光纤的速度及其他耗用而造成约22毫秒的来回行程延时。 For example, each 1000 miles, since the speed of light through the optical fiber and other overhead caused by the round-trip latency for about 22 milliseconds.

[0061] 额外延时可由于经由互联网流动的数据的数据速率而造成。 [0061] Additional latency can occur due to the resulting data via the Internet data flow rate. 举例而言,若用户具有以“6Mbps DSL服务”出售的DSL服务,则在实践中,用户将很可能最多得到小于5Mbps 的下行输送量,且将可能周期性地看见由于各种因素(诸如,峰值载入时间期间在数字用户线接入复用器(DSLAM)处的拥挤)产生的连接降级。 For example, if a user has DSL service to "6Mbps DSL service" for sale, then in practice, the user will probably get less than 5Mbps downlink transport amount up to, and will likely see periodically due to various factors (such as, during the time of peak load congestion digital Subscriber line access multiplexer (DSLAM) at) connected to generate degradation. 若经由相邻者循环的本地共用同轴电缆中存在拥挤或电缆调制解调器系统网络中的其他地方存在拥挤,则类似问题可出现, 从而将用于以“6Mbps电缆调制解调器服务”出售的连接的电缆调制解调器的数据速率减小至远小于该数据速率。 If the cycle via the local shared coaxial cable neighbor there is congestion or congestion exists elsewhere in the cable modem system network, the similar problems may occur, such that for a cable modem connection as "6Mbps cable modem service" sold the data rate is reduced to much less than the data rate. 若使4Mbps的稳定速率下的数据分组以用户数据报协议(UDP)格式单向地从服务器中心经由所述连接而流动,若一切都适当地工作,则数据分组将通过而不造成额外延时,但若存在拥挤(或其他妨碍)且仅3. 5Mbps可用于使数据流动到用户,则在典型情形下,包将被丢弃,导致丢失数据,或者分组将在拥挤点处排队直至它们可被发送为止,从而引入了额外延时。 If data packets in User Datagram Protocol (UDP) format from a server flows unidirectionally through the center of the connection at a steady rate of 4Mbps, if everything is working well, the data packets will pass through without incurring additional latency However, if there is congestion (or other impediments) and only 3. 5Mbps can be used to stream data to the user, then in a typical situation either packets will be dropped, resulting in lost data, or packets will queue up at the point of congestion, until they can be send up, introducing additional delay. 不同拥挤点具有用于保存被延迟的分组的不同队列容量,因此在一些状况下,立即将不可成功解决拥挤的分组丢弃。 Different points of congestion have different queuing capacity to hold delayed packets, so in some cases, immediately through the congestion of packet dropping. 在其他状况下,将几百万比特的数据排队且最终将其发送。 In other cases, several megabits of data are queued up and eventually be sent. 但是,在几乎所有状况下,拥挤点处的排队具有容量限制,且一旦超过该限制,队列将溢出且分组将被丢弃。 However, in almost all cases, queues at points of congestion have capacity limits, and once those limits are exceeded, the queues will overflow and packets will be dropped. 因此,为了避免造成额外延时(或更糟地,分组丢失),必须避免超过从游戏或应用服务器到用户的数据速率容量。 Therefore, in order to avoid incurring additional latency (or worse, packet loss), you must avoid exceeding the game or application from the server to the user's data rate capacity.

[0062] 还由于在服务器中压缩视频及在客户端设备中解压缩视频所需的时间而造成延时。 [0062] Due to further compress video in the server and the time required to decompress video in the client device and cause delay. 当在服务器上执行的视频游戏正在计算待显示的下一个帧时,进一步造成延时。 When the video game executing on servers in a frame to be displayed is calculated, resulting in further delay. 当前可用的视频压缩算法受到高数据速率或高延时。 Currently available video compression algorithms suffer from high data rates or high latency. 举例而言,运动JPEG为仅帧内有损的压缩算法,该压缩算法特征为低延时。 For example, motion JPEG is an intraframe-only lossy compression algorithm, compression algorithm that is characterized by low-latency. 视频的每个帧独立于视频的每个其他帧而压缩。 Each frame of video is independent of every other frame of video and compression. 当客户端设备接收经压缩的运动JPEG视频的一个帧时,其可立即解压缩该帧且显示该帧,从而导致非常低的延时。 When a client device receives a frame of compressed motion JPEG video, it can immediately decompress the frame and display of the frame, resulting in very low latency. 但因为每个帧分开进行压缩,所以算法不能够利用连续帧之间的类似性,且因此仅帧内视频压缩算法受到非常高俄数据速率。 But because each frame is compressed separately, the algorithm is unable to exploit similarities between successive frames, and thus intraframe-only video compression algorithms suffer from very high data rates. 举例而言,60fps (每秒帧数)640X480 运动JPEG视频可能需要40Mbps (每秒百万比特)或40Mbps (每秒百万比特)以上的数据。 For example, 60fps (frames per second) 640X480 motion JPEG video may require 40Mbps (megabits per second) or 40Mbps (megabits per second) or more of data. 用于所述低分辨率视频窗的所述高数据速率在许多宽带应用程序中将是过于昂贵的(且对于大多数消费者的基于互联网的应用程序的确如此)。 The high data rate for the low-resolution video window in a number of applications in the broadband is too expensive (and true for most consumers of Internet-based applications). 另外,因为每个帧经独立压缩,所以可能由于有损压缩而产生的帧中的假影可能出现于连续帧中的不同位置处。 Further, because each frame is compressed independently by, the frame may be due to lossy compression generated artifacts may appear at different locations in successive frames. 这可导致当解压缩视频时,在观看者看来为移动的视觉假影。 This can result when decompressing video, the viewer seems to moving visual artifacts.

[0063] 其他压缩算法(诸如,来自Microsoft公司的MPEG2、H. 264或VC9)当用于先前技术的配置中时,可实现高压缩比率,但以高延时为代价。 [0063] Other compression algorithms (such as, MPEG2 from Microsoft Corporation, H. 264 or VC9) used in prior art configurations, can achieve high compression ratios, but at the cost of high latency. 所述算法利用帧间压缩以及帧内压缩。 Algorithms utilize interframe as well as intraframe compression. 周期性地,所述算法执行帧的仅帧内压缩。 Periodically, such algorithms perform an intraframe-only compression of a frame. 这种帧被称为关键帧(通常称作“I”帧)。 Such a frame is known as a key frame (typically referred to as "I" frame). 接着,该算法通常将I帧与先前帧与相继帧两者相比较。 Then, these algorithms typically compare the I frame with both prior frames and successive frames. 并非独立地压缩先前帧及相继帧, 而是算法确定图像从I帧到先前帧及相继帧有什么改变,且接着将该改变储存为:“B”帧(对于I帧之前的改变)及“P”帧(对于I帧之后的改变)。 Not compressed independently of prior frames and successive frames, the algorithm determines the image from the I frame to the prior and successive frames there is any change, and then stores those changes as: "B" frame (I-frame prior to the change), and " P "frames (for changes following the I frame). 这导致比仅帧内压缩低得多的数据速率。 This results in much lower than intraframe-only compression data rate. 但是,其通常以较高延时为代价。 However, it is usually at the cost of higher latency. I帧通常比B帧或P帧大得多(常常大10 倍),且因此,以给定数据速率传输成比例地花费较长的时间。 Than the I frame is typically so much B or P frame (often 10 times larger), and to proportionally take longer a given data rate transmission.

[0064] 考虑(例如)一个i情形:其中I帧为B帧及P帧的大小的10倍,且对于每个单个I帧内,存在29个B帧+30个P帧=59个帧间,或对于每个“帧群”(GOP)总共60个帧。 [0064] considerations (e.g.) a case where i: wherein B is 10 times the size of the frames and P frames I frames, and for every single I frame, there are 29 B frames + 30 P frames = 59 inter , or 60 frames total for each "frame group" (GOP). 因此,在60fps下,每秒存在1个60帧G0P。 Thus, in the of 60 fps, there is 1 60 G0P per second. 假设传输信道具有2Mbps的最大数据速率。 Suppose the transmission channel has a maximum data rate of 2Mbps. 为了在信道中达成最高质量的视频,压缩算法将产生2Mbps数据流,且给定上述比率,这将产生每帧内2百万比特(Mb)/(59+10) = 30,394个比特及每I帧303,935个比特。 To achieve the highest quality video in the channel, the compression algorithm would produce a 2Mbps data stream, and given the above ratios, this would result in 2 Megabits (Mb) / (59 + 10) = 30,394 bits and 303,935 bits per I frame. 当通过解压缩算法接收经压缩的视频流时,为了稳定地播放视频,需要以规则间隔(例如,60fps)解压缩及显示每个帧。 When the compressed video stream received by the decompression algorithm, in order to stably video, need to be decompressed and displayed at regular intervals of each frame (e.g., 60fps). 为了实现该结果,若任何帧受到传输延时,则需要将所有帧延迟至少该延时,因此最糟状况的帧延时将限定用于每个视频帧的延时。 To achieve this result, if any frame is subject to transmission latency, all of the frames need to be delayed at least that latency, so the worst-case frame latency will define the latency for every video frame. 因为I帧最大,所以I帧引入最长传输延时,且整个I帧将必须在可解压缩及显示I帧(或取决于I帧的任何帧间)之前接收。 Since the largest I frame, the I frames introduce the longest transmission delay, and an entire I frame would have to be decompressed and displayed before receiving an I frame (or any interframe dependent on the I frame) in a. 假定信道数据速率为2Mbps,则传输I帧将花费303,935/2Mb = 145毫秒。 Given that the channel data rate is 2Mbps, the transmission of the I frame it will take 303,935 / 2Mb = 145 msec.

[0065] 使用传输信道的带宽的大百分比的帧间视频压缩系统(如上所述)将由于I帧相对于帧的平均大小的大的大小而经受长延时。 Large percentage of [0065] the use of the transmission channel bandwidth interframe video compression system (described above) due to the large size of the average size of an I frame relative to the frame subject to long latencies. 或者,换言之,当先前技术帧间压缩算法达成比仅帧内压缩算法低的平均每帧数据速率(例如,2Mbps对40Mbps)时,其由于大I帧而仍遭受高的峰值每帧数据速率(例如,303,935*60 = 18. 2Mbps)。 Or, in other words, while prior art interframe compression algorithms achieve a lower than average per-frame data rate algorithm intraframe-only compression (eg, 2Mbps vs. 40Mbps) when, because of the large I frames still suffer from a high peak per-frame data rate ( e.g., 303,935 * 60 = 18. 2Mbps). 但请记住:上述分析假定P 帧及B帧均比I帧小得多。 But remember: The above analysis assumes that the P and B frames are much smaller than the I frames. 尽管这大体成立,但对于具有与先前帧、高运动或场景改变不相关的高图像复杂度的帧,这不成立。 While this is generally true, but with the prior frame, high motion, or scene change frame is not associated with high image complexity, it does not hold. 在所述情形下,P帧或B帧可变得与I帧一般大(若P 帧或B帧变得比I帧大,则尖端压缩算法通常将“强制”I帧且用I帧替换P帧或B帧)。 In such cases, P or B frames can become as large as I frames (if a P-frame or B frame gets larger than an I frame, a sophisticated compression algorithm will typically be "forced" I frame with an I frame and replace the P or B-frame). 因此,I帧大小的数据速率峰值可在任何时刻出现于数字视频流中。 Thus, I frame-sized data rate peaks can occur at the digital video stream at any time. 因此,对于经压缩的视频, 当平均视频数据速率接近传输信道的数据速率容量时(经常为该状况,给定对于视频的高数据速率要求),来自I帧或大的P帧或B帧的高峰值数据速率导致高帧延时。 Thus, with compressed video, when the average video data rate approaches transmission channel data rate capacity (as is frequently the case, given the high data rate demands for video) from P or B frames I frames or large of peak data rates result in a high frame latency.

[0066] 当然,上述论述仅特征化由GOP中的大的B帧、P帧或I帧产生的压缩算法延时。 [0066] Of course, the above discussion only characterizes the GOP is large B-frame, P-frame or I-frame compression algorithm latency created. 若使用B帧,则延时将更高。 If B frames are used, the latency will be even higher. 原因是因为在可显示B帧之前,必须接收B帧之后的所有B帧及I帧。 Reason why is because before a B frame can be displayed, must receive all B frames after the B frame and the I frame. 因此,在诸如BBBBBIPPPPPBBBBBIPPPPP的图片群(GOP)序列中,其中在每个I帧之前存在5个B帧,只有在接收到随后的B帧及I帧之后才可由视频解压缩器显示第一B帧。 Thus, in the sequence such as BBBBBIPPPPPBBBBBIPPPPP group of pictures (GOP), where there are 5 B frames before each I frame, only the first B-frame before display by the video decompressor, after receiving a subsequent B frames and I frame . 因此,若使视频以60fps (也即,16. 67毫秒/帧)流动,则在可解压缩第一B帧之前,不管信道带宽如何快,接收五个B帧及I帧将花费16. 67*6 = 100毫秒,且这是仅5个B帧的情况。 Thus, before if video at 60fps (i.e., 16.67 ms / frame), then the first B frame can be decompressed, no matter how fast the channel bandwidth, five B frames and the received I frame will take 16.67 * 6 = 100 msec, and this is the case just 5 B frames. 具有30个B帧的经压缩的视频序列相当普遍。 Video sequences with 30 B frames are quite common compressed. 此外,在如2Mbps的低信道带宽下,由于I帧的大小而引起的延时影响很大程度上增加到由于等待B帧到达而产生的延时影响。 Further, in such a low channel bandwidth like 2Mbps, since the size of the I frame is largely caused by increased latency impact due to waiting delays impact generated B frames to arrive. 因此,在2Mbps信道上,在大量B帧的情况下,使用先前技术视频压缩技术超过500毫秒或500毫秒以上的延时相当容易。 Thus, on a 2Mbps channel, in a case where a large number of B frames more using prior art video technology than 500 milliseconds or 500 milliseconds is compressed relatively easily. 若不使用B帧(对于给定质量水准,以较低压缩比率为代价),则不招致B帧延时,但仍招致上文所描述的由于峰值帧大小而引起的延时。 If B-frames (for a given quality level, the cost of a lower compression ratio), the B frame latency is not incurred, but still incurred due to the delay caused by the peak frame sizes, described above.

[0067] 问题恰恰由于许多视频游戏的性质而加重。 [0067] issue precisely because of the nature of many video games and heavier. 利用上文所描述的GOP结构的视频压缩算法很大程度上被最佳化以用于连同要用于被动观看的现场直播的视频或电影材料一起使用。 Video using the GOP structure described above compression algorithms have been largely optimized for use with passive viewing to be used in conjunction with live video or motion picture material. 通常,相机(真实相机,或者计算机产生的动画的状况下的虚拟相机)及场景相对稳定,仅因为若相机或场景太颠簸地来回移动,则视频或电影材料(a)通常观看起来令人不愉快,且(b)若其正被观看,当相机突然来回颠簸时,观看者通常不能够紧密地跟随该动作(例如,若相机在拍摄吹灭生日蛋糕上的蜡烛的孩子时被扰动且突然在蛋糕之间来回颠簸,则观看者通常集中于孩子及蛋糕上,而不理会相机突然移动时的简短中断)。 In general, the (virtual camera in the situation animation real camera, or computer-generated) camera and the scene is relatively stable, only because if the camera or the scene is too bumpy move back and forth, the video or movie material (a) usually watch them unpleasant and (b) if it is being viewed, when the camera jerks around suddenly, the viewer is usually not closely following the action (for example, if the camera is disturbed when shooting blowing out the candles on a birthday cake and a child suddenly thrashing back and forth between the cake, the viewer usually focus on children and cake, without regard to a brief interruption when the camera suddenly moved). 在视频会谈或视频电话会议的状况下,可将相机固持于固定位置中且根本不移动,从而导致根本非常少的数据峰值。 In the case of video conference or video conference call, you can hold the camera in a fixed position and does not move at all, resulting in very little data peak. 但3D高动作视频游戏通过恒定运动来被特征化(例如,考虑3D竞赛,其中整个帧在竞赛的持续时间中处于快速运动中,或者考虑第一人称射击游戏,其中虚拟相机恒定地颠簸地来回移动)。 But 3D high action video games by constant motion to be characterized (e.g., consider a 3D racing, where the entire frame is in rapid motion for the duration of the race, or consider first-person shooters, where the virtual camera is constantly moving around jerkily ). 所述视频游戏可产生具有大的及频繁的峰值的帧序列,其中用户可能需要清楚地看见在该突然运动期间发生了什么。 The video games can result in frame sequences with large and frequent peaks where the user may need to clearly see what is happening during those sudden motions. 因此,在3D高动作视频游戏中,压缩假影远不可容忍。 Therefore, high action video game in 3D, the compression artifacts are far less tolerable. 因此,许多视频游戏的视频输出(由于其性质)产生具有非常高且频繁的峰值的经压缩的视频流。 Thus, the video output of many video games (by its nature) generates a compressed video stream with very high and frequent peaks.

[0068] 假定快动作视频游戏的用户对于高延时具有小的容忍度,且给定所有上述延时原因,至今存在对于使视频在互联网上流动的服务器主机代管的视频游戏的限制。 [0068] Assume that the user fast action video games for high latency have little tolerance, and given all of the above causes of latency, to date there is a limit to stream video on the Internet to server-hosted video games. 另外,若需要高度互动性的应用程序被主机代管于通用互联网上且使视频流动,则所述应用程序的用户遭受类似限制。 Further, if desired, highly interactive applications are hosted on the general Internet and stream video, then the user application program suffer from similar limitations. 所述服务需要网络配置,其中主机代管服务器直接设置于前端(在电缆宽带的状况下)或中央办公室(在数字用户线(DSL)的状况下)中,或商业背景中的LAN内(或特别分级的私用连接上),以便控制自客户端设备至服务器的路线及距离以最小化延时且可适应峰值而不造成延时。 Such services require a network configuration in which the hosting servers are directly disposed at the front end (in the case of cable broadband) or the central office (of Digital Subscriber Line (DSL) in), or in a commercial setting, in the LAN (or on a specially-graded private connection) in order to control from the client device to the server route and distance to minimize latency and peaks can be accommodated without incurring latency. LAN(通常额定在lOOMbps-lGbps)及具有足够带宽的租用线路通常可支持峰值带宽要求(例如,18Mbps峰值带宽为100Mbps LAN容量的一小部分)。 LAN (typically rated at lOOMbps-lGbps) typically can support peak bandwidth requirements and leased lines with adequate bandwidth (e.g., 18Mbps peak bandwidth is a small fraction of 100Mbps LAN capacity).

[0069] 若进行特殊适应,则峰值带宽要求也可由住宅宽带基础架构来适应。 [0069] If the special adaptation, the peak bandwidth requirements but also by residential broadband infrastructure to adapt. 举例而言,在电缆TV系统上,可为数字视频通信给出专用带宽,该专用带宽可处理诸如大I帧的峰值。 For example, on a cable TV system, it can be given dedicated bandwidth digital video traffic, dedicated bandwidth which can handle peaks, such as large I frames. 此外,在DSL系统上,可供应较高速度的DSL调制解调器(考虑高峰值),或可供应可处理较高数据速率的特别分级的连接。 Further, on a DSL system, a higher speed DSL modem can be supplied (considering the peak value), or a specially-graded provisioned which can handle a higher data rate connection. 但是,附接至通用互联网的传统电缆调制解调器及DSL基础架构对于用于压缩的视频的峰值带宽要求而言远不能容忍。 However, attached to the general Internet cable modem and DSL conventional infrastructure for peak bandwidth requirements for compressing video far less tolerance. 因此,在线服务(将视频游戏或应用程序主机代管于距客户端设备长距离的服务器中心中,且接着经由传统的住宅宽带连接经由互联网而使经压缩的视频输出流动)遭受显著的延时及峰值带宽要求-尤其对于需要非常低的延时的游戏及应用程序(例如,第一人称射击游戏及其他多用户、互动式动作游戏,或需要快响应时间的应用程序)。 Therefore, the online service (a video game or application hosting to a client device from the server center in long distance, and then connect the video output via the Internet, the flow of compressed via a conventional residential broadband) suffer significant delay and peak bandwidth limitations - in particular, the need for very low latency games and applications (e.g., first person shooters and other multi-user, interactive action games, or applications requiring a fast response time). 附图说明 BRIEF DESCRIPTION

[0070] 根据下面的详细描述并根据附图可以更完整地理解本公开,然而,所述附图并不用来将公开的主题限制到所示特定实施方式中,而只是用作解释和理解。 [0070] and the accompanying drawings will be more fully understood from the present disclosure the following detailed description, however, not be taken to limit the disclosed subject matter to the specific embodiments shown, but are for explanation and understanding only.

[0071] 图1示出了先前技术视频游戏系统的架构。 [0071] FIG 1 illustrates an architecture of a prior art video gaming system.

[0072] 图2a至图2b示出了根据一个实施例的高级系统架构。 [0072] Figures 2a-2b illustrate a high level system architecture according to an embodiment.

[0073] 图3示出了用于客户端与服务器之间的通信的实际的、额定的及所需的数据速率。 [0073] FIG. 3 illustrates actual, rated, and required data rates for communication between the client and the server.

[0074] 图4a示出了根据一个实施例而使用的主机服务及客户端。 [0074] Figure 4a illustrates a hosting service and a client according to one embodiment used in the embodiment.

[0075] 图4b示出了与客户端与主机服务之间的通信相关的例示性延时。 [0075] FIG 4b illustrates exemplary latencies associated with communication between a client and a hosting service.

[0076] 图4c示出了根据一个实施例的客户端设备。 [0076] FIG 4c illustrates a client device according to an embodiment.

[0077] 图4d示出了根据另一个实施例的客户端设备。 [0077] FIG 4d illustrates a client device according to another embodiment.

[0078] 图4e示出了图4c中的客户端设备的实例方块图。 [0078] FIG 4e illustrates an example block diagram of FIG. 4c client device.

[0079] 图4f示出了图4d中的客户端设备的实例方块图。 [0079] FIG 4f illustrates an example block diagram of FIG. 4d client device.

[0080] 图5示出了可根据一个实施例而使用的视频压缩的一个实例。 [0080] FIG. 5 illustrates an example of an embodiment according to the video compression used.

[0081] 图6a示出了可在另一个实施例中使用的视频压缩的一个实例。 [0081] Figure 6a illustrates an example embodiment may be used in another video compression embodiment.

[0082] 图6b示出了与传输低复杂度、低动作视频序列相关的数据速率中的峰值。 [0082] Figure 6b shows a low transmission peak complexity, low action video sequence in data rate associated.

[0083] 图6c示出了与传输高复杂度、高动作视频序列相关的数据速率中的峰值。 [0083] Figure 6c shows a high transmission peak complexity, high action video sequence in data rate associated.

[0084] 图7a至图7b示出了在一个实施例中使用的实例视频压缩技术。 [0084] Figures 7a-7b illustrate example video compression used in one embodiment of the technique.

[0085] 图8示出了在一个实施例中使用的额外实例视频压缩技术。 [0085] FIG. 8 illustrates additional example embodiment of a compression used in the video technology.

[0086] 图9a至图9c示出了在一个实施例中使用以用于缓解数据速率峰值的实例技术。 [0086] Figures 9a through 9c illustrate examples of techniques used in the Examples for relieve embodiment of a data rate peaks.

[0087] 图IOa至图IOb示出了将图像图像块有效地封装于分组内的一个实施例。 [0087] FIGS IOa to IOb illustrate an embodiment of the image block is effectively encapsulated within the packet.

[0088] 图Ila至图Ild示出了使用前向纠错技术的实施例。 [0088] FIGS Ild Ila to illustrate an embodiment of error correction techniques prior to use.

[0089] 图12示出了使用多核心处理单元来进行压缩的一个实施例。 [0089] FIG. 12 illustrates the use of multi-core processing units for compression according to one embodiment.

[0090] 图13a至图13b示出了根据各种实施例的主机服务之间的地理定位及通信。 [0090] Figures 13a to 13b illustrate geographical positioning and communication between hosting services according to various embodiments.

[0091] 图14示出了与客户端与主机服务之间的通信相关的示例性延时。 [0091] FIG. 14 illustrates exemplary latencies associated with communication between a client and a hosting service.

[0092] 图15示出了实例主机服务服务器中心架构。 [0092] FIG. 15 illustrates an example hosting service server center architecture.

[0093] 图16示出了包括多个现场直播的视频窗的用户接口的一个实施例的实例屏幕拍摄。 [0093] FIG. 16 shows an example of a screen embodiment includes a plurality of live video windows user interface shooting.

[0094] 图17示出了在选择特定视频窗之后的图16的用户接口。 [0094] FIG. 17 shows the user interface after selecting a particular video window 16 of FIG.

[0095] 图18示出了在将特定视频窗放大至全屏幕大小之后的图17的用户接口。 [0095] FIG. 18 illustrates a user interface of Figure 17 after amplifying the particular video window to full screen size.

[0096] 图19示出了叠加在多人游戏的屏幕上的实例合作用户视频数据。 [0096] FIG. 19 shows an example of superimposed on the screen of a multiplayer game collaborative user video data.

[0097] 图20示出了用于主机服务上的游戏玩家的实例用户页面。 [0097] FIG. 20 illustrates a game player on a hosting service for example user page.

[0098] 图21示出了实例3D互动式广告。 [0098] FIG. 21 illustrates an example 3D interactive advertisement.

[0099] 图22示出了用于自现场直播的表演的表面捕获产生具有纹理表面的照片般逼真的图像的实例步骤序列。 [0099] FIG. 22 shows the surface for performing capture of a live picture from a textured surface having a generating photorealistic images example sequence of steps.

[0100] 图23示出了允许选择线性媒体内容的实例用户接口页面。 [0100] FIG 23 shows a selection of linear media content allows example user interface page.

[0101] 图24为示出了在现场直播网页之前消逝的时间量与连接速度的曲线图。 [0101] FIG. 24 is a diagram illustrating the amount of time versus connection speed before the web page is live elapsed. 具体实施方式 Detailed ways

[0102] 在以下描述中列举了特定细节(诸如,设备类型、系统配置、通信方法等),以便提供对本公开的彻底理解。 [0102] Specific details are listed (such as device types, system configurations, communication methods, etc.) In the following description, in order to provide a thorough understanding of the present disclosure. 然而,一般本领域的技术人员应了解,实践所描述的所述实施例可能不需要这些特定细节。 Generally, however, those skilled in the art will appreciate that the practice of the described embodiments may be practiced without these specific details.

[0103] 图2a至图2b提供两个实施例的高级架构,其中视频游戏及软件应用程序由主机服务210主机代管且在订阅服务下由用户场所211 (注意,“用户场所”意思是用户所定位的无论何处的位置,若使用移动设备则包括室外)处的客户端设备205经由互联网206 (或其他公众网络或私用网络)来存取。 [0103] Figures 2a-2b provide a high-embodiments in which video games and software applications are hosted by a hosting service 210 and the host service subscribed by the user in the premises 211 (note that "customer premises" means that the user anywhere in the located position, if the mobile device 205 includes a client device outdoors) via the Internet at 206 (or other public or private network) to access. 客户端设备205可为具有至互联网的有线或无线连接、 具有内部或外部显示设备222的通用计算机(诸如,以Microsoft Windows或Linux为基础的PC或Apple公司的Macintosh计算机),或者其可为将视频及音频输出到监视器或电视机222的诸如机顶盒的专用客户端设备(具有至互联网的有线或无线连接),或者其可为推测起来具有至互联网的无线连接的行动设备。 The client device 205 may have to the Internet wired or wireless connection, with internal or external display a general purpose computer (such as in Microsoft Windows or Linux-based PC or Apple, Inc. Macintosh computer) device 222, or it may be video and audio to a monitor or TV set-top box 222, such as a dedicated client device (with to the Internet wired or wireless connection), or it may be presumably mobile wireless device having a connection to the Internet.

[0104] 所述设备中的任一者可具有其自身的用户输入设备(例如,键盘、按钮、触摸屏幕、追踪板(track pad)或惯性感测棒(inertial-sensing wand)、视频捕获相机和/或运动追踪相机等),或者其可使用通过有线或无线连接的外部输入设备221 (例如,键盘、鼠标、游戏控制器、惯性感测棒、视频捕获相机和/或运动追踪相机等)。 [0104] The apparatus may have any of their own user input devices (e.g., keyboard, buttons, touch screens, track pads (track pad), or inertia sensing rod (inertial-sensing wand), video capture cameras and / or motion tracking cameras, etc.), or they may use external input devices 221 connected through a wired or wireless (e.g., keyboard, mouse, game controllers, inertial sensing wand, video capture cameras and / or motion tracking cameras, etc.) . 如下文更详细描述, 主机服务210包括各种性能水平的服务器(包括具有高能力CPU/GPU处理能力的那些服务器)。 As described in greater detail below, the hosting service 210 includes servers of various levels of performance (including those with high-powered CPU / GPU processing capabilities of those). 在播放游戏或使用主机服务210上的应用程序期间,家庭或办公室客户端设备205 接收来自用户的键盘和/或控制器输入,且接着其将控制器输入经由互联网206传输至主机服务210,主机服务210作为响应来执行游戏程序代码并产生用于游戏或应用程序软件的视频输出(视频图像序列)的相继帧(例如,若用户按压将会指引屏幕上的人物向右移动的按钮,则游戏程序接着将产生显示人物向右移动的视频图像序列)。 During play the game or using an application on the hosting service 210, a home or office client device 205 receives keyboard and / or controller input from the user, and then it is input to the controller 210 transfer the hosting service 206 via the Internet, the host consecutive frames service 210 that executes the gaming program code in response and generates a video output (a sequence of video images) for the game or application software (e.g., if the user presses on the guidance screen character will move to the right button, then the game program would then create a sequence of video images showing the character moving to the right). 接着使用低延时视频压缩器压缩该视频图像序列,且主机服务210接着经由互联网206而传输低延时视频流。 Then using a low- latency video compressor compresses the sequence of video images, and the hosting service 210 then transmits the low- latency video stream through the Internet 206. 家庭或办公室客户端设备接着解码经压缩的视频流并将经解压缩的视频图像再现于监视器或TV上。 Home or office client device then decodes the compressed video stream and the decompressed video image reproduced on a monitor or TV. 因此,显著地减少客户端设备205的计算及图形硬件要求。 Thus, significantly reducing the client device 205 computing and graphical hardware requirements. 客户端205仅需要具有用于将键盘/控制器输入转发到互联网206且解码并解压缩从互联网206所接收的经压缩的视频流的处理能力,实际上现今任何个人计算机均能够在其CPU上以软件来进行这些(例如,以约2GHz执行的Intel公司双核CPU能够解压缩使用诸如H. 264及Windows 媒体VC9的压缩器编码的720p HDTV)。 The client 205 only needs to have means for forwarding the keyboard / controller input to the Internet 206 and decode and decompress the compressed video stream processing capabilities 206 received from the Internet, in fact, it can be any personal computer in which both the CPU these in software (e.g., Intel Corporation performed at approximately 2GHz is capable of decompressing dual-core CPU, such as the use of H. 264 and 720p HDTV compressed encoded Windows media VC9). 此外,在任何客户端设备的状况下,专用芯片也可以比通用CPU低得多的成本及比通用CPU少得多的电力消耗(诸如,现代PC所需的)来实时地执行用于所述标准的视频解压缩。 Further, in the condition of any client devices, dedicated chips can also be much lower than the cost of a general purpose CPU and a general purpose CPU is much less than the power consumption (such as required by modern PC) be performed in real time for the standard video decompression. 值得注意地,为了执行转递控制器输入及解压缩视频的功能,家庭客户端设备205不需要任何专门化的图形处理单元(GPU)、光学驱动器或硬盘驱动器(诸如,图1中所显示的先前技术视频游戏系统)。 Notably, to perform forwarding controller input and decompressing video compression, home client devices 205 do not require any specialized graphics processing units (the GPU), optical drive or hard drives (such as the one shown in FIG. prior art video game system).

[0105] 随着游戏及应用程序软件变得更复杂及更具照片般逼真感,其将需要较高性能的CPU、GPU、较多RAM,及较大且较快的磁盘驱动器,且可使主机服务210处的计算能力不断地升级,但终端用户将不需要使家庭或办公室客户端平台205升级,因为将通过给定视频解压缩算法而使家庭或办公室客户端平台205的处理要求对于显示分辨率及帧速率保持恒定。 [0105] As games and applications software become more complex and more photorealistic, it will require higher performance CPU, GPU, more RAM, and larger and faster disk drives, and can computing the host service 210 continually upgraded, but the end user will not need to make a home or office client platform 205 upgrades, because the compressed video decompression algorithm given the home or office client platform 205 relative to the display processing request resolution and frame rate remains constant. 因此,图2a至图2b中所说明的系统中不存在现今所见的硬件限制及相容性问题。 Thus, the hardware limitations and compatibility issues seen today do not exist in the system Figures 2a-2b in the illustrated.

[0106] 另外,因为游戏及应用程序软件仅在主机服务210中的服务器中执行,所以在用 [0106] In addition, because the game only server and application software in the host service 210 is executed, it is in use

20户的家庭或办公室(除非另外有条件,否则如本文中所使用的“办公室”将包括任何非住宅背景,包括(例如)教室)中决不存在游戏或应用程序软件的复本(光学媒体的形式,或者为下载的软件)。 20 home or office (unless otherwise qualified, as used herein, "office" would include any non-residential background, including (for example) the classroom) never exist in replica (optical media game or application software form, or as a downloadable software). 这显著减轻游戏或应用程序软件被非法复制(盗版)的可能性,以及减轻可由游戏或应用程序软件使用的有价值的数据库被盗版的可能性。 This significantly reduced the possibility of a game or application software is illegally copied (pirated), as well as to reduce the possibility of a game can be a valuable database or application software being pirated. 实际上,若需要专门化的服务器(例如,需要非常昂贵的、大的或有噪音的设备)来播放对于家庭或办公室使用不可行的游戏或应用程序软件,则即使获得游戏或应用程序软件的盗版复本,其也将不可在家庭或办公室中操作。 In fact, if you need specialized server (for example, need to be very expensive, big or noisy equipment) to play for home or office use infeasible game or application software, then even if a game or application software pirated copies, which will also be unavailable in the home or office.

[0107] 在一个实施例中,主机服务210向设计视频游戏的游戏或应用程序软件开发商(其大体指代软件开发公司、游戏或电影工作室,或游戏或应用程序软件出版商)220提供软件开发工具,以使得其可设计能够在主机服务210上执行的游戏。 [0107] In one embodiment, the hosting service 210 provides the game or application software developers to design video games (which refers generally to software development companies, film studios or game, or a game or application software publishers) 220 software development tools, so that it can design a game that can be performed on the host service 210. 所述工具允许开发商利用主机服务的特征(所述特征通常在独立PC或游戏控制台中将不可用)(例如,快速存取复杂几何形状的非常大的数据库(除非另外有条件,否则“几何形状”将在本文中用于指代限定3D数据集的多边形、纹理、索具、照明、行为及其他组件及参数))。 The tool allows developers to use the hosting service features (a feature not normally available in a standalone PC or game console) (e.g., fast access to very large complex geometry database (unless otherwise qualified, "Geometry shape "will be used herein to refer to a 3D data set defining polygons, textures, rigging, lighting, behaviors and other components and parameters)).

[0108] 在该架构下,不同商业模型是可能的。 [0108] In this framework, different business models are possible. 在一个模型下,主机服务210从终端用户收取订阅费用且向开发商220支付版税,如图2a中所显示。 In a model, the hosting service 210 charged for the subscription from end users to pay royalties to the developers and 220, shown in Figure 2a. 在替代实施中(图2b中所显示),开发商220直接从用户收取订阅费用且向主机服务210支付用于主机代管游戏或应用程序内容的费用。 In an alternative embodiment (shown in FIG. 2b), the developers 220 collects a subscription fee directly from the user and paid to the hosting service 210 for hosting the game or application content. 这些基本原理不限于用于提供在线游戏或应用程序主机代管的任何特定商业模型。 These basic principles are not limited to any particular business model for online gaming or application hosting.

[0109] 经压缩的视频特性 [0109] Characteristics of compressed video

[0110] 如先前所论述,在线提供视频游戏服务或应用程序软件服务的一个显著问题在于延时。 [0110] As previously discussed, a significant problem video game service or application software services online that delay. 70毫秒-80毫秒的延时(自输入设备被用户致动(actuate)的时刻到在显示设备上显示响应时的时刻)为用于需要快响应时间的游戏及应用程序的上限。 70-80ms latency (the time from the input device is actuated by the user (Actuate) to the display time when the response on a display device) as an upper limit for games and requiring a fast response time of the application. 然而,由于大量实际及物理约束而使得这在图2a及图2b中所显示的架构的情况下非常难以达成。 However, since a large number of practical and physical constraints such that under such circumstances the architecture shown in Figures 2a and 2b are very difficult to achieve.

[0111] 如图3中所指示,当用户订阅互联网服务时,连接通常额定为到用户的家庭或办公室的标定最大数据速率301。 [0111] indicated in Figure 3, when a user subscribes to an Internet service, the connection is typically rated by a nominal home or office user maximum data rate 301. 取决于提供者的策略及路由设备能力,该最大数据速率可或多或少被严格地强制执行,但通常由于许多不同原因中的一者而使得实际可用数据速率较低。 Depending on the provider's policies and routing equipment capabilities, that maximum data rate may be more or less strictly enforced, but typically the number of different reasons in one of the actual available data rate is lower. 举例而言,可能在DSL中央办公室处或在本地电缆调制解调器回路上存在过多网络通信通信,或可能在电缆线上存在噪音,从而引起丢弃的分组,或提供者可能建立每用户每月最大数目的比特。 For example, there may be too much network traffic at the DSL central office or on the local cable modem loop, or there may be noise on the cabling causing dropped packets, or the provider may establish a maximum number of per user per month bit. 当前,用于电缆及DSL服务的最大下行数据速率通常在数百千比特/秒(Kbps)到30Mbps的范围内。 Currently, the maximum downstream data rate for cable and DSL services typically ranges from several hundred Kilobits / second (Kbps) in the range of 30Mbps. 蜂窝式服务通常限于数百Kbps的下行数据。 Cellular services are typically limited to hundreds of Kbps of downstream data. 然而,宽带服务的速度及订阅宽带服务的用户的数目将随着时间而急剧增加。 However, the number of users speed broadband services and subscribe to broadband services will increase dramatically over time. 当前,一些分析者估计33% 的美国宽带用户具有2Mbps或2Mbps以上的下行数据速率。 Currently, some analysts estimate that 33% of US broadband subscribers have a downstream data rate of 2Mbps or more. 举例而言,一些分析者预测:至2010年止,超过85%的美国宽带用户将具有2Mbps或2Mbps以上的数据速率。 For example, some analysts predict that: by 2010, over 85% of US broadband subscribers will have a data rate of 2Mbps or more.

[0112] 如图3中所指示,实际可用最大数据速率302可随着时间而波动。 [0112] indicated in Figure 3, the actual available max data rate 302 may fluctuate over time. 因此,在低延时、在线游戏或应用程序软件情况下,有时难以预测用于特定视频流的实际可用数据速率。 Thus, at the low-latency, online gaming or application software context it is sometimes difficult to predict the actual available data rate for a particular video stream. 若对于特定量的场景复杂度及运动在给定数目的每秒帧数(fps)下以给定分辨率(例如, 640X480i60fps)维持给定质量水平所需的数据速率303升高高于实际可用最大数据速率302 (如通过图3中的峰值指示),则可出现若干问题。 If the complexity and quantity of motion for a particular scene in a given number of frames per second (fps) at a given resolution (e.g., 640X480i60fps) maintain a given data rate required quality level 303 rises above the actual available max data rate 302 (as indicated by the peak in Figure 3), then several problems may occur. 举例而言,一些互联网服务将仅丢弃分组,从而导致用户的视频屏幕上的丢失的数据及失真的/丢失的图像。 For example, some Internet service will simply drop packets, resulting in the loss of the user's data and video screen distortion / loss of image. 其他服务将暂时 Other services will be temporarily

21缓冲(也即,排队)额外分组且以可用数据速率将所述分组提供到客户端,从而导致延时的增加-对于许多视频游戏及应用程序而言为不可接受的结果。 21 buffer (i.e., queue up) the additional packets and the available data rate to provide the packets to the client, thereby resulting in an increase in latency - for many video games and applications for the unacceptable results. 最后,一些互联网服务提供者将数据速率的增加视为恶意攻击(诸如,否认服务攻击(由计算机黑客用以使网络连接停用的公知技术)),且将在特定时间周期中切断用户的互联网连接。 Finally, some Internet service providers will increase the data rate as a malicious attack (such as denial of service attacks (by the computer hackers to disable network connections well known technique)), and will cut off Internet users in a specific time period connection. 因此,本文中所描述的实施例设法确保用于视频游戏的所需数据速率不会超过最大可用数据速率。 Thus, embodiments herein described embodiments seek to ensure that the required data rate for a video game does not exceed the maximum available data rate.

[0113] 主机服务架构 [0113] Host Services Architecture

[0114] 图4a说明根据一个实施例的主机服务210的架构。 [0114] Figure 4a illustrates the host service architecture 210 according to one embodiment. 主机服务210可位于单个服务器中心中,或者可跨越多个服务器中心而分散(以为具有比其他者低延时的至特定服务器中心的路径的用户提供低延时连接,以在用户之间提供负载平衡,且在一或多个服务器中心出故障的状况下提供冗余)。 The hosting service 210 may be located in a single server center, or across multiple server centers (than other users that have lower latency paths to certain server centers to provide for lower latency connections to provide load between users balance, and provides redundancy in the one or more server centers have failed condition). 主机服务210最终可包括成千上万个或甚至数百万个服务器402,从而服务非常大的用户基础(user base)。 The hosting service 210 may eventually include hundreds of thousands or even millions of servers 402, serving a very large user base (user base). 主机服务控制系统401提供对主机服务210的总体控制,且指引路由器、服务器、视频压缩系统、计费及帐务系统等。 Hosting service control system 401 provides overall control of the host service 210, and directs routers, servers, video compression systems, billing and accounting systems. 在一个实施例中,主机服务控制系统401实施在基于Linux的分散式处理系统上,该处理系统绑定到用于储存用于用户信息、服务器信息及系统统计数据的数据库的RAID阵列。 In one embodiment, the hosting service control system 401 is implemented on a distributed processing Linux-based system, it is bound to the processing system for storing user information, RAID arrays database server information, and system statistics. 在上述描述中, 除非归因于其他特定系统,否则由主机服务210实施的各种动作由主机服务控制系统401 来起始及控制。 In the above description, unless attributed to other specific systems, by the host service 210 or embodiment 401 of the various actions are initiated and controlled by the hosting service control system.

[0115] 主机服务210包括许多服务器402,诸如当前可从Intel (因特尔公司)、IBM(美国国际商用机器公司)及Hewlett Packard(惠普公司)及其他者得到的所述服务器。 [0115] The hosting service 210 includes a number of servers 402 such as those currently available from the server, the Intel (Intel Corporation), the IBM (International Business Machines Corporation) and Hewlett Packard (HP) and others were obtained. 或者,可将服务器402装配成定制组件配置,或者最终可将服务器402整合以便将整个服务器实施为单个芯片。 Alternatively, server 402 may be assembled into a custom configuration of components, or can eventually be integrated so the server 402 as a single chip implementation of the whole server. 尽管此图为说明起见而显示少数服务器402,但在实际部署中,可能存在少至一个服务器402或多达数百万个或数百万个以上服务器402的服务器。 Although this diagram illustration shows a small number server 402, but in the actual deployment, there may be as few as one or as many as 402 servers over millions of millions or 402 servers in the server. 服务器402均可以相同方式配置(作为一些配置参数的实例,具有相同CPU类型及性能;具有或不具有GPU,且若具有GPU,则具有相同GPU类型及性能;具有相同数目的CPU及GPU ;具有相同量及相同类型/速度的RAM ;及具有相同RAM配置),或服务器402的各种子集可具有相同配置(例如,25%的服务器可以一个特定方式配置,50%的服务器以不同方式配置,且25%的服务器以又一个方式配置),或每个服务器402可不同。 Server 402 can be the same arranged (as a number of examples of the configuration parameters, with the same CPU type and performance; CPU and GPU has the same number;; with or without a GPU, and if with a GPU, with the same GPU type and performance with RAM same amount of and type / speed; and with the same RAM configuration), or various subsets of the servers 402 may have the same configuration (e.g., 25% of the servers can be configured a certain way, 50% of the server configuration in different manners , and 25% yet another way of configuration server), or every server 402 may be different.

[0116] 在一个实施例中,服务器402无磁盘,也即,并非具有其自身的本地大容量储存器(其为光学或磁性储存器,或者基于半导体的储存器,诸如闪存或服务类似功能的其他大容量储存装置),每一个服务器经由快速底板或网络连接而存取共用的大容量储存器。 [0116] In one embodiment, the servers 402 are diskless, i.e., rather than having its own local mass storage (be it optical or magnetic storage, or semiconductor-based storage such as flash memory or similar service features other mass storage), connected to each server accesses shared mass storage through fast backplane or network. 在一个实施例中,该快速连接为连接到独立冗余磁盘阵列(RAID)405系列的储存区域网络(SAN)403,在使用超高速以太网实施的设备之间具有连接。 In one embodiment, this fast connection is connected to a separate redundant disk array (RAID) 405 series Storage Area Network (SAN) 403, with connections between devices implemented using Gigabit Ethernet. 如本领域的技术人员已知的, SAN 403可用于将许多RAID阵列405组合在一起,从而导致极高的带宽_接近或可能超过可自用于当前游戏控制台及PC中的RAM得到的带宽。 As is known to those skilled in the art, SAN 403 may be used to combine many RAID arrays 405 together, resulting in extremely high bandwidth _ approaching or potentially exceeding the bandwidth available from the current gaming consoles and the RAM PC obtained. 此外,尽管基于诸如磁性媒体的旋转媒体的RAID阵列经常具有显著的搜寻时间存取延时,但基于半导体储存器的RAID阵列可实施为具有低得多的存取延时。 In addition, although the RAID arrays based on rotating media, such as magnetic media, frequently have significant seek-time access latency, RAID arrays based on semiconductor storage can be implemented with much lower access latency. 在另一配置中,一些或所有服务器402在本地提供一些或所有其自身的大容量储存器。 In another configuration, some or all of the servers 402 provide some or all of their own mass storage locally. 举例而言,服务器402可将频繁存取的信息(诸如,其操作系统及视频游戏或应用程序的复本)储存在基于低延时本地闪存的储存器上,但其可利用SAN来存取基于旋转媒体的具有较高搜寻延时的RAID阵列405,以较不频繁地存取几何形状或游戏状态信息的大数据库。 For example, the server 402 may store frequently-accessed information (such as its operating system and a copy of a video game or application program) stored in the reservoir based on low-latency local Flash memory, but it may utilize the SAN to access with higher seek RAID arrays based on rotating media, delay 405, to access large databases of geometry more frequently or game state information. [0117] 另外,在一个实施例中,主机服务210使用下文详细描述的低延时视频压缩逻辑404。 [0117] Further, in one embodiment, the hosting service 210 described in detail below using the low-latency video compression logic 404. 视频压缩逻辑404可以软件、硬件或其任何组合来实施(下文描述其特定实施例)。 Video compression logic 404 may be software, hardware, or any combination thereof (certain embodiments of which are described below). 视频压缩逻辑404包括用于压缩音频以及视觉材料的逻辑。 Video compression logic 404 includes logic for compressing audio as well as visual material.

[0118] 在操作中,当经由键盘、鼠标、游戏控制器或其他输入设备421而玩视频游戏或使用用户场所211处的应用程序时,客户端415上的控制信号逻辑413将表示由用户促使的按钮按压(及其他类型的用户输入)的控制信号406a-b(通常为UDP分组的形式)传输到主机服务210。 [0118] In operation, when via a keyboard, mouse, game controller or other input device 421 while playing a video game or using an application at the user premises 211, the client control signal logic 413 on the client 415 by the user causes represented button presses (and other types of user inputs) control signals 406a-b (typically in the form of UDP packets) transmitted to the host service 210. 将来自给定用户的控制信号路由到适当服务器(或若多个服务器响应于用户的输入设备,则路由至多个服务器)402。 Routes the control signals from a given user to the appropriate server (or servers, if a plurality of devices in response to user input, to route the plurality of servers) 402. 如图4a中所说明,可经由SAN而将控制信号406a路由至服务器402。 Illustrated in Figure 4a, via the SAN control signals 406a may be routed to the servers 402. 可替换地或另外,可经由主机服务网络(例如,基于以太网的区域网络)而将控制信号406b直接路由至服务器402。 Alternatively or in addition, via the hosting service network (e.g., Ethernet-based local area network), control signals 406b routed directly to the server 402. 不管控制信号406a-b是如何被传输,该或所述服务器均响应于控制信号406a_b而执行游戏或应用程序软件。 Control signals 406a-b regardless of how they are transmitted, the server or servers in response to a control signal 406a_b execute the game or application software. 尽管图4a中未说明, 但各种网络连接组件(诸如,防火墙和/或网关)可处理主机服务210的边缘处(例如,主机服务210与互联网410之间)和/或用户场所211的边缘处(互联网410与家庭或办公室客户端415之间)的传入及传出的通信。 Although not illustrated in Figure 4a, various networking components (such as a firewall and / or gateway) may be processed at the edge of the hosting service 210 (e.g., between the hosting service 210 and the Internet 410) and / or the edge of the user premises 211 incoming and outgoing traffic at (410 Internet and home or office client 415) a. 所执行的游戏或应用程序软件的图形及音频输出(也即,新的视频图像序列)提供至低延时视频压缩逻辑404,低延时视频压缩逻辑404 根据低延时视频压缩技术(诸如,本文中所描述的所述技术)而压缩视频图像序列且经由互联网410(或,如下文所描述,经由绕过通用互联网的最佳高速网络服务)而将经压缩的视频流(通常具有经压缩或未经压缩的音频)传输回至客户端415。 Executed game or application software graphics and audio output (i.e., a new sequence of video images) to the low-latency video compression logic 404, low-latency video compression logic 404 in accordance with the low latency video compression techniques (such as, the techniques described herein) is compressed and the described sequence of video images via the Internet 410 (or, as described below, via the high-speed bypass best general Internet network service) and the compressed video stream (typically having a compressed or uncompressed audio) transmitted back to the client 415. 接着,客户端415上的低延时视频解压缩逻辑412解压缩视频及音频流并再现经解压缩的视频流,且通常在显示设备422上播放经解压缩的音频流。 Next, the low-latency video compression logic 412 the client 415 decompresses the video and audio streams and renders the decompressed video stream, and typically plays the decompressed display audio stream device 422. 或者,可在与显示设备422分开的扬声器上播放音频或根本不播放音频。 Alternatively, the audio can be played on a separate display device 422 or not playing audio speakers. 注意,尽管输入设备421及显示设备422在图2a及图2b中显示为独立式设备,但其可集成在诸如便携式计算机或行动设备的客户端设备内。 Note that, although the input device 421 and display device 422 are shown as in Figures 2a and 2b are stand-alone devices, but may be integrated within client devices such as portable computers or mobile devices.

[0119] 家庭或办公室客户端415 (先前在图2a及图2b中描述为家庭或办公室客户端205)可为非常低廉且低能力的设备,其具有非常有限的计算或图形性能且可能具有非常有限的本地大容量储存器或不具有本地大容量储存器。 [0119] Home or office client 415 (described previously in Figures 2a and 2b as a home or office client 205) may be a very inexpensive and low- power device, with very limited computing or graphics performance and may well have very limited local mass storage or no local mass storage. 相比之下,耦合至SAN 403及多个RAID 405的每一个服务器402可为格外高性能的计算系统,且实际上,若多个服务器以并列处理配置合作地使用,则几乎不存在对可承受的计算量及图形处理能力的限制。 In contrast, each server is coupled to a SAN 402 may be an exceptionally high performance computing system 403 and a plurality of RAID 405, and indeed, if multiple servers are used cooperatively in a parallel-processing configuration, there is almost no be limitation receiving computational processing power and graphics. 此外,由于低延时视频压缩404及低延时视频解压缩412(由用户感知地),所以将服务器402的计算能力提供给用户。 Further, due to the low latency video compression 404 and low-latency video compression 412 (perceptually to the user), the computing power of the server 402 will be provided to the user. 当用户按压输入设备421上的按钮时,显示器422上的图像被响应于按钮按压而更新(在感知上无有意义的延迟),好像游戏或应用程序软件在本地执行。 When the user presses a button on input device 421, the image on the display 422 is updated in response to the button press (perceptually with no meaningful delay), as if the game or application software were running locally. 因此,对于为非常低性能的计算机或只是实施低延时视频解压缩及控制信号逻辑413的低廉芯片的家庭或办公室客户端415,自看来在本地可用的远程位置有效地为用户提供任意计算能力。 Thus, the compression control signal logic 413 and low chip home or office client very low performance computer or just an embodiment of a low-latency video decompressor 415, since it seems effective to provide users with a local computing any available remote location ability. 此为用户给出用于玩最高级、处理器密集的(通常为新的)视频游戏及最高性能的应用程序的能力。 This gives users the power play the most advanced, processor-intensive (typically new) video game application's ability and highest performance.

[0120] 图4c显示非常基础且低廉的家庭或办公室客户端设备465。 [0120] Figure 4c shows a very basic and inexpensive home or office client device 465. 该设备为根据图4a 及图4b的家庭或办公室客户端415的一个实施例。 The apparatus according to one embodiment of home or office client Figures 4a and 4b of the end 415. 其大约2英寸长。 Which is about 2 inches long. 其具有与具有以太网供电(PoE)的以太网电缆相接口的以太网插孔462,该设备从以太网插孔462得到其电力及其到互联网的连接性。 Having an interface with an Ethernet cable with Power over Ethernet (PoE) the Ethernet jack 462, which derives its power from the Ethernet jack 462 and its connectivity to the Internet. 该设备能够在支持网络地址转换(NAT)的网络内执行NAT。 The apparatus is capable of performing within the network NAT Network Address Translation (NAT) is. 在办公室环境中,许多新的以太网交换器具有PoE且将PoE直接带到办公室中的以太网插孔。 In an office environment, many new Ethernet switch with PoE and PoE directly to the office Ethernet jack. 在此种情形下,所需的为从壁式插孔到客户端465的以太网电缆。 In this case, as required from the wall jack to the client 465. Ethernet cable. 若可用的以太网连接不运输电力(例如,在具有DSL或电缆调制解调器但无PoE的家庭中),则存在可用的低廉的壁式“砖块(brick),,(也即,电源),其将接受无电力的以太网电缆且输出具有PoE的以太网。 If the available Ethernet connection does not carry power (e.g., having a DSL or cable modem, but no PoE), then there are inexpensive wall "type bricks (brick) ,, (i.e., power supplies) will accept an unpowered Ethernet cable and output Ethernet with PoE.

[0121] 客户端465含有耦合至蓝牙无线接口的控制信号逻辑413 (图4a),该蓝牙无线接口与诸如键盘、鼠标、游戏控制器和/或麦克风和/或耳机的蓝牙输入设备479相接口。 [0121] The client 465 contains control signal logic is coupled to a Bluetooth wireless interface 413 (FIG. 4A), the Bluetooth wireless interface such as a keyboard, mouse, game controller and / or microphone and / or headset Bluetooth input devices 479, interfaces . 而且,客户端465的一个实施例在与显示设备468耦合的情况下能够以120fps输出视频,显示设备468能够支持120fps视频且向一对遮光眼镜466发信号(通常经由红外)以对于每个相继帧交替地遮蔽一只眼接着遮蔽另一只眼。 Further, a client 465 coupled in the case of the embodiment of the display device 468 capable of outputting video at 120fps embodiment, the display device 468 able to support 120fps video and the pair of the shutter eyeglasses signal 466 (typically via IR) to for each successive an eye shield frame alternately then the other eye shield. 用户所感觉的效果为“跳出”显示屏幕的立体3D图像。 The effect perceived by the user is "out" 3D stereoscopic image display screen. 支持该操作的一种该显示设备468为Samsung HL-T5076S。 For supporting the operation of the display device 468 Samsung HL-T5076S. 因为用于每一只眼的视频流是单独的,所以在两个独立视频流由主机服务210压缩的一个实施例中,帧在时间上交错,且帧在客户端465内以两个独立解压缩过程来解压缩。 Because for each eye is separate video stream, so in two independent video streams compressed by the hosting service 210 according to one embodiment, the frames are interleaved in time, and the two separate solutions to the frame 465 at the client compression process to decompress.

[0122] 客户端465也含有低延时视频解压缩逻辑412,其解压缩传入的视频及音频且经由HDMI (高清晰度多媒体接口)、连接器463输出,HDMI (高清晰度多媒体接口)、连接器463 插入到SDTV (标准清晰度电视)或HDTV (高清晰度电视)468中,从而向TV提供视频及音频,或插入到支持HDMI的监视器468中。 [0122] The client 465 also contains low latency video decompression logic 412, which decompresses the incoming video and audio and via an HDMI (High Definition Multimedia Interface), an output connector 463, an HDMI (High Definition Multimedia Interface) , connector 463 which plugs into an SDTV (standard definition television) or HDTV (high definition television) 468, providing the TV video and audio, or into a monitor 468 that supports HDMI. 若用户的监视器468不支持HDMI,则可使用HDMI 至DVI (数字视觉接口),但音频将丢失。 If the user's monitor 468 does not support HDMI, you can use the HDMI to DVI (Digital Visual Interface), but the audio will be lost. 在HDMI标准下,显示能力(例如,所支持的分辨率,帧速率)464从显示设备468表达,且接着经由互联网连接462将该信息传回到主机服务210,因此主机服务210可使经压缩的视频以适合于该显示设备的格式流动。 In the HDMI standard, the display capabilities (e.g. supported resolutions, frame rates) 464 expression from the display apparatus 468, and then this information via an Internet connection 462 back to the hosting service 210, the hosting service 210 so it can stream compressed the video format suitable for the flow of the display device.

[0123] 图4d显示家庭或办公室客户端设备475,除了客户端设备475具有更多外部接口之外,其与图4c中所显示的家庭或办公室客户端设备465相同。 [0123] Figure 4d shows a home or office client device 475, client device 475 in addition to having more than an external interface, which is the same home or office client device 4c shown in FIG. 465. 而且,客户端475可接受PoE来供电,或者其可占用插入墙壁中的外部电源适配器(未图示)。 Also, client 475 can accept either PoE for power, or it can run off of an external power adapter (not shown) in the wall. 视频相机477使用客户端475USB输入将经压缩的视频提供到客户端475,经压缩的视频由客户端475上传到主机服务210以用于下文所描述的用途。 A video camera 477 using the client 475USB inputs be provided to the client 475 the compressed video, compressed video uploaded by the client 475 to hosting service 210 for use described below. 将利用下文所描述的压缩技术将低延时压缩器创建到相机477中。 The use of the compression techniques described below will create a camera 477 is a low-latency compressor.

[0124] 除具有用于其互联网连接的以太网连接器之外,客户端475还具有到互联网的802. Ilg无线接口。 [0124] In addition to having an Ethernet connector for its Internet connection, client 475 also has a wireless interface 802. Ilg Internet. 两种接口均能够在支持NAT的网络内使用NAT。 Both interfaces are capable of using NAT in the network of support NAT.

[0125] 而且,除具有用于输出视频及音频的HDMI连接器之外,客户端475还具有双链接DVI-I连接器,双链接DVI-I连接器包括模拟输出端(且具有将提供VGA输出的标准适配器电缆)。 [0125] Further, in addition to having an HDMI connector to output video and audio, client 475 also has a Dual Link DVI-I connector, double-link DVI-I connector, which includes analog output (and with will provide VGA a standard adapter cable output). 其还具有用于复合视频及S视频的模拟输出端。 It also has analog outputs for composite video and S-video.

[0126] 对于音频,客户端475具有左/右模拟立体RCA插孔,且对于数字音频输出,其具有T0SLINK (光纤)输出端。 [0126] having a left / right analog stereo RCA jacks for audio, the client 475, and for digital audio output it has T0SLINK (optical) output.

[0127] 除了到输入设备479的蓝牙无线接口之外,其还具有用于接口到输入设备的USB 插孔。 [0127] In addition to a Bluetooth wireless interface other than the input device 479, which also has USB jacks to interface to input devices.

[0128] 图4e显示客户端465的内部架构的一个实施例。 [0128] Figure 4e shows internal architecture of client 465 according to one embodiment. 该图中所显示的所有设备或一些设备可实施在场可程序化逻辑阵列、定制ASIC中或若干个离散设备(定制设计或者现成的)中。 Either all or some of the devices shown in the Figures may be implemented in an Field Programmable Logic Array, an custom ASIC or in several discrete devices (either custom designed or off the shelf) in.

[0129] 具有PoE的以太网497附接到以太网接口481。 [0129] Ethernet with PoE 497 is attached to the Ethernet interface 481. 电力499才具有PoE的以太网497 得到且连接到客户端465中的其余设备。 Power 499 Ethernet with PoE 497 and is connected to the rest of the client device 465. 总线480为用于设备之间的通信的公同总线。 Bus 480 is a common bus for communication between devices. [0130] 执行来自闪存476的小客户端控制应用程序的控制CPU 483 (几乎任何小CPU都是适当的,诸如具有嵌入式RAM的IOOMHz下的MIPS R4000系列CPU)实施用于网络(也即, 以太网接口)的协议栈且还与主机服务210通信,并配置客户端465中的所有设备。 Small Client [0130] Flash memory 476 is performed from the application control CPU 483 (almost any small CPU are suitable, such as a MIPS R4000 series CPU at IOOMHz embedded RAM) is implemented for the network (i.e., Ethernet interface) protocol stack and also communicate with the host service 210, and configure all the devices in the client 465. 其还处理与输入设备469的接口并将分组(必要时,连同受前向纠错保护的用户控制器数据一起)发送回至主机服务210。 It also handles interfaces with the input devices 469 and the packet (if necessary, along with the user data to the error correction controller is protected by the front) sent back to the hosting service 210. 而且,控制CPU 483监视分组通信(例如,分组是丢失还是延迟,以及其到达的时间戳)。 Further, the control CPU 483 monitors the packet communication (e.g. if packets are lost or delayed and also timestamps their arrival). 将此信息发送回至主机服务210,以使得其可恒定地监视网络连接且相应地调整其发送的内容。 Transmits this information back to the hosting service 210 so that it can constantly monitor the network connection and adjust what it sends accordingly content. 最初在制造时为闪存476载入用于控制CPU 483的控制程序以及对于特定客户端465单元而言唯一的序号。 Initially loaded at the time of manufacturing a flash memory 476 a control program for controlling the CPU 483 and for a particular customer a unique serial number 465 unit end. 此序号允许主机服务210唯一地识别客户端465单元。 This serial number allows the hosting service 210 to uniquely identify the Client 465 unit.

[0131] 蓝牙接口484经由其天线(在客户端465内部)无线地通信至输入设备469。 [0131] Bluetooth interface 484 communicates wirelessly through its antenna to an input device (internal client 465) 469.

[0132] 视频解压缩器486为经配置以实施本文中所描述的视频解压缩的低延时视频解压缩器。 [0132] Video decompressor 486 is a low-latency video decompressor configured to implement the video decompression described herein, a compressor for compressing. 大量视频解压缩设备存在,或者为现成产品,或者作为具有可整合在FPGA或定制ASIC中的设计的知识产权(IP)。 A large number of video decompression devices exist, either as a ready-made product, or as intellectual property (IP) that can be integrated in FPGA or custom ASIC design. 一个提供用于H. 264解码器的IP的公司为澳大利亚新南威尔士州(NSW Australia)的Ocean LogicofManly0使用IP的优点在于:本文中所使用的压缩技术与压缩标准不相符。 IP provides an advantage for H. 264 decoders companies using IP Australia New South Wales (NSW Australia) is Ocean LogicofManly0 comprising: compression techniques used herein do not match the standard compression. 一些标准解压缩器足够灵活以经配置以适应本文中的压缩技术,但一些标准解压缩器可能并非如此。 Some standard decompression is flexible enough to be configured to accommodate compression techniques in this article, but some standard decompression may not. 但是,在IP的情况下,在视需要而重新设计解压缩器中存在完全灵活性。 However, in the case of IP, as required in the re-design solution there is complete flexibility in the compressor.

[0133] 视频解压缩器的输出端耦合至视频输出子系统487,视频输出子系统487将视频耦合至HDMI接口490的视频输出端。 An output terminal [0133] Video decompressor is coupled to the video output subsystem 487, the video output subsystem 487 couples the video to the video output terminal 490 of the HDMI interface.

[0134] 音频解压缩子系统488或者使用可用的标准音频解压缩器来实施,或者其可实施为IP,或者可在可(例如)实施VorbiS音频解压缩器的控制处理器483内实施音频解压缩。 [0134] The audio decompression subsystem may be (e.g.) 488 or embodiment using available standard audio decompressor implemented, or it may be implemented as IP, or the audio Solutions embodiment Vorbis audio decompressor 483 to the control processor compression.

[0135] 实施音频解压缩的设备耦合到音频输出子系统489,音频输出子系统489将音频耦合至HDMI接口490的音频输出端。 [0135] The audio decompression device is coupled to the audio output subsystem audio output terminal 489, an audio output subsystem 489 that couples the audio to the HDMI interface 490.

[0136] 图4f显示客户端475的内部架构的一个实施例。 [0136] Figure 4f shows internal architecture of client 475 according to one embodiment. 如可见,除额外接口及来自插入墙壁中的电源适配器的可选外部DC电力(且若如此使用,则可选外部DC电力替换将来自以太网PoE 497的电力)之外,该架构与客户端465的架构相同。 As can be seen, in addition to additional interfaces and optional external power adapter from the DC power into the wall (and, if so used, the optional external DC power from the power replace the Ethernet PoE 497), the architecture of the client 465 of the same architecture. 下文中将不重复与客户端465共同的功能性,但将额外功能性描述如下。 465 will not be repeated hereinafter common functionality and client, but the additional functionality is described as follows.

[0137] CPU 483与额外设备通信且配置额外设备。 [0137] CPU 483 communicating with the additional equipment and additional equipment configuration.

[0138] WiFi子系统482经由其天线提供无线互联网存取,作为对以太网497的替代。 [0138] WiFi subsystem 482 provides wireless Internet access via its antenna, as an alternative to Ethernet 497. WiFi 子系统可购自多家制造商,包括美国加州的圣克拉拉的AtherosCommunications (阿特赫鲁斯通信公司)。 WiFi subsystems are available from several manufacturers, including AtherosCommunications Santa Clara, California (阿特赫鲁斯 Communications).

[0139] USB子系统485提供对用于有线USB输入设备479的蓝牙通信的替代。 [0139] USB subsystem 485 provides an alternative to Bluetooth communication for wired USB input devices 479. FIG. USB子系统相当标准且可容易地用于FPGA及ASIC,且经常创建到执行如视频解压缩的其他功能的现成设备中。 USB subsystems are quite standard and readily available for FPGA and ASIC, frequently built into the-shelf devices performing other functions, like video decompression in such.

[0140] 与客户端465内的视频输出相比较,视频输出子系统487产生较宽范围的视频输出。 [0140] Video output in comparison with the client 465, the video output subsystem 487 produces a wider range of video outputs. 除提供HDMI 490视频输出之外,其提供DVI-I 491、S-视频492及合成视频493。 In addition to providing HDMI 490 video output, but that provides DVI-I 491, S- 493 the video 492, and composite video. 而且,当DVI-I 491接口用于数字视频时,将显示能力464自显示设备传回至控制CPU 483,以使得其可向主机服务210通知显示设备478的能力。 Also, when the DVI-I 491 interface is used for digital video, display capabilities 464 are passed back from the display device to the control CPU 483, so that it can service the host device 478 capabilities notification display 210. 由视频输出子系统487提供的所有接口均为相当标准的接口且容易以许多形式可用。 All interfaces provided by the video output subsystem 487 are quite standard interfaces and readily available in many forms.

[0141] 音频输出子系统489经由数字接口494(S/PDIF和/或Toslink)数字地输出音频且经由立体模拟接口495输出模拟形式的音频。 [0141] Audio output subsystem 489 via digital interface 494 (S / PDIF and / or Toslink) and audio outputs audio digitally through stereo analog interface 495 in analog form.

[0142] 来回行程延时分析 [0142] TRIP LATENCY ANALYSIS

[0143] 当然,为了实现前一段的利益,用户使用输入设备421的动作与在显示设备420上看见该动作的后果之间的来回行程延时应不大于70毫秒-80毫秒。 [0143] Of course, in order to achieve the benefits of the preceding paragraph, using the user input device 421 and seeing the operation of the round trip between the consequence of that action on display device 420 should be no greater than the delay of 70-80ms. 此延时必须考虑在自用户场所211中的输入设备421到主机服务210及再次返回到用户场所211至显示设备422 的路径中的所有因素。 This delay must be taken into account in the user premises 211 from input device 421 to the hosting service 210 and back again to the user premises all the factors in the path of the device 422 to the display 211. 图4b说明各种组件及网络(信号必须经由该组件或网络行进),且该组件及网络上方的为时间线,该时间线列出实际实施中可预期的例示性延时。 Figure 4b illustrates the various components and networks (signal must travel through the assembly or network), and the components and networks is a timeline above, the time-line lists a practical implementation contemplated exemplary latencies. 注意,图4b 经简化以便仅显示重要路径路由。 Note that Figure 4b is simplified so that only the critical path routing. 下文描述用于系统的其他特征的数据的其他路由。 Other routes are described below for other features of the system data. 双头箭头(例如,箭头453)指示来回行程延时且单头箭头(例如,箭头457)指示单向延时,且“〜”表示近似量测。 Double-headed arrows (e.g., arrow 453) indicate round-trip latency and a single-headed arrow (e.g., arrow 457) indicate one-way latency, and "-" denote an approximate measure. 应指出,将存在所列的延时不可达成的真实世界情形,但在大量状况下,在美国,使用至用户场所211的DSL及电缆调制解调器连接,该延时可在下一段中所描述的情形中达成。 It should be noted, there will be real-world situations where the latencies listed can not be achieved, but in a large number of cases in the US, to the user premises using DSL and cable modem connections 211, which may be delayed in some circumstances described in the next reached. 而且,注意,尽管至互联网的蜂窝式无线连接性的确将在所显示的系统中工作,但大多数当前美国蜂窝式数据系统(诸如,EVD0)招致非常高的延时且将不能够达成图4b中所显示的延时。 Also, note that although the cellular wireless connectivity to the Internet will certainly work in the system shown, most current US cellular data systems (such as, EVD0) incur very high latencies and would not be able to reach FIG. 4b delay as shown. 然而,这些基本原理可在可能能够实施该水平的延时的未来蜂窝式技术上实施。 However, these underlying principles may be implemented in a possible embodiment of the horizontal delay future cellular technologies.

[0144] 自用户场所211处的输入设备421开始,一旦用户致动输入设备421,就将用户控制信号发送至客户端415 (其可为诸如机顶盒的独立设备,或其可为在诸如PC或行动设备的另一设备中执行的软件或硬件),且将其分组化(在一个实施例中以UDP格式)并为分组给出目的地地址以到达主机服务210。 [0144] from the input device 421 at user premises 211 begins, once the user actuates the input device 421, a user control signal is sent to client 415 (which may be a standalone device such as a set-top box, or it may be such as a PC or software or hardware device to another mobile device in execution), and is packetized (in one embodiment, UDP format), and the packet is given a destination address to reach hosting service 210. 分组将也含有用于指示控制信号来自哪个用户的信息。 Packet will also contain information to indicate which user the control signals from. 接着经由防火墙/路由器/NAT(网络地址转换)设备443将控制信号分组转发到WAN 接口442。 Then forwarded through Firewall / Router / the NAT (Network Address Translation) device 443 to WAN interface signal packet 442. WAN接口442为由用户的ISP (互联网服务提供者)提供到用户场所211的接口设备。 ISP (Internet Service Provider) WAN interface 442 by the user is provided to the user premises 211 interface device. WAN接口442可以是电缆或DSL数据机、WiMax收发器、光纤收发器、蜂窝式数据接□、电网十办i^din (InternetProtocol-over-powerline interface), ^ilJSKN 的许多接口中的任何其他接口。 The WAN interface 442 may be a Cable or DSL modem, WiMax transceiver, a Fiber transceiver, a Cellular data interface □, many interfaces grid ten do i ^ din (InternetProtocol-over-powerline interface), ^ ilJSKN in any other interfaces . 另外,可将防火墙/路由器/NAT设备443 (及(可能地) WAN接口442)整合到客户端415中。 Further, Firewall / Router / NAT device 443 (and (possibly) the WAN interface 442) may be integrated into the client 415. 该一个实例将为移动电话,其包括用于实施家庭或办公室客户端415的功能性的软件,以及用于经由某标准(例如,802. Ilg)而无线地路由及连接到互联网的装置。 One example of this would be a mobile phone, which includes a functional embodiment of home or office client end software 415, and means for through some standard (e.g., 802. Ilg) to route and connect wirelessly to the Internet.

[0145] WAN接口442接着将控制信号路由至本文中所称的用于用户的互联网服务提供者(ISP)的“存在点”441,WAN接口442为提供连接至用户场所211的WAN输送器与通用互联网或私用网络之间的接口的设施。 [0145] WAN Interface 442 then routes the control signals to the internet service provider for the user referred to herein (ISP) "point of presence" 441, WAN 442 to provide an interface connected to the user premises 211 and the WAN transport universal Internet or a private network interface facilities between. 存在点的特性将视所提供的互联网服务的性质而改变。 Characteristic point of presence will depend on the nature of the Internet services provided by the change. 对于DSL,其通常是DSLAM位于的电话公司中央办公室。 For DSL, which is typically a telephone company central office DSLAM is located. 对于电缆调制解调器,其通常是电缆多系统运营商(MSO)前端。 For cable modems, which is typically a cable multiple system operator (MSO) front end. 对于蜂窝式系统,其通常是与蜂窝式塔相关的控制室。 For cellular systems, which are generally associated with cellular tower control room. 但无论存在点的性质怎样,其均将接着将控制信号分组路由至通用互联网410。 But no matter what the nature of the point of presence, it will then route the control signal packets to the general Internet 410. 接着经由将最可能系光纤收发器接口的接口将控制信号分组路由至WAN接口444至主机服务210。 WAN interface 444 to the host via the then most likely will be a fiber transceiver interface control interface 210 signals the packet is routed to the service. WAN 444将接着将控制信号分组路由至路由逻辑409 (其可以许多不同方式来实施,包括以太网交换器及路由服务器),路由逻辑409估计用户的地址且将控制信号路由至用于给定用户的正确的服务器402。 WAN 444 will then route the control signal packets to routing logic 409 (which may be implemented in many different ways, including Ethernet switches and routing servers), which evaluates the user's routing address and routes the control signals to the user for a given the correct server 402.

26[0146] 服务器402接着将所述控制信号视为在服务器402上执行的游戏或应用程序软件的输入且使用所述控制信号来处理游戏或应用程序的下一个帧。 26 [0146] The server 402 then takes the control signals as input for the game or application software executing on the server 402 and uses the control signals to process the next frame of the game or application. 一旦产生下一个帧,就将视频及音频自服务器402输出至视频压缩器404。 Once the next frame, it will output video and audio from the server 402 to video compressor 404. 可经由各种装置将视频及音频自服务器402输出至压缩器404。 Via various means may output video and audio from the server 402 to compressor 404. 首先,可将压缩器404创建到服务器402中,因此可在服务器402 内在本地实施压缩。 First, the compressor 404 may be created to the server 402, so the compression may be inherent in the local server 402 embodiment. 或者,可经由到网络(其或者是服务器402与视频压缩器404之间的私用网络,或者是经由诸如SAN 403的共用网络的网络)的网络连接(诸如,以太网连接) 以分组化的形式输出视频和/或音频。 Alternatively, via a network (which is either the server 402 and video compressor 404 between a private network, or a shared network SAN via a network such as 403) connected (such as an Ethernet connection) to a packetized network in the form of output video and / or audio. 或者,可经由视频输出连接器(诸如,DVI或VGA连接器)将视频自服务器402输出,且接着由视频压缩器404来捕获。 Alternatively, and then captured by video compressor 404 via the video output connector (such as, DVI or VGA connector) to the video output 402 from the server. 而且,可将音频自服务器402输出为数字音频(例如,经由T0SLINK或S/PDIF连接器)或模拟音频,模拟音频由视频压缩器404内的音频压缩逻辑来数字化并编码。 Also, the audio may be output from server 402 to a digital audio (e.g., via T0SLINK or S / PDIF connector) or as analog audio, the audio by the audio compressor 404 within the video compression logic digitized and encoded.

[0147] —旦视频压缩器404已捕获来自服务器402的视频帧及在该帧时间期间所产生的音频,则视频压缩器将使用下文所描述的技术压缩视频及音频。 [0147] - Once video compressor 404 has captured the video frame and the audio generated during that frame time from server 402, then video compressor will use the techniques described below compressed video and audio. 一旦视频及音频被压缩,就通过一地址将其分组化以将其发送回至用户的客户端415,且将其路由至WAN接口444,WAN 接口444接着经由通用互联网410路由视频及音频分组,通用互联网410接着将视频及音频分组路由至用户的ISP的存在点441,存在点441将视频及音频分组路由至用户场所处的WAN接口442,WAN接口442将视频及音频分组路由至防火墙/路由器/NAT设备443,防火墙/路由器/NAT设备443接着将视频及音频分组路由至客户端415。 Once the video and audio is compressed it is an address which is packetized to send it back to the user client 415, and it is routed to the WAN interface 444, a WAN interface 444 then packets through the general Internet 410 routes the video and audio, general Internet 410 then the video and audio packets to the user's ISP point of presence 441, a point 441 the video and audio packets to the WAN Interface 442 user places at, WAN interface 442 the video and audio packets to the firewall / router / NAT device 443, firewall / router / NAT device 443 then routes the video and audio packets to the client 415.

[0148] 客户端415解压缩视频及音频,且接着在显示设备422(或客户端的内置显示设备)上显示视频并将音频发送到显示设备422或到单独的放大器/扬声器或到创建到客户端中的放大器/扬声器。 [0148] The client 415 decompresses the video and audio, and then the display 422 apparatus (built-in or client display device) and sends the audio to the display device 422 or to separate amplifier / speakers or to the client on the display the amplifier / speaker.

[0149] 为使用户感觉到刚才所描述的整个过程在感知上没有滞后,来回行程延迟需要小于70毫秒或80毫秒。 [0149] To enable users to perceive that the entire process just described is perceptually without lag, the round-trip delay needs be less than 70 or 80ms. 所描述的来回行程路径中的一些延时延迟受主机服务210和/或用户的控制,而其他的延时延迟不受主机服务210和/或用户的控制。 The described round-trip path delay some delay by the hosting service and / or user control 210, while the other delay from delay the hosting service 210 and / or user control. 尽管如此,基于大量真实世界情况的分析及测试,以下为近似量测。 Nevertheless, based on extensive real-world case analysis and testing, the following are approximate measurements.

[0150] 用于发送控制信号的单向传输时间451通常小于1毫秒,经由用户场所的来回行程路由452通常使用以太网上的易得的消费者级防火墙/路由器/NAT交换器而在约1毫秒内完成。 [0150] one-way transmission time to send the control signals 451 is typically less than 1 ms, the roundtrip routing through the user premises 452 is typically accomplished via use of readily available consumer-grade Firewall / Router / NAT switches over Ethernet in about 1 ms completed within. 用户ISP广泛地改变其来回行程延迟453,但在DSL及电缆调制解调器提供者的情况下,通常看见其在10毫秒与25毫秒之间。 User ISP vary widely in their round trip delays 453, but with DSL and cable modem providers, we typically see between 10 and 25ms. 通用互联网410上的来回行程延时可视频务被如何路由及路在线是否存在任何故障(且该问题在下文加以论述)而极大地改变, 但通常通用互联网提供相当最佳的路由且延时很大程度上由光穿过光纤的速度(给定到目的地的距离)来确定。 Common Internet 410 round trip delay can be on how traffic is routed and whether there is any fault (and these issues are discussed below) vary greatly, but usually quite common Internet offers the best route and delay it the speed of light through the fiber (for a given distance to the destination) is determined to a large extent. 如下文进一步论述,已确定1000英里作为期望将主机服务210远离用户场所211置放的大致最远距离。 As discussed further below, we have established 1000 miles as a desired host service 210 away from user premises 211 disposed roughly the furthest distance. 在1000英里处(来回行程2000英里),用于信号经由互联网的实际传输时间为约22毫秒。 At 1000 miles (2000 miles round trip), the practical transit time for a signal through the Internet is approximately 22ms. 至主机服务210的WAN接口444通常为具有可忽略的延时的商业级光纤高速接口。 WAN Interface 441 to the hosting service 210 is typically a commercial-grade fiber having a high-speed interface with negligible latency. 因此,通用互联网延时454通常在1毫秒与10毫秒之间。 Thus, the general Internet latency 454 is typically between 1 and 10ms. 经由主机服务210的单向路由455延时可在小于1毫秒内达成。 Host service 210 via a one-way routing 455 latency can be achieved in less than 1 ms. 服务器402通常将在小于一帧时间(其在60fps下为16. 7毫秒)的时间中计算用于游戏或应用程序的新帧,因此16毫秒为将使用的合理的最大单向延时456。 Server 402 typically in less than one frame time (which is at 60fps 16. 7 ms) of calculating a new frame of the game or application, so 16ms is a reasonable maximum one-way latency 456 to use. 在本文中所描述的视频压缩及音频压缩算法的最佳硬件实施中,压缩457可在1毫秒内完成。 Video herein described compression and audio compression algorithms optimized hardware implementation, the compression 457 can be completed in 1 ms. 在次佳版本中,压缩可花费多达6毫秒(当然,更欠佳的版本可花费更长时间,但所述实施将影响来回行程的总延时且将需要其他延时较短(例如,可减小经由通用互联网的可允许的距离)以维持70毫秒-80毫秒延时目标)。 In less optimized versions, the compression may take as much as 6ms (of course even less optimized versions could take longer, but such implementations would impact the overall latency round trip and would require other latencies to be shorter (e.g., It may reduce the allowable distance through the general Internet) to maintain the 70-80ms latency target). 已经考虑互联网454、用户ISP 453及用户场所路由452的来回行程延时,因此剩余的为视频解压缩458延时,视频解压缩458延时取决于视频解压缩458是实施于专用硬件中还是实施于客户端设备415(诸如,PC或行动设备)上的软件中,可视显示器的大小及解压缩CPU的性能而改变。 Has been considered Internet 454, User ISP 453, and User Premises Routing 452 round trip delay, thus remaining compressed video decompression 458 latency which, depending on the video decompression 458 latency video decompression 458 is implemented in dedicated hardware, or if implemented the client device 415 (such as, PC or mobile device) software, the size of the display and the compression performance of the CPU is changed. 通常,解压缩458花费1毫秒与8毫秒之间的时间。 Typically, decompression 458 takes between 1 millisecond and time 8 ms.

[0151] 因此,可通过将在实践中所见的所有最糟状况的延时加在一起来确定图4a中所显示的系统的用户可预期将经历的最糟状况的来回行程延时。 [0151] Thus, with the user to determine the round-trip latency of the system shown in Figure 4a may be expected to be experience by the worst-case delay seen in practice, the worst-case all added. 他们是:1+1+25+22+1+16+6+8 =80毫秒。 They are: 1 + 1 + 1 + 25 + 22 + 16 + 6 + 8 = 80ms. 此外,实际上,在实践中(具有下文所论述的防止误解的说明),此大致为使用图4a中所显示的系统的原型版本(在美国使用现成的Windows PC作为客户端设备及家庭DSL及电缆调制解调器连接)所见的来回行程延时。 In addition, in fact, in practice (with a description discussed below caveats), a prototype version of this figure 4a generally use the display system (used in the US as a ready-made Windows PC client device and home DSL and cable modem connection) round-trip latency seen. 当然,优于最糟状况的情况可导致短得多的延时,但不可依赖其来开发广泛使用的商业服务。 Of course, the situation is better than the worst case can result in a much shorter delay, but can not be relied upon in developing business services widely used.

[0152] 为了经由通用互联网达成图4b中所列出的延时,需要客户端415中的视频压缩器404及视频解压缩器412 (来自图4a)产生具有非常特定的特性的分组流,以使得经由自主机服务210至显示设备422的整个路径所产生的分组序列不经受延迟或过多分组丢失,且详言之,始终如一地落在经由用户的互联网连接(经由WAN接口442及防火墙/路由器/ NAT 433)而可用于用户的带宽的约束内。 Packet stream [0152] To achieve the latencies listed in Figures 4b over the general Internet, it requires the client 415 to the video compressor 404 and video decompressor 412 (from FIG. 4a) generated which very particular characteristics, to such that the packet sequence from the hosting service via path 210 to the entire display device 422 is not subject to delays produced or excessive packet loss and detail, consistently falls via the user's Internet connection (WAN via the interface 442 and firewall / router / NAT 433) and the constraints of the bandwidth available for the user. 另外,视频压缩器必须产生足够强健的分组流,以使得其可容忍在正常互联网及网络传输中出现的不可避免的分组丢失及分组重排序。 Further, the video compressor must create a packet stream which is sufficiently robust so that it can be tolerated in normal Internet and network transmissions inevitable packet loss and packet reordering.

[0153] 低延时视频压缩 [0153] low-latency video compression

[0154] 为了完成上述目标,一个实施例采用新的视频压缩方法,该方法降低用于传输视频的延时及峰值带宽要求。 [0154] In order to accomplish the foregoing goals, one embodiment takes a new approach to video compression which decreases the latency and the peak bandwidth requirements for transmitting video. 在描述该实施例之前,将关于图5及图6a至图6b提供对当前视频压缩技术的分析。 Prior to the embodiment described, it will provide an analysis of current video compression techniques with respect to FIGS. 5 and 6a-6b. 当然,若用户具备足以处理所述技术所需的数据速率的带宽,则该技术可根据基本原理来使用。 Of course, if the user is provided with sufficient data rate required by the processing bandwidth technology, the technique may be used in accordance with the basic principles. 注意,本文中不解决音频压缩,而是陈述音频压缩与视频压缩同时且同步地来实施。 Note that this article does not solve the audio compression, audio compression statement but with video compression is implemented simultaneously and synchronously. 满足用于该系统的要求的先前技术音频压缩技术存在。 Meet the requirements for this system of prior art audio compression techniques exist.

[0155] 图5说明用于压缩视频的一个特定先前技术,其中由压缩逻辑520使用特定压缩算法来压缩每个个别视频帧501-503以产生一系列经压缩的帧511-513。 [0155] Figure 5 illustrates one particular prior art video compression, where the compression logic 520 using a particular compression algorithm to compress each individual video frame 501-503 to generate a series of compressed frames 511-513. 该技术的一个实施例是“运动JPEG”,其中根据联合图像专家群(JPEG)压缩算法基于离散余弦变换(DCT) 来压缩每一帧。 One embodiment of this technique is "motion JPEG", wherein the compression algorithm to compress each frame based discrete cosine transform (DCT) according to the Joint Photographic Experts Group (JPEG). 可使用各种不同类型的压缩算法,然而,仍遵守该基本原理(例如,以小波为基础的压缩算法,诸如JPEG-2000)。 Using a variety of different types of compression algorithms, however, still comply with the basic principles (for example, wavelet-based compression algorithms, such as JPEG-2000).

[0156] 此类型压缩的一个问题在于:其减小了每个帧的数据速率,但其不利用连续帧之间的类似性来减小总视频流的数据速率。 [0156] A problem with this type of compression is that: it reduces the data rate of each frame, but to reduce the data rate of the overall video stream does not exploit similarities between successive frames. 举例而言,如图5中所说明,假定640 X 480 X 24比特/像素=640*480*24/8/1024 = 900千字节/帧(KB/帧)的帧速率,则对于给定质量的图像,运动JPEG可能仅将该流压缩1/10,从而产生90KB/帧的数据流。 For example, in Figure 5, assuming a 640 X 480 X 24 bits / pixel = 640 * 480 * 24/8/1024 = 900 Kilobytes / frame (KB / frame) frame rate, then for a given the quality of image, motion JPEG may only compress the stream 10, resulting in a data stream 90KB / frame. 在60帧/秒下,此将需要90KB*8比特*60帧/秒=42. 2Mbps的信道带宽,其对于美国现今几乎所有的家庭互联网连接而言将为极高的带宽,且对于许多办公室互联网连接而言为过高的带宽。 At 60 frames / sec, this would require 90KB * 8 bits * 60 frames / sec = 42 channel bandwidth of 2Mbps, which is for the United States today almost all home Internet connection will be very high in terms of bandwidth, and for many offices Internet connection is too high in terms of bandwidth. 实际上,假定其在此种高带宽的情况下要求恒定数据流,且其将仅服务一个用户,则即使在办公室LAN环境中,其也将消耗IOOMbps以太网LAN的带宽的大百分比及支持LAN的负担沉重的以太网交换器。 Indeed, given that it requires a constant flow of data in the case of such a high bandwidth, and it will only serve one user, even in the office LAN environment, it will also consume a large percentage IOOMbps Ethernet LAN bandwidth and support LAN the burden of Ethernet switches. 因此,当与其他压缩技术(诸如下文所描述的那些技术)相比较时,用于运动视频的压缩无效率。 Thus, when compared with other compression techniques (such as those techniques described below), the compression for motion video is inefficient. 此外,使用有损压缩算法的单个帧压缩算法(如JPEG及JPEG-2000) 产生在静止图像中可能不引人注意的压缩假影(例如,场景中的密集树叶内的假影可能不呈现为假影,因为眼并不确切地知道密集树叶应如何呈现)。 Further, a single frame compression algorithms lossy compression algorithm (such as JPEG and JPEG-2000) to generate a still image may not be noticeable compression artifacts (e.g., an artifact within dense foliage in the scene may not appear as artifacts, because the eye does not know exactly how dense foliage should appear). 但是,一旦场景在运动中,假影就可能突出,因为眼侦测到自帧至帧而改变的假影,尽管假影在场景的区域(在该区域中, 假影在静止图像中可能不引人注意)中。 However, once the scene in motion artifacts may stand out because the eye detects change from frame to frame and artifacts, although artifacts in areas of the scene (in this region, artifacts may not be in a still image interesting) in. 此导致帧序列中的“背景噪音”的感知,该“背景噪音”的外观与边缘模拟TV接收期间可见的“雪花”杂音类似。 This results in a sequence of frames "background noise" in the perception of "background noise" and the appearance of the visible during marginal analog TV reception "snow" noise similar. 当然,该类型的压缩仍可用于本文中所描述的特定实施例中,但一般而言,为了避免场景中的背景噪音,对于给定感知质量,需要高数据速率(也即,低压缩比率)。 Of course, this type of compression may still be used in certain embodiments described herein, but generally speaking, to avoid background noise in the scene, for a given perceptual quality, a high data rate (i.e., a low compression ratio) .

[0157] 其他类型的压缩(诸如,H. 264,或Windows媒体VC9、MPEG2及MPEG4)在压缩视频流中均更有效,因为其利用连续帧之间的类似性。 [0157] Other types of compression (such as, H. 264, or Windows Media VC9, MPEG2 and MPEG4) were more effective at compressing a video stream because they exploit the similarities between successive frames. 这些技术均依赖于用于压缩视频的相同的一般技术。 These techniques all rely upon the same general techniques to compress video. 因此,尽管将描述H. 264标准,但相同的一般原理适用于各种其他压缩算法。 Thus, although the H. 264 standard will be described, but the same general principles apply to various other compression algorithms. 大量H. 264压缩器及解压缩器可用,包括用于压缩H. 264的x264开放源软件库及用于解压缩H. 264的FFmpeg (—种视频和音频流方案)开放源软件库。 H. large compressor 264 and decompressor are available, including for compressing H. x264 open source software libraries for decompressing and 264 H. FFmpeg 264 (the - kind of the program video and audio streams) open source software library.

[0158] 图6a及图6b说明例示性先前技术压缩技术,其中由压缩逻辑620将一系列未经压缩的视频帧501-503,559-561压缩成一系列“I帧” 611、671 ;“P帧” 612-613 ;及“B 帧” 670。 [0158] Figures 6a and 6b illustrate an exemplary prior art compression technique embodiments, wherein the compression logic 620 to a series of uncompressed video frames 501-503, 559-561 are compressed into a series of "the I frame" 611,671; "P-frames" 612-613; and "B frame" 670. 图6a中的垂直轴大体表示经编码的帧中的每一者的所得大小(尽管所述帧未按比例进行绘制)。 The vertical axis in Figure 6a generally designated by the resulting size of each encoded frame (although the frames are not drawn to scale). 如上所述,本领域的技术人员良好地理解使用I帧、B帧及P帧的视频写码。 As described above, the present art is well understood in the art using I frames, B frames and P frames of video write code. 简言之,I帧611为完全未压缩的帧501的基于DCT的压缩(类似于如上所述的经压缩的JPEG图像)。 Briefly, I-frame 611 is complete uncompressed frame 501 DCT-based compression (compressed JPEG image as described above) is based. P帧612-613的大小通常显著小于I帧611的大小,因为其利用先前I帧或P帧中的数据;也即,其含有指示先前I帧或P帧之间的改变的数据。 P frames 612-613 generally are significantly smaller than the size of the I frames 611 because they exploit the previous I frame or P frame; i.e., they contain data indicating the previous I frame or P frame change between. 除B帧使用随后参考帧中的帧以及(可能地)之前参考帧中的帧之外,B帧670类似于P帧。 Followed except that B frames use the frame in the reference frame, and (possibly) in a previous frame other than the reference frame, B frames 670 are similar to P-frames.

[0159] 对于以下论述,将假定所要的帧速率为60帧/秒,每个I帧为约160Kb,平均P帧及B帧为16Kb且每隔一秒产生一新的I帧。 [0159] For the following discussion, it will be assumed to be a frame rate of 60 frames / second, that each I frame is 160Kb about, the average P frame and B frame is generated every second 16Kb, and a new I-frame. 在该组参数下,平均数据速率将为: 160Kb+16Kb*59 = 1. 1Mbps。 In this set of parameters, the average data rate would be: 160Kb + 16Kb * 59 = 1. 1Mbps. 该数据速率适当地落在用于到家庭及办公室的许多当前宽带互联网连接的最大数据速率内。 The data rate falls well into many current maximum data rate of broadband Internet connections to homes and offices. 该技术也倾向于避免来自仅帧内编码的背景噪音问题,因为P帧及B帧追踪帧之间的差异,因此压缩假影倾向于不自帧至帧而呈现及消失,从而减少上文所描述的背景噪音问题。 This technique also tends to avoid the background noise problem from intraframe-only encoding because the difference between the P-frames and B frames track frames, so compression artifacts tend not from frame-to-frame to appear and disappear, thereby reducing the above background noise problem described.

[0160] 上述类型的压缩的一个问题在于:尽管平均数据速率相对低(例如,1. IMbps),但单个I帧可能花费若干个帧时间来传输。 [0160] Compression of the type described above a problem in that: although the average data rate is relatively low (for example, 1 IMbps), a single I frame may take several frame times to transmit. 举例而言,使用先前技术,2. 2Mbps网络连接(例如,DSL或电缆调制解调器,其具有来自图3a的最大可用数据速率302的2. 2Mbps峰值) 通常将足够使视频以1. IMbps流动,每60个帧一个160Kbps的I帧。 For example, using the prior art, 2. 2Mbps network connection (e.g., DSL or cable modem, which has a maximum available data rate 302 from Figure 3a 2. 2Mbps peak) would typically be adequate to 1. IMbps video flow, each 60 a 160Kbps I frame of. 这将通过使解压缩器在解压缩视频之前将1秒视频排入队列来完成。 This will be done by having the decompressor before decompressing the video into the video queue 1 second. 在1秒内,将传输1. 1Mb的数据,其将通过2. 2Mbps最大可用数据速率来容易地适应,即使假定可用数据速率可能周期性地下降多达50%也如此。 Within 1 second, to transfer data 1. 1Mb, which would be easily accommodated by 2. 2Mbps maximum available data rate, even assuming that the available data rate might dip periodically by as much as 50% true. 遗憾地,此先前技术方法将由于接收器处的1秒视频缓冲而导致视频的1秒延时。 Unfortunately, this prior art approach due to the 1 -second video buffer at the receiver result in a 1 second delay video. 此种延迟对于许多先前技术应用程序(例如,线性视频的回放)而言足够,但对于不可容忍大于70毫秒-80毫秒的延时的快动作视频游戏而言其为极长的延时。 Such a delay is adequate for many prior art applications (e.g., playback of linear video) is sufficient in terms of, for fast action video games but not tolerate more than 70-80ms of latency, which is very long in terms of delay.

[0161] 若进行尝试来消除1秒视频缓冲,其仍将不会导致用于快动作视频游戏的足够延时减少。 [0161] If an attempt to eliminate the 1 -second video buffer, it still would not result in a sufficient delay is reduced fast action video games. 举例而言,如先前所述,B帧的使用将需要接收I帧的前的所有B帧以及I帧。 For example, as previously described, the use of B frames need to receive all of the B frames and the I frame before the I frame. 若假定在P帧与B帧之间大致分裂59个非I帧,则将存在至少29个B帧且可显示任何B帧之前所接收的I帧。 If we assume the 59 non-I are roughly split between P frames and B frames, at least 29 B frames and the I frame may be displayed prior to any received frame B will exist. 因此,不管信道的可用带宽如何,均需要29+1 = 30个帧每一者1/60 秒持续时间的延迟,或500毫秒的延时。 Thus, regardless of the available bandwidth of the channel, 29 + 1 = 30 frames for each 1/60 second delay duration, or 500 milliseconds is required. 显而易见,该时间极长。 Clearly, the time is very long. [0162] 因此,另一方法将是消除B帧且仅使用I帧及P帧。 [0162] Thus, another approach would be to eliminate B frames and only use I and P frames. (其中一个后果是:对于给定质量水平,数据速率将增加,但出于此实例中的一致性起见,继续假定每个I帧的大小为160Kb且平均P帧的大小为16Kb,且因此数据速率仍为1. IMbps)。 (One consequence of this is: For a given level of quality, the data rate would increase, but for the sake of consistency in this example, continue to assume that each I frame is 160Kb and the average P frame is 16Kb in size, and thus the data rate is still 1. IMbps). 该方法消除了由B帧引入的不可避免的延时,因为每个P帧的解码仅依赖于先前所接收的帧。 This approach eliminates the unavoidable latency introduced by B frames, since the decoding of each P frame is only reliant on the previously received frame. 该方法仍存在的问题在于:1帧比平均P帧大得多,以致在低带宽信道上(如大多数家庭中及许多办公室中典型的),I帧的传输添加实质延时。 This method is still problematic in that: a much larger than an average P frame, that on a low bandwidth channel (e.g., most homes and in many offices, typically), transmitting the I frame adds substantial latency. 此在图6b中加以说明。 This is illustrated in Figure 6b. 视频流数据速率624低于可用最大数据速率621 (除对于I帧之外),其中I帧所需的峰值数据速率623远超过可用最大数据速率622 (且甚至超过额定最大数据速率621)。 Video stream data rate 624 is below the available maximum data rate 621 (except for the I frame), where I is the peak data rate desired frame 623 far exceeds the available maximum data rate 622 (and even the rated max data rate 621). P帧所需的数据速率小于可用最大数据速率。 P-frame data rate required less than the available maximum data rate. 即使在2. 2Mbps的可用最大数据速率峰值稳定地保持在其2. 2Mbps峰值速率,也将花费160Kb/2. 2Mb = 71毫秒来传输I帧,且若可用最大数据速率622下降50% (1. 1Mbps),则将花费142毫秒来传输I帧。 Even if the available max data rate peaks at 2. 2Mbps stably held at its peak rate 2. 2Mbps, will take 160Kb / 2. 2Mb = 71 ms to transmit the I frame, and if the available max data rate 622 dips by 50% (1 . 1Mbps), it will take 142ms to transmit the I frame. 因此,传输I帧中的延时将落在71毫秒与142毫秒之间的某处。 Thus, the delay in transmitting the I frame will fall somewhere in between 71 milliseconds and 142 milliseconds. 该延时添加到图4b中所识别的延时(所述延时在最糟状况下总计达70毫秒),因此, 这将导致从用户致动输入设备421的时刻直到图像呈现于显示设备422上的为141-222毫秒的总来回行程延时,其极高。 This latency is additive to Figure 4b, the identified time delay (the delay up to 70 ms in the worst case), and therefore, this will result from the user actuates input device 421 until an image appears on display device 422 141- 222ms on total round trip latency that high. 且若可用最大数据速率下降至低于2. 2Mbps,则延时将进一步增加。 And if the available maximum data rate falls below 2. 2Mbps, the latency will increase further.

[0163] 还注意到,以远超过可用数据速率622的峰值数据速率623使ISP “堵塞”通常存在严重后果。 [0163] Also note that the available data rate far in excess of peak data rate 622 623 that ISP "jam" generally are severe consequences. 不同ISP中的设备将不同地表现,但当以比可用数据速率622高得多的数据速率接收分组时,以下行为在DSL及电缆调制解调器ISP当中相当普遍:(a)通过将分组排入队列而使分组延迟(引入延时),(b)丢弃一些或所有分组,(c)将连接停用一时间周期(最可能因为ISP担忧其为恶意攻击,诸如“否认服务”攻击)。 ISP in different devices will behave differently, but to receive a packet data rate than the available data rate is much higher 622, the following behaviors are quite common among DSL and cable modem ISP: (a) by the packet enqueued the packet delay (latency introduced), (b) to discard some or all of the groups, (c) connecting disable a period of time (most likely because the ISP is concerned it is a malicious attack, such as "denial of service" attack). 因此,以全数据速率传输分组流(具有诸如图6b中所显示的所述特性的特性)并非为可行的选项。 Therefore, the transmission packet stream at full data rate (such as having a characteristic of the characteristic shown in FIG. 6b) is not a viable option. 可在主机服务210 处将峰值623排入队列且以低于可用最大数据速率的数据速率进行发送,从而引入前一段中所描述的不可接受的延时。 At the hosting service 210 may be queued up at peak 623 is below the available maximum data rate transmission data rate, introducing the unacceptable latency described in the preceding paragraph.

[0164] 另外,图6b中所显示的视频流数据速率序列624为非常“驯服的(tame) ”视频流数据速率序列,且将是由于压缩来自视频序列的视频而预期产生的该种数据速率序列,该视频序列并不改变很大且具有非常少的运动(例如,如在视频电话会议中将是普遍的,在视频电话会议中,相机处于固定位置中且具有非常少的运动,且场景(例如,就座的人谈话) 中的对象显示较少运动)。 [0164] Further, the video stream data rate in Figure 6b is shown in sequence 624 is "tamed (TAME)" video stream data rate sequence and would be the sort of data rate due to the compressed video from a video sequence and the expected resulting sequence, the video sequence does not change very much and has very little motion (e.g., as in a video teleconference is common in video teleconferencing where the cameras are in a fixed position and have little motion, and scenes (for example, seated people talking) in, show little movement).

[0165] 图6c中所显示的视频流数据速率序列634为从具有多得多的动作的视频(诸如, 可能在电影或视频游戏中或在某应用程序软件中产生)预期可见的典型序列。 [0165] Figure 6c video stream data rate sequence 634 shown in a typical sequence the expected visible (such as, or may generate an application software in the video game or movie) from video with far more action. 注意,除I 帧峰值633之外,还存在相当大且在许多场合上超过可用最大数据速率的P帧峰值(诸如, 635及636)。 Note that, in addition to the I frame peaks 633, there are also quite large and exceed the available on many occasions, P frame peak maximum data rate (such as 635 and 636). 尽管所述P帧峰值并不如I帧峰值一般相当大,但其仍极大以致不能由信道以全数据速率来运输,且如同I帧峰值一样,P帧峰值必须缓慢地传输(借此增加延时)。 Although these P frame peaks are not quite as large I frame peaks, they still are far that at full data rate carried by the channel, and as with the I frame peaks, they P frame peaks must be transmitted slowly (thereby increasing delay Time).

[0166] 在高带宽信道(例如,100Mbps LAN,或高带宽IOOMbps私用连接)上,网络将能够容忍诸如I帧峰值633或P帧峰值636的大峰值,但原则上,可维持低延时。 [0166] On a high bandwidth channel (e.g., 100Mbps LAN, or a high bandwidth IOOMbps private connection) the network would be able to tolerate such a frame peak large peak I 633 or P frame peaks 636, but in principle, be maintained low latency . 但是,所述网络经常在许多用户当中共用(例如,在办公室环境中),且该“有峰”数据将影响LAN的性能,尤其在网络通信经路由至私用共用连接(例如,从远程数据中心到办公室)的情况下。 However, such networks are frequently shared amongst many users (e.g., in an office environment), and such "peak" data would impact the performance of the LAN, particularly if the network traffic was routed to a private shared connection (e.g., from a remote data the case of the center to the office) is. 首先,记住此实例是60fps下640X480像素的相对低分辨率的视频流的实例。 First, keep in mind that this example is a video stream at a relatively low resolution of 640X480 pixels 60fps. 60fps 下的1920X1080的HDTV流容易由现代计算机及显示器来处理,且60fps下的2560X1440 HDTV streams 1920X1080 at 60fps are readily handled by modern computers and displays, and the lower the 2560X1440 60fps

30分辨率显示器日益可用(例如,Apple公司的30"显示器)。使用H. 264压缩,60fps下的1920X1080的高动作视频序列可能需要4. 5Mbps以获得合理质量水平。若假定I帧峰值为标定数据速率的10倍,则其将产生45Mbps峰值,以及较小但仍相当大的P帧峰值。若若干个用户正在同一IOOMbps网络(例如,办公室与数据中心之间的私用网络连接)上接收视频流,则容易看见来自若干用户的视频流的峰值可如何碰巧对准,从而淹没网络的带宽,且可能淹没网络上支持用户的交换器的底板的带宽。即使在超高速以太网的状况下,若足够的用户具有同时对准的足够峰值,则其可淹没网络或网络交换器。此外,一旦2560X1440 分辨率视频变得更常见,平均视频流数据速率就可能为9. 5Mbps,从而或许产生95Mbps峰值数据速率。不用说,数据中心与办公室之间的IOOMbps连接(其在现今为格外快的 Resolution of the display 30 are increasingly available (e.g., Apple, Inc. 30 "display) using the H. 264 compression, high action video sequence at 60fps 1920X1080 4. 5Mbps may be necessary to obtain a reasonable quality level. If we assume the I frames peak Calibration 10 times the data rate, that would result in 45Mbps peaks, as well as smaller, but still considerable, P frame peak. If several users were receiving the same is IOOMbps network (e.g., a private network connection between an office and data center) video stream, a video stream is easy to see the peaks from several users could happen to align, overwhelming the bandwidth of the network, and potentially overwhelming the bandwidth of the backplanes of the switches supporting the users on the network device. even under conditions of ultra high-speed Ethernet , if enough users had enough peaks aligned at once, it could overwhelm the network or the network switches. Additionally, once 2560X1440 resolution video becomes more commonplace, the average video stream data rate may be 9. 5Mbps, resulting in perhaps a 95Mbps peak data rate. Needless to say, IOOMbps connection (which today is exceptionally fast between the data center and office 接) 将完全被来自单一用户的峰值通信击溃。因此,即使LAN及私用网络连接对有峰流动视频可具有更高容忍度,具有高峰值的流动视频也是不需要的且可能需要办公室的IT部门的特殊计划及适应。 Video peak traffic flow routed connection) would be completely from a single user. Therefore, even if the LAN and private network connections of peaky streaming video can be more tolerant, with high peaks is not desirable and might require office IT special programs department and adapt.

[0167] 当然,对于标准线性视频应用程序,该问题并不是问题,因为数据速率在传输点“经平滑化”且用于每一帧的数据低于最大可用数据速率622,且在解压缩I帧、P帧及B帧序列之前,客户端中的缓冲器储存I帧、P帧及B帧序列。 [0167] Of course, for standard linear video applications these issues are not a problem because the data rate in the transmission point "smoothed" and the data for each frame below the max available data rate 622, and decompression I preceding frame, P frame and B frame sequence, stored in the client buffer I frame, P frame and B frame sequence. 因此,网络上的数据速率保持接近于视频流的平均数据速率。 Thus, the data rate over the network remains close to the average data rate of the video stream. 很遗憾地,这引入延时,即使不使用B帧,对于诸如需要快响应时间的视频游戏及应用程序的低延时应用程序而言,这也是不可接受的。 Unfortunately, this introduces latency, even if B frames are not used, such as the need for rapid response in terms of low-latency applications and video game application time, which is unacceptable.

[0168] 用于减轻具有高峰值的视频流的一个先前技术解决方法是使用常常被称作“恒定比特速率”(CBR)编码的技术。 [0168] for reducing a peak value of the video stream with the prior art solution is to use is often referred to as "constant bit rate" (the CBR) encoding techniques. 尽管术语CBR看来似乎暗示将所有帧压缩以具有相同比特速率(也即,大小),但其经常提及的为压缩范例,在压缩范例中,允许跨越特定数目的帧(在我们的状况下,1个帧)的最大比特速率。 Although the term CBR would seem to imply that all frames are compressed to have the same bit rate (i.e., size), what it usually refers to is a compression paradigm, in the compression example, across a certain number of frames (in our case conditions , 1 frame) is the maximum bit rate. 举例而言,在图6c的状况下,若对编码施加CBR 约束(其将比特速率限于(例如)额定最大数据速率621的70% ),则压缩算法将限制所述帧中的每一者的压缩,以使得通常将使用额定最大数据速率621的70%以上来压缩的任何帧将以较少比特来压缩。 For example, in the case of Figure 6c, if a CBR constraint were applied to the encoding (which limited the bit rate (e.g.) rated maximum data rate 621 is 70%), then the compression algorithm would limit each of the frames of the compression, such that any frame will typically be used rated maximum data rate 621 is compressed to 70% or more will be compressed with less bits. 这个结果是:将使通常将需要更多比特来维持给定质量水平的帧“极度缺乏”比特且所述帧的图像质量将比不需要比额定最大数据速率621的70%多的比特的其他帧的图像质量糟。 The result is: that would normally require more bits to maintain a given level of quality of the frame is "starved" of bits and the image quality of those frames would not require more than the rated maximum data rate 621 is 70% more than other bits bad image quality frames. 此方法对于特定类型的经压缩视频(其中(a)预期较少运动或场景改变且(b)用户可接受周期性的质量降级)可产生可接受的结果。 This method for certain types of compressed video by (wherein (a) little motion or scene changes are expected and (b) the users can accept periodic quality degradation) can produce acceptable results. 适合CBR的应用的良好实例为视频电话会议,因为存在较少峰值,且在质量暂时降级的情况下(例如,若使相机扫视,从而导致显著场景运动及大峰值,则在扫视期间可能不存在足够的比特用于高质量图像压缩,其将导致降级的图像质量),大多数用户可接受。 Good examples of applications suitable for CBR video conference call, because there are few peaks, and in the case of temporary downgrading of quality (for example, if the camera is panned, resulting in a significant scene motion and large peak, it may not be present during the panning enough bits for high- quality image compression, which would result in degraded image quality), acceptable for most users. 很遗憾地,CBR并非良好地适合具有高复杂度的场景或大量运动和/或需要合理恒定的质量水平的许多其他应用。 Unfortunately, CBR is not well suited to have scenes of high complexity or a lot of movement and / or many other applications where a reasonably constant level of quality.

[0169] 在一个实施例中所使用的低延时压缩逻辑404使用若干不同技术来解决流动低延时经压缩视频同时维持高质量的许多问题。 [0169] In the low-latency compression used in one embodiment of logic 404 implemented using several different techniques to address many problems streaming low-latency compressed video, while maintaining high quality. 首先,低延时压缩逻辑404仅产生I帧及P 帧,从而缓解等待若干个帧时间来解码每个B帧的需要。 First, the low-latency compression logic 404 generates only I frames and P frames, thereby alleviating wait several frame times to decode each B frame. 另外,如图7a中所说明,在一个实施例中,低延时压缩逻辑404将每个未经压缩的帧701-760再分成一系列“图像块(tile) ” 且将每个图像块个别地编码为I帧或P帧。 Further, as illustrated in Figure 7a, in one embodiment, the low-latency compression logic 404 subdivides each uncompressed frame 701-760 into a series of "(the tile)" for each image block and the individual encoded as an I or P frame. 在本文中将该群经压缩的I帧及P帧称作“R 帧”711-770。 Herein The group of compressed I frames and P frames are referred to as "R frames" 711-770. 在图7a中所显示的特定实例中,将每个未经压缩的帧再分成4X4矩阵的16 个图像块。 In the specific example shown in Figure 7a, each uncompressed frame is subdivided into 16 image blocks of the 4X4 matrix. 然而,所述基本原理不限于任何特定再分机制。 However, these underlying principles are not limited to any particular subdivision scheme.

[0170] 在一实施例中,低延时压缩逻辑404将视频帧划分成许多图像块,且将来自每个 [0170] In one embodiment, the low-latency compression logic 404 into a video frame into a number of tiles, and from each

31帧的一个图像块编码(也即,压缩)为I帧(也即,将该图像块压缩,好像其为全图像的大小的1/16的单独视频帧,且用于此“迷你型”巾贞的压缩为I帧压缩)并将剩余图像块编码为P帧(也即,用于每个“迷你型” 1/16帧的压缩为P帧压缩)。 An image block coding (i.e., compression) of 31 as an I frame (i.e., the tile is compressed as if it is a separate video frame size is 1/16 of the full image, and for this "mini" towel Fok I frame compression) and the remaining tiles as P frames (i.e., for each "mini" 1/16 compressed frame is P frame compression). 经压缩为I帧的图像块及经压缩为P帧的图像块将分别被称作“I图像块”及“P图像块”。 The compressed image block as compressed I frames and P frames of the image will be referred to as "I tiles" and "P tiles." 随着每个相继视频帧而改变待编码为I图像块的图像块。 With each successive video frame, the tile to be encoded as I tile. 因此,在给定帧时间中,视频帧中的所述图像块中仅一个图像块为I图像块,且所述图像块中的剩余者为P图像块。 Thus, in a given frame time, the image block in the video frame only one image block is an I tile, and the remainder of the image blocks as P tiles. 举例而言,在图7a中,未经压缩的帧701的图像块0经编码为I图像块IO且剩余的1-15图像块经编码为P图像块(Pl至P15) 以产生R帧711。 For example, in Figure 7a, tile 0 of uncompressed frame 701 is encoded as I tile IO and the remaining 1 -15 tiles are encoded as P tiles (Pl is to P15) to produce R frame 711 . 在下一个未经压缩的视频帧702中,未经压缩的帧701的图像块1经编码为I图像块Il且剩余的图像块0及2至15经编码为P图像块(P0,及P2至P15)以产生R 帧712。 An uncompressed video frames 702 in the next frame of uncompressed image block 701 is encoded as an I tile Il and the remaining tiles 0 and 2 through 15 are encoded as P tiles (P0, P2 and to P15) to produce R frame 712. 因此,用于图像块的I图像块及P图像块在相继帧上逐渐地在时间上交错。 Thus, I tiles and P tiles for tiles are progressively interleaved in time over successive frames. 该过程继续,直至产生R图像块770 (矩阵中最末图像块经编码为I图像块(也即,115))为止。 This process continues until a R tile 770 is generated (the last tile in the matrix encoded as an I tile (i.e., 115)) so far. 该过程接着重新开始,从而产生诸如帧711(也即,对于图像块0,编码I图像块)等的另一个R帧。 The process then starts over, generating such as frame 711 (i.e., the image block 0, encoding an I tile) another R frame or the like. 尽管图7a中未说明,但在一个实施例中,R帧的视频序列的第一R帧仅含有I图像块(也即,以使得随后的P帧具有参考图像数据(自其开始计算运动))。 Although not illustrated in Figure 7a, in one embodiment, the first R frame of the video sequence of R frames contains only I tiles (i.e., so that subsequent P frames have reference image data (counted from its movement) ). 或者,在一实施例中,启动序列使用与正常相同的I图像块型样,但不包括用于尚未连同I图像块一起编码的所述图像块的P图像块。 Alternatively, in one embodiment, the normal startup sequence uses the same I tile pattern, but does not include P tiles for the image block together with the coded Not I tile. 换言之,在第一I图像块到达之前不连同任何数据一起编码特定图像块,从而避免图9a中的视频流数据速率934中的启动峰值,其在下文进一步详细说明。 In other words, not together with the first I tile arrives before any data with encoding a particular image block, thereby avoiding startup peaks in the video stream data rate 934 in FIG. 9A, which is explained in further detail below. 此外,如下所述,各种不同大小及形状可用于所述图像块同时仍遵守所述基本原理。 Further, as described below, various different sizes and shapes may be used for the tiles while still complying with these underlying principles.

[0171] 在客户端415上执行的视频解压缩逻辑412解压缩每个图像块,好像其为小I帧及P帧的单独视频序列,且接着将每个图像块再现给驱动显示设备422的帧缓冲器。 [0171] Solutions of the video decompression logic 412 executing on the client 415 decompresses each tile compression, if it is a small separate video sequence of I frames and P frames, and then renders each tile to the driving of the display device 422 frame buffer. 举例而言,使用来自R帧711至770的IO及PO来解压缩并再现视频图像的图像块0。 For example, the IO and PO from R frames 711 to 770 are used to decompress and render tile 0 of the video image. 类似地, 使用来自R帧711至770的Il及Pl来重建图像块1,等等。 Similarly, the use of Il Pl and from R frames 711 to 770 are used to reconstruct tile 1, and so on. 如上所述,I帧及P帧的解压缩是这项技术中众所熟知的,且I图像块及P图像块的解压缩可通过使视频解压缩器的多个执行个体在客户端415上执行来完成。 Mentioned above, I frame and P frame compression solutions are all well known in the art, and I P tiles and tiles solution may be performed by compression of video decompressor plurality of individual client 415 execution to complete. 尽管倍增过程看来似乎增加客户端415上的计算负担,但实际上其不会增加客户端415上的计算负担,因为图像块本身成比例地较小(相对于额外处理的数目而言),因此所显示的像素的数目相同,好像存在一个处理且使用传统的全大小的I帧及P帧。 Although multiplying processes would seem to increase the computational burden on client 415, it actually does not increase the computational burden on client 415, because the tile themselves are proportionally smaller (relative to the number of additional processing), Thus the same number of pixels displayed, as if there were one process and using conventional full sized I and P frames.

[0172] 该R帧技术显著减轻通常与I帧相关联的带宽峰值(图6b及图6c中所说明),因为任何给定帧主要是由通常比I帧小的P帧构成。 [0172] This R frame technique significantly mitigates the bandwidth peaks typically associated with I frames (FIGS. 6b and 6c illustrated), because any given frame is mostly made of typically smaller than I frames P frames. 举例而言,再次假定典型I帧为160Kb, 则图7a中所说明的帧中的每一者的I图像块将为该量的大致1/16或10Kb。 For example, again assume that a typical I frame is 160Kb, 7a in each of the frames illustrated in FIG. I tile will be the amount that is approximately 1/16 or 10Kb. 类似地,假定典型P帧为16Kb,则用于图7a中所说明的图像块中的每一者的P帧可为大致1Kb。 Similarly, assuming that a typical P frame is 16Kb, for each of the tiles illustrated in Figure 7a in a P frame may be approximately 1Kb. 最终结果为约10Kb+15*lKb = 25Kb的R帧。 The final result is approximately 10Kb + 15 * lKb = R frame is 25Kb. 因此,每一60帧序列将是25Kb*60 = 1. 5Mbps。 Thus, each 60-frame sequence would be 25Kb * 60 = 1. 5Mbps. 因此,在60帧/秒下,这将需要能够维持1. 5Mbps的带宽的信道,但由于I图像块是贯穿60 帧间隔而分散而使得具有低得多的峰值。 Thus, at 60 frames / sec, this would require a channel capable of sustaining a bandwidth of 1. 5Mbps, but due to I tiles being dispersed throughout the 60-frame interval such that the peak has a much lower.

[0173] 注意,在先前实例中,在用于I帧及P帧的相同假定数据速率情况下,平均数据速率为1. 1Mbps。 [0173] Note that in previous examples, at the same assumed data rates for I frames and P frames, the average data rate was 1. 1Mbps. 这是因为在先前实例中,每隔60个帧时间仅引入新I帧,而在此实例中,构成I帧的16个图像块在16个帧时间中循环,且因此,每隔16个帧时间引入I帧的均等物, 从而导致稍高的平均数据速率。 This is because in the previous example, every 60 frame times only introduced a new I frame, and in this example, 16 image blocks constituting the I frame 16 in the frame of the cycle time, and therefore, every 16 frames introducing time equivalent of an I frame, resulting in a slightly higher average data rate. 尽管如此,但在实践中,引入更频繁的I帧并不会线性地增加数据速率。 Nevertheless, in practice, introducing more frequent I frames does not increase the data rate linearly. 这是由于以下事实:P帧(或P图像块)主要编码自先前帧至下一个帧的差异。 This is due to the fact that: a difference between a previous frame to the next frame from the P frame (or a P tile) primarily encodes. 因此,若先前帧与下一个帧相当类似,则P帧将非常小,若先前帧与下一个帧相当不同, 则P帧将非常大。 Thus, if the prior frame is quite similar to the next frame, the P frame will be very small, if the prior frame is quite different from the next frame, the P frame will be very large. 但因为P帧很大程度上是自先前帧导出,而非自实际帧导出,所以所得的经编码帧可能含有比具有足够数目的比特的I帧多的错误(例如,视觉假影)。 But because a P frame is largely derived from the previous frame, rather than from the actual frame, the resulting encoded frame may contain a sufficient number of bits than I-frames many errors (e.g., visual artifacts). 此外,当一个P帧跟随另一P帧时,可出现错误累加(当存在长P帧序列时,变得更糟)。 Further, when one P frame follows another P frame, an accumulation of errors (when there is a long sequence of P frames worse) may occur. 现在,尖端的视频压缩器将侦测到图像的质量在一序列P帧的后降级的事实,且必要时,其将更多比特分配给随后的P帧以提高质量,或若其为最有效的动作过程,则用I帧替换P帧。 Now, a sophisticated video compressor will detect the quality of the image is degrading after a sequence of P frames of a fact, and if necessary, it will allocate more bits to subsequent P frames to improve the quality, or if it is the most effective the course of action, replace a P frame with an I frame. 因此,当使用长P帧序列(例如,59个P帧,如上文先前实例中)时,特定言的当场景具有大量复杂度和/或运动时,通常,对于P帧而言需要更多比特(因为其变得距I帧更远)。 Thus, when long sequences of P frames (e.g., 59 P frames, as in the previous example), when the scene has a particular large complexity and / or motion, typically, for P frames as more bits are needed (as it becomes farther away from the I-frame).

[0174] 或者,自相对观看点看P帧,紧密地跟随I帧的P帧倾向于需要比距I帧更远的P 帧少的比特。 [0174] Alternatively, from the opposite point P frames see, closely follow an I frame I tend to require less than the P-frames from an I frame to P frame bits. 因此,在图7a中所显示的实例中,没有P帧比距I帧隔开15个帧更远(在I 帧之前),而在先前实例中,P帧可能从I帧隔开59个帧。 Thus, the example shown in Figure 7a, no P frame is an I frame than to the frames 15 spaced farther (before I-frame), while in the previous example, a P frame could be 59 frames spaced from the I-frame . 因此,在更频繁的I帧情况下,P 帧较小。 Thus, with more frequent I frames, the P frames are smaller. 当然,确切相对大小将基于视频流的性质而改变,但在图7a的实例中,若I图像块为10Kb,则P图像块的大小平均可为仅0. 75kb,从而导致10Kb+15*0. 75Kb = 21. 25Kb,或在60帧/秒下,数据速率将为21. 25Kb*60 = 1. 3Mbps,或比1. IMbps下的具有I帧随后跟着59个P帧的流的数据速率高约16%。 Of course, the exact relative sizes will vary based on the nature of the video stream changes, but in the example of Figure 7a, if an I tile is 10Kb, P tiles on average, may be only 0. 75kb, resulting in 10Kb + 15 * 0 . 75Kb = 21. 25Kb, or at 60 frames / second, the data rate would be 21. 25Kb * 60 = 1. 3Mbps, or than the subsequent data rate with an I frame followed by 59 P frames stream under 1. IMbps height of about 16%. 再一次,用于视频压缩的所述两种方法之间的相对结果将视视频序列而改变,但通常,我们凭经验发现,对于给定质量水平,使用R帧比使用I/P 帧序列需要多约20%的比特。 Once again, the relative results between these two approaches to video compression will vary depending on the video sequence, but typically, we have found empirically that, for a given level of quality than using R-frames require I / P frame sequences about 20% more bits. 但是,当然,R帧急剧地减少峰值,这使视频序列在远小于I/ P帧序列的延时下可用。 But, of course, R frames dramatically reduce the peaks which make the video sequences usable with far less latency than I / P frame sequences.

[0175] 可视视频序列的性质、信道的可靠性及可用数据速率而以多种不同方式来配置R 帧。 Properties [0175] of the video sequence, the reliability and the available data rate channel and a number of different ways to configure a frame R. 在替代实施例中,在4X4配置中使用不同于16的数目的图像块。 In an alternative embodiment, used in a 4X4 configuration 16 different from the number of image blocks. 举例而言,可在2X1 或1X2配置中使用2个图像块,可在2X2、4X1或1X4配置中使用4个图像块,可在3X2、 2X3、6X1或1X6配置中使用6个图像块或可在4X2(如图7b中所显示)、2X4、8X 1或1 X 8配置中使用8个图像块。 For example, two tiles may be used in 2X1 or 1X2 configuration, 4 tiles may be used in 2X2,4X1 or 1X4 configuration, 6 tiles may be used in or 3X2, 2X3,6X1 or 1X6 configuration in 4X2 (shown in Figure 7b), 2X4,8X 1 or 1 X 8 configuration using eight tiles. 注意,图像块不需要为方形,视频帧也不必为方形,或甚至矩形。 Note that the tiles need not be square, but also the video frame be square, or even rectangular. 可将图像块分解成最佳地适合所使用的视频流及应用程序的无论什么形状。 The tiles can be decomposed into a video stream and the application used for optimally whatever shape.

[0176] 在另一实施例中,I图像块及P图像块的循环不锁定到图像块的数目。 [0176] In another embodiment, the cycling of the I tiles and P tiles is not locked to the number of blocks. 举例而言, 在8图像块4X2配置中,仍可如图7b中所说明而使用16循环序列。 For example, in an 8-tile 4X2 configuration, as illustrated in Figure 7b still the 16-cycle sequence. 顺序的未经压缩的帧721、722、723各自经划分成8个图像块0_7,且每个图像块被个别压缩。 Uncompressed frame order 721,722,723 are each divided into 8 tiles 0_7, and each tile is compressed individually. 自R帧731,仅图像块0被压缩为I图像块,且剩余图像块被压缩为P图像块。 From R frame 731, only tile 0 is compressed as an I tile, and the remaining tiles are compressed as P tiles. 对于随后的R帧732,所有8 个图像块被压缩为P图像块,且接着对于随后的R帧733,图像块1被压缩为I图像块且其他图像块均被压缩为P图像块。 For subsequent R frame 732 all of the 8 tiles are compressed as P tiles, and then for subsequent R frame 733, tile 1 is compressed as an I tile and the other tiles are all compressed as P tiles. 此外,如此对于16个帧继续进行排序,仅每隔一帧产生一个I图像块,因此在第15个帧时间期间(图7b中未图示)及在第16个帧时间期间产生用于图像块7的最末I图像块(使用所有的P图像块压缩R帧780)。 Further, the case for the sequencing continues frame 16, an I tile generated only every other one, so during the time frame 15 (not shown in Figure 7b), and generates an image during a frame time of 16 last I tile block 7 (using all P tiles R frame 780 is compressed). 接着,序列再次以图像块0被压缩为I图像块且其他图像块被压缩为P图像块开始。 Then, the sequence begins again with tile 0 compressed as an I tile and the other tiles compressed as P tiles. 如在先前实施例中,整个视频序列的第一帧通常将均为I图像块,以提供用于从该点向前的P图像块的参考。 As in the previous embodiment, the first frame of the entire video sequence would typically be all I tiles, to provide a reference for P tiles from that point forward. I图像块及P图像块的循环甚至不需要为图像块的数目的偶倍数。 Cyclic I tiles and P tiles need not even be an even number of image blocks in multiples. 举例而言,在8个图像块的情况下,在使用另一I图像块之前,具有一个I图像块的每一帧之后可为所有都为P图像块的2个帧。 For example, in the case of the 8 tiles, before another I tile, with each frame of I picture after a block may have two tiles as P frames for all. 在又一实施例中,若(例如)已知屏幕的特定区域具有更多运动(需要更频繁的I 图像块),而其他区域更为静态(例如,显示游戏的分数)(需要较不频繁的I图像块),则与其他图像块相比,可更经常地将特定图像块连同I图像块一起进行排序。 In yet another embodiment, if (for example) a specific region of the screen are known to have more motion (more frequent I tile), while other regions are more static (e.g., the game score display) (requires less frequent I tile), as compared with other image block may be more frequently a particular image block together with an I tile sorting. 此外,尽管在图7a-图7b中说明每个帧具有单个I图像块,但可在单个帧中编码多个I图像块(取决于传输信道的带宽)。 Further, although illustrated in FIG. 7b FIG 7a- each frame has a single I tile, but multiple I tiles may be encoded (depending on the bandwidth of the transmission channel) in a single frame. 相反地,特定帧或帧序列可在不具有I图像块(也即,仅P图像块)的情况下传输。 Conversely, certain frames or frame sequences may no I tiles (i.e., only P tiles) of the transmission case.

[0177] 前一段的方法适当起作用的原因在于:尽管不具有跨越每个单个帧而分散的I图像块看来似乎导致较大峰值,但系统的行为并不如此简单。 [0177] Cause of the preceding paragraph works well is that method: while not having dispersed across every single frame of an I tile seem to be result in larger peaks, the behavior of the system is not that simple. 因为每个图像块是与其他图像块分开进行压缩,所以当图像块变小时,每个图像块的编码可变得较不有效,因为给定图像块的压缩器不能够利用来自其他图像块的类似图像特征及类似运动。 Since each tile is compressed separately from the other tiles, it becomes small when the image blocks, each image block may be coded becomes less efficient, because the compressor of a given tile is not able to use the other image block from the similar image features and similar motion. 因此,与将屏幕划分成8个图像块相比较,将屏幕划分成16个图像块通常将导致较不有效的编码。 Thus, the screen is divided into 8 tiles compared to the screen into 16 tiles generally will result in a less efficient encoding. 但是,若将屏幕划分成8个图像块且其引起每隔8个帧(而非每隔16个帧)引入一个完全I帧的数据,则其导致总体上高得多的数据速率。 However, if the screen is divided into 8 tiles and it causes every 8 frames (instead of every 16 frames) introducing a full I frame data, it results in a much higher data rate overall. 因此,通过每隔16个帧(而非每隔8个帧)引入一个完全I帧,减小了总数据速率。 Thus, every 16 frames (instead of every 8 frames) introducing a full I frame, the overall data rate is reduced. 而且,通过使用8个较大图像块(而非16个较小图像块),减小了总数据速率,其也将由较大图像块引起的数据峰值减轻至某种程度。 Also, by using 8 larger tiles (tiles instead of 16 smaller), the overall data rate is reduced, which is also caused by the larger tiles mitigates to some degree the data peaks.

[0178] 在另一实施例中,图7a及图7b中的低延时视频压缩逻辑404通过基于待压缩的视频序列的已知特性而通过设定预先配置或者基于每个图像块中的图像质量的正在进行的分析而自动地控制至R帧中的各图像块的比特的分配。 [0178] In another embodiment, FIGS. 7a and 7b of the low latency video compression logic based on the known characteristics of the video sequence to be compressed configuration or image set in advance based on each image block by the 404 ongoing quality analysis to automatically control the distribution of bits of each image block R frame. 举例而言,在一些竞赛视频游戏中,玩家的汽车(其为场景中相对无运动的)的前方占据屏幕的下半部的大部分,而屏幕的上半部完全被填满正接近的道路、建筑物及风景,其几乎总是在运动中。 For example, in some racing video game, the player's car (which is relatively free movement of the scene) occupies most of the front of the lower half of the screen, while the top half of the screen is completely filled with the oncoming road , buildings and scenery, it is almost always in motion. 若压缩逻辑404 将相等数目的比特分配给每个图像块,则图7b中的未经压缩的帧721中的屏幕的下半部上的图像块(图像块4-7)通常将以比图7b中的未经压缩的帧721中的屏幕的上半部中的图像块(图像块0-3)高的质量而压缩。 If the compression logic 404 to allocate an equal number of bits to each tile, then the tiles (tiles 4-7) in uncompressed on the bottom half of the frame 721 7b in FIG screen ratio will generally FIG. the upper half of uncompressed frame 721 7b in the block of the image screen (tiles 0-3) in a high quality compression. 若已知该特定游戏或游戏的此特定场景具有所述特性,则主机服务210的运营商可配置压缩逻辑404以将更多比特分配给屏幕的顶部的图像块(与分配给屏幕的底部处的图像块的比特相比)。 If this particular scene of the game or particular game is known to have said characteristic, the operator of the hosting service 210 can configure the compression logic 404 to the top of the image block more bits are allocated to the screen (assigned to the bottom of the screen compared bit image block). 或者,压缩逻辑404可在压缩帧之后估计图像块的压缩质量(使用许多压缩质量度量中的一者或多者,诸如峰值信号噪音比(PSNR)),且若确定在特定时间窗上,特定图像块始终如一地产生较佳质量结果,则其逐渐地将更多比特分配给产生较低质量结果的图像块,直至各种图像块达到类似水平的质量为止。 Or, the compression logic 404 may be compressed frame after compression quality estimation image blocks (a number of quality metrics using one or more of compression, such as peak signal to noise ratio (the PSNR)), and if it is determined at a specific time window, specific tile consistently producing better quality results, then it gradually allocates more bits to tiles that are producing lower quality results, until the various tiles reach a similar level of quality. 在替代实施例中,压缩器逻辑404分配比特以在特定图像块或图像块群中达成较高质量。 In an alternative embodiment, the compressor logic 404 allocates bits to achieve higher quality in a particular tile or group of tiles. 举例而言,其可提供较佳的总体感知外观,以在屏幕的中心具有比边缘处高的质量。 For example, it may provide a better overall perceptual appearance to the center of the screen than at the edges with a high quality.

[0179] 在一个实施例中,为了改良视频流的特定区域的分辨率,视频压缩逻辑404使用较小图像块来编码视频流的具有相对多的场景复杂度和/或运动的区域(与视频流的具有相对少的场景复杂度和/或运动的区域相比)。 [0179] In one embodiment, for improved resolution of certain regions of the video stream, the video compression logic 404 uses smaller tiles to region (video encoder video stream with relatively more scene complexity and / or motion compared flow region having relatively few scene complexity and / or motion). 举例而言,如图8中所说明,在一个R帧811 (可能随后跟着具有相同图像块大小的一系列R帧(未图示))的一个区域中的移动人物805的周围使用较小图像块。 For example, as illustrated in Figure 8, in one R frame 811 (potentially followed by a series of R frames with the same tile sizes (not shown)) in a region around a moving character 805 uses smaller Piece. 接着,当人物805移动至图像的新区域时,在另一R帧812 内的此新区域的周围使用较小图像块,如所说明。 Then, when the character 805 moves to a new area of ​​the image around this new area within another R frame 812 uses smaller tiles, as illustrated. 如上所述,各种不同大小及形状可用作“图像块”同时仍遵守所述基本原理。 As described above, various different sizes and shapes may be "tiles" while still complying with these underlying principles.

[0180] 尽管上文所描述的循环I/P图像块实质上减小视频流的数据速率中的峰值,但其并不完全消除峰值,尤其在快速改变或高度复杂的视频图像(诸如在电影、视频游戏及某一应用程序软件下出现)的状况下。 [0180] While the cyclic I described hereinabove / P tiles substantially reduce the peak data rate of the video stream, but it does not eliminate the peaks entirely, particularly in the rapidly-changing or highly complex video images (such as movies under occurs under certain video games and application software) situation. 举例而言,在突然场景转变期间,复杂帧可能随后跟着完全不同的另一复杂帧。 During example, a sudden change in the scene, complex frame may be followed completely different frame of another complex. 即使若干个I图像块可领先于场景转变仅几个帧时间,其在该情形下也没有帮助,因为新帧的材料与先前I图像块无关。 Even though several I tiles may have preceded the scene transition by only a few frame times, they do not help in this situation because the new frame regardless of the material of the previous I tiles. 在这种情形下(及在即使并非一切都改变,大量图像也改变的其他情形下),视频压缩器404将确定将许多(若并非所有)P 图像块更有效地写码为I图像块,且所导致的是所述帧的数据速率中的非常大的峰值。 In this case (and in the case even though not everything changes, much of the image changes in other cases), the video compressor 404 will determine that many (if not all) P tiles are more efficiently coded as I tiles, and the resulting data rate of the frame is very large peak.

[0181] 如先前所论述,其仅为对于大多数消费者级互联网连接(及许多办公室连接)的状况,其仅不可“堵塞”超过图6c中显示为622的可用最大数据速率以及额定最大数据速率621的数据。 [0181] As previously discussed, it is simply the most consumer-grade Internet connections (and many office connections), and it simply is not "jam" is displayed in Figure 6c exceed the available maximum data rate of 622 and a maximum rated data data rate 621. 注意,额定最大数据速率621 (例如,“6Mbps DSL”)实质上为对于考虑购买互联网连接的用户的销售数字,但通常其不保证性能水平。 Note that the rated maximum data rate 621 (eg, "6Mbps DSL") essentially for considering the purchase of the sales figures of the user's Internet connection, but usually it does not guarantee performance levels. 出于该应用的目的,其是不相关的,因为我们仅关注经由连接使视频流动时的可用最大数据速率622。 For the purposes of this application, it is not relevant, because we only focus on the available maximum data rate of the video via the connecting flow at 622. 因此,在图9a及图9c中,当描述对峰值问题的解决方法时,自曲线图省略额定最大数据速率,且仅显示可用最大数据速率922。 Thus, in Figures 9a and 9c, when we describe a solution to the peaking problem, from the graph rated maximum data rate is omitted, and only the available maximum data rate 922. 视频流数据速率不得超过可用最大数据速率922。 The video stream data rate must not exceed the maximum available data rate 922.

[0182] 为了解决此问题,视频压缩器404进行的第一件事是确定峰值数据速率941,其为信道能够稳定地处理的数据速率。 [0182] In order to solve this problem, the video compressor 404 is the first thing to determine the peak data rate 941, which is a data rate the channel is able to handle steadily. 该速率可通过许多技术来确定。 This rate can be determined by a number of techniques. 一种该技术是将越加变高的数据速率测试流自主机服务210逐渐发送至客户端415 (图4a及图4b中),且使客户端将关于分组丢失及延时的水平的反馈提供至主机服务。 One such technique is increasingly higher data rate test stream from the hosting service 210 is gradually transmitted to the client 415 (FIG. 4a and FIG. 4b), and having the client about the level of packet loss and latency provide feedback to the hosting service. 当分组丢失和/或延时开始显示尖锐增加时,其为达到可用最大数据速率922的指示。 As the packet loss and / or latency begins to show a sharp increase, that is the available maximum data rate indicator reaches 922. 之后,主机服务210可逐渐地减小测试流的数据速率,直至客户端415报告在合理的时间周期中已接收到测试流(分组丢失及延时的可接受水平接近最小)为止。 After that, the hosting service 210 may (packet loss and latency close to the minimum acceptable level) gradually reduce the data rate of the test stream until the client 415 reports the test stream has been received within a reasonable period of time so far. 这确定峰值最大数据速率941,其接着将用作用于流动视频的峰值数据速率。 This establishes a peak maximum data rate 941, which will then be used as a peak data rate for streaming video. 随着时间的推移,峰值数据速率941将波动(例如,若家庭中的另一用户开始严重地使用互联网连接),且客户端415将需要恒定地监视峰值数据速率941以观看分组丢失或延时是否增加(指示可用最大数据速率922下降至低于先前所确定的峰值数据速率941),且若如此,则峰值数据速率941。 Over time, the peak data rate 941 will fluctuate (e.g., if another user in a household starts to heavily use the Internet connection), and the client 415 will need to constantly monitor the peak data rate 941 to packet loss or latency view whether to increase (indicating the available max data rate 922 is dropping below the previously established peak data rate 941), and if so the peak data rate 941. 类似地,若随着时间的推移,客户端415发现分组丢失及延时保持在最佳水平,则其可请求视频压缩器缓慢地增加数据速率以观看可用最大数据速率是否增加(例如,若家庭中的另一用户已停止对互联网连接的严重使用),且再次等待直至分组丢失和/或较高延时指示已超过可用最大数据速率922为止,且可再次发现用于峰值数据速率941的较低水平,但该较低水平可能高于测试增加的数据速率之前的水平。 Similarly, if over time the client 415 finds that the packet loss and latency remain at optimal levels, it can request that the video compressor slowly increases the data rate to see whether the available maximum data rate has increased (e.g., if the family in another user has stopped heavy use of the Internet connection), and again waiting until packet loss and / or higher latency indicates that the available maximum data rate has been exceeded until the 922, and may be found again a lower level for the peak data rate 941 but lower levels may be higher than the level before testing an increased data rate. 因此,可通过使用该技术(及类似其的其他技术)而发现峰值数据速率941,且视需要而周期性地进行调整。 Thus, by using this technique (and other techniques like it) a peak data rate 941, and optionally periodically adjusted. 峰值数据速率941确定可由视频压缩器404使用以使视频流动至用户的最大数据速率。 Determining the peak data rate 941 by video compressor 404 to stream video to the user's maximum data rate. 用于确定峰值数据速率的逻辑可在用户场所211处和/或在主机服务210上加以实施。 It means for determining the peak data rate may be logical and / or on the hosting service 210 to be implemented at the user 211 places. 在用户场所211处,客户端设备415执行计算以确定峰值数据速率且将此信息传输回至主机服务210 ;在主机服务210处,主机服务处的服务器402执行计算以基于自客户端415所接收的统计数据(例如,峰值丢失、延时、最大数据速率等)而确定峰值数据速率。 At the user premises 211, the client device 415 performs calculations to determine the peak data rate and transmit this information back to the hosting service 210; at the hosting service 210, a server 402 at the hosting service performs the calculations based on the received from the client 415 statistical data (e.g., the peak loss, latency, maximum data rate, etc.) to determine the peak data rate.

[0183] 图9a显示具有实质场景复杂度和/或运动的实例视频流数据速率934,其是使用先前所描述且在图7a、图7b及图8中加以说明的循环I/P图像块压缩技术而产生。 [0183] Figure 9a shows an example video stream data rate has substantial scene complexity and / or motion 934, which is used as previously described, and in FIG. 7a, be cyclic I / P tiles described in Fig. 7b and Fig. 8, the compression technology and production. 视频压缩器404经配置而以低于峰值数据速率941的平均数据速率输出经压缩的视频,且注意, 大部分时间,视频流数据速率保持低于峰值数据速率941。 Video compressor 404 but below the peak data rate to an average data rate 941 configured to output compressed video, and note that, most of the time, the video stream data rate remains below the peak data rate 941. 数据速率934与图6c中所显示的视频流数据速率634 (其是使用I/P/B或I/P帧而产生)的比较显示循环I/P图像块压缩产生平滑得多的数据速率。 Comparative 634 (which is the use of I / P / B or I / P frames generated) data rate 934 with video stream data shown in FIG. 6c speed display cyclic I / P tile compression produces a much smoother data rate. 但在帧2倍峰值952 (其接近2倍峰值数据速率942)及帧4 倍峰值954 (其接近4倍峰值数据速率944)下,数据速率仍超过峰值数据速率941,这是不可接受的。 Still, at frame 2x peak 952 (which approaches 2x the peak data rate 942) and frame 4x peak 954 (which approaches 4x the peak data rate 944), the data rate exceeds the peak data rate 941, which is unacceptable. 在实践中,即使对于来自快速改变的视频游戏的高动作视频,超过峰值数据速率941的峰值也在小于2%的帧中出现,超过2倍峰值数据速率942的峰值很少出现,且超过3倍峰值数据速率943的峰值难得出现。 In practice, even with high peak action video from rapidly changing video games, exceeds the peak data rate 941 occur in less than 2% of frames, peaks in excess of 2x peak data rate 942 occur rarely, and more than 3 peak times of peak data rate 943 occur rarely. 但是,当其确实出现时(例如,在场景转变期间), 其所需的数据速率必须产生良好质量的视频图像。 However, when they do occur (e.g., during a scene transition), the required data rate must produce a good quality video image.

[0184] 解决此问题的一个方式是简单地配置视频压缩器404以使得其最大数据速率输出为峰值数据速率941。 One embodiment of [0184] to solve this problem is simply to configure the video compressor 404 such that its maximum data rate output is the peak data rate 941. 遗憾地,峰值帧期间的所得视频输出质量不良,因为压缩算法“极度缺乏”比特。 Unfortunately, the resulting video output quality during the peak frames is poor since the compression algorithm is "starved" bit. 所导致的为当存在突然转变或快速运动时出现压缩假影,且及时地,用户开始认识到:当存在突然改变或快速运动时假影总是突然出现,且其可变得相当讨厌。 The cause is when there is a sudden change in appearance or fast motion compression artifacts, and in a timely manner, users began to realize: When there is a sudden change or fast motion artifacts are always sudden, and it can become quite annoying.

[0185] 尽管人的视觉系统对在突然改变或快速运动期间出现的视觉假影相当敏感,但对在所述情形下侦测到帧速率的减小并不是非常敏感。 [0185] Although the human visual system is quite sensitive to visual artifacts that appear during sudden changes or rapid motion, but to detecting a reduction in frame rate in the case of not very sensitive. 事实上,当所述突然改变出现时,看来似乎人的视觉系统专注于追踪所述改变,且若帧速率暂时从60fps下降到30fps且接着立即返回到60fps,则人的视觉系统不会注意到。 In fact, when the sudden change in appearance, seems to focus on the human visual system to track the change, and if a temporary drop in the frame rate from 60fps to 30fps to 60fps and then immediately returns, the human visual system does not pay attention to. 此外,在非常急剧的转变(如突然场景改变) 的状况下,若帧速率下降到20fps或甚至15fps且接着立即返回到60fps,则人的视觉系统不会注意到。 Further, in the case of a very dramatic transition (e.g., a sudden scene change), and if the frame rate drops to 20fps or even 15fps, and then returns immediately to of 60 fps, the human visual system does not notice. 只要帧速率减小仅偶尔出现,对于人观察者而言,看来似乎视频以60fps不断地执行。 As long as the frame rate is reduced only occasionally, for a human observer, it seems to 60fps video continue to perform.

[0186] 通过图9b中所说明的技术来利用人的视觉系统的此特性。 [0186] exploit the characteristics of the human visual system by the technique illustrated in Figure 9b. 服务器402 (来自图4a 及图4b)以稳定帧速率(在一个实施例中,在60fps下)产生未经压缩的视频输出流。 Rate server 402 (from Figures 4a and 4b) to stabilize (In one embodiment, the at 60fps) produces an uncompressed video output stream. 时间线显示每个1/60秒每个帧961-970输出。 A timeline shows each frame 961-970 output each 1/60 seconds. 自帧961开始,将每个未经压缩的视频帧输出至低延时视频压缩器404,低延时视频压缩器404在小于一帧时间的时间中压缩所述帧,产生用于第一帧的经压缩的帧1981。 Since the start frame 961, each uncompressed video frame output to the low-latency video compressor 404, the low-latency video compressor 404 compresses the frame in less than a frame time, producing for the first frame the compressed frame 1981. 经产生用于经压缩的帧1981的数据可视如先前所描述的许多因素而较大或较小。 1981 data produced for the compressed frame, depending upon many factors, as previously described larger or smaller. 若数据足够小以致可以峰值数据速率941在一帧时间(1/60秒) 或小于一帧时间内将其传输至客户端415,则在传输时间(xmit时间)991(指示传输时间的持续时间的箭头的长度)期间将其传输。 If the data is small enough that it can peak data rate 941 in one frame time (1/60 sec) or less at a time it is transmitted to client 415, then 991 (indicating a transmission time duration of the transmission time (time XMIT) during the length of the arrow) to its transmission. 在下一个帧时间中,服务器402产生未经压缩的帧2962,将其压缩成经压缩的帧2982,且在小于一帧时间的传输时间992期间以峰值数据速率941将其传输至客户端415。 In the next frame time, server 402 produces uncompressed frame 2 962, it is compressed to compressed frame 2 982, and during the transmission time is less than a frame time at peak data rate 992, 941 be transmitted to the client 415.

[0187] 接着,在下一个帧时间中,服务器402产生未经压缩的帧3963。 [0187] Next, in the next frame time, server 402 produces uncompressed frame 3963. 当由视频压缩器404来压缩未经压缩的帧3963时,所得的经压缩的帧3983为比可以峰值数据速率941在一帧时间中传输的数据多的数据。 When the data 3963, the resulting compressed frame 3,983,941 transmitted by video compressor 404 to compress uncompressed frame than the peak data rate may be a multi-time data. 因此,在传输时间(2倍峰值)993期间将其传输,其占据所有帧时间及下一个帧时间的一部分。 Thus, the transmit time (2x peak) 993 during its transmission, which occupy a frame time and part of the next frame time. 现在,在下一个帧时间期间,服务器402产生另一未经压缩的帧4964且将其输出至视频压缩器404,但数据被忽略且通过974来说明。 Now, during the next frame time, server 402 produces another uncompressed frame 4 964 and outputs it to video compressor 404 but the data is ignored and illustrated by 974. 这是因为视频压缩器404经配置以忽略在其仍传输先前经压缩的帧时到达的其他未经压缩的视频帧。 This is because the video compressor 404 is configured to ignore its still transmit uncompressed video frames other previously reached by the compression of the frame. 当然,客户端415的视频解压缩器将未能接收到帧4,但其简单地继续在显示设备422 上显示帧3历时2个帧时间(也即,暂时将帧速率自60fps减小至30fps)。 Of course, the client 415 video decompressor will fail to receive frame 4, but it simply continues to display on display device 422 frame 3 for 2 frame times (i.e., briefly reduces the frame rate from 60fps to 30fps ).

[0188] 对于下一个帧5,服务器402输出未经压缩的帧5965,将其压缩成经压缩的帧5985 且在传输时间995期间在1帧内将其传输。 [0188] For the next frame 5, server 402 outputs uncompressed frame 5 965, it is compressed to compressed frame 5 985 and 995 during the transmission time of one frame in its transmission. 客户端415的视频解压缩器解压缩帧5并将其显示于显示设备422上。 Solutions of video client 415 the decompressor decompresses frame 5 and displays it on the display device 422. 接着,服务器402输出未经压缩的帧6966,视频压缩器404将其压缩成经压缩的帧6986,但此时所得的数据非常大。 Next, server 402 outputs uncompressed frame 6 966, video compressor 404 compresses it into compressed frame 6 986, but this time the resulting data is very large. 在传输时间(4倍峰值)996期间以峰值数据速率941传输经压缩的帧,但花费几乎4个帧时间来传输帧。 At the peak data rate 941 of the compressed frame is transmitted at the transmit time (4x peak) period 996, but it takes almost 4 frame times to transmit the frame. 在接下来的3个帧时间期间,视频压缩器404忽略来自服务器402的3个帧,且客户端415的解压缩器将帧6稳定地保持在显示设备422上历时4个帧时间(也即,暂时将帧速率自60fps减小至15fps)。 During the next 3 frame times, video compressor 404 ignores 3 frames from server 402, and client 415 decompressor holds frame 6 steadily maintained for 4 frames times on the display device 422 (i.e. temporarily reduce the frame rate from 60fps to 15fps). Meet

36着最后,服务器402输出帧10970,视频压缩器404将其压缩成经压缩的帧10987,且在传输时间997期间将其传输,且客户端415的解压缩器解压缩帧10并将其显示于显示设备422 上且再一次视频以60fps重新开始。 Finally the 36, the server 402 outputs frame 10 970, video compressor 404 compresses it into compressed frame 10 987, and during transmission a transmission time 997, the decompressor 415's and the client decompresses frame 10 and displays it on the display device 422 and once again the video resumes at 60fps.

[0189] 注意,尽管视频压缩器404丢弃了来自由服务器402产生的视频流的视频帧,但其不会丢弃音频数据(不管音频是以什么形式来的),且当丢弃视频帧时视频压缩器404继续压缩音频数据并将其传输至客户端415,客户端415继续解压缩音频数据并将音频提供至由用户使用以回放音频的无论什么设备。 [0189] Note that although video compressor 404 drops video frames of the video server 402 streams generated, it does not drop audio data (regardless of what form the audio come), and when the video frames are dropped Video Compression 404 continues to compressed audio data and transmit them to client 415, client 415 continues to decompress the audio data and provide the audio to play back audio using whatever device by the user. 因此在丢弃帧的周期期间,音频继续而不减弱。 Therefore, during a period when frames are dropped, audio continues unabated. 与经压缩的视频相比,经压缩的音频消耗相对小百分比的带宽,且因此不会对总数据速率有较大影响。 Compared with the compressed video, compressed audio consumes a relatively small percentage of bandwidth, and therefore does not have great influence on the overall data rate. 尽管在数据速率图中的任一者中都未说明,但峰值数据速率941内总是存在经保留用于经压缩音频流的数据速率容量。 Although any one of the data rate is not illustrated in the drawing, there is always data rate capacity reserved for after the compressed audio stream within the peak data rate 941.

[0190] 选择刚刚在图9b中所描述的实例来说明在数据速率峰值期间帧速率如何下降, 但未说明的是当使用先前所描述的循环I/P图像块技术时,所述数据速率峰值及随的发生的丢弃的帧很少,即使在高场景复杂度/高动作序列(诸如在视频游戏、电影及某个应用程序软件中出现的那些)期间也如此。 When [0190] The example just described in Figure 9b was chosen to illustrate how the frame rate drops during data rate peaks, but does not illustrate is that when the cyclic I previously described / P tile techniques, the peak data rate and dropped frames occur with very little, even in scenes of high complexity / high action sequences also do so (such as those appearing in video games, movies and software in an application) period. 因此,减小的帧速率罕有且暂时,且人的视觉系统不会侦测到它们。 Thus, the reduced frame rates are infrequent and brief, and the human visual system does not detect them.

[0191] 若将刚刚所描述的帧速率减小机制应用于图9a中所说明的视频流数据速率,则在图9c中说明所得的视频流数据速率。 [0191] If the frame rate reduction mechanism just described is applied to the video stream data rate illustrated in Figure 9a, the resulting video stream data rate is illustrated in Figure 9c. 在此实例中,2倍峰值952已减小至平坦化的2倍峰值953,且4倍峰值955已减小至平坦化的4倍峰值955,且整个视频流数据速率934保持处于或低于峰值数据速率941。 In this example, 2x peak 952 has been reduced to flattened 2x peak 953, and 4x peak 955 has been reduced to flattened 4x peak 955, and the entire video stream data rate 934 remains at or below peak data rate 941.

[0192] 因此,使用上文所描述的技术,可经由通用互联网及消费者级互联网连接而以低延时来传输高动作视频流。 [0192] Thus, using the techniques described above, may be connected through the general Internet and consumer-grade Internet with low latency to transmit high action video stream. 另外,在LAN(例如,IOOMbs以太网或802. Ilg无线网络)上或私用网络(例如,数据中心与办公室之间的IOOMbps连接)上的办公室环境中,可在无峰值情况下传输高动作视频流,以使得多个用户(例如,以4. 5Mbps传输60fps下的1920X 1080) 可使用LAN或共用私用数据连接,而不使重迭峰值淹没(overwhelm)网络或网络交换器底板。 Additionally, or private network (e.g., IOOMbps connection between a data center an offices) on a LAN (e.g., IOOMbs Ethernet or wireless network 802. Ilg) on ​​the office environment, high action can be transmitted without peaks in video stream, so that multiple users (e.g., in at 1920X 1080 4. 5Mbps transmission of 60 fps) can use the LAN or shared private data connection without having overlapping peaks overwhelming (overwhelm) network or the network switch backplanes.

[0193] 数据速率调整 [0193] DATA RATE ADJUSTMENT

[0194] 在一个实施例中,主机服务210最初评估信道的可用最大数据速率622及延时以确定用于视频流的适当数据速率且接着响应于此而动态地调整数据速率。 [0194] In one embodiment, the hosting service 210 initially assesses the available maximum channel data rate and delay 622 to determine an appropriate data rate of the video stream in response thereto and then dynamically adjusts the data rate. 为了调整数据速率,主机服务210可(例如)修改待发送至客户端415的视频流的图像分辨率和/或每秒帧数。 To adjust the data rate, the hosting service 210 may be (e.g.) sent to the client to modify the image resolution of the video stream and / or 415 frames per second. 而且,主机服务可调整经压缩视频的质量水平。 Moreover, the hosting service can adjust the level of quality compressed video. 当改变视频流的分辨率时(例如,自1280 X 720分辨率至640 X 360),客户端415上的视频解压缩逻辑412可将图像按比例增加以在显示屏幕上维持相同图像大小。 When changing the resolution of the video stream (e.g., from a resolution of 1280 X 720 to 640 X 360), the client 415 on the end of the video decompression logic 412 compressed images can be scaled to maintain the same image size on the display screen.

[0195] 在一个实施例中,在信道完全退出的情形下,主机服务210将游戏暂停。 [0195] In one embodiment, in a situation where the channel completely drops out, the hosting service 210 pauses the game. 在多人游戏的状况下,主机服务向其他用户报告该用户已退出游戏和/或将游戏暂停以用于其他用户。 In a multiplayer game, the hosting service to other users report that the user has to exit the game and / or suspend the game for other users.

[0196] 丢弃或延迟的分组 [0196] dropped or delayed packets

[0197] 在一个实施例中,若数据由于图4a或图4b中的视频压缩器404与客户端415之间的分组丢失而丢失,或由于到达得过晚以致不能解压缩及满足经解压缩帧的延时要求的分组被无次序地接收而丢失,则视频解压缩逻辑412能够减轻视觉假影。 [0197] In one embodiment, if the data of FIG. Since the video compressor 4a or 4b 404 in lost packets between the client 415 is lost, or because that arrives too late to decompress and meet the decompressed packet latency requirements frame is received out of order lost, the video decompression logic 412 is able to mitigate the visual artifacts. 在流动I/P帧实施中,若存在丢失/延迟的分组,则整个屏幕受影响,从而可能引起屏幕完全冻结一段时间周期或显示其他屏幕宽视觉假影。 In streaming I / P frame implementation, if there is lost / delayed packet, the entire screen is impacted, potentially causing the screen to completely freeze for a period of time or show other screen-wide visual artifacts. 举例而言,若丢失/延迟的分组引起I帧的丢失,则在接收新的I帧之前,解压缩器将缺乏用于跟随的所有P帧的参考。 For example, if a lost / delayed packet causes the loss of an I frame, prior to receiving the new I-frame, the decompressor will lack a reference for all of the P frames that follow. 若丢失P帧,则其将影响跟随的用于整个屏幕的P帧。 If you lose a P-frame, it will follow the impact of P frames for the entire screen. 视I帧出现之前有多久,这将具有较长或较短的视觉影响。 How long before I frame appears, it will have a longer or shorter visual impact. 使用如图7a及图7b中所显示的交错I/P图像块,丢失/延迟的分组不太可能影响整个屏幕,因为其仅影响受影响的分组中所含有的图像块。 7a and FIG using interleaved I / P tiles as shown in Figure 7b, a lost / delayed packet is much less likely to impact the entire screen since it will only affect the image block in the affected packet contained. 若每个图像块的数据是在个别分组内发送, 则若分组丢失,则其仅影响一个图像块。 If the data of each image block is sent within an individual packet, then if a packet is lost, it will only affect one tile. 当然,视觉假影的持续时间将取决于I图像块分组是否丢失及在P图像块丢失的情况下在I图像块出现之前将花费多少个帧。 Of course, the duration of the visual artifacts will depend on the I tiles packet is lost, and in the case of a P tile missing before I tile will cost how many frames appear. 但是,假定屏幕上的不同图像块是通过I帧非常频繁地(可能每个帧)更新,则即使屏幕上的一图像块受影响,其他图像块也可能不受影响。 However, different tiles on the screen are being updated with I frames very frequently (potentially every frame), even if one tile on the screen is affected, other tiles may not be affected. 另外,若某一事件引起若干分组同时丢失(例如,邻接DSL线的电力中的暂时中断数据流的尖峰信号),则一些图像块将比其他图像块受到更大影响,但因为一些图像块将通过新的I图像块迅速地更新,所以其仅暂时受影响。 Further, if some event cause loss of several packets at once (e.g., power next to a DSL line that briefly disrupts the data flow spikes), then some of the blocks than other tiles affected more, but because some tiles will by rapidly updating a new I tile, they will be only briefly affected. 而且, 在流动I/P帧实施的情况下,不仅I帧为最关键帧,而且I帧极大,因此若存在引起丢弃/ 延迟的分组的事件,则与小得多的I图像块相比,I帧受影响存在较高机率(也即,若I帧的任何部分丢失,则根本不可能可解压缩I帧)。 Further, in the case of streaming I / P frame implementation, not only are the I frames the most critical frame, but the I frames are extremely large, so if there is an event caused by the packets dropped / delayed, compared with a much smaller I tile , I frame affected by the presence a high probability (that is, any portion of the frame if I lost, it is impossible to be decompressed I-frame). 由于所有所述原因,与I/P帧的情况相比, 当分组被丢弃/延迟时,使用I/P图像块导致小得多的视觉假影。 For all of these reasons, as compared with the case of I / P frame when packets are dropped / delayed, using I / P tiles results in far fewer visual artifacts.

[0198] 一个实施例试图通过将经压缩的图像块智能地封装于TCP(传输控制协议)分组或UDP(用户数据报协议)分组内而减少丢失分组的效应。 [0198] One embodiment attempts to intelligently by the image block compressed encapsulated in TCP (transmission control protocol) packets or UDP (User Datagram Protocol) packets to reduce the effect of lost packets. 举例而言,在一个实施例中,只要可能,即将图像块与分组边界对准。 For example, in one embodiment, wherever possible, tiles are aligned with packet boundaries. 图IOa说明可如何在不实施此特征的情况下将图像块封装于一系列分组1001-1005内。 FIG IOa case illustrates how the features of this embodiment will not tiles packed within a series of packets 1001-1005. 具体言之,在图IOa中,图像块越过分组边界且经无效率地封装以致单一分组的丢失导致多个帧的丢失。 Specific, in the FIG IOa, tiles cross packet boundaries and are packed inefficiently so that the loss of a single packet results in the loss of multiple frames. 举例而言,若分组1003或1004丢失,则丢失三个图像块,导致视觉假影。 For example, if packets 1003 or 1004 are lost, three tiles are lost, resulting in visual artifacts.

[0199] 相比之下,图IOb说明用于将图像块智能地封装于分组内以减少分组丢失的效应的图像块封装逻辑1010。 [0199] In contrast, FIG. IOb described for intelligently packing tiles within packets to reduce the tile packing logic 1010 packet loss effect. 首先,图像块封装逻辑1010将图像块与分组边界对准。 First, the tile packing logic 1010 tiles are aligned with packet boundaries. 因此,图像块T1、T3、T4、T7及Τ2分别与分组1001-1005的边界对准。 Thus, tiles T1, T3, T4, T7 and Τ2 are aligned with the boundaries of packets 1001-1005. 图像块封装逻辑也试图以可能的更有效的方式将图像块组合于分组内,而不越过分组边界。 The tile packing logic also attempts to fit a more efficient manner possible tiles within packets, without crossing packet boundaries. 基于图像块中的每一者的大小,将图像块Tl与Τ6组合于一个分组1001中;将Τ3与Τ5组合于一个分组1002中;将图像块Τ4与Τ8组合于一个分组1003中;将图像块Τ8添加至分组1004 ;且将图像块Τ2添加至分组1005。 Based on the size of each of image blocks, tiles Tl and Τ6 combined in one packet 1001; with the Τ3 Τ5 combined in one packet 1002; tiles Τ4 with Τ8 combined in one packet 1003; image block Τ8 added to packet 1004; and adding to the packet 1005 Τ2 tiles. 因此,在此方案下,单个分组丢失将导致不多于2个图像块(而非如图IOa 中所说明的3个图像块)的丢失。 Thus, under this scheme, a single packet loss will result in no more than 2 tiles (not illustrated in FIG IOa tiles 3) is lost.

[0200] 图IOb中所显示的实施例的一个的外益处在于:图像块是以其在图像内被显示的不同次序进行传输。 [0200] outside of a benefit of the embodiment shown in FIG. IOb comprising: image block is displayed in a different order thereof in the image transmission. 若邻近分组由于干扰传输的同一事件(其将影响屏幕上彼此不接近的区域)而丢失,则此方式在显示器上产生较不引人注意的假影。 If the adjacent interference packet transmission because the same event (which will not affect the on-screen area close to each other) and lost, this way produce a less noticeable artifacts on display.

[0201] 一个实施例使用前向纠错(FEC)技术来保护视频流的特定部分以使其免受信道错误的影响。 [0201] A particular embodiment of the front portion to protect the video stream using a forward error correction (FEC) techniques embodied in from channel errors. 如此项技术中已知,诸如里德-所罗门及Viterbi (维特比)的FEC技术产生纠错数据信息并将其附加至经由通信信道而传输的数据。 As it is known in the art, such as Reed - Solomon and the Viterbi (Viterbi) techniques of FEC data and error correction data appended to the information transmitted via the communication channel. 若错误在基本数据(例如,I帧) 中出现,则FEC可用于校正该错误。 If an error occurs in the underlying data (e.g., I-frame), then the FEC may be used to correct the error.

[0202] FEC码增加传输的数据速率,因此理想地,其仅在最需要时使用。 [0202] FEC codes increase the data rate of the transmission, so ideally, they are only used where they are most needed. 若数据正被发送, 且其将不导致非常引人注意的视觉假影,则可较佳不使用FEC码来保护数据。 If data is being sent that would not result in a very noticeable visual artifact, it may be preferable to not use FEC codes to protect the data. 举例而言,紧接于丢失的I图像块之前的P图像块将仅在屏幕上产生1/60秒的视觉假影(也即,屏幕上的图像块将不被更新)。 For example, immediately following an I tile is lost before the P-picture blocks produced only 1/60 second visual artifact (i.e., on tile on the screen will not be updated) on the screen. 此种视觉假影几乎不能被人眼侦测到。 Such a visual artifact hardly detected by the human eye. 随着P图像块自I图像块进一步向后,丢失P图像块越加变得更引人注意。 As P tiles are further back from an I tile, losing a P tile becomes increasingly more noticeable. 举例而言,若图像块循环型样为在I图像块再次可用之前I图像块随后跟着15个P图像块,则若紧接于I图像块之后的P图像块丢失,则其导致该图像块显示不正确的图像历时15个帧时间(在60fps下,这将为250毫秒)。 For example, if a tile cycle pattern is an I tile followed by 15 P tiles, if the block immediately following an I tile is lost after a prior P I tile is available again, it causes the image block display an incorrect image for 15 frame times (at 60fps, which will be 250 ms). 人眼将容易侦测到250毫秒的流的中断。 The human eye will readily detect a disruption of the flow of 250 ms. 因此,P图像块距新的I图像块越向后(也即,P 图像块跟随I图像块越接近),则假影越引人注意。 Thus, P tile is from a new I tile rearwardly (i.e., P tiles follows an I tile closer), the more noticeable the artifact. 如先前所论述,尽管如此,但一般而言, P图像块跟随I图像块越接近,用于该P图像块的数据越小。 As previously discussed, though, in general, P tiles follows an I tile closer, for the image data of the block P is smaller. 因此,跟随I图像块的P图像块不仅对于保护以免丢失而言更关键,而且其大小较小。 Therefore, I follow the image block P tiles not only to protect from being lost even more critical, and its size is small. 此外,一般而言,需要保护的数据越小,保护其所需的FEC码越小。 Further, in general, the smaller the data to be protected, which protected the smaller the FEC code.

[0203] 因此,如图Ila中所说明,在一个实施例中,由于I图像块在视频流中的重要性,仅I图像块具备FEC码。 [0203] Thus, as illustrated in FIG Ila, in one embodiment, because of the importance of I tiles in the video stream, only I tiles are provided with FEC codes. 因此,FEC 1101含有用于I图像块1100的纠错码且FEC 1104含有用于I图像块1103的纠错码。 Thus, FEC 1101 contains error correction code for I tile 1100 and FEC 1104 contains error correction code for I tile 1103. 在此实施例中,对于P图像块不产生FEC。 In this embodiment, for the P tiles no FEC.

[0204] 在图lib中所说明的一个实施例中,对于在丢失时最可能引起视觉假影的P图像块也产生FEC码。 [0204] FIG. Lib, in one embodiment illustrated embodiment, if lost most likely to cause visual artifacts P tiles FEC codes are also generated. 在此实施例中,FEC 1105提供用于前3个P图像块但不用于跟随的P图像块的纠错码。 In this embodiment, FEC 1105 provide error correction codes for the first 3 P tiles, but not for the P tiles that follow. 在另一实施例中,对于数据大小最小的P图像块产生FEC码(其将倾向于自选在I图像块的后最早出现的P图像块,其对于保护最为关键)。 In another embodiment, FEC codes are generated for the data block size of the smallest P picture (which will tend to self- select P tiles following I tiles occurring the soonest, which is most critical to protect).

[0205] 在另一实施例中,并非将FEC码连同图像块一起发送,而是将图像块传输两次,每次在不同分组中传输。 [0205] embodiment, rather than sending an FEC code with a tile In another embodiment, instead of the image block is transmitted twice, each time in a different packet. 若一分组丢失/延迟,则使用另一分组。 If one packet is lost / delayed, the other packet is used.

[0206] 在图Ilc中所显示的一个实施例中,产生分别用于与视频同时自主机服务传输的音频分组1110及1112的FEC码1111及1113。 [0206] one embodiment, are generated for audio packets from the video hosting service simultaneously transmitted FEC codes 1110 and 1112, 1111 and 1113 are shown in FIG Ilc. 维持视频流中的音频的完整性特别重要,因为失真的音频(例如,滴答声或嘶嘶声)将导致特别不合需要的用户体验。 To maintain the integrity of the audio video stream is particularly important because the distortion of the audio (for example, clicking or hissing) will result in the user experience is particularly undesirable. FEC码帮助确保音频内容在客户端计算机415处无失真地再现。 FEC codes to help ensure distortion-free audio content 415 reproduced on the client computer.

[0207] 在另一实施例中,并非将FEC码连同音频数据一起发送,而是将音频数据传输两次,每次在不同分组中传输。 [0207] embodiment, rather than sending an FEC code with audio data, In another embodiment, the audio data is transmitted twice, each time in a different packet. 若一个分组丢失/延迟,则使用另一分组。 If one packet is lost / delayed, the other packet is used.

[0208] 另外,在图Ild中所说明的一个实施例中,FEC码1121及1123分别用于自客户端415上行传输到主机服务210的用户输入命令(例如,按钮按压)1120及1122。 [0208] Further, as explained in FIG Ild one embodiment, the FEC codes 1121 and 1123 are used for upstream transmission from the client 415 to the hosting service 210, the user input commands (e.g., button presses) 1120 and 1122. 这是重要的,因为在视频游戏或应用程序中漏掉按钮按压或鼠标运动可能导致不合需要的用户体验。 This is important, because missing in a video game or an application button presses or mouse movements may lead to undesirable user experience.

[0209] 在另一实施例中,并非将FEC码连同用户输入命令数据一起发送,而是将用户输入命令数据传输两次,每次在不同分组中传输。 [0209] embodiment, rather than sending an FEC code with user input command data in a further embodiment, but the user input command data is transmitted twice, each time in a different packet. 若一个分组丢失/延迟,则使用另一分组。 If one packet is lost / delayed, the other packet is used.

[0210] 在一个实施例中,主机服务210评估与客户端415的通信信道的质量,以确定是否使用FEC,且若使用,则确定应对视频、音频及用户命令的何部分应用FEC。 [0210] In one embodiment, the hosting service 210 assesses the quality of the communication channel with the client 415, to determine whether to use FEC, and if used, is determined FEC should be applied to deal with video, audio and user commands. 评估信道的“质量”可包括如上所述的诸如估计分组丢失、延时等的功能。 Evaluate the channel "quality" described above may include such as evaluating packet loss, latency, and other functions. 若信道特别不可靠,则主机服务210可对所有I图像块、P图像块、音频及用户命令应用FEC。 If the channel is particularly unreliable, then the hosting service 210 may apply FEC to all of I tiles, P tiles, audio and user commands. 相比之下,若信道可靠,则主机服务210可仅对音频及用户命令应用FEC,或可不对音频或视频应用FEC,或可根本不使用FEC。 In contrast, if the channel is reliable, then the hosting service 210 may apply only to the FEC audio and user commands, or may not use the FEC to audio or video, or may not be used at FEC. 可使用FEC的应用的各种其他排列,同时仍遵守所述基本原理。 Applications may use various other arrangements of the FEC, while still complying with these underlying principles. 在一个实施例中, 主机服务210不断地监视信道的状况且相应地改变FEC策略。 In one embodiment, the hosting service 210 continually monitors the conditions of the channel and changes the FEC policy accordingly.

[0211] 在另一实施例中,参看图4a及图4b,当分组丢失/延迟,从而导致图像块数据的丢失时,或若可能由于特别糟的分组丢失而使得FEC不能够校正丢失的图像块数据,客户端415评估在将接收新的I图像块之前剩余多少个帧且将其与自客户端415至主机服务210 的来回行程延时相比较。 [0211] In another embodiment, referring to Figures 4a and 4b, when a packet is lost / delayed resulting in the loss of tile data or if, perhaps because of a particularly bad packet loss, the FEC is unable to correct lost block data, the client 415 assesses how many frames are left before a new I picture received and to block 415 to hosting service 210 round-trip latency from the client is compared with. 若来回行程延时小于新的I图像块应到达之前的帧的数目,则客户端415向主机服务210发送消息,请求新的I图像块。 Trip latency is the number of frames should arrive before a new I tile is less than, then the client 415 to the host service 210 sends a message requesting a new I tile. 将此消息路由至视频压缩器404, 且其并非产生用于数据已丢失的图像块的P图像块,而是产生I图像块。 This message is routed to the video compressor 404, and rather than generating a P tile for the image data has been lost block, it generates an I tile. 假定图4a及图4b中所显示的系统经设计以提供通常小于80毫秒的来回行程延时,则这导致图像块被校正于80毫秒内(在60fps下,帧具有16. 67毫秒的持续时间,因此在全帧时间中,80毫秒延时将导致83. 33毫秒内的经校正的图像块,83. 33毫秒为5个帧时间,其为引人注意的中断,但远不及(例如)对于15个帧250毫秒中断引人注意)。 And FIG assumed system is designed as shown in FIG. 4b 4a to provide a round-trip latency is typically less than 80ms, this results in a tile being corrected within 80ms (at of 60 fps, the frame has a duration of 16.67 ms , thus in full frame times, 80ms latency would result in a corrected image block within 83.33 msec, 83.33 ms frame time is 5, which is a noticeable disruption, but far less (e.g.) for 15 ms frames 250 noticeable disruption). 当压缩器404脱离其通常的循环次序而产生此种I图像块时,若I图像块将引起所述帧的带宽超过可用带宽,则压缩器404将延迟其他图像块的循环,以使得其他图像块在所述帧时间期间接收P图像块(即使在所述帧期间一个图像块通常将应为I图像块),且接着通常的循环将自下一个帧开始继续, 且通常将已接收到先前帧中的I图像块的图像块将接收I图像块。 When the compressor 404 out of its usual cyclic order such an I tile is generated, if the I tile would cause the bandwidth of that frame to exceed the available bandwidth, then the compressor 404 will delay the cycles of the other tiles so that the other images blocks received during the frame time P tiles (one tile would normally be due an I tile during that frame even), and then the usual cycling will continue with the next frame starts, and normally would have received prior to the I tile image frame will receive an I tile. 尽管此动作暂时延迟R 帧循环的阶段,但其通常将在视觉上不引人注意。 Although this action briefly delays the phase of the R frame cycling, it normally will not be noticeable visually.

[0212] 视频及音频压缩器/解压缩器实施 [0212] Video and audio compressor / decompressor embodiments

[0213] 图12说明一个特定实施例,其中使用多核和/或多处理器1200来并行地压缩8个图像块。 [0213] Figure 12 illustrates one particular embodiment in which a multi- core and / or multi-processor 1200 in parallel to compress 8 tiles. 在一实施例中,使用在2. 66GHz或更高下执行的双核处理器、四核Xeon (至强)CPU 计算机系统,每个核心作为独立过程实施开源x264H. 264压缩器。 In an embodiment, using higher or 2. 66GHz performed under dual core processor, quad core Xeon (Xeon) CPU computer system, as an independent process of each core implementing the open source x264H. 264 compressor. 然而,可使用各种其他硬件/软件配置,同时仍遵守所述基本原理。 However, various other hardware / software configuration, while still complying with these underlying principles. 举例而言,CPU核心中的每一者可通过以FPGA实施的H. 264压缩器来替换。 For example, each of the CPU cores can be replaced by H. 264 compressor implemented in an FPGA. 在图12中所显示的实例中,核心1201-1208用于作为八个独立线绪来同时处理I图像块及P图像块。 In the example shown in Figure 12, cores 1201 -1208 are used as eight independent threads to concurrently process the I tiles and P tiles. 如此项技术中众所周知的,当前多核及多处理器计算机系统与诸如Microsoft Windows XP专业版(64比特版或者32比特版)及Linux的多线绪处理操作系统整合时,其固有地能够进行多线绪处理。 As is well known in the art when the current multi-core and multi-processor computer systems such as Microsoft Windows XP Professional Edition (64-bit or the 32-bit version) and Linux multi-threading operating system integration, which is inherently capable of multi-line thread processing.

[0214] 在图12中所说明的实施例中,因为该8个核心中的每一者仅负责一个图像块, 所以其很大程度上独立于其他核心而操作,每一者执行x264的单独实例化。 [0214] In the embodiment illustrated in FIG. 12, since each of the 8 cores is responsible for just one tile, it operates largely independently of the other cores, each running a separate x264 instantiated. 使用以PCI Express xl 为基础的DVI 捕获卡(诸如,来自Netherlands 的Microtronix of Oosterhout 的Sendero视频成像IP开发板)来捕获640X480、800X600或1280X720分辨率下的未经压缩的视频,且卡上的FPGA使用直接存储器存取(DMA)来将所捕获的视频经由DVI总线传送至系统RAM中。 PCI Express xl used based DVI capture card (such as, the Netherlands from Microtronix of Oosterhout the Sendero Video Imaging IP Development Board) to capture uncompressed video at 640X480,800X600 or 1280X720 resolution, and the FPGA on the card using a direct memory access (DMA) to transfer the captured video through the DVI bus into system RAM. 将所述图像块配置成4X2配置1205 (尽管其说明为方形图像块,但在该实施例中,其具有160 X 240分辨率)。 The tiles are arranged in 4X2 arrangement 1205 (although they are illustrated as square tiles, in this embodiment, which has a 160 X 240 resolution). x264的每个实例化被配置成压缩该8个160 X 240 图像块中的一者,且其经同步化以使得在初始I图像块压缩的后每一核心进入一循环,每一帧与另一帧不同相,以压缩一I图像块继的以七个P图像块,如图12中所说明。 Each instantiation of x264 is configured to compress the 8 160 X 240 image block one, and it was synchronized so that each of the core after the initial I tile compression enters a loop, each frame with another a different phase, to compress one I seven P tiles tile followed by, as illustrated in FIG. 12.

[0215] 在每一帧时间,使用先前所描述的技术将所得的经压缩图像块组合成分组流,且接着将经压缩图像块传输至目的地客户端415。 [0215] In each time frame, using the techniques previously described and the resulting compressed tiles are combined into a packet stream, and then transmitted to a destination client 415. compressed image block.

[0216] 尽管图12中未说明,但若组合的8个图像块的数据速率超过指定峰值数据速率941,则所有8个x264过程将暂时中止历时达必要的帧时间,直至已传输用于组合的8个图像块的数据为止。 [0216] Although not illustrated in FIG. 12, if the data rate of the combined 8 tiles exceeds a specified peak data rate 941, then all 8 x264 processes are suspended for as many frame times necessary until the transport has been used in the compositions eight data blocks up image.

[0217] 在一个实施例中,将客户端415实施为执行FFmpeg的8个实例化的PC上的软件。 [0217] In one embodiment, client 415 is implemented as eight FFmpeg instantiation of software on your PC. 接收过程接收8个图像块,且将每一图像块路由至FFmpeg实例化,FFmpeg实例化解压缩图像块并将其再现至显示设备422上的适当图像块位置。 A receiving process receives the 8 tiles, and each tile is routed to an FFmpeg instantiation, which decompresses the tile FFmpeg instantiation and renders it to an appropriate tile location on the display device 422.

[0218] 客户端415接收来自PC的输入设备驱动器的键盘、鼠标或游戏控制器输入并将其传输到服务器402。 [0218] The client 415 receives input from the device driver of the PC keyboard, mouse, or game controller input 402 and transfers it to the server. 服务器402接着应用所接收的输入设备数据并将其应用于在服务器402 上执行的游戏或应用程序,服务器402为使用Intel 2. 16GHz双核CPU执行Windows的PC。 Game or application server 402 then applies the received input device data and their application executing on server 402, server 402 using a dual-core CPU Intel 2. 16GHz running Windows your PC. 服务器402接着产生新帧并经由其DVI输出端将新帧自以主机板为基础的图形系统或者经由NVIDIA8800GTX PCI卡的DVI输出端输出。 Server 402 then produces a new frame and the new frame from a motherboard-based graphics system, or output via an output terminal DVI NVIDIA8800GTX PCI card via its DVI output.

[0219] 同时,服务器402经由其数字音频输出端(例如,S/PDIF)输出由游戏或应用程序产生的音频,该数字音频输出端耦合至实施视频压缩的以双四核Xeon为基础的PC上的数字音频输入端。 [0219] Simultaneously, the server 402 through its digital audio output (e.g., S / PDIF) audio output generated by the game or application, the audio output coupled to the digital to implement video compression in the dual quad-core Xeon-based PC on the digital audio input. Vorbis开源音频压缩器用于使用可用于处理线绪的无论什么核心来与视频同时地压缩音频。 A Vorbis open source audio compressor is used for the process thread using whatever core is compressed simultaneously with the video and audio. 在一个实施例中,完成压缩其图像块的核心首先执行音频压缩。 In one embodiment, the compressed core is completed tile first executes the audio compression. 接着将经压缩的音频连同经压缩的视频一起传输,并在客户端415上使用Vorbis音频解压缩器来解压缩经压缩的音频。 Then transfer the compressed audio along with the compressed video, and using the Vorbis audio decompressor to decompress the compressed audio on the client 415.

[0220] 主机服务服务器中心分配 [0220] server hosting service distribution center

[0221] 经由玻璃(诸如,光纤)的光以光在真空中的速度的某一部分行进,且因此可确定光在光纤中的确切传播速度。 [0221] In light of the speed of light in a certain part of the travel through a vacuum glass (such as optical fiber), and thus the light can be determined in an exact propagation speed of the optical fiber. 但是,在实践中,考虑用于路由延迟、传输无效率及其他耗用的时间,我们观察到互联网上的最佳延时反映较接近光速的50%的传输速度。 However, in practice, be considered for routing delays, inefficiencies and other overhead transmission time, we observed the best reflection of the delay on the Internet speeds closer to 50% of the speed of light. 因此,最佳1000英里来回行程延时为约22毫秒,且最佳3000英里来回行程延时为约64毫秒。 Thus, an optimal 1000 mile round trip latency is approximately 22ms, and an optimal 3000 mile round trip latency is about 64ms. 因此, 一美国海岸上的单个服务器将距离过远以致不能以所要的延时伺服另一海岸上的客户端(其可能达3000英里远)。 Thus, a single server on one US coast will be too far away can not clients on the other coast to the desired latency (which may be up to 3000 miles away). 然而,如图13a中所说明,若主机服务210服务器中心1300定位于美国的中心(例如,堪萨斯州、内布拉斯加州等),以致至美国大陆中的任何点的距离为约1500英里或1500英里以下,来回行程互联网延时可低至32毫秒。 However, as described in Figure 13a, if the hosting service 210 server center 1300 is located in the center of the US (e.g., Kansas, Nebraska, etc.), so that the distance to any point in the continental US is approximately 1500 miles or 1500 miles or less, the round trip Internet latency could be as low as 32 ms. 参看图4b,注意:尽管用户ISP 453所允许的最糟状况延时为25毫秒,但通常,在DSL及电缆调制解调器系统的情况下我们观察到较接近10-15毫秒的延时。 Referring to Figure 4b, note: User ISP 453, although the worst-case latencies allowed is 25 msec, but usually, in the case of DSL and cable modem systems we observed closer to 10- 15ms delay. 而且,图4b假定自用户场所211至主机代管中心210的最大距离为1000英里。 Also, Figure 4b assumes from the user premises 211 to the hosting center 210 of the maximum distance of 1000 miles. 因此,在所使用的典型的15毫秒的用户ISP来回行程延时及对于32毫秒的来回行程延时的1500英里的最大互联网距离的情况下,自用户致动输入设备421的时刻至在显示设备422上看见响应的总来回行程延时为1+1+15+32+1+16+6+8 =80毫秒。 Thus, in a typical user ISP 15 milliseconds is used for the round-trip latency and 1500 miles round trip delay of 32 ms and a maximum Internet distance from the user actuates input device 421 in time to the display device 422 sees a response on the total round trip delay of 15 + 1 + 1 + 32 + 1 + 16 + 6 + 8 = 80ms. 因此,通常可在1500英里的互联网距离上达成80毫秒响应时间。 Thus, typically 80 millisecond response time can be achieved over an Internet distance of 1500 miles. 这将允许美国大陆中具有足够短的用户ISP延时453的任何用户场所存取在中心定位的单个服务器中心。 This would allow any user continental US place a short enough user ISP latency 453 to access a single server center that is centrally located.

[0222] 在图13b中所说明的另一实施例中,主机服务210服务器中心HS1-HS6战略上定位于美国(或其他地理区域)的周围,特定较大的主机服务服务器中心接近高人口中心而定位(例如,HS2及HS5)。 [0222] surrounding embodiment, the hosting service 210 server centers HS1-HS6 strategic positioning in the United States (or other geographical region) is further illustrated in Figure 13b embodiment, with certain larger hosting service server centers positioned close to high population centers positioned (e.g., HS2 and HS5). 在一个实施例中,服务器中心HS1-HS6经由网络1301交换信息, 网络1301可为互联网或私用网络或两者的组合。 In one embodiment, the server centers HS1-HS6 exchange information via a network 1301, network 1301 may be the Internet or a private network or a combination of both. 在多个服务器中心的情况下,可以较低延时向具有高用户ISP延时453的用户提供服务。 In the case where a plurality of server centers, the delay 453 may be lower latency to users having a high user ISP services.

[0223] 尽管互联网上的距离的确为对经由互联网的来回行程延时有影响的因素,但有时很大程度上与延时无关的其他因素也起作用。 [0223] Although the distance on the Internet is certainly a factor in the delay to affect round trip via the Internet, but sometimes other factors largely unrelated to the delay also play a role. 有时经由互联网将分组流路由至距离远的位置且再次返回,从而导致来自长循环的延时。 Sometimes a packet via the Internet and back again, resulting in latency from the long loop flow path to a far away location. 有时在路径上存在不适当操作的路由设备,从而导致传输的延迟。 Sometimes there is routing equipment on the path that is not operating, resulting in a delay of the transmission. 有时存在使路径超载的通信,其引入延迟。 Sometimes there is a traffic overloading a path which introduces delay. 此外,有时,根本是存在防止用户的ISP路由至给定目的地的故障。 Also, sometimes, there is the presence of the user's ISP routes to prevent failure to a given destination. 因此,尽管通用互联网通常以相当可靠且最佳的 Thus, while the general Internet usually quite reliable and best

41路由及延时来提供从一点到另一点的连接,该相当可靠且最佳的路线及延时很大程度上是通过距离来确定(尤其是在导致路由至用户的本地区域的外部的长距离连接的情况下), 但该可靠性及延时得不到任何保证且常常不可自用户的场所至通用互联网上的给定目的地而达成。 Routing and delay 41 to provide a connection from one point to another, which is fairly reliable and optimal route and latency that is largely determined by distance (especially in the external route to lead the user's local area long when the distance of the connection), but the reliability and latency is by no means guaranteed and often can not given destination on the general Internet user's premises to be reached.

[0224] 在一个实施例中,当用户客户端415最初连接到主机服务210以玩视频游戏或使用应用程序时,客户端在启动时与可用的主机服务服务器中心HS1-HS6中的每一者通信(例如,使用上文所描述的技术)。 [0224] In one embodiment, when a user client 415 initially connects to the hosting service to play a video game or use an application, the client at the start of each of the available hosting service server centers HS1-HS6 210 communication (e.g., using the techniques described above). 若延时对于特定连接而言足够低,则使用该连接。 If the latency is low enough for a particular connection, then that connection. 在一个实施例中,客户端与所有主机服务服务器中心或主机服务服务器中心的一个子集通信, 选择具有最低延时连接的主机服务服务器中心。 In one embodiment, the client and a hosting service server centers to all sub-hosting service server center, or set of communication, selecting a hosting service server centers the lowest latency connection. 客户端可选择具有最低延时连接的服务中心,或服务器中心可识别具有最低延时连接的服务器中心并将此信息(例如,以互联网地址的形式)提供给客户端。 The client may select the service center with the lowest latency connection or the service centers may identify the one with the lowest latency connection and provide this information (e.g., in the form of an Internet address) to the client.

[0225] 若特定主机服务服务器中心超载和/或用户的游戏或应用程序可容忍至另一、较少载入的主机服务服务器中心的延时,则可将客户端415重定向至另一主机服务服务器中心。 [0225] If and / or delay the game or application the user can tolerate the latency to another, less loaded hosting service server center of a particular hosting service server center is overloaded, the client 415 may be redirected to the other host service server center. 在此种情形下,将使用户正执行的游戏或应用程序在用户的超载服务器中心处的服务器402上暂停,且将游戏或应用程序状态数据传送至另一主机服务服务器中心处的服务器402。 In this situation, the game or application the user is running on the server pauses at the center of users overloaded server 402, and transmits a game or application state data to the server 402 at the center of another server hosting services. 接着将重新开始该游戏或应用程序。 Then restart the game or application. 在一个实施例中,主机服务210将等待直至游戏或应用程序达到自然暂停点(例如,游戏中的级别之间,或者在用户在应用程序中起始“保存”操作之后)才进行传送。 In one embodiment, the hosting service 210 would wait until the game or application reached a natural pausing point (e.g., between levels in a game, or after the user in an application start "save" operation) to do the transfer. 在又一实施例中,主机服务210将等待直至用户活动停止历时指定时间周期(例如,1分钟)为止且接着将在这时起始传送。 In yet another embodiment, the hosting service 210 would wait until user activity ceases for a specified period of time (e.g., 1 minute) and then would initiate the transfer at that time.

[0226] 如上所述,在一个实施例中,主机服务210订阅图14的互联网旁路服务440以试图将得到保证的延时提供给其客户端。 [0226] As described above, in one embodiment, the hosting service 210 subscribes to an Internet bypass 14 of FIG service 440 to attempt to provide guaranteed latency to its clients. 如本文中所使用的互联网旁路服务是提供自互联网上的一点至另一点的具有得到保证的特性(例如,延时、数据速率等)的私用网络路线的服务。 Internet bypass services, as used herein, is a private network routes having characteristics (e.g., latency, data rate, etc.) to ensure the service point to another on the Internet to provide self. 举例而言,若主机服务210正使用在圣弗朗西斯科提供的AT&T的DSL服务接收来自用户的大量通信(而非路由至以AT&T的圣弗朗西斯科为基地的中央办公室),则主机服务210将在以圣弗朗西斯科为基地的中央办公室与用于主机服务210的服务器中心中的一或多者之间租用来自服务提供者(可能为AT&T本身或另一提供者)的高容量私用数据连接。 For example, if the hosting service 210 is using AT & T DSL service offering in San Francisco receiving large amount of traffic from a user (rather than routing to AT & T central office in the San Francisco-based), then the hosting service 210 will be San Francisco-based central office to the rental for hosting service from the service provider (perhaps aT & T itself or another provider) a high-capacity private data connection between the central server 210 in one or more of. 接着,若自所有主机服务服务器中心HS1-HS6经由通用互联网至圣弗朗西斯科中使用AT&T DSL的用户的路线导致过高延时,则可改为使用私用数据连接。 Then, if routes from all hosting service server centers HS1-HS6 user of AT & T DSL line via the Internet the most holy general Francisco resulted in too high latency, you can instead use private data connection. 尽管私用数据连接通常比经由通用互联网的路线更昂贵,但只要其保持主机服务210的一小百分比连接至用户,总成本影响就低,且用户将体验到更一贯的服务体验。 Although private data connections are typically more expensive than the route through the general Internet, but as long as it remains a small percentage of the hosting service 210 is connected to the user, the overall cost impact is low, and the user will experience a more consistent service experience.

[0227] 在电力故障的情况下,服务器中心常常具有两个备用电力层。 [0227] In the event of power failure, the server centers often have two layers of backup power. 第一层通常为来自电池(或来自替代的立即可用的能量源,诸如保持运转且附接至发电机的飞轮)的备用电力,其在电力干线出故障时立即提供电力且保持服务器中心运转。 The first layer is typically from a battery (or immediately available energy source from an alternative, such as a holding operation and is attached to a flywheel generator) backup power, which provides power immediately when the power mains fail and keeps the server center running. 若电力故障为暂时的,且电力干线迅速返回(例如,在一分钟内),则电池所需的是保持服务器中心运转。 If the power failure is brief, and the power mains return quickly (e.g., within a minute), then the batteries are required to keep the server center running. 但若电力故障历时较长的时间周期,则通常启动发电机(例如,柴油机供电)来取代电池且发电机只要具有燃料即可运转。 But if the power failure is for a longer period of time, then typically generators (e.g., diesel-powered) for the batteries and can run for as long as they have fuel. 所述发电机极昂贵,因为其必须能够产生多达服务器中心通常自电力干线所得到的电力。 Such generators are extremely expensive since they must be able to generate electric power as much as the server center normally obtained from the power mains.

[0228] 在一个实施例中,主机服务HS1-HS5中的每一者彼此共用用户数据,以便在一个服务器中心具有电力故障时,其可将在进行中的游戏及应用程序暂停,且接着将游戏或应用程序状态数据自每个服务器402传送至其他服务器中心处的服务器402,且接着将通知每一用户的客户端415以指导其传达至新的服务器402。 [0228] In one embodiment, each of the hosting services HS1-HS5 user data in common with each other, so that if one server center has a power failure, which may be in progress and a game application to pause and then game or application state data to the server 402 at other server centers from each server 402, and then will notify the client 415 of each user to direct it communications to the new server 402. 假定所述情形偶尔出现,则将用户转移至不能够提供最佳延时的主机服务服务器中心(也即,用户将仅必须容忍较高延时历时电力故障的持续时间)可为可接受的,其将允许用于转移用户的宽得多的范围的选项。 Assumed that such situations occur infrequently, it will not be able to transfer a user to provide optimal latency hosting service server centers (i.e., the user will simply have to tolerate higher latency for the duration of the power failure) may be acceptable, it will option allows the user to transfer a much wider range. 举例而言,给定跨越美国的时区差,则东海岸上的用户在11 : 30PM可能将要睡眠,而西海岸上的用户在8:30PM正开始在视频游戏使用上达到峰值。 For example, given the time zone differences across the United States, on the East Coast users in 11: 30PM might be going to sleep while users on the West Coast are starting to peak in video game usage at 8:30 PM. 若那时西海岸上的主机服务服务器中心中存在电力故障,则其他主机服务服务器中心处可能不存在用于处理所有用户的足够的西海岸服务器402。 If there is a power failure hosting service server center on the West Coast in time, the other hosting service server may not exist at the center of the user's enough to handle all of the West Coast server 402. 在此种情形下,可将一些用户转移至东海岸上具有可用服务器402 的主机服务服务器中心,且对于用户而言的唯一后果将是较高延时。 In such cases, some users may be transferred to the East Coast with an available server hosting service server center 402, and for the user of the only consequence would be higher latency. 一旦将用户自失去电力的服务器中心转移,服务器中心接着就可开始其服务器及设备的有序切断,以便在电池(或其他立即电力备用)耗尽之前切断所有设备。 Once the user server center loses power transfer, the server center can then commence an orderly cutting device and the server, so that all the cutting devices before the batteries (or other immediate power backup) is exhausted. 以此方式,可避免用于服务器中心的发电机的成本。 In this way, you can avoid the cost of a generator of the center server.

[0229] 在一个实施例中,在主机服务210的严重载入的时间期间(或者由于峰值用户载入,或者因为一个或多个服务器中心出故障),基于用户正使用的游戏或应用程序的延时要求将用户转移至其他服务器中心。 Severe loading time period [0229] In one embodiment, the hosting service 210 (either due to peak user loading, or because one or more server centers have failed), based on the game or application the user is using the delay required to transfer the user to another server center. 因此,将为使用需要低延时的游戏或应用程序的用户给出对存在有限供应的可用低延时服务器连接的优选。 Thus, low latency will need to use the game or application is given preference to available low latency server connections there is a limited supply.

[0230] 主机服务特征 [0230] hosting features

[0231] 图15说明在以下特征描述中利用的用于主机服务210的服务器中心的组件的实施例。 [0231] FIG. 15 illustrates the use of features described in the following embodiment for the host service component 210 server center. 如同图2a中所说明的主机服务210 —样,除非另外有条件,否则此服务器中心的组件由主机服务210控制系统401来控制及协调。 As illustrated in Figure 2a host service 210-- like, unless otherwise qualified, the components of this server center by the host service 401 control system 210 are controlled and coordinated.

[0232] 将来自用户客户端415的入埠互联网通信1501指引至入埠路由1502。 Inbound internet traffic 1501 [0232] from the user client 415 to the inbound routing 1502. 通常,入埠互联网通信1501将经由至互联网的高速光纤连接而进入服务器中心,但具有足够带宽、可靠性及低延时的任何网络连接装置将是足够的。 Typically, inbound internet traffic 1501 will enter the server center connected to the Internet via a high-speed optical fiber, but of adequate bandwidth, reliability and low latency any network connection means would be sufficient. 入埠路由1502是网络(该网络可实施为以太网、光纤信道网络,或经由任何其他输送装置)交换器及支持所述交换器的路由服务器的系统,其取得到达的分组且将每个分组路由到适当应用程序/游戏服务器1521-1525。 Inbound routing 1502 is a network (which may be implemented as an Ethernet network, a fiber channel network, or through any other transport means) system switches and routing servers supporting the switches which takes the arriving packets and each packet routed to the appropriate application / game server 1521-1525. 在一实施例中,传送至特定应用程序/游戏服务器的分组表示自客户端所接收的数据的一子集和/或可由数据中心内的其他组件(例如,网络连接组件,诸如网关及路由器)来转译/改变。 In one embodiment, the transfer to the packet a particular app / game server represents a subset from the client received data and / or other components within the data center (e.g., networking components such as gateways and routers) be translated / changed. 在一些状况下,(例如)若游戏或应用程序同时并行地在多个服务器上执行,则每次将分组路由至一个以上服务器1521-1525。 In some cases, (for example) if the game or application simultaneously performed in parallel on multiple servers, each time the packet is routed to more than one server 1521 -1525. RAID阵列1511-1512连接至入埠路由网络1502,以使得应用程序/游戏服务器1521-1525可读取RAID阵列1511-1512及写入RAID 阵列1511-1512。 RAID array 1511-1512 are connected to the inbound routing network 1502, such that the app / game servers 1521 -1525 can read and write RAID array 1511-1512 RAID array 1511-1512. 另外,RAID阵列1515 (其可实施为多个RAID阵列)也连接至入埠路由1502,且来自RAID阵列1515的数据可自应用程序/游戏服务器1521-1525来读取。 Further, a RAID array 1515 (which may be implemented as multiple RAID arrays) is also connected to the inbound routing 1502 and data from RAID array 1515 can be read from app / game servers 1521 -1525. 入埠路由1502可在多种先前技术网络架构(包括树结构的交换器,入埠互联网通信1501在其根部)中实施;在互连所有各种设备的网状结构中实施;或作为互连的子网络序列(互通设备当中的集中通信与其他设备当中的集中通信隔离)来实施。 The inbound routing 1502 may be a variety of prior art network architectures (including the tree structure of switches, with the inbound internet traffic 1501 at its root) in the embodiment; In mesh structure interconnecting all of the various devices; or as an interconnect sub-network sequence (isolated among the centralized communication with other devices among the centralized communication interworking device) FIG. 一类型的网络配置为SAN(存储区域网络),其尽管通常用于储存设备,但其也可用于设备之间的通用高速数据传送。 Type of network configuration is a SAN (storage area network), although typically used for storage devices, it can also be used for general high-speed data transfer among devices. 又, 应用程序/游戏服务器1521-1525可各自具有至入埠路由1502的多个网络连接。 Also, the app / game servers 1521 -1525 may each have multiple network to the inbound routing 1502 is connected. 举例而言,服务器1521-1525可具有至附接至RAID阵列1511-1512的子网络的网络连接及至附接至其他设备的子网络的另一网络连接。 For example, a server 1521 -1525 may have a network attached to RAID Arrays 1511-1512 network connection up device attached to the other sub-network connected to another network. [0233] 应用程序/游戏服务器1521-1525可经相同地、有些不同地或全部不同地来配置, 如先前关于图4a中所说明的实施例中的服务器402所描述的。 [0233] The app / game servers 1521 -1525 may be the same, some differently, or all differently configured, embodiments as previously described in Figure 4a in the server 402 described herein. 在一实施例中,每一用户当使用主机服务时通常为至少一应用程序/游戏服务器1521-1525。 In one embodiment, each user, when using the hosting service is typically at least one app / game server 1521 -1525. 出于说明的简单起见,将假定给定用户正使用应用程序/游戏服务器1521,但多个服务器可由一用户使用,且多个用户可共用单一应用程序/游戏服务器1521-1525。 For simplicity of explanation, I shall assume a given user is using app / game server 1521, but multiple servers could be used by one user, and multiple users could share a single app / game server 1521 -1525. 自客户端415 (如先前所描述)所发送的用户的控制输入经接收为入埠互联网通信1501,且经由入埠路由1502而路由至应用程序/游戏服务器1521。 (As previously described) from the client 415 to control the transmitted user input is received as inbound Internet traffic 1501, and the inbound routing 1502 to app / game server 1521 via. 应用程序/游戏服务器1521使用用户的控制输入作为至在服务器上执行的游戏或应用程序的控制输入,且计算视频及与其相关联的音频的下一个帧。 App / game server 1521 uses the user's control input as control input to the game or application is executing on the server, and computes the next frame of video and the audio associated therewith. 应用程序/游戏服务器1521接着将未经压缩的视频/音频1529输出至共用视频压缩1530。 App / game server 1521 then outputs the uncompressed video / audio 1529 to shared video compression 1530. 应用程序/游戏服务器可经由任何装置(包括一或多个超高速以太网连接)而输出未经压缩的视频,但在一实施例中,视频是经由DVI (交互式数字视频系统)连接而输出,且音频及其他压缩及通信信道状态信息系经由通用串列总线(USB)连接而输出。 App / game server via any means (including one or more Gigabit Ethernet connections) output the uncompressed video, in one embodiment, the video is connected via a DVI (Digital Video Interactive Systems) outputs embodiment and the audio and other compression and communication channel state information is output and outputs a universal serial bus (USB) connection via.

[0234] 共用视频压缩1530压缩来自应用程序/游戏服务器1521-1525的未经压缩的视频及音频。 [0234] The shared video compression 1530 compresses / game server 1521 -1525 from the application of non-compressed video and audio. 该压缩可完全以硬件或以执行软件的硬件来实施。 The compression maybe implemented entirely in hardware, or in hardware running software implemented. 可存在用于每个应用程序/ 游戏服务器1521-1525的专用压缩器,或若压缩器足够快,则可使用给定压缩器来压缩来自一个以上应用程序/游戏服务器1521-1525的视频/音频。 May be present for each app / game server 1521 -1525 is dedicated compressor, or if the compressors are fast enough, a given compressor can be used to compress / from one or more game server 1521 -1525 application video / audio . 举例而言,在60fps下,视频帧时间为16. 67毫秒。 For example, in the case of 60 fps, the video frame time is 16.67 milliseconds. 若压缩器能够在1毫秒内压缩1帧,则该压缩器可用于通过取得来自一个接一个的服务器的输入而压缩来自多达16个应用程序/游戏服务器1521-1525的视频/音频,该压缩器保存每个视频/音频压缩过程的状态且当其在来自服务器的视频/ 音频流当中循环时切换背景。 If a compressor is able to compress a frame in 1 ms, then that compressor could be used by taking input from one server connected to a compressing / game server applications from 1521-1525 up to 16 video / audio, the compression saves each video / audio compression process and switching context as it cycles from the server in the video / audio streams. 这导致压缩硬件的实质成本节省。 This results in substantial cost savings in hardware compression. 因为不同服务器将在不同时间完成帧,所以在一个实施例中,压缩器资源是处于具有用于储存每个压缩过程的状态的共用储存装置(例如,RAM,闪存)的共用集区1530中,且当服务器1521-1525帧完整且准备被压缩时,控制装置确定那时哪个压缩资源可用,为该压缩资源提供服务器的压缩过程的状态及待压缩的未经压缩的视频/音频的帧。 Since different servers will be completing frames at different times, in one embodiment, the compressor resources are in a medium having shared storage means (e.g., RAM, Flash) state of each compression process for storing a shared pool 1530, and when a server 1521 -1525 frame is complete and ready to be compressed, a control means determines which compression resource is available, the server provides status of the compression process and be compressed uncompressed video / audio compression resource frames.

[0235] 注意,用于每个服务器的压缩过程的状态的一部分包括关于压缩本身的信息,诸如先前帧的经解压缩的帧缓冲数据(其可用作用于P图像块的参考)、视频输出的分辨率; 压缩的质量;图像块结构;每图像块的比特的分配;压缩质量、音频格式(例如,立体声、环绕音效、Dolby ® AC-3 (杜比® ) AC-3)。 [0235] Note that the compression process state for each server includes information about the compression part itself, such as the decompressed frame buffer data (which may be used as a reference for P tiles) of a previous frame, the video output resolution; compressed mass; image block structure; bits allocated per image block; compression quality, the audio format (e.g., stereo, surround sound, Dolby ® AC-3 (Dolby ®) AC-3). 但是压缩过程状态也包括关于以下的通信信道状态信息:峰值数据速率941,及先前帧(如图9b中所说明)当前是否正被输出(且因此应忽略当前帧),及潜在地是否存在应在压缩中考虑的(诸如,过多分组丢失)影响压缩决策(例如,在I图像块的频率方面,等)的信道特性。 But the compression process state also includes the following communication channel state information: the peak data rate 941 and whether a previous frame (as illustrated in Fig 9b) is currently being output (and thus should ignore the current frame), and potentially whether there should be (such as excessive packet loss) affect decisions for the compression (e.g., in terms of frequency of I tiles, etc) channel characteristics considered in compression. 因为峰值数据速率941或其他信道特性随着时间而改变,如由支持每个用户监视从客户端415发送的数据的应用程序/游戏服务器1521-1525所确定的,所以应用程序/游戏服务器1521-1525将相关信息发送至共用硬件压缩1530。 As the peak data rate 941 or other channel characteristics change over time, such as by the application supporting each user monitoring data sent from the client 415 / game server 1521 -1525 determined, the app / game server 1521- 1525 transmits the relevant information to the shared hardware compression 1530.

[0236] 共用硬件压缩1530也使用诸如先前所描述的所述装置的装置将经压缩的视频/ 音频分组化,且在适当时,应用FEC码,复制特定数据,或采取其他步骤,以便充分地确保视频/音频数据流由客户端415接收且以可行的高质量及可靠性解压缩的能力。 [0236] The shared hardware compression 1530 also using a device such as the previously described device packetizes the compressed video / audio, and if appropriate, applying FEC codes, duplicating certain data, or taking other steps to adequately Make sure the video / audio data stream and the ability to receive high quality and reliability as feasible decompressed by the client 415.

[0237] —些应用程序(诸如,下文所描述的应用程序)需要给定应用程序/游戏服务器1521-1525的视频/音频输出同时在多个分辨率下(或以其他多个格式)可用。 [0237] - Some applications (such as the application described below) required for a given app / game server 1521 -1525 video / audio output simultaneously at multiple resolutions (or in other multiple formats) simultaneously. 若应用程序/游戏服务器1521-1525如此通知共用硬件压缩1530资源,则该应用程序/游戏服务器1521-1525的未经压缩的视频音频1529将被以不同格式、不同分辨率和/或在不同分组/纠错结构中同时压缩。 If the app / game server 1521 -1525 so notifies the shared hardware compression 1530 resource, then the app / game server 1521 -1525 uncompressed video audio 1529 is in different formats, different resolutions, and / or in a different packet / error correction structures while compressing. 在一些状况下,一些压缩资源可在压缩同一视频/音频的多个压缩过程当中共用(例如,在许多压缩算法中,存在借以在应用压缩的前将图像按比例调整成多个大小的步骤。若需要输出不同大小的图像,则该步骤可用于同时服务若干个压缩过程)。 In some cases, some compression resources can be compressed in a plurality of the same video / audio compression in the process of sharing (e.g., in many compression algorithms, there is a step whereby before applying compression image into a plurality of adjusting the size of the scale. If desired output images of different sizes, the step can be used to serve several compression processes at once). 在其他状况下,对于每个格式将需要单独的压缩资源。 In other cases, for each compression format will require a separate resource. 在任何状况下,将用于给定应用程序/游戏服务器1521-1525 ( 一或多个)所需的所有各种分辨率及格式的经压缩的视频/音频1539同时输出到出埠路由1540。 In any case, for a given app / game server 1521 -1525 (one or more) video compressed in all of the various resolutions and formats required / audio output 1539 while the outbound routing 1540. 在一个实施例中,经压缩的视频/音频1539的输出是处于UDP格式,因此其为单向分组流。 In one embodiment, the output of the compressed video / audio 1539 is in UDP format, so it is a unidirectional stream of packets.

[0238] 出埠路由网络1540包含一系列路由服务器及交换器,该系列路由服务器及交换器将每个经压缩的视频/音频流经由出埠互联网通信1599接口(其通常将连接到至互联网的光纤接口)而指引到有意的用户或其他目的地和/或返回至延迟缓冲器1515,和/或返回至入埠路由1502,和/或经由私用网络(未图示)而输出以供进行视频分配。 [0238] The outbound routing network 1540 comprises a series of routing servers and switches in the series of routing servers and switches which direct each compressed video / audio stream outbound Internet traffic 1599 interface (which typically would connect to the Internet fiber optic interface) directed to the intended user or other destination and / or back to the delay buffer 1515, and / or back to the inbound routing 1502, and / or via a private network (not shown) for output for video distribution. 注意(如下所述):出埠路由1540可将给定视频/音频流同时输出至多个目的地。 Note that (as described below): the outbound routing 1540 may be given video / audio stream to multiple output destinations. 在一实施例中, 这是使用互联网协议(IP)多播来实施,其中广播意欲同时流动到多个目的地的给定UDP 流,且该广播由出埠路由1540中的路由服务器及交换器来重复。 In one embodiment, this is implemented using Internet Protocol (IP) multicast in which the broadcast is intended to flow simultaneously to a plurality of destinations for a given UDP stream, and the outbound routing 1540 is broadcast by the routing servers and switches repeat. 广播的该多个目的地可经由互联网而至多个用户的客户端415、经由入埠路由1502而到多个应用程序/游戏服务器1521-1525,和/或到一个或多个延迟缓冲器1515。 The multiple destinations of the broadcast may be to multiple users via the Internet terminal 415, via inbound routing 1502 to multiple app / game servers 1521 -1525, and / or to one or more delay buffers 1515.. 因此,将给定服务器1521-1522的输出压缩成一个或多个格式,且将每个经压缩的流指引到一个或多个目的地。 Thus, the output of a given server 1521-1522 is compressed into one or multiple formats, and each compressed stream is directed to one or multiple destinations.

[0239] 另外,在另一实施例中,若多个应用程序/游戏服务器1521-1525同时由一个用户使用(例如,在用于产生具有复杂场景的3D输出的并行处理配置中)且每个服务器产生所得图像的部分,则可由共用硬件压缩1530将多个服务器1521-1525的视频输出组合成一组合帧,且自该点向前如上所述处理该组合帧,好像其来自单个应用程序/游戏服务器1521-1525。 [0239] Further, in another embodiment, if multiple app / game servers 1521-1525 are used simultaneously by one user (e.g., in a parallel processing configuration to create the 3D output of a complex scene) and each the resulting image generating part of the server, by the shared hardware compression 1530 may be a combination of multiple servers 1521 -1525 video output into a combined frame, and from that point forward it is handled as described above, if it came from a single app / game server 1521-1525.

[0240] 注意,在一个实施例中,将由应用程序/游戏服务器1521-1525产生的所有视频的复本(至少以由用户观看的视频的分辨率或更高分辨率)记录于延迟缓冲器1515中历时至少某一数目的分钟(在一个实施例中为15分钟)。 [0240] Note that, a copy of all of the videos in the embodiment, by the app / game server 1521 -1525 generated (at least the resolution or higher of video viewed by the user) recorded in delay buffer 1515 for at least some number of minutes (in one embodiment 15 minutes). 这允许每个用户“回放”来自每个会话的视频,以便核查先前工作或业绩(在游戏的状况下)。 This allows each user to "replay" video from each session in order to review previous work or exploits (in the case of games). 因此,在一个实施例中,将路由至用户客户端415的每个经压缩视频/音频输出1539流也多播至延迟缓冲器1515。 Thus, in one embodiment, it is routed to a user client 415 through each compressed video / audio output 1539 stream also being multicasted to a delay buffer 1515. 当将视频/音频储存于延迟缓冲器1515上时,延迟缓冲器1515上的目录提供应用程序/游戏服务器1521-1525 (其为延迟的视频/音频的来源)的网络地址与延迟缓冲器1515上可发现延迟的视频/音频的位置之间的交叉参考。 When the video / audio is stored on a delay buffer 1515, a directory on the delay buffer 1515 provides app / game server 1521 -1525 (which is the delayed video / audio source) of the network address of the delay buffer 1515 a cross reference between the delayed video / audio position.

[0241 ] 现场直播的、可即刻观看的、可即刻播放的游戏 [0241] live, can instantly view, you can instantly play games

[0242] 应用程序/游戏服务器1521-1525不仅可用于执行用户的给定应用程序或视频游戏,而且其可用于建立用于支持经由主机服务210的导航及其他特征的主机服务210的用户接口应用程序。 [0242] app / game servers 1521-1525 not only for the user to perform a given application or video game, and it can be used to build support for a host via a host service navigation and other features of the user interface 210 of the service application 210 program. 一种该用户接口应用程序的屏幕拍摄显示在图16中(“游戏取景器(Game Finder) ”屏幕)。 One such user interface application display screen shooting ( "Game Finder (Game Finder)" screen) in FIG. 16. 该特定用户接口屏幕允许用户观看由其他用户现场玩的(或延迟的)15个游戏。 The specific user interface screen allows users to watch live play by other users (or delay) 15 games. “缩略图”视频窗中的每一者(诸如,1600)为在运动中的现场直播的视频窗,其显示来自一个用户的游戏的一个视频。 "Thumbnail" of each of the video window (such as 1600) for the live video window in motion showing a video from a user's game. 缩略图中所显示的视图可为用户正看的同一视图,或其可为延迟的视图(例如,若用户正玩搏斗游戏,则用户可能不希望其他用户看见其隐藏在哪里且其可选择将其游戏播放的任何视图延迟一时间周期(例如,10分钟))。 The same view in the thumbnail view may be displayed as the user is seeing, or it may be a delayed view (e.g., if a user is playing a combat game, a user may not want other users to see where and she may choose to hide the any delay in view of a game player for a time period (e.g., 10 min)). 视图也可为不同于任何用户的视图的游戏的相机视图。 Views can also be different from the game's camera view any user's view. 通过菜单选择(此说明中未图示),用户可基于多种标准来选择要同时观看的游戏的选择。 By menu selection (not shown in this illustration), a user may be selected based on various criteria to select the game while watching. 作为例示性选择的小取样,用户可选择游戏的随机选择(诸如图16中所显示的游戏)、所有一个类别的游戏(均由不同玩家来玩)、仅游戏的顶级玩家、游戏中的给定级别的玩家,或较低级玩家(例如,若玩家正学习基础)、是“搭档”(或为竞争者)的玩家、具有最多数目观看者的游戏等。 As an exemplary selection of a small sample, the user may select a random selection of games (such as the game shown in FIG. 16), all of one kind of games (played by different players), only the top of the game players to the game given the level of the player, or a lower-level players (for example, if a player is learning the basics), is a "partner" (or as competitors) players, the game has the largest number of viewers and so on.

[0243] 注意,通常,每个用户将决定来自其游戏或应用程序的视频是否可由他人观看,且若如此,则决定该视频可由哪些他人观看及何时可由他人观看,决定该视频是否只可在具有延迟的情况下观看。 [0243] Note that in general, each user will decide whether video games or from its application by watching others, and if so, by what others decide which video to watch and when to watch others may decide whether or not the video only Watch in the case of a delay.

[0244] 产生图16中所显示的用户接口屏幕的应用程序/游戏服务器1521-1525通过向每个用户(该应用程序/游戏服务器1521-1525正请求来自该用户的游戏)的应用程序/游戏服务器1521-1525发送消息而获取该15个视频/音频馈送。 Application [0244] Application user interface screen is generated as shown in FIG. 16 / game server 1521 -1525 (the app / game game server 1521 -1525 is requesting from the user) to each user by / game 1521-1525 message server acquires the 15 video / audio feeds. 该消息经由入埠路由1502或另一网络来发送。 The message is sent through the inbound routing 1502 or another network. 该消息将包括被请求的视频/音频的大小及格式,且将识别观看用户接口屏幕的用户。 The message will include the requested video / audio format and size, and will identify the user viewing the user interface screen. 给定用户可选择选择“盗版”模式且不准许任何其他用户观看其游戏的视频/音频(自其观看点或者自另一观看点),或如先前的段中所描述,用户可选择允许观看来自其游戏的视频/音频,但延迟所观看的视频/音频。 A given user may select "privacy" mode and not permit any other users to view video / audio (either from his point of view or from another point of view), or as described in the previous paragraph, the user may choose to allow viewing video / audio from its games, but delayed watched video / audio. 用户应用程序/游戏服务器1521-1525 (其接收并接受允许其视频/音频被观看的请求)将因此向请求服务器确认, 且其也将通知共用硬件压缩1530需要产生被请求格式或屏幕大小(假定格式及屏幕大小不同于已经产生的格式及屏幕大小)的额外经压缩视频流,且其也将指示经压缩视频的目的地(也即,请求服务器)。 User app / game server 1521 -1525 (receiving and accepting a request to allow its video / audio to be viewed) will acknowledge as such to the requesting server, and it will also notify the shared hardware compression 1530 of the need to generate the requested format or screen size (assuming format and screen size is different from the format that has been produced and screen size) of the additional compressed video stream, and it will also indicate the destination for the compressed video (i.e., the requesting server). 若被请求的视频/音频仅被延迟,则请求应用程序/游戏服务器1521-1525将被这样通知,且其将通过查找延迟缓冲器1515上的目录中的视频/音频的位置及为延迟的视频/音频的来源的应用程序/游戏服务器1521-1525的网络地址而自延迟缓冲器1515获取延迟的视频/音频。 If the requested video / audio is only delayed, then the requesting app / game server 1521 -1525 will be so notified, and which directory on the delay buffer 1515 video / audio and the location by searching for the delayed video source / audio app / game server network address of the delay buffer from 1521-1525 and 1515 to obtain delayed video / audio. 一旦所有该请求被产生并处理,则将高达15个现场直播的缩略图大小的视频流从出埠路由1540路由到入埠路由1502、到产生用户接口屏幕的应用程序/游戏服务器1521-1525,且将由该服务器来解压缩及显示。 Once all of these requests have been generated and processed, up to 15 live thumbnail-sized video streams will be routed from the outbound routing 1540 to the inbound routing 1502 to the application program generating the user interface screen / game server 1521 -1525, and will be decompressed and displayed at the server. 延迟的视频/音频流可能处于过大的屏幕大小,如果这样,则应用程序/游戏服务器1521-1525将解压缩所述流并将视频流按比例缩减至缩略图大小。 Delayed video / audio streams may be in too large a screen size, and if so, the app / game server 1521 -1525 will decompress the streams and scale down the video streams to thumbnail size. 在一个实施例中,将对音频/视频的请求发送到与图4a的主机服务控制系统类似的中央“管理”服务(图15中未显示)(且由中央“管理”服务来管理),中央“管理”服务接着将所述请求重定向到适当应用程序/游戏服务器1521-1525。 In one embodiment, sends a request for the audio / video to the hosting service control system of Figure 4a is similar to the central "Management" service (not shown in FIG. 15) (and managed by the central "Management" service), the central "management" service then redirects the requests to the appropriate app / game server 1521 -1525. 此外,在一个实施例中,可能不需要请求,因为缩略图被“推送”至允许其的那些用户的客户端。 Further, in one embodiment, the request may not be required because the thumbnails are "pushed" to those users that allow their clients.

[0245] 来自15个游戏的所有同时混合的音频可能产生刺耳的声音。 [0245] All the while mixing audio from 15 games might create a cacophony of sound. 用户可选择以此方式将所有声音混合在一起(可能就为得到由被观看的所有动作产生的“喧嚣”的感觉),或者用户可选择每次仅收听来自一个游戏的音频。 In this way the user can choose to mix all the sounds together (probably "noisy" feeling generated by all the action being viewed to obtain), or the user may choose to just listen to the audio from a game. 单个游戏的选择是通过将黄色选择帧1601 移动到给定游戏来完成(黄色帧移动可通过使用键盘上的箭头键、通过移动鼠标、通过移动操纵杆或通过推动诸如移动电话的另一设备上的方向按钮来完成)。 The choice is a single game (the yellow box movement can be accomplished by using arrow keys on a keyboard, by moving a mouse, by moving a joystick, or by pushing another device such as a mobile phone by moving the yellow selection box 1601 to a given game to complete direction button to complete). 一旦选择了单个游戏,则只有来自该游戏的音频播放。 Once you select a single game, an audio player from the game only. 而且,显示游戏信息1602。 Also, the display game information 1602. 在该游戏的状况下,例如, 出版商标志(“EA”)及游戏标志“极品飞车卡本峡谷”及橙色横条在相对条件下指示在该的数目(在此状况下,许多,因此游戏为“热门”)。 In the case of this game, for example, the publisher logo ( "EA") and the game logo "Need for Speed ​​Carbon" and an orange horizontal bar indicates in relative terms in the number of (in this case, a lot, so the game as "hot"). 另外提供“状态”,指示存在145个玩家正积极地玩极品飞车游戏的80个不同具体实例(也即,该游戏可通过个别玩家游戏或多人游戏来玩),且存在680个观看者(此用户是其中之一)。 Further "state" indicates the presence of 80 different instantiations of 145 players actively playing Need for Speed ​​Game (i.e., it can be played by an individual player game or multiplayer game), and there are 680 viewers ( this user is one of them). 注意,该统计数据(及其他统计数据)由主机服务控制系统401来收集并储存于RAID阵列1511-1512上,以用于保持主机服务210操作的日志且用于适当地向用户计费并向提供内容的出版商支付费用。 Note that these statistics (and other statistics) are collected and 401 stored on a RAID array 151 1 -1512 by the hosting service control system for maintaining a log of the hosting service 210 operation and for appropriately billing users and provide content publishers to pay. 一些统计数据是由于由服务控制系统401进行的动作而记录,且一些统计数据是由个别应用程序/游戏服务器1521-1525报告给服务控制系统401。 Some statistics are due to be recorded by the operation of 401 service control system, and some statistics by individual app / game server 1521-1525 reported to the service control system 401. 举例而言,当游戏正被观看时(及当游戏被停止观看时),执行此游戏取景器应用程序的应用程序/游戏服务器1521-1525向主机服务控制系统401发送消息,以使得主机服务控制系统401 可更新多少个游戏处于观看中的统计数据。 For example, when the games are being viewed (and when the game is stopped watching), the application performs this Game Finder app / game server 1521 -1525 401 sends a message to the hosting service control system, so that the hosting service control how many game system 401 may be updated in view of the statistics. 一些统计数据可为用户接口应用程序(诸如, 此游戏取景器应用程序)所用。 Some statistics are available for user interface applications (such as this Game Finder application) used.

[0246] 若用户单击其输入设备上的启动按钮,则其将看见黄色帧中的缩略图视频放大同时缩略图视频保持现场直播为全屏幕大小。 [0246] If the user clicks the start button on their input device, it will see the yellow frame thumbnail video thumbnail to enlarge while maintaining broadcast live video to full screen size. 该效果显示在图17中的过程中。 This effect is shown in process in FIG. 17. 注意,视频窗1700的大小增大。 Note that the size of the video window 1700 is increased. 为了实施该效果,应用程序/游戏服务器1521-1525从运行选定的游戏的应用程序/游戏服务器1521-1525请求具有路由到其的游戏的全屏幕大小(以用户的显示设备422的分辨率)的视频流的复本。 To implement this effect, the app / game server 1521 -1525 running routes with the selected game from the app / game server 1521 -1525 requests to full screen size of the game (the user's display device resolution 422) Copies of the video stream. 执行游戏的应用程序/游戏服务器1521-1525通知共用硬件压缩器1530不再需要游戏的缩略图大小的复本(除非另一应用程序/游戏服务器1521-1525需要这种缩略图),且接着其指引共用硬件压缩器1530将视频的全屏幕大小的复本发送至放大视频的应用程序/游戏服务器1521-1525。 1521-1525 execution notification application / game server shared hardware compressor 1530 that a copy of the game is no longer required thumbnail size (unless another app / game server 1521 -1525 requires such a thumbnail), and then it guidelines shared hardware compressor 1530 that a copy of the application will send a fullscreen size of the video to the video amplifying / game server 1521 -1525. 玩该游戏的用户可或可不具有分辨率与将游戏放大的用户的所述显示设备的分辨率相同的显示设备422。 The user playing the game may or may not have resolution as the user zooming up the game devices display the same resolution as the display device 422. 另外,游戏的其他观看者可或可不具有分辨率与将游戏放大的用户相同的显示设备422(且可具有不同的音频回放装置,例如,立体声或环绕音效)。 Further, other viewers of the game may or may not have the same resolution and displayed to the user zooming up the game device 422 (and may have different audio playback means, e.g., stereo or surround sound). 因此,共用硬件压缩器1530确定是否已经产生满足请求视频/音频流的用户的要求的合适的经压缩视频/音频流,且若合适的经压缩视频/音频流确实存在,则共用硬件压缩器1530通知出埠路由1540将该流的复本路由至放大该视频的应用程序/游戏服务器1521-1525,且若合适的经压缩视频/音频流不存在,则压缩视频的适合于所述用户的另一复本并指导出埠路由将该流发送回至入埠路由1502及放大该视频的应用程序/游戏服务器1521-1525。 Thus, the shared hardware compressor 1530 determines whether it has generated the right by satisfying the request video / user audio stream requires compressed video / audio stream and if appropriate compressed video / audio stream does exist, the shared hardware compressor 1530 replica notifies the outbound routing 1540 routes the video stream to the amplification of the app / game server 1521 -1525, and if a suitable compressed video / audio stream does not exist, the compressed video is suited to the user further a copy of the routing and instructs the outbound stream is sent back to the inbound routing 1502 and the video application / game server 1521 -1525. 现在接收选定视频的全屏幕版本的该服务器将解压缩该全屏幕版本并将其逐渐地按比例放大至全大小。 Now receiving a full screen version of the selected video will decompress the server version of the full-screen and gradually scale it up to full size.

[0247] 图18说明在将游戏完全放大至全屏幕且以用户的显示设备422的全分辨率显示游戏之后屏幕看起来如何(如通过箭头1800指向的图像所指示的)。 [0247] Figure 18 illustrates after the game has completely zoomed up to full screen and the game display at the full resolution of the display device 422 a user how the screen looks (e.g., the image 1800 is indicated by an arrow pointing). 执行游戏取景器应用程序的应用程序/游戏服务器1521-1525向提供缩略图的其他应用程序/游戏服务器1521-1525发送消息以指示所述缩略图不再需要且向主机服务控制服务器401发送消息以指示不再观看其他游戏。 Game Finder application executing application / game server 1521 -1525 to other applications provide thumbnails / game server 1521 -1525 sends a message to indicate that the thumbnail images 401 are no longer needed and send a message to the hosting service control server to no longer being viewed other games. 此时,产生的唯一显示为屏幕顶部的叠加层(overlay) 1801,其将信息及菜单控制提供给用户。 In this case, the only display the generated screen to the top of the overlay (overlay) 1801, which provides information and menu controls to the user. 注意,随着该游戏进展,观众增长至2,503个观看者。 Note that, with the progress of the game, the audience has grown to 2,503 viewers. 在如此多的观看者的情况下,必然存在具有显示设备422的许多观看者,所述显示设备422具有相同或接近的分辨率(每个应用程序/游戏服务器1521-1525具有按比例调整视频以用于调整配合度的能力)。 In the case of so many viewers, there are bound to be many viewers with display devices 422, the display apparatus 422 having the same or nearly the resolution (each app / game server 1521 -1525 has the video is scaled to capability for adjusting the fitting).

[0248] 因为所显示的游戏为多人游戏,所以用户可决定在某时刻加入该游戏。 Game [0248] because the show is a multiplayer game, so the user can decide to join the game at some point. 由于多种原因,主机服务210可或可不允许用户加入该游戏。 For various reasons, the host service 210 may or may not allow the user to join the game. 举例而言,用户可能必须支付玩游戏的 For example, users may have to pay to play the game

47费用而选择不支付,用户可能不具有足以加入所述特定游戏的足够等级(例如,对于其他玩家而言,其将不有竞争性),或者用户的互联网连接可能不具有足以允许用户玩的足够低的延时(例如,不存在用于观看游戏的延时约束,因此可在无延时关注的情况下观看被远距离地(实际上,在另一大陆上)玩的游戏,但对于待玩的游戏而言,延时必须足够低以使用户(a)享受该游戏,且(b)处于与可能具有较低延时连接的其他玩家相等的地位)。 While 47 chose not to pay the cost, the user may not have sufficient enough to join that particular game level (for example, for the other players, it will not be competitive), or the user's Internet connection may not have enough to allow the user to play sufficiently low latency (e.g., not a latency constraint for viewing games, watch can thus be far away (indeed, on another continent) to play the game without latency concerns the case, but for to be playing a game, the delay must be low enough so that the user (a) enjoy the game, and (b) are in equal status with the other players may have lower latency connections). 若准许用户玩,则为用户提供游戏取景器用户接口的应用程序/游戏服务器1521-1525将请求主机服务控制服务器401初始化(也即,定位并启动)被合适地配置以用于播放特定游戏的应用程序/游戏服务器1521-1525从而从RAID阵列1511-1512载入该游戏,且接着主机服务控制服务器401将指导入埠路由1502将来自用户的控制信号传送至现在对游戏进行主机代管的应用程序/游戏服务器且现在对游戏进行主机代管的应用程序/游戏服务器将指导共用硬件压缩1530自压缩来从主机代管游戏取景器应用程序的应用程序/游戏服务器的视频/音频切换至压缩来自现在对游戏进行主机代管的应用程序/游戏服务器的视频/音频。 If the user is permitted to play, then the user providing the Game Finder user interface application / game server 1521 -1525 requests the hosting service control server 401 initiate (i.e., locate and start up) is suitably configured for playing the particular game app / game server 1521 -1525 to load the game from a RAID array 1511-1512, and then the hosting service control server 401 will instruct the user control signal from the inbound routing 1502 to now hosting the game application app / game server and the game will now hosting the app / game server 1530 will instruct the shared hardware compression from the compression / game server hosting the game application from the finder application of video / audio from the switch to the compression video applications are now hosting a game / game server / audio. 游戏取景器应用程序/游戏服务与对游戏进行主机代管的新的应用程序/游戏服务器的垂直同步并不同步,且因此在该两个同步之间可能存在时间差。 Game Finder app / game service and the new application program hosting the game / game server vertical synchronization are not synchronized, and thus there may be a time difference between the two synchronization. 因为共用视频压缩硬件1530将在应用程序/游戏服务器1521-1525完成视频帧之后即开始压缩视频,所以来自新服务器的第一帧可比旧服务器的全帧时间完成得早,来自新服务器的第一帧可能在先前经压缩的帧完成其传输之前(例如,考虑图9b的传输时间992 :若未经压缩的帧3963 早一帧时间的一半地完成,则其将影响(impinge)传输时间992)。 Because the shared video compression hardware 1530 will begin compressing video upon an app / game server 1521-1525 complete video frame, the first full-frame time frame than the old server from a new server is completed early, the first from the new server before the previous frame may be compressed frame completing its transmission (e.g., consider transmit time 992 of Figure 9b: if uncompressed half of the frame 3963 to complete an earlier time, then it will impact (impinge) the transmit time 992) . 在这种情形下,共用视频压缩硬件1530将忽略来自新服务器的第一帧(例如,如忽略(974)帧4964),且客户端415将来自旧服务器的最末帧保持额外一帧时间,且共用视频压缩硬件1530将开始压缩来自对游戏进行主机代管的新应用程序/游戏服务器的下一帧时间视频。 In such a situation the shared video compression hardware 1530 will ignore the first frame (e.g., such as ignoring (974) 4964 frames), and the client 415 from the old server an extra hold the last frame from the new server time, and the shared video compression hardware 1530 will begin compressing the new application from the game had been hosting the next frame time video / game server. 对于用户而言,在视觉上,从一个应用程序/游戏服务器至另一应用程序/游戏服务器的转变将是无缝的。 For the user, visually, from one app / game server to the other app / game server transition will be seamless. 主机服务控制服务器401接着将通知对游戏取景器进行主机代管的应用程序/游戏服务器1521-1525切换至闲置状态,直至再次需要其为止。 The hosting service control server 401 will then notify that had been hosting the Game Finder app / game server 1521 -1525 switch to an idle state, until it is needed again far.

[0249] 用户接着能够玩该游戏。 [0249] The user can then play the game. 此外,例外的是游戏将在感知上即刻地播放(因为游戏已被以十亿比特/秒速度从Raid阵列1511-1512载入到应用程序/游戏服务器1521-1525 上),且将通过理想的驱动器、暂存器配置(在Windows的状况下)将游戏连同被正确配置以用于该游戏的操作系统一起载入到准确适合于该游戏的服务器上,且没有可能与该游戏的操作竞争的其他应用程序在该服务器上执行。 In addition, exceptional is the game will play perceptually instantly (since the games at gigabit / second speed from a Raid array 1511-1512 to the app / game server 1521 -1525), and preferably through drive configuration registers (in the case of Windows) will play along with the operating system to be configured correctly for the game loaded with the exact fit for the game on the server, and not likely to compete with the operation of the game other applications execute on the server.

[0250] 而且,随着用户在游戏中进展,游戏的片段中的每一者将以十亿比特/秒的速度(也即,在8秒内载入十亿字节)从RAID阵列1511-1512载入服务器中,且由于RAID阵列1511-1512的巨大储存容量(因为其为许多用户的共用资源,所以其可能非常大,但仍具成本效益),使得可预先计算几何形状设置或其他游戏片段设置并将其储存于RAID阵列1511-1512上且极快速地进行载入。 [0250] Also, as the progress of the user, each of the segments of the game will speed gigabit / second speed (i.e., 1 gigabyte loads in 8 seconds) from the RAID arrays 1511- game Loading server 1512, and because of the vast storage capacity of the RAID array 1511-1512 (since it is a shared resource among many users, it can be very large, yet still be cost effective), can be calculated in advance so that the geometry setup or other game fragment set and stored on the RAID array 151 1 -1512 and loaded extremely rapidly. 此外,因为每个应用程序/游戏服务器1521-1525的硬件配置及计算能力是已知的,所以可预先计算像素及顶点着色。 Further, since each app / game server 1521 -1525 hardware configuration and computational capabilities are known, it is possible to pre-calculate the pixel and vertex shaders.

[0251] 因此,游戏可几乎即刻启动,其将在理想环境中执行,且随后的片段将几乎即刻载入。 [0251] Thus, the game will start up almost instantly, it will run in an ideal environment, and subsequent segments will load almost instantly.

[0252] 但是,除这些优点之外,用户将能够观看他人玩游戏(经由先前所描述的游戏取景器,及其他装置),且两者均决定游戏是否有趣,如果这样,则自观看他人来学习技巧。 [0252] However, in addition to these advantages, users will be able to watch others play games (Game Finder described by, and other devices), and both decide whether the game is fun, and if so, from watching others study skills. 此外,用户将能够即刻地演示该游戏,而不必等待大的下载和/或安装,且用户将能够即刻玩该游戏(可能在较小费用的试用基础上,或在较长期的基础上)。 In addition, users will be able to demo the game, without having to wait for large download and / or install, and the user will be able to instantly play the game (perhaps on a trial basis for a small fee, or on a longer-term basis). 此外,用户将能够通过足够低延时的无线连接而在Windows PC、MaCint0Sh上、在电视机上、在家里、在行进时且甚至在移动电话上玩该游戏。 In addition, users will be able to wirelessly connect sufficiently low latency and on Windows PC, MaCint0Sh, on television, at home, and even play the game on a mobile phone while traveling. 此外,这均可在并非曾经实体拥有游戏复本的情况下完成。 In addition, it may not have been in the case of entities owned copy of the game.

[0253] 如先前所叙述,用户可决定不允许其游戏播放可被他人观看,允许其游戏可在延迟之后观看,允许其游戏可被选定用户观看,或允许其游戏可被所有用户观看。 [0253] As previously described, the user can decide not to allow his game to be viewable by others, to allow his game to be viewable after a delay to allow his game to be viewable by selected users, or allow his game to be viewable by all users. 不管怎样, 在一个实施例中,将视频/音频储存于延迟缓冲器1515中历时15分钟,且用户将能够“回倒”并观看其先前的游戏播放,且将游戏暂停,将游戏缓慢地回放,将游戏快进等,正如其在观看具有数字视频记录器(DVR)的TV时所能够进行的。 In any case, in one embodiment, the video / audio is stored on a delay buffer 1515 for 15 minutes, and the user will be able to "rewind" and view his prior game play, and the play pause, play it back slowly , the fast forward, etc., just as when watching TV with a digital video recorder (DVR) being able to. 尽管在该实例中,用户在玩游戏, 但若用户正使用应用程序,则相同“DVR”能力是可用的。 Although in this example, a user playing a game, but if the user is using the application, the same "DVR" capability is available. 这在核查先前工作中及在如下详述的其他应用中可是有用的。 This prior work and may be useful in other applications as detailed below in the verification. 另外,若游戏经设计为具有基于利用游戏状态信息而回倒的能力,以便可改变相机视图等,则也将支持此“3D DVR”能力,但其将需要将游戏设计为支持“3D DVR”能力。 In addition, if the game was designed to have a capacity based on utilizing game state information back down the order can change the camera view, etc., it is also supported by "3D DVR" capability, but it will require the game to be designed to support "3D DVR" ability. 使用延迟缓冲器1515的“DVR”能力将连同任何游戏或应用程序(当然,限于在使用游戏或应用程序时所产生的视频)一起起作用,但在具有3D DVR能力的游戏的状况下,用户可控制先前所播放的片段的3D “飞越(fly-through) ”,且使延迟缓冲器1515记录所得视频并记录游戏片段的游戏状态。 Using a delay buffer 1515 under "DVR" will work with any game or application (of course, limited to a video game or when an application using the generated) function together, but with 3D DVR capability of the status of a game, the user may control a previously played segment 3D "overflew (fly-through)", and have the delay buffer 1515 record the resulting video and game state of the game segment record. 因此,将特定“飞越”记录为经压缩的视频,但因为也将记录游戏状态,所以不同的飞越将可能在游戏的同一片段的稍后日期。 Thus, a particular "over" is recorded as compressed video, but also because the game state record, a different fly will be possible at a later date in the same segment of the game.

[0254] 如下所述,主机服务210上的用户将各自具有一个用户页面,在该用户页面中,用户可公布关于其本身的信息及其他数据。 [0254] as described below, users on the hosting service 210 will each have a User Page, the user page, the user can post information about themselves and other data is. 用户将能够公布的事情之一为来自其已保存的游戏播放的视频片段。 One thing users will be able to publish a video clip from their saved game to play. 举例而言,若用户已克服游戏中的特别困难的挑战,则用户可刚好“回倒”到其在游戏中获得其大成果的地点之前,且接着指导主机服务210将某一持续时间(例如,30秒)的视频片段保存在用户的用户页面上以供其他用户观看。 For example, if the user has a particularly difficult challenge to overcome in the game, the user can "rewind" before the place had their great achievements in the game to, and then instruct the hosting service 210 will some duration (eg , 30 seconds) of the video clips in the user's user page for other users to watch. 为了实施这些,用户正使用的应用程序/游戏服务器1521-1525仅要做的事情是将储存于延迟缓冲器1515中的视频回放至RAID阵列1511-1512且接着在用户的用户页面上给所述视频片段编索引。 To implement this, the user is using an application / 1521-1525 do game server is a playback video stored in a delay buffer 1515 to a RAID array 1511-1512 and then the user on the user's page index that video segment.

[0255] 若游戏具有3D DVR的能力,如上所述,则也可由用户来记录3D DVR所需的游戏状态信息且使其为用户的用户页面可用。 [0255] If the game has the capability of 3D DVR, as described above, the user may also be required to record 3D DVR game state information and made available for the user's User Page.

[0256] 在游戏经设计为除具有活跃玩家外还具有“旁观者”(也即,能够在不参与的情况下在3D世界行进并观察到动作的用户)的情况下,则游戏取景器应用程序将使用户能够作为旁观者以及玩家加入游戏。 Case [0256] In addition to a game designed for the active players have "spectators" (that is, able to participate without traveling in a 3D world and observe the action of the user), the Game Finder application program will allow users to join the game as spectators and players. 从观看的实施点看,对于主机系统210而言,用户是旁观者而不是活跃玩家是不存在差异的。 From the implementation point of view to see, for the host system 210, the user is a spectator rather than an active player does not exist differences. 将游戏载入应用程序/游戏服务器1521-1525上且用户将控制该游戏(例如,控制观看世界的虚拟相机)。 The game loaded onto an app / game server 1521-1525 and the user will be controlling the game (for example, see the world of virtual camera control). 唯一的差异是用户的游戏体验。 The only difference is the user's gaming experience.

[0257] 多个用户合作 [0257] multiple users to collaborate

[0258] 主机服务210的另一特征是多个用户在观看现场直播的视频的同时合作的能力(即使使用迥然不同的设备来观看也如此)。 [0258] Another feature 210 hosting service is the ability of multiple users to collaborate while viewing live video (even with very different devices for viewing so). 当玩游戏时及当使用应用程序时,这都有用。 When playing the game and when when using the application, which are useful.

[0259] 许多PC及移动电话装备有视频相机且具有进行实时视频压缩的能力(尤其当图像小时)。 [0259] Many PC and mobile phone equipped with a video camera and has the ability for real-time video compression (especially when the image is small). 而且,小相机是可用的,可附接到电视,且以软件或使用用于压缩视频的许多硬件压缩设备中的一者来实施实时压缩并不困难。 Moreover, a small camera is available, it can be attached to the TV, and to use the software or device for real-time compression is not difficult in one of the many hardware compression video compression is implemented. 而且,许多PC及所有移动电话具有麦克风, 且耳机在具有麦克风情况下可用。 Moreover, many PC and all mobile phones have microphones, and headsets are available with microphones.

[0260] 组合有本地视频/音频压缩能力(具体来说,使用本文中所描述的低延时视频压缩技术)的所述相机和/或麦克风将使用户能够将视频和/或音频连同输入设备控制数据一起从用户场所211传输到主机服务210。 [0260] combined with local video / audio compression capability (particularly employing the low latency video compression described herein technique) of the camera and / or microphone will allow user to transmit video and / or audio input devices in conjunction with 211 with transmission control data from the user premises 210 to the hosting service. 当使用所述技术时,则可实现图19中所说明的能力:用户可使其视频及音频1900出现在另一用户的游戏或应用程序内的屏幕上。 When using the technology, the ability can be realized illustrated in FIG. 19: a user can have his video and audio 1900 appear on the user's game or another application on the screen. 该实例为多人游戏,其中队友在赛车中合作。 This example is a multiplayer game, in which teammates in the car. 用户的视频/音频仅可被其队友选择性地观看/听到。 User video / audio only by their teammates selectively viewable / hearable. 此外,因为使用上文所描述的技术将有效地不存在延时,所以玩家将能够实时地彼此谈话或进行运动而没有可感知的延迟。 In addition, since the techniques described above are used effectively no latency, players would be able to talk or make motions to each other in real-time without perceptible delay.

[0261] 该视频/音频整合是通过使来自用户的相机/麦克风的经压缩视频和/或音频作为入埠互联网通信1501到达而完成。 [0261] This video / audio integration is accomplished by the user from the camera / microphone compressed video and / or audio as inbound internet traffic 1501 arrival. 接着,入埠路由1502将该视频和/或音频路由到被准许观看/听到视频和/或音频的应用程序/游戏服务器1521-1525。 Then the inbound routing 1502 routes the video and / or audio routing are permitted to view / hear the video and / or audio app / game server 1521-1525. 接着,选择使用视频和/或音频的各别应用程序/游戏服务器1521-1525的用户解压缩视频和/或音频且视需要而将其整合以出现于游戏或应用程序内,诸如通过1900所说明的。 Next, choose a video and / or audio of the respective app / game game servers 1521 -1525 that video and / or audio decompress it and integrate as desired to appear within the game or application, such as described by 1900 of.

[0262] 图19的实例显示如何在游戏中使用该合作,但该合作可为用于应用程序的极其强大的工具。 Examples [0262] Figure 19 shows how to use the cooperation in the game, but this cooperation can be an extremely powerful tool for the application. 考虑一各情形:其中一大建筑物正由在芝加哥的建筑师为以纽约为基地的房地产开发商为纽约市设计,但该决策涉及在旅游中且碰巧在迈阿密机场的财务投资者,且需要进行关于建筑物的特定设计要素(在其如何搭配其附近的建筑物方面)的决策,以满足投资者与房地产开发商两者。 Consider a situation: one of the major real estate developers with a structure that is New York-based New York City, designed by the architect in Chicago, but the decision involves traveling and happens to financial investors in the Miami airport, and the need be specific about the design elements of the building (in terms of how its mix of buildings in its vicinity) of the decision, in order to satisfy both investors and real estate developers. 假定建筑公司在芝加哥具有具有附接到PC的相机的高分辨率监视器,房地产开发商在纽约具有带相机的笔记本计算机,且投资者在迈阿密具有带相机的移动电话。 The company has assumed building in Chicago with a camera attached to a PC monitor's high resolution, real estate developers have a laptop with a camera in New York, and the investor has a camera with a mobile phone in Miami. 建筑公司可使用主机服务210来对能够进行高度逼真3D再现的强大的建筑设计应用程序进行主机代管,且其可利用纽约市的建筑物的大数据库,以及正设计的建筑物的数据库。 Building company to use the hosting service 210 can be a powerful architectural design applications were highly realistic 3D rendering of hosting, and it can take advantage of a large database of buildings in New York City, is designing and building database. 建筑设计应用程序将在应用程序/游戏服务器1521-1525中的一者上(或若其需要大量计算能力,则在若干者上)执行。 Architectural design application (or if it requires a lot of computational power on several,) executed on the application / game servers 1521-1525 one. 处于全异(disparate)位置处的3个用户中的每一者将连接至主机服务210,且每一者将具有对建筑设计应用程序的视频输出的同时观看,但其将被针对每个用户具有的给定设备及网络连接特性而由共用硬件压缩1530 适当地设定大小(例如,建筑公司可经由20Mbps商用互联网连接看见2560X 144060fps显示,纽约的房地产开发商可经由其笔记本计算机上的6Mbps DSL连接看见1280X72060fps 图像,且投资者可经由其移动电话上的250Kbps蜂窝式数据连接看见320X 18060fps图像)。 In disparate (disparate) for each user at a location 3 will connect to the hosting service 210, and each will have a video output of the architectural design application while viewing, but it will be for each user having a given device and network connection characteristics by the shared hardware compression 1530 is appropriately sized (e.g., the architectural firm may see through a 20Mbps commercial Internet 2560X 144060fps display, the real estate developer in New York via 6Mbps DSL on their computer 1280X72060fps connection see the image, and the investor via which a 250Kbps cellular data connection on the mobile phone 320X 18060fps visible image). 每一方将听到其他方的语音(将通过应用程序/游戏服务器1521-1525中的许多广泛可用的会议呼叫套装软件中的任一者来处理会议呼叫),且经由用户输入设备上的按钮的致动,用户将能够使用其本地相机使视频出现。 Each party will hear the voice of the other party (through the application of many widely available conference / game servers 1521-1525 suite of call in either one to handle conference calls) and user input via a button on the device actuation, the user will be able to make video appear of their local camera. 随着会议进行,建筑师将能够通过极具照片般逼真感的3D再现显示当其使建筑物旋转且使其邻接该区域中的另一建筑物飞越(fly-by)时建筑物看起来像什么,且所有方均将在各方的显示设备的分辨率下可见到相同视频。 As the meeting, the architects will be able to photo-realism by great 3D playback display when it is rotated and allowed the building to another building adjacent to the area flew (fly-by) when the building looks like What, and all parties are to be seen the same video at a resolution of the display device of the parties. 由任何方使用的本地设备中的任一者均能够以该真实感处理3D动画不是问题,更不用说下载或甚至储存再现纽约市的周围建筑物所需的巨大数据库。 Any local device used by any party is incapable of handling 3D animation with such realism, let alone download or even storing the huge database needed to surrounding buildings in New York City. 自用户中的每一者的观点看,尽管距离很远,且尽管是全异本地设备,但其将简单地在难以置信的真实感程度下具有无缝体验。 Since each of the users point of view in watching, although far apart, and despite the disparate local device, but it will simply have a seamless experience with an incredible degree of realism. 此外,当一方希望其面部被看来较佳地传达其情绪状态时,其可如此进行。 Further, when one party wants their face to be seen to better convey their emotional state, they can do so. 另外,若房地产开发商或投资者希望控制建筑程序且使用其自身的输入设备(其为键盘、鼠标、小键盘或触摸屏幕),则其可如此,且其可以用无感知的延时来响应(假定其网络连接不具有不合理的延时)。 In addition, if real estate developers or investors want to control construction program and use their own input device (which is a keyboard, mouse, keypad or touch screen), it may be so, and it can be used without perceptible delay in response (assuming their network connection does not have unreasonable latency). 举例而言,在移动电话的状况下,若移动电话连接至机场的WiFi网络,则其将具有非常低的延时。 For example, in the case of the mobile phone, if the mobile phone is connected to a WiFi network airport, it will have very low latency. 但若其使用美国现今可用的蜂窝式数据网络,则其很可能将 But if it is the United States using cellular data network available today, it is likely to

50遭受引人注意的滞后。 50 suffer from noticeable lag. 但是,对于会议的大多数目的(其中投资者正观看建筑师控制建筑物飞越或正谈论视频电话会议),甚至蜂窝式延时也应是可接受的。 However, for most purpose of the meeting (in which investors are viewing the architect or building fly control are talking about a video conference call), and even cellular latency should be acceptable.

[0263] 最后,在合作性会议呼叫结束时,房地产开发商及投资者将进行其评论且从主机服务停播,建筑公司将能够“回倒”已记录在延迟缓冲器1515上的会议的视频且核查在会议期间进行的应用于建筑物的3D模型的评论、面部表情和/或动作。 [0263] Finally, at the end of the collaborative conference call, real estate developers and investors made their comments and service off the air, construction companies will be able to "rewind" the meeting has been recorded on a delay buffer 1515 from the host's video and verification of 3D models used in the building made comments during the meeting, facial expressions and / or action. 若存在其希望保存的特定片段,则可将视频/音频的所述片段自延迟缓冲器1515移动至RAID阵列1511-1512 以用于档案储存及稍后回放。 If the particular segments they want to save, those segments may be the video / audio from a delay buffer 1515 to a RAID array 1511-1512 moved for archival storage and later playback.

[0264] 而且,从成本观点看,若建筑师仅需要使用纽约市的计算能力及大数据库历时15 分钟的会议呼叫,则其仅需要支付所述资源被使用的时间的费用,而不必拥有高能力的工作台且不必购买大数据库的昂贵复本。 [0264] Moreover, from a cost point of view, if an architect in New York City only need to use the computing power and large databases over a 15-minute conference call, it is only required to pay the cost of the resources used time, without having to have a high table and having to buy expensive large database replicas.

[0265] 视频丰富的社区服务 [0265] rich video community service

[0266] 主机服务210使得有空前机会来在互联网上建立视频丰富的社区服务。 [0266] The hosting service 210 enables an unprecedented opportunity to build rich video community service on the Internet. 图20显示用于主机服务210上的游戏玩家的例示性用户页面。 Figure 20 shows for a game player on the hosting service 210 illustrated exemplary User Page. 如同游戏取景器应用程序一样,用户页面为在应用程序/游戏服务器1521-1525中的一者上执行的应用程序。 As with the Game Finder application as the application user to a page on one of the app / game servers 1521-1525 executed. 该页面上的所有缩略图及视频窗显示恒定地移动的视频(若片段短,则其循环)。 All thumbnails and video windows on this page show constantly moving video (if the segments are short, they loop).

[0267] 使用视频相机或通过上传视频,用户(其用户名为“KILLHAZARD”)能够公布其本身的视频2000 (其他用户可观看该视频)。 [0267] Using a video camera or by uploading a video, users (whose user name is "KILLHAZARD") can publish their own video 2000 (other users can view the video). 该视频储存于RAID阵列1511-1512上。 The video is stored on a RAID array 1511-1512. 而且, 当其他用户来到KILLHAZARD的用户页面时,若KILLHAZARD此时正使用主机服务210,则将显示其正进行的无论什么的现场直播的视频2001 (假定KILLHAZARD准许观看其用户页面的用户观看该视频)。 Also, when other users come KILLHAZARD user page, if KILLHAZARD at this time is using the hosting service 210, no matter what the live video of its ongoing 2001 (assuming that KILLHAZARD permit users to view their user viewing the page will be displayed video). 这将由对用户页面应用程序进行主机代管的应用程序/游戏服务器1521-1525从服务控制系统401请求KILLHAZARD是否为活跃的(且若如此,则请求其正使用的应用程序/游戏服务器1521-1525)来完成。 This will be a page for user application hosting app / game server 1521-1525 request from the service control system 401 KILLHAZARD whether active (and if so, which applications are using the requests / game server 1521-1525 )To be done. 接着,使用由游戏取景器应用程序使用的相同方法,将合适分辨率及格式的经压缩的视频流发送至执行用户页面应用程序的应用程序/游戏服务器1521-1525且将其显示。 Next, using the same methods used by the Game Finder application sends compressed video stream in a suitable resolution and format the User Page application to the app / game server 1521 -1525 and the display. 若用户选择具有KILLHAZARD的现场直播的游戏播放的窗口且接着适当地单击其输入设备,则该窗口将放大(再次使用与游戏取景器应用程序相同的方法),且现场直播的视频将以观看用户的显示设备422的分辨率(适合于观看用户的互联网连接的特性)填充屏幕。 If a user selects with KILLHAZARD live game play window and then appropriately clicks on their input device, the window will zoom up (with the Game Finder application again in the same manner), and the live video will be viewed 422 resolution of the user's display device (viewing characteristics adapted to the user's Internet connection) fill the screen.

[0268] 这优于先前技术方法的关键优点是:观看用户页面的用户能够看见用户不拥有的现场直播地播放的游戏,且可不具有能够玩该游戏的本地计算机或游戏控制台。 [0268] This is advantageous over the prior art methods of key advantages are: User page viewing can see the user does not own the game played live, and may not be able to play the game with a local computer or game console. 其为用户提供看用户页面中显示为“活动中”的用户玩游戏的极好机会,且这是了解观看用户可能希望尝试或较擅长的游戏的机会。 It provides users with an excellent opportunity to see the games as "activities" of the user display a user page, and this is an opportunity to try to understand better at the game or watching a user may wish.

[0269] 来自KILLHAZARD的搭档2002的相机记录的或上传的视频剪辑也显示于用户页面上,且每个视频剪辑的下方为指示该搭档是否在线玩游戏的文字(例如,six shot正玩游戏“龙骑士(Eragon) ”且MrSnUggleS99离线等)。 [0269] KILLHAZARD partner from the camera or upload recorded 2002 video clip is also displayed on the user page, and underneath each video clip to indicate whether the partner online play word games (for example, six shot is playing the game. " dragon Knight (Eragon) "and MrSnUggleS99 offline, etc.). 通过单击菜单项(未图示),搭档视频剪辑自显示已记录的或上传的视频切换至当前正玩主机服务210上的游戏的搭档在所述时刻在其游戏中正在进行的内容的现场直播的视频。 By clicking a menu item (not shown), partner or upload video clips from the show recorded video is currently being switched to playing the game on the hosting service 210 ongoing partner in their game at the moment the content of the site live video. 因此,其变成为搭档分群的游戏取景器。 Therefore, it becomes partner grouping game viewfinder. 若选择搭档的游戏且用户单击该游戏,则该游戏将放大至全屏幕,且用户将能够观看全屏幕现场直播地播放的游戏。 If you choose to partner game and the user clicks the game, then the game will be enlarged to full screen, and the user will be able to watch the game full-screen live broadcast.

[0270] 再次,观看搭档的游戏的用户不拥有游戏的复本,也不拥有用于玩该游戏的本地 [0270] Again, the user viewing the buddy's game does not own a copy of the game, nor the local for playing the game

51计算/游戏控制台资源。 51 computing / game console resources. 游戏观看是有效瞬时的。 Game viewing is effectively instantaneous.

[0271] 如上文先前所描述,当用户玩主机服务210上的游戏时,用户能够“回倒”游戏且发现其希望保存的视频片段,且接着将该视频片段保存到其用户页面。 [0271] As previously described above, when a user plays a game on the hosting service 210, the user can "rewind" the game and find a video segment which wants to save, and then saves the video segment to his User Page. 这被称为“自赏剪辑(Brag Clip)”。 This is called "Brag Clip (Brag Clip)". 视频片段2003都是由KILLHAZARD从其所玩的先前游戏保存的自赏剪辑2003。 Video clips from 2003 are playing it by the KILLHAZARD previously saved game Brag Clip 2003. 数字2004显示自赏剪辑已被观看多少次,及自赏剪辑何时被观看,用户具有对其评定等级的机会,且橙色钥匙孔形状的图示2005的数目指示等级是多高。 2004 figures show Brag Clip has been viewed many times, and when the Brag Clip is viewed, users have the opportunity of its rating, and keyhole-shaped orange icon indicating the level of 2005 the number is high. 当用户观看用户页面时,自赏剪辑2003连同页面上的其余视频一起恒定地循环。 When a user viewing a user page, Brag Clip 2003 constantly circulating along with the rest of the video on the page. 若用户选择并单击自赏剪辑2003中的一者,则其放大以呈现自赏剪辑2003,以及允许播放、暂停、回倒、快进、步进等该剪辑的DVR控制。 If the user selects and click Brag Clip one in 2003, it zooms up to present the Brag Clip 2003, and allowed to play, pause, rewind, fast forward, step, etc. DVR control of the clip.

[0272] 自赏剪辑2003回放是通过应用程序/游戏服务器1521-1525载入用户记录自赏剪辑时储存于RAID阵列1511-1512上的经压缩视频片段且将其解压缩并将其回放来实施。 [0272] The Brag Clip 2003 playback is implemented by the app / game server 1521 -1525 loading the user recorded stored on a RAID array 151 1 -1512 when the Brag Clip segment and the compressed video decompressing it and playing it back .

[0273] 自赏剪辑2003也可为来自支持3D DVR能力的游戏的“3D DVR”视频片段(也即, 来自可被重放且允许用户改变相机观看点的游戏的游戏状态序列)。 [0273] Brag Clips 2003 can also be "3D DVR" video segments from 3D DVR & game (i.e., from the user can be replayed and allows the game to change the camera view point of the game state sequence). 在此状况下,除用户在记录游戏片段时进行的特定“飞越”的经压缩视频记录之外,也储存游戏状态信息。 In this case, in addition to the particular "fly" by the user when recording a compressed video game segment recorded game state information is also stored. 当用户页面正被观看且所有缩略图及视频窗均恒定地循环时,3D DVR自赏剪辑2003将使在用户记录游戏片段的“飞越”时记录为经压缩视频的自赏剪辑2003恒定地循环。 When a user page is being viewed, and all the thumbnails and video windows are constantly circulating, 3D DVR Brag Clip 2003 will record the user record when the game segment "Beyond" video is compressed by the Brag Clip 2003 constant cycle . 但是,当用户选择3D DVR自赏剪辑2003并单击3D DVR自赏剪辑2003时,除允许播放经压缩视频自赏剪辑的DVR控制之外,用户将能够单击给出其用于游戏片段的3D DVR能力的按钮。 However, when the user selects a 3D DVR Brag Clip 2003 and click 3D DVR Brag Clip 2003, in addition to allow the compressed video Brag Clip DVR controls outside, the user will be able to click to be given for the game segment button 3D DVR capability. 其将能够独立地控制游戏片段期间的相机“飞越”,且若其希望(且拥有用户页面的用户允许如此),则其将能够以经压缩的视频的形式记录替代性自赏剪辑“飞越”,替代性自赏剪辑“飞越”将接着可为用户页面的其他观看者所用(立即地,或者在用户页面的拥有者具有核查自赏剪辑的机会之后)。 They will be able to control the camera during a game segment "over", and if it wants (and has a User page allows so), it will be able to record the compressed video in the form of alternative Brag Clip "over" other viewers, alternative Brag clip "Beyond" will then be used by the user of the page (either immediately, or have the opportunity to Brag clip verification in the owner's user page after).

[0274] 该3D DVR自赏剪辑2003能力是通过启动将要在另一应用程序/游戏服务器1521-1525上重放已记录的游戏状态信息的游戏来启用。 [0274] The 3D DVR Brag Clip 2003 capability is enabled by activating the state of the game to be played back the recorded information to enable the game on another app / game server 1521-1525. 因为游戏可被几乎瞬时地启动(如先前所描述),所以启动其(其播放限于由自赏剪辑片段记录的游戏状态)且接着允许用户在将经压缩视频记录到延迟缓冲器1515的同时用相机进行“飞越”并不困难。 Since the game can be activated almost instantaneously (as previously described), so that start (with its play limited by the Brag Clip recorded game state), and then allow the user to simultaneously recording the compressed video to a delay buffer 1515 by camera "fly" is not difficult. 一旦用户完成进行“飞越”,则将游戏撤销启动。 Once the user has completed a "fly through" withdrawal will start the game.

[0275] 从用户的观点看,启动具有3D DVR自赏剪辑2003的“飞越”并不比控制线性自赏剪辑2003的DVR控制难。 [0275] From the user's point of view, with a 3D DVR Brag Clip 2003 "over the" no better than control linear Brag Clip DVR 2003 is difficult to control. 用户可不知道该游戏或甚至不知道如何玩该游戏。 Users do not know the game or do not even know how to play this game. 用户指示盯着看另一操作者记录的游戏片段期间的3D世界的虚拟相机操作者。 Peering into a 3D world during a game segment recorded by another virtual camera operator.

[0276] 用户将也能够将其自身的音频加录于自赏剪辑上(或者自麦克风记录或者上传)。 [0276] Users will also be able to overdub their own audio Brag (either recorded from microphones or uploaded) on the clip. 以这种方式,可使用自赏剪辑来使用来自游戏的人物及动作产生定制动画。 In this way, you can use the Brag Clip to use characters and action from the game to create custom animations. 此动画制作技术通常被称为“游戏电影(machinima) ”。 This animation technique is commonly referred to as "game movies (machinima)".

[0277] 随着用户在游戏中进展,其将达成不同技能级别。 [0277] As users progress through the game, they will achieve differing skill levels. 所播放的游戏将成果报告给服务控制系统401,且所述技能级别也将显示于用户页面上。 The games played will report the results to the service control system 401, and the skill level will be displayed on the user page.

[0278] 互动式动画广告 [0278] Interactive animated ads

[0279] 在线广告已自文字转变至静态图像、视频,且现在转变至互动式片段,通常使用如Adobe Flash的动画精简型客户端来实施。 [0279] Online advertising has been transformed from text to still images, video, and now to interactive segments, typically using animation thin clients such as Adobe Flash to implement. 使用动画精简型客户端的原因在于:用户通常对于因向其推销产品或服务的特权而被延迟较无耐心。 The reason for using animation thin clients are: users typically have little patience for the products pitched to them the privileges or services is delayed. 而且,精简型客户端在非常低性能的PC上执行,且因此广告商可具有高度信心:互动式广告将适当地工作。 Also, thin client on the PC to perform very low performance, and therefore advertisers can have a high degree of confidence: Interactive advertising will work properly. 很遗憾地,诸如Adobe Flash的动画精简型客户端在互动性的程度及体验(以减少下载时间)的持续时间上受限制。 Limited duration regret, such as Adobe Flash animation thin clients in the degree of interactivity and experience (to reduce download time).

[0280] 图21说明一个互动式广告,其中用户将在汽车在陈列室中旋转时选择汽车的外部及内部色彩,同时实时射线追踪显示汽车看起来如何。 [0280] FIG. 21 illustrates an interactive advertisement where the user will select the external and internal color of the car while the car rotates in a showroom, while real-time ray tracing show how the car looks. 接着用户选择角色来驾驶汽车,且接着用户可采用该汽车来用于在竞赛轨道上或者穿过诸如摩纳哥的外国场所驾驶。 Then the user selects a role to drive a car, and then the user can take the car for the race track, or through foreign places such as Monaco driving. 用户可选择较大引擎或较佳轮胎,且接着可看见改变的配置如何影响汽车加速或保持稳定的能力。 Users can select a larger engine, or better tires, and then can see how the changed configuration affects the ability of the car to accelerate or hold the road.

[0281] 当然,广告有效地的为尖端的3D视频游戏。 [0281] Of course, the advertisement is effectively a sophisticated 3D video game. 但对于可在PC或视频游戏控制台上播放的这种广告,其将需要可能100MB下载,且在PC的状况下,其可能需要安装特殊驱动器, 且可能在PC缺乏足够CPU或GPU计算能力时根本不执行。 But for this kind of advertising that can be played on a PC or video game console, it will probably need 100MB download, and in the case of a PC, it may need to install special drivers, and may lack sufficient CPU or GPU computing power in PC not at all. 因此,所述广告在先前技术配置中不切实际。 Thus, such advertisements are impractical in prior art configurations.

[0282] 在主机服务210中,所述广告几乎即刻地投放,且较佳地执行,无论用户的客户端415能力如何。 [0282] In the hosting service 210, such advertisements launch almost instantly, and run perfectly, no matter the user's client 415 capabilities. 因此,其比精简型客户端互动式广告更迅速地投放,体验上更加丰富,且高度可靠。 So, put it more quickly than the thin client interactive advertising, the richer the experience, and highly reliable.

[0283] 实时动画期间流动的几何形状 Flow [0283] During the real-time animated geometry

[0284] RAID阵列1511-1512及入埠路由1502可提供如此快的数据速率且具有如此低的延时,以致有可能设计依赖于RAID阵列1511-1512及入埠路由1502来在实时动画(例如, 具有复杂数据库的飞越)期间于游戏播放的中间或应用程序中可靠地直接传送几何形状的视频游戏及应用程序。 [0284] RAID array 1511-1512 and the inbound routing 1502 can provide data rates so quickly and with such a low latency, so that it is possible to design relies on a RAID array 1511-1512 and the inbound routing 1502 to the real-time animation (e.g. , intermediate application or flying) with a complex database during game play to reliably deliver geometry video games and applications.

[0285] 在先前技术的系统(诸如,图1中所显示的视频游戏系统)下,可用的大量储存设备(尤其是在实用的家庭设备中)极缓慢以致不能在游戏播放期间流动几何形状(除了所需的几何形状稍微可预测的情形之外)。 [0285] In the prior art systems (such as in FIG. 1 shows a video game system), available mass storage devices (particularly in practical home devices) are far too slow to stream geometry in during game play ( except in situations where the required geometry was somewhat predictable). 举例而言,在存在指定道路的驾驶游戏中,可合理地适当预测用于进入视野内的建筑物的几何形状且大量储存设备可提前搜寻即将到来的几何形状所定位的位置。 For example, in the presence of designated roads driving game, it can reasonably be predicted to enter the appropriate geometry of the building within the field of view and a mass storage device can search for the location of the upcoming geometry positioning in advance.

[0286] 但在具有不可预测的改变的复杂场景中(例如,在周围具有复杂人物的战役场景中),若PC或视频游戏系统上的RAM完全被填满用于当前在视图中的对象的几何形状,且接着用户突然将其人物转向以观看其人物之后是什么,若未将几何形状预先载入RAM中,则可能在可显示几何形状之前存在延迟。 [0286] But in a complex scene with unpredictable changes (for example, in a battle scene with complex characters) if RAM on the PC or video game system is completely filled with objects in the current view of the geometry, and then the user suddenly turns their character around to view what is behind their character, if the geometry has been preloaded into RAM, then there may be a delay before the display geometry.

[0287] 在主机服务210中,RAID阵列1511-1512可以超过超高速以太网速度的速度使数据流动,且在SAN网络下,有可能达到优于10个十亿比特以太网或优于其他网络技术的100 亿比特/秒的速度。 [0287] In the hosting service 210, RAID arrays 151 1 -1512 can exceed the speed of Gigabit Ethernet speed so that the data flow, and with a SAN network, it is possible to achieve better than 10 gigabit Ethernet or over other networks rate of 100 gigabits / sec technology. 100亿比特/秒将在小于一秒内载入十亿字节的数据。 10 gigabits / second will load a gigabyte of data in less that a second. 在60fps帧时间(16. 67毫秒)内,可载入约170百万比特(21MB)的数据。 In a 60fps frame time (16.67 ms), a load of about 170 megabits (21MB) of data. 当然,甚至在RAID配置中,旋转媒体也仍将导致大于一帧时间的延时,但以闪存为基础的RAID储存器最终将与旋转媒体RAID阵列一般大且将不会招致该高延时。 Of course, even in a RAID configuration, rotation of the media will still incur latencies greater than a frame time, but Flash-based RAID storage will eventually be as large as rotating media RAID arrays and will not incur such high latency. 在一各实施例中,使用经由大量RAM写入的高速缓存来提供非常低延时的存取。 In one embodiment each of the embodiments, a large RAM write through cache to provide very low latency access.

[0288] 因此,在足够高的网络速度,以及足够低延时的大量储存器下,可与CPU和/或GPU 可处理3D数据一般快地将几何形状流动到应用程序/游戏服务器1521-1525中。 [0288] Thus, at a sufficiently high network speed, and sufficiently low enough latency mass storage, which can be the CPU and / or GPU may process the 3D geometry data to flow as fast as the app / game server 1521 -1525 in. 因此,在先前所给出的实例中,其中用户突然将其人物转向且向后看,可在人物完成旋转之前载入其身后的所有人物的几何形状,且因此,对于用户而言,将看来似乎其处于与现场直播的动作一般真实的照片般逼真的世界中。 Thus, in the example given previously, where a user turns their character around suddenly and looks behind, to be completed before the rotation geometry of loading all of the characters behind the character, and therefore, for the user, to look it seems to be in the live action generally true photo-realistic world.

[0289] 如先前所论述,照片般逼真的计算机动画中的最后的边界中的一者是人脸,且由于人眼对于不完全性的敏感性,来自照片般逼真的面部的最轻微错误可导致来自观看者的负面反应。 [0289] As discussed previously, the last frontiers in photorealistic computer animation in one human face, and because of the sensitivity of the human eye is not complete, and from photo-realistic face of the slightest error can lead to negative reaction from viewers. 图22显示使用Contour™真实性捕获技术(以下共同待审申请中的申请的主题:2004 年9 月15 日申请的第10/942,609 号“Apparatus and method for capturing the motion of a performer,,;2004 年9 月15 日申请的第10/942,413 号"Apparatus and method for capturing theexpression of a performer,,;2005 年2 月25 日申请的第11/066,954 号“Apparatusand method for improving marker identification within a motion capturesystem,,;2005年3 月10 日申请的第11/077,628 号"Apparatus and method forperforming motion capture using shutter synchronization,,;2005 年10 月20 El 串i青白勺第11/255, 854 ^ "Apparatus and method for performing motion captureusing a random pattern on capture surfaces,,;2006 年6 月7 日申请的第11/449,131 号“System and method for performing motion capture usingphosphor application techniques"; 2006 ^ 6 7 H ^itWH 11/449, 043 ^-"System and method for performing motion capture by strobing af lu Figure 22 shows the use Contour ™ Reality Capture technology (the co-pending application, the subject of the trial: September 15, 2004 Application No. 10 / 942,609 "Apparatus and method for capturing the motion of a performer ,, ; September 15, 2004 application No. 10 / 942,413 "Apparatus and method for capturing theexpression of a performer ,,; February 25, 2005 application No. 11 / 066,954" Apparatusand method for improving marker identification within a motion capturesystem ,,; March 10, 2005 application No. 11 / 077,628 "Apparatus and method forperforming motion capture using shutter synchronization ,,; October 2005 20 El sequence i plainer spoon of 11/255 , 854 ^ "Apparatus and method for performing motion captureusing a random pattern on capture surfaces ,,; June 7, 2006 application No. 11 / 449,131" System and method for performing motion capture usingphosphor application techniques "; 2006 ^ 6 7 H ^ itWH 11/449, 043 ^ - "System and method for performing motion capture by strobing af lu orescentlamp" ;2006 年6 月7 日申请的第11/449,127 号"System and method for threedimensional capture of stop-motion animated characters”,上述申请中的每一者都被转让给本CIP申请的受让人)捕获的现场直播的表演如何导致非常平滑的捕获表面,既而达成高多边形计数的追踪表面(也即,多边形运动精确地追随面部的运动)。 orescentlamp "; June 7, 2006, Application No. 11 / 449,127" System and method for threedimensional capture of stop-motion animated characters ", each of the above applications are assigned to the assignee of the present CIP application how people) captured live performances led to the capture very smooth surface, subsequently track surface reached a high polygon count (ie, polygon motion precisely follow the movement of the face). 最后,当将现场直播的表演的视频映射于追踪表面上以产生纹理表面时,产生照片般逼真的结果。 Finally, when the live performances of the video mapping on the track surface to create a textured surface to produce photo-realistic results.

[0290] 尽管当前GPU技术能够再现追踪表面及纹理中的许多多边形且实时地照明该表面,但若多边形及纹理每一帧时间改变(其将产生最具照片般逼真感的结果),则其将迅速地消耗现代PC或视频游戏控制台的所有可用RAM。 [0290] Although current GPU technology is able to track the number of polygons and the reproducing light the surface in real-time, if the polygons and textures changing every frame time (which will produce the most realistic sense result) of surface and texture, it will quickly consume all available RAM modern PC or video game console.

[0291] 使用上文所描述的流动几何形状技术,将几何形状不断地馈送至应用程序/游戏服务器1521-1525中以使得其可不断地动画制作照片般逼真的面部从而允许产生具有几乎不能区别于现场直播的动作面部的面部的视频游戏变得实际。 Flow described [0291] using geometry techniques described above, will continuously feed geometry into the app / game servers 1521 -1525 so that they can continue photo-realistic animation allowing generating portion have almost indistinguishable in the face of live action video games to become the face of reality.

[0292] 线性内容与互动式特征的整合 Integration [0292] LINEAR CONTENT WITH INTERACTIVE FEATURES

[0293] 电影、电视节目及音频材料(统称“线性内容”)广泛地以许多形式可用于家庭及办公室用户。 [0293] films, television programs and audio material (collectively, the "linear content") are widely available in many forms for home and office users. 线性内容可在如⑶、DVD、HD-DVD及蓝光媒体的实体媒体上获取。 Linear content available on the physical media, ⑶, DVD, HD-DVD and Blu-ray media. 其也可通过来自卫星及电缆TV广播的DVR来记录。 It may also be recorded by the DVR from satellite and cable TV broadcast. 此外,其可以经由卫星及电缆TV的即付即看(PPV) 内容及以电缆TV上的视频点播(VOD)可用。 Further, it may look i.e. (PPV) content through satellite and cable TV and pay-in of video on demand (VOD) on cable TV available.

[0294] 日益增加的线性内容可经由互联网以下载的内容及流动内容可用。 [0294] increasing linear content is available to download content and flow content via the Internet. 现今,确实不存在一个能体验与线性媒体相关的所有特征的位置。 Today, there is not a really able to experience all of the features of linear media position. 举例而言,DVD及其他视频光学媒体通常具有在其他位置处不可用的互动式特征(如导演的评论、“花絮”短片等)。 For example, DVD and other video optical media typically have interactive features not available at other locations (e.g., director's commentaries, "making" movies, etc.). 在线音乐站点具有通常在CD上不可用的封面艺术及歌曲信息,但并非所有CD在线可用。 Online music sites have generally not available on the CD cover art and song information, but not all CD available online. 且与电视节目相关联的网站常常具有额外特征、博客及有时来自演员或创作人员的评论。 And television programs and websites associated often have extra features, blog and sometimes comments from the actors or creators.

[0295] 另外,在许多电影或运动事件的情况下,通常有经常连同线性媒体一起发行(在电影的状况下)或(在运动的状况下)可以被紧密地联系到真实世界事件(例如,玩家的 [0295] In addition, in the case of many movies or sports events, there is usually always together with the linear media distribution together (in the case of a movie) or (in the case of sports) may be closely linked to real-world events (for example, players

54交易)的视频游戏。 54 transactions) video games.

[0296] 主机服务210非常适合于在将全异形式的相关内容连结在一起时传送线性内容。 [0296] The hosting service 210 is well suited for the disparate forms of related content delivery of linear content in linking together. 的确,传送电影不如传送高度互动式视频游戏有挑战,且主机服务210能够将线性内容传送至家庭或办公室中的多种设备,或传送至移动设备。 Indeed, the transfer film that delivering highly interactive video games and challenges, and the hosting service 210 can transmit linear content to the home or office in a variety of devices, or to mobile devices. 图23显示用于主机服务210的例示性用户接口页面,其显示线性内容的选择。 23 shows a hosting page exemplary user interface 210 that shows a selection of linear content.

[0297] 但是,不同于大多数线性内容传送系统,主机服务210还能够传送相关的互动式成份(例如,DVD上的菜单及特征、HD-DVD上的互动式叠加层,及网站上的Adobe Flash动画(如下文所述))。 [0297] However, unlike most linear content delivery system, hosting service 210 is also able to deliver related interactive components (for example, menus and features on the DVD, the interactive overlays on HD-DVD, and Adobe on site Flash animation (as described below)). 因此,客户端设备415限制不再引入哪些特征可用的限制。 Thus, the client device 415 limitations no longer introduce limitations which features are available.

[0298] 另外,主机代管系统210能够动态且实时地将线性内容与视频游戏内容连结在一起。 [0298] Further, the hosting system 210 is able to dynamically and in real time linear content with video game content are joined together. 举例而言,若用户正观看哈利波特电影中的Quidditch(魁地奇)比赛,且决定其愿意尝试玩Quidditch,则其可仅仅单击按钮且电影将暂停且其将被立即输送到哈利波特视频游戏的Quidditch片段。 For example, if a user is watching a movie in the Harry Potter Quidditch (Quidditch) competition, and to determine their willingness to try to play Quidditch, you can just click a button and the movie will pause and it will be immediately transported to Kazakhstan Quidditch fragment Potter video games. 在玩Quidditch比赛的后,另一次单击按钮,且电影将即刻重新开始。 After playing Quidditch game, another click of the button, and the movie will start again immediately.

[0299] 在照片般逼真的图形及制作技术的情况下,其中摄影捕获的视频不能区别于现场直播的动作人物,当用户进行自现场直播的动作电影中的Quidditch游戏至主机服务上的视频游戏中的Quidditch游戏的转变时(如本文中所述),该两个场景实际上不能区别。 [0299] Under-realistic graphics and production technology in the photo case in which photography can not capture video from the live action characters, when the user from live action movie Quidditch game to the host video game service the transition of the Quidditch game (as described herein), the two scenes are virtually indistinguishable. 此为线性内容与互动式(例如,视频游戏)内容两者的导演提供全新的创作选项,因为该两个世界之间的线变得不能区别。 This linear content (eg, video games) both of which are directed to provide new creative options, because the line between the two worlds become indistinguishable interactive.

[0300] 利用图14中所显示的主机服务架构,可将3D电影中的虚拟相机的控制提供给观看者。 [0300] FIG hosting architecture 14 shows, 3D movie of the virtual camera control can be provided to the viewer. 举例而言,在发生于列车内的场景中,将有可能允许观看者在故事进展时控制虚拟相机且环顾列车。 For example, in a scene that takes place inside the train, there will likely allow the viewer to control the virtual camera in the story progresses and look around the car. 此假定列车中的所有3D对象(“资产”)以及能够实时地再现所述场景以及原始电影的足够计算能力水平可用。 This assumes that all of the 3D objects ( "assets") in the train can be reproduced and a sufficient level of computing capability as well as the original movie scenes are available in real time.

[0301] 且甚至对于非计算机产生的娱乐,存在可提供的非常刺激的互动式特征。 [0301] And even for non-computer generated entertainment, there are interactive features that can provide a very exciting. 举例而言,2005电影“傲慢与偏见”具有装饰华丽的旧英国大厦中的许多场景。 For example, the 2005 film "Pride and Prejudice" has ornate old English mansions of many scenes. 对于特定大厦场景, 用户可将视频暂停且接着控制相机以巡视大厦,或可能的周围区域。 For certain mansion scenes, the user may pause the video and then control the camera to tour of the mansion, or perhaps the surrounding area. 为了实施这些,可携带具有鱼眼镜头的相机穿过大厦,当其追踪其位置时,非常类似于实施先前技术Apple (苹果)公司的QuickTimeVR。 To implement this, the camera can carry through the building with a fisheye lens, it keeps track of its position, much like prior art Apple (Apple) company QuickTimeVR. 各种帧接着将被转换,因此图像不失真,且接着其连同电影一起被储存于RAID阵列1511-1512上,且在用户选择继续虚拟巡视时被回放。 Various frames would then be transformed so the images are not distorted, and then along with the movie is stored on a RAID array 1511-1512, and when the user chooses to go on a virtual tour playback.

[0302] 在运动事件的情况下,可经由主机服务210来使现场直播的运动事件(诸如,篮球比赛)流动以供用户观看(如同其对于常见TV所想要的那样)。 [0302] In the case of sports events, hosting service 210 via the live sports events (such as a basketball game) flow for users to watch (as they would for regular TV as). 在用户观看特定播放之后,游戏的视频游戏(最终篮球玩家看起来与真实玩家一般照片般逼真)可赶上在同一位置中开始的玩家,且用户(可能各自控制一玩家)可重新玩以观看其是否可比所述玩家做得更佳。 After the user to view a particular player, game video game (final basketball players looking photo-realistic real players in general) can catch up with the players starting in the same location, and the user (a player may control their own) can play to watch again whether it is better than the players do.

[0303] 本文中所描述的主机服务210极其适合于支持此未来世界,因为其能够承受不切实际以致不能安装于家庭中或大多数办公室背景中的计算能力及大容量储存资源,而且其计算资源总是最新的(在可用的最新的计算硬件的情况下),但是在家庭背景中,将总是存在具有较旧代的PC及视频游戏的家庭。 [0303] hosting services described herein 210 ideally suited to support this futuristic world, because it can withstand impractical to install in a home or office in the background most of the computing power and mass storage resources, and its calculation resources always up to date (in the case of the latest available computing hardware), but in a home setting, will always be homes with older generation of PC and video games. 此外,在主机服务210中,用户被隐瞒所有此计算复杂度,因此,即使用户可能正使用非常尖端的系统,自用户的观点看,也如改变电视上的信道一般简单。 Further, in the hosting service 210, the user is concealed all of this computing complexity, and therefore, even though the user may be using very sophisticated systems, from the user's point of view, but also as changing channels on a television simple. 另外,用户将能够存取所有计算能力及计算能力将自任何客户端415带来的体验。 In addition, users will be able to access all the computing power and computing capacity from 415 to bring the experience to any client.

[0304] 多人游戏 [0304] Multiplayer

[0305] 对于游戏为多人游戏的程度,则游戏将能够不仅经由入埠路由1502网络传达到应用程序/游戏服务器1521-1525,而且通过网络桥接器传达至具有不在主机服务210中执行的服务器或游戏机器的互联网(未图示)。 [0305] extent the game is a multiplayer game, the game will be able communicate through the inbound routing network 1502 to app / game server 1521-1525, and communicate through a network bridge to have a hosting service 210 server is not executed a game machine or the Internet (not shown). 当通过通用互联网上的计算机玩多人游戏时,则应用程序/游戏服务器1521-1525将具有极快访问互联网的益处(与游戏在家庭中的服务器上执行的情况相比),但其将受在较缓慢连接上玩游戏的其他计算机的能力限制, 且也潜在地受互联网上的游戏服务器被设计以适应最少共同点(common denominator)(所述游戏服务器是相对较慢的消费者互联网连接上的家庭计算机)的事实限制。 When playing multiplayer computer games via the Internet on the general, then the app / game server 1521-1525 will have the benefit of extremely fast access to the Internet (compared with the case of a game executed on the home server), but it will be subject to the ability of other computers to play games on a slow connection limits, and also potentially (the game server is relatively slow Internet connection on consumers by the game server on the Internet are designed to accommodate the least common denominator (common denominator) home computer) the fact that limit.

[0306] 但当完全在主机服务210服务器中心内玩多人游戏时,则可达到极大差异。 [0306] However, when fully in the hosting service 210 server center when playing a multiplayer game, you can achieve a great difference. 对用于用户的游戏进行主机代管的每个应用程序/游戏服务器1521-1525将与其他应用程序/ 游戏服务器1521-1525以及用极高速度、极低延时连接性及巨大、非常快的储存阵列对针对多人游戏的中央控制进行主机代管的任何服务器互连。 Each application on the user's game for hosting / game server 1521-1525 will work with other applications / game servers 1521-1525 and with high speed, low latency connectivity and vast, very fast storage arrays central control for the multiplayer game as any servers that are hosting. 举例而言,若超高速以太网用于入埠路由1502网络,则应用程序/游戏服务器1521-1525将在彼此当中传达,且传达到以十亿比特/秒速度在潜在的仅1毫秒或1毫秒以下的延时下对针对多人游戏的中央控制进行主机代管的任何服务器。 For example, if Gigabit Ethernet is used for the inbound routing 1502 network, then the app / game server 1521 -1525 will be communicating among each other and communicating to at gigabit / second speed with potentially only 1 ms or 1 central control for the multiplayer game servers hosting any case the delay in milliseconds or less. 另外,RAID阵列1511-1512将能够非常快速地响应且接着以十亿比特/秒速度传送数据。 Also, RAID arrays 151 1 -1512 will be able to respond very rapidly and then transfer data at gigabit / second speeds. 作为一个实例,若用户在外表及服装方面定制人物,以使得人物具有对于人物而言唯一的大量几何形状及行为,在限于在家庭中在PC或游戏控制台上执行的游戏客户端的先前技术系统下,若所述人物将进入另一用户的视野中,则用户将必须等待直到长的缓慢下载完成为止,以便将所有几何形状及行为数据载入其计算机中。 As an example, if the user to customize the appearance and character in the clothing, so that a large number of characters has a unique geometry and behavior for the character in the game is limited to executing on the client PC or game console in the home end of the prior art system next, if that character into another field of view of the user, the user would have to wait until a long, slow download completes so that all of the geometry and behavior data loads into their computer. 在主机服务210内,所述相同下载可优于以十亿比特/秒速度从RAID阵列1511-1512服务的超高速以太网。 In the hosting service 210, that same download could be over Gigabit Ethernet at gigabit / second speed from a RAID array 1511-1512 and services. 即使家庭用户具有8Mbps互联网连接(其根据现今的标准来看极快),超高速以太网也快100倍。 Even home users have 8Mbps Internet connection (which is fast by today's standards), Gigabit Ethernet is 100 times faster. 因此,在快的互联网连接上花费一分钟进行的工作在超高速以太网上将花费小于一秒。 Therefore, a fast Internet connection would take a minute, would take less than one second over Gigabit Ethernet.

[0307] 顶级玩家分群及锦标赛 [0307] grouping and tournament top players

[0308] 主机服务210极其适合于锦标赛。 [0308] The hosting service 210 is extremely suitable for the championship. 因为无游戏在本地客户端中执行,所以不存在用户作弊的机会。 Because no game is running on the local client, the opportunity for users to cheat does not exist. 而且,由于输出路由1540多播UDP流的能力,使得主机服务210能够同时向观众中的数千人广播较大锦标赛。 Moreover, since the output routing 1540 to multicast UDP streams, the Hosting Service 210 can simultaneously broadcast to an audience of thousands of people in major tournaments.

[0309] 事实上,当存在如此风行以致数千名用户正接收相同流的特定视频流时(例如,显示较大锦标赛的视图),可以更有效的将视频流发送到内容传送网络(⑶N)(诸如, Akamai (阿卡迈公司)或Limelight (聚光灯公司))以用于大量分配到许多客户端设备415。 [0309] In fact, when there are certain video streams that are so popular that thousands of users are receiving the same stream (e.g., showing views of a major tournament), can be more efficient to send the video stream to a Content Delivery Network (⑶N) (such as, Akamai (Akamai) or Limelight (spotlight Corporation)) for mass distribution to many client devices 415.

[0310] 当使用CDN来显示顶级玩家分群的游戏取景器页面时,可获得类似水平的效率。 [0310] When using a CDN to display the top player groupings Game Finder page, obtain a similar level of efficiency.

[0311] 对于较大锦标赛,可使用现场直播的名人解说员来在特定比赛期间提供评论。 [0311] For larger tournaments, you can use a live celebrity announcer to provide comments during a particular game. 尽管大量用户将是在观看较大锦标赛,且相对小数目将是在锦标赛中玩。 Despite the large number of users will be watching a major tournament, and a relatively small number will be playing in the tournament. 可将来自名人解说员的音频路由到对在锦标赛中玩的用户进行主机代管且对锦标赛中的游戏的任何旁观者模式复本进行主机代管的应用程序/游戏服务器1521-1525,且可将音频加录于游戏音频之上。 Can route the audio from a celebrity narrator for the user to play in the tournament and hosting a tournament of any spectator mode copies of the game had been hosting the application / game server 1521-1525, and can be audio overdub on top of game audio. 可在游戏上(也可能刚好在旁观者视图上)叠加名人解说员的视频。 It can be in the game (perhaps just a spectator view) superimposed celebrity narrator of the video.

[0312] 网页载入的加速[0313] 全球信息网的主要传送协议、超文本传输协议(HTTP)是被构想并被限定在其中仅商业具有高速互联网连接,且在线的消费者使用拨号数据机或ISDN的时代中。 [0312] page load acceleration [0313] The main global information network transport protocol, Hypertext Transfer Protocol (HTTP), were conceived and defined only businesses had high speed Internet connection therein, and the consumers who were online were using dialup modems or the ISDN era. 此时,用于快速连接的“黄金标准”是Tl线,其对称地提供1. 5Mbps数据速率(也即,两个方向中具有相等数据速率)。 In this case, a quick-connect "gold standard" is a Tl line which provided symmetrically 1. 5Mbps data rate (i.e., two directions with equal data rate).

[0314] 现今,情形完全不同。 [0314] Today, the situation is completely different. 大量发达世界中经由DSL或电缆调制解调器连接的平均家庭连接速度具有比Tl线高得多的下行数据速率。 The average home connection speed large developed world connected via DSL or cable modem has a much higher downstream data rate than a Tl line. 事实上,在世界的一些地方中,光纤到路边(fiber-to-the-curb)正将高达50Mbps至IOOMbps的数据速率带入家庭。 In fact, in some parts of the world, the fiber to the curb (fiber-to-the-curb) is bringing data rates of 50Mbps to IOOMbps into the home.

[0315] 遗憾地,HTTP没有被架构(也没有被实施)成有效地利用该急剧的速度改善。 [0315] Unfortunately, HTTP was not architected (nor has it been implemented) to effectively take advantage of these dramatic speed improvements. 网站为远程服务器上的档案的集合。 Web site is a collection of files on the remote server. 非常简单地说,HTTP请求第一档案,等待下载该档案,且接着请求第二档案,等待下载该档案等。 In very simple terms, HTTP requests the first file, waits for the file downloaded, and then requests the second file, waits for the file to download the like. 事实上,HTTP允许一个以上“开放连接”(也即,每次请求一个以上档案),但由于商定的标准(及防止网络服务器被超载的愿望)而使得仅准许非常少的开放连接。 In fact, HTTP allows for a more "open connection" (that is, more than one file per request), but due to agreed standards (and to prevent the network server is overloaded desire) so that permit only very few open connections. 此外,由于网页被构造的方式,浏览器常常未意识到可用于立即下载的多个同时页面(也即,仅在剖析一个页面之后才变得显而易见:需要下载如图像的新档案)。 In addition, due to the way Web pages are constructed, the browser more often not immediately available for download at the same time aware of the page (that is, only after parsing a page does it become apparent: you need to download the image as a new file). 因此,网站上的档案实质上是逐个地载入。 Therefore, the file on the site is essentially a loaded one by one. 此外,由于由HTTP使用的请求及响应协议,存在与所载入的每个档案相关的大致(访问美国的典型网络服务器)100毫秒的延时。 Further, since the request and response protocol used by HTTP, there is roughly (accessing typical web servers in the US) a 100ms latency associated with each file that is loaded.

[0316] 在相对低速连接的情况下,这不会引入大量问题,因为用于档案本身的下载时间决定网页的等待时间。 [0316] In the case of relatively low speed connections, this does not introduce much problem because the download time for the files themselves dominates the waiting time page. 但是,随着连接速度增大(尤其是复杂网页情况下),开始引起问题。 However, as connection speeds increase (especially complex web pages), began to cause problems.

[0317] 在图24中所显示的实例中,显示了典型商业网站(该特定网站来自较大的运动鞋商标)。 [0317] In the example shown in Figure 24, a typical commercial website (this particular website was from a major athletic shoe brand). 网站上具有54个档案。 It has 54 files on the site. 档案包括HTML、CSS、JPEG、PHP、JavaScript及Flash档案,且包括视频内容。 Archives, including HTML, CSS, JPEG, PHP, JavaScript and Flash files, and include video content. 在现场直播网页(也即,用户可单击该网页并开始使用该网页)之前,必须载入总共1.5M字节。 Before the web page is live (ie, the user can click on it and start using the web page), you must load a total of 1.5M bytes. 对于大量档案存在许多原因。 For many reasons there is a large number of files. 首先,该网页为复杂且尖端的网页,且其次,该网页为基于关于存取该页面的用户的信息动态地组合的网页(例如,用户来自哪个国家,何种语言,用户之前是否进行购买等),且视所有这些因素而下载不同档案。 First of all, this is a complex and sophisticated web pages, and secondly, the web page is based on information about the user accessing the page dynamically combination (for example, users from any country, whether to make a purchase before what language, users, etc. ), and depending on all these factors and download different files. 但是,其仍为非常典型的商业网页。 However, it is still a very typical business website.

[0318] 图24显示随着连接速度增大在现场直播网页之前经过的时间量。 [0318] Figure 24 shows the amount of time before the connection speed grows through the web page is live. 在1. 5Mbps连接速度2401下,使用具有传统网络浏览器的传统网络服务器,在现场直播网页之前花费13. 5 秒。 1. 5Mbps the connection speed 2401, using a conventional web server with a convention web browser, it takes before the web page is live 13.5 seconds. 在12Mbps连接速度2402下,载入时间减少至6. 5秒,或约快一倍。 In a 12Mbps connection speed 2402, the load time is reduced to 6.5 seconds, or about twice as fast. 但在96Mbps连接速度2403下,载入时间仅减少至约5. 5秒。 But with a 96Mbps connection speed 2403, the load time is only reduced to about 5.5 seconds. 这个原因是因为在这种高下载速度下,下载档案本身的时间最小,但每个档案各自大致100毫秒的延时仍保持,从而导致54个档案*100 毫秒=5. 4秒的延时。 The reason why is because at such a high download speed, the time to download files themselves is minimal, but per file, roughly 100ms of latency remains, resulting in 54 files * 100ms = 5.4 seconds of latency. 因此,无论到家庭的连接多快,该网站在现场直播之前将总是花费至少5. 4秒。 Therefore, the connection to the home regardless of how fast the site before the live broadcast will always spend at least 5.4 seconds. 另一因素是服务器侧排队;每个HTTP请求是在队列的后部添加,因此在忙碌的服务器上这将具有显著影响,因为对于要从网络服务器得到的每个小项目,HTTP请求需要等待其返回。 Another factor is the server-side queuing; every HTTP request is added in the back of the queue, so on a busy server this will have a significant impact because for every small item to get from the web server, HTTP requests needs to wait thereof return.

[0319] 解决这些问题的一个方式是废弃或重新限定HTTP。 [0319] One way to solve these issues is to discard or redefine HTTP. 或者,可能使网站拥有者较佳地将其档案合并成单一档案(例如,以Adobe Flash格式)。 Alternatively, it may make the site owner to better consolidate its files into a single file (for example, in Adobe Flash format). 但是,作为一个实际问题,该公司以及许多他人在其网站架构中具有大量投资。 However, as a practical matter, the company and many others have invested heavily in its Web site architecture. 另外,尽管一些家庭具有12-lOOMbps连接,但大多数家庭仍具有较缓慢的速度,且HTTP在缓慢速度下确实工作良好。 In addition, although some families have a 12-lOOMbps connection, but most families still have a slower speed, and HTTP does work well at slow speeds.

[0320] 一个替代方法是在应用程序/游戏服务器1521-1525上对网络浏览器进行主机代管,且在RAID阵列1511-1512上(或潜在地在对网络浏览器进行主机代管的应用程序/ [0320] One alternative is to host web browsers on app / game server 1521 -1525, and on the RAID arrays 1511-1512 (or potentially in web browser application hosting /

57游戏服务器1521-1525上的RAM中或本地储存器上)对用于网络服务器的档案进行主机代管。 57 RAM or on local storage on the game server 1521 -1525) of the archives for the web servers hosting. 由于经由入埠路由1502 (或至本地储存器)的非常快的互连,并非在使用HTTP下每档案具有100毫秒的延时,而是在使用HTTP下将存在每档案最小延时。 Because through the inbound routing 1502 (or to local storage) of the very fast interconnect, not per file using HTTP having 100 ms delay, but there will be used the next minimis latency per file HTTP. 接着,并非使家庭中的用户经由HTTP存取网页,而是用户可经由客户端415存取网页。 Then, instead of the user's home page access via HTTP, but the user can access the web page through client 415.. 接着,甚至在1.5Mbps 连接下(因为此网页不需要大量带宽来用于其视频),网页也将在每个线2400小于1秒的时间内处于现场直播。 Then, even with a 1.5Mbps connection (because this web page does not require much bandwidth for its video), the webpage will be at 2400 in less than 1 second per line live. 实质上,在应用程序/游戏服务器1521-1525上执行的网络浏览器显示现场直播的页面之前将不存在延时,且在客户端415显示来自网络浏览器的视频输出之前将不存在可侦测到的延时。 In essence, executing on the app / game server 1521-1525 Web browser will display there is no delay before the live page and the client 415 will show there is no detectable before the video output from the web browser to delay. 当用户使用鼠标搜寻网页和/或在网页上键入字时,将用户的输入信息发送至在应用程序/游戏服务器1521-1525上执行的网络浏览器,且网络浏览器将相应地作出响应。 When the user mouses and / or type a word on the page to send the user's input to the web browser running on the app / game server 1521-1525, and the web browser will respond accordingly.

[0321] 此方法的一个不利之处是:若压缩器正恒定地传输视频数据,则使用带宽,即使网页变成静态也如此。 One disadvantage [0321] This method is: if the compressor is constantly transmitting video data, then bandwidth is used, even if the web page becomes static. 这可通过配置压缩器以仅在(并且如果)网页改变时才传输数据且接着仅将数据传输到发生改变的页面部分来补救。 This is accomplished by configuring the compressor to only (and if) the web page changes, and then, only transmit data when transmit data to the parts of the page changes to remedy. 当存在具有恒定地改变的闪烁标语等的一些网页时,所述网页往往令人讨厌,且除非存在要移动某物(例如,视频剪辑)的原因,否则通常网页为静态的。 When some web pages with flashing banners, etc. that are constantly changing, the web tend to be annoying, and unless there is a reason for something to be moving (e.g., a video clip), or usually web pages are static. 对于所述网页,可能为以下状况:使用主机服务210将传输较少数据(与传统网络服务器相比),因为将仅传输实际显示的图像,没有精简型客户端可执行代码,且没有可能从不被观看的大对象(诸如,滚动翻转图像)。 For such web pages, may be the following condition: using the hosting service 210 will transmit less data (as compared to a conventional web server), because the image actually displayed will only be transmitted, no thin client executable code, and may range from no large object is not viewed (such as rollover images).

[0322] 因此,使用主机服务210来对旧版网页进行主机代管,可将网页载入时间减少到打开网页是类似改变电视上的频道的程度:有效地即刻地现场直播该网页。 [0322] Thus, using a host service 210 to host legacy web pages, web page load time is reduced to open the web page is like changing channels on the TV: Effective immediately the web page is live.

[0323] 便于游戏及应用程序的除错 [0323] facilitate debugging games and applications

[0324] 如先前所述,具有实时图形的视频游戏及应用程序为非常复杂的应用程序且通常当其被发行到该领域中时,其含有缺陷。 [0324] As mentioned previously, video games and applications with realtime graphics are very complex applications and typically when they are released into the field they contain bugs. 尽管软件开发商将自用户得到关于缺陷的反馈,且其可能具有用于在崩溃之后将机器状态传回的一些方式,但确切地识别是什么引起游戏或实时应用程序崩溃或不适当地执行非常困难。 Although software developers will get feedback from users about the defect, and it may have some way for the state of the machine after a crash returned, but identify exactly what is causing the game or real-time application to crash or execute very inappropriately difficult.

[0325] 当游戏或应用程序在主机服务210中执行时,将游戏或应用程序的视频/音频输出恒定地记录在延迟缓冲器1515上。 [0325] When a game or application runs in the hosting service 210, the video game or application / audio output constantly recorded on a delay buffer 1515. 另外,看门狗进程执行每个应用程序/游戏服务器1521-1525,该看门狗进程将向主机服务控制系统401定期地报告应用程序/游戏服务器1521-1525正平稳地执行。 In addition, a watchdog process runs each app / game server 1521-1525, the watchdog process will be hosting service control system 401 periodically report the application / game server 1521-1525 is running smoothly. 若看门狗进程未能报告,则服务器控制系统401将试图与应用程序/游戏服务器1521-1525通信,且若成功,则将收集可用的无论什么机器状态。 If the watchdog process fails to report, the server control system 401 will attempt to communicate with the 1521-1525 application / game server, and if successful, no matter what the state of the machine will be available for collection. 将无论什么可用的信息连同由延迟缓冲器1515记录的视频/音频一起发送到软件开发商。 Whatever information is available transmitted along with the video recorded by the delay buffer 1515 / audio to software developers.

[0326] 因此,当游戏或应用程序软件开发商自主机服务210得到崩溃的通知时,其得到导致崩溃的原因的帧接帧的纪录。 [0326] Thus, when the game or application software developers from the hosting service 210 is notified of the crash, which get the record frame-by-frame caused the crash of. 该信息在追踪到缺陷并将缺陷修复中可能是极具价值的。 The information in tracking down bugs and repair may be valuable.

[0327] 还应该注意,当应用程序/游戏服务器1521-1525崩溃时,在最近的可重新启动的时刻重新启动服务器,且将消息提供给用户,从而就技术困难道歉。 [0327] It should also be noted that, when the application / game server 1521-1525 crash, restart the server can be restarted recent times, and the message is provided to the user apologizing for the technical difficulty.


[0329] 图4a及图4b中所显示的系统为终端用户与游戏及应用程序开发商两者提供多种益处。 [0329] Figures 4a and 4b in the system shown in FIG provide a variety of benefits for both end users and game and application developers. 举例而言,通常,家庭及办公室客户端系统(例如,PC或游戏控制台)仅在一周中的小百分比的小时中处于使用中。 For example, typically, home and office client system (for example, PC or game console) is in use only in a small percentage of the week hours. 根据由Nielsen(尼尔森)娱乐“Active Gamer BenchmarkStudy (活跃游戏者基准点研究)” (http://www. prnewswire. com/cgi-bin/stories, pi ? ACCT = 104&ST0RY = /www/story/10-05-2006/0004446115&EDATE =)的2006 年10 月5 日通信稿,活跃的游戏者一周平均花费14个小时来在视频游戏控制台上玩且约一周17个小时在手持设备上玩。 According to a Nielsen (Nielsen) Entertainment "Active Gamer BenchmarkStudy (Point of player base active)" (http:..? // www prnewswire com / cgi-bin / stories, pi ACCT = 104 & ST0RY = / www / story / 10- 05-2006 / 0004446115 & EDATE =) 2006 October 5 draft communication, active gamers spend an average of 14 hours one week to play on video game consoles and about 17 hours a week playing on a handheld device. 该报告还陈述:对于所有游戏播放活动(包括控制台、手持设备及PC游戏播放),活跃的游戏者平均一周13个小时。 The report also states that: For all the game play activities (including console, handheld and PC games played), active players average 13 hours a week. 考虑较高数字的控制台视频游戏播放时间,存在一周24*7 = 168个小时,这暗示在活跃游戏者的家中,视频游戏控制台仅在一周的17/168 = 10%的小时中处于使用中。 Consider higher figure of console video game playing time, there are 24 * 7 = 168 hours, which implies an active player in the home video game console only during the week of 17/168 = 10% of the hours in use in. 或者,90%的时间,视频游戏控制台是闲置的。 Or 90% of the time, a video game console is idle. 给定视频游戏控制台的高成本,及制造商资助所述设备的事实,这是昂贵资源的非常无效率的使用。 To high costs, and given the fact that the video game console manufacturers subsidize the device, which is a very inefficient use of expensive resources. 商业内的PC通常也仅在一周的一部分小时中使用,尤其是高端应用程序(诸如, Autodesk Maya)常常所需的非便携式台式PC。 PC within the business usually use only part of the week hours, especially non-portable high-end desktop PC applications (such as, Autodesk Maya) often required. 尽管一些商业在所有小时及假日都操作,且一些PC(例如,带回家以用于在晚上进行工作的便携式PC)是在所有小时及假日使用,但大多数商业活动倾向于在给定商业时区中集中在从周一至周五的约9AM至5PM、较少的假日以及休息时间(诸如,午餐),且因为大多数PC使用在用户积极地利用PC时出现,所以其遵循:台式PC的利用倾向于遵循这些操作小时数。 Although some businesses operate at all hours and holidays, and some of the PC (for example, home to a portable PC for work at night) are at all hours and holiday use, but most business activities tend to set in business time zone focus from about 9AM to 5PM Monday to Friday, less holidays and break times (such as lunch), and since most PC when users actively engaged with the PC, so it follows: a desktop PC the use of these operations tend to follow a few hours. 若假定一周中的五天的自9AM至5PM不断地使用PC,则这将暗示PC在一周的40/168 = 24%的小时中被使用。 Assuming a five-day week from 9AM to 5PM constantly using a PC, then this would imply that the PC is used in a 40/168 = 24% of the hours in a week. 高性能台式PC是用于商业的非常昂贵的投资,且这反映了非常低的利用度。 High-performance desktop PC is a very expensive investment for the business, and this reflects the very low bioavailability. 在台式计算机上教学的学校可在一周的甚至更小部分中使用计算机,且尽管其视教学的小时数而改变,但大多数教学在自周一至周五的日间小时期间出现。 Teaching in the school on a desktop computer can use the computer in an even smaller part of the week, and although it depends on the number of hours of teaching varies, but most of the teaching in the period from Monday to Friday, daytime hours appear. 因此,一般而言,PC及视频游戏控制台仅在一周的小部分小时中被利用。 Thus, in general, PC and video game consoles only use a small portion of the hours in a week.

[0330] 值得注意地,因为许多人在非假日的周一至周五的日间小时期间在商业或在学校工作,所以这些人通常在这些小时期间不玩视频游戏,且因此当其确实玩视频游戏时,其通常是在其他小时期间(诸如,晚上、周末及假日)。 [0330] Notably, because a lot of people during daytime hours Monday through Friday non-holiday business or work in the school, so these people do not usually play video games during these hours, and so when they do play video the game, which is usually during other hours (such as evenings, weekends and holidays).

[0331] 给定图4a中所显示的主机服务的配置,则上述两段中所描述的使用模式导致资源的非常有效的利用。 [0331] to the configuration of the hosting service shown given in FIG. 4a, the above described use mode two paragraphs result in very efficient utilization of resources. 显而易见,存在对于可在给定时间由主机服务210来服务的用户的数目的限制,尤其在用户需要用于复杂应用程序(如尖端3D视频游戏)的实时响应性的情况下。 Clearly, there is a limit to the number of users may be at a given time by the host services 210 and services, especially in the case of real-time response to user needs for complex applications (like sophisticated 3D video game). 但是,不同于家庭中的视频游戏控制台或由商业使用的PC(其通常在大多数时间闲置放置),服务器402可由不同用户在不同时间重新利用。 However, unlike the home video game console or a PC used by the commercial (which is usually placed in the idle most of the time), the server 402 may be different users to re-use at different times. 举例而言,具有高性能双CPU及双GPU及大量RAM的高性能服务器402可由商业及学校在非假日的9AM至5PM利用,但由玩尖端视频游戏的游戏者在晚上、周末及假日利用。 For example, with high performance dual CPU and large amounts of RAM and dual GPU performance server 402 by a businesses and schools use on non-holidays 9AM to 5PM, but by gamers playing a sophisticated video game in the evenings, weekends and on holidays. 类似地,低性能应用程序可由商业及学校在商业小时期间在具有Celeron (赛扬)CPU、无GPU (或非常低端的GPU)及有限RAM的低性能服务器402上利用且低性能游戏可在非商业小时期间利用低性能服务器402。 Similarly, low-performance applications by a businesses and schools on a commercial h having Celeron (Celeron) the CPU, and a low-performance game can utilize on 402 no GPU (or a very low-end GPU) and limited RAM low-performance servers using low performance server 402 during non-business hours.

[0332] 另外,在本文中所描述的主机服务配置的情况下,资源是在数千名(若非数百万名)用户当中有效地共用。 [0332] Further, the case where the configuration of the hosting service as described herein, resources are shared efficiently among thousands (if not millions) of users. 一般而言,在线服务仅具有其总用户基础的小百分比在给定时间使用服务。 Generally small percentage, on-line service has only its total subscriber base at a given time using the service. 若考虑先前所列出的Nielsen视频游戏使用统计数据,则容易了解为什么。 Considering the previously listed Nielsen video game usage statistics, it is easy to understand why. 若活跃游戏者一周仅17个小时玩控制台游戏,且若假定游戏的峰值使用时间是在晚上(5-12AM,7*5天=35小时/周)及周末(8AM-12AM,16*2 = 32小时/周)的典型非工作、 非商业小时期间,则对于17个小时的游戏播放,一周存在35+32 = 65个峰值小时。 If active gamers play one week only 17 hours of game consoles, and if we assume that the peak usage time of the game in the evenings (5-12AM, 7 * 5 days = 35 hours / week) and weekend (8 AM-12AM,16*2 = 32 hours / week) typical non-work, non-business hours period, then for 17 hours of game play, the presence of one week 35 + 32 = 65 peak hours. 由于以下许多原因而使得难以估计系统上的确切峰值用户负载:一些用户将在峰值外时间期间玩,可能存在特定日间时间存在用户的丛集(clustering)峰值,峰值时间可受所玩游戏的 Since many of the following reasons The exact peak user load on the system is difficult to estimate: some users will play during off-peak times, there may be certain day times when there is a user clusters (Clustering) the peak times can be affected by the play

59类型(例如,孩子的游戏将可能是在晚上的较早时间玩)等影响。 Affect 59 types (for example, children's games will likely be played earlier in the evening) and so on. 但是,假定当游戏者可能玩游戏时,游戏者玩的平均小时数远小于日间的小时数,则仅主机服务210的一部分数目的用户将是在给定时间使用主机服务210。 However, when it is assumed that the player may play the game, the average number of hours played by a gamer is far less than the number of hours of the day, only a portion of the number of users the hosting service 210 will be at a given time using the hosting service 210. 为了该分析,我们假定峰值负载为12.5%。 For this analysis, we assume that the peak load was 12.5%. 因此,仅12. 5%的计算、压缩及带宽资源是在给定时间使用,从而由于资源的再使用而导致仅12. 5%的硬件成本来支持给定用户玩性能游戏的给定级别。 Thus, only 12.5% ​​of the computing, compression and bandwidth resources are used at a given time, due to reuse of resources resulting in only 12.5% ​​of the hardware cost to support a given user to play a given level of performance game.

[0333] 此外,假定一些游戏及应用程序需要比其他者多的计算能力,则可基于被用户玩的游戏或由用户执行的应用程序来动态地分配资源。 [0333] In addition, given that some games and applications need more computing power than the other person, you can dynamically allocate resources based on a user playing a game or application program executed by the user. 因此,选择低性能游戏或应用程序的用户将被分配低性能(较低廉)服务器402,且选择高性能游戏或应用程序的用户将被分配高性能(较昂贵)服务器402。 Therefore, the user selected the low-performance game or application will be allocated a low-performance (less expensive) server 402, and selects high-performance game or applications will be allocated a high-performance (more expensive) server 402. 实际上,给定游戏或应用程序可能具有游戏或应用程序的较低性能及较高性能区,且可在游戏或应用程序的区之间将用户从一个服务器402切换到另一服务器402,以保持用户在满足游戏或应用程序的需要的最低成本服务器402上执行。 Indeed, a given game or game or application may have lower-performance applications and higher-performance region, and the user can be switched from one server 402 to another server 402 between sections of the game or application to keep the user needs to perform in a game or application of the lowest-cost server 402. 注意,比单个磁盘快得多的RAID阵列405将可以被甚至低性能服务器402所用,这具有较快磁盘传送速率的益处。 Note that, much faster than a single disk, RAID arrays 405 can even low-performance servers 402, which has the benefit of the faster disk transfer rates. 因此,跨越所有所玩游戏或所使用的应用程序的每服务器402平均成本比玩最高性能游戏或应用程序的大多数昂贵服务器402的成本小得多,然而,即使低性能服务器402也会从RAID阵列405得到磁盘性能益处。 Thus, the average cost per server 402 across all applications or games played used than most expensive server to play the highest performance games or applications cost much less 402, however, even low-performance server 402 will be from the RAID array 405 derive disk performance benefits.

[0334] 另外,主机服务210中的服务器402可能只是不具有磁盘或周边接口(不同于网络接口)的PC主机板,且恰好,可向下整合成刚好具有到SAN 403的快速网络接口的单个芯片。 Single [0334] Further, the hosting service 210 server 402 may simply without a disk or peripheral interfaces (than a network interface) than a PC motherboard, and in time, may be integrated down to just having a fast network interface to the SAN 403 chip. 而且,RAID阵列405可能将在比存在磁盘的情况多得多的用户当中共用,因此每个活跃的用户的磁盘成本将远小于一个磁盘驱动器。 Also, RAID Arrays 405 likely will be shared amongst far many more than there are disks among the users, so that each active user will be far less than one disk cost disk drives. 所有该设备将可能驻留于环境上受控制的服务器室环境中的支架中。 All of this equipment will likely reside in an environmentally controlled server room environment stent. 若服务器402出故障,则其可容易地在主机服务210处进行修理或替换。 If a server 402 fails, it can be readily repaired or replaced at the host service 210. 相比之下,家庭或办公室中的PC或游戏控制台必须坚固,必须能够幸免于合理的磨损及撕裂以防被重击或降落的独立器具需要外壳,具有至少一个磁盘驱动器,必须幸免于不利的环境条件(例如,被勉强塞入具有其他用具的过热AV橱柜中),需要服务保证, 必须被封装及装运,且由可能收取零售利润的零售商来出售。 In contrast, a home or office PC or game console must be strong, must be able to survive reasonable wear and tear from being hit or landed standalone appliance, having at least one disk drive, you have to survive adverse environmental conditions (for example, being crammed into an overheated AV cabinet with other gear), service assurance needs, must be packaged and shipped, and be sold by retailers may charge a retail profits. 另外,PC或游戏控制台必须被配置以满足将在未来某一时刻使用的计算上最密集的预期游戏或应用程序的峰值性能, 即使较低性能游戏或应用程序(或游戏或应用程序的区)也可能在大多数时间玩。 In addition, PC or game console must be configured to meet the peak performance in the calculation used at some point in the future the most intensive anticipated game or application, or even lower performance gaming application area (or a game or application ) may also play most of the time. 此外, 若PC或控制台出故障,则使其得到修理是昂贵且耗时的过程(不利地影响制造商、用户及软件开发商)。 In addition, if the PC or console fails, to get it repaired is expensive and time-consuming process (adversely affecting manufacturers, users and software developers).

[0335] 因此,假定图4a中所显示的系统将相当于本地计算资源的体验的体验提供给用户,以供用户在家庭、办公室或学校中体验给定水平的计算能力,则通过图4a中所显示的架构提供所述计算能力要低廉得多。 [0335] Thus, the system shown in Figure 4a corresponds to displayed will experience local computing resources experience provided to the user, the computing power for a given level of the user experience in the home, office or school, Figure 4a through display architecture provides the computing power is much cheaper.

[0336] 消除对升级的需要 [0336] eliminate the need for upgrade

[0337] 另外,用户不必再担忧将PC和/或控制台升级以玩新游戏或处理较高性能的新应用程序。 [0337] In addition, users no longer have to worry about upgrading the PC and / or console to play new games or handle higher performance new applications. 主机服务210上的任何游戏或应用程序(不管所述游戏或应用程序需要何类型的服务器402)均可为用户所用,且所有游戏及应用程序接近即刻地执行(也即,快速地从RAID阵列405或服务器402上的本地储存器载入)且适当地具有最新更新及缺陷修复(也即,软件开发商将能够选择用于执行给定游戏或应用程序的服务器402的理想服务器配置,且接着将服务器402配置有最佳驱动器,且接着随着时间的推移,开发商将能够同时将更新、缺陷修复等提供给主机服务210中的游戏或应用程序的所有复本)。 Any game or applications on the hosting service 210 (either the game or application needs what type of server 402) may be used by the user, and all games and applications run nearly instantly (i.e., rapidly from the RAID array local storage on the server 402 or the load 405) and properly with the latest updates and bug fixes (i.e., software developers will be able to select a server for executing a given game or application server 402 is disposed over, and then the server 402 is configured with the optimum drive, and then as time goes on, developers will be able to provide updates, bug fixes, and other services available to all copies of hosts 210 games or applications). 实际上,在用户开始使用主机服务210之后,用户可能发现游戏及应用程序继续提供较佳体验(例如,经由更新和/或缺陷修复)且可能是以下状况:用户一年后发现新游戏或应用程序可用于利用计算技术(例如,较高性能的GPU)(其在一年前甚至不存在)的服务210上,因此对于用户而言,将不可能购买将在一年后玩游戏或执行应用程序的一年前的技术。 In fact, the user starts using the hosting service 210, the user may find games and applications continue to provide a better experience (for example, through updates and / or bug fixes) and possibly the following conditions: Users discover new game or application after one year service on the 210 program can be used to take advantage of computing technology (for example, higher-performance GPU) (which did not even exist a year ago), so for the user, it will be impossible to play games or purchase applications execute in a year a year ago the technology program. 因为玩游戏或执行应用程序的计算资源对于用户而言不可见(也即,自用户的观点看,用户仅选择开始接近即刻地执行的游戏或应用程序_更像用户改变电视上的信道),所以用户的硬件将在用户甚至未意识到升级的情况下已被“升级”。 Because computing resources to play the game or execution of the application is not visible to the user (that is, from the user's point of view, the user selects only a game or application to start nearly instantly executed _ more like the user to change the channel on the TV), so the user's hardware will have to be "upgraded" in case the user is not even aware of the upgrade.

[0338] 消除对于备份的需要 [0338] eliminating the need for backup

[0339] 对于商业、学校及家庭中的用户的另一较大问题是备份。 [0339] Another major problem for businesses, schools and home users in the backup. 若磁盘出故障,或若存在无意擦除,则储存在本地PC或视频游戏控制台中的信息(例如,在控制台的状况下,用户的游戏成果及等级)可能丢失。 If a disk fails, or if there is an inadvertent erasure of information on the local PC or video game console (for example, in the case of a console, the user's game achievements and ranking) may be lost if storage. 存在提供用于PC的手动或自动备份的许多可用的应用程序, 且可将游戏控制台状态上传至在线服务器以供备份,但通常将本地备份复制至必须储存于安全且有组织的某处的另一本地磁盘(或其他非挥发性储存设备),且由于经由典型低成本互联网连接可用的缓慢上行速度而使得对于在线服务的备份常常有限。 There is provided a number of applications available for the PC manual or automatic backup, and can be uploaded to an online state gaming console server for backup, but usually the local backup copy to be stored in a safe and organized somewhere another local disk (or other non-volatile storage device), and due to the connection via the slow upstream speed available typical low-cost Internet online backup services for the often limited. 在图4a的主机服务210下,储存于RAID阵列405中的数据可使用为本领域技术人员所熟知的先前技术RAID配置技术来配置,以使得当磁盘出故障时,将不丢失数据,且将通知在容纳出故障的磁盘的服务器中心处的技术员,且接着技术员将替换该磁盘,该磁盘接着将被自动地更新以使得RAID阵列再一次容忍故障。 In FIG. 4A is the hosting service 210, the data stored in RAID arrays 405 can be configured using prior art RAID technology known to those skilled in the art to configure such that when a disk fails, the data is not lost, and the receiving a notification of a technician at the center of the failed disk server, and then will replace the disk, which then will be automatically updated so that the RAID array is once again failure tolerant. 另外,因为所有磁盘驱动器彼此接近且其间具有经由SAN 403的快速本地网络,所以在服务器中心中将所有磁盘系统配置定期地备份到次级储存器(其可储存于服务器中心处或者经易地重新定位)并不困难。 Further, since all disk drives are near one another and between them through the SAN 403 with fast local networks, so in a server center for all of the disk systems to be backed up periodically secondary reservoir (which can be stored at the server center or re placeshifted positioning) is not difficult. 从主机服务210的用户的观点看,其数据始终完全安全,且其从不必考虑备份。 210 from the hosting service user's point of view, the data is always completely safe, and it does not have to back up from consideration.

[0340] 对演示的存取 [0340] Access to demo

[0341] 用户经常希望在购买游戏或应用程序的前试用游戏或应用程序。 [0341] Users often want to try a game or application before buying a game or application. 如先前所述,存在先前技术装置,通过该先前技术装置来演示(“演示”的动词形式意思是试用演示版本,演示版本也被称为“演示”,但作为名词)游戏及应用程序,但其中的每一者遭受限制和/或不便利。 As described previously, there are prior art means by which to demonstrate the prior art devices ( "demo" is a verb meaning to try the demo version of the form, demo version is also called "Demo", but as a noun) games and applications, but each of which is subject to limitations and / or inconveniences. 使用主机服务210,对于用户而言,容易且便于试用演示。 Using the hosting service 210, for the user, easy and convenient to try the demo. 实际上,用户所进行的系经由用户接口(诸如,下文所描述的用户接口)选择演示且试用该演示。 Indeed, all performed by the user via a user interface (such as user interfaces described below) and try out the demo select the demo. 演示将几乎即刻地载入适合于该演示的服务器402上,且其将完全类似任何其他游戏或应用程序而执行。 The demonstration will load almost instantly suited to the presentation server 402, and it will be exactly like any other game or application and execution. 无论演示需要非常高性能的服务器402还是低性能的服务器402,且无论用户使用的家庭或办公室客户端415是何类型,自用户的观点看,演示均将工作。 Whether demo requires very high performance server 402 or 402 low-performance server and client regardless of home or office users is what type of end 415, from the user's point of view, the demo will work. 游戏演示或应用程序演示的软件出版商将能够确切地控制准许用户试用何演示及试用多长时间,且当然,演示可包括为用户提供获得对所演示的游戏或应用程序的全版本的存取机会的用户接口要素。 Game demo or presentation applications software publishers will be able to control exactly what permit users to try the demo and try for how long, and of course, the presentation may include provide users gain access to the full version of the demo game or application of the user interface elements of chance.

[0342] 因为演示可能是低于成本价或免费提供,所以一些用户可能试图使用重复的演示(尤其是重复地玩可能有趣的游戏演示)。 [0342] Since the presentation may be provided free or below cost, so some users may try to use demos repeated (in particular, repeatedly playing the game may be interesting demo). 主机服务210可使用各种技术来限制用于给定用户的演示使用。 The hosting service 210 can employ various techniques to limit demo use for a given user. 最直接的方法是建立用于每个用户的用户ID且限制允许给定用户ID播放演示的次数。 The most direct way is to create a user ID for each user and limit the number of times a given user ID allows playback of the presentation. 然而,用户可设置多个用户ID,尤其是其是自由的情况下。 However, the user may be provided a plurality of user ID, which is a particular case of freedom. 用于解决此问题的一个技术是限制允许给定客户端415播放演示的次数。 A technique for solving this problem is to limit the number of times a given client allows 415 players presentation. 若客户端为独立设备,则该设备将具有一序号,且主机服务210可限制演示可由具有所述序号的客户端存取的次数。 If the client is a standalone device, then the device will have a serial number, and the hosting service 210 can limit the number of times a demo can be accessed by a client of the number of. 若客户端415正以PC或其他设备上的软件执行,则可由主机服务210来指派序号且将该序号储存于PC上并使用该序号来限制演示使用,但假定PC可由用户来重新程序化,且序号被擦除或改变,则另一选项是主机服务210保持PC网络适配器媒体访问控制(MAC)地址(和/ 或其他机器特定识别符,诸如硬盘驱动器序号等)的纪录并将演示使用限制于该MAC地址。 If the client 415 is running as software on a PC or other device, may be assigned by the hosting service 210 and the serial number stored on the PC and used to limit demo use, but it is assumed that the user PC may be reprogrammed, and the serial number erased or changed, another option is for the hosting service 210 held PC network adapter media access control (MAC) address (and / or other machine specific identifiers such as hard-drive serial numbers, etc.) and limit demo usage record to the MAC address. 假定可改变网络适配器的MAC地址,然而,这并非极简单的方法。 It may be assumed to change MAC addresses of network adapters, however, this is not a foolproof method. 另一方法是限制演示可被播放到给定IP地址的次数。 Another approach is to limit the number of presentation can be played to a given IP address. 尽管可由电缆调制解调器及DSL提供者来周期性地重新指派IP地址,但其在实践中不会非常频繁地发生,且若可确定(例如,通过联系ISP) IP是处于用于住宅DSL或电缆调制解调器存取的IP地址的区块中,则通常可建立用于给定家庭的小数目的演示使用。 Although provided by cable modem and DSL may be periodically re-assigned IP address, it does not happen in practice very frequently, and if it can determine (e.g., by contacting the ISP) in the IP is used for residential DSL or cable modem block IP address access, it can usually be established for a given number of small family demo use. 而且,在家庭中在共用同一IP地址的NAT路由器之后可能存在多个设备, 但通常在住宅背景中,将存在有限数目的所述设备。 Moreover, the presence in the home after a NAT router sharing the same IP address may be multiple devices, but typically in a residential setting, there will be a limited number of the device. 若IP地址是处于服务商业的区块中, 则可建立用于商业的较大数目的演示。 If the IP address is in the service business block, you can establish a large number of commercial presentation. 但是,最后,所有先前所述方法的组合是限制PC上的演示的数目的最佳方式。 But eventually, all combinations of the previously described method is the best way to limit the number of presentations on a PC. 尽管可能不存在使得所确定的且技术上熟练的用户可能在重复播放演示的数目中受到限制的极简单的方式,但建立大量障碍可建立足够阻碍以使得大多数PC用户不值得费神去滥用演示系统,且相反,其在其意欲试用新游戏及应用程序时使用演示。 Although there may not be such that the determined and technically very simple way skilled users may be limited in the number of repeat presentation but the establishment of a large number of obstacles can hinder the establishment of enough so that most PC users are not worth the bother to abuse demo system, and on the contrary, when it is intended to use the demo to try out new games and applications.

[0343] 对学校、商业及其他机构的益处 [0343] benefits to schools, business and other institutions

[0344] 显著益处尤其出现在利用图4a中所显示的系统的商业、学校及其他机构。 [0344] Significant benefits accrue particularly to businesses, schools and other institutions in the system shown by FIG. 4a. 商业及学校具有与安装、维护及升级PC相关联的实质成本,尤其当谈及执行诸如Maya的高性能应用程序的PC时。 Businesses and schools have with the installation, maintenance and real costs associated with PC upgrade, particularly when it comes to perform a high-performance PC applications such as Maya. 如先前所陈述,PC通常仅在一周的小时的一部分中被利用,且如在家庭中, 具有给定水平的性能能力的PC在办公室或学校环境中的成本远高于在服务器中心环境中的成本。 As previously stated, PC are generally utilized only in a portion of the hours of the week, and as in the family, with a given level of performance capability cost PC in the office or school environment is much higher than in the server center environment cost.

[0345] 在较大商业或学校(例如,大的大学)的状况下,所述实体的IT部门设置服务器中心且维护经由LAN级连接而远程地存取的计算机可以是实际的。 [0345] under conditions in larger businesses or schools (for example, large universities), IT departments of the entity set up and maintain the central server connected via a LAN level and the computer can be accessed remotely practical. 存在用于经由LAN 或经由办公室之间的私用高带宽连接而远程存取计算机的许多解决方法。 The presence of a number of solutions via the LAN or through a private high bandwidth connection between offices and remote access of computers. 举例而言, 通过Microsoft的Windows终端机服务器,或者通过虚拟网络计算应用程序(如来自RealVNC(远程控制)有限公司的VNC)或者通过来自Sun Microsystems (太阳计算机系统公司)的精简型客户端装置,用户可获得对PC或服务器的远程存取,在图形响应时间及用户体验中具有一定范围的质量。 For example, by Microsoft's Windows Terminal Server, or through virtual network computing applications (such as from RealVNC (remote control) Limited VNC) or through thin client means from Sun Microsystems (Sun Microsystems), and users get to access a remote PC or server, with a range of quality in graphics response time and user experience. 另外,所述自行管理的服务器中心通常专用于单个商业或学校,且因此不能够利用在全异应用程序(例如,娱乐及商业应用程序)在一周的不同时间利用同一计算资源时所可能的使用的重叠。 Further, the self-managed server centers are typically dedicated to a single business or school, and therefore can not take advantage of the computing resources disparate applications (e.g., entertainment and business applications) utilize the same at different times of the week is possible to use It overlaps. 因此,许多商业及学校缺乏独立设置具有至每一用户的LAN速度的网络连接的服务器中心的规模、资源或专门技能。 So, many businesses and schools lack separate settings for each user to have LAN speed server-centric network connection size, resources or expertise. 实际上,大百分比的学校及商业具有与家庭相同的互联网连接(例如,DSL、电缆调制解调器)。 In fact, a large percentage of schools and businesses have the same home Internet connection (eg, DSL, cable modem).

[0346] 然而,所述组织仍可能具有对于非常高性能的计算的需要(或者定期地或者周期性地)。 [0346] Yet such organizations may still have the need for very high- performance computing (or periodically or periodically). 举例而言,小建筑公司可能仅具有小数目的建筑师,当进行设计工作时,具有相对适度的计算需要,但其可能周期性地需要非常高性能的3D计算(例如,当建立用于客户端的新建筑设计的3D飞越时)。 For example, a small architectural firm may have only a small number of architects, when doing design work, with relatively modest computing needs, but it may periodically require very high-performance 3D computing (e.g., when the establishment of a client when the new building is designed to end a 3D fly). 图4a中所显示的系统极其适合于所述组织。 The system in Figure 4a is extremely well suited for such organizations. 所述组织仅需要为提供至家庭的同一种类的网络连接(例如,DSL、电缆调制解调器)且通常非常低廉。 The tissue is only required to provide the same type of home network connection (e.g., DSL, cable modems) and are typically very inexpensive. 其可利用低廉的PC作为客户端415,或者完全没有PC也可以,而利用简单实施控制信号逻辑413及低延时视频解压缩412的低廉的专用设备。 It can utilize inexpensive PC as the client 415, or no PC can, with a simple embodiment of the control signal logic 413 and low-latency video decompression 412. inexpensive dedicated devices. 该特征对于可能具有PC的偷窃或对PC 内的专用组件的损坏的问题的学校特别有吸引力。 This feature may be a problem for theft or damage to specific components within the PC has a PC of schools particularly attractive. [0347] 这种配置解决了用于所述组织的许多问题(且许多这种优点也为进行通用计算的家庭用户共用)。 [0347] Such an arrangement solves a number of problems for such organizations (and many of these advantages are also shared by home users for general purpose computing). 举例而言,操作成本(其最终必须以某种形式传递回至用户以便具有可行的商业)可能低得多,因为(a)计算资源是与在一周中具有不同峰值使用时间的其他应用程序共用,(b)所述组织可仅在需要时获得(且招致成本)对高性能计算资源的存取, (c)所述组织不必提供用于备份或以其他方式维护高性能计算资源的资源。 For example, the operating cost (which ultimately must be passed back in some form to the users in order to have a viable business) can be much lower because (a) the computing resources are shared with different peak week time using other applications , (b) the organizations can gain only when needed (and incur the cost of) high performance computing resources accessed, (c) the organizations do not provide a backup resource or otherwise maintaining the high performance computing resources.

[0348] 盗版的消除 [0348] eliminate piracy

[0349] 另外,游戏、应用程序、互动式电影等可能不再如现今这样被盗版。 [0349] In addition, games, applications, interactive movies may no longer be pirated as they are today. 因为游戏是在服务器中心处执行,所以用户不具备对于基本程序码的存取,因此不存在盗版。 Because game is executed at the service center, so the user does not have access to the underlying program code, so there is nothing to pirate. 即使用户将要复制原始码,用户也不能够在标准游戏控制台或家庭计算机上执行该码。 Even if the user is to copy the source code, the user is not able to execute the code on a standard game console or home computer. 此打开了标准视频游戏不可用的世界各地(诸如,中国)的市场。 This open standard video games around the world are not available (such as China) market. 已使用的游戏的重新销售也是不可能的。 Re-sale of used games is also not possible.

[0350] 对于游戏开发商而言,如同现今的状况,存在较少市场不连续性。 [0350] For game developers, as is the case today, there are fewer market discontinuities. 与全新的一代技术迫使用户及开发商升级且游戏开发商取决于硬件平台的及时传送的当前情形对比,可随着时间随着游戏要求改变而逐渐地更新主机服务210。 And a new generation of technology users and developers to upgrade and forcing game developers depend on the current situation timely delivery of hardware platforms contrast, can change over time as the game requires progressively update the host service 210.


[0352] 以上描述提供由以通用互联网为基础的低延时流动互动式视频(其隐含地也包括连同视频一起的音频,如本文中所使用)的新颖基本概念致能的多种应用。 [0352] described above to provide a novel basic concept of general Internet-based, low-latency streaming interactive video (which implicitly also includes audio together with the video, as used herein) in a variety of applications enabled. 经由互联网而提供流动视频的先前技术系统仅具有可通过高延时互动实施的所致能的应用。 Provided streaming video via the Internet prior art systems having only applications which can implemented with high latency interactions. 举例而言,用于线性视频的基本回放控制(例如,暂停、回倒、快进)在高延时下适当地工作,且有可能在线性视频馈送当中进行选择。 For example, basic playback controls for linear video (e.g. pause, rewind, fast forward) work properly at high latency, and there is possible to select among linear video feeds. 此外,如先前所陈述,一些视频游戏的性质允许其以高延时来播放。 In addition, as previously stated, some of the properties of video games allow them to play with high latency. 但是,用于流动视频的先前技术方法的高延时(或低压缩比率)严重限制流动视频的潜在应用或使其部署变窄到专门化的网络环境,且甚至在所述环境中,先前技术也引入网络上的实质负担。 However, the potential applications of streaming video for the prior art method of high latency (or low compression ratio) or severely restrict flow of video narrowed their deployments to specialized network environments, and even in such environments, prior art techniques introduce substantial burdens on the network. 本文中所描述的技术打开了在经由互联网的低延时流动互动式视频下可能的多种应用的大门,尤其是经由消费者级互联网连接而致能的所述应用。 Techniques described herein may be opened at the low-latency streaming interactive video through the Internet door variety of applications, particularly those enabled the application is connected via a consumer-grade Internet.

[0353] 实际上,在与图4c的客户端465 —般小的客户端设备下,足以通过有效的任意量的计算能力、任意量的快速储存及强大服务器之间的极快网络连接而提供增强的用户体验,其使新的计算时代成为可能。 [0353] Indeed, with client 465 of Figure 4c the end - as a client device under a small, effective enough to pass any amount of computing power, fast network between arbitrary amount of fast storage, and a server connected to provide strong enhanced user experience, which makes possible a new era of computing. 另外,因为带宽要求并不随着系统的计算能力增长而增长(也即,因为带宽要求仅关于显示分辨率、质量及帧速率),所以一旦宽带互联网连接性是普遍存在的(例如,经由分布广的低延时无线涵盖)、可靠的且具有足以满足所有用户的显示设备422的需要的足够高的带宽,则问题将是典型消费者及商业应用所必要的是厚重客户端(诸如,执行Windows、Linux、OSX等的PC或移动电话)还是甚至精简型客户端(诸如,Adobe Flash 或Java)。 Further, because the bandwidth requirements do not grow as the computing power of the system grows (i.e., because the bandwidth requirements are only tied to display resolution, quality and frame rate), once broadband Internet connectivity is ubiquitous (e.g., through widespread the low-latency wireless coverage), reliable and has enough to meet the needs of all users display device 422 is high enough bandwidth, then the problem would be the typical consumer and commercial applications is essential that the thick client (such as Windows execution , Linux, OSX such as a PC or mobile phone) or even thin clients (such as, Adobe Flash or Java).

[0354] 流动互动式视频的出现导致关于计算架构的结构的假定的重新考虑。 [0354] advent of streaming interactive video led to a rethinking of assumptions about the structure of the computing infrastructure. 该一个实例是图15中所显示的主机服务210服务器中心实施例。 The one example is displayed in Figure 15 the host service 210 server center embodiment. 用于延迟缓冲G器和/或分群视频1550的视频路径是反馈回路,其中应用程序/游戏服务器1521-1525的经多播的流动互动式视频输出经由路径1552而实时地或者经由路径1551在可选择的延迟之后被反馈回到应用程序/游戏服务器1521-1525中。 G for delay buffer and / or group video 1550 is a feedback loop of the video, wherein the app / game server 1521 -1525 by multicast streaming interactive video output in real time via path 1552 or via path 1551 may be after selecting the delay is fed back to the application / game servers 1521-1525. 这使得通过先前技术服务器或本地计算架构将是不可能或不可行的多种实际应用(例如,诸如图16、图17及图20中所说明的应用)成为可能。 This makes it possible through prior art server or local computing architectures will be possible or feasible many practical applications (e.g., 16, 17 such as those illustrated and described in FIG. 20). 但是,作为更一般的架构特征,反馈回路1550所提供的是流动互动式视频水平下的递归,因为可在应用程序需要视频时将视频无限地循环。 However, as a more general architectural feature, what feedback loop 1550 provides is recursion at the streaming interactive video level, since video can be looped back indefinitely as the application requires the video. 这使得之前从未可用的多种应用可能性成为可能。 This enables a variety of application possibilities never before available as possible.

[0355] 另一关键架构特征在于:视频流是单向UDP流。 [0355] Another key architectural feature in that: the video streams are unidirectional UDP streams. 这有效地实现流动互动式视频的任意程度的多播(相比之下,诸如TCP/IP流的双向流将随着用户的数目增加而在来自来回通信的网络上产生越来越多的通信停滞)。 This enables effectively an arbitrary degree of streaming interactive video multicast (in contrast, such as TCP / IP streams, would generate more traffic on the network from the back- and- forth communications as the number of users stagnation). 多播是服务器中心内的重要能力,因为其允许系统对互联网用户(且实际上,世界的人口)的增长的需要作出响应以在一对多或甚至多对多基础上通信。 Multicasting is an important capability within the server center because it allows the system to Internet users (and, indeed, the world's population) growth needed to respond to the many-to-many, or even on the basis of communication. 再次,本文中所论述的说明流动互动式视频递归与多播两者的使用的实例(诸如,图16)仅为具有可能性的非常大的冰山的尖端。 Again, as discussed herein described streaming interactive video recursion and instances (such as in FIG. 16) using both a multicast only the tip of a very large iceberg of possibilities.

[0356] 在一个实施例中,本文中所说明的各种功能模组及相关联的步骤可由含有用于执行所述步骤的固线式逻辑的特定硬件组件(诸如,特殊用途集成电路(“ASIC”))或由被编程的计算机组件与定制硬件组件的任何组合来执行。 [0356] In one embodiment, the various functional modules and the associated steps may be described herein as comprising a particular hardware components (such as a specific integrated circuit (hardwired logic for performing the steps, " ASIC ")), or be performed by any combination of programmed computer components and custom hardware components.

[0357] 在一个实施例中,可将所述模组实施于诸如Texas仪器的TMS320x架构(例如, TMS320C6000, TMS320C5000,...等)的可编程数字信号处理器(“DSP”)上。 [0357] In one embodiment, the modules may be implemented in a programmable digital signal processor such as Texas Instruments' TMS320x architecture (e.g., TMS320C6000, TMS320C5000, ..., etc.) ( "DSP") on. 可使用各种不同的DSP,同时仍遵守所述基本原理。 You can use a variety of DSP, while still complying with these underlying principles.

[0358] 实施例可包括如上文所阐述的各种步骤。 [0358] Embodiments may include various steps as set forth above. 所述步骤可体现于引起通用或专用处理器执行特定步骤的机器可执行指令中。 The steps may be embodied in a general purpose or special purpose processor cause the machine to perform certain steps executable instructions. 已将与这些基本原理无关的各种元件(诸如,计算机存储器、硬碟机、输入设备)从图中省去以避免混淆相关方面。 Various elements (such as computer memory, hard drive, input devices) are not relevant to these underlying principles omitted from the figures to avoid obscuring the pertinent aspects.

[0359] 所公开的标的物的要素也可作为用于储存机器可执行指令的机器可读媒体来提供。 [0359] Elements of the disclosed subject matter can also be used as machine-executable instructions for storing the machine-readable medium is provided. 机器可读媒体可包括(但不限于)闪存、光碟、CD-ROM、DVDROM、RAM、EPR0M、EEPR0M、磁卡或光卡、传播媒体或适合于储存电子指令的其他类型的机器可读媒体。 The machine-readable medium may include (but are not limited to) a flash memory, DVD, CD-ROM, DVDROM, RAM, EPR0M, EEPR0M, magnetic or optical cards, propagation media or other type of machine is suitable for storing electronic instructions readable medium. 举例而言,本发明可作为计算机程序来下载,该计算机程序可经由通信链路(例如,数据机或网络连接)借助于体现在载波或其他传播媒体中的数据信号而从远程计算机(例如,服务器)传送到请求计算机(例如,客户端)。 For example, the present invention may be downloaded as a computer program, the computer program (e.g., a modem or network connection) via a communication link a data signal embodied in a carrier wave or other propagation medium from a remote computer (e.g., server) to a requesting computer (e.g., client).

[0360] 还应理解,所公开的标的物的要素也可作为计算机程序产品来提供,该计算机程序产品可包括在上面储存有指令的机器可读媒体,所述指令可用于程序化计算机(例如, 处理器或其他电子设备)以执行一序列操作。 [0360] It should also be understood that elements of the disclosed subject matter may also be provided as a computer program product, the computer program product may include instructions stored in the above machine-readable medium, the instructions may be used to program a computer (e.g. , a processor or other electronic device) to perform a sequence of operations. 或者,所述操作可通过硬件与软件的组合来执行。 Alternatively, the operations may be performed by a combination of hardware and software. 机器可读媒体可包括(但不限于)软盘、光盘、CD-ROM,及磁光盘、ROM、RAM、EPROM、 EEPR0M、磁卡或光卡、传播媒体或适合于储存电子指令的其他类型的媒体/机器可读媒体。 The machine-readable medium may include (but are not limited to) a flexible disk, optical disk, CD-ROM, and magneto-optical disks, ROM, RAM, EPROM, EEPR0M, magnetic or optical cards, propagation media or other type of media suitable for storing electronic instructions / machine-readable medium. 举例而言,所公开的标的物的要素可作为计算机程序产品来下载,其中程序可经由通信链路(例如,数据机或网络连接)借助于体现于载波或其他传播媒体中的数据信号而自远程计算机或电子设备传送至请求过程。 For example, elements of the disclosed subject matter may be downloaded as a computer program product, wherein the program may be by means of a data signal embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection) from the a remote computer or electronic device to a requesting process.

[0361] 另外,尽管已结合特定实施例描述所公开的标的物,但众多修改及变更将适当地处于本公开的范畴内。 [0361] Further, although a particular embodiment has been described in conjunction with the subject matter disclosed, numerous modifications and alterations are well within the scope of the present disclosure. 因此,说明书及图式应视为说明性的而非限制性的意义。 Accordingly, the specification and drawings should be regarded as illustrative rather than restrictive.

64 64

Claims (3)

  1. 一种用于执行视频压缩的计算机实施的方法,该方法包含:在图像序列中检测在该图像序列中的不同区域处发生的运动;逻辑上将该图像序列中的每一个图像再分成多个图像块,每个图像块的大小基于在放置所述图像块的区域中检测到的运动的量而被选择;以及使用第一压缩格式对所述图像序列中的每一个图像中的一个或多个图像块进行编码,且使用第二压缩格式对所述图像序列中每一个图像中的剩余图像块进行编码。 The computer-implemented method for performing video compression, the method comprising: detecting motion at different areas in the image sequence occurring in the image sequence; each of the logical sequence of images subdivided into a plurality of image blocks, each image block size based on the amount detected in the region of the image block is placed in motion is selected; and the use of a first compression format for each of the images in sequence or image block is encoded, and the sequence of images of the remaining image blocks in each image is encoded using the second compression format.
  2. 2.根据权利要求1所述的方法,其中所述第一压缩格式包含帧内编码。 2. The method according to claim 1, wherein the first compression format comprises intraframe coding.
  3. 3.根据权利要求2所述的方法,其中所述第二压缩格式包含帧间编码。 3. The method according to claim 2, wherein the second compression format comprises interframe coding.
CN 200880119436 2007-12-05 2008-12-04 System and method for compressing video by adjusting tile size based on detected intraframe motion or scene complexity CN101918956A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US99965707A true 2007-12-05 2007-12-05
US11/999,657 2007-12-05
PCT/US2008/085600 WO2009073825A1 (en) 2007-12-05 2008-12-04 System and method for compressing video by adjusting tile size based on detected intraframe motion or scene complexity

Publications (1)

Publication Number Publication Date
CN101918956A true CN101918956A (en) 2010-12-15



Family Applications (1)

Application Number Title Priority Date Filing Date
CN 200880119436 CN101918956A (en) 2007-12-05 2008-12-04 System and method for compressing video by adjusting tile size based on detected intraframe motion or scene complexity

Country Status (10)

Country Link
EP (1) EP2218037A4 (en)
JP (1) JP2011507346A (en)
KR (1) KR20100112567A (en)
CN (1) CN101918956A (en)
AU (1) AU2008333827A1 (en)
CA (1) CA2707715A1 (en)
NZ (1) NZ585904A (en)
RU (1) RU2487407C2 (en)
TW (2) TW200952496A (en)
WO (1) WO2009073825A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8926435B2 (en) 2008-12-15 2015-01-06 Sony Computer Entertainment America Llc Dual-mode program execution
US8613673B2 (en) 2008-12-15 2013-12-24 Sony Computer Entertainment America Llc Intelligent game loading
US8147339B1 (en) 2007-12-15 2012-04-03 Gaikai Inc. Systems and methods of serving game video
KR20130090898A (en) 2010-09-13 2013-08-14 소니 컴퓨터 엔터테인먼트 아메리카 엘엘씨 Dual mode program execution and loading
US8506402B2 (en) 2009-06-01 2013-08-13 Sony Computer Entertainment America Llc Game execution environments
US8888592B1 (en) 2009-06-01 2014-11-18 Sony Computer Entertainment America Llc Voice overlay
US8968087B1 (en) 2009-06-01 2015-03-03 Sony Computer Entertainment America Llc Video game overlay
US8771064B2 (en) 2010-05-26 2014-07-08 Aristocrat Technologies Australia Pty Limited Gaming system and a method of gaming
US8676591B1 (en) 2010-08-02 2014-03-18 Sony Computer Entertainment America Llc Audio deceleration
KR20190086039A (en) 2010-09-13 2019-07-19 소니 인터랙티브 엔터테인먼트 아메리카 엘엘씨 Add-on Management
CN104835187A (en) * 2015-05-19 2015-08-12 北京三六三互动教育科技有限公司 Animation editor and editing method thereof

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5608458A (en) * 1994-10-13 1997-03-04 Lucent Technologies Inc. Method and apparatus for a region-based approach to coding a sequence of video images
AU1977397A (en) * 1996-03-04 1997-09-22 Ericsson Inc. Digital communication system for adapting communications protocol based on a current communication channel condition
JP3413759B2 (en) * 1998-07-17 2003-06-09 株式会社ケンウッド Bs digital broadcasting receiver
JP3439361B2 (en) * 1999-02-04 2003-08-25 日本電気株式会社 Image coding apparatus and a moving picture transmission system
US6711283B1 (en) * 2000-05-03 2004-03-23 Aperio Technologies, Inc. Fully automatic rapid microscope slide scanner
US6999432B2 (en) * 2000-07-13 2006-02-14 Microsoft Corporation Channel and quality of service adaptation for multimedia over wireless networks
KR100915107B1 (en) * 2001-07-19 2009-09-03 톰슨 라이센싱 에스.에이. Fade resistant digital transmission and reception system
CA2669171C (en) * 2001-09-26 2012-05-01 Interact Devices System and method for communicating media signals
EP1706702A2 (en) * 2003-12-21 2006-10-04 KREMEN, Stanley H. System and apparatus for recording, transmitting, and projecting digital three-dimensional images
US20080207322A1 (en) * 2005-03-21 2008-08-28 Yosef Mizrahi Method, System and Computer-Readable Code For Providing a Computer Gaming Device
EP1768346A1 (en) * 2005-09-23 2007-03-28 Microsoft Corporation Provision of game applications across a network according to the display characteristics of a user terminal
JP4649615B2 (en) * 2006-02-16 2011-03-16 国立大学法人三重大学 Video encoding / decoding device, video encoding / decoding method, and program thereof
US8403757B2 (en) * 2006-04-13 2013-03-26 Yosef Mizrachi Method and apparatus for providing gaming services and for handling video content

Also Published As

Publication number Publication date
JP2011507346A (en) 2011-03-03
TW200935916A (en) 2009-08-16
TW200952496A (en) 2009-12-16
KR20100112567A (en) 2010-10-19
EP2218037A4 (en) 2013-06-05
RU2487407C2 (en) 2013-07-10
NZ585904A (en) 2013-06-28
RU2010127314A (en) 2012-01-10
AU2008333827A1 (en) 2009-06-11
WO2009073825A1 (en) 2009-06-11
CA2707715A1 (en) 2009-06-11
EP2218037A1 (en) 2010-08-18

Similar Documents

Publication Publication Date Title
RU2510591C2 (en) System for combining plurality of views of real-time streaming interactive video
TWI530163B (en) System and method for compressing video based on latency measurements and other feedback, and machine-readable medium
TWI475843B (en) System and method for multi-stream video compression
EP2656888B1 (en) Method for user session transitioning among streaming interactive video servers
RU2503998C2 (en) Hosting and broadcasting virtual events using streaming interactive video
AU2011247835B2 (en) System and method for remote-hosted video effects
TWI592198B (en) System and method for accelerated machine switching
US9669306B2 (en) System and method for capturing text for an online application
RU2510590C2 (en) System and method of compressing streaming interactive video
EP2411943B1 (en) System and method for multi-stream video compression using multiple encoding formats
RU2493667C2 (en) Method of combining linear content and interactive content compressed together as streaming interactive video
EP2229224B1 (en) System for reporting recorded video preceding system failures
RU2500022C2 (en) Streaming interactive video client apparatus
RU2501167C2 (en) System for streaming databases serving real-time applications used through streaming interactive video
RU2496135C2 (en) Streaming interactive video integrated with recorded video segments
RU2509419C2 (en) Method for multicasting views of real-time streaming interactive video
RU2504908C2 (en) System for collaborative conferencing using streaming interactive video
RU2493583C2 (en) System for recursive reconstruction of streaming interactive video
CA2756692C (en) System and method for utilizing forward error correction with video compression
US10130891B2 (en) Video compression system and method for compensating for bandwidth limitations of a communication channel
US8711923B2 (en) System and method for selecting a video encoding format based on feedback data
RU2525441C2 (en) Method and apparatus for time decoder
RU2496141C2 (en) System for combining recorded application state with application streaming interactive video output
US9968847B2 (en) System and method for compressing video frames or portions thereof based on feedback information from a client device
EP2450088A2 (en) System and Method for Remote-Hosted Video Effects

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1149828

Country of ref document: HK

ASS Succession or assignment of patent right

Owner name: OLA 2 CORP.


Effective date: 20130814



Effective date: 20130814

C41 Transfer of patent application or patent right or utility model
C02 Deemed withdrawal of patent application after publication (patent law 2001)
REG Reference to a national code

Ref country code: HK

Ref legal event code: WD

Ref document number: 1149828

Country of ref document: HK