WO2014124544A1 - Method and system for interactive online audience participation in multicast gaming - Google Patents

Abstract

A method is provided for interactive remote game spectating of video games played on host computers from client devices (spectators). It uses a protocol for delivery of multi-media over a peer to peer network such as Real-Time Media Flow Protocol (RTMFP protocol) or Web RTC to permit communications between non-players and players in a multi-user real-time video game, and in particular for audio and video rendering.

Description

METHOD AND SYSTEM FOR INTERACTIVE ONLINE AUDIENCE PARTICIPATION IN MULTICAST GAMING

Cross Reference To Related Application

[0001] The present application claims the benefits, under 35 U.S.C.§ 119(e), of U.S.

Provisional Application Serial No. 61/765,755 filed February 17, 2013 entitled "Method and System for Interactive Online Audience Participation in Multicast Gaming" which is incorporated herein by this reference

Technical Field

[0002] The invention relates to the field of online computer games and in particular

interactive online multi-user games.

Background

[0003] Currently video games may be played online among multiple players at separate stations or terminals, each of which has the game application. Where the players are skilled there may be sufficient interest in the real time game play to generate an audience but there is no current method for real time audience participation and feedback. In some cases the game play may be recorded and watched on Youtube, with the number of hits on Youtube reflecting the popularity of the game event.

However there is a need for a system to let an audience participate in real time in a multi-cast game.

[0004] Real-Time Media Flow Protocol (RTMFP protocol) is a protocol developed by Adobe Systems for delivery of multimedia over peer-to-peer networks. Where there are real-time live communications among peers, such as is the case with multi-user games, it uses the Flash Player to communicate video and audio among peers. It has not to date been used however to permit communications between non-players and players in a multi-user real-time video game, and in particular for audio and video rendering.

[0005] The foregoing examples of the related art and limitations related thereto are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent to those of skill in the art upon a reading of the specification and a study of the drawings. Summary

[0006] The following embodiments and aspects thereof are described and illustrated in conjunction with systems, tools and methods which are meant to be exemplary and illustrative, not limiting in scope. In various embodiments, one or more of the above-described problems have been reduced or eliminated, while other

embodiments are directed to other improvements.

[0007] The disclosed method and system provides interactive remote game spectating

(referred to herein as PlayQuests) from Windows-based computers (hosts) to client devices (spectators). It uses Real-Time Media Flow Protocol (RTMFP protocol) or

WebRTC to permit communications between non-players and players in a multi-user real-time video game, and in particular for audio and video rendering.

[0008] More particularly the invention provides a method of providing real time interactive remote game spectating of a multi-user real-time video game played on a host computer to spectating client devices to permit communications between spectators and players in the multi-user real-time video game, the host computer having audio and graphics capability, and the client devices provided with software for accessing and interactively communicating with the host computer via a computer network, comprising:

a) providing a host computer having audio and graphics capability and game software for playing a game, and software for multi-casting a broadcast of the game to multiple recipients using a protocol for delivery of multi-media over a peer to peer network;

b) providing a plurality of spectating client devices;

c) multi-casting the game play to the plurality of spectating client devices using a protocol for delivery of multi-media over a peer to peer network; d) one or more of the plurality of spectating client devices generating multimedia interactive information in connection with the game play; e) delivering the multi-media interactive information over a peer to peer network to the host computer and the plurality of clients. [0009] In addition to the exemplary aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the drawings and by study of the following detailed descriptions.

Brief Description of Drawings

[00010] Exemplary embodiments are illustrated in referenced figures of the drawings.

It is intended that the embodiments and figures disclosed herein are to be considered illustrative rather than restrictive.

[00011] Fig. 1 is a schematic diagram illustrating the network or system used to carry out the invention;

[00012] Fig. 2 is a flowchart illustrating the method of the invention;

[00013] Fig. 3 and 4 are schematic diagrams illustrating the network or system used to carry out the invention; and

[00014] Fig. 5 through 18 are images of screen displays illustrating the method of the invention.

Throughout the following description specific details are set forth in order to provide a more thorough understanding to persons skilled in the art.

However, well known elements may not have been shown or described in detail to avoid unnecessarily obscuring the disclosure. Accordingly, the description and drawings are to be regarded in an illustrative, rather than a restrictive, sense. [00016] With reference to Figure 1, the method of the invention is carried out by users via a plurality of user computer terminals 10 ("host computers"), whether desktop, laptop, mobile device or the like, and provided with the application software ("host application"), and web browser client software, accessing the system web portal site maintained by a web server 12, for registering and downloading via the Internet 14 or other computer network. Spectators use a social network hosting server 22 such as Facebook™ and may use mobile client players 11. The Facebook server 22 also accesses web server 12 via the Internet 14 and communicates with the web service server 16 for user authentication, game playing/spectating ("PlayQuests") and metrics tracking.

[00017] Host computers 10 require audio and graphics capability and sufficiently large processors and RAM. The Host computers 10 may also have the client player application, and mobile devices 11 may be used for purposes only of the Mobile Client Player applications. In that case the devices may be iOS mobile devices such as iPod Touch 4, iPhone 4, iPad 2 using OS: iOS 4.3 and Network: 1000 Kbps down, 500 Kbps up, 3G/4G/LTE; or Android mobile devices:such as ARMv7 processor with vector FPU, minimum 800MHz,

H.264 and AAC HW decoders, OS: Android 2.3 and Network: 1000 Kbps down, 500 Kbps up, 3G/4G/LTE.

[00018] With reference to Fig. 3, the PlayQuest platform is comprised of the

following:

I. Host Application 20. Captures and enables remote viewing of games (PlayQuests) on host computer 10.

2. Client Player. Facilitates remote spectating and interaction with games (PlayQuests) hosted on a host computer 10.

3. User Web Portal 30. Web site for registering a PlayQuest account, downloading the Host Application installer, and user support.

4. PlayQuest Service 40. Central web service for user authentication, PlayQuests, metrics tracking.

5. Cumulus Service 50. Rendezvous server for multicast connections.

6. Multicaster 70 for sending a multicast of the game play to spectators 11

(see Fig. 4).

[00019] As shown in Fig. 2 the steps in the method are:

1 [Host Computer] User registration.

2. [Host Computer] Install PlayQuest Host Application from Web Portal 30.

3. [Host Computer] Start PlayQuest Host Application and log in via

Facebook.

4. [Host Computer] Start a game. 5. [Host Computer] Start PlayQuest (capture and streaming).

6. [PlayQuest Service] Post update to user's Facebook profile as well as link to distribute to friends to watch.

7. [Client Play] Connect to PlayQuest stream.

8. [Client Play] Chat and interact with others watching the stream.

9. [Client Play] Create events and perform limited interaction with the game player.

10. [Host Computer] View stats and metrics and PlayQuests. User Account Registration

[00020] This is the user information gathered during the registration process, for example done on Facebook. The user selects and enters the following:

1. User name (handle).

2. Full name (first, last).

3. Avatar image (optionally replace default avatar, uploaded or URL reference).

4. Email address.

New accounts are then validated via email. Host Application

[00021] The PlayQuest Host Application is installed on a supported computer through the web portal 30. The Host Application does the following:

1. Register for PlayQuest account (one-time). User is directed to web portal for registration. See User Account Registration above.

2. Log in to PlayQuest account. With "remember me" and "sign out" options.

3. Target a game running on the host computer.

4. Encode the game video/audio.

5. Facilitate remote viewing of the game from a remote player (PlayQuest).

6. Game Identification. Captured games must be identified.

7. Metrics. Periodically, the Host Application will detect and submit basic computer information: such as OS, CPU, GPU, RAM.

8. On every PlayQuest session, the Host Application will detect and submit the PlayQuest information: such as Total time viewed (remote), Capture/encode settings (see Capture/Encode below), Faults and warnings, QoS, IP.

9. Help. For first-time users a wizard-style walk-through is provided to start a PlayQuest. An option also displays the User Support section of the User Web Portal.

Capture/Encode

[00022] For games running under DirectX 9/10/11, video is captured directly from the game video buffer. Video is encoded with H.264 in software. Audio is captured using Windows 7 & 8 "loop-back" audio. Audio is encoded with AAC. HTML5 may also be used for audio and video rendering. When a PlayQuest is started, total video+audio bit-rate is automatically adjusted to available bandwidth on current network connection on host computer. This bit-rate is clamped between a minimum and maximum range. User will be warned of expectations of quality for lower bit-rates at lower range. Realtime adjustment of bit-rate may be based on quality of service. Fig. 3 illustrates the PlayQuest video game capture 80, ffdshow video encode 82, MONOGRAM x264 encoder 84, virtual-audio-capturer 86, MONOGRAM AAC encoder 88, MONOGRAM FLV Mux 90, and PlayQuest socket writer 92.

[00023] The host computer should comply with the following minimum

specifications:

- OS: Windows Vista, 7 and 8

- CPU: Intel i5, AMD K10 Series (Opteron, Phenom)

- GPU: Radeon HD series; GeForce 8, 9, 100-series, 200-series, 300-series, 400-series and 500-series; Intel Sandy Bridge HD Graphics 2000 and 3000; DirectX 9 and Shader Model 2.0 capable

- RAM: 4 GB

- Network: 1000 Kbps down, 500 Kbps up

[00024] The host computer works optimally with the following recommended

specifications: - OS: Windows 7 and 8

- CPU: Intel i7, AMD Kl l Series (Bulldozer)

- GPU: Radeon HD series; GeForce 9, 200-series, 300-series, 400-series and 500-series; DirectX 9 and Shader Model 2.0 capable

- RAM: 8 GB

- Network: 4000 Kbps down, 2000 Kbps up

Client Player

[00025] The PlayQuest client player plays the codecs with a flash video object such as

Adobe Flash 9+. The Client Player will detect and submit basic device information: Device identifier; OS. On every PlayQuest session, the Client Player detects and submits the following PlayQuest information:

- PlayQuest session identification

- Total time played (remote)

- Faults and warnings

- Network settings (cellular data, Wi-Fi)

- QoS

- Touch UI

- IP

Help. First-time users will be displayed a wizard-style walk-through to connect to and play a PlayQuest. An option displays the User Support section of the User Web Portal.

User Web Portal

[00026] The user web portal 30 has the following functions:

1. Register for PlayQuest user account (one-time via Facebook). See User Account Registration above

2. Basic user account management. User name, name, email, password reset, avatar.

3. Download Host Application for installation on a supported host computer.

4. Version 2 mobile client. Links to downloads for Client Player applications (Apple Store and Google Play Store).

5. Basic User Support: a. Quick Start Tutorial

b. FAQs

c. Ticket-based support.

Fig. 5-18 illustrate the screens viewed by a user on a host computer. In Fig. 5 the user is presented a home page at the web portal for registering and downloading the Host Application. The web portal manages user accounts and billing, allows viewing of live games, archived games and video (see Fig. 6) and hosts user chat and forums. To sign in the user visits the web portal shown in Fig. 5. The download page is shown in Fig. 7 for downloading the Host Application. Once the application is installed the user logs in using his or her Facebook account as shown in Fig. 8. Once logged in the user sees, as shown in Fig. 9, a list 32 of Facebook friends who also have registered for the application, as well as any live multi-casts of games 34 that can be joined and watched. Fig. 10 illustrates the user selecting a game process to start a Playquest. The user opens the game and sees an overlay inserted by the system using the graphics driver, inserting a box to list spectators at 36.

Fig. 11 illustrates that various spectators 38 have joined the Playquest and are watching the game in Facebook. The spectators have chat capability and buttons to rate the play and can trigger overlays in the game. Fig. 12 illustrates an overlay 42 added to the game display using the multicast feature. Audio can also be multicast from a spectator. Fig. 13-16 illustrate text messaging sent with the client application. Fig. 16 shows that snapshots of in game content can be saved and marked up, and can then be posted to

Facebook. As shown in Fig. 17 the system can utilize a site on Facebook to share and distribute the Host Application. Fig. 18 illustrates various tools displayed to the spectator at 50 which can be used to interact with the game, add markups, drawings or comments or send text messages.

Preferably the Web portal will be minimal in functionality, and mostly for purposes of user registration and application installer downloads. The system is aimed at a large target number of registered users, with thousands of users simultaneously active online, though with scaling up for additional load capacity with be done on an as needed basis via the Azure hosting services. H.264 video encoding is preferred using CPU or GPU. Bit-rate dynamic adaptation may be used according to available bandwidth and dependent on the H.264 encoder. While a number of exemplary aspects and embodiments have been discussed above, those of skill in the art will recognize certain modifications, permutations, additions and sub-combinations thereof. It is therefore intended that the following appended claims and claims hereafter introduced are interpreted to include all such modifications, permutations, additions and sub-combinations as are within their true spirit and scope.

Claims

WHAT IS CLAIMED IS:

1. A method of providing real time interactive remote game spectating of a multi-user real-time video game played on a host computer to spectating client devices to permit communications between spectators and players in a multi-user real-time video game, said host computer having audio and graphics capability, and said client devices provided with software for accessing and interactively communicating with said host computer via a computer network, comprising:

a) providing a host computer having audio and graphics capability and game software for playing a game, and software for multi-casting a broadcast of said game to multiple recipients using a protocol for delivery of multi-media over a peer to peer network;

b) providing a plurality of spectating client devices;

c) multi-casting the game play to said plurality of spectating client devices using a protocol for delivery of multi-media over a peer to peer network;

d) one or more of said plurality of spectating client devices generating multi-media interactive information in connection with said game play;

e) delivering said multi-media interaction over a peer to peer network to said host computer and said plurality of clients.

2. The method of claim 1 wherein said protocol for delivery of multi-media over a peer to peer network is Real-Time Media Flow Protocol (RTMFP protocol) or WebRTC.

3. The method of claim 1 wherein said spectating client devices are mobile client

devices.

4. The method of claim 1 wherein spectators use a social network hosting server to

communicate with said host computer.

5. The method of claim 1 wherein said host computer encodes the game video and

audio to facilitate remote viewing of the game from a remote player.

6. The method of claim 1 wherein the client player is Adobe Flash. The method of claim 1 wherein said multi-media interaction is audio and video rendering.

A computer implemented system for providing real time interactive remote game spectating of a multi-user real-time video game played on a host computer to spectating client devices to permit communications between spectators and players in a multi-user real-time video game, said host computer having audio and graphics capability, and said client devices provided with software for accessing and interactively communicating with said host computer via a computer network, comprising:

a) a host computer having audio and graphics capability and game software for playing a game, and software for multi-casting a broadcast of said game to multiple recipients using a protocol for delivery of multi-media over a peer to peer network; b) a plurality of spectating client devices;

c) computer-implemented means for multi-casting the game play to said plurality of spectating client devices using a protocol for delivery of multi-media over a peer to peer network;

d) computer-implemented means for one or more of said plurality of spectating client devices generating multi-media interactive information in connection with said game play; and

e) computer-implemented means for delivering said multi-media interaction over a peer to peer network to said host computer and said plurality of clients.