JP2012531668A - Networked computer graphics rendering system having multiple display devices - Google Patents

Abstract

A plurality of computing platforms are networked to render graphics on a plurality of display devices, and each computing platform independently has an application for rendering graphics for display on the display device. Run. The client computing platform adds a directional offset to the view state information received from the server computing platform to position the graphics rendered by the server and client in the same world scene representation.
[Selection] Figure 1

Description

  This application is a co-pending application (Attorney Docket Number: 8720P006, Title of Invention “Networked Computer Graphics Rendering System with Multiple Display Devices for Displaying Multiple View Frustums”, 2009 Related to application and application (Attorney docket number: 8720P007, title of invention “Display of computer graphics rendering system with multiple display devices and setting of audio parameters”, filed June 26, 2009) To do.

  The present invention relates generally to computer graphics rendering systems, and more particularly to a graphics rendering system having a plurality of display devices.

  Computer graphics applications continue to improve the realism of rendered scenes in order to improve the entertainment or simulation value of computer games or CAD applications. An effect known as “immersion” contributes to the realism of the rendered scene. Immersion means moving the viewpoint of the user of the graphics application from the viewpoint of the bystander who looks at the scene to a partial viewpoint of the scene. However, it is difficult to achieve an effective immersion in resources within the limits of the most common computing platform hardware and processing. For example, many first-person and third-person computer games render a wide field of view (FOV) of approximately 60 degrees as a display on a display device. By increasing the FOV, the “tunnel vision” sensation can be reduced, but a wide FOV results in a “fish-eye” view, sacrificing detail and making it impossible to achieve the immersion effect.

  Graphics applications often require high computing power platforms such as one or more multi-core central processing units (CPUs) and multiple graphics processing units (GPUs) downstream of the CPUs. Such a graphics platform CPU typically generates a single graphics domain model, which each performs rendering to independently drive one of a plurality of display devices connected to the platform. It is processed by the GPU. However, since this method is extremely hardware intensive, the graphics platform market is small and few application titles have been created that take advantage of the capabilities of such platforms.

  Embodiments of the invention are particularly pointed out and distinctly claimed in the last part of the specification. However, embodiments of the present invention may be best understood by referring to the following description and the accompanying drawings, as well as the purpose, features, and advantages of both the mechanism and method of operation.

1 is a perspective view of a multi-display graphics rendering system according to one embodiment of the present invention in which each computing platform is a game console. FIG. FIG. 5 is a flow diagram illustrating an example of a method for rendering graphics with multiple computing platforms and display devices provided in, for example, a multi-display graphics rendering system. It is a figure which shows the example of the server view frustum corresponding to the system shown in FIG. It is a figure which shows the top view of the multi-display graphics rendering system based on embodiment of this invention. 1 is a block diagram of two multi-display graphics rendering systems communicatively connected to each other. FIG. FIG. 3 illustrates hardware and user interfaces that can be used to determine the position of a controller in connection with one embodiment of the present invention. FIG. 6 illustrates additional hardware available for instruction processing in one embodiment of the invention.

  Described herein is a system having a plurality of display devices for rendering and displaying graphics and a method for rendering and displaying graphics on the system. In the following description, specific numerical details are set forth in order to provide a thorough understanding of embodiments of the present invention. However, it will be apparent to one skilled in the art that other embodiments may be practiced without these specific details. In some instances, well-known methods, procedures, structures, and circuits have not been described in detail to avoid obscuring the present invention. Some portions of the detailed descriptions that follow are presented in terms of algorithms and symbolic representations of operations on data bits or binary digital signals within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the results to others skilled in the art.

  Some portions of the detailed descriptions that follow are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the results to others skilled in the art. An algorithm is understood herein and generally as a self-consistent sequence of steps to achieve a desired result. A step is a physical operation requiring a physical quantity. Usually, though not necessarily, these quantities take the form of storable electrical or magnetic signals that are transmitted, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for common uses, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

  As is clear from the discussion below, unless otherwise stated, discussions using words such as “processing”, “calculation”, “transformation”, “adjustment”, and “decision” throughout the detailed description Manipulates computer system registers and data represented in memory as physical (eg, electrical) quantities, and physical quantities in computer system memory or registers, or other similar information storage, communication or display devices It is understood that it refers to the operation and processing of a computer system or similar electrical computing device that translates into other data that is also expressed in the same manner.

  The present invention also relates to an apparatus for performing the procedures herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. In one embodiment, an apparatus for performing the procedures herein includes a game console (eg, Sony PlayStation®, Nintendo Wii®, Microsoft Xbox®, etc.). The computer program is, for example, any type of disk including a floppy (registered trademark) disk, an optical disk (for example, a compact disk read-only memory (CD-ROM), a digital video disk (DVD), a Blu-ray disk (registered trademark), etc.) And computer readable such as magneto-optical disks, read only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical card, or any type of medium suitable for storing electrical instructions It may be stored in a different recording medium.

  In the following description and claims, the terms “coupled” and “connected” are used along with their derivatives. It should be understood that these terms are not intended as synonyms for each other. The term “coupled” is used to indicate that two or more elements that are or are not in direct physical contact with each other cooperate or interact with each other. The term “connected” is used to indicate the establishment of communication between two or more elements coupled together.

  In an embodiment, a multi-display graphics rendering system includes a plurality of computing platforms for generating and rendering graphics for display on a display device of the system, respectively. FIG. 1 shows a perspective view of a multi-display graphics rendering system 100 according to one embodiment of the present invention where each computing platform is a game console. In another embodiment, the illustrated game consoles may each be replaced by any conventional computer having a programmable processor. As shown, the system 100 includes three game consoles, a view server console 101 and view client consoles 102 and 103, although any number of consoles may be included in the system.

  In an embodiment, each computing platform of the multi-display graphics rendering system is connected to one display device. In such embodiments, a one-to-one ratio between the computing platform and the display device is maintained regardless of the number of computing platforms included in the system. Each display device of the system is for displaying graphics generated and rendered by a computing platform connected thereto. In another embodiment, a computing platform may be connected to more than one display device, in which case the computing platform generates and renders graphics for two or more connected display devices. . In the particular embodiment shown in FIG. 2, the view server console 101 is connected to the view server display device 111 by a video data input / output link 151. Accordingly, the view server display device 111 displays graphic outputs such as the graphics objects 171 and 172 generated by the view server console 101. The view server display device 111 may be of any display technology known in the technical field. For example, it may be an LCD, a plasma display, a projection display, a cathode ray tube (CRT), etc., but is not limited thereto. Similarly, the video data input / output link 151 may utilize any technique known in the technical field for transmitting video. For example, component video, S-video, composite video, high definition multimedia interface (HDMI), and the like may be used, but the present invention is not limited thereto. View client consoles 102 and 103 are similarly connected to view client display devices 112 or 113, respectively, via video data input / output links 152 or 153, respectively. The view client display device 112 displays a graphic output such as the graphics object 172 generated by the view client console 102, and the view client display device 113 displays a graphic output such as the graphics object 173 generated by the view client console 103. Is displayed.

  In an embodiment, the display devices of the multi-display graphics rendering system have known physical positions relative to each other. As shown in FIG. 1, each of the view client display devices 112-113 has a linear position and orientation (that is, an angular position) on the table 109 with respect to the reference coordinate system of the view server display device 111. Arranged to have. In the illustrated embodiment, the view client display device 113 has an angular position rotated by θ with respect to the Y axis of the reference coordinate system of the view server display device 111, so the screens of the display devices 111 and 113 A vector N perpendicular to has an angular position offset of approximately 45 degrees and converges to the same reference center C. The view client display device 112 has a linear position along the X axis of the reference coordinate system of the view server display device 111. Since the view client display device 112 is not rotated about the Y axis, the vector N perpendicular to the screen does not converge to the point F. Nevertheless, in order to define the position of the view client display device 112 relative to the view server display device 111 in terms of physical angular position offset (ie, view angle offset), the line from reference C is displayed in the view client display. It may be the center line of the device 112. In further embodiments, the display device may be rotated about one or more of the X and Z axes of the reference coordinate system. For example, a view client display device arranged above the view server display device 111 may have an angular position that is rotated about the X axis of the reference coordinate system and tilted downward.

  In one embodiment, each computing platform of the multi-display graphics rendering system may be connected to each other via a network of communication links. The network provides a sufficiently high bandwidth, for example, to achieve a good implementation of a graphics rendering method that updates the display state about 20-100 times per second, as described elsewhere herein. It may be any network known in the art that is capable of. In the embodiment shown in FIG. 1, the network is a local area network (LAN) 110. The LAN 110 may be a conventional LAN that is common in the technical field. For example, it may be one of the Ethernet family of frame-based short-range computer network technologies, but is not limited to this.

  In one embodiment, only the computing platform, which is a physical node of the LAN 110, actively generates and renders one of the multiple displays of the system 100. In one embodiment, the view server and view client all share the same local high speed network switch. Thus, such an embodiment eliminates a wide area network (WAN) that includes a large number of physical nodes that only forward the data packets specific to the multi-display graphics rendering system without processing them. In general, many modern WANs provide insufficient bandwidth (possibly only allowing less than 10 view state updates per second) to achieve good execution of the graphics rendering methods described herein. .

  In one embodiment, at least one of the computing platforms of the multi-display graphics rendering system is displayed when viewing at least one of a plurality of display devices designed to be used by a user of the system. Combined with a controller for controlling the graphics. In certain embodiments, only one computing platform of a multi-display graphics rendering system is connected to a controller that controls graphics displayed as independent views on all of the graphics rendering system display devices. . For example, as shown in FIG. 1, the view server console 101 is connected to the controller 125 via the server controller data input / output link 126. The controller 125 may be a pointing device. For example, it may be a conventional game controller having a joystick, but is not limited thereto. The server controller data input / output link 126 may be any conventional wired or wireless communication.

  FIG. 2 shows a flow diagram illustrating an example method 200 for rendering graphics, eg, with multiple computing platforms and display devices provided in the multi-display graphics rendering system 100. The method 200 begins with each of the first, second, and nth computing platforms connected by a communication link, each starting a calibrated application to establish a server-client relationship in the system. Here, “in-system” refers to a view client-server relationship that spans only the computing platform that calibrates the multi-display graphics rendering system.

  In step 205, a first application is started on the first computing platform. In general, the first application that is started may be any type of computer application that is known in the art and emphasizes graphics. For example, it may be a computer game application, a CAD application, a video editing application, or the like, but is not limited thereto. In certain embodiments, the application that is started is a first-person or third-person perspective three-dimensional computer game application. For example, it may be a third-person viewpoint three-dimensional game SOCOM (registered trademark) issued by Sony Computer Entertainment Inc., but is not limited thereto. In step 206, a second application is started on the second computing platform. The second application to be started may be of any type as described for the first application. In certain embodiments, the second application started on the second computing platform is the same application as the first application started on the first computing platform, and both platforms are independently the same. Application (e.g., SOCOM).

  In procedure 210, for the first and second applications running on the first and second computing platforms, the first computing platform is designated as the view server and the second platform becomes the view client. Similarly, in a system having three or more computing platforms, one view server may be designated and all remaining platforms may be client platforms. The nomination of the first computing platform may be performed by a setup routine that is executed automatically or that requires user input. The configuration routine is appointed based on one or more of a comparison of processing capabilities of the first and second computing platforms, a physical relationship between display devices connected to the computing platform, and a comparison of display device characteristics. . For example, the setup routine may determine which system's display device has a larger display area or higher resolution and nominate a computing platform connected to that display device to the view server. As another example, the setting routine may be set to indicate which display device the user wants to center on the field of view (FOV). In another example, a computing platform with higher computing power, such as a Sony PlayStation 3 (R) computing platform, may be designated as a view server relative to the Sony PlayStation 2 (R) computing platform of the same system. .

  In the example procedure 215 of the method 200, the view server sets the view angular position offset (ie, viewing direction offset) to a default of zero. By zeroing the view angular position offset, the camera view rendered by the view server is by default when executed on a computing platform that has no view client-server relationship in the system (ie, a single computing platform). The camera view rendered by the application. However, in step 220, the view client sets the viewing direction offset to a non-zero value. In one such embodiment, configuration settings associated with viewing direction offsets are accessed from a storage location on the view client or view server. In one embodiment, the viewing direction offset is specific to the view client and is based on a physical orientation offset between a display device connected to the view server and a display device connected to a particular view client. Determined in advance. For example, returning to FIG. 1, the viewing direction offset of the view client console 103 is set to be approximately equal to the physical direction offset θ between the view client display device 113 and the view server display device 111. Similarly, the viewing direction offset of the view client console 102 is set to be approximately equal to the physical direction offset between the view client display device 112 and the view server display device 111. In an embodiment, the viewing direction offset is determined based on a calibration routine that is performed by the view server and / or the view client to deduce the physical relative direction between the display devices.

  Method 200 proceeds to procedure 225, where the server's field of view value is reduced to less than 50 degrees in response to being networked to another platform that controls another display device. A second (and third, etc.) view client display located adjacent to the view server display device while reducing the fisheye effect in the view server display device by providing a larger field of view across multiple display devices Devices can be combined to provide an immersive environment for the user. Similarly, in step 226, the view client field of view value is also reduced to less than 50 degrees and transmitted to the view server. In a particular embodiment, each of the view server and view client has a field of view value set between 20 degrees and 50 degrees, and in an embodiment having both a view server and a view client, The value is set to approximately 25 degrees.

  On the view server side, the view server executes the geometry pipeline in steps 230, 235 and 250 to display the server camera view on the view server display device in step 255. On the view client side, each view client executes a geometry pipeline at steps 245, 260, and 265 to display the client camera's view on the view client display device at step 270. In an embodiment, the execution of these geometry pipelines by the view server and the view client may be performed by an internal view transmitted by the view server to render and display the graphics as a total across multiple display devices at substantially the same time. Adjustments are made based on the state information and one or more viewing direction offsets.

  While executing the first application on the view server, the input vertex data associated with the input received from the view control input device, such as the controller 125, is converted to model space using any conventional technique. . In procedure 230, the view server generates a world space from the model space. In general, “space” is a coordinate system for positioning an object with respect to a reference frame. In the left-handed coordinate system, similarly to the reference coordinate system shown in FIG. 1, the Z axis (the axis in the depth direction) extends from the user to the scene. The model space is a reference frame that defines the vertices of the modeled object relative to the model's local origin. Each object rendered in the world scene, such as graphics objects 172, 172, and 173, has a corresponding model that includes a plurality of vertices in model space. Thus, for a model relating to the graphics object 171, the model space may generate vertices for the origin located at the center of the human shape. To create world space, the global origin is defined as a common reference point for determining different positions in the scene, and the vertices of all modeled objects are defined relative to that global origin. .

  In step 235, the world space generated in step 230 is converted into a server view space or “server view frustum”. FIG. 3 shows an example of a server view frustum 300 corresponding to the system 100 shown in FIG. As shown in the figure, the server view frustum 300 is a reference frame for observing a conical view volume clipped by the front surface 301 and the back surface 302 in the positive direction of the Z-axis when the observer is at the origin. Have. The outer boundary of the server view frustum 300 converges to the position of the server camera 305. Accordingly, the server view frustum 300 has a truncated pyramid shape. The server camera 305 defines a viewpoint in the world space where the observer observes the world scene. Accordingly, the world space coordinate system is transferred and aligned to a world space viewpoint to provide server view frustum 300 at server camera 305.

  In step 240, the view server scans the internal state of all world space objects and sends the state of each object in such a way that all view clients can accurately represent the same world scene. To do. For example, if the view server is executing a game application, internal game state information may be broadcast to view clients of the system. The transmission mechanism may be UDP or TCP, and may depend on the selection of the title. UDP transmissions have additional state information saved by the view server console so that if an update packet is lost, the information can be resent with more recent information. TCP may be used, but additional latency deviations can be expected. However, TCP is highly reliable and does not require application interaction to confirm data transmission. In a local network such as LAN 110, latency is not a problem, especially when the view server / client console shares a local high-speed network switch.

  In the embodiment, the view state indicating the world space position and the world space direction of the server camera 305 is transmitted to the view client via the communication link. In other words, the server camera's view matrix is sent to the view client. In one such embodiment, the view server multicasts game view state information about 20 to 100 times per second. Object animation information may be further transmitted as part of the internal state data transmitted by the view server. Additional information may also be sent by the view server to synchronize locally generated objects such as particle effects (eg, objects generated by the view client console). In traditional online game configurations, such locally generated objects have only been roughly approximated because remote players do not need to see these effects in full synchronization, In an embodiment of the present invention, close synchronization of locally generated objects is based on view state information sent by the view server to advantageously provide the most desirable view to the user of the system. Achieved.

  In procedure 245, the view client receives the transmitted state information and generates a world space that is substantially the same as that generated by the view server. In other words, in response to each received view state update, the view client replicates the graphics object generated by the view server and renders a second view in the same world space. Is regenerated. In the embodiment, the second view has a viewpoint position at the position of the server camera 305. The server and the client share a common camera set by the view server and transmitted to the view client. The view client applies the viewing direction offset determined in step 220 to angularly offset or rotate the client view frustum from the world space direction so that the client view frustum does not overlap the server view frustum. May be. At this time, in order to render the client view depending on the server view, a viewing direction specific to each client is applied to the received server state information.

  For example, as shown in FIG. 3, the client view frustum 310 sets a viewing direction offset 350 from the Z axis around the server view frustum 300 based on the received server view state information and the viewing direction offset specific to the client. Generated by the view client to have (as defined for the world space direction). The client view frustum 310 is offset to be adjacent to the right side of the server view frustum 300. The graphics object 172 is duplicated by the view client and a portion of the graphics object 172 outside the server view frustum 300 is rendered. The portion of the graphics object 172 that is in the client view frustum 310 may further depend on the client's field of view according to procedure 226. As further shown in FIG. 3, another view client that receives a broadcast of view state information from the view server generates a third client view frustum 320 having a viewing direction offset 351 from the server view frustum 300. Client view frustum 320 is offset adjacent to the left side of server view frustum 300 and is outside server view frustum 300 (although generated by the server as part of world space and sent to the client). The graphics object 173 is rendered.

  In one embodiment, each of the view frustums 300-302 has a field of view of less than 50 degrees, such as 25 degrees to 50 degrees. The viewing direction offset adjusts the physical gap between adjacent display devices (such as due to display bezels) to provide a continuous FOV across the view server and view client display devices. It may be set to less than double. For example, if the physical direction offset between the view server display device 111 and the view client display device 112 is between 40 degrees and 50 degrees, and the FOV of each display device is 25 degrees, the viewing direction offset is almost It may be set to 40 degrees. In another embodiment, the viewing direction offset is set to at least twice the FOV of the display so that adjacent view frustums are spaced apart and the field of view across the view server and view client display is discontinuous. May be. At this time, the object in the world space may be “hidden” from the view in the area between the display devices.

  With appropriately set viewing direction offsets, the view server and view client render a common world space view frustum to form a mesh of view space for the same world scene. In this way, multiple display devices are tailored for rendering performed by independent computing platforms. This reduces the computing power of each platform. In addition, any application title created for stand-alone execution on a single computing platform having a single display device, such as the view server console 101, can be easily multiple, with little special application code required. The display device can be supported. Computing platform suppliers can provide a common library of functions that provide support for multi-display graphics rendering that can be incorporated by any developer of a computing platform application. Titles need only be created to have more or fewer graphics objects as a function of the field of view provided by the multi-display graphics rendering system. For example, an application title running in a single platform or “stand-alone” mode may generate a graphics object with a 60 degree deflection view from the camera. The same application title running in a multi-display system will include more fields of view based on the number of consoles on the network and the cumulative field of view available determined by the settings of each console's field of view. The deflection for generating the graphics object may be increased.

  After generating the view space corresponding to each display device, further matrix operations may be performed by the view server and view client to render the view of each display device. For example, the server view frustum 300 is deep in the scene, as is known in the art, to make the object 171 appear smaller than the object 172 based on the relative position of the object in the server view frustum 300. May be rendered to give Further matrix operations may be performed to generate a server screen space that is displayed on the server display device in step 255 with reference frames whose screen XY positions in the frame buffer are directly related to the coordinate system. For example, the server view frustum 300 may be rendered in a server screen space that includes a graphics object 171 and a portion of the graphics object 172 within the server view frustum 300, as shown in FIG. In step 270, a similar rendering activity is performed to display the client screen space substantially simultaneously with the server screen space being rendered so that the combined view across the display device of the system is displayed side by side. It may be executed by the view client. For example, the client view frustum 310 is substantially equivalent to the server view frustum 300 being rendered in a server screen space that includes a portion of the graphics object 172 within the server view frustum, as further shown in FIG. At the same time, it may be rendered in a client screen space that includes a portion of the graphics object 172 within the client view frustum 310 as determined by the field of view and orientation offset 350.

  In procedure 260, any change to the input matrix received from the controller coupled to the view server triggers an update of the server view space sent in procedure 240. The view client re-renders the corresponding client view space to reflect changes in the input matrix. In one embodiment, only the view server includes a view control input device such as the controller 125. A view client computing platform that is responsible for rendering a secondary view of the world scene through which the viewr is controlled need not include a controller. Because the view client system is a passive rendering mode, in some embodiments the network communication in the system may be a substantially one-way communication where the view server broadcasts to downstream view clients. Often, view clients do not need to communicate with each other. Thus, the network bandwidth may be dedicated to the view server substantially to allow a high internal state refresh rate across all platforms of the system.

  FIG. 4 shows a plan view of a multi-display graphics rendering system 400 according to an embodiment of the present invention. As shown, the system 400 includes all the components of the system 100 and four views added around the user's chair 450 with the view client display device 112 rotated about the Y axis of FIG. Client display devices 414, 415, 416 and 417 are included. Each view client display device is connected to view client consoles 404, 405, 406, and 407 in substantially the same manner as described for system 100. As shown, each display device has a physical orientation relative to the view server display device 111. For example, the view client display device 112 is at a position having a first physical direction offset 422 relative to the view server display device 111, and the view client display device 416 is second relative to the view server display device 111. With a physical direction offset of 426. In certain embodiments, each view client display device has a first physical direction offset 422 of approximately 40 to 50 degrees and a second physical direction offset 426 relative to the view server display device 111. It is at a fixed incremental position of approximately 40 to 50 degrees, such as approximately 80 to 100 degrees. When performing a graphics rendering method, such as method 200, each computing platform provides a viewing direction offset value based on the physical directional angle to provide a total view of approximately 360 degrees of the same world scene. May modify the server view state and render the view space. Any number of display devices may be combined to provide a 360 degree field of view in any dimension.

  In an embodiment, a multi-display rendering system such as system 400 may be networked to another multi-display rendering system. FIG. 5 shows a block diagram of two multi-display graphics rendering systems communicatively connected to each other. In the illustrated embodiment, a multi-display graphics rendering system 515 that includes a view server console 501 connected to a view client console 502 via a LAN 510, each connected to one of the display devices 503, is similar. A view server console 601 connected to the view client console 602 via the LAN 610, each connected to another multi-display graphics rendering system 615 connected to one of the display devices 603. The view server console 601 is further connected to the view server console 501 via the WAN 550 to enable communication between server consoles.

  In one such embodiment, a client-server relationship between systems is established between the view server consoles 501 and 601 separately from the view server-client relationship in the system established in the systems 515 and 615. Is done. In implementation, no system-to-system interaction between view client console 502 and view client console 602 is required. The inter-system client-server relationship is such that only a computing platform that is visible or active on the inter-system network of a multi-display rendering system (e.g., system 515 or system 615) is the system's view server (e.g., server console 501 and 601) may be established using conventional one-to-one or one-to-many communication technologies. For example, to allow a player using the multi-display graphics rendering system 515 to play the same game as a player using the system 615, the object position and animation information can be viewed using, for example, UDP transmission. Conventional online gaming techniques may be applied, such as being sent periodically between server consoles 501 and 601. The inter-system refresh rate between the system view server consoles 501 and 601 may be significantly lower than the intra-system refresh rate within the respective systems 515 and 615.

  In an embodiment of a networked multi-display rendering system, all consoles in the network (eg, view server consoles 501, 601 and respective view client consoles 502 and 602) generate the same world space graphics. And run the same application to render. The display device 503 is the respective display device from the first server camera at the first world position (eg, the position of the first player in world space) rendered by the view server console 501 and the view client console 502. The graphics by the direction offset in the camera view displayed by 503 is displayed. The display device 603 is the respective display device from the second server camera at the second world position (eg, the position of the second player in world space) rendered by the view server console 601 and the view client console 602. The graphics by the direction offset in the camera view displayed by 603 is displayed.

  FIG. 6 illustrates the hardware and user interface that can be used to generate and render a camera view of an application as a component of a multi-display graphics rendering system in connection with one embodiment of the present invention. FIG. 6 schematically shows the overall system configuration of a Sony PlayStation 3 (registered trademark) entertainment device which is a console compatible with the implementation of the multi-display graphics rendering system according to one embodiment of the present invention. Along with the system unit 1400, various peripheral devices that can be connected to the system unit 1400 are provided. The system unit 1400 includes a cell processor 1428, a Rambus® dynamic random access memory (XDRAM) unit 1426, a reality synthesizer graphics unit 1430 provided with a dedicated video random access memory (VRAM) unit 1432, and an input / output bridge. 1434. The system unit 1400 further includes a Blu-ray (registered trademark) disk BD-ROM optical disk reader 1440 for reading from the disk 1440A and a removable slot-in hard disk drive (HDD) 1436 that can be accessed via the input / output bridge 1434. . The system unit 1400 further comprises a memory card reader 1438 for reading a CompactFlash® memory card, a Memory Stick® memory card, etc., optionally accessible via the input / output bridge 1434 as well. .

  The input / output bridge 1434 further includes a plurality of universal serial bus (USB) 2.0 ports 1424, a Gigabit Ethernet (registered trademark) port 1422, an IEEE 802.11b / g wireless network (Wi-Fi) port 1420, seven Bluetooth ( It connects to a Bluetooth wireless link port 1418 that can support up to a registered trademark connection.

  In operation, the input / output bridge 1434 handles all wireless, USB and Ethernet data including data from one or more game controllers 1402-1403. For example, when a user plays a game, the input / output bridge 1434 receives data from the game controllers 1402-1403 via the Bluetooth link and passes it to the cell processor 1428. Cell processor 1428 updates the current state of the game as such.

  In addition to the game controllers 1402 to 1403, the wireless, USB and Ethernet (registered trademark) ports include a remote controller 1404, a keyboard 1406, a mouse 1408, a portable entertainment device 1410 such as a Sony PlayStation Portable (registered trademark) entertainment device, an eye toy ( Connections to other peripheral devices such as a video camera 1412 such as a registered video camera, a microphone headset 1414, and a microphone 1415 are also provided. These peripheral devices may in principle be connected to the system unit 1400 wirelessly. For example, the portable entertainment device 1410 may communicate via a Wi-Fi ad hoc connection, and the microphone headset 1414 may communicate via a Bluetooth link.

  These interfaces are provided by PlayStation 3 devices with other peripheral devices such as digital video recorders (DVRs), set-top boxes, digital cameras, portable media players, voice over IP phones, cell phones, printers, and scanners. It also means that they have potential compatibility.

  In addition, a legacy memory card reader 1416 that allows reading of the type of memory card 1448 used by the PlayStation® or PlayStation 2® device may be connected to the system unit via the USB port 1424. .

  The game controllers 1402 to 1403 can wirelessly communicate with the system unit 1400 via a Bluetooth link, or can be connected via a USB port to provide a power source for charging the game controllers 1402 to 1403. . Game controllers 1402 to 1403 include a memory, a processor, a memory card reader, a permanent memory such as a flash memory, a light irradiation unit such as LED or infrared light, a microphone and a speaker for ultrasonic communication, an acoustic chamber, a digital camera, an internal Recognizable shapes such as a clock, a sphere facing the game console, and wireless communication using protocols such as Bluetooth and WiFi may also be included.

  The game controller 1402 is a controller designed to be used with both hands, and the game controller 1403 is a controller that is provided with a ball and is handled with one hand. In addition to one or more analog joysticks and conventional control buttons, the game controller is capable of three-dimensional position determination. Thus, game controller user gestures and movements may be translated as input to the game in addition to or instead of conventional button or joystick commands. Optionally, other wirelessly communicable peripheral devices such as a PlayStation portable device may be used as the controller. In the case of a PlayStation portable device, additional game or control information (eg, number of control instructions or life) may be provided on the screen of the device. Single or several large buttons for a dance mat (not shown), maucap ball (not shown), light gun (not shown), steering wheel and pedal (not shown), or fast-press quiz game Other alternative or additional control devices such as a dedicated controller (not shown) may be used.

  The remote controller 1404 can wirelessly communicate with the system unit 1400 via the Bluetooth link. The remote controller 1404 includes controls suitable for operation of the Blu-ray Disc BD-ROM reader 1440 and navigation of the disc contents.

  The Blu-ray Disc BD-ROM reader 1440 can read CD-ROMs compatible with PlayStation and PlayStation 2 devices in addition to conventional recorded and recordable CDs and so-called super audio CDs. The reader 1440 can further read a DVD-ROM compatible with the PlayStation 2 and PlayStation 3 devices in addition to the conventional recorded and recordable DVD. The reader 1440 can read not only a conventional recorded and recordable Blu-ray disc but also a BD-ROM compatible with the PlayStation 3 device.

  The system unit 1400 receives audio and video that has been generated or decoded by the PlayStation 3 device via the reality synthesizer graphics unit 1430 via the audio and video connector and a monitor or display 1444 and one or more loudspeakers. It can be supplied to a display and audio output device 1442 such as a television set having 1446. Audio connector 1450 may include conventional analog and digital outputs, and video connector 1452 may include component video, S-video, composite video, one or more high-resolution multimedia interface (HDMI) outputs, and the like. As a result, the video output may be in PAL or NTSC, or 720p, 1080i or 1080p high resolution format.

  Audio processing (generation, decoding, etc.) is executed by the cell processor 1428. The PlayStation 3 device operating system supports Dolby 5.1 surround sound, Dolby Theater Surround (DTS), and 7.1 surround sound decoding from Blu-ray Discs.

  According to one embodiment, video camera 1412 includes a single charge coupled device (CCD), LED indicator, and hardware-based real-time data compression encoding device, and the compressed video data is stored in system unit 1400. May be transmitted in an appropriate format, such as an inter-picture based MPEG (motion picture expert group) standard. The camera LED indicator may be lit in response to appropriate control data from the system unit 1400, for example to signal a backlight condition. Embodiments of the video camera 1412 may connect to the system unit 1400 via a USB, Bluetooth, or Wi-Fi communication port. Video camera embodiments may include one or more coupled microphones and may be capable of transmitting audio data. In video camera embodiments, the CCD may have a resolution suitable for high definition video capture. In use, an image captured by a video camera may be incorporated into a game, for example, or interpreted as a game control input. In another embodiment of the camera, the camera is an infrared camera suitable for detecting infrared light.

  In general, appropriate software, such as a device driver, should be provided for successful data communication occurring with a peripheral device such as a video camera or remote controller via one of the communication ports of the system unit 1400. It is. The technology of the device driver is known and will not be described in detail here, but it will be understood by those skilled in the art that a device driver or similar software interface is required in this embodiment.

  FIG. 7 illustrates additional hardware available for instruction processing in one embodiment of the invention. FIG. 7 shows the components of the cell processor 1500. The cell processor 1500 in FIG. 7 includes an external input / output configuration including a memory controller 1560 and dual bus interface controllers 1570A and 1570B, a main processor called a power processing element (PPE) 1550, and synergistic processing elements (SPE) 1510A to H. It has a configuration with four basic elements of eight coprocessors, an element interconnection bus 1580 called a circular data bus connecting the above elements. The overall floating point operation of the cell processor is 218G flops, compared to 6.2G flops for the Emotion Engine of the PlayStation 2 device.

  The power processing element (PPE) 1550 is based on a bi-directional parallel multi-thread power 1470 compatible with a PowerPC core (PPU) 1555 that operates with an internal clock of 3.2 GHz. It comprises a 512kB level 2 (L2) cache and a 32kB level 1 (L1) cache. The PPE 1550 can perform eight single precision operations per clock cycle, which is 25.6 gigaflops at 3.2 GHz. The primary role of the PPE 1550 is to act as a controller for the SPEs 1510A-H that handle most computing workloads. In operation, PPE 1550 maintains a queue of jobs, schedules jobs for SPEs 1510A-H, and monitors their progress. Each SPE 1510A-H executes a kernel whose role is to fetch and execute a job, and synchronizes it with the PPE 1550.

  Each SPE 1510A-H includes a synergistic processing unit (SPU) 1520A-H, and each memory flow controller (MFC) 1540A-H includes a dynamic memory access controller (DMAC) 1542A. -H, memory management units (MMU) 1544A-H, and a bus interface (not shown). Each SPU 1520A-H is a RISC processor with a local RAM 1530A-H of 256 kB with a clock of 3.2 GHz and expandable to 4 GB in principle. Each SPE theoretically gives a single precision performance of 25.6 GFLOPS. The SPU can perform four single precision floating point operations, four 32-bit operations, eight 16-bit integer operations, or sixteen 8-bit integer operations in one clock cycle. Memory instructions can also be executed in the same clock cycle. The SPUs 1520A to H do not directly access the system memory XDRAM 1426. The 64-bit addresses generated by SPUs 1520A-H are passed to MFCs 1540A-H and commanded to DMA controllers 1542A-H for accessing memory via element interconnect bus 1580 and memory controller 1560.

  Element interconnect bus (EIB) 1580 is a logically circulating communication bus within cell processor 1500. The EIB connects a total of twelve elements: the processor elements described above: PPE 1550, memory controller 1560, dual bus interface 1570A, B, and eight SPEs 1510A-H. Elements can read and write the bus simultaneously at a rate of 8 bytes every clock cycle. As described above, each SPE 1510A-H includes a DMAC 1542A-H for scheduling a longer read or write sequence. The EIB has four channels, two in clockwise direction and two in counterclockwise direction. Among the 12 elements, the longest data flow between any two elements is 6 steps in the appropriate direction. The theoretical peak instant of the EIB 12-slot band is 96 bytes per clock when all is utilized through arbitration between elements. This is the same as the theoretical peak bandwidth of 307.2 GB / s (gigabyte per second) when the clock rate is 3.2 GHz.

  The memory controller 1560 includes an XDRAM interface 1562 developed by Rambus. The memory controller theoretically has a peak bandwidth of 25.6 gigabytes per second and connects to the Rambus XDRAM.

  The dual bus interface 1570A, B includes a Rambus Flex IO (registered trademark) system interface 1572A, B. The interface is organized into 12 channels of 5 inputs and 7 outputs, each 8 bits wide.

  It should be understood that the above description is intended to be illustrative and not limiting. Many other embodiments will be apparent to persons of ordinary skill in the art upon reading and understanding the above description. Although the invention has been described with reference to several specific embodiments, the invention is not limited to the described embodiments and can be modified and replaced within the spirit and scope of the appended claims. It will be appreciated by those skilled in the art. The detailed description is therefore to be regarded as illustrative instead of limiting. The scope of the invention should be determined with reference to the claims, along with the full scope of equivalents to which the claims are entitled.

Claims (40)

With a client connected to the communication link,
The client includes a client display device and a client computing platform for executing a client application,
The client computing platform periodically receives view state information from the server computing platform via the communication link;
The view state information includes a server camera position corresponding to an origin of a server view space generated by the server computing platform,
The client computing platform generates a client view space for display on the client display device based on the received view state information,
The graphics rendering system, wherein the generated client view space also has the server camera position as an origin. The server includes a server display device,
The system of claim 1, wherein the server computing platform generates the server view space for display on the server display device.   The system of claim 1, wherein the client computing platform generates a client view space in a direction offset from the server view space. The client display device has a physical orientation relative to the server display device;
The system of claim 2, wherein a directional offset between the server view space and the client view space is determined based on a relative physical direction of the client display device.   The system of claim 4, wherein the client computing platform stores data relating to the direction offset locally.   The system according to claim 2, wherein the server and the client application are the same game or CAD application. The communication link is a local area network;
The server computing device periodically broadcasts the view state information of the server via the communication link using an unreliable transmission protocol (UDP),
The system of claim 1, wherein the server and client computing platform are game consoles.   The system of claim 1, wherein only the server of the server and client computing platforms is connected to a controller that controls both the server view space and the client view space. Receiving view state information via a local area network (LAN);
Generating a client view space having an origin at a position of a server camera provided by the view state information based on the received view state information by a client application executed on a client computing platform;
Displaying rendered graphics on the client display device based on the generated client view space;
A method of rendering a graphic display comprising: Generating a server view space having an origin at a position of the server camera by a server application executed on a server computing platform;
Displaying the rendered graphics based on the server view space on a server display connected to the server computing platform;
Periodically transmitting the view state information including a server camera view matrix via the LAN;
10. The method of claim 9, further comprising:   The method of claim 10, wherein the client view space is generated in a direction offset from the server view space. The client display device has a physical orientation relative to the server display device;
The method of claim 11, wherein a directional offset between the server view space and the client view space is determined based on a relative physical direction of the client display device.   13. The method of claim 12, further comprising: executing a calibration routine at the server computing platform and the client computing platform to determine an offset in the direction based on the relative physical direction. The method described.   The method of claim 10, wherein the server and client application are the same game or CAD application.   The step of periodically transmitting view state information via the local area network (LAN) further includes the step of broadcasting the view state information to all clients connected to the LAN. Item 11. The method according to Item 10. The view state information further includes object position information and animation information,
The step of generating a client view space based on the view state information generates an object in the client view space based on the position information of the object, and displays an animation in the client view space based on the animation information. The method of claim 9, further comprising a step. A computer readable recording medium that stores a set of instructions that, when executed, cause a processing system to perform a method,
The method
Receiving view state information via a local area network (LAN);
Generating a client view space having an origin at a position of a server camera provided by the view state information based on the received view state information;
Displaying rendered graphics on the client display device based on the generated client view space;
A recording medium comprising:   The recording medium according to claim 17, wherein the client view space is generated in a direction offset from a server view space generated by a server that has transmitted the view state information. The client display device has a physical orientation relative to the server display device;
The recording medium according to claim 18, wherein an offset in a direction between the server view space and the client view space is determined based on a relative physical direction of the client display device.   The recording of claim 19, wherein the method further comprises performing a calibration routine to determine an offset in the direction based on a relative physical direction of the client display device. Medium. A first display device and a first computing platform, wherein the first computing platform generates a world space, and from the generated world space, displays the first display device on the first display device. Executing a first application that renders one view frustum, the first view frustum having an origin at a world space position of the application;
Connected to the server via a communication link and including a second display device and a second computing platform, wherein the second computing platform receives the transmission of the world space location and executes a second application Then, the second application regenerates the world space, renders a second view frustum for display on the display device of the proxy w from the regenerated world space, and Two view frustums, a client having an origin at a position in the world space in a direction offset from the first view frustum;
A multi-display graphics rendering system comprising: The first and second applications are the same game or CAD application,
The first and second view frustums represent separate parts of the same world scene from the position in the world space;
The system of claim 21, wherein the second view frustum does not overlap the first view frustum. The second display device has a physical orientation relative to the first display device;
The directional offset between the first view frustum and the second view frustum is based on the relative physical direction of the second display device. system.   24. The system of claim 23, wherein the first view frustum has a field of view from 20 degrees to 50 degrees. The second view frustum has a field of view equal to the first view frustum;
25. The system of claim 24, wherein a directional offset between the first view frustum and the second view frustum is less than twice the field of view. The second display device is physically oriented from 40 to 50 degrees to the left of the first display device;
The field of view is approximately 25 degrees;
26. The system of claim 25, wherein the second camera view is offset in a direction less than 50 degrees from the first camera view. A plurality of clients connected to the server via a communication link;
Each client
A client display device;
A client computing platform for receiving a transmission of the world space location from the server and executing a client application;
The client application generates a client view frustum having an origin at a position in the world space, each client view frustum having a different amount of offset direction from the first view frustum, The system of claim 21, wherein a client renders the client view frustum for display on the client display device. The plurality of clients includes a third client having a third display device physically oriented to the left of the second display device;
The third display device has a third view frustum having an offset in a second direction that is larger than the offset in the direction between the first and second view frustums from the first view frustum. 28. The system according to claim 27, wherein:   28. The system of claim 27, wherein the server and the plurality of clients generate an effective field of view greater than 120 degrees from the world space location. The first and second computing platforms are game consoles;
The communication link is a local area network (LAN) link,
The said server transmits the position of the said world space to all the clients connected to the said LAN with the broadcast message transmitted using unreliable transmission protocol (UDP). System.   Of the first and second computing platforms, only the first computing platform is connected to a controller configured to control both the first and second view frustums. The system of claim 21. Generating a world space with a first computing platform;
Generating from the generated world space a first view frustum having an origin at a position in the world space and a direction in the world space;
Displaying the first view frustum on a first display device connected to the first computing platform;
Transmitting the world space position and world space direction to a second computing platform via a communication link;
Regenerating the world space in the second computing platform;
Rendering from a regenerated world space a second view frustum having an origin at a position in the world space and a direction offset from the first view frustum;
Displaying the second camera view on a second display device;
A multi-display graphics rendering method comprising: Accessing configuration information stored in the second computing platform indicating an offset in a direction between the first view frustum and the second view frustum;
Applying an offset in the direction to the direction of the world space to render the second view frustum from the regenerated world space;
The method of claim 32, further comprising: The world space is generated by a game or CAD application executed on the first computing platform,
The method of claim 32, wherein the world space is regenerated by the same game or CAD application running on the second computing platform. Scanning the internal state of all world space objects generated by the application running on the first computing platform;
Transmitting the internal state to the second computing platform;
35. The method of claim 34, further comprising: Receiving a view control command for deflecting the position of the world space or the direction of the world space;
Re-rendering the first view frustum in response to the received view control command;
Receiving via the communication link a deflected world space position or world space direction and re-rendering the second view frustum accordingly;
The method of claim 32, further comprising: A computer readable recording medium that stores a set of instructions that, when executed, cause a processing system to perform a method,
The method
Creating a world space;
Generating from the generated world space a first view frustum having an origin at a position in world space received from a remote computing platform and angularly offset from the direction of the received world space; ,
Displaying the first view frustum on a first display device;
A recording medium comprising:   38. The recording medium of claim 37, wherein the world space is generated based on object and animation information received from the remote computing platform. Accessing stored configuration information indicating an angular offset between the first view frustum and a second view frustum rendered by the remote computing platform;
Applying the direction offset to the received direction of the world space to render the second view frustum from the regenerated world space;
The recording medium according to claim 37, further comprising:   The method further comprises receiving a modified world space location or world space orientation from the remote computing platform and re-rendering the first view frustum accordingly. The recording medium according to claim 37.

Images