JP2014149836A - Interactive user controlled avatar animations - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique that controls avatars in a virtual space accessed through a computer network.SOLUTION: There is provided a method including a step to capture the activity of a console controller, and a step to process the captured activity of the console controller for identifying an input parameter. Capturing and mapping are continuously executed for the definition of a correlation between the activity of the console controller and an avatar as a virtual space expression of a user.

Description

  The present invention relates generally to interactive multimedia entertainment, and more particularly to interactive user control and user manipulation representation in virtual space.

  The video game industry has experienced many changes over the years. As computing power has expanded, video game developers have similarly created game software that takes advantage of this increased computing power. To this end, video game developers have been coding games that employ advanced arithmetic and mathematics to create extremely realistic gaming experiences.

  An example of a game platform is Sony PlayStation or Sony PlayStation 2 (PS2), each sold in the form of a game console. As is well known, game consoles are designed to be connected to a monitor (usually a television) and allow user interaction with a handheld controller. The game console is a specialized processing hardware such as a CPU, a graphic synthesizer for graphic operations with a large amount of processing, a vector device for performing geometry conversion, and other connecting hardware, that is, glue hardware, firmware And designed with software etc. In addition, the game console is designed with an optical disc tray for receiving a game compact disc for local play by the game console. Also possible are online games where the user can play against other users on the Internet or interactively play with other users.

  Given the complexity of the game that attracts players' interest, game and hardware manufacturers have continued to innovate to realize additional interactive features and computer programs. Some computer programs define a virtual world. A virtual world is a simulated environment in which users can interact via one or more computer processors. The user can be displayed on the video screen in the form of an expression called an avatar. The degree of interaction between the avatar and the simulated environment is implemented by one or more computer applications that govern the interaction, such as simulated physical phenomena, the exchange of information between users. The nature of interaction between users in the virtual world is often limited by the constraints of the system implementing the virtual world.

  The embodiments of the present invention have been made in this situation.

  The embodiments defined herein allow computer-controlled systems and programs to map interface inputs to specific features of virtual world animated characters displayed on the screen. In one embodiment, a particular button on a controller (eg, a game controller) is mapped to a particular body part of the avatar that defines the animated character of the virtual world. In some embodiments, not only buttons are mapped to avatar features, but positions, movements, triggers, placements, and combinations thereof are also mapped, resulting in real-world users being presented on the screen Can be controlled accurately.

  As described in more detail below, real-world users control avatars across a virtual world of many places and spaces, and other avatars (which can be controlled by other real-world users or computer-controlled bots) Can be exchanged with each other, or can be interfaced with an object, an object, or the environment, or can perform a communication operation. The communication operation can be controlled by the controller by a transformation mapping that is transferred to the avatar in the form of visual, audio or a combination thereof. Thus, the following embodiments show the buttons, movements (and combinations) of a specific controller (eg, a game controller or general purpose computer control peripheral) to a specific or selected body part of the avatar, the entire body of the avatar. Examples of possible controls by mapping to movement, avatar body reactions, avatar face reactions, avatar emotions, etc. should be considered broadly.

  In one embodiment, a method for controlling an avatar in a virtual space accessed via a computer network using a console is disclosed. The method begins with capturing console controller activity (eg, activity) and processing the captured activity of the console controller to identify input parameters. The next operation of the method is the step of mapping selected parameters of the input parameters to the portion of the avatar that is a virtual space representation of the user. To capture the correlation between the console controller activity and the avatar, which is the virtual space representation of the user, the capture, the processing and the mapping are performed sequentially.

  In another embodiment, a method for interactively controlling an avatar over a computer network using a console is disclosed. The method begins with providing a console controller and determining a first position of the console controller. The method is followed by capturing input to the console controller including detecting movement of the console controller to a second position. Another step is processing the input to the console controller and the relative movement of the console controller between the first position and the second position. The next step in the method is to map the relative movement between the first position and the second position of the console controller to the animated body part of the avatar. In order to define the correlation between the relative movement of the console controller and the avatar, the capture, the processing and the mapping are performed continuously.

  In yet another embodiment, a computer-implemented method for interactively controlling an avatar in a virtual environment is disclosed. In this embodiment, a computer program executed on at least one computer in a computer network generates the avatar and the virtual environment. The method begins with providing a controller that is interfaced with the computer program and mapping controller inputs so that a user can control selected portions of the avatar. The method is followed by capturing controller input and controller movement between a first position and a second position. The next step of the method is to process the captured controller input and controller movement and apply the captured movement to interactively animate the selected portion of the avatar within the virtual environment. It is a step to do. In order to define the correlation between controller movement and avatar animation, the capture of the controller input and controller movement and the process are performed continuously.

  Other aspects and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.

  The invention and further advantages will be better understood when the following description is read with reference to the accompanying drawings.

1 is a schematic diagram illustrating an avatar control system 100 according to an embodiment of the present invention. FIG. 6 is a schematic diagram illustrating various methods of transmitting motion and position detection between the controller 108 and the console 110, according to one embodiment of the present invention. FIG. 3 is a schematic diagram of a user 102a interacting with an avatar control system, according to one embodiment of the present invention. FIG. 3 is a schematic diagram of a user 102a interacting with an avatar control system, according to one embodiment of the present invention. FIG. 6 is an exemplary diagram of motion capture of relative motion of a controller that changes avatars, according to one embodiment of the invention. 6 is a flowchart illustrating a method in which controller button selection can be used to define and supplement an avatar control system, in accordance with one embodiment of the present invention. FIG. 7 is an exemplary diagram of multiple motion captures of relative motion of a controller that changes avatars, according to one embodiment of the invention. FIG. 6 is another exemplary diagram of multiple motion captures of relative motion of a controller that changes avatars, in accordance with an embodiment of the present invention. FIG. 5 is a schematic diagram illustrating mapping of controller buttons for controlling a specific body part of an avatar according to an embodiment of the present invention. FIG. 4 is a schematic diagram illustrating control of various forms of an avatar during display of an avatar animation, according to an embodiment of the present invention. FIG. 6 is a schematic diagram illustrating control of various movements of the avatar's head, in accordance with an embodiment of the present invention. 1 is a schematic diagram that schematically illustrates the overall system architecture of a Sony® PlayStation 3® entertainment device, which is a console having a controller for implementing an avatar control system, in accordance with an embodiment of the present invention. FIG. 9 is a schematic diagram of a Cell processor 928 according to an embodiment of the present invention.

  An invention is disclosed for allowing real-world users to control avatar movements and actions in a virtual world. According to one embodiment of the present invention, the user can interact with the virtual world. As used herein, the term “virtual world” refers to an actual user that can be sensed via one or more display devices and / or interacted via one or more user interfaces. Refers to a representation of a real or fictional environment with rules of interaction that are simulated by one or more processors. As used herein, the term “user interface” refers to an actual device that allows a user to send input to or receive output from the virtual world. The virtual world can be simulated by one or more processor modules. A plurality of processor modules can be interconnected via a network. A user may interact with the virtual world via a user interface device that can communicate with the processor module and other user interface devices via a network. Certain features of the virtual world can be presented to the user in a graphical form on a graphic display such as a computer monitor, television monitor or similar display. Other particular aspects of the virtual world may be presented to the user in an audible form with speakers that may be associated with the graphic display.

  Within the virtual world, the user can be represented by an avatar. Each avatar in the virtual world can be uniquely associated with a different user. The user's name or pseudonym can be displayed next to the avatar so that the users can easily identify each other. A particular user's interaction with the virtual world may be represented by one or more corresponding actions of the avatar. Different users can interact through their avatars. An avatar representing a user can have the same appearance as a person, animal or object. The avatar in the form of a person may have the same or different gender as the user. The avatar can be displayed on the display so that the user can see the avatar along with other objects in the virtual world.

  In another embodiment, the display may display the world from the avatar's viewpoint without displaying the avatar. The user's (or avatar's) viewpoint in the virtual world can be regarded as the field of view of the virtual camera. As used herein, a “virtual camera” refers to a viewpoint in a virtual world and can be used to draw a two-dimensional image of a three-dimensional scene in the virtual world. Users can interact through their avatars via chat channels associated with each lobby. The user can enter text for chatting with other users via the user interface. This text can be displayed on or next to the user avatar, for example, in the form of a comic balloon (sometimes also called a “chat balloon”). Such a chat can be easily performed using a chat system of a fixed expression sometimes called “quick chat”. In quick chat, the user may select one or more chat expressions from a menu. Other examples include: (1) British Patent Application No. 0703964.6 filed on March 1, 2007, “ENTERTAINMENT DEVICE”; (2) British Patent Application No. 0704225 filed on March 5, 2007; No. 2 “ENTERTAINMENT DEVICE AND METHOD”; (3) UK Patent Application No. 070425.1 filed March 5, 2007 “ENTERTAINMENT DEVICE AND METHOD”; ) British Patent Application No. 070427.8 filed March 5, 2007, “ENTERTAINMENT DEVICE AND METHOD”; and (5) British Patent Application No. 0704246.8, filed March 5, 2007. See also “ENTERTAINMENT DEVICE AND METHOD” Each of which is incorporated herein by reference.

  In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without some or all of these details. In some instances, well known process operations have not been described in detail in order not to unnecessarily obscure the present invention.

  FIG. 1A is a schematic diagram illustrating an avatar control system 100 according to an embodiment of the present invention. A user 102 a operates a controller 108 that can communicate with the console 110. In some embodiments, the console 110 may comprise a storage medium that can store and retrieve data. Examples of types of storage media include, but are not limited to, magnetic storage devices, optical storage devices, flash memories, random access memories, and read only memories.

  The console 110 may also have a network interface such as an Ethernet port and a wireless network function including a number of wireless network standards belonging to IEEE 802.11. The console 110 network interface allows the user 102a to connect to a remote server, which provides real-time and interactive game play, software updates, media services, and social networking services with other users of the console 110. Can be provided.

  The console 110 can also include a central processing unit and a graphics processing unit. The central processing unit is used to process instructions obtained from the storage medium, and the graphic processing unit can process and render graphics displayed on the screen 106.

  When the screen 106 is connected to the console 110, the console 106 can display the virtual space 104 including the avatar 102b. In one embodiment, the virtual space 104 may be maintained on a remote server that is accessed using the network interface of the console 110. In another embodiment, a portion of virtual space 104 may be stored on console 110 and another portion stored on a remote server. In some embodiments, the virtual space 104 is a virtual three-dimensional world displayed on the screen 106. Virtual space 104 may be a virtual representation of the real world where terrain, weather, flora, fauna, currency and politics exist. Similar to the real world, the virtual space 104 may include urban, suburban and rural areas. However, unlike the real world, the laws of physics may change in the virtual space 104. The features of the virtual space 104 listed above are intended to be illustrative and are not intended to be limiting or limiting. As a virtual space, the range that can be simulated and modeled may include anything in the real world and is limited only by the limits of human imagination.

  The user can interact with the virtual space 114 using the avatar 102b. The avatar 102b is drawn in three dimensions, and can be configured by the user 102a so that the user 102a is represented in the virtual space 104 realistically or by fantasy. User 102a has full control over multiple forms of the avatar, including hairstyle, head shape, eye shape, eye color, nose shape, nose size, ear size, Examples include, but are not limited to, lip shape, lip color, clothing, footwear and accessories. In another embodiment, the user 102a can input his / her actual facial photo, which may be mapped to the head and body of a three-dimensional wireframe.

  FIG. 1B illustrates various methods of transmitting motion and position detection between the controller 108 and the console 110 according to one embodiment of the invention. User 102 a can control avatar 102 b in virtual space 104 using controller 108. In one embodiment, the controller 108 can wirelessly send a signal to the console 110. Because multiple controllers can be used in a single console 110, interference occurs between individual controllers by transmitting radio signals at specific frequencies or using radio and communication protocols such as Bluetooth. be able to. The controller 108 includes a plurality of buttons and joysticks that can be manipulated by the user to achieve various effects such as moving and selecting items from the screen menu. Similarly, the buttons and joysticks of the controller 108 can be mapped in the control form of a computer program executed by the console 110.

  The controller 108 can also include a motion sensor that can detect translation and rotation in the x, y, and z axes. In one embodiment, the motion sensor is an inertial sensor that can detect the movement, acceleration and deceleration of the controller 108. In another embodiment, a gyro may be used to detect controller 108 movement in all axes. The controller 108 can wirelessly transmit data from the motion sensor to the console 110 for processing, which results in the action being displayed on the screen.

  A camera 112 may also be connected to the console 110 to assist in providing visual detection of the controller 108. In one embodiment, a camera 112 and LEDs located on the controller 108 provide the console 110 with visual detection of the controller 108. The LEDs can emit light in or out of the visible spectrum and may be integrated into the controller as an array to help the controller determine if it is off axis from the camera 112. In another embodiment, the LEDs may be attached to the controller 108 as a module.

  The camera 112 is configured to receive light emitted by the LEDs, and the console 110 can calculate the movement of the controller 108 based on the changes in the LEDs relative to the camera 112. Further, in embodiments where multiple controllers are associated with a single console 110, individual blink patterns or light frequencies can be used to distinguish between LEDs of different controllers.

  In another embodiment, camera 112 is a depth camera that can assist in determining the distance between camera 112 and controller 108. In some embodiments, the depth camera may have a maximum scan depth. In this situation, depth values are calculated only for objects within the maximum scan depth. As shown in FIG. 1B, the camera 112 has a maximum scan depth of Z. Since the controller 108 is within the maximum scan depth, the distance between the camera 112 and the controller 108 can be calculated. In yet another embodiment, a combination of inertial sensor, LED detection and depth camera may be used to increase the accuracy of motion and position detection. In one embodiment, the camera 112 may be integrated with the console 110. In another embodiment, the camera 112 may be placed independently of the console 110.

  FIG. 2 shows a diagram of a user 102a interacting with an avatar control system, according to one embodiment of the present invention. The user 102a has the controller 108 and is shown with his waist bent. As the user 102a bends, the controller 108 is pitched down from the initial substantially horizontal position to the position shown in FIG. In operation 120, the downward pitch motion of the controller 108 may be captured by the motion capture system. In operation 122, computer analysis is performed by the console and the motion capture of the controller 108 is mapped to a specific body part of the avatar 102b. Operation 124 draws an animation of avatar 102b, which may be output from console to screen 106.

  As shown in FIG. 2, the motion capture of the controller 108 may be mapped to the waist of the avatar 102b. In another embodiment, motion capture of the movement of the controller 108 may be mapped to different body parts of the avatar 102b, such as legs, arms, hands and head. In yet another embodiment, motion capture from the controller 108 may be combined with other forms of user input to change the avatar 102b. For example, when the user 102a speaks, the microphone and camera system may be used for the monitor, and as a result, the animation of the mouth of the avatar 102b may be displayed. The user 102a can also use buttons on the controller 108 to change and customize the reaction and movement of the avatar 102b.

  FIG. 3 shows a diagram of a user 102a interacting with an avatar control system, according to one embodiment of the present invention. In this embodiment, the user 102 is shown pitching the controller 108 down from the initial substantially horizontal position to the position shown in FIG. Similar to FIG. 2, in operation 120, the controller's downward pitch motion may be captured by the motion capture system. Operation 122 performs a computer analysis of the motion capture, and operation 124 may render the motion capture to the avatar 102b.

  Comparison of FIG. 2 and FIG. 3 shows that motion capture of the relative motion of the controller can change the avatar 102b. In FIG. 2, when the user 102 bends, the motion capture system may detect a change in the position of the controller 108. In FIG. 3, the controller 108 can be pitched downward by the wrist movement of the user 102a. Although the body movement performed by the user 102a is different, the downward pitch movement of the controller 108 is a relative movement that is captured and analyzed by the controller. Thus, when different body movements of the user 102a causing the controller 108 to cause the same relative motion are mapped to the same avatar body part, the avatar 102b is caused to have a similar animation.

  FIG. 4A is an exemplary diagram of motion capture of relative motion of a controller that changes avatars, in accordance with one embodiment of the present invention. In this embodiment, avatar 102b represents the display before motion capture, and avatar 102b 'represents the display after motion capture view. In the initial state, the user 102a having the controller 108 is represented by an avatar 102b. When the user 102a causes the controller 108 to yaw 90 ° to the right of the user 102a, the motion capture of the relative motion of the controller 108 is analyzed and applied to the avatar 102b. In this embodiment, the movement of the controller 108 is mapped to the whole body of the avatar 102b, and the avatar 102b 'is displayed by the 90 ° yaw movement of the controller 108.

  FIG. 4B is a flowchart illustrating a method by which the selection of buttons on the controller can be used to define and supplement an avatar control system according to one embodiment of the present invention. As explained above, different features of the avatar can be controlled by various relative movements of the controller. In order to improve the avatar's interactivity and reality, it may be advantageous to allow the user to control their avatar's facial expressions, hand gestures, and other features, expressions, and emotions. In order to achieve this level of avatar control, auxiliary inputs other than motion capture of the controller's relative motion may be used. Controller buttons can be mapped to select, control and manipulate avatar expressions and emotional variations that are endless. The flowchart of FIG. 5 illustrates how an avatar control system can be defined and the selection of buttons on the controller can be used to supplement this. In operation 500, the motion detection system detects the initial position of the controller. This is followed by operation 502 which detects controller movement relative to the initial position of the controller. In operation 504, it is determined whether any button on the controller has been selected. In operation 506, computer button selection and relative motion of the controller are performed. This is followed by operation 508 in which controller movements and button selections are mapped to avatars.

  FIG. 5A is an exemplary illustration of multiple motion captures of relative motion of a controller that changes avatars, in accordance with one embodiment of the present invention. The user 102a performs the motion A by performing a downward pitch motion on the controller 108. The motion capture of the motion A is mapped to the waist of the avatar 102b. As a result, the avatar 102b bends the waist. User 102a performs motion B to pitch the controller up to a substantially horizontal position and roll the controller to the user's right while yawing the controller to the user's right. In this embodiment, the yaw movement of the controller is mapped in the direction the avatar is facing. For this reason, the avatar 102b turns to face the front by the yaw movement of the user to the right, and is displayed as the avatar 102b ′. As explained above, in this embodiment, the pitch movement of the controller 108 is mapped to the avatar's hip movement. Therefore, when the movement B is performed, the avatar moves from the bent position of the avatar 102b to the upright position of the avatar 102b 'by the upward pitch movement of the controller 108 to the substantially horizontal position. In this embodiment, the roll motion of the controller 108 is mapped to the motion of the avatar tilting from the waist, and the avatar 102b ′ tilted to the right of the avatar is displayed by the roll motion of the controller user to the right. Is done.

  FIG. 5B is another exemplary diagram of multiple motion captures of relative motion of a controller that changes avatars, in accordance with one embodiment of the present invention. The user performs motion A by performing a downward pitch motion on the controller 108. The motion capture of the motion A is mapped to the waist of the avatar 102b. As a result, the avatar 102b bends the waist. User 102a 'performs motion B without returning controller 108 to a substantially horizontal position. User 102a 'performs motion B to roll the controller to the left of the user. In this embodiment, when the controller is rolled, the avatar 102b tilts to the left of the avatar. For this reason, the combination of the movement A and the movement B causes the avatar 102b to bend forward and tilt to the left of the avatar. Since the controller 108 includes a sensor capable of measuring acceleration / deceleration, the animation of the avatar 102b can be correlated with the actual movement of the controller 108 by the user 102a / a '.

  FIG. 6 illustrates a mapping of controller buttons for controlling a specific body part of an avatar according to one embodiment of the present invention. The controller 108 may include various buttons including a digital control pad, indicated by DU, DR, DD and DL. The controller may also include a left shoulder button 108a that includes LS1 and LS2. Similarly, the right shoulder button 108b includes RS1 and RS2. The controller 108 also includes analog sticks AL and AR, and the analog stick can also function as a button when pressed. The controller may also include selection buttons indicated by □, Δ, ○, and X in FIG. Although specific names and symbols have been used to describe the controller 108, the names are exemplary and are not intended to be limiting.

  In one embodiment, the various buttons of the controller 108 may be mapped to enable control of specific body parts of the avatar. As shown in the avatar mapping 600, pressing the AR allows the user to control the head of the avatar. The user can control the right arm or the right leg of the avatar by pressing RS1 or RS2. Similarly, LS1 and LS2 are mapped to control the left arm and left leg of the avatar, respectively. The user can control various parts of the avatar and can start and modify pre-rendered avatar animations. The user can initiate an avatar animation by pressing a button or buttons, moving the controller one or more times, or combining the button press sequence and controller movement.

  As shown in the dance animation 601, the avatar animation can be considered as a sequence of various states. In one embodiment, state 1 602 indicates that the user avatar is in a rest position or a position before the start of the dance animation. State 2 604 can be considered to be the state of the avatar immediately after the start of the dance animation. In this embodiment, the avatar is tilted to the left. In state 3 606 of the final state of the dance animation 601, the user's avatar tilts to the right and raises his right arm. Since the dance animation 601 is intended to show various states of the avatar, transition frames between various states are not shown. It will be apparent to those skilled in the art that additional frames may be needed to smoothly display the avatar's animation between the various states. Since the dance animation 601 is an example and is not intended to be limiting, the number of states may increase or decrease in other embodiments of avatar animation.

  As explained above, the controller 108 can detect translation and rotational motion acceleration and deceleration in three axes. This allows the user to interactively control the movement with the direction of the animation and the rate animation of the avatar based on user input such as the actual acceleration and deceleration of the translation and rotation motion of the controller. In addition, the mapping of controller buttons that enable control of a particular body part of the avatar allows the user to specify one or more body parts of the avatar to be interactively animated. Thus, since the user directly controls the animation of a specific body part of the avatar, a unique avatar animation is obtained. Avatar animations that respond to direct control from the user are different from avatar animations that are pre-mapped, pre-defined, and pre-rendered, as in other forms of avatar animation.

  For example, some systems allow control of animated characters in the game, but in one embodiment, the controller movement, controller input, and controller position disclosed herein can be assigned to specific parts of the avatar. By mapping, it becomes possible to specify in detail the characteristics of the avatar to be controlled, the degree of control, and the application of such control to the avatar to be animated. Furthermore, the avatar character is not bound by a specific predetermined game, game scene, or environment or game level experience. For example, avatars controlled by real-world users can define places to visit, things to interact with, things to see and experiences to enjoy. In response to requests for input dictated by controller activity, defined by real-world users, avatar experiences, movements, reactions and body movements in a virtual environment are created.

  FIG. 7 illustrates control of various forms and features of an avatar during display of an avatar animation according to an embodiment of the present invention. In state 700, a combination of button presses by controller 108 may be used to initiate state 1 602 of an avatar dance animation on screen 106. As shown in state 700, controller 108 is in a substantially horizontal position. In state 702, the user holds down LS1 to control the avatar's left arm. For this reason, when the user pitches the controller 108 upward, the left arm of the avatar goes up to the position shown in the state 604a on the screen 106. Moving to state 704, the user pitches controller 108 down to a substantially horizontal position while holding LS1. When the controller is pitched down, on screen 106, the avatar's left arm is lowered to the position shown in state 606a.

  In another embodiment, if the user presses and releases the button corresponding to the selected portion of the avatar, control of that portion of the avatar can continue until the button is pressed next time. In order to easily identify which part of the avatar the user is controlling, it is possible to highlight the part where the avatar is controlled on the screen 106. This highlight display is displayed only for the user who is controlling the avatar and may not be visible to other users of the virtual space.

  FIG. 8 illustrates the control of various movements of the avatar's head, according to one embodiment of the present invention. State 800 shows the avatar's head turning when the controller 108 is yawed while the right analog stick button (AR) is pressed. For this reason, the user who is performing the avatar control in the state 800 can shake the avatar's head sideways in order to convey “no” without using words. Conversely, in state 802, if the user pitches the controller 108 up and down while holding down AR, the avatar's head can be swung up and down to convey “yes” without using words. In state 804, if the controller 108 is rolled to the left or right while pressing AR, the head of the avatar tilts to the left or right. It will be apparent to those skilled in the art that the avatar's head can be compounded based on a combination of user input controllers selected from yaw, pitch and roll. Similarly, complex movements based on yaw, pitch and roll can also be mapped to other features of avatar animation.

  FIG. 9 schematically illustrates the overall system architecture of a Sony® PlayStation 3® entertainment device, which is a console having a controller for implementing an avatar control system, according to one embodiment of the present invention. The system unit 900 includes various peripheral devices that can be connected to the system unit 900. The system unit 900 includes a Cell processor 928, a Rambus® dynamic random access memory (XDRAM) unit 926, a Reality Synthesizer graphic unit 930 having a dedicated video random access memory (VRAM) unit 932, and an I / O bridge 934. Prepare. The system unit 900 also has a Blu-ray (registered trademark) disk BD-ROM (registered trademark) for reading from the disk 940 a and a removable slot-in hard disk drive (HDD) 936 accessible via the I / O bridge 934. An optical disk reader 940 is also provided. Optionally, the system unit 900 also includes a memory card reader 938 for reading a compact flash memory card, memory stick® memory card, etc., which is also accessible via the I / O bridge 934.

  The I / O bridge 934 also includes six universal serial bus (USB) 2.0 ports 924, a Gigabit Ethernet port 922, an IEEE 802.11b / g wireless network (Wi-Fi) port 920, and up to seven Bluetooth connections. Is also connected to a Bluetooth (registered trademark) wireless link port 918 capable of supporting the above.

  In operation, the I / O bridge 934 processes all wireless, USB, and Ethernet data, including data from one or more game controllers 902. For example, while the user is playing a game, the I / O bridge 934 receives data from the game controller 902 via the Bluetooth link and forwards it to the Cell processor 928, which in turn, may Update.

  In addition to the game controller 902, other peripheral devices can be connected via the wireless, USB, and Ethernet ports. Examples of such peripheral devices include a remote control 904, a keyboard 906, a mouse 908, a portable entertainment device 910 such as a Sony PlayStation Portable (registered trademark) entertainment device, a video camera such as an EyeToy (registered trademark) video camera 912, and a microphone head. There is a set 914 and the like. Therefore, these peripheral devices can be wirelessly connected to the system unit 900 in principle. For example, portable entertainment device 910 may communicate over a Wi-Fi ad hoc connection and microphone headset 914 may communicate over a Bluetooth link.

  By providing these interfaces, the PlayStation 3 device can be used with other peripherals such as digital video recorders (DVR), set-top boxes, digital cameras, portable media players, VoIP phones, cell phones, printers, and scanners. Some can be compatible.

  In addition, an old-style memory card reader 916 can be connected to the system unit via the USB port 924, and the type of memory card 948 used in the PlayStation® device or PlayStation 2® can be read. It becomes.

  In this embodiment, the game controller 902 is operable to communicate wirelessly with the system unit 900 via a Bluetooth link. However, instead, the game controller 902 may be connected to a USB port, which also provides power to charge the battery of the game controller 902. The game controller has one or more analog joysticks and conventional control buttons, as well as sensing six degrees of freedom movement corresponding to the translation and rotation of each axis. Thus, in addition to or instead of conventional buttons or joystick commands, gestures and movements made by the user of the game controller can be converted as input to the game. Optionally, other wireless-enabled peripherals such as PlayStation portable devices can be used as the controller. In the case of a PlayStation portable device, additional game information or control information (eg, number of control instructions or live) can be presented on the screen of the device. Other alternative or auxiliary controls may be used, such as a dance mat (not shown), a light gun (not shown), a handle and pedal (not shown), or an immediate answer quiz There are custom controllers such as one or more large buttons (also not shown) for the game.

  The remote controller 904 is also operable to communicate wirelessly with the system unit 900 via a Bluetooth link. The remote control 904 includes controls suitable for operating the Blu-ray Disc BD-ROM reader 940 and viewing the contents of the disc.

  Blu-ray Disc BD-ROM reader 940 is operable to read conventional recorded CDs, recordable CDs, and so-called super audio CDs as well as CD-ROMs compatible with PlayStation devices and PlayStation 2 devices. The reader 940 is operable to read DVD-ROMs compatible with the PlayStation 2 device and the PlayStation 3 device in addition to the conventional recorded DVD and recordable DVD. In addition, the reader 940 is operable to read BD-ROMs compatible with PlayStation 3 devices, as well as conventional recorded Blu-ray discs and recordable Blu-ray discs.

  The system unit 900 displays audio and video generated or decoded by the PlayStation 3 device via the Reality Synthesizer graphics unit 930 via the audio connector and the video connector, with a display 944 and one or more speakers 946. It is operable to provide audio output device 942 (such as a monitor or television receiver). The audio connector 950 includes conventional analog and digital outputs, while the video connector 952 includes a variety of component video, S-video, composite video, and one or more high definition multimedia interface (HDMI) outputs. sell. Thus, the video output may be in a format such as PAL or NTSC, or a high resolution of 720p, 1080i or 1080p.

  Audio processing (generation, decoding, etc.) is performed by the Cell processor 928. The PlayStation 3 device operating system supports Dolby® 5.1 surround sound, Dolby® Theater Surround (DTS), and 7.1 surround sound decoding from Blu-ray® discs. Yes.

  In this embodiment, the video camera 912 comprises one charge-coupled device (CCD), LED indicator, and hardware-based real-time data compression and encoding device so that the compressed video data can be decoded by the system unit 900. Intra-image based MPEG (Motion Picture Expert Group) standards are transmitted in an appropriate format. The LED indicator of the camera is arranged to emit light upon receiving appropriate control data from the system unit 900, for example, indicating adverse lighting conditions. Embodiments of the video camera 912 may connect to the system unit 900 in various ways via USB, Bluetooth or Wi-Fi communication ports. Video camera embodiments include one or more associated microphones and can transmit audio data. In video camera embodiments, the CCD may have a resolution suitable for high resolution video capture. In use, an image captured by a video camera can be captured, for example, in a game or interpreted as a game control input.

  In general, it is necessary to provide appropriate software such as a device driver so that data communication with a peripheral device such as a video camera or a remote controller can be successfully performed via one of the communication ports of the system unit 900. Device driver technology is well known and will not be described in detail here, but those skilled in the art will appreciate that a device driver or similar software interface may be required in the described embodiment.

  FIG. 10 is a schematic diagram of a Cell processor 928 according to one embodiment of the invention. The Cell processor 928 includes an external input / output structure including a memory controller 1060 and dual bus interface controllers 1070A, 10B, a main processor called a power processing element 1050, eight coprocessors called synergistic processing elements (SPE) 1010A to H, It has an architecture with four basic components, an annular data bus connected to the above components, called the element interconnect bus 1080. The Emotion Engine of the PlayStation 2 device is 6.2 gigaflops, whereas the total floating point performance of the Cell processor is 218 gigaflops.

  The power processing element (PPE) 1050 operates on an internal clock of 3.2 GHz and is based on a PowerPC core (PPU) 1055 compliant with bidirectional simultaneous multithreading Power970. The PPE 1050 includes a 512 kB level 2 (L2) cache and a 32 kB level 1 (L1) cache. The PPE 1050 is capable of eight single position (SP) operations per clock cycle, which corresponds to 25.6 gigaflops at 3.2 GHz. The primary role of the PPE 1050 is to function as a controller for the synergistic processing elements 1010A-H that process most of the computational workload. In operation, the PPE 1050 maintains a job queue, schedules jobs for the synergistic processing elements 1010A-H and monitors their progress. Thus, each synergistic processing element 1010A-H executes a kernel, whose role is to retrieve the job, execute it, and synchronize with the PPE 1050.

  Each synergistic processing element (SPE) 1010A-H includes a synergistic processing unit (SPU) 1020A-H and a memory flow controller (MFC) 1040A-H, respectively, and the MFC 1040A-H is a dynamic memory access controller (DMAC) 1042A. To H, memory management units (MMU) 1044A to H, and a bus interface (not shown). Each SPU 1020A-H is a RISC processor with a 256kB local RAM 1030A-H that is clocked at 3.2GHz and can in principle scale up to 4GB. Each SPE theoretically exhibits 25.6 gigaflops in single precision processing performance. One SPU can process four single precision floating point units, four 32-bit numbers, eight 16-bit integers, or sixteen 8-bit integers in one clock cycle. Memory operations can also be performed in the same clock cycle. SPUs 1020A-H do not access system memory XDRAM 926 directly. 64-bit addresses created by the SPUs 1020A-H are passed to the MFCs 1040A-H so that the MFCs 1040A-H can access their DMA controllers 1042A-H via the element interconnect bus 1080 and the memory controller 1060. To instruct.

  Element Interconnect Bus (EIB) 1080 is a logically circular communication bus within Cell processor 928 that includes the processor elements described above (ie, PPE 1050, memory controller 1060, dual bus interfaces 1070A, B, and 8 SPE1010A-H, a total of 12 participants). Participants can simultaneously read and write to the bus at a rate of 8 bytes per clock cycle. As explained above, each SPE 1010A-H comprises a DMAC 1042A-H for scheduling a long read or write sequence. The EIB comprises four channels, two of which are clockwise and the other two are counterclockwise. Therefore, for 12 participants, the longest data flow in a stepped manner between two participants is 6 steps in the appropriate direction. Thus, when arbitration is performed between participants and full utilization is achieved, the theoretical peak instantaneous EIB bandwidth of 12 slots is 96B per clock. This corresponds to a theoretical peak bandwidth of 307.2 GB / s (gigabytes / second) at a clock rate of 3.2 GHz.

  The memory controller 1060 includes an XDRAM interface 1062 developed by Rambus Incorporated. The memory controller interfaces with the Rambus XDRAM 926 with a theoretical peak bandwidth of 25.6 GB / s.

  The dual bus interface 1070A, B includes a Rambus FlexIO (registered trademark) system interface 1072A, B. Each interface is organized into 12 channels of 8 bit width, of which 5 paths are for reception and 7 paths are for transmission. As a result, the theoretical peak bandwidth between the Cell processor and the I / O bridge 700 via the controller 170A and between the Reality Simulator graphic unit 200 via the controller 170B is 62.4 GB / s (transmission 36.4 GB). / S, reception 26 GB / s).

  The data sent to the Reality Simulator graphic unit 930 by the Cell processor 928 typically includes a display list, which is used for drawing vertices, pasting textures on polygons, specifying lighting conditions, etc. A series of commands.

  In embodiments, depth data may be captured to more accurately identify real-world users and guide avatar or scene activity. The object may be an object held by a person or a person's hand. As used herein, the terms “depth camera” and “three-dimensional camera” refer to any camera capable of obtaining distance information, ie depth information, as well as two-dimensional pixel information. For example, the depth camera can acquire distance information using controlled infrared illumination. Another exemplary depth camera is a stereo camera pair, which uses two reference cameras to determine distance information by triangulation. Similarly, the term “depth sensing device” refers to any type of device capable of obtaining distance information in addition to two-dimensional pixel information.

  Recent advances in 3D images have opened up the possibility of real-time interactive computer animation. More specifically, the new “depth camera” can provide the ability to capture and map the third dimension in addition to the normal two-dimensional video image. New depth data allows various embodiments of the invention to place computer-generated objects in various locations in the video scene, such as behind other objects, in real time.

  Furthermore, various embodiments of the present invention provide users with a real-time interactive gaming experience. For example, the user can interact with various computer-generated objects in real time. Furthermore, the video scene can be changed in real time to make the user's gaming experience more realistic. For example, a computer-generated costume can be placed on top of a user's clothes or a virtual shadow can be displayed in a video scene using a computer-generated light source. Thus, using various embodiments of the present invention and a depth camera, a user can experience an interactive game environment in his / her living room. Similar to a normal camera, a depth camera captures two-dimensional data of a plurality of pixels that make up a video image. These values are pixel color values and are typically the red, green, and blue (RGB) values of each pixel. In this way, the object captured by the camera is displayed on the monitor as a two-dimensional object.

  Various embodiments of the present invention also consider distributed image processing configurations. For example, the present invention is not limited to the processing of a captured image and a display image performed at one place (CPU or the like) or two places (CPU and another one element or the like). For example, input image processing can be easily performed in an associated CPU, processor or device capable of performing the processing, and basically all of the image processing can be distributed throughout the interconnect system. . Thus, the present invention is not limited to a specific image processing hardware circuit and / or software. Also, the embodiments described herein are not limited to specific combinations of general hardware circuitry and / or software, and are not limited to specific sources of instructions executed by processing components.

  In view of the above embodiments, it should be understood that the present invention may use various computer-implemented operations that use data stored in a computer system. These operations include operations that require physical manipulation of physical quantities. This physical quantity typically takes the form of an electrical or magnetic signal that can be manipulated, stored, transferred, combined, compared, etc., but is not necessarily limited thereto. Furthermore, the operations performed are often referred to as generation, identification, determination or comparison.

  The above described invention may be practiced with other computer system configurations such as portable devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers and the like. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.

  The present invention may also be implemented as computer readable code on a computer readable medium. The computer readable medium may be any data storage device that can store data which can be read later by a computer system, including an electromagnetic carrier wave. Examples of computer readable media include hard disks, network attached storage (NAS), read only memory, random access memory, CD-ROM, CD-R, CD-RW, magnetic tape and other optical data storage and non- There are optical data storage devices and the like. The computer readable medium may also be distributed via a computer system coupled to a network so that the computer readable code is stored and executed in a distributed fashion.

  Although the invention has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. Accordingly, the embodiments are illustrative and not limiting and the invention is not limited to the details described herein, but the appended claims and their equivalents. May be changed within.

100 avatar control system 102 user 104 virtual space 106 screen 108 controller 110 console 112 camera 114 virtual space 200 graphic unit 900 system unit 902 game controller 904 remote control 906 keyboard 908 mouse 910 portable entertainment device 912 video camera 914 microphone headset

Claims (20)

A method for controlling an avatar in a virtual space accessed via a computer network using a console executing a computer program comprising:
Capturing console controller activity;
Processing the captured activity of the console controller to identify input parameters;
Mapping a selected parameter of the input parameters to a portion of the avatar that is a virtual space representation of a user;
The method wherein the capture, the processing and the mapping are performed sequentially to define a correlation between the console controller activity and the avatar that is the virtual space representation of the user.   The method of claim 1, wherein the console controller activity includes selecting a controller button.   The method of claim 1, wherein the console controller activity includes detecting rotation and translation of the console controller in a three-dimensional space.   The method of claim 3, wherein the input parameters include a rate of change of the rotation and translation speed of the console controller.   The method of claim 1, wherein the avatar portion includes a specific body part that is animated.   5. The method of claim 4, wherein the mapping of selected parameters of the input parameters to the avatar portion is performed in proportion to acceleration and deceleration of the rotation and translation speed of the console controller.   The method of claim 1, wherein the input parameter includes a selection of a controller button, and the controller button is mapped to initiate user control of a different portion of the avatar animation upon selection.   The method of claim 7, wherein the avatar animation is responsive to and proportional to the three-axis translation and rotation movements of the controller, acceleration and deceleration controlled by a user. A method for interactively controlling an avatar over a computer network using a console, comprising:
Providing a console controller;
Determining a first position of the console controller;
Capturing input to the console controller comprising detecting movement of the console controller to a second position;
Processing the input to the console controller and the relative movement between the first position and the second position of the console controller;
Mapping the relative movement between the first position and the second position of the console controller to an animated body part of the avatar;
The method wherein the capture, the processing and the mapping are performed sequentially to define a correlation between the relative motion of the console controller and the avatar.   The method of claim 9, wherein the input to the controller includes selection of a console button.   The method of claim 10, wherein input to the console changes the mapping of relative motion of the console controller to a different part of the avatar.   The method of claim 9, wherein the movement of the console controller is detectable on the three axes including three axis translational and rotational movements.   The method of claim 9, wherein acceleration of the console controller is included as part of capturing the input and detecting the movement of the console controller.   The method of claim 9, wherein the avatar portion includes a specific body part that is animated.   The method of claim 9, wherein the avatar is a virtual representation of a user in a virtual environment, and the virtual environment for the avatar is accessed via the computer network and rendered by the console. A computer implemented method for interactively controlling an avatar in a virtual environment for an avatar and virtual environment generated by a computer program executed on at least one computer in a computer network, comprising:
Providing a controller interfaced with the computer program;
Mapping controller inputs so that a user can control selected portions of the avatar;
Capturing controller input and movement of the controller between a first position and a second position;
Processing the captured controller input and controller movement and applying the captured movement to interactively animate the selected portion of the avatar within the virtual environment; and
A method wherein the capture of the controller input and controller movement and the processing are performed sequentially to define a correlation between controller movement and avatar animation.   17. The method of claim 16, wherein the movement of the controller is detectable on all three axes, including three axis translational and rotational movements.   The method of claim 16, wherein capture of controller movement includes six-axis capture of acceleration and deceleration of the controller in rotation and translation of the controller.   The method of claim 16, wherein the controller input includes a selection of a controller button, and the controller button is mapped to initiate user control of a different portion of the avatar animation upon selection.   20. The method of claim 19, wherein the animation of the selected portion of the avatar is responsive to and proportional to user-controlled acceleration and deceleration of the three-axis translation and rotation motion of the controller.

Images