JP6246991B2 - Information processing program, information processing apparatus, information processing system, and information processing method - Google Patents

Description

  The present invention relates to an information processing program, an information processing apparatus, an information processing system, and an information processing method, and more particularly to an information processing program, an information processing apparatus, an information processing system, and an information processing method for performing processing for displaying a virtual world. .

  Conventionally, there is a game in which a user holds and operates a portable game device and displays a game screen corresponding to the operation on the portable game device (see, for example, Patent Document 1). In the portable game device described in Patent Document 1, a game image is generated based on various user operations performed on operation keys, a touch panel, and the like, and a game is displayed on a display unit provided in the portable game device. The game is progressing with images displayed.

JP 2011-56049 A

  However, in the portable game device disclosed in Patent Document 1, a game image is displayed on the basis of a predetermined direction on a display unit provided in the portable game device. Therefore, when the direction in which the user holds the portable game device is changed, the game image displayed on the display unit is difficult to see.

  Therefore, an object of the present invention is to display an image suitable for a device regardless of the direction in which the user holds the device when displaying the image of the virtual world on a display device that the user can hold and view the screen. An information processing program, an information processing apparatus, an information processing system, and an information processing method that can be displayed are provided.

  In order to achieve the above object, the present invention may employ the following configuration, for example. When interpreting the description of the claims, it is understood that the scope should be interpreted only by the description of the claims, and the description of the claims and the description in this column are contradictory. In that case, priority is given to the claims.

  One configuration example of the information processing program of the present invention is an information processing apparatus capable of displaying an image on the portable display device that outputs at least data according to the attitude and / or movement of the portable display device body. Run on computer. The information processing program causes the computer to function as rotation information calculation means, object control means, first virtual camera control means, and display control means. The rotation information calculation means calculates the rotation direction of the portable display device around a predetermined direction in the real space based on data output from the portable display device. The object control means controls the posture of the object by rotating the object arranged in the virtual world in the rotation direction around the direction corresponding to the predetermined direction set in the virtual world. The first virtual camera control means rotates a first virtual camera for generating an image of a virtual world including at least an object in a rotation direction around a direction corresponding to a predetermined direction set in the virtual world. The attitude of the first virtual camera is controlled. The display control means displays a first image indicating the virtual world viewed from the first virtual camera on the portable display device.

  The information processing apparatus may be an apparatus that executes a game process and generates an image based on the game process, or may be a versatile apparatus such as a general personal computer. The portable display device may be a size that can be carried by a user, and is typically a display device that allows a user to visually recognize an image displayed on the portable display device in a posture held by both hands. Also good. Further, the portable display device is a device other than a means for outputting at least data corresponding to the attitude and / or movement of the portable display device body and a means for displaying the first image, like a terminal device in an embodiment described later. Other configurations may or may not be provided.

  According to the above, when displaying an image of the virtual world on the portable display device, it is possible to display a suitable image on the portable display device regardless of the direction in which the user holds the portable display device. Moreover, according to the attitude | position of a portable display apparatus, while being able to display a suitable image, the attitude | position of the object of the virtual world currently displayed on the portable display apparatus can also be controlled simultaneously.

  The rotation information calculation means may further calculate the rotation amount of the portable display device around a predetermined direction in the real space based on data output from the portable display device. The object control means controls the posture of the object by rotating the object in the rotation direction by a rotation amount based on the rotation amount of the portable display device around the direction corresponding to the predetermined direction set in the virtual world. May be. The first virtual camera control means rotates the first virtual camera in the rotation direction by a rotation amount based on the rotation amount of the portable display device around the direction corresponding to the predetermined direction set in the virtual world. You may control the attitude | position of a 1st virtual camera.

  According to the above, a suitable image can be displayed according to the rotation amount of the portable display device, and the attitude of the object in the virtual world displayed on the portable display device can be simultaneously controlled.

  The first virtual camera control means may control the attitude of the first virtual camera by rotating the first virtual camera in the rotation direction by the same rotation amount as the rotation amount of the portable display device.

  According to the above, in order to rotate the virtual camera that generates the virtual world to be displayed on the portable display device with the same rotation direction and rotation amount as the rotation amount of the portable display device, the user can You can enjoy the feeling of looking through the virtual world through the device.

  The information processing program may further cause the computer to function as gravity direction calculation means. The gravity direction calculating means calculates the gravity direction with respect to the portable display device based on data output from the portable display device. The rotation information calculation unit may calculate the rotation direction and the rotation amount of the portable display device based on the direction of gravity. When the first image is displayed on the portable display device, the first virtual camera control means is displayed as the gravity direction of the portable display device displaying the first image and the first image. You may control the attitude | position of a 1st virtual camera so that the gravity direction of a virtual world may correspond.

  According to the above, since the direction of the virtual world displayed on the portable display device can be changed so as to coincide with the direction of gravity applied to the portable display device, the user can change the real space into the virtual world. You can feel as if you were.

  In addition, the object control means rotates the object in the rotation direction by the same rotation amount as the rotation amount of the portable display device, and controls the object to a posture and position where the positional relationship with the first virtual camera is fixed. Also good.

  According to the above, an object in the virtual world can be displayed at a predetermined position on the display screen of the portable display device, and the operation of the object according to the attitude of the portable display device becomes easy.

  The rotation information calculation unit may calculate a rotation direction and an amount of rotation of the portable display device around the predetermined axis with a predetermined axis provided in the portable display device as a predetermined direction.

  According to the above, it is possible to control the posture of the first virtual camera or the object by moving the portable display device or changing the posture so as to rotate at least around a predetermined axis provided in the portable display device. It becomes possible.

  The rotation information calculation means calculates a rotation direction and a rotation amount of the portable display device around the depth direction with the depth direction of the display surface displaying the first image on the portable display device as a predetermined axis. Also good. The first virtual camera control means sets the line-of-sight direction of the first virtual camera as a direction corresponding to the depth direction, and rotates the first virtual camera in the rotation direction around the line-of-sight direction with the same rotation amount as the rotation amount of the portable display device. May be rotated.

  According to the above, by moving the portable display device or changing the posture so as to roll at least around the depth direction of the display surface of the portable display device, the posture of the first virtual camera or object is changed to the first virtual camera. It is possible to roll around the viewing direction of the camera.

  In addition, the rotation information calculation unit is configured to perform the portable display in the vertical direction and the horizontal direction, respectively, with the vertical and horizontal directions of the portable display device orthogonal to the depth direction and orthogonal to each other as predetermined axes. The rotation direction and rotation amount of the device may be further calculated. The first virtual camera control means has a first virtual direction with the same rotation direction and rotation amount as the rotation direction and rotation amount around the vertical direction of the portable display device, with the vertical direction orthogonal to the viewing direction of the first virtual camera as the center. The first virtual camera is rotated with the same rotation direction and rotation amount as the rotation direction and rotation amount around the horizontal direction of the portable display device around the line-of-sight direction of the first virtual camera and the horizontal direction orthogonal to the vertical direction. The camera may be rotated.

  According to the above, in addition to rolling around the depth direction of the display surface of the portable display device, it is also possible to move the portable display device or change its posture so as to pitch and yaw the portable display device. The attitude of the first virtual camera and the object can be controlled.

  The first virtual camera control means may control the attitude of the first virtual camera in the virtual world so that the attitude is the same as the attitude of the portable display device in real space.

  According to the above, since the posture of the first virtual camera is controlled to be the same as the posture of the portable display device, the portable display device is, for example, according to the user pointing the portable display device in the direction that the user wants to see. It is possible to provide the user with an image as if he is peeking through the virtual world through the user, and to give the user a sense of being in the virtual world.

  The display control means may output image data indicating the first image to the portable display device. The portable display device may include image data acquisition means and display means. The image data acquisition unit acquires image data output from the information processing apparatus. The display means displays the first image indicated by the image data acquired by the image data acquisition means.

  Based on the above, the portable display device can function as a so-called thin client terminal that does not execute information processing.

  The information processing program may further cause the computer to function as compressed image generation means. The compressed image generation means generates compressed image data by compressing image data indicating the first image. In this case, the display control means may output the compressed image data generated by the compressed image generation means to the portable display device. The image data acquisition unit may acquire compressed image data output from the information processing apparatus. The portable display device may further include display image expansion means. The display image expansion means expands the compressed image data to obtain image data indicating the first image. The display unit may display the first image indicated by the image data acquired by the image data acquisition unit and expanded by the display image expansion unit.

  According to the above, since the first image is compressed and output from the information processing device to the portable display device, the first image can be output at high speed, and the first image is generated after the first image is generated. Can be delayed until the image is displayed on the portable display device.

  The display control means may further display a second image indicating the virtual world viewed from the second virtual camera on another display device connected to the information processing device, separately from the first image.

  The other display device is a display device connected to the information processing device like the monitor 2 in an embodiment described later, and may be a separate body from the portable display device, and is generated by the information processing device. Any other image can be used as long as it can display the second image. For example, the another display device may be configured integrally with the information processing device (in one housing).

  According to the above, when processing based on an operation for changing or moving the attitude of the portable display device is performed, the processing result is not only transmitted to the portable display device but also to another display device connected to the information processing device. Can also be displayed. Therefore, for example, the user can use the images displayed on the two devices according to the operation status and preference, and can also view an image suitable for the user operation. Further, an image to be displayed on another display device connected to the information processing device can be used as an image for other people to view, for example, different from the user, and is also suitable when a plurality of people view the processing result. Can be a comfortable viewing environment.

  The information processing program may further cause the computer to function as compressed image generation means. The compressed image generation means generates compressed image data by compressing image data indicating the first image. In this case, the display control unit outputs the compressed image data generated by the compressed image generation unit to the portable display device, and separately from the compressed image data, the image data indicating the second image is not compressed. You may output to a display apparatus. The portable display device may include an image data acquisition unit, a display image expansion unit, and a display unit. The image data acquisition unit acquires the compressed image data output from the information processing apparatus. The display image expansion means expands the compressed image data to obtain image data indicating the first image. The display means displays the first image indicated by the image data acquired by the image data acquisition means and expanded by the display image expansion means.

  According to the above, since the first image is compressed and output from the information processing device to the portable display device, the first image can be output at high speed, and the first image is generated after the first image is generated. Can be delayed until the image is displayed on the portable display device.

  The information processing program may further cause the computer to function as second virtual camera control means. Based on the position of the object in the virtual world, the second virtual camera control means sets the second virtual camera for generating an image of the virtual world at a position different from the first virtual camera and the object is the second Set to be included in the image.

  According to the above, the same object is displayed not only on the portable display device but also on another display device, and images of virtual worlds having different viewpoints are displayed. Therefore, when operating the object, the user can use the images displayed on the two devices according to the operation status and preference.

  The second virtual camera control means may set the second virtual camera at a position farther from the object than the distance from the object to the first virtual camera. The display control means may display a range wider than the range of the virtual world indicated by the first image on another display device as the second image.

  According to the above, the virtual world image having a wider display range than the virtual world image displayed on the portable display device is displayed on another display device connected to the information processing device. An image suitable for the user operation when presented to the user can be displayed on each display device.

  Further, the second virtual camera control means may set the second virtual camera at a position where the object is bird's-eye view in the virtual world. The display control means may display an image in which a bird's eye view of an object placed in the virtual world is displayed on another display device as a second image.

  According to the above, an image of a virtual world corresponding to the attitude of the portable display device is displayed on the portable display device, and an image of bird's-eye view of the virtual world is displayed on another display device connected to the information processing device. Thus, for example, an image suitable for a user operation when presenting the state of the virtual world to the user can be displayed on each display device.

  The portable display device may include at least one of a gyro sensor and an acceleration sensor that outputs data according to the attitude and / or movement of the portable display device body. In this case, the rotation information calculation unit may calculate the rotation direction and the rotation amount of the portable display device based on data output from at least one of the gyro sensor and the acceleration sensor.

  According to the above, by using the data indicating the angular velocity generated in the portable display device output from the gyro sensor and / or the data indicating the acceleration generated in the portable display device output from the acceleration sensor, The attitude and movement of the portable display device can be accurately calculated.

  In addition, the present invention may be implemented in the form of an information processing apparatus and an information processing system including the above-described units and an information processing method including operations performed by the above units.

  According to the present invention, when displaying an image of the virtual world on the portable display device, it is possible to display a suitable image on the portable display device regardless of the direction in which the user holds the portable display device. .

1 is an external view showing an example of a game system 1 according to an embodiment of the present invention. Functional block diagram showing an example of the game apparatus body 5 of FIG. The figure which shows an example of an external appearance structure of the terminal device 6 of FIG. The figure which shows an example of a mode that the user hold | gripped the terminal device 6. The block diagram which shows an example of an internal structure of the terminal device 6 of FIG. The perspective view which shows an example of the external appearance of the board type controller 9 of FIG. The figure which shows an example of the AA sectional drawing of the board-type controller 9 shown in FIG. 6, and an example by which the part of the corner where the load sensor 94 is arrange | positioned is enlargedly displayed. FIG. 6 is a block diagram showing an example of an electrical configuration of the board type controller 9 of FIG. The figure which shows an example of the mode of the user who operates using the terminal device 6 and the board type controller 9 The figure which shows an example of the image displayed on LCD61 of the terminal device 6 The figure which shows an example of the image displayed on the monitor 2 The figure which shows an example of the image displayed on LCD61 when the terminal device 6 is seen in a normal vision state. 11 is a diagram showing an example in which the image shown in FIG. 11A is displayed on the LCD 61 of the terminal device 6 in the normal viewing state. The figure which shows an example of the image displayed on LCD61 when the terminal device 6 is rolled in the counterclockwise direction from a normal viewing state. FIG. 12A is a diagram showing an example in which the image shown in FIG. 12A is displayed on the LCD 61 of the terminal device 6 rotated in a counterclockwise direction from the normal viewing state. The figure which shows an example of the image displayed on LCD61 when the terminal device 6 is rolled in the counterclockwise direction from the normal viewing state and viewed in the vertical holding state The figure which shows an example which displayed the image shown by FIG. 13A on LCD61 of the terminal device 6 used as the vertical holding state The figure for demonstrating an example of the relationship of the operation instruction | indication direction projected on the vertical plane of the virtual space of the terminal device projected on the vertical plane of real space, and the rotation operation corresponding to a terminal device upward direction (roll angle) The figure which shows an example of main data and a program memorize | stored in the main memory of the game device main body 5 of FIG. The flowchart which shows an example of the information processing performed in the game device main body 5 of FIG. A subroutine showing an example of the game control process in step 44 in FIG. A subroutine showing an example of the player object setting process in step 83 in FIG. Subroutine showing an example of the player object action setting process in step 127 in FIG. The figure for demonstrating an example of the relationship between the real space reference direction and terminal device depth direction which were projected on the horizontal surface of real space, and the virtual world reference direction and operation instruction direction which were projected on the horizontal surface of the virtual world

  With reference to FIG. 1, an information processing apparatus that executes an information processing program according to an embodiment of the present invention and an information processing system including the information processing apparatus will be described. Hereinafter, for the sake of specific explanation, a stationary game apparatus body 5 will be used as an example of the information processing apparatus, and a game system including the game apparatus body 5 will be described. FIG. 1 is an external view showing an example of a game system 1 including a stationary game apparatus 3. FIG. 2 is a block diagram illustrating an example of the game apparatus body 5. Hereinafter, the game system 1 will be described.

  In FIG. 1, a game system 1 includes a home television receiver (hereinafter referred to as a monitor) 2 which is an example of display means, and a stationary game apparatus 3 connected to the monitor 2 via a connection cord. Composed. The monitor 2 includes a speaker 2a for outputting the audio signal output from the game apparatus 3 as audio. In addition, the game apparatus 3 executes an optical disc 4 on which a program (for example, a game program) as an example of the information processing program of the present invention is recorded, and executes the program on the optical disc 4 to display and output a game screen on the monitor 2. A game apparatus body 5 equipped with a computer, a terminal device 6, a controller 7 for giving the game apparatus body 5 operation information necessary for operating an object or the like displayed on the display screen, and a board-type controller 9 including. The game system 1 executes a game process in the game apparatus body 5 based on a game operation using at least one of the terminal device 6, the controller 7, and the board type controller 9, and monitors a game image obtained by the game process 2 and / or the terminal device 6. Note that the game apparatus body 5, the terminal device 6, the controller 7, and the board-type controller 9 are connected so as to be capable of wireless communication by radio. For example, the wireless communication is executed in accordance with the Bluetooth (registered trademark) standard or the IEEE 802.11n standard, but may be executed in accordance with other standards such as infrared rays.

  An optical disc 4, which is an example of an information storage medium that can be used interchangeably with respect to the game device main body 5, is detachably inserted into the game device main body 5. The optical disc 4 stores an information processing program (typically a game program) to be executed in the game apparatus body 5. An insertion slot for the optical disc 4 is provided on the front surface of the game apparatus body 5. The game apparatus body 5 executes the game process by reading and executing the information processing program stored in the optical disc 4 inserted into the insertion slot.

  The monitor 2 is connected to the game apparatus body 5 via a connection cord. The monitor 2 displays a game image obtained by a game process executed in the game apparatus body 5. The monitor 2 has a speaker 2a, and the speaker 2a outputs game sound obtained as a result of the game processing. In other embodiments, the game apparatus body 5 and the stationary display apparatus may be integrated. Communication between the game apparatus body 5 and the monitor 2 may be wireless communication.

  The game apparatus body 5 is equipped with a flash memory 17 (see FIG. 2) that functions as a backup memory that stores data such as save data in a fixed manner. The game apparatus main body 5 displays the result as a game image on the monitor 2 and / or the terminal device 6 by executing a game program or the like stored on the optical disc 4. The game program or the like is not limited to the optical disc 4 and may be executed in advance recorded in the flash memory 17. Further, the game apparatus body 5 can reproduce the game state executed in the past by using the save data stored in the flash memory 17 and display the game image on the monitor 2 and / or the terminal device 6. . Then, the user of the game device 3 operates at least one of the terminal device 6, the controller 7, and the board type controller 9 while watching the game image displayed on the monitor 2 and / or the terminal device 6, thereby playing the game You can enjoy the progress.

  The controller 7 and the board-type controller 9 wirelessly transmit transmission data such as operation information to the game apparatus body 5 incorporating the controller communication module 19 by using, for example, Bluetooth technology. The controller 7 is an operation means for mainly selecting an option displayed on the display screen of the monitor 2. The controller 7 is provided with a housing large enough to be held with one hand, and a plurality of operation buttons (including a cross key and the like) exposed on the surface of the housing. Further, as will be apparent from the description below, the controller 7 includes an imaging information calculation unit that captures an image viewed from the controller 7. Then, as an example of the imaging target of the imaging information calculation unit, two LED modules (hereinafter referred to as markers) 8L and 8R are installed near the display screen of the monitor 2 (upper side of the screen in FIG. 1). Although details will be described later, the user (player) can perform a game operation to move the controller 7, and the marker 8 is used by the game apparatus body 5 to calculate the movement, position, posture, and the like of the controller 7. The marker 8 includes two markers 8L and 8R at both ends thereof. The marker 8R (same for the marker 8L) is specifically one or more infrared LEDs (Light Emitting Diodes), and outputs infrared light toward the front of the monitor 2. The marker 8 is connected to the game apparatus body 5, and the game apparatus body 5 can control lighting of each infrared LED included in the marker 8. The marker 8 is portable, and the user can install the marker 8 at a free position. Although FIG. 1 shows a mode in which the marker 8 is installed on the monitor 2, the position and orientation in which the marker 8 is installed are arbitrary. In addition, the controller 7 can receive transmission data wirelessly transmitted from the controller communication module 19 of the game apparatus body 5 by the communication unit, and can generate sound and vibration corresponding to the transmission data.

  In other embodiments, the controller 7 and / or the board-type controller 9 and the game apparatus body 5 may be connected by wire. In this embodiment, the game system 1 includes one controller 7 and one board-type controller 9, but the game apparatus body 5 can communicate with a plurality of controllers 7 and a plurality of board-type controllers 9. By using the number of controllers 7 and the board type controller 9 at the same time, it is possible to play a game with a plurality of people.

  The controller 7 has a housing formed by plastic molding, for example, and a plurality of operation portions (operation buttons) are provided in the housing. Then, the controller 7 transmits operation data indicating an input state to the operation unit (whether or not each operation button is pressed) to the game apparatus body 5.

  In addition, the controller 7 has an imaging information calculation unit that analyzes the image data captured by the imaging unit and determines a region having high luminance in the region, thereby calculating the position of the center of gravity, the size, and the like of the region. For example, the imaging information calculation unit includes imaging means fixed to the housing of the controller 7 and targets a marker that outputs infrared light such as the marker unit 65 and / or the marker 8 of the terminal device 6 as an imaging target. Then, the imaging information calculation unit calculates the position of the imaging target in the captured image captured by the imaging unit, and transmits marker coordinate data indicating the calculated position to the game apparatus body 5. Since this marker coordinate data changes corresponding to the direction (inclination angle) and position of the controller 7 itself, the game apparatus body 5 can calculate the direction and position of the controller 7 using this marker coordinate data.

  Further, the controller 7 includes an acceleration sensor and / or a gyro sensor. The acceleration sensor detects acceleration (including gravitational acceleration) generated in the controller 7 and transmits acceleration data indicating the detected acceleration to the game apparatus body 5. Since the acceleration detected by the acceleration sensor changes in accordance with the direction (tilt angle) and movement of the controller 7 itself, the game apparatus body 5 calculates the direction and movement of the controller 7 using the acquired acceleration data. Can do. The gyro sensor detects angular velocities around the three axes set in the controller 7 and transmits angular velocity data indicating the detected angular velocities to the game apparatus body 5. Since the angular velocity detected by the gyro sensor changes in accordance with the direction (tilt angle) and movement of the controller 7 itself, the game apparatus body 5 calculates the direction and movement of the controller 7 using the acquired angular velocity data. Can do. In this way, the user can perform a game operation by pressing an operation unit provided in the controller 7 and moving the controller 7 itself to change its position and posture (tilt).

  The controller 7 is provided with a speaker and a vibrator. The controller 7 processes the sound data transmitted from the game apparatus body 5 and outputs sound corresponding to the sound data from the speaker. Further, the controller 7 processes the vibration data transmitted from the game apparatus body 5 and activates the vibrator according to the vibration data to generate vibration. In the embodiment of the present invention to be described later, it is possible to play a game without using the controller 7. The detailed configuration of the board type controller 9 will be described later.

  The terminal device 6 is large enough to be gripped by the user, and the user can use the terminal device 6 by holding it in his hand or by placing the terminal device 6 in a free position. This is a portable device. Although the detailed configuration will be described later, the terminal device 6 includes an LCD (Liquid Crystal Display) 61 that is a display means, and input means (a touch panel 62, a gyro sensor 604, etc., which will be described later). The terminal device 6 and the game apparatus body 5 (terminal communication module 28 (see FIG. 2)) can communicate wirelessly (may be wired). The terminal device 6 receives data of an image (for example, a game image) generated by the game apparatus body 5 from the game apparatus body 5 and displays an image indicated by the data on the LCD 61. In this embodiment, an LCD is used as the display device. However, the terminal device 6 may have any other display device such as a display device using EL (Electro Luminescence). . In addition, the terminal device 6 transmits operation data representing the content of the operation performed on the own device to the game apparatus body 5 including the terminal communication module 28.

  Next, the internal configuration of the game apparatus body 5 will be described with reference to FIG. FIG. 2 is a block diagram illustrating an example of the configuration of the game apparatus body 5. The game apparatus body 5 includes a CPU (Central Processing Unit) 10, a system LSI (Large Scale Integration) 11, an external main memory 12, a ROM / RTC (Read Only Memory / Real Time Clock) 13, a disk drive 14, and an AV-IC. (Audio Video-Integrated Circuit) 15 and the like.

  The CPU 10 executes processing by executing a program stored on the optical disc 4 and functions as a game processor. The CPU 10 is connected to the system LSI 11. In addition to the CPU 10, an external main memory 12, a ROM / RTC 13, a disk drive 14, and an AV-IC 15 are connected to the system LSI 11. The system LSI 11 performs processing such as control of data transfer between components connected thereto, generation of an image to be displayed, and acquisition of data from an external device. The internal configuration of the system LSI 11 will be described later. The volatile external main memory 12 stores a program read from the optical disk 4, a program read from the flash memory 17, and various data, and stores a work area and a buffer of the CPU 10. Used as a region. The ROM / RTC 13 includes a ROM (so-called boot ROM) in which a program for starting up the game apparatus body 5 is incorporated, and a clock circuit (RTC) that counts time. The disk drive 14 reads program data, texture data, and the like from the optical disk 4 and writes the read data to the internal main memory 35 or the external main memory 12 described later.

  The system LSI 11 includes an input / output processor (I / O processor) 31, a GPU (Graphics Processor Unit) 32, a DSP (Digital Signal Processor) 33, a VRAM (Video RAM) 34, and an internal main memory 35. Although not shown, these components 31 to 35 are connected to each other by an internal bus.

  The GPU 32 forms part of the drawing means and generates an image in accordance with a graphics command (drawing command) from the CPU 10. The VRAM 34 stores data (data such as polygon data and texture data) necessary for the GPU 32 to execute the graphics command. When an image is generated, the GPU 32 creates image data using data stored in the VRAM 34. In the present embodiment, the game apparatus body 5 may generate both a game image to be displayed on the monitor 2 and a game image to be displayed on the terminal device 6. Hereinafter, a game image displayed on the monitor 2 may be referred to as a “monitor game image”, and a game image displayed on the terminal device 6 may be referred to as a “terminal game image”.

  The DSP 33 functions as an audio processor, and generates sound data using sound data and sound waveform (tone color) data stored in the internal main memory 35 and the external main memory 12. In the present embodiment, both game sound output from the speaker 2a of the monitor 2 and game sound output from the speaker of the terminal device 6 may be generated for the game sound as well as the game image. Hereinafter, the game sound output from the monitor 2 may be referred to as “monitor game sound”, and the game sound output from the terminal device 6 may be referred to as “terminal game sound”.

  Of the images and sounds generated in the game apparatus body 5 as described above, the image data and sound data output to the monitor 2 are read out by the AV-IC 15. The AV-IC 15 outputs the read image data to the monitor 2 via the AV connector 16 and outputs the read audio data to the speaker 2 a built in the monitor 2. As a result, an image is displayed on the monitor 2 and a sound is output from the speaker 2a.

  Of the images and sounds generated in the game apparatus body 5, image data and sound data output to the terminal device 6 are transmitted to the terminal device 6 by the input / output processor 31 or the like. Data transmission to the terminal device 6 by the input / output processor 31 and the like will be described later.

  The input / output processor 31 transmits / receives data to / from components connected to the input / output processor 31 or downloads data from an external device. The input / output processor 31 is connected to the flash memory 17, network communication module 18, controller communication module 19, expansion connector 20, memory card connector 21, and codec LSI 27. An antenna 22 is connected to the network communication module 18. An antenna 23 is connected to the controller communication module 19. The codec LSI 27 is connected to a terminal communication module 28, and an antenna 29 is connected to the terminal communication module 28.

  The game apparatus body 5 can connect to a network such as the Internet and communicate with an external information processing apparatus (for example, another game apparatus or various servers). That is, the input / output processor 31 is connected to the network via the network communication module 18 and the antenna 22 and can communicate with an external information processing apparatus connected to the network. The input / output processor 31 periodically accesses the flash memory 17 to detect the presence / absence of data that needs to be transmitted to the network. If the data is present, the data is sent via the network communication module 18 and the antenna 22. To the network. Further, the input / output processor 31 receives data transmitted from the external information processing apparatus or data downloaded from the download server via the network, the antenna 22 and the network communication module 18, and receives the received data in the flash memory 17. To remember. By executing the program, the CPU 10 reads out the data stored in the flash memory 17 and uses it in the program. In the flash memory 17, in addition to data transmitted and received between the game apparatus body 5 and the external information processing apparatus, save data (process result data or intermediate data) of a game played using the game apparatus body 5 is stored. It may be stored. The flash memory 17 may store a program such as a game program.

  Further, the game apparatus body 5 can receive operation data from the controller 7 and / or the board type controller 9. That is, the input / output processor 31 receives operation data and the like transmitted from the controller 7 and / or the board-type controller 9 via the antenna 23 and the controller communication module 19 and buffers the internal main memory 35 or the external main memory 12. Store in the area (temporary storage). As in the case of the external main memory 12, the internal main memory 35 may store a program read from the optical disc 4, a program read from the flash memory 17, or various data. The CPU 10 may be used as a work area or a buffer area.

  Further, the game apparatus body 5 can transmit and receive data such as images and sounds to and from the terminal device 6. When the game image (terminal game image) is transmitted to the terminal device 6, the input / output processor 31 outputs the game image data generated by the GPU 32 to the codec LSI 27. The codec LSI 27 performs predetermined compression processing on the image data from the input / output processor 31. The terminal communication module 28 performs wireless communication with the terminal device 6. Therefore, the image data compressed by the codec LSI 27 is transmitted to the terminal device 6 via the antenna 29 by the terminal communication module 28. In the present embodiment, the image data transmitted from the game apparatus body 5 to the terminal device 6 is used for the game. In the game, if the displayed image is delayed, the operability of the game is adversely affected. For this reason, it is preferable that the transmission of image data from the game apparatus body 5 to the terminal device 6 should be as delayless as possible. Therefore, in this embodiment, the codec LSI 27 is, for example, H.264. The image data is compressed using a highly efficient compression technique such as H.264 standard. Other compression techniques may be used, and when the communication speed is sufficient, the image data may be transmitted without compression. The terminal communication module 28 is a communication module that has received, for example, Wi-Fi authentication. For example, the terminal communication module 28 uses a MIMO (Multiple Input Multiple Output) technology adopted in the IEEE 802.11n standard, for example. Wireless communication may be performed at high speed, or another communication method may be used.

  In addition to the image data, the game apparatus body 5 transmits audio data to the terminal device 6. That is, the input / output processor 31 outputs the audio data generated by the DSP 33 to the terminal communication module 28 via the codec LSI 27. The codec LSI 27 performs compression processing on the audio data in the same manner as the image data. The compression method for the audio data may be any method, but a method with a high compression rate and less deterioration of the sound is preferable. In other embodiments, audio data may be transmitted without being compressed. The terminal communication module 28 transmits the compressed image data and audio data to the terminal device 6 via the antenna 29.

  Furthermore, the game apparatus body 5 transmits various control data to the terminal device 6 as needed in addition to the image data and the sound data. The control data is data representing a control instruction for the constituent elements included in the terminal device 6, for example, an instruction for controlling lighting of the marker unit (marker unit 65 shown in FIG. 5) or a camera (camera 66 shown in FIG. 5). Indicates an instruction to control imaging. The input / output processor 31 transmits control data to the terminal device 6 in accordance with an instruction from the CPU 10. With respect to this control data, the codec LSI 27 does not perform data compression processing in the present embodiment, but may perform compression processing in other embodiments. Note that the above-described data transmitted from the game apparatus body 5 to the terminal device 6 may or may not be encrypted as necessary.

  Further, the game apparatus body 5 can receive various data from the terminal device 6. Although details will be described later, in the present embodiment, the terminal device 6 transmits operation data, image data, and audio data. Each data transmitted from the terminal device 6 is received by the terminal communication module 28 via the antenna 29. Here, the image data and audio data from the terminal device 6 are subjected to the same compression processing as the image data and audio data from the game apparatus body 5 to the terminal device 6. Therefore, these image data and audio data are sent from the terminal communication module 28 to the codec LSI 27, subjected to expansion processing by the codec LSI 27, and output to the input / output processor 31. On the other hand, the operation data from the terminal device 6 has a smaller data amount than images and sounds, and therefore may not be subjected to compression processing. Further, encryption may or may not be performed as necessary. Therefore, the operation data is received by the terminal communication module 28 and then output to the input / output processor 31 via the codec LSI 27. The input / output processor 31 stores (temporarily stores) the data received from the terminal device 6 in the buffer area of the internal main memory 35 or the external main memory 12.

  Further, the game apparatus body 5 can be connected to other devices and external storage media. That is, the expansion connector 20 and the memory card connector 21 are connected to the input / output processor 31. The expansion connector 20 is a connector for an interface such as USB or SCSI, and connects a medium such as an external storage medium, a peripheral device such as another controller, or a wired communication connector. By connecting, communication with the network can be performed instead of the network communication module 18. The memory card connector 21 is a connector for connecting an external storage medium such as a memory card. For example, the input / output processor 31 can access an external storage medium via the expansion connector 20 or the memory card connector 21 to store data or read data.

  The game apparatus body 5 (for example, the front main surface) is provided with a power button 24, a reset button 25, an insertion port for attaching / detaching the optical disk 4, an eject button 26 for removing the optical disk 4 from the insertion port of the game apparatus body 5, and the like. It has been. The power button 24 and the reset button 25 are connected to the system LSI 11. When the power button 24 is turned on, power is supplied to each component of the game apparatus body 5. When the reset button 25 is pressed, the system LSI 11 restarts the startup program of the game apparatus body 5. The eject button 26 is connected to the disk drive 14. When the eject button 26 is pressed, the optical disk 4 is ejected from the disk drive 14.

  In other embodiments, some of the components included in the game apparatus body 5 may be configured as expansion devices that are separate from the game apparatus body 5. At this time, the expansion device may be connected to the game apparatus body 5 via the expansion connector 20, for example. Specifically, the expansion device includes, for example, the codec LSI 27, the terminal communication module 28, and the antenna 29, and may be detachable from the expansion connector 20. According to this, the said game device main body can be set as the structure which can communicate with the terminal device 6 by connecting the said extended apparatus with respect to the game device main body which is not provided with said each component.

  Next, the configuration of the terminal device 6 will be described with reference to FIGS. FIG. 3 is a diagram illustrating an example of an external configuration of the terminal device 6. 3A is a front view of the terminal device 6, FIG. 3B is a top view, FIG. 3C is a right side view, and FIG. 3D is a bottom view. FIG. 4 is a diagram illustrating an example of a state in which the user holds the terminal device 6.

  As shown in FIG. 3, the terminal device 6 includes a housing 60 that is roughly a horizontally-long rectangular plate shape. The housing 60 is large enough to be gripped by the user. Accordingly, the user can move the terminal device 6 or change the arrangement position of the terminal device 6.

  The terminal device 6 has an LCD 61 on the surface of the housing 60. The LCD 61 is provided near the center of the surface of the housing 60. Therefore, the user can move the terminal device 6 while looking at the screen of the LCD 61 by holding the housings 60 on both sides of the LCD 61 as shown in FIG. FIG. 4 shows an example in which the user holds the terminal device 6 sideways by holding the housings 60 on both the left and right sides of the LCD 61 (that is, the terminal device 6 is held with the long side direction in the horizontal direction). However, it is also possible to hold the terminal device 6 vertically (that is, hold the terminal device 6 with the long side direction vertical).

  As illustrated in part (a) of FIG. 3, the terminal device 6 includes a touch panel 62 on the screen of the LCD 61 as an operation unit. In the present embodiment, the touch panel 62 is a resistive film type touch panel. However, the touch panel 62 is not limited to the resistive film type, and any type of touch panel such as a capacitance type can be used. The touch panel 62 may be a single touch method or a multi touch method. In this embodiment, a touch panel 62 having the same resolution (detection accuracy) as the resolution of the LCD 61 is used. However, the resolution of the touch panel 62 and the resolution of the LCD 61 are not necessarily matched. The input to the touch panel 62 is normally performed using a touch pen. However, the input to the touch panel 62 is not limited to the touch pen and can be performed by a user's finger. The housing 60 may be provided with a storage hole for storing a touch pen used for performing an operation on the touch panel 62. Thus, since the terminal device 6 includes the touch panel 62, the user can operate the touch panel 62 while moving the terminal device 6. That is, the user can directly input (by the touch panel 62) to the screen while moving the screen of the LCD 61.

  As shown in FIG. 3, the terminal device 6 includes two analog sticks 63A and 63B and a plurality of operation buttons 64A to 64L as operation means. Each analog stick 63A and 63B is a device that indicates a direction. Each analog stick 63A and 63B is configured such that the stick portion operated by the user's finger can slide or tilt in any direction (any angle in the up / down / left / right and diagonal directions) with respect to the surface of the housing 60. Has been. The left analog stick 63A is provided on the left side of the screen of the LCD 61, and the right analog stick 63B is provided on the right side of the screen of the LCD 61. Therefore, the user can input with the left or right hand to instruct the direction using the analog stick 63A or 63B. Further, as shown in FIG. 4, the analog sticks 63 </ b> A and 63 </ b> B are provided at positions where the user can operate while holding the left and right portions of the terminal device 6. Also, the analog sticks 63A and 63B can be easily operated.

  The operation buttons 64A to 64L are operation means for performing predetermined inputs. As shown below, each operation button 64A-64L is provided in the position which a user can operate in the state which hold | gripped the right-and-left part of the terminal device 6 (refer FIG. 4). Therefore, the user can easily operate these operation means even when the user moves the terminal device 6.

  As shown in part (a) of FIG. 3, among the operation buttons 64A to 64L, a cross button (direction input button) 64A and operation buttons 64B to 64H are provided on the surface of the housing 60. These operation buttons 64A to 64G are arranged at positions where they can be operated with the thumb of the user (see FIG. 4).

  The cross button 64A is provided on the left side of the LCD 61 and below the left analog stick 63A. That is, the cross button 64A is arranged at a position where it can be operated with the left hand of the user. The cross button 64 </ b> A has a cross shape and is a button capable of instructing the vertical and horizontal directions. The operation buttons 64B to 64D are provided below the LCD 61. These three operation buttons 64B to 64D are arranged at positions that can be operated by both the left and right hands. The four operation buttons 64E to 64H are provided on the right side of the LCD 61 and below the right analog stick 63B. That is, the four operation buttons 64E to 64H are arranged at positions that can be operated with the user's right hand. Further, the four operation buttons 64E to 64H are arranged so as to have a vertical / left / right positional relationship (relative to the center position of the four operation buttons 64E to 64H). Therefore, the terminal device 6 can also function the four operation buttons 64E to 64H as buttons for instructing the user in the up / down / left / right directions.

  Further, as shown in FIGS. 3A, 3B, and 3C, the first L button 64I and the first R button 64J are provided on an oblique upper portion (upper left portion and upper right portion) of the housing 60. Provided. Specifically, the first L button 64I is provided at the left end of the upper side surface of the plate-like housing 60, and protrudes from the upper and left side surfaces. The first R button 64J is provided at the right end of the upper side surface of the housing 60, and protrudes from the upper and right side surfaces. In this way, the first L button 64I is disposed at a position operable with the user's left index finger, and the first R button 64J is disposed at a position operable with the user's right hand index finger (see FIG. 4).

  As shown in FIGS. 3B and 3C, the second L button 64K and the second R button 64L are provided on the back surface of the plate-like housing 60 (that is, the surface opposite to the surface on which the LCD 61 is provided). Are protruded from the leg portions 68A and 68B. Specifically, the second L button 64K is provided slightly above the left side (left side when viewed from the front side) of the housing 60, and the second R button 64L is provided on the right side (from the front side) of the back side of the housing 60. It is provided slightly above the right side when viewed. In other words, the second L button 64K is provided at a position substantially opposite to the left analog stick 63A provided on the front surface, and the second R button 64L is provided at a position substantially opposite to the right analog stick 63B provided on the front surface. It is done. The second L button 64K is disposed at a position that can be operated by the user's left hand middle finger, and the second R button 64L is disposed at a position that can be operated by the user's right hand middle finger (see FIG. 4). Further, as shown in FIG. 3C, the second L button 64K and the second R button 64L are provided on the diagonally upward surfaces of the feet 68A and 68B, and have button surfaces that are diagonally upward. When the user grips the terminal device 6, it is considered that the middle finger moves in the vertical direction. Therefore, the user can easily press the second L button 64K and the second R button 64L by turning the button surface upward. Further, by providing the foot portions 68A and 68B on the back surface of the housing 60, the user can easily grip the housing 60, and the operation buttons are provided on the foot portions 68A and 68B, so that the housing 60 is held. It becomes easy to operate.

  3, since the second L button 64K and the second R button 64L are provided on the back surface, the terminal device 6 is placed with the screen of the LCD 61 (the surface of the housing 60) facing upward. The screen may not be completely horizontal. Therefore, in other embodiments, three or more legs may be formed on the back surface of the housing 60. According to this, since the terminal device 6 can be placed so that three or more feet are in contact with the floor surface when the screen of the LCD 61 is facing upward, the terminal device 6 is placed so that the screen is horizontal. can do. Alternatively, the terminal device 6 may be placed horizontally by attaching a detachable foot.

  Functions corresponding to the game program are appropriately assigned to the operation buttons 64A to 64L. For example, the cross button 64A and the operation buttons 64E to 64H may be used for a direction instruction operation or a selection operation, and the operation buttons 64B to 64E may be used for a determination operation or a cancel operation.

  Although not shown, the terminal device 6 has a power button for turning on / off the terminal device 6. Further, the terminal device 6 has an operation button for turning on / off the screen display of the LCD 61, an operation button for setting connection (pairing) with the game apparatus body 5, and a speaker (a speaker 607 shown in FIG. 5). ) May have an operation button for adjusting the volume.

  As shown in part (a) of FIG. 3, the terminal device 6 includes a marker part (marker part 65 shown in FIG. 5) including a marker 65 </ b> A and a marker 65 </ b> B on the surface of the housing 60. As an example, the marker unit 65 is provided on the upper side of the LCD 61. Like the markers 8L and 8R of the marker 8, the marker 65A and the marker 65B are each composed of one or more infrared LEDs. Similar to the marker 8 described above, the marker unit 65 is used by the game apparatus body 5 to calculate the movement of the controller 7 with respect to the marker unit 65. Further, the game apparatus body 5 can control lighting of each infrared LED included in the marker unit 65.

  The terminal device 6 includes a camera 66 that is an imaging unit. The camera 66 includes an imaging device (for example, a CCD image sensor or a CMOS image sensor) having a predetermined resolution, and a lens. For example, the camera 66 is provided on the surface of the housing 60. Therefore, the camera 66 can take an image of the face of the user who has the terminal device 6, and can take an image of the user who is playing the game while viewing the LCD 61 as an example.

  The terminal device 6 includes a microphone (a microphone 609 shown in FIG. 5) that is a voice input unit. A microphone hole 60 b is provided on the surface of the housing 60. The microphone 609 is provided inside the housing 60 that is inside the microphone hole 60b. The microphone 609 detects sounds around the terminal device 6 such as a user's voice.

  The terminal device 6 includes a speaker (speaker 607 shown in FIG. 5) that is an audio output means. As shown in part (d) of FIG. 3, a speaker hole 60 a is provided on the lower side surface of the housing 60. The output sound of the speaker 607 is output from the speaker hole 60a. In the present embodiment, the terminal device 6 includes two speakers, and a speaker hole 60a is provided at each position of the left speaker and the right speaker.

  The terminal device 6 includes an expansion connector 67 for connecting other devices to the terminal device 6. In the present embodiment, the extension connector 67 is provided on the lower side surface of the housing 60 as shown in FIG. Note that any other device connected to the expansion connector 67 may be used. For example, a controller (such as a gun-type controller) used for a specific game or an input device such as a keyboard may be used. If it is not necessary to connect another device, the expansion connector 67 may not be provided.

  In addition, regarding the terminal device 6 shown in FIG. 3, the shape of each operation button and the housing 60, the number of components, the installation position, and the like are merely examples, and other shapes, numbers, and installation positions are included. Also good.

  Next, the internal configuration of the terminal device 6 will be described with reference to FIG. FIG. 5 is a block diagram illustrating an example of the internal configuration of the terminal device 6. As shown in FIG. 5, in addition to the configuration shown in FIG. 3, the terminal device 6 includes a touch panel controller 601, a magnetic sensor 602, an acceleration sensor 603, a gyro sensor 604, a user interface controller (UI controller) 605, a codec LSI 606, a speaker. 607, a sound IC 608, a microphone 609, a wireless module 610, an antenna 611, an infrared communication module 612, a flash memory 613, a power supply IC 614, a battery 615, and a vibrator 619. These electronic components are mounted on an electronic circuit board and stored in the housing 60.

  The UI controller 605 is a circuit for controlling data input / output with respect to various input / output units. The UI controller 605 includes a touch panel controller 601, an analog stick 63 (analog sticks 63A and 63B), operation buttons 64 (operation buttons 64A to 64L), a marker unit 65, a magnetic sensor 602, an acceleration sensor 603, a gyro sensor 604, and a vibrator. 619. The UI controller 605 is connected to the codec LSI 606 and the expansion connector 67. Further, a power supply IC 614 is connected to the UI controller 605, and power is supplied to each unit via the UI controller 605. A built-in battery 615 is connected to the power supply IC 614 to supply power. The power supply IC 614 can be connected to a charger 616 or a cable that can acquire power from an external power supply via a connector or the like. The terminal device 6 can be connected to the power supply IC 614 from an external power supply using the charger 616 or the cable. Power supply and charging can be performed. The terminal device 6 may be charged by attaching the terminal device 6 to a cradle having a charging function (not shown).

  The touch panel controller 601 is a circuit that is connected to the touch panel 62 and controls the touch panel 62. The touch panel controller 601 generates touch position data in a predetermined format based on a signal from the touch panel 62 and outputs it to the UI controller 605. The touch position data represents the coordinates of the position where the input is performed on the input surface of the touch panel 62. The touch panel controller 601 reads signals from the touch panel 62 and generates touch position data at a rate of once per predetermined time. Various control instructions for the touch panel 62 are output from the UI controller 605 to the touch panel controller 601.

  The analog stick 63 outputs to the UI controller 605 stick data representing the direction and amount in which the stick unit operated by the user's finger has slid (or tilted). In addition, the operation button 64 outputs operation button data representing the input status (whether or not it has been pressed) to each of the operation buttons 64A to 64L to the UI controller 605.

  The magnetic sensor 602 detects the direction by detecting the magnitude and direction of the magnetic field. The azimuth data indicating the detected azimuth is output to the UI controller 605. Also, a control instruction for the magnetic sensor 602 is output from the UI controller 605 to the magnetic sensor 602. Regarding the magnetic sensor 602, an MI (magnetic impedance) element, a flux gate sensor, a Hall element, a GMR (giant magnetoresistance) element, a TMR (tunnel magnetoresistance) element, an AMR (anisotropic magnetoresistance) element, or the like was used. Although there is a sensor, any sensor may be used as long as it can detect the direction. Strictly speaking, in a place where a magnetic field is generated other than the geomagnetism, the obtained azimuth data does not indicate the azimuth, but even in such a case, the terminal device 6 moves. Since the azimuth data changes, the change in the attitude of the terminal device 6 can be calculated.

  The acceleration sensor 603 is provided inside the housing 60 and detects the magnitude of linear acceleration along the direction of the three axes (the xyz axis shown in FIG. 3A). Specifically, the acceleration sensor 603 sets the long side direction of the housing 60 to the x axis (in the state in which the marker unit 65 is arranged on the upper side of the LCD 61, the right direction along the long side direction toward the display screen of the LCD 61). x-axis positive direction), the short side direction of the housing 60 is the y-axis (the marker unit 65 is arranged on the upper side of the LCD 61, and the upward direction along the short side direction toward the display screen of the LCD 61 is the y-axis. The magnitude of the linear acceleration of each axis is detected with the direction perpendicular to the surface of the housing 60 as the z-axis (the display screen depth direction of the LCD 61 is the z-axis positive direction). Acceleration data representing the detected acceleration is output to the UI controller 605. Further, a control instruction for the acceleration sensor 603 is output from the UI controller 605 to the acceleration sensor 603. The acceleration sensor 603 is, for example, a capacitive MEMS acceleration sensor in the present embodiment, but other types of acceleration sensors may be used in other embodiments. Further, the acceleration sensor 603 may be an acceleration sensor that detects a uniaxial or biaxial direction.

  The gyro sensor 604 is provided inside the housing 60 and detects angular velocities around the three axes of the x-axis, the y-axis, and the z-axis. Angular velocity data representing the detected angular velocity is output to the UI controller 605. Further, a control instruction for the gyro sensor 604 is output from the UI controller 605 to the gyro sensor 604. Any number and combination of gyro sensors may be used for detecting the triaxial angular velocity, and the gyro sensor 604 may be composed of a two-axis gyro sensor and a one-axis gyro sensor. Good. Further, the gyro sensor 604 may be a gyro sensor that detects a uniaxial or biaxial direction.

  The vibrator 619 is, for example, a vibration motor or a solenoid, and is connected to the UI controller 605. In response to a control instruction output from the UI controller 605 to the vibrator 619, the vibrator 619 operates to generate vibration in the terminal device 6. As a result, a so-called vibration-compatible game in which the vibration is transmitted to the user's hand holding the terminal device 6 can be realized.

  The UI controller 605 outputs operation data including touch position data, stick data, operation button data, azimuth data, acceleration data, and angular velocity data received from the above components to the codec LSI 606. When another device is connected to the terminal device 6 via the expansion connector 67, the operation data may further include data representing an operation on the other device.

  The codec LSI 606 is a circuit that performs compression processing on data transmitted to the game apparatus body 5 and expansion processing on data transmitted from the game apparatus body 5. Connected to the codec LSI 606 are an LCD 61, a camera 66, a sound IC 608, a wireless module 610, a flash memory 613, and an infrared communication module 612. The codec LSI 606 includes a CPU 617 and an internal memory 618. For example, although the terminal device 6 is configured not to perform the game process itself, it is necessary to execute a minimum program for management and communication of the terminal device 6. As an example, when the power is turned on, a program stored in the flash memory 613 is read into the internal memory 618 and executed by the CPU 617, whereby the terminal device 6 is activated. A part of the internal memory 618 is used as a VRAM for the LCD 61.

  The camera 66 captures an image in accordance with an instruction from the game apparatus body 5 and outputs the captured image data to the codec LSI 606. Control instructions for the camera 66 such as an image capturing instruction are output from the codec LSI 606 to the camera 66. Note that the camera 66 can also capture moving images. That is, the camera 66 can repeatedly capture images and repeatedly output image data to the codec LSI 606.

  The sound IC 608 is a circuit that is connected to the speaker 607 and the microphone 609 and controls input / output of audio data to and from the speaker 607 and the microphone 609. That is, when audio data is received from the codec LSI 606, the sound IC 608 outputs an audio signal obtained by performing D / A conversion on the audio data to the speaker 607, and outputs sound from the speaker 607. The microphone 609 detects sound (such as user's voice) transmitted to the terminal device 6 and outputs a sound signal indicating the sound to the sound IC 608. The sound IC 608 performs A / D conversion on the audio signal from the microphone 609 and outputs audio data in a predetermined format to the codec LSI 606.

  The codec LSI 606 transmits image data from the camera 66, audio data from the microphone 609, and operation data from the UI controller 605 to the game apparatus body 5 via the wireless module 610 as terminal operation data. In the present embodiment, the codec LSI 606 performs the same compression processing as the codec LSI 27 on the image data and audio data. The compressed image data and audio data and the terminal operation data are output to the wireless module 610 as transmission data. An antenna 611 is connected to the wireless module 610, and the wireless module 610 transmits the transmission data to the game apparatus body 5 via the antenna 611. The wireless module 610 has the same function as the terminal communication module 28 of the game apparatus body 5. That is, the wireless module 610 has a function of connecting to a wireless LAN by a method compliant with, for example, the IEEE 802.11n standard. Data transmitted from the wireless module 610 may or may not be encrypted as necessary.

  As described above, the transmission data transmitted from the terminal device 6 to the game apparatus body 5 includes operation data (terminal operation data), image data, and audio data. When another device is connected to the terminal device 6 via the extension connector 67, the data received from the other device may be further included in the transmission data. In addition, the infrared communication module 612 performs infrared communication with other devices in accordance with, for example, the IRDA standard. The codec LSI 606 may transmit the data received by infrared communication to the game apparatus body 5 by including the data in the transmission data as necessary.

  Further, as described above, compressed image data and audio data are transmitted from the game apparatus body 5 to the terminal device 6. These data are received by the codec LSI 606 via the antenna 611 and the wireless module 610. The codec LSI 606 decompresses the received image data and audio data. The expanded image data is output to the LCD 61, and an image corresponding to the image data is displayed on the LCD 61. The expanded audio data is output to the sound IC 608, and a sound corresponding to the audio data is output from the speaker 607.

  When the control data is included in the data received from the game apparatus body 5, the codec LSI 606 and the UI controller 605 issue a control instruction to each unit according to the control data. As described above, the control data is for each component included in the terminal device 6 (in this embodiment, the camera 66, the touch panel controller 601, the marker unit 65, the sensors 602 to 604, the vibrator 619, and the infrared communication module 612). Data representing a control instruction. In the present embodiment, as the control instruction represented by the control data, an instruction to operate each of the above components or to stop (stop) the operation can be considered. In other words, components that are not used in the game may be paused in order to reduce power consumption. In this case, the transmission data transmitted from the terminal device 6 to the game device body 5 includes data from the paused components. Do not include. In addition, since the marker unit 65 is an infrared LED, the control may simply be ON / OFF of power supply.

  As described above, the terminal device 6 includes operation means such as the touch panel 62, the analog stick 63, and the operation button 64. However, in other embodiments, instead of these operation means or together with these operation means. The configuration may include other operation means.

  Further, the terminal device 6 includes a magnetic sensor 602, an acceleration sensor 603, and a gyro sensor 604 as sensors for calculating the movement of the terminal device 6 (including changes in position and orientation, or position and orientation). In other embodiments, the configuration may include only one or two of these sensors. Moreover, in other embodiment, it may replace with these sensors or the structure provided with another sensor with these sensors may be sufficient.

  Moreover, although the terminal device 6 is a structure provided with the camera 66 and the microphone 609, in other embodiment, it does not need to be provided with the camera 66 and the microphone 609, and may be provided only with either one. Good.

  Further, the terminal device 6 is configured to include a marker unit 65 as a configuration for calculating the positional relationship between the terminal device 6 and the controller 7 (the position and / or orientation of the terminal device 6 viewed from the controller 7). In other embodiments, the marker unit 65 may not be provided. Moreover, in other embodiment, the terminal device 6 may be provided with another means as a structure for calculating the said positional relationship. For example, in another embodiment, the controller 7 may include a marker unit, and the terminal device 6 may include an imaging element. Furthermore, in this case, the marker 8 may be configured to include an imaging element instead of the infrared LED.

  Next, the configuration of the board type controller 9 will be described with reference to FIGS. FIG. 6 is a perspective view showing an example of the appearance of the board-type controller 9 shown in FIG. As shown in FIG. 6, the board-type controller 9 includes a base 9a on which the user rides (a user's foot is placed), and at least four load sensors 94a to 94d for detecting a load applied to the base 9a. . Each of the load sensors 94a to 94d is included in the base 9a (see FIG. 7), and in FIG. 6, their arrangement positions are indicated by broken lines. In the following description, the four load sensors 94a to 94d may be described as the load sensor 94 when collectively described.

  The base 9a is formed in a substantially rectangular parallelepiped and has a substantially rectangular shape when viewed from above. For example, the base 9a has a rectangular short side set to about 30 cm and a long side set to about 50 cm. The upper surface of the base 9a is formed flat, and a pair of surfaces are set for the user to put on both feet and ride. Specifically, on the upper surface of the base 9a, a surface on which the user places the left foot (a region surrounded by a double line on the left back side in FIG. 6) and a surface on which the left foot is placed (the right front side in FIG. 6). (Regions surrounded by double lines) are set. And the side surfaces of the four corners of the base 9a are formed, for example, so as to partially protrude in a columnar shape.

  In the table 9a, the four load sensors 94a to 94d are arranged at a predetermined interval. In this embodiment, the four load sensors 94a to 94d are arranged at the peripheral edge of the base 9a, specifically at the four corners. The interval between the load sensors 94a to 94d is set to an appropriate value so that the intention of the game operation according to the user's load applied to the table 9a can be detected with higher accuracy.

  FIG. 7 shows an example of an AA sectional view of the board-type controller 9 shown in FIG. 6 and an example in which a corner portion where the load sensor 94 is arranged is enlarged. In FIG. 7, the base 9 a includes a support plate 90 and legs 92 for the user to ride. The load sensors 94a to 94d are respectively provided at locations where the legs 92 are disposed. In this embodiment, since the four legs 92 are provided at the four corners, the four load sensors 94a to 94d are respectively arranged at the four corners. The leg 92 is formed in a substantially bottomed cylindrical shape by plastic molding, for example, and the load sensor 94 is disposed on a spherical component 92 a provided on the bottom surface in the leg 92. The support plate 90 is supported by the legs 92 via the load sensor 94.

  The support plate 90 includes an upper layer plate 90a that forms the upper surface and the upper side surface, a lower layer plate 90b that forms the lower surface and the lower side surface, and a middle layer plate 90c provided between the upper layer plate 90a and the lower layer plate 90b. The upper layer plate 90a and the lower layer plate 90b are formed by plastic molding, for example, and are integrated by adhesion or the like. The middle layer plate 90c is formed by press molding of one metal plate, for example. The middle layer plate 90c is fixed on the four load sensors 94a to 94d. The upper layer plate 90a has lattice-like ribs (not shown) on the lower surface thereof, and is supported by the middle layer plate 90c via the ribs. Therefore, when the user gets on the base 9a, the load is transmitted to the four legs 92 via the support plate 90 and the load sensors 94a to 94d. As indicated by arrows in FIG. 7, the reaction from the floor caused by the input load is transmitted from the leg 92 to the upper layer plate 90a via the spherical component 92a, the load sensors 94a to 94d, and the middle layer plate 90c. To do.

  The load sensor 94 is, for example, a strain gauge (strain sensor) type load cell, and is a load converter that converts an input load into an electric signal. In the load sensor 94, in response to the load input, the strain generating body 95 is deformed to cause distortion. This strain is converted into a change in electrical resistance by a strain sensor 96 attached to the strain generating body 95 and further converted into a change in voltage. Therefore, the load sensor 94 can output a voltage signal indicating the input load from the output terminal.

  The load sensor 94 may be another type of load sensor such as a tuning fork vibration type, a string vibration type, a capacitance type, a piezoelectric type, a magnetostrictive type, or a gyro type.

  Returning to FIG. 6, the board type controller 9 is further provided with a power button 9c. When the power button 9c is operated (for example, pressing the power button 9c) while the board type controller 9 is not activated, power is supplied to each circuit component (see FIG. 8) of the board type controller 9. However, the board-type controller 9 may be turned on in accordance with an instruction from the game apparatus body 5 to start supplying power to each circuit component. The board-type controller 9 may be automatically turned off when the state in which the user is not on continues for a certain time (for example, 30 seconds) or longer. Further, when the power button 9c is operated again while the board-type controller 9 is activated, the power supply may be turned off and the power supply to each circuit component may be stopped.

  FIG. 8 is a block diagram showing an example of the electrical configuration of the board-type controller 9. In FIG. 8, the flow of signals and data is indicated by solid arrows, and the supply of power is indicated by broken arrows.

  In FIG. 8, the board type controller 9 includes a microcomputer 100 for controlling the operation thereof. The microcomputer 100 includes a CPU, a ROM, a RAM, and the like (not shown), and the CPU controls the operation of the board type controller 9 according to a program stored in the ROM.

  The microcomputer 100 is connected to the power button 9c, the AD converter 102, the DC-DC converter 104, and the wireless module 106. Further, an antenna 106 a is connected to the wireless module 106. The four load sensors 94a to 94d are connected to the AD converter 102 via the amplifier 108, respectively.

  Further, the board-type controller 9 accommodates a battery 110 for supplying power to each circuit component. In another embodiment, an AC adapter may be connected to the board type controller 9 instead of the battery 110 so that commercial power is supplied to each circuit component. In this case, instead of the DC-DC converter 104, it is necessary to provide in the board type controller 9 a power supply circuit that converts alternating current into direct current and steps down and rectifies the direct current voltage. In this embodiment, power supply to the microcomputer 100 and the wireless module 106 is performed directly from the battery 110. That is, power from the battery 110 is always supplied to some components (CPU) and the wireless module 106 in the microcomputer 100, and whether the power button 9c is turned on or not is turned on from the game apparatus body 5. It is detected whether or not a command instructing is transmitted. On the other hand, power from the battery 110 is supplied to the load sensors 94 a to 94 d, the AD converter 102, and the amplifier 108 via the DC-DC converter 104. The DC-DC converter 104 converts the voltage value of the direct current from the battery 110 into a different voltage value and supplies the voltage value to the load sensors 94 a to 94 d, the AD converter 102, and the amplifier 108.

  The power supply to the load sensors 94a to 94d, the AD converter 102, and the amplifier 108 may be performed as necessary under the control of the DC-DC converter 104 by the microcomputer 100. In other words, the microcomputer 100 controls the DC-DC converter 104 to operate the load sensors 94a to 94d to detect the load, thereby controlling the load sensors 94a to 94d, the AD converter 102, and Power may be supplied to the amplifier 108.

  When power is supplied, the load sensors 94a to 94d each output a signal indicating the input load. These signals are amplified by the amplifiers 108, converted from analog signals to digital data by the AD converter 102, and input to the microcomputer 100. Identification information of the load sensors 94a to 94d is given to the detection values of the load sensors 94a to 94d so that the detection values of the load sensors 94a to 94d can be identified. In this way, the microcomputer 100 can acquire data indicating the load detection values of the four load sensors 94a to 94d at the same time.

  On the other hand, when it is determined that the load sensors 94a to 94d do not need to be operated, that is, when the load detection timing is not reached, the microcomputer 100 controls the DC-DC converter 104 to load sensors 94a to 94d and the AD converter 102. , And the power supply to the amplifier 108 is stopped. Thus, since the board type controller 9 can detect the load and the distance by operating the load sensors 94a to 94d only when necessary, the power consumption for load detection can be suppressed. .

  The case where load detection is necessary is typically when the game apparatus body 5 (FIG. 1) wants to acquire load data. For example, when the game apparatus body 5 requires load information, the game apparatus body 5 transmits an information acquisition command to the board-type controller 9. When the microcomputer 100 receives an information acquisition command from the game apparatus body 5, the microcomputer 100 controls the DC-DC converter 104 to supply power to the load sensors 94 a to 94 d and detect the load. On the other hand, when the microcomputer 100 has not received the information acquisition command from the game apparatus body 5, the microcomputer 100 controls the DC-DC converter 104 to stop the power supply to the load sensors 94 a to 94 d and the like.

  Note that the microcomputer 100 may control the DC-DC converter 104 by determining that the load detection timing comes at regular intervals. When such periodic load detection is performed, the period information may be given to the microcomputer 100 of the board-type controller 9 from the game apparatus body 5 at the start of the game, for example, or stored in the microcomputer 100 in advance. May be installed.

  Data indicating detection values from the load sensors 94a to 94d is transmitted from the microcomputer 100 to the game apparatus body 5 via the wireless module 106 and the antenna 106a as board operation data (input data) of the board type controller 9. For example, when a load is detected in response to a command from the game apparatus body 5, the microcomputer 100 receives the detected value data of the load sensors 94 a to 94 d from the AD converter 102 and receives the detected value data as a game. It transmits to the apparatus main body 5. Note that the microcomputer 100 may transmit the detection value data to the game apparatus body 5 at regular intervals. When the transmission cycle is longer than the load detection cycle, data including load values at a plurality of detection timings detected up to the transmission timing may be transmitted.

  The wireless module 106 can communicate with the same wireless standard (Bluetooth, wireless LAN, etc.) as the controller communication module 19 of the game apparatus body 5. Therefore, the CPU 10 of the game apparatus body 5 can transmit an information acquisition command to the board-type controller 9 via the controller communication module 19 or the like. Thus, the board type controller 9 can receive commands from the game apparatus body 5 via the wireless module 106 and the antenna 106a. Further, the board-type controller 9 can transmit board operation data including load detection values (or load calculation values) of the load sensors 94 a to 94 d to the game apparatus body 5.

  For example, in the case of a game that is executed using a mere total value of four load values detected by the four load sensors 94a to 94d, the user performs the four load sensors 94a to 94d of the board type controller 9. Can take any position. That is, the user can play the game by riding in any direction on any position on the table 9a. However, depending on the type of game, it is necessary to perform processing by identifying which direction the load value detected by the four load sensors 94 is viewed from the user. That is, it is necessary to grasp the positional relationship between the four load sensors 94 of the board-type controller 9 and the user. In this case, for example, it may be assumed that the positional relationship between the four load sensors 94 and the user is defined in advance, and the user gets on the table 9a so as to obtain the predetermined positional relationship. Typically, a positional relationship in which there are two load sensors 94a to 94d on each of the front, rear, left and right of the user who rides on the center of the base 9a, that is, the user gets on the center of the base 9a of the board type controller 9 A positional relationship is defined. In this case, in this embodiment, the base 9a of the board-type controller 9 is formed in a rectangular shape in plan view, and the power button 9c is provided on one side (long side) of the rectangle. As a mark, the user decides in advance that the user should get on the base 9a so that the long side provided with the power button 9c is in a predetermined direction (front, back, left or right). . In this way, the load values detected by the load sensors 94a to 94d are load values in a predetermined direction (right front, left front, right rear, and left rear) as viewed from the user. Therefore, the board-type controller 9 and the game apparatus body 5 use the identification information of the load sensor 94 included in the load detection value data and the arrangement data indicating the position or direction of the load sensor 94 set (stored) with respect to the user. Based on this, it is possible to grasp which direction each load detection value corresponds to when viewed from the user. Thereby, for example, it is possible to grasp the intention of the game operation by the user, such as the front / rear / left / right operation direction or the distinction of the raised foot.

  Next, before describing specific processing performed by the game apparatus body 5, an outline of information processing performed by the game apparatus body 5 will be described with reference to FIGS. 9 to 14. FIG. 9 is a diagram showing an example of a state of a user who operates using the terminal device 6 and the board type controller 9. FIG. 10A is a diagram illustrating an example of an image displayed on the LCD 61 of the terminal device 6. FIG. 10B is a diagram illustrating an example of an image displayed on the monitor 2. FIG. 11A is a diagram illustrating an example of an image displayed on the LCD 61 when the terminal device 6 is viewed in the normal viewing state. FIG. 11B is a diagram showing an example in which the image shown in FIG. 11A is displayed on the LCD 61 of the terminal device 6 in the normal viewing state. FIG. 12A is a diagram illustrating an example of an image displayed on the LCD 61 when the terminal device 6 is rotated counterclockwise from the normal viewing state. FIG. 12B is a diagram showing an example in which the image shown in FIG. 12A is displayed on the LCD 61 of the terminal device 6 that is rolled counterclockwise from the normal viewing state. FIG. 13A is a diagram illustrating an example of an image displayed on the LCD 61 when the terminal device 6 is viewed in a vertically held state by rotating the terminal device 6 in the counterclockwise direction from the normal viewing state. FIG. 13B is a diagram showing an example in which the image shown in FIG. 13A is displayed on the LCD 61 of the terminal device 6 in a vertically held state. FIG. 14 illustrates an example of the relationship between the terminal device depth direction projected onto the vertical plane of the real space and the operation instruction direction projected onto the vertical plane of the virtual world, and the rotation operation corresponding to the terminal device upward direction (roll angle). It is a figure for doing.

  As shown in FIG. 9, the user operates using the terminal device 6 and the board type controller 9. Specifically, the user rides on the board type controller 9 while holding the terminal device 6. Then, the user operates on the board-type controller 9 while watching the image displayed on the monitor 2 or the image displayed on the LCD 61 of the terminal device 6 (for example, a stepping action or a bending action), and the terminal device 6 itself. Play by moving the. The player object Po operates in the virtual world on the LCD 61 and the monitor 2 of the terminal device 6 in accordance with the direction and orientation of the terminal device 6 held by the user and the user operation on the board type controller 9 (for example, A game image in which the position and posture of the virtual camera set in the virtual world is changed according to the position and posture of the player object Po. .

  As shown in FIG. 10A, the player object Po moving in the virtual world is displayed on the LCD 61 of the terminal device 6. In the example shown in FIG. 10A, a virtual camera is set near the back of the player object Po swimming in the sea, and the state in the virtual world viewed from the virtual camera is displayed together with the player object Po. In this way, by displaying the virtual world viewed from behind the player object Po on the LCD 61, the user holding the terminal device 6 is given a sense of realism in the virtual world, and the user moves the player object Po in the moving direction. And intuitively know the speed of movement. Further, the orientation of the terminal device 6 is changed in the vertical direction (pitch), the terminal device 6 is rotationally moved so that the orientation of the terminal device 6 is changed (yaw), and the terminal device 6 is rotated around the depth direction (z-axis direction) of the display screen. By rotating (rolling), the posture and moving direction of the player object Po change, and the position and posture of the virtual camera also change according to the posture change. For example, by linking the attitude and orientation of the terminal device 6 with the attitude and orientation of the virtual camera, the user can feel as if he is looking into the virtual world via the LCD 61 of the terminal device 6.

  Further, as shown in FIG. 10B, the same virtual world as the virtual world displayed on the LCD 61 is also displayed on the monitor 2. In the example shown in FIG. 10B, a state in the virtual world where the player object Po swimming in the sea is viewed from a distance is displayed together with the player object Po. In the example shown in FIG. 10B, the movement locus Lp of the player object Po that has swam in the sea is displayed in the virtual world. Thus, by displaying on the monitor 2 the state in the virtual world where the player object Po is viewed from a distance, the user can easily understand the situation around the player object Po and the user is playing. It can also be enjoyed that the other person watching is moving the player object Po in the virtual world.

  As an example, when the user performs an action of stepping on the board type controller 9, the player object Po performs an action of swimming in the sea while flapping at a speed corresponding to the stepping action. Further, when the user bends and stretches on the board-type controller 9, the player object Po performs a dolphin kick and swims in the sea at a speed corresponding to the bend / stretch operation. In this way, the user can change the swimming style and the moving speed of the player object Po by the operation on the board type controller 9.

  For example, as described above, the board type controller 9 outputs a load detection value corresponding to the user operation on the board type controller 9. If the load detection value is used, it is possible to calculate the total load applied to the board type controller 9 and the position of the center of gravity of the load applied to the board type controller 9. Further, if the change in the total load or the change in the position of the center of gravity is used, it is possible to estimate what operation the user is performing on the board type controller 9. The swimming style and moving speed of the player object Po are set according to the user action estimated on the board type controller 9 as described above.

  Further, the posture and moving direction of the player object Po swimming in the sea change according to the direction and posture of the terminal device 6 held by the user. As an example, a posture in which the player object Po swims in the sea in accordance with the change in direction (direction in which the player object Po faces in the virtual world) when the user directs the terminal device 6 in the vertical and horizontal directions (that is, pitch and yaw). ) Changes in conjunction with the direction change, and the swimming direction also changes. Specifically, when the user changes the direction of the terminal device 6 itself so that the back surface of the terminal device 6 faces upward (that is, when pitched in the elevation direction), the direction in which the player object Po rises in the sea surface direction. Change the posture to swim to face the sea, and change the swim direction to rise to the sea level. Further, when the user changes the direction of the terminal device 6 itself so that the back surface of the terminal device 6 is directed to the left (that is, when yawed to the left), the posture of swimming to the left as viewed from the player object Po is changed. And change the direction of swimming to the left. In this way, the user can change the posture and moving direction of the player object Po according to the direction and posture of the terminal device 6 to be held. For example, by linking the posture and orientation of the terminal device 6 with the posture and movement direction of the player object Po, the user can perform a realistic operation as if the user became the player object Po using the terminal device 6. . Further, as described above, since the virtual camera is set near the back of the player object Po swimming in the sea, the posture and position of the virtual camera change as the posture and movement direction of the player object Po change. For example, by linking the attitude and orientation of the terminal device 6 with the attitude and orientation of the virtual camera, the user can feel as if he is looking into the virtual world via the LCD 61 of the terminal device 6.

  Further, when the terminal device 6 gripped by the user is rotated around the depth direction (z-axis direction) of the LCD 61 (that is, when the user rolls), the player object Po moves underwater according to the direction and orientation of the terminal device 6. The posture of swimming changes.

  As shown in FIG. 11A and FIG. 11B, when the terminal device 6 is viewed in a normal view, the player object Po is in a posture in which the front side of the trunk is directed downward of the virtual world (that is, the rear side of the trunk is directed above the virtual world). Controlled to swim in the sea. Then, the player object Po and the virtual world are displayed on the LCD 61 of the terminal device 6 so that the vertical direction of the virtual world in which the player object Po is arranged is displayed in the vertical direction of the LCD 61. Here, the normal viewing state of the terminal device 6 means that the user holds the terminal device 6 sideways by holding the housings 60 on both the left and right sides of the LCD 61 in a state where the marker portion 65 is disposed on the upper side of the LCD 61 (that is, And the terminal device 6 with the long side direction set to the side), and the x-axis positive direction is directed to the right side of the user along the horizontal direction in the real space. Typically, the normal viewing state of the terminal device 6 in the present embodiment is such that the x-axis direction is oriented in the horizontal direction in the real space, and the upper direction in the real space (that is, the direction within the predetermined elevation angle range centering on the vertical direction). The y-axis positive direction is directed to the z-axis, and the z-axis positive direction is directed to a direction within a predetermined angle range centered on the horizontal direction in real space.

  When the terminal device 6 in the normal viewing state is rotated counterclockwise around the depth direction (z-axis direction) of the LCD 61 (that is, when rolled in the A direction shown in FIG. 11B), FIG. 12A and FIG. An image as shown in FIG. 12B is displayed on the LCD 61. As shown in FIGS. 12A and 12B, when the terminal device 6 is rotated counterclockwise from the normal viewing state, the angle at which the terminal device 6 is rotated (roll angle; for example, the y axis rotates around the z axis) The player object Po is controlled so as to swim in the sea in a posture in which the body is rotated in the roll-rotated direction (roll direction; for example, the direction in which the y-axis rotates about the z-axis).

  As an example, as shown in FIG. 14, when the terminal device 6 rolls in the A direction by the roll angle θ (that is, the y axis rotates around the z axis by the angle θ in the A direction), the player object Po is The posture is controlled by rolling in the virtual world in the virtual direction by the roll angle θ around the moving direction (for example, the body back direction is rotated in the A direction by the angle θ around the moving direction). Then, the player object Po and the virtual world are displayed so that the vertical direction of the virtual world in which the player object Po is arranged is displayed in a direction rotated from the vertical direction of the LCD 61 by the roll angle θ in the reverse direction of the roll direction. Is displayed on the LCD 61 of the terminal device 6. 11A and 12A, the player object Po rolls in the virtual world in the A direction by the roll angle θ, but the virtual world displayed on the LCD 61 is in the A direction. Since the same roll angle θ is rotated and displayed in the opposite direction, the result is displayed on the LCD 61 in the same state, and the virtual world other than the player object Po is displayed as if rotated in the opposite direction of the A direction. The 11B and 12B, the terminal device 6 is rotated by the roll angle θ in the A direction from the normal viewing state, but there is a virtual world other than the player object Po displayed on the LCD 61. Since the same roll angle θ is rotated and displayed in the reverse direction of the A direction, the virtual world is displayed in the same direction as a result, and the player object Po is displayed with respect to the virtual world and the real space. It is displayed as if it was rotated in the A direction.

  When the image of the virtual world in which the player object Po shown in FIGS. 11A and 11B is arranged is changed to the image shown in FIGS. 12A and 12B according to the roll rotation of the terminal device 6, the player object Po It is conceivable that the virtual camera arranged in the vicinity of the back also rolls around the line-of-sight direction in the same manner as the player object Po. For example, as illustrated in FIG. 14, when the terminal device 6 rolls in the A direction by the roll angle θ, the virtual camera disposed in the vicinity of the back is centered on the visual line direction (Z-axis positive direction in the drawing) of the virtual camera. The posture may be controlled by rolling in the A direction by the roll angle θ. As a result, the virtual camera operates with the same roll rotation as the player object Po. As a result, the player object Po displayed on the LCD 61 is displayed on the LCD 61 in the same state, and a virtual world other than the player object Po. Is displayed as if it is rotated in the direction opposite to the A direction.

  When the terminal device 6 roll-rotated in the A direction is further rotated in the A-direction to be in a vertically held state (that is, rotated further in the A-direction until the y-axis becomes horizontal around the z-axis), FIG. An image as shown in FIG. 13B is displayed on the LCD 61. As shown in FIG. 13A and FIG. 13B, when the terminal device 6 is rolled in the counterclockwise direction from the normal viewing state to the vertical holding state, the body is moved in the roll direction by the roll angle that the terminal device 6 has further rotated. The player object Po is controlled to swim in the sea in a rotated posture (for example, a posture rotated and moved further in the A direction until the body back direction becomes horizontal around the moving direction). Then, the player object Po and the virtual world are displayed on the LCD 61 of the terminal device 6 so that the direction from the top to the bottom of the virtual world where the player object Po is arranged is displayed from the right to the left of the LCD 61. Is done. 11A and 13A, the player object Po rolls in the virtual world in the A direction by a roll angle of 90 °, but the virtual world displayed on the LCD 61 is A. Since the same roll angle is rotated by 90 ° in the opposite direction, the result is displayed in the same state on the LCD 61, and the virtual world other than the player object Po is rotated by 90 ° in the reverse direction of the A direction. Is displayed. 11B and 13B, the terminal device 6 is rotated from the normal viewing state to the vertical holding state by a roll angle of 90 ° in the A direction, but the player object Po displayed on the LCD 61 is displayed. Since the virtual world other than is rotated and displayed in the reverse direction of the A direction by the same roll angle of 90 °, as a result, the virtual world is displayed in the same direction with respect to the real space, and the player object Po is displayed in the virtual world. Or displayed as if rotated 90 ° in the A direction with respect to the real space.

  Thus, the user can roll-rotate the player object Po in the virtual world by rolling the terminal device 6, and the roll angle and roll direction of the terminal device 6 and the roll angle and roll direction of the player object Po can be achieved. , The user can use the terminal device 6 to perform a realistic operation as if he / she became the player object Po. Further, as described above, a virtual camera is set in the vicinity of the back of the player object Po swimming in the sea, and by rotating in the same manner as the player object Po, the user is as if using the LCD 61 of the terminal device 6 as a viewing window. You can enjoy the feeling of looking inside.

  In addition, when the attitude control corresponding to the pitch and yaw of the terminal apparatus 6 described above is also taken into account in the attitude control of the virtual camera and the player object Po by the rolling of the terminal apparatus 6 as a result, the real space is eventually obtained. The attitude of the virtual camera in the virtual world may be controlled so as to have the same attitude as the terminal device 6 in FIG. That is, the z-axis positive direction (terminal device depth direction) of the terminal device 6 in the real space is made to correspond to the Z-axis positive direction (gaze direction) of the virtual camera in the virtual world, and the y-axis positive direction ( Terminal device upward direction) corresponds to the positive Y-axis direction (virtual camera upward direction) of the virtual camera in the virtual world, and the positive x-axis direction (right direction of the terminal device) of the terminal device 6 in the real space corresponds to the virtual camera in the virtual world. The attitude of the virtual camera is controlled in correspondence with the positive X-axis direction (right direction of the virtual camera). Then, the positional relationship including the roll direction between the player object Po and the virtual camera is fixed, and the position of the virtual camera is set based on the position of the player object Po moving in the virtual world. In this way, by setting the virtual camera and the player object Po in the virtual world, images of the virtual world as shown in FIGS. 11A, 12A, and 13A, respectively, are displayed in FIGS. 11B, 12B, and 13B. Each of them can be displayed on the LCD 61 of the terminal device 6 in the posture as shown.

  For example, the terminal device 6 outputs acceleration data and angular velocity data corresponding to the posture change of the terminal device 6. And if the acceleration which the said acceleration data shows is used, since the direction of the gravitational acceleration which is acting on the terminal device 6 is computable, what kind of attitude | position is the terminal device 6 on the basis of the perpendicular direction in real space? That is, it is possible to estimate the xyz-axis direction of the terminal device 6 with respect to the vertical direction. In addition, if the angular velocity indicated by the angular velocity data and / or the acceleration indicated by the acceleration data is used, the angular velocity and / or dynamic acceleration acting on the terminal device 6 can be known. By using this, it is possible to estimate a change in posture from the initial posture of the terminal device 6 in real space (that is, a change in the direction of the xyz axis around each xyz axis). The posture and movement direction of the player object Po and the posture and position of the virtual camera are set according to the posture change (change in the xyz axis direction) of the terminal device 6 thus estimated.

  Next, details of information processing performed in the game system 1 will be described. First, main data used in the information processing will be described with reference to FIG. FIG. 15 shows main data and programs stored in the external main memory 12 and / or the internal main memory 35 of the game apparatus body 5 (hereinafter, the two main memories are collectively referred to as main memory). It is a figure which shows an example.

  As shown in FIG. 15, the data storage area of the main memory includes board operation data Da, terminal operation data Db, load value data Dc, gravity center position data Dd, terminal device direction / posture data De, operation direction data Df, and motion posture. Data Dg, movement vector data Dh, position data Di, virtual camera data Dj, movement locus data Dk, operation mode flag data Dm, image data Dn, and the like are stored. In addition to the data included in the information shown in FIG. 15, the main memory appropriately stores data necessary for game processing such as image data of various objects displayed on the monitor 2 and the LCD 61 and sound data used in the game. Remembered. Various program groups Pa constituting the information processing program are stored in the program storage area of the main memory.

  The board operation data Da stores a series of operation information (board operation data) transmitted as transmission data from the board type controller 9 and is updated to the latest board operation data. For example, the board operation data Da includes load data Da1 and the like. The load data Da1 is data indicating load detection values detected by the load sensors 94a to 94d of the board type controller 9, respectively.

  The terminal operation data Db stores a series of operation information (terminal operation data) transmitted from the terminal device 6 as transmission data, and is updated to the latest terminal operation data. For example, the terminal operation data Db includes acceleration data Db1, angular velocity data Db2, and the like. The acceleration data Db1 is data representing the acceleration (acceleration vector) detected by the acceleration sensor 603. For example, the acceleration data Db1 represents a three-dimensional acceleration having acceleration components in the xyz three-axis directions shown in FIG. 3 as components, but in another example, acceleration data in any one or more directions is expressed. It may represent. The angular velocity data Db2 is data representing the angular velocity detected by the gyro sensor 604. For example, the angular velocity data Db2 represents the angular velocities around the three axes xyz shown in FIG. 3, but in other examples, the angular velocity data Db2 may represent angular velocities around any one or more axes. .

  Note that the game apparatus body 5 includes data (for example, load detection values) included in operation information transmitted at predetermined intervals (for example, every 1/200 second) from the controller 7, the board-type controller 9, and the terminal device 6, respectively. , Data indicating acceleration and angular velocity) are sequentially received. For example, the received data is sequentially stored in the main memory by the input / output processor 31. In the processing flow described later, the CPU 10 reads the latest board operation data and terminal operation data from the main memory every frame (for example, every 1/60 seconds), and updates the board operation data Da and the terminal operation data Db, respectively. An example is used.

  Further, the operation information transmitted from the controller 7, the board type controller 9, and the terminal device 6 at each predetermined period may be temporarily stored in a buffer (not shown) provided in the controller communication module 19, the terminal communication module 28, or the like. In this case, the data stored in the buffer is read for each frame, and the board operation data Da (for example, load data Da1) and terminal operation data Db (for example, acceleration data Db1 and angular velocity data Db2) of the main memory are read out. ) Is updated and used. At this time, since the cycle for receiving the operation information and the processing cycle are different, the operation information received at a plurality of times is described in the buffer. Of the operation information received at a plurality of times, The process is executed using only the latest operation information.

  The load value data Dc is a set of data indicating the load value detected by the board type controller 9. For example, the load value data Dc is a set of data indicating the total value (total load value) of the loads detected by the load sensors 94a to 94d. Specifically, the load value data Dc is an array of data indicating the total load value within a predetermined period calculated in time series, and data indicating the total load value in each element of the array is time-series. Stored.

  The gravity center position data Dd is a set of data indicating the gravity center position of the load applied to the board type controller 9. For example, the center-of-gravity position data Dd is a set of data indicating the center-of-gravity position calculated using predetermined formulas from the load values detected by the load sensors 94a to 94d. Specifically, the centroid position data Dd is an array of data indicating the centroid position within a predetermined period calculated in time series, and data indicating the centroid position is stored in each element of the array in time series. The

  The terminal device direction / attitude data De includes real space reference direction data De1, terminal device depth direction data De2, and terminal device upward direction data De3. The real space reference direction data De1 is data indicating the reference direction (posture; real space reference direction) of the terminal device 6 in the real space. The terminal device depth direction data De2 is data indicating the current depth direction (terminal device depth direction) of the terminal device 6 in the real space. The terminal device upward data De3 is data indicating the current upward direction (terminal device upward direction) of the terminal device 6 in the real space. For example, the real space reference direction data De1, the terminal device depth direction data De2, and the terminal device upward direction data De3 are calculated based on the acceleration data Db1 and the angular velocity data Db2 included in the terminal operation data Db. The calculation method of the real space reference direction, the terminal device depth direction, and the terminal device upward direction will be described later.

  The operation direction data Df includes virtual world reference direction data Df1, operation instruction direction data Df2, operation upward direction data Df3, and the like. The virtual world reference direction data Df1 is data indicating a reference direction set in the virtual world. The operation instruction direction data Df2 is data indicating the operation instruction direction currently instructed by the user with respect to the virtual world. The operation upward direction data Df3 is data indicating the operation upward direction currently instructed by the user with respect to the virtual world. A method for calculating the virtual world reference direction, the operation instruction direction, and the operation upward direction will be described later.

  The motion posture data Dg is data indicating the motion and posture of the player object Po in the virtual world. The movement vector data Dh is data indicating the moving speed and moving direction of the player object Po in the virtual world (for example, data indicating the movement vector in the virtual world). The position data Di is data indicating the position of the player object Po in the virtual world. The virtual camera data Dj is data related to a virtual camera set in the virtual world. For example, the virtual camera data Dj includes data relating to a first virtual camera for generating a game image to be displayed on the LCD 61 of the terminal device 6 and data relating to a second virtual camera for generating a game image to be displayed on the monitor 2. including. The movement trajectory data Dk is data indicating a trajectory that the player object Po has moved in the virtual world.

  The operation mode flag data Dm is data indicating an operation mode flag that is set according to an operation (for example, swimming style) set for the player object Po. As an example, when the user bends and stretches on the board-type controller 9, the action of the player object Po is set to the action of dolphin kicking and swimming in the sea, and the action mode flag is set to on.

  The image data Dn includes player object data Dn1, background image data Dn2, and the like. The player object data Dn1 is data for arranging the player object Po in the virtual world and generating a game image. The background image data Dn2 is data for generating a game image by arranging a background in the virtual world.

  Next, details of information processing performed in the game apparatus body 5 will be described with reference to FIGS. FIG. 16 is a flowchart showing an example of information processing executed in the game apparatus body 5. FIG. 17 is a subroutine showing an example of the game control process of step 44 in FIG. FIG. 18 is a subroutine showing an example of the player object setting process in step 83 in FIG. FIG. 19 is a subroutine showing an example of the player object action setting process in step 127 in FIG. FIG. 20 is a diagram for explaining an example of the relationship between the real space reference direction and the terminal device depth direction projected onto the horizontal plane of the real space, and the virtual world reference direction and the operation instruction direction projected onto the virtual world horizontal plane. Here, in the flowcharts shown in FIG. 16 to FIG. 19, the processing in which the player object Po is operated and displayed according to the user's operation using the terminal device 6 and the board type controller 9 is mainly included in the processing. Detailed explanations of other processes not described and directly related to the present invention will be omitted. 16 to 19, each step executed by the CPU 10 is abbreviated as “S”.

  When the power of the game apparatus body 5 is turned on, the CPU 10 of the game apparatus body 5 executes a startup program stored in the ROM / RTC 13, thereby initializing each unit such as the main memory. Then, the information processing program stored in the optical disc 4 is read into the main memory, and the CPU 10 starts executing the program. The flowchart shown in FIGS. 16-19 is a flowchart which shows the process performed after the above process is completed.

  In FIG. 16, the CPU 10 executes initial processing (step 40), and proceeds to the next step. For example, in the initial process in step 40, the CPU 10 constructs a virtual world, places the player object Po and each object in the virtual world at an initial position, and sets initial values of various parameters used in the game process. .

  Next, the CPU 10 sets a reference direction based on the data transmitted from the terminal device 6 (step 41), and proceeds to the next step. Hereinafter, an example in which the CPU 10 sets the reference direction will be described.

  The terminal device 6 repeatedly transmits data as described above to the game apparatus body 5. In the game apparatus body 5, the terminal communication module 28 sequentially receives the data, and the input / output processor 31 sequentially stores the terminal operation data, camera image data, and microphone sound data in the main memory. In step 41, the CPU 10 reads the latest terminal operation data from the main memory and updates the acceleration data Db1 and the angular velocity data Db2.

  Next, the CPU 10 calculates the direction and orientation of the terminal device 6 in real space. For example, the CPU 10 calculates the current direction and posture of the terminal device 6 as a reference direction (initial posture) in the real space based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2, and the calculated terminal device The real space reference direction data De1 is updated using the data indicating the six reference directions. For example, the CPU 10 can calculate the amount of rotation (direction change amount) in the real space per unit time of the terminal device 6 using the angular velocity indicated by the angular velocity data Db2. Further, in a state where the terminal device 6 is almost stationary in a real space (static state), the acceleration applied to the terminal device 6 is a gravitational acceleration, and therefore the terminal device 6 is accelerated by the acceleration indicated by the acceleration data Db1. The direction of gravity to be applied (that is, the attitude of the terminal device 6 with respect to the vertical direction in the real space) can be calculated. Therefore, the CPU 10 can calculate the initial posture of the terminal device 6 based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2. In the following description, when step 41 is executed, the actual direction is determined from the direction in which the back surface of the terminal device 6 faces in real space (the z-axis positive direction shown in FIG. 3 and the terminal device depth direction). Set the spatial reference direction.

  The initial posture of the terminal device 6 may be calculated based on the acceleration indicated by the acceleration data Db1, may be calculated based on the direction of magnetism detected by the magnetic sensor 602, or the terminal device 6 is specified. The specific posture at the time when the predetermined operation is performed may be used as the initial posture by causing the user to perform the predetermined operation in the state of the posture. When calculating the attitude of the terminal device 6 as an absolute attitude based on a predetermined direction in real space, it is necessary to calculate the initial attitude. The timing for setting the initial posture, that is, the timing for executing the step 41 may be automatically performed at the start of the game, or the user can use the terminal device 6 to perform a predetermined operation (for example, a predetermined operation button). The operation may be performed in response to the operation of pressing 64).

  In step 41, the current direction in which the player object Po is facing in the virtual world (for example, the direction that becomes the moving direction when moving as it is) is calculated as the reference direction (initial posture) in the virtual world. The virtual world reference direction data Df1 is updated using the data indicating the reference direction of the virtual world.

  Following step 41, the process of step 42 is executed. Thereafter, a processing loop composed of a series of processes from step 42 to step 51 is repeatedly executed at a rate of once per predetermined time (one frame time).

  In step 42, the CPU 10 acquires board operation data transmitted from the board type controller 9, and proceeds to the next step. Here, the board-type controller 9 repeatedly transmits board operation data to the game apparatus body 5. Therefore, in the game apparatus body 5, the controller communication module 19 sequentially receives the board operation data, and the received board operation data is sequentially stored in the main memory by the input / output processor 31. The board operation data transmission interval of the board-type controller 9 is preferably shorter than the game processing time (one frame time), for example, 1/200 second. In step 42, the CPU 10 reads the latest board operation data from the main memory and updates the load data Da1. The board operation data includes data indicating identification information of the load sensors 94a to 94d and data indicating load detection values detected by the load sensors 94a to 94d, respectively. Is used to update the load data Da1.

  Next, the CPU 10 acquires various data transmitted from the terminal device 6 (step 43), and proceeds to the next step. Here, the terminal device 6 repeatedly transmits the data to the game apparatus body 5. Therefore, in the game apparatus body 5, the terminal communication module 28 sequentially receives the data, and the camera image data and the microphone sound data are sequentially subjected to expansion processing by the codec LSI 27. Then, the input / output processor 31 sequentially stores terminal operation data, camera image data, and microphone sound data in the main memory. In step 43, the CPU 10 reads the latest terminal operation data from the main memory and updates the acceleration data Db1 and the angular velocity data Db2.

  Next, the CPU 10 performs a game control process (step 44), and proceeds to the next step. The game control process is a process of executing a process of moving the player object Po in the virtual world in accordance with a game operation by the user and advancing the game. In the game example, the user can play various games using the terminal device 6 and the board-type controller 9. Hereinafter, the game control process in step 44 will be described with reference to FIG.

  In FIG. 17, the CPU 10 calculates the load value and the center of gravity position (step 81), and proceeds to the next step. For example, the CPU 10 calculates the total load value by adding the load detection values indicated by the load data Da1, and updates the latest data in the time series data array in the load value data Dc using the data indicating the total load value. To do. Specifically, since the load data Da1 indicates the latest load detection values detected by the load sensors 94a to 94d, the total load value is calculated by summing the load detection values. Further, the CPU 10 calculates the gravity center position using the load detection value indicated by the load data Da1, and updates the latest data in the time series data array in the gravity center position data Dd using the data indicating the gravity center position. Hereinafter, an example of a calculation method of the center of gravity position will be described.

The position of the center of gravity is the position of the center of gravity of the load applied to the base 9a of the board-type controller 9, and is determined by the load value detected by each of the load sensors 94a to 94d (see FIG. 6). For example, the position of the center of gravity is a predetermined coordinate system corresponding to the position of the board type controller 9 on the base 9a (for example, the center of the base 9a is the origin, the long side direction of the base 9a is the X1 axis direction, and the short side direction is Y1. It is represented by a coordinate value based on the axial direction (X1Y1 coordinate system). When the load value detected by the load sensor 94a is a, the load value detected by the load sensor 94b is b, the load value detected by the load sensor 94c is c, and the load value detected by the load sensor 94d is d, the center of gravity The X1-axis coordinate value (X1) and the Y1-axis coordinate value (Y1) can be calculated using the following mathematical formula.
X1 = ((a + c) − (b + d)) × m
Y1 = ((c + d) − (a + b)) × n
Here, m and n are predetermined constants.

  The total load value and the center-of-gravity position calculated in this way vary according to the user operation on the board-type controller 9 and weight shift (posture). As an example, when the user repeats stepping on the board type controller 9, the total load value fluctuates according to the stepping cycle, and the position of the center of gravity is between the position where the user puts the left foot and the position where the right foot is put. It changes to reciprocate according to. As another example, when the user repeats bending and stretching movements on the board-type controller 9, the total load value varies according to the bending and stretching period, but the change in the center of gravity position is relatively small.

  Next, the CPU 10 calculates the direction change and posture of the terminal device 6 (step 82), and proceeds to the next step. For example, the CPU 10 determines the terminal device depth direction (z-axis positive direction) and the terminal device upward direction (y-axis positive direction) of the terminal device 6 in real space based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2. And the terminal device depth direction data De2 and the terminal device upward data De3 are updated using the calculated data indicating the terminal device depth direction and the terminal device upward direction of the terminal device 6.

  Here, the CPU 10 can calculate the amount of rotation (direction change amount) in the real space per unit time of the terminal device 6 using the angular velocity indicated by the angular velocity data Db2. Further, in a state where the terminal device 6 is almost stationary in a real space (static state), the acceleration applied to the terminal device 6 is a gravitational acceleration, and therefore the terminal device 6 is accelerated by the acceleration indicated by the acceleration data Db1. The direction of gravity to be applied (that is, the attitude of the terminal device 6 with respect to the vertical direction in the real space and the xyz-axis direction with respect to the vertical direction) can be calculated. Therefore, the CPU 10 can calculate the direction change and the posture of the terminal device 6 based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2.

  In the present embodiment, the direction change and orientation of the terminal device 6 are calculated based on the data indicating the acceleration and angular velocity detected by the terminal device 6, but in other embodiments, any one of the data or You may calculate the direction change and attitude | position of the terminal device 6 using three or more data. For example, the magnetic sensor 602 provided in the terminal device 6 detects the magnetism applied to the terminal device 6, and has a predetermined orientation (that is, a predetermined direction) with respect to the terminal device 6 from the direction of geomagnetism applied to the terminal device 6. The attitude of the terminal device 6 with respect to the azimuth can be calculated. Note that the amount of rotation of the terminal device 6 can be calculated even when a magnetic field other than geomagnetism is generated in the real space where the terminal device 6 is disposed. Therefore, the CPU 10 can calculate the direction change and the posture of the terminal device 6 by using at least one of data indicating acceleration, angular velocity, and magnetism detected by the terminal device 6.

  The specific calculation method of the attitude of the terminal device 6 may be any method. For example, the acceleration data Db1 indicates the attitude of the terminal device 6 calculated based on the angular velocity indicated by the angular velocity data Db2. A method of correcting using the indicated acceleration and the direction of magnetism detected by the magnetic sensor 602 is conceivable.

  Specifically, the CPU 10 first calculates the attitude of the terminal device 6 based on the angular velocity indicated by the angular velocity data Db2. Any method may be used to calculate the attitude of the terminal device 6 from the angular velocity, but as an example, the previous attitude (direction of the xyz axis calculated last time) and the current angular speed (in the current processing loop). And the angular velocity obtained in step 42). The CPU 10 calculates a new xyz-axis direction by rotating the previous xyz-axis direction by a unit time around each axis at the current angular velocity. The previous xyz-axis direction is represented by terminal device depth direction data De2 and terminal device upward direction data De3, and the current angular velocity is represented by angular velocity data Db2. Accordingly, the CPU 10 reads the terminal device depth direction data De2, the terminal device upward direction data De3, and the angular velocity data Db2, and calculates the attitude (new xyz axis direction) of the terminal device 6. As described above, in step 41, the initial posture of the terminal device 6 is determined. Therefore, when calculating the attitude of the terminal device 6 from the angular velocity, the CPU 10 can calculate the current attitude of the terminal device 6 with reference to the initially calculated initial attitude of the terminal device 6.

  Next, CPU10 correct | amends the attitude | position (xyz-axis direction) of the terminal device 6 calculated based on angular velocity using the acceleration which acceleration data Db1 shows. Specifically, the CPU 10 calculates the attitude (xyz axis direction) of the terminal device 6 based on the acceleration indicated by the acceleration data Db1. Here, in a state where the terminal device 6 is substantially stationary, the acceleration applied to the terminal device 6 is a gravitational acceleration. Therefore, in this state, since the direction of gravity acceleration (gravity direction) can be calculated using the direction of acceleration indicated by the acceleration data Db1, the direction of the terminal device 6 with respect to the gravity direction (xyz with respect to the gravity direction). Axial direction) can be calculated.

  When the posture of the terminal device 6 based on the acceleration is calculated, the CPU 10 corrects the posture based on the angular velocity using the posture based on the acceleration. Specifically, the CPU 10 brings the attitude of the terminal device 6 calculated based on the angular velocity (in the xyz axis direction) closer to the attitude of the terminal device 6 calculated based on the acceleration (in the xyz axis direction) at a predetermined rate. Make corrections. The predetermined ratio may be a predetermined fixed value, or may be set according to an acceleration or the like indicated by the acceleration data Db1. In addition, regarding the attitude of the terminal device 6 calculated based on the acceleration, the attitude cannot be calculated for the rotation direction with the gravity direction as an axis, so the CPU 10 does not correct the rotation direction. Also good. When correcting the attitude of the terminal device 6 calculated based on the angular velocity based on the magnetic direction detected by the magnetic sensor 602, the CPU 10 determines the attitude of the terminal device 6 calculated based on the angular velocity. The terminal device 6 calculated based on the direction of magnetism may be approximated at a predetermined rate. Based on the above, the CPU 10 can accurately calculate the attitude of the terminal device 6.

  Next, the CPU 10 sets a player object Po (step 83), and proceeds to the next step. Hereinafter, the player object setting process in step 83 will be described with reference to FIG.

  In FIG. 18, the CPU 10 calculates a horizontal angle difference between the real space reference direction and the terminal device depth direction (step 120), and proceeds to the next step. Here, the horizontal angle difference is an angle difference (for example, angle C shown in FIG. 20) obtained by projecting the angle difference between the real space reference direction and the terminal device depth direction in the real space onto the horizontal plane, and the terminal device 6 3 shows an angle obtained by changing the direction of the terminal device 6 (the direction in which the back surface of the terminal device 6 is facing (the z-axis positive direction shown in FIG. 3)) from the initial posture of FIG. For example, the CPU 10 calculates the horizontal angle difference using the real space reference direction indicated by the real space reference direction data De1 and the terminal device depth direction indicated by the terminal device depth direction data De2.

  Next, the CPU 10 calculates an operation instruction direction with respect to the virtual world reference direction according to the horizontal angle difference and the attitude (vertical angle) of the terminal device 6 in the vertical direction (step 121), and proceeds to the next step. . For example, as illustrated in FIG. 20, the CPU 10 uses the virtual world reference direction indicated by the virtual world reference direction data Df1 to generate an angle generated when the virtual world reference direction and the operation instruction direction are projected onto the horizontal plane of the virtual world. The difference is the horizontal angle difference and has the same positional relationship (that is, if the terminal device depth direction is rotating counterclockwise with respect to the real space reference direction, the operation instruction direction is also relative to the virtual world reference direction. The operation instruction direction in the virtual world is calculated so that the positional relationship is rotated to the left (angle C shown in FIG. 20). Further, as shown in FIG. 14, the CPU 10 uses the terminal device depth direction indicated by the terminal device depth direction data De2, and the angular difference between the horizontal direction in the virtual world and the operation instruction direction is different from the horizontal direction in the real space. The angle difference from the terminal device depth direction and the same positional relationship (that is, when the terminal device depth direction is downward with respect to the horizontal direction of the real space, the operation instruction direction with respect to the horizontal direction of the virtual world) The operation instruction direction in the virtual world is calculated so as to be downward (angle B shown in FIG. 14). Then, the CPU 10 updates the operation instruction direction data Df2 using the calculated operation instruction direction.

  Next, the CPU 10 calculates the operation upward direction in the virtual world based on the terminal device upward direction of the terminal device 6 (step 122), and proceeds to the next step. For example, the CPU 10 determines the operation upward direction using the terminal device depth direction indicated by the terminal device depth direction data De2, the terminal device upward direction indicated by the terminal device upward direction data De3, and the operation instruction direction indicated by the operation instruction direction data Df2. calculate. Specifically, as shown in FIG. 14, the operation upward direction in the virtual world based on the operation instruction direction is the same as the terminal device upward direction in the real space based on the terminal device depth direction. Calculate the operational direction. Here, the depth direction of the terminal device and the upward direction of the terminal device are set as directions in the real space with respect to the gravity direction (vertical direction), respectively, and the operation instruction direction is also set as a direction in the virtual world. , It can also be treated as a direction based on the direction of gravity set in the virtual world. Therefore, by using the terminal device depth direction, the terminal device upward direction, and the operation instruction direction, it is possible to calculate the operation direction of the virtual world corresponding to the terminal device upward direction in the real space. Then, the CPU 10 updates the operation upward direction data Df3 using the calculated operation upward direction.

  Next, the CPU 10 determines whether or not the player object Po is moving (step 123). For example, the CPU 10 determines that the player object Po is moving when a movement vector is set in the movement vector data Dh. Then, when the player object Po is moving, the CPU 10 proceeds to the next step 124. On the other hand, when the player object Po has not moved, the CPU 10 advances the processing to the next step 130.

In step 124, the CPU 10 calculates the rotational movement angle in accordance with the horizontal angle difference calculated in step 120, and proceeds to the next step. For example, the CPU 10 sets the rotational movement angle D (angle D shown in FIG. 20) as D = C × c1.
Calculate with Here, C is a horizontal angle difference, and c1 is a positive constant smaller than 1 (for example, 0.001).

  Next, the CPU 10 changes the virtual world reference direction according to the rotational movement angle calculated in step 124 (step 125), and proceeds to the next step. For example, when the virtual world reference direction indicated by the virtual world reference direction data Df1 is projected onto the horizontal plane of the virtual world, the CPU 10 approaches the operation instruction direction so as to rotate around the gravity direction of the virtual world by the rotational movement angle. The virtual world reference direction is changed, and the virtual world reference direction data Df1 is updated using the changed virtual world reference direction.

  Next, the CPU 10 sets the orientation and orientation of the player object Po in the virtual world based on the operation instruction direction indicated by the operation instruction direction data Df2 and the operation upward direction indicated by the operation upward direction data Df3 (step 126). Proceed to the next step. As an example, the CPU 10 sets the operation instruction direction indicated by the operation instruction direction data Df2 as it is as the direction (moving direction) of the player object Po. Then, the CPU 10 rolls the player object Po so that the operation upward direction indicated by the operation upward direction data Df3 is the upward direction of the player object Po in the virtual world (in the above-described game example, the trunk rear direction shown in FIG. 14). Set posture. Then, based on the set orientation and orientation of the player object Po, the motion orientation data Dg is updated. Therefore, the orientation and posture of the player object Po in the virtual world are set based on the posture of the terminal device 6 in the real space.

  For example, when the user turns the back surface of the terminal device 6 upward (that is, the pitch is upward), the orientation of the player object Po is set to an elevation angle corresponding to the angle at which the back surface is facing the vertical direction. Is done. Further, when the user turns the back surface of the terminal device 6 downward (that is, the pitch is downward), the orientation of the player object Po is at a depression angle corresponding to the angle at which the back surface is made with respect to the vertical direction. Is set. When the user rotates the terminal device 6 around the depth direction (that is, roll), the player object Po also rolls in the virtual world according to the angle rolled in the direction in which the terminal device 6 rolls. Furthermore, the left / right orientation of the player object Po is set based on the change in the left / right direction of the terminal device 6 with respect to the real space reference direction. For example, when the terminal device 6 rotates in the yaw direction (see FIG. 3) that is the rotation direction around the y axis, the direction of the player object Po also changes according to the rotation amount. Specifically, when the terminal device 6 rotates around the y axis in the left direction when viewed from the negative y axis direction, the direction of the player object Po also changes in the left direction as viewed from the player object Po according to the amount of rotation. To do. Further, when the user maintains the terminal device 6 rotated left or right with respect to the real space reference direction, the virtual world reference direction continues to change to the left or right. As a result, the operation instruction direction continues to change to the left or right according to the change of the virtual world reference direction. Therefore, when the user maintains the state in which the user rotates the terminal device 6 left or right with respect to the real space reference direction, the orientation of the player object Po continues to change further to the left or right. That is, even if the user wants to change the direction of the player object Po in the reverse direction, for example, it is not necessary to change the direction of the terminal device 6 in the reverse direction or the like. Can be improved.

  Next, the CPU 10 performs player object motion setting processing (step 127), and proceeds to the next step. Hereinafter, the player object action setting process performed in step 127 will be described with reference to FIG.

  In FIG. 19, the CPU 10 determines whether or not the operation mode flag indicated by the operation mode flag data Dm is set to OFF (step 140). Then, when the operation mode flag is set to OFF, the CPU 10 proceeds to the next step 141. On the other hand, when the operation mode flag is set to ON, the CPU 10 advances the processing to the next step 148.

  In step 141, the CPU 10 determines whether or not the total load value is equal to or greater than a predetermined value. For example, the CPU 10 determines whether or not the latest total load value indicated by the load value data Dc is equal to or greater than the predetermined value. The predetermined value is a threshold value for determining that the user has bent and stretched on the board-type controller 9, and as an example, a predetermined ratio (for example, the weight of the user riding on the board-type controller 9) 20%) is added. Here, the weight of the user riding on the board-type controller 9 may be calculated based on the total load value (for example, an average value of the total load values obtained within a predetermined period), It may be an input value. And CPU10 advances a process to the following step 142, when a total load value is beyond a predetermined value. On the other hand, if the total load value is less than the predetermined value, the CPU 10 proceeds to the next step 144.

  In step 142, the CPU 10 sets the object motion to the motion of hitting the dolphin kick, and proceeds to the next step. For example, the CPU 10 determines that the user has bent and extended on the board type controller 9 when the total load value is equal to or greater than the predetermined value. In this case, the CPU 10 sets the motion of the player object Po to the motion of swimming by the dolphin kick swimming method, sets the motion of hitting the dolphin kick, and updates the motion posture data Dg using the set motion.

  Next, the CPU 10 sets the operation mode flag to ON (step 143), and ends the processing by the subroutine. For example, the CPU 10 turns on the operation mode flag indicated by the operation mode flag data Dm.

  On the other hand, if it is determined in step 141 that the total load value is less than the predetermined value, the CPU 10 determines whether or not the center of gravity position indicated by the center of gravity position data Dd has moved from outside the right foot region into the right foot region ( Step 144). Then, when the gravity center position moves from outside the right foot region into the right foot region, the CPU 10 proceeds to the next step 145. On the other hand, if the gravity center position has not moved from outside the right foot region into the right foot region, the CPU 10 proceeds to the next step 146.

  Here, a predetermined coordinate system corresponding to a position on the board 9a of the board-type controller 9 (for example, the center of the board 9a is the origin, the long side direction of the board 9a is the X1 axis direction, and the short side direction is the Y1 axis direction. X1Y1 coordinate system) includes a region in which the position of the center of gravity moves (right foot region) when the user raises the left foot and lowers the right foot in the stepping motion, and a case where the user raises the right foot and lowers the left foot in the stepping motion. An area (left foot area) in which the center of gravity moves is set. For example, the left foot region and the right foot region are set on the left and right (for example, the left and right with the power button 9c down) on the base 9a, and a neutral region having a predetermined width extending in the front-rear direction is set between the left foot region and the right foot region. The In the process of step 144, an affirmative determination is made when the latest center of gravity position indicated by the center of gravity position data Dd has moved from outside the right foot region into the right foot region, such as when moving from the neutral region into the right foot region.

  Note that it is conceivable that the user steps on the board type controller 9 while changing the direction. In this case, if the positions of the left foot region, the right foot region, and the neutral region are fixed, it is considered that accurate stepping determination cannot be performed. In such a case, the left foot region, the right foot region, and the neutral region in the coordinate system may be moved based on the movement of the gravity center position in the coordinate system. For example, when the movement distance of the center of gravity position in a predetermined period is equal to or longer than a predetermined length, the neutral region is moved so as to extend perpendicularly to the movement locus at the center position of the movement locus of the center of gravity position in the predetermined period. The left foot region and the right foot region are moved according to the movement of the neutral region. As described above, by moving the left foot region, the right foot region, and the neutral region, even if the user steps on the board-type controller 9, the stepping determination can be performed accurately.

  Further, in the above-described stepping determination, it is determined whether the foot that the user has stepped on the board type controller 9 is the left foot or the right foot, but such a left / right distinction may not be made. For example, when processing is performed with the “left foot” in the above description as “one foot” and the “right foot” as the “other foot”, the foot that the user steps on and the foot that the player object Po strikes with a fluttering foot Although there is a possibility that the above-described process is reversed, an operation instruction in which the timing of the butterfly motion of the player object Po is synchronized with the user's stepping motion can be performed.

  In step 145, the CPU 10 sets the object motion to the motion of hitting the right foot with the right foot, and ends the processing by the subroutine. For example, the CPU 10 determines that the user has performed a stepping action of raising the left foot and lowering the right foot on the board-type controller 9 when the position of the center of gravity moves from outside the right foot region into the right foot region. In this case, the CPU 10 sets the motion of the player object Po to a motion to swim by the butterfly swimming method, sets the motion to hit the butterfly with the right foot, and updates the motion posture data Dg using the set motion.

  In step 146, the CPU 10 determines whether or not the center of gravity position indicated by the center of gravity position data Dd has moved from outside the left foot region into the left foot region. Then, when the gravity center position moves from outside the left foot region into the left foot region, the CPU 10 proceeds to the next step 147. On the other hand, if the center of gravity position has not moved from outside the left foot region into the left foot region, the CPU 10 proceeds to the next step 151.

  In step 147, the CPU 10 sets the object motion to the motion of hitting the left foot with the left foot, and ends the processing by the subroutine. For example, the CPU 10 determines that the user has performed a stepping action of raising the right foot and lowering the left foot on the board type controller 9 when the position of the center of gravity moves from outside the left foot region into the left foot region. In this case, the CPU 10 sets the motion of the player object Po to a motion that swims by the butterfly swimming method, sets the motion to hit the butterfly with the left foot, and updates the motion posture data Dg using the set motion.

  When it is determined in step 140 that the operation mode flag is on, the CPU 10 determines whether or not a predetermined time has elapsed since the operation mode flag was turned on (step 148). And CPU10 advances a process to the following step 149, when predetermined time passes. On the other hand, if the predetermined time has not elapsed, the CPU 10 advances the processing to the next step 150. Here, the predetermined time is a time from when the dolphin kick is hit until the next dolphin kick operation input or butterfly operation input is enabled, and until the predetermined time elapses, the board-type controller 9 The user's motion determination using is not performed. As a result, it is possible to prevent erroneous determination of the user operation immediately after performing the dolphin kick.

  In step 149, the CPU 10 sets the operation mode flag to OFF, and ends the processing by the subroutine. For example, the CPU 10 changes the operation mode flag indicated by the operation mode flag data Dm to off.

  On the other hand, in step 150, the CPU 10 sets an object motion during the dolphin kick, and ends the processing by the subroutine. For example, the CPU 10 continuously sets the action of the player object Po to the action of swimming by the dolphin kick swimming method, sets the action during the dolphin kick, and updates the action posture data Dg using the set action.

  The operation mode flag is off (determined negative in step 140), the total load value is less than a predetermined value (determined negative in step 141), and the center of gravity position has not moved from outside the right foot region into the right foot region (in step 144). If the determination is negative, and the center of gravity position has not moved from outside the left foot region into the left foot region (negative determination in step 146), the CPU 10 sets the operation that is currently set to continue (step 151). Then, the processing by the subroutine is finished. For example, if the action of hitting the foot (for example, the action of hitting the foot) is set, the CPU 10 cancels the action and continues to set the swimming style of the player object Po set as the object action. The motion posture data Dg is updated using the set motion.

  Returning to FIG. 18, after the player object motion setting process in step 127, the CPU 10 sets the movement vector of the player object Po (step 128), and proceeds to the next step. For example, the CPU 10 determines the player object based on the time-series arrangement of the total load value indicated by the load value data Dc (change history of the total load value) and the action (swimming) of the player object Po set in step 127. The moving speed of Po is calculated. As an example, the CPU 10 calculates the speed at which the player object Po moves faster as the period at which the total load value changes is shorter. As another example, the CPU 10 calculates the moving speed of the player object Po faster as the maximum value or the change width of the total load value is larger. For example, the shorter the pitch that the user steps on the board-type controller 9, the faster the moving speed of the player object Po swimming with a butterfly. Also, the shorter the pitch at which the user bends and stretches on the board-type controller 9, the faster the moving speed of the player object Po swimming with the dolphin kick, and the stronger the strength to bend and stretch, the higher the player object Po swimming with the dolphin kick. Increases movement speed. And CPU10 correct | amends the calculated moving speed with the coefficient by the set swimming method, and determines the moving speed of player object Po. In addition, the moving speed of the player object Po is set to be relatively fast at the timing of hitting a foot and gradually decreased until the timing of hitting the next foot.

  Further, the CPU 10 sets the operation instruction direction in the virtual world indicated by the operation instruction direction data Df2 as the moving direction of the player object Po. Then, the CPU 10 calculates a movement vector in the virtual world using the movement speed and movement direction of the player object Po calculated as described above, and updates the movement vector data Dh using the calculated movement vector. For example, the CPU 10 sets the length of the movement vector using the calculated movement speed of the player object Po, and sets the direction of the movement vector using the operation instruction direction indicated by the operation instruction direction data Df2.

  Next, the CPU 10 calculates the position of the player object Po (step 129), and ends the processing by the subroutine. For example, the CPU 10 moves the position of the player object Po indicated by the position data Di in the virtual world based on the movement vector indicated by the movement vector data Dh, and updates the position data Di using the position after the movement.

  On the other hand, when it is determined in step 123 that the player object Po is not moving, the CPU 10 determines the player object Po in the virtual world based on the operation instruction direction and the operation upward direction data Df3 indicated by the operation instruction direction data Df2. The direction is set (step 130), and the process proceeds to the next step. Note that the processing of step 130 is the same as the processing of step 126, and thus detailed description thereof is omitted.

  Next, the CPU 10 sets the object motion in a stationary state (step 131), and ends the processing by the subroutine. For example, the CPU 10 sets the motion of the player object Po to a stationary state (for example, a state drifting without swimming in water), and updates the motion posture data Dg using the set motion.

  Returning to FIG. 17, after the player object setting process in step 83, the CPU 10 sets parameters relating to the virtual camera (step 84), and proceeds to the next step. For example, the terminal game image and the monitor game image are respectively obtained by a three-dimensional CG image obtained by arranging a virtual camera in a virtual world (virtual space) and calculating a game space viewed from the virtual camera. Generated. Specifically, the first virtual camera for generating the terminal game image is set so as to include a state in the virtual world viewed from the vicinity of the back of the player object Po arranged in the virtual world. Further, the second virtual camera for generating the monitor game image is set in the same virtual world where the first virtual camera is set, and the player object Po arranged in the virtual world is viewed from a distance. It is set to include the situation in the virtual world. The CPU 10 determines the positions of the first virtual camera and the second virtual camera in the virtual world based on the position of the player object Po indicated by the position data Di and the operation instruction direction (direction of the player object Po) according to the operation instruction direction data Df2. (Parameters relating to the first virtual camera and the second virtual camera) are set. In this way, since the terminal game image and the monitor game image become virtual world game images viewed from different viewpoints, the virtual world game viewed from different viewpoints also on the LCD 61 and the monitor 2 on which they are displayed. An image will be displayed.

  Specifically, the CPU 10 controls the posture of the first virtual camera in the virtual world so as to have the same posture as the terminal device 6 in the real space. That is, the operation instruction direction (operation instruction direction indicated by the operation instruction direction data Df2) set based on the z-axis positive direction of the terminal device 6 is made to correspond to the Z-axis positive direction (gaze direction) of the first virtual camera, The operation upward direction (operation upward direction indicated by the operation upward direction data Df3) set based on the y-axis positive direction of the device 6 is made to correspond to the Y-axis positive direction (virtual camera upward direction) of the first virtual camera. 1 Control the attitude of the virtual camera. Then, the positional relationship including the roll direction between the player object Po and the first virtual camera is fixed, and the position of the first virtual camera is set based on the position of the player object Po moving in the virtual world. In this way, by setting the first virtual camera for the player object Po in the virtual world, images of the virtual world as shown in FIGS. 11A, 12A, and 13A, respectively, are displayed in FIGS. 11B, 12B, And can be displayed on the LCD 61 of the terminal device 6 in the posture as shown in FIG. 13B. That is, the user can control the operations of the first virtual camera and the player object Po that generate the virtual world image displayed on the LCD 61 of the terminal device 6 by moving the main body of the terminal device 6. Since the posture and orientation of the terminal device 6 are linked to the posture and orientation of the first virtual camera, the user can feel as if he is looking into the virtual world using the LCD 61 of the terminal device 6 as a viewing window.

  Next, the CPU 10 sets the movement trajectory of the player object Po (step 85), and ends the processing by the subroutine. For example, the CPU 10 adds the current position of the player object Po indicated by the position data Di to the movement locus indicated by the movement locus data Dk, and updates the movement locus data Dk using the movement locus to which the current position is added.

  Returning to FIG. 16, after the game control process in step 44, the CPU 10 and the GPU 32 generate a monitor game image to be displayed on the monitor 2 (step 45), and the process proceeds to the next step. For example, the CPU 10 and the GPU 32 read out each data representing the result of the game control process in step 44 from the main memory, read out data necessary for generating a monitor game image from the VRAM 34, generate a game image, and generate The monitored game image is stored in the VRAM 34. The monitor game image may be generated by any method as long as it represents the result of the game control process in step 44. For example, the monitor game image is obtained by arranging the second virtual camera in the virtual world based on the parameters related to the second virtual camera indicated by the virtual camera data Dj, and in the virtual world based on the motion posture data Dg and the position data Di It is generated by a three-dimensional CG image obtained by arranging Po, arranging the movement locus Lp in the virtual world based on the movement locus data Dk, and calculating the virtual world viewed from the second virtual camera.

  Next, the CPU 10 and the GPU 32 generate a terminal game image to be displayed on the terminal device 6 (step 46), and proceed to the next step. For example, the CPU 10 and the GPU 32 read out each data representing the result of the game control process in step 44 from the main memory, read out data necessary for generating a terminal game image from the VRAM 34, generate a game image, and generate The monitored game image is stored in the VRAM 34. The terminal game image may be generated by any method as long as it represents the result of the game control process in step 44 as in the case of the monitor game image. Further, the terminal game image may be generated by the same method as the monitor game image or may be generated by a different method. For example, in the terminal game image, the first virtual camera is arranged in the virtual world based on the parameters related to the first virtual camera indicated by the virtual camera data Dj, and the player object is placed in the virtual world based on the motion posture data Dg and the position data Di. It is generated by a three-dimensional CG image obtained by arranging Po and calculating the virtual world viewed from the first virtual camera.

  Note that when the action of the player object Po indicated by the action posture data Dg indicates an action of hitting a foot (an action of hitting a dolphin kick or an action of hitting a butterfly), the player object Po performs an action of hitting a foot. It may be placed in a virtual world. Thus, the player object Po can be operated in synchronism with the actions (stepping action and bending action) performed by the user with the board type controller 9.

  Next, the CPU 10 generates a monitor game sound to be output to the speaker 2a of the monitor 2 (step 47), and proceeds to the next step. For example, the CPU 10 causes the DSP 33 to generate a monitor game sound to be output from the speaker 2a in accordance with the result of the game control process in step 44. As an example, in the virtual world set in accordance with the result of the game control process in step 44 above, the CPU 10 sounds, motion sounds, and sound effects of each object assumed to be heard based on the position of the second virtual camera. The DSP 33 generates a monitor game sound in which BGM or the like to be output from the monitor 2 is added.

  Next, the CPU 10 generates a terminal game sound to be output to the speaker 607 of the terminal device 6 (step 48), and proceeds to the next step. For example, the CPU 10 causes the DSP 33 to generate terminal game sound to be output from the speaker 607 in accordance with the result of the game control process in step 44. As an example, in the virtual world set according to the result of the game control process in step 44 described above, the CPU 10 sounds, motion sounds, and sound effects of each object assumed to be heard based on the position of the first virtual camera. In addition, the DSP 33 generates a terminal game sound in which BGM or the like to be output from the terminal device 6 is added. The terminal game sound may be the same as or different from the monitor game sound. Further, the terminal game sound may be a part of the sound different from the monitor game sound (for example, sound effects are different but BGM is the same). If the monitor game sound and the terminal game sound are the same, the generation process of the terminal game sound may not be executed in step 48.

  Next, the CPU 10 outputs the monitor game image and the monitor game sound to the monitor 2 (step 49), and proceeds to the next step. For example, the CPU 10 sends the monitor game image data stored in the VRAM 34 and the monitor game sound data generated by the DSP 33 to the AV-IC 15. In response to this, the AV-IC 15 outputs data indicating the monitor game image and the monitor game sound to the monitor 2 via the AV connector 16. As a result, the monitor game image is displayed on the monitor 2 and the monitor game sound is output from the speaker 2a.

  Next, the CPU 10 transmits the terminal game image and the terminal game sound to the terminal device 6 (step 50), and proceeds to the next step. For example, the CPU 10 sends the terminal game image data stored in the VRAM 34 and the terminal game sound data generated by the DSP 33 to the codec LSI 27, and the codec LSI 27 performs a predetermined compression process. Further, the terminal game image data and the terminal game sound data subjected to the compression processing are transmitted to the terminal communication module 28 by the codec LSI 27 and transmitted to the terminal device 6 by the terminal communication module 28 via the antenna 29. The The terminal game image data and the terminal game sound data transmitted from the game apparatus body 5 are received by the wireless module 610 of the terminal apparatus 6 and subjected to predetermined decompression processing by the codec LSI 606. The terminal game image data that has undergone the decompression process is output to the LCD 61, and the data sound data of the terminal game sound that has undergone the decompression process is output to the sound IC 608. As a result, the terminal game image is displayed on the LCD 61 and the terminal game sound is output from the speaker 607.

  Next, the CPU 10 determines whether or not to end the game (step 51). As a condition for ending the game, for example, a condition that a game is over or a game is cleared is satisfied, or a user performs an operation to end the game. If the game is not finished, the CPU 10 returns to step 42 and repeats the process. If the game is finished, the process according to the flowchart is finished. Thereafter, the series of processing from step 42 to step 51 is repeatedly executed until it is determined in step 51 that the game is to be ended.

  Thus, according to the above-described processing, when displaying an image of the virtual world on the terminal device 6 that the user can hold and view the screen, the terminal device regardless of the direction in which the user holds the terminal device 6. 6 can be displayed. Further, according to the above-described processing, the user moves the main body of the terminal device 6 to move (attitude) the first virtual camera and the player object Po that generate the virtual world image displayed on the LCD 61 of the terminal device 6. Can be controlled. Since the posture and orientation of the terminal device 6 are linked to the posture and orientation of the first virtual camera, the user can feel as if he is looking into the virtual world using the LCD 61 of the terminal device 6 as a viewing window.

  In the game example described above, control (position, direction, and attitude control) of the virtual camera (first virtual camera) for generating an image to be displayed on the LCD 61 based on the attitude of the terminal device 6 is performed. . By such control, an image as if looking into the virtual world via the LCD 61 can be provided to the user, and the user can feel as if he is in the virtual world. Further, the operation using the attitude of the terminal device 6 includes a horizontal swing motion (yaw) around the vertical direction (for example, around the y-axis direction) and a vertical swing motion (pitch) around the horizontal direction (for example, around the x-axis direction). ), Can be rotated in three directions like a horizontal rotation (roll) around the front and back in the horizontal direction (for example, around the z-axis direction), and is suitable for controlling a virtual camera capable of similar movement in the virtual world. By controlling the attitude of the virtual camera in the virtual world so as to be the same as the attitude of the terminal device 6 in the real space, it is possible to provide an image that looks in the direction desired by the user in the virtual world it can. In the above-described game example, the player object moves (for example, moves) in response to the user moving on the board-type controller 9. In other words, the user is given a sense of being in the virtual world by the image displayed on the LCD 61, and is further given a sense of operation as if he / she became a player character in real space. The feeling of being in a virtual world will increase.

  In the above-described game example, the posture of the player object Po displayed on the LCD 61 is controlled based on the posture of the terminal device 6. By such control, it is possible to provide the user with an operating environment as if the terminal device 6 itself became the player object Po, and the feeling that the user became the player object Po in the virtual world or the player object Po It can give the feeling of operating directly. Further, the operation using the attitude of the terminal device 6 includes a horizontal swing motion (yaw) around the vertical direction (for example, around the y-axis direction) and a vertical swing motion (pitch) around the horizontal direction (for example, around the x-axis direction). ), It is possible to perform a rotation operation in three directions like a left-right rotational movement (roll) around the front-rear horizontal direction (for example, around the z-axis direction), and control the player object Po that can perform the same movement in the virtual world. Is preferred. For example, in the case of the above game example, the horizontal swing motion (yaw) around the vertical direction (y-axis direction) along the LCD 61 of the terminal device 6 is made to correspond to the posture change (yaw) of the player object Po in the horizontal direction. The vertical movement (pitch) about the horizontal direction (x-axis direction) along the vertical direction of the player object Po corresponds to the change in the vertical posture (pitch) of the player object Po, and the horizontal rotation movement about the depth direction (z-axis direction) of the LCD 61 By associating (roll) with a rotation posture change (roll) around the front direction (movement direction) of the player object Po, an object image that changes to a posture desired by the user in the virtual world can be provided. Further, in the above-described game example, an action (for example, a movement action) of the player object Po is performed in response to the user's action on the board type controller 9. That is, the user operates one player object Po with a plurality of devices (the terminal device 6 and the board-type controller 9), and an unprecedented operation is possible. In addition, the attitude control of the player object Po based on the attitude of the terminal device 6 gives a feeling that the user has become the player object Po in the virtual world or that the user is directly operating the player object Po. Thus, the operation on the board-type controller 9 gives a sense of operation as if the player character Po in the real space is further given, so that the sense of being in the virtual world is further enhanced. In the above-described game example, the player object Po can be moved in the front direction of the player object Po. Therefore, the user can set the posture (front direction) of the player object Po according to the posture of the terminal device 6 and can also set the moving direction of the player object Po. The operation for setting the moving direction is intuitive and the movement is performed. It becomes easy to match the direction to the direction desired by the user. Further, similarly to the above-described posture of the player object Po, the moving direction of the player object Po can be set in accordance with the rotation operation of the terminal device 6 in three directions. The direction can be set as it is.

  In the above-described game example, the moving direction of the player object Po is set based on the attitude of the terminal device 6. By setting the moving direction as described above, it is possible to provide the user with an operating environment as if the terminal device 6 itself became the player object Po, and the player feels that the user has become the player object Po in the virtual world. A feeling as if the object Po is directly operated can be given. Further, the operation using the attitude of the terminal device 6 includes a horizontal swing motion (yaw) around the vertical direction (for example, around the y-axis direction) and a vertical swing motion (pitch) around the horizontal direction (for example, around the x-axis direction). ) Rotational operation in three directions is possible like a left-right rotational movement (roll) around the front-rear horizontal direction (for example, around the z-axis direction). This is suitable for controlling the player object Po that can be set. For example, in the case of the above game example, the horizontal swing motion (yaw) around the vertical direction (y-axis direction) along the LCD 61 of the terminal device 6 is made to correspond to the movement direction change (yaw) in the left-right direction of the player object Po. By causing the vertical swing motion (pitch) about the left-right direction (x-axis direction) along the LCD 61 to correspond to the change in the movement direction (pitch) of the player object Po in the vertical direction, the movement direction desired by the user in the virtual world is achieved. Moving object control can be provided. Further, in the above-described game example, the movement operation (for example, setting of the movement speed) of the player object Po is performed in response to the user moving on the board type controller 9. That is, the user operates the moving direction and moving speed of one player object Po with a plurality of devices (the terminal device 6 and the board-type controller 9), and an unprecedented operation becomes possible. In addition, the setting of the moving direction of the player object Po according to the attitude of the terminal device 6 gives a feeling that the user has become the player object Po in the virtual world or that the user is directly operating the player object Po. In addition, since the operation on the board-type controller 9 gives an operational feeling as if the player object Po in the real space is given, the feeling as if in the virtual world is further increased.

  In the above-described game example, the depth direction of the virtual world displayed on the LCD 61 of the terminal device 6 can be the moving direction of the player object Po. Therefore, the user can set the moving direction of the player object Po according to the attitude of the terminal device 6, and the virtual world with the moving direction as the depth direction is displayed on the LCD 61. Therefore, the operation for setting the moving direction is intuitive. Thus, it becomes easy to adjust the moving direction to a direction desired by the user. In addition, since the moving direction of the player object Po can be set in accordance with the rotation operation of the terminal device 6 in three directions in the same manner as the direction of the virtual camera described above, the moving direction desired by the user in the virtual world Can be set as they are.

  Further, in the above-described game example, the virtual world image including at least the player object Po is displayed on the LCD 61 of the terminal device 6, but the virtual world image of another aspect may be displayed on the LCD 61. For example, the virtual world image of the subjective viewpoint viewed from the player object Po may be displayed on the LCD 61 without displaying the player object Po. In this case, the vertical direction of the virtual world displayed from the subjective viewpoint on the LCD 61 is rolled in the opposite direction according to the roll of the terminal device 6 (that is, the same operation as the roll of the terminal device 6 in the real space is performed in the virtual world. 1 virtual camera may roll). Further, the depth direction of the virtual world displayed on the LCD 61 from the subjective viewpoint may be the same as the moving direction of the player object Po, or may be different from the moving direction of the player object Po. When the depth direction of the virtual world displayed on the LCD 61 from the subjective viewpoint is the same as the movement direction of the player object Po, the movement direction is set because the virtual world with the movement direction as the depth direction is displayed on the LCD 61. It goes without saying that the operation to perform is intuitive and it is easy to adjust the moving direction to the direction desired by the user.

  In the above-described game example, a processing example in which the control of the position and orientation of the virtual camera is immediately linked according to the operation instruction direction determined based on the orientation of the terminal device 6 is used. However, a process in which the position and orientation of the first virtual camera are controlled with a predetermined time delay according to a change in the operation instruction direction or a change in the operation upward direction may be performed. In this case, after the orientation and orientation of the player object Po change, the position and orientation of the first virtual camera change following the orientation and orientation after a predetermined time delay.

  In the above-described game example, the positional relationship including the roll direction between the player object Po and the first virtual camera is fixed, and the position of the first virtual camera is determined based on the position of the player object Po moving in the virtual world. Although set, the positional relationship between the player object Po and the first virtual camera may be changed. As an example, when a rotation operation such as a left-right rotational motion (roll) around the front-rear horizontal direction (for example, around the z-axis direction) is performed on the terminal device 6, the terminal device 6 rolls in the rolled direction. The first virtual camera is rolled around the viewing direction (Z-axis positive direction) in the virtual world by an angle. On the other hand, the player object Po moves around the moving direction in the virtual world at an angle smaller than the rolled angle in the rolled direction (in the same direction as the visual line direction when the moving direction and the visual line direction match). To roll. As a result, at least the positional relationship between the player object Po and the first virtual camera in the roll direction changes. Thus, even when the roll angle at which the player object Po changes its posture in the roll direction is smaller than the roll angle of the terminal device 6, the player object Po responds to the roll operation of the user terminal device 6. Since it is the same that the roll operation is, the same operation feeling can be tasted.

  As another example, when a rotation operation such as a left-right rotational motion (roll) around the front-rear horizontal direction (for example, around the z-axis direction) is performed on the terminal device 6, the terminal device 6 rolls in the direction in which the terminal device 6 rolls. The first virtual camera is rolled around the viewing direction (Z-axis positive direction) in the virtual world by the rolled angle. On the other hand, the player object Po moves around the moving direction in the virtual world by the same angle as the rolled angle in the rolled direction (in the same direction as the visual line direction when the moving direction and the visual line direction match). And then the posture of the player object Po is changed at a predetermined roll rotation speed so that the direction of the back of the body is directed upward in the virtual world. As a result, at least the positional relationship between the player object Po and the first virtual camera in the roll direction changes after the roll rotation movement. Thus, even when the control is performed such that the posture of the player object Po in the roll direction returns to the direction based on the vertical direction of the virtual world, the player object Po responds to the roll operation of the user terminal device 6. Since the player object Po can be once rolled, the same operation feeling can be experienced when the terminal device 6 is rolled.

  In the above description, the attitude of the first virtual camera in the virtual world is controlled so as to be the same attitude as the attitude of the terminal device 6 in the real space. Thereby, according to the user pointing the terminal device 6 in a direction that the user wants to see, for example, an image as if looking into the virtual world through the terminal device 6 can be provided to the user, and the virtual world can be provided to the user. However, when such an effect is not expected, the attitude of the terminal device 6 in the real space may not completely match the attitude of the first virtual camera in the virtual world. Absent. For example, when the terminal device 6 rotates about a predetermined axis (for example, the z-axis) with the rotation amount θt, the first virtual camera rotates about the direction of the virtual world (for example, the line-of-sight direction) corresponding to the predetermined axis. It may be rotated with a smaller amount of rotation or a larger amount of rotation. Further, the first virtual camera is further rotated at a predetermined rotation speed so that the first virtual camera rotates at a rotation amount smaller than or greater than the rotation amount θt and then has the same posture as that of the terminal device 6 in the real space. You can rotate it.

  In the information processing example described above, in order to set the movement of the terminal device 6 around the z axis and the direction in which the first virtual camera and the player object Po are rolled, the terminal device upward direction (y-axis positive direction) Although the direction is used, once the orientation of the roll direction in the virtual world of the first virtual camera and the player object Po corresponding to the orientation of the terminal device 6 is set, the subsequent processing does not need to use these directions. It doesn't matter. For example, after the posture of the first virtual camera and the player object Po corresponding to the posture of the terminal device 6 in the virtual world is set, the rotation angle corresponding to the angular velocity generated around the z-axis of the terminal device 6 is set. If the first virtual camera and the player object Po are directly associated with the posture in the roll direction in the virtual world, the same posture control is possible.

  In the above description, the virtual world is fixed and the virtual camera and the player object Po are moved with respect to the virtual world. However, it is only necessary that the virtual camera and the player object Po move relative to the virtual world. Needless to say. For example, when the direction, position, posture change or the like of the virtual camera in the virtual world changes according to the direction, movement, posture change, etc. of the terminal device 6, the direction, position, posture of the virtual camera is fixed and the terminal device 6 is fixed. The entire virtual world may be moved based on the virtual camera in accordance with the direction, movement, and posture change. Further, when the direction, position, posture, etc. of the player object Po in the virtual world changes according to the direction, movement, posture change, etc. of the terminal device 6, the direction, position, posture of the player object Po is fixed and the terminal device is fixed. The entire virtual world other than the player object Po may be moved based on the player object Po in accordance with the direction, movement, and posture change of 6.

  In the above description, in order to set the movement of the terminal device 6 around the z axis and the direction in which the first virtual camera and the player object Po are rolled, the terminal device upward direction (y-axis positive direction) and the operation upward direction are set. Although used, other axes orthogonal to the z-axis, the line-of-sight direction of the first virtual camera, and other directions orthogonal to the moving direction (operation instruction direction) of the player object Po may be used. For example, the movement around the z-axis of the terminal device 6 and the direction in which the first virtual camera and the player object Po are rolled may be set using the terminal device left direction (x-axis positive direction) or the operation left direction.

  In the above-described game example, the game image displayed on the LCD 61 of the terminal device 6 and the game image displayed on the monitor 2 are both images showing the same virtual world (virtual space). It is an image from which the viewpoint and the range which see the said virtual world (virtual space) differ. Therefore, the user can see the virtual world (virtual space) that has different views and display ranges displayed on the two display screens, and can view a suitable game image as appropriate according to the game situation and the like. It becomes. In the above-described game example, the user holds the terminal device 6 and operates it to change the position and posture of the player object Po and the virtual camera according to the posture and position of the terminal device 6 in the real space, and the LCD 61. The image displayed on the screen can also be changed according to the position and posture of the player object Po and the virtual camera. Therefore, it is possible to give the user a sense of reality in the virtual world (virtual space) to the user who views the image displayed on the LCD 61 while holding the terminal device 6. On the other hand, if only the image displayed on the LCD 61 is viewed, it may be difficult to grasp the position of the entire virtual world (virtual space) and the situation around the player object Po. By displaying a wide range of virtual world (virtual space), such a problem can be solved.

  Further, in the above-described game example, the user can perform an operation by changing the load applied to the board-type controller 9 and can perform an operation (main body posture) by holding the terminal device 6 on which a game image including the player object Po is displayed. And operation by position, touch operation, button operation, etc.) are also possible. Then, the player object Po displayed on the terminal device 6 operates in the virtual world according to an operation and / or an operation using the terminal device 6 due to a load change applied to the board type controller 9. Therefore, giving the user a feeling that the user himself / herself operates as the player object Po and sees the virtual world, or gives the user a feeling that the player object Po is operated in the real space. Is possible.

  Here, in the operation using the board-type controller 9 described above, the presence / absence of a load applied to the board-type controller 9, a change in load, and the position of the center of gravity of the load are used for various motion controls of the player object Po. . For example, in the above-described game example, the action, moving speed, moving direction, and the like of the player object are changed depending on the load change applied to the board-type controller 9 and the position of the center of gravity of the load. In the above game example, the direction in the virtual world (for example, the moving direction of the player object Po, the front direction, etc.) is set according to the operation direction of the user, but the direction is the attitude of the terminal device 6. It is set based on (direction). This is because when the operation based on the attitude of the terminal device 6 and the operation using the board-type controller 9 are compared, the operation based on the attitude of the terminal device 6 is easier to set the direction, and the presence of the game This is an example in which the direction that increases is selected, and an appropriate one of the attitude of the terminal device 6 and the center of gravity position of the load applied to the board-type controller 9 may be selected as an operation for giving a direction instruction according to the game content. . As described above, when the terminal device 6 and the board-type controller 9 are used as operation means, when setting the direction in the virtual world according to the operation of the user, it is possible to select a suitable method using a plurality of operations as options. Become.

  Note that the game system 1 can play various games as described above using the terminal device 6 and the board-type controller 9 as operation means. While the terminal device 6 can be used as a portable display or a second display, the terminal device 6 can also be used as a controller that performs input by operation of the main body, touch operation, button operation, etc. According to the system 1, it becomes possible to implement a wide range of games. That is, since the terminal device 6 functions as a display device, a game in which the terminal device 6 is used as a display unit without using the monitor 2 or the controller 7 and the board type controller 9 is used as an operation unit. There can also be a system-like form. Further, since the terminal device 6 functions as a display device and also as an operation device, the terminal device 6 is used as a display means without using the monitor 2 and the controller 7, and the terminal device 6 and the board are used. There may be a form such as a game system using the mold controller 9 as an operation means. Furthermore, since the terminal device 6 functions as a display device and also as an operation device, the terminal device 6 is used as a display means without using the monitor 2, the controller 7, and the board type controller 9. There may be a game system that uses the terminal device 6 as an operation means.

  Moreover, in the said embodiment, the terminal device 6 functions as what is called a thin client terminal which does not perform a game process. However, in the terminal device 6, at least a part of the series of game processes executed by the game apparatus body 5 in the above embodiment may be executed by the terminal device 6. As an example, the terminal device 6 may execute a part of processing (for example, processing for generating a terminal game image). As another example, the terminal device 6 may execute all the series of game processes executed by the game apparatus body 5. In this case, since the terminal device 6 functions as a display device and also functions as a processing device that executes game processing, the terminal device 6 does not use the monitor 2, the game device body 5, and the controller 7. Can be used as a display means, a board type controller 9 is used as an operation means, and a terminal system 6 is used as a processing means. In the game system, only the terminal device 6 and the board type controller 9 are connected wirelessly or by wire, and various kinds of games can be performed by outputting board operation data from the board type controller 9 to the terminal device 6. Needless to say, when the board-type controller 9 is not used, the terminal device 6 may be used as a display unit, an operation unit, and a processing unit.

  In the above embodiment, posture data (for example, magnetic sensor 602, acceleration) used to calculate the posture and / or movement of the terminal device 6 (including the position and posture itself, or changes in the position and posture). At least one data output from the sensor 603 and the gyro sensor 604) is output from the terminal device 6 to the game apparatus body 5, and the attitude and / or movement of the terminal apparatus 6 is calculated by information processing in the game apparatus body 5. Yes. However, the attitude and / or movement of the terminal device 6 calculated in the game apparatus main body 5 may be calculated in the terminal device 6. In this case, data indicating the posture and / or movement of the terminal device 6 calculated by the terminal device 6 (that is, the position and posture of the terminal device 6 itself calculated using the posture data, or the position and posture of the terminal device 6). Data indicating a change) is output from the terminal device 6 to the game apparatus main body 5, and the data is used in information processing in the game apparatus main body 5.

  Further, in the above description, a mode in which the terminal device 6 and the game device body 5 and the board type controller 9 and the game device body 5 are connected by wireless communication is used. Wireless communication between them may be performed. As a first example, the terminal device 6 functions as a relay device for other wireless communication. In this case, the board operation data of the board type controller 9 is wirelessly transmitted to the terminal device 6, and the terminal device 6 wirelessly transmits the terminal operation data of the terminal device 6 to the game apparatus body 5 together with the received board operation data. In this case, the terminal device 6 and the game apparatus body 5 are directly connected by wireless communication, but the board-type controller 9 is connected to the game apparatus body 5 by wireless communication via the terminal device 6. . As a second example, the board type controller 9 functions as another wireless communication relay device. In this case, the terminal operation data of the terminal device 6 is wirelessly transmitted to the board type controller 9, and the board type controller 9 wirelessly transmits the board operation data of the board type controller 9 to the game apparatus body 5 together with the received terminal operation data. In this case, the board type controller 9 and the game apparatus body 5 are directly connected by wireless communication, but the terminal device 6 is connected to the game apparatus body 5 by wireless communication via the board type controller 9. Become. When another device relays and transmits operation data to the game apparatus main body 5, an electrical connection is established between the device that generates the operation data and another device that relays the operation data via a cable. It doesn't matter.

  Further, the terminal device 6 and / or the board-type controller 9 and the game apparatus body 5 may be electrically connected via a cable. In this case, the cable connected to the terminal device 6 and / or the board type controller 9 is connected to the connection terminal of the game apparatus body 5. As a first example, the terminal device 6 and the game apparatus body 5 are electrically connected via a first cable, and the board-type controller 9 and the game apparatus body 5 are electrically connected via a second cable. Connected. As a second example, the terminal device 6 and the game apparatus body 5 are electrically connected via a cable. In this case, the board operation data of the board type controller 9 may be transmitted wirelessly to the terminal device 6 and then transmitted to the game apparatus body 5 via the cable, or directly from the board type controller 9 to the game apparatus body 5. It may be transmitted wirelessly. As a third example, the board type controller 9 and the game apparatus body 5 are electrically connected via a cable. In this case, the terminal operation data of the terminal device 6 may be transmitted wirelessly to the board type controller 9 and then transmitted to the game apparatus body 5 via the cable, or directly from the terminal apparatus 6 to the game apparatus body 5. May be sent.

  In the above embodiment, the game system 1 has one terminal device 6 and one board-type controller 9, but the game system 1 has a plurality of sets of terminal devices 6 and a board-type controller 9. It may be. That is, the game apparatus main body 5 can wirelessly communicate with a plurality of sets of terminal devices 6 and board type controllers 9, and transmits game image data, game sound data, and control data to each terminal device 6. Operation data, camera image data, microphone sound data, and board operation data may be received from each terminal device 6 and the board type controller 9. The game apparatus body 5 performs wireless communication with each of the plurality of terminal apparatuses 6 and the board type controller 9. At this time, the game apparatus body 5 performs wireless communication with each terminal apparatus 6 and the board type controller 9. It may be performed by time division or may be performed by dividing the frequency band.

  As described above, when a plurality of sets of terminal devices 6 and the board-type controller 9 are provided, more types of games can be played using the game system 1. For example, when the game system 1 has two sets of terminal devices 6 and a board type controller 9, two users can play a game simultaneously. Further, when the game system 1 has two sets of terminal devices 6 and a board-type controller 9, the game system 1 has three display devices, so that game images for each of the three users can be displayed. It can be generated and displayed on each display device.

  In the above description, the board-type controller 9 is provided with a plurality of load sensors 94. However, if information on the center of gravity of the load applied to the board-type controller 9 in the above processing is unnecessary, the load sensor 94 is provided. At least one board-type controller 9 may be provided.

  In the above embodiment, the stationary game apparatus 3 has been described. However, the information processing program of the present invention is executed by an information processing apparatus such as a portable game apparatus or a general personal computer. May be realized. In another embodiment, the present invention is not limited to a game device, and may be any portable electronic device such as a PDA (Personal Digital Assistant), a mobile phone, a personal computer, a camera, or the like. In any device, the present invention can be realized by connecting the device, the terminal device 6 and the board-type controller 9 wirelessly or by wire.

  In the above description, the example in which the information processing is performed by the game apparatus body 5 is used. However, at least a part of the processing steps in the information processing may be performed by another apparatus provided outside the game system 1. . For example, when the game apparatus body 5 is configured to be communicable with another apparatus (for example, a server or another game apparatus), the game apparatus body 5 and the other apparatus cooperate in the processing step in the information processing. It may be executed by doing. As an example, processing for setting a player object, a virtual world, and the like is performed in another device, and data related to the motion and posture of the player object is transmitted from the game device body 5 to the other device, and the above information processing is performed. It is possible. Then, image data indicating the virtual world generated by another device is transmitted to the game apparatus body 5 and the virtual world is displayed on the monitor 2 or the LCD 61. In this way, by performing at least a part of the processing steps in the information processing with another device, processing similar to the information processing described above can be performed. Note that at least a part of the processing steps in the information processing may be performed by the board-type controller 9 (microcomputer 100). Further, the above-described information processing can be executed by cooperation between one processor or a plurality of processors included in an information processing system including at least one information processing apparatus. In the above embodiment, the CPU 10 of the game apparatus body 5 executes the predetermined program, so that the process according to the above-described flowchart is performed. All may be done.

  Further, the shape of the game apparatus main body 5 described above, the shape of the terminal device 6, the controller 7, and the board-type controller 9, and the shapes, numbers, and installation positions of various operation buttons and sensors are merely examples. It goes without saying that the present invention can be realized even in the shape, number, and installation position. In addition, the processing order, setting value, display mode, value used for determination, and the like used in the information processing described above are merely examples, and the present invention can be realized even in other orders, display modes, and values. Needless to say.

  The information processing program (game program) may be supplied not only to the game apparatus body 5 through an external storage medium such as the optical disc 4 but also to the game apparatus body 5 through a wired or wireless communication line. Further, the information processing program may be recorded in advance in a nonvolatile storage device inside the game apparatus body 5. The information storage medium for storing the information processing program may be a CD-ROM, DVD, or an optical disk storage medium similar to them, a flexible disk, a hard disk, a magneto-optical disk, a magnetic tape, or the like. The information storage medium for storing the information processing program may be a nonvolatile semiconductor memory or a volatile memory. Such a storage medium can be referred to as a computer-readable recording medium. For example, the various functions described above can be provided by causing a computer or the like to read and execute the programs of these recording media.

  Although the present invention has been described in detail above, the above description is merely illustrative of the present invention in all respects and is not intended to limit the scope thereof. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. It is understood that the scope of the present invention should be construed only by the claims. Moreover, it is understood that those skilled in the art can implement an equivalent range from the description of the specific embodiments of the present invention based on the description of the present invention and the common general technical knowledge. In addition, it is to be understood that the terms used in the present specification are used in the meaning normally used in the art unless otherwise specified. Thus, unless defined otherwise, all technical and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In case of conflict, the present specification, including definitions, will control.

  An information processing program, an information processing device, an information processing system, and an information processing method according to the present invention allow a user to display an image of a virtual world on a display device that can be held and viewed by a user. An image suitable for the device can be displayed regardless of the holding direction, and is useful as an information processing program, an information processing device, an information processing system, and an information processing method for performing processing for displaying an image of a virtual world. .

DESCRIPTION OF SYMBOLS 1 ... Game system 2 ... Monitor 3 ... Game device 4 ... Optical disk 5 ... Game device main body 10,617 ... CPU
11 ... System LSI
12 ... External main memory 13 ... ROM / RTC
14 ... Disk drive 15 ... AV-IC
16 ... AV connector 17 ... Flash memory 18 ... Network communication module 19 ... Controller communication module 20 ... Expansion connector 21 ... Memory card connectors 22, 23, 29, 611, 754 ... Antenna 24 ... Power button 25 ... Reset button 26 ... Eject Button 27 ... Codec LSI
28 ... Terminal communication module 31 ... Input / output processor 32 ... GPU
33 ... DSP
34 ... VRAM
35 ... Internal main memory 6 ... Terminal device 60 ... Housing 61 ... LCD
62 ... touch panel 63 ... analog stick 64 ... operation button 65 ... marker part 66 ... camera 67 ... expansion connector 68 ... foot part 601 ... touch panel controller 602 ... magnetic sensor 603 ... acceleration sensor 604 ... gyro sensor 605 ... UI controller 606 ... codec LSI
607 ... Speaker 608 ... Sound IC
609 ... Microphone 612 ... Infrared communication module 613 ... Flash memory 614 ... Power supply IC
616: Charger 618 ... Internal memory 619 ... Vibrator 7 ... Controller 8 ... Marker 9 ... Board type controller 90 ... Support plate 92 ... Leg 94 ... Load sensor 95 ... Strain body 96 ... Strain sensor 102 ... AD converter 104 ... DC- DC converter 106, 610, 753 ... wireless module 108 ... amplifier 100, 751 ... microcomputer 110, 615 ... battery

Claims (20)

An information processing program executed by a computer of an information processing device capable of displaying an image on the portable display device that outputs at least data according to the attitude and / or movement of the portable display device body,
The computer,
Rotation information calculation means for calculating the rotation direction of the portable display device around a predetermined direction in real space based on data output from the portable display device;
Object control means for controlling the posture of the object by rotating the object arranged in the virtual world in the rotation direction around a direction corresponding to the predetermined direction set in the virtual world;
A first virtual camera for generating an image of the virtual world including at least the object is arranged at a position away from the object, and the direction corresponding to the predetermined direction set in the virtual world is the center. First virtual camera control means for controlling the posture of the first virtual camera by rotating in a rotation direction;
An information processing program for causing a first image indicating a virtual world viewed from the first virtual camera to function as display control means for displaying on the portable display device. The rotation information calculation means further calculates a rotation amount of the portable display device around the predetermined direction in real space based on data output from the portable display device,
The object control means rotates the object in the rotation direction by the rotation amount based on the rotation amount of the portable display device around the direction corresponding to the predetermined direction set in the virtual world. Control the posture of the
The first virtual camera control means rotates the first virtual camera in the rotation direction based on the rotation amount of the portable display device around a direction corresponding to the predetermined direction set in the virtual world. The information processing program according to claim 1, wherein the information processing program controls the posture of the first virtual camera by rotating by an amount.   The first virtual camera control means controls the posture of the first virtual camera by rotating the first virtual camera in the rotation direction by the same rotation amount as the rotation amount of the portable display device. Information processing program described in 1. The information processing program further causes the computer to function as a gravity direction calculating unit that calculates a gravity direction with respect to the portable display device based on data output from the portable display device.
The rotation information calculation means calculates a rotation direction and a rotation amount of the portable display device with the gravity direction as a reference,
When the first image is displayed on the portable display device, the first virtual camera control means displays the gravity direction of the portable display device displaying the first image as the first image. The information processing program according to claim 3, wherein the posture of the first virtual camera is controlled so that the gravity direction of the virtual world being matched matches.   The object control means rotates the object in the rotation direction by the same rotation amount as the rotation amount of the portable display device, so that the object is in a posture and position in which the positional relationship with the first virtual camera is fixed. The information processing program according to claim 3 or 4 which controls.   The rotation information calculation means calculates a rotation direction and a rotation amount of the portable display device around the predetermined axis, with a predetermined axis provided in the portable display device as the predetermined direction. The information processing program as described in any one. The rotation information calculation means calculates a rotation direction and an amount of rotation of the portable display device around the depth direction with the depth direction of the display surface displaying the first image on the portable display device as the predetermined axis. ,
The first virtual camera control means sets the line-of-sight direction of the first virtual camera as a direction corresponding to the depth direction, and the same rotation amount as the rotation amount of the portable display device in the rotation direction around the line-of-sight direction The information processing program according to claim 6, wherein the first virtual camera is rotated. The rotation information calculation means is further portable with respect to the vertical direction and the horizontal direction, respectively, with the vertical and horizontal directions of the portable display device orthogonal to the depth direction and orthogonal to each other as the predetermined axis. Further calculate the direction and amount of rotation of the display device,
The first virtual camera control means has the same rotation direction and rotation amount as the rotation direction and rotation amount around the vertical direction of the portable display device, with the vertical direction orthogonal to the viewing direction of the first virtual camera as the center. The first virtual camera is rotated, and the rotation direction and rotation are the same as the rotation direction and the rotation amount around the horizontal direction of the portable display device around the line-of-sight direction of the first virtual camera and the horizontal direction orthogonal to the vertical direction. The information processing program according to claim 7, wherein the first virtual camera is rotated by an amount.   The said 1st virtual camera control means controls the attitude | position of the said 1st virtual camera in a virtual world so that it may become the attitude | position same as the attitude | position of the said portable display apparatus in real space. Information processing program described in 1. The display control means outputs image data indicating the first image to the portable display device,
The portable display device is:
Image data acquisition means for acquiring image data output from the information processing apparatus;
The information processing program according to claim 1, further comprising: a display unit that displays the first image indicated by the image data acquired by the image data acquisition unit. And further causing the computer to function as compressed image generation means for generating compressed image data by compressing image data representing the first image,
The display control means outputs the compressed image data generated by the compressed image generation means to the portable display device,
The image data acquisition means acquires the compressed image data output from the information processing apparatus;
The portable display device further includes display image expansion means for expanding the compressed image data to obtain image data indicating the first image,
The display means, the image data acquisition means acquires the display image Shin Zhang means displays the first image represented by the Shin Zhang image data storage medium according to claim 10.   The display control means further displays, separately from the first image, a second image showing the virtual world viewed from a second virtual camera on another display device connected to the information processing device. The information processing program according to any one of 1 to 9. And further causing the computer to function as compressed image generation means for generating compressed image data by compressing image data representing the first image,
The display control means outputs the compressed image data generated by the compressed image generation means to the portable display device, and separately from the compressed image data, the separate image data indicating the second image is not compressed. Output to the display device
The portable display device is:
Image data acquisition means for acquiring compressed image data output from the information processing apparatus;
Display image decompression means for decompressing the compressed image data to obtain image data representing the first image;
And display means for the image data acquisition means acquires the display image Shin Zhang means displays the first image represented by the Shin Zhang image data storage medium according to claim 12.   Based on the position of the object in the virtual world, the second virtual camera for generating an image of the virtual world is at a position different from the first virtual camera, and the object is included in the second image 14. The information processing program according to claim 12 or 13, further causing the computer to function as second virtual camera control means for setting so as to be performed. The second virtual camera control means sets the second virtual camera at a position farther from the object than the distance from the object to the first virtual camera,
The information processing program according to claim 14, wherein the display control means displays a range wider than the range of the virtual world indicated by the first image as the second image on the other display device. The second virtual camera control means sets the second virtual camera at a position where the object is bird's-eye view in the virtual world,
The information processing program according to claim 14 or 15, wherein the display control unit displays an image obtained by bird's-eye view of the object arranged in the virtual world as the second image on the another display device. The portable display device includes at least one of a gyro sensor and an acceleration sensor that outputs data according to the attitude and / or movement of the portable display device body,
The rotation information calculation unit calculates a rotation direction and a rotation amount of the portable display device based on data output from at least one of the gyro sensor and the acceleration sensor. Information processing program described in 1. An information processing apparatus capable of displaying an image on the portable display device that outputs at least data according to the attitude and / or movement of the portable display device body,
Rotation information calculation means for calculating the rotation direction of the portable display device around a predetermined direction in real space based on data output from the portable display device;
Object control means for controlling the posture of the object by rotating the object arranged in the virtual world in the rotation direction around a direction corresponding to the predetermined direction set in the virtual world;
A first virtual camera for generating an image of the virtual world including at least the object is arranged at a position away from the object, and the direction corresponding to the predetermined direction set in the virtual world is the center. First virtual camera control means for controlling the posture of the first virtual camera by rotating in a rotation direction;
An information processing apparatus comprising: a display control unit configured to display a first image indicating a virtual world viewed from the first virtual camera on the portable display device. An information processing system configured to be capable of communicating with a plurality of devices and displaying an image on the portable display device that outputs at least data according to the attitude and / or movement of the portable display device body,
Rotation information calculation means for calculating the rotation direction of the portable display device around a predetermined direction in real space based on data output from the portable display device;
Object control means for controlling the posture of the object by rotating the object arranged in the virtual world in the rotation direction around a direction corresponding to the predetermined direction set in the virtual world;
A first virtual camera for generating an image of the virtual world including at least the object is arranged at a position away from the object, and the direction corresponding to the predetermined direction set in the virtual world is the center. First virtual camera control means for controlling the posture of the first virtual camera by rotating in a rotation direction;
An information processing system comprising: display control means for displaying a first image showing a virtual world viewed from the first virtual camera on the portable display device. The portable display device that outputs at least data according to the attitude and / or movement of the portable display device main body is included in an information processing system including at least one information processing device capable of displaying an image. An information processing method executed by cooperation between one processor or a plurality of processors,
A rotation information calculation step for calculating a rotation direction of the portable display device around a predetermined direction in real space based on data output from the portable display device;
An object control step for controlling the posture of the object by rotating the object arranged in the virtual world in the rotation direction around a direction corresponding to the predetermined direction set in the virtual world;
A first virtual camera for generating an image of the virtual world including at least the object is arranged at a position away from the object, and the direction corresponding to the predetermined direction set in the virtual world is the center. A first virtual camera control step of controlling the posture of the first virtual camera by rotating in the rotation direction;
A display control step of displaying a first image showing a virtual world viewed from the first virtual camera on the portable display device.

Images