JP5872136B2 - Posture calculation device, posture calculation program, game device, game program, posture calculation system, and posture calculation method - Google Patents

Description

  The present invention relates to an attitude calculation device or an attitude calculation program, and more specifically to an attitude calculation device or an attitude calculation program for calculating the attitude of an input device.

  Conventionally, techniques for calculating the attitude of an input device using an acceleration sensor and a gyro sensor have been considered. For example, Patent Document 1 describes a game device that uses an input control device including an acceleration sensor and a gyro sensor. This game device controls a sword possessed by a game character in accordance with the movement of an input control device. Specifically, data on the motion of swinging the sword is created based on the output of the acceleration sensor, and data on the posture of the sword is created based on the output of the gyro sensor.

JP 2000-308756 A

  When the attitude is calculated using a gyro sensor as in Patent Document 1, an error occurs between the calculated attitude and the actual attitude of the input control device. When the input control device moves slowly, the input controller's angular velocity is not detected by the gyro sensor. Conversely, when the input controller moves excessively, the input controller's angular velocity exceeds the detectable range of the gyro sensor. This is because there is a case where it may occur. An error may also occur when the angular velocity changes suddenly during a period shorter than the output interval of the angular velocity data. Then, since the angular velocity error is accumulated with time in the posture calculated from the angular velocity, the posture error becomes large. In Patent Document 1, since the posture error calculated from the gyro sensor is not assumed, the posture may not be accurately calculated.

  Therefore, an object of the present invention is to provide an attitude calculation device or an attitude calculation program that can accurately calculate the attitude of an input device using an angular velocity sensor.

  The present invention employs the following configuration in order to solve the above problems. Note that the reference numerals in parentheses and supplementary explanations in this section show the correspondence with the embodiments described later in order to help understanding of the present invention, and do not limit the present invention.

The present invention obtains data from an input device (8) having at least an angular velocity sensor (gyro sensors 55, 56) and an acceleration sensor (37), and calculates the posture of the input device in a three-dimensional space. It is a calculation device. The posture calculation device includes posture calculation means (CPU 10 that executes step S4; hereinafter, only the step number will be described), acceleration vector calculation means (S2), and first correction means (S5). The attitude calculation means acquires an angular velocity detected by the angular velocity sensor, and calculates a new attitude (first attitude data 68) by rotating the previous attitude in accordance with the acquired angular velocity . The acceleration vector calculation means calculates an acceleration vector (Va1 or Va2) indicating the acceleration of the input device based on the acceleration data (64) from the acceleration sensor. The first correction unit corrects the attitude of the input device in accordance with the change in the direction of the acceleration vector so that the direction of the acceleration vector in the space approaches or matches the vertical downward direction in the space. The first correction means, with respect to the orientation of the input device, is set for the input device, a predetermined axis you correspond to the attitude of the input device before the correction of the (Z-axis vector M1Z shown in FIG. 19) orientation And the orientation of the input device are corrected so that the change between the correction value and the corrected direction is minimized.

  According to the present invention, since the posture calculated using the angular velocity sensor is corrected based on the acceleration data, the posture error calculated from the angular velocity sensor can be corrected, and the input device using the angular velocity sensor can be corrected. Can be calculated accurately.

  Further, according to the present invention, posture correction is performed so that the change between the orientation before correction and the orientation after correction is minimized with respect to a predetermined axis representing the posture. For this reason, in the game process in which the posture of the object in the virtual game space is changed according to the posture of the input device, the change in the orientation of the object corresponding to the predetermined axis is minimized. That is, even if the posture is corrected, the change in the orientation of the object corresponding to the predetermined axis can be made as small as possible. Therefore, according to the present invention, it becomes difficult for the player to notice the change in the posture of the object due to the correction of the posture, and it is possible to prevent the player from feeling uncomfortable due to the correction. As a result, a game operation with good operability can be provided.

  In addition, the first correction unit projects a direction (projection acceleration vector Vap) in which an acceleration vector is projected onto a plane perpendicular to a predetermined axis (plane XY shown in FIG. 19), and a direction (projection gravity) in which a vertical downward direction is projected onto the plane. Direction of the first transformation (first transformation matrix mtx1) that rotates about a predetermined axis so as to approach the vector Gp) and the acceleration vector (acceleration vector Va ′ shown in FIG. 20) on which the first transformation has been performed. And the attitude of the input device may be corrected by a conversion consisting of a second conversion (second conversion matrix mtx2) that performs rotation to bring the vertical direction downward.

  According to the above, the attitude of the input device is determined by the first transformation that rotates about a predetermined axis and the second transformation that rotates to bring the direction of the acceleration vector subjected to the first transformation closer to the vertical downward direction. Therefore, the change in the direction of the predetermined axis due to the correction can be easily and surely minimized.

  The first correction unit may correct the attitude of the input device so that the direction of the acceleration vector coincides with the vertical downward direction (S31-S37).

  According to the above, the posture can be quickly corrected by correcting the posture of the input device so that the direction of the acceleration vector coincides with the vertically downward direction. In particular, in the present invention, since the change in the predetermined axis due to the correction is reduced, even if the posture is corrected quickly, the player is less likely to notice the change in the posture of the object due to the correction of the posture.

  The first correction means controls the correction amount of the attitude of the input device so that the direction of the acceleration vector and the vertical downward direction in the space become closer as the acceleration vector is closer to the gravitational acceleration. Also good.

  According to the above, the posture of the input device is corrected more greatly as the magnitude of the acceleration detected by the acceleration sensor is closer to the magnitude of the gravitational acceleration. Here, the detection result of the acceleration sensor is estimated that the closer the magnitude of acceleration is to the magnitude of gravity acceleration, the more accurately the direction of gravity acceleration is indicated, and the direction of gravity acceleration is obtained accurately. It is guessed. According to the above, the more accurate the direction of gravity acceleration is obtained, the more strongly the correction based on the gravity acceleration is reflected, and when the direction of gravity acceleration is not obtained accurately, the gravity The correction based on the acceleration is not reflected so much. Thereby, the corrected posture can be calculated more accurately.

  The first correction unit may correct the attitude of the input device only when the difference between the magnitude of the acceleration vector and the magnitude of the gravitational acceleration is smaller than a predetermined reference.

  According to the above, when the difference between the magnitude of the acceleration vector and the magnitude of the gravitational acceleration is equal to or greater than a predetermined reference, the first correction unit does not perform correction. In other words, if it is assumed that the acceleration vector does not accurately indicate the direction of gravitational acceleration, the correction of the (inaccurate) acceleration vector is not performed, resulting in a more accurate posture. Can be calculated.

  The input device may further include an imaging unit (imaging element 40). At this time, the posture calculation device further includes second correction means (S6) for further correcting the posture of the input device based on a predetermined image to be captured (captured image) picked up by the image pickup means.

  According to the above, the attitude of the input device can be calculated more accurately by further correcting the attitude of the input device based on the image of the imaging means.

  The second correction unit may correct the posture of the input device by adding rotation about a predetermined axis (Z axis) to the posture of the input device.

  According to the above, the correction based on the image of the imaging unit is performed with respect to the rotation around the predetermined axis. If the rotation is about a predetermined axis, the correction is not easily noticed by the player, so that the attitude of the input device can be calculated more accurately while maintaining the operability of the input device.

  Further, the present invention may be provided as a game device that performs the game process (S7) using the posture corrected by the posture calculation device as the posture of the input device.

  According to the above, the game player can perform the game operation using the accurate attitude of the input device corrected based on the acceleration data and the image to be captured as the game input. The operability of game operations can be improved.

  In addition, the present invention may be implemented in the form of an attitude calculation program or a game program that causes a computer of an information processing apparatus to function as each of the above means.

  According to the present invention, the posture calculated using the angular velocity sensor is corrected based on the acceleration data. As a result, an error in the posture calculated from the angular velocity sensor can be corrected, so that the posture of the input device can be accurately calculated using the angular velocity sensor.

External view of game system Functional block diagram of game device The perspective view which shows the external appearance structure of an input device Perspective view showing the external configuration of the controller Diagram showing the internal structure of the controller Diagram showing the internal structure of the controller Block diagram showing the configuration of the input device The figure which shows the vector which shows a 1st attitude | position and a 2nd attitude | position The figure which shows vector v3 which shows correction amount The figure which shows the vector which shows the 1st attitude | position after correction | amendment by a 1st correction process. The figure which shows the vector which shows a 1st attitude | position and a 3rd attitude | position The figure which shows the 1st attitude | position after correction | amendment by a 2nd correction process. The figure which shows the main data memorize | stored in the main memory of a game device Main flowchart showing the flow of processing executed in the game device The flowchart which shows the flow of the 1st process example of the 1st correction process (step S5) shown in FIG. A diagram showing vectors representing the direction of gravity, acceleration, and posture in a spatial coordinate system The figure which shows the state which rotated the 1st attitude | position (Z-axis vector M1Z) by the method of rotating the acceleration vector Va by the shortest distance from the state shown in FIG. Flowchart showing the flow of the second processing example of the first correction processing (step S5) shown in FIG. Diagram showing projected acceleration vector and projected gravity vector The figure which shows the state which performed the 1st conversion from the state shown in FIG. The figure which shows the state which performed the 2nd conversion from the state shown in FIG. Flowchart showing the flow of the second correction process (step S6) shown in FIG. The figure which shows the two-dimensional coordinate corresponding to a captured image

[Overall configuration of game system]
With reference to FIG. 1, a game system 1 including a game apparatus, which is an example of an attitude calculation apparatus according to an embodiment of the present invention, will be described. FIG. 1 is an external view of the game system 1. Hereinafter, the game apparatus and the game program of the present embodiment will be described using a stationary game apparatus as an example. In FIG. 1, the game system 1 includes a television receiver (hereinafter simply referred to as “TV”) 2, a game device 3, an optical disk 4, an input device 8, and a marker unit 6. In the present system, game processing is executed by the game device 3 based on a game operation using the input device 8.

  An optical disk 4 that is an example of an information storage medium that can be used interchangeably with the game apparatus 3 is detachably inserted into the game apparatus 3. The optical disc 4 stores a game program to be executed on the game apparatus 3. An insertion slot for the optical disk 4 is provided on the front surface of the game apparatus 3. The game apparatus 3 executes a game process by reading and executing a game program stored in the optical disc 4 inserted into the insertion slot.

  A television 2 which is an example of a display device is connected to the game apparatus 3 via a connection cord. The television 2 displays a game image obtained as a result of the game process executed in the game device 3. In addition, a marker unit 6 is installed around the screen of the television 2 (upper side of the screen in FIG. 1). The marker unit 6 includes two markers 6R and 6L at both ends thereof. The marker 6R (same for the marker 6L) is specifically one or more infrared LEDs, and outputs infrared light toward the front of the television 2. The marker unit 6 is connected to the game apparatus 3, and the game apparatus 3 can control lighting of each infrared LED included in the marker unit 6.

  The input device 8 gives operation data indicating the content of the operation performed on the own device to the game device 3. In the present embodiment, the input device 8 includes a controller 5 and a gyro sensor unit 7. Although details will be described later, the input device 8 has a configuration in which a gyro sensor unit 7 is detachably connected to the controller 5. The controller 5 and the game apparatus 3 are connected by wireless communication. In the present embodiment, for example, Bluetooth (registered trademark) technology is used for wireless communication between the controller 5 and the game apparatus 3. In other embodiments, the controller 5 and the game apparatus 3 may be connected by wire.

[Internal configuration of game device 3]
Next, the internal configuration of the game apparatus 3 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration of the game apparatus 3. The game apparatus 3 includes a CPU 10, a system LSI 11, an external main memory 12, a ROM / RTC 13, a disk drive 14, an AV-IC 15 and the like.

  The CPU 10 executes a game process by executing a game program stored on the optical disc 4, and functions as a game processor. The CPU 10 is connected to the system LSI 11. In addition to the CPU 10, an external main memory 12, a ROM / RTC 13, a disk drive 14, and an AV-IC 15 are connected to the system LSI 11. The system LSI 11 performs processing such as control of data transfer between components connected thereto, generation of an image to be displayed, and acquisition of data from an external device. The internal configuration of the system LSI will be described later. The volatile external main memory 12 stores a program such as a game program read from the optical disc 4 or a game program read from the flash memory 17, or stores various data. Used as a work area and buffer area. The ROM / RTC 13 includes a ROM (so-called boot ROM) in which a program for starting the game apparatus 3 is incorporated, and a clock circuit (RTC: Real Time Clock) that counts time. The disk drive 14 reads program data, texture data, and the like from the optical disk 4 and writes the read data to an internal main memory 11e or an external main memory 12 described later.

The system LSI 11 includes an input / output processor (I / O processor) 11a, a GPU (Graphics Processor Unit) 11b, a DSP (Digital Signal Processor) 11c, and a VRA.
1d and an internal main memory 11e are provided. Although not shown, these components 11a to 11e are connected to each other by an internal bus.

  The GPU 11b forms part of a drawing unit and generates an image according to a graphics command (drawing command) from the CPU 10. The VRAM 11d stores data (data such as polygon data and texture data) necessary for the GPU 11b to execute the graphics command. When an image is generated, the GPU 11b creates image data using data stored in the VRAM 11d.

  The DSP 11c functions as an audio processor, and generates sound data using sound data and sound waveform (tone color) data stored in the internal main memory 11e and the external main memory 12.

  The image data and audio data generated as described above are read out by the AV-IC 15. The AV-IC 15 outputs the read image data to the television 2 via the AV connector 16, and outputs the read audio data to the speaker 2 a built in the television 2. As a result, an image is displayed on the television 2 and a sound is output from the speaker 2a.

  The input / output processor 11a performs transmission / reception of data to / from components connected to the input / output processor 11a and downloads data from an external device. The input / output processor 11a is connected to the flash memory 17, the wireless communication module 18, the wireless controller module 19, the expansion connector 20, and the memory card connector 21. An antenna 22 is connected to the wireless communication module 18, and an antenna 23 is connected to the wireless controller module 19.

  The input / output processor 11a is connected to the network via the wireless communication module 18 and the antenna 22, and can communicate with other game devices and various servers connected to the network. The input / output processor 11a periodically accesses the flash memory 17 to detect the presence / absence of data that needs to be transmitted to the network. If there is such data, the input / output processor 11a communicates with the network via the wireless communication module 18 and the antenna 22. Send. Further, the input / output processor 11a receives data transmitted from other game devices and data downloaded from the download server via the network, the antenna 22 and the wireless communication module 18, and receives the received data in the flash memory 17. Remember. By executing the game program, the CPU 10 reads out the data stored in the flash memory 17 and uses it in the game program. In the flash memory 17, in addition to data transmitted and received between the game apparatus 3 and other game apparatuses and various servers, save data (game result data or intermediate data) of the game played using the game apparatus 3 May be stored.

  The input / output processor 11a receives operation data transmitted from the controller 5 via the antenna 23 and the wireless controller module 19, and stores (temporarily stores) the data in the buffer area of the internal main memory 11e or the external main memory 12.

  Further, an expansion connector 20 and a memory card connector 21 are connected to the input / output processor 11a. The expansion connector 20 is a connector for an interface such as USB or SCSI, and connects a medium such as an external storage medium, a peripheral device such as another controller, or a wired communication connector. By connecting, communication with the network can be performed instead of the wireless communication module 18. The memory card connector 21 is a connector for connecting an external storage medium such as a memory card. For example, the input / output processor 11a can access an external storage medium via the expansion connector 20 or the memory card connector 21 to store data in the external storage medium or read data from the external storage medium.

  The game apparatus 3 is provided with a power button 24, a reset button 25, and an eject button 26. The power button 24 and the reset button 25 are connected to the system LSI 11. When the power button 24 is turned on, power is supplied to each component of the game apparatus 3 via an AC adapter (not shown). When the reset button 25 is pressed, the system LSI 11 restarts the boot program for the game apparatus 3. The eject button 26 is connected to the disk drive 14. When the eject button 26 is pressed, the optical disk 4 is ejected from the disk drive 14.

[Configuration of Input Device 8]
Next, the input device 8 will be described with reference to FIGS. FIG. 3 is a perspective view showing an external configuration of the input device 8. FIG. 4 is a perspective view showing an external configuration of the controller 5. 3 is a perspective view of the controller 5 as seen from the upper rear side, and FIG. 4 is a perspective view of the controller 5 as seen from the lower front side.

  3 and 4, the controller 5 includes a housing 31 formed by plastic molding, for example. The housing 31 has a substantially rectangular parallelepiped shape whose longitudinal direction is the front-rear direction (the Z-axis direction shown in FIG. 3), and is a size that can be gripped with one hand of an adult or a child as a whole. The player can perform a game operation by pressing a button provided on the controller 5 and moving the controller 5 itself to change its position and posture.

  The housing 31 is provided with a plurality of operation buttons. As shown in FIG. 3, a cross button 32a, a first button 32b, a second button 32c, an A button 32d, a minus button 32e, a home button 32f, a plus button 32g, and a power button 32h are provided on the upper surface of the housing 31. It is done. In the present specification, the upper surface of the housing 31 on which these buttons 32a to 32h are provided may be referred to as a “button surface”. On the other hand, as shown in FIG. 4, a recess is formed on the lower surface of the housing 31, and a B button 32i is provided on the rear inclined surface of the recess. A function corresponding to the game program executed by the game apparatus 3 is appropriately assigned to each of the operation buttons 32a to 32i. The power button 32h is for remotely turning on / off the main body of the game apparatus 3. The home button 32 f and the power button 32 h are embedded in the upper surface of the housing 31. This can prevent the player from pressing the home button 32f or the power button 32h by mistake.

  A connector 33 is provided on the rear surface of the housing 31. The connector 33 is used to connect another device (for example, the gyro sensor unit 7 or another controller) to the controller 5. Further, locking holes 33a are provided on both sides of the connector 33 on the rear surface of the housing 31 in order to prevent the other devices from being easily detached.

  A plurality (four in FIG. 3) of LEDs 34 a to 34 d are provided behind the upper surface of the housing 31. Here, the controller type (number) is assigned to the controller 5 to distinguish it from other main controllers. The LEDs 34a to 34d are used for the purpose of notifying the player of the controller type currently set in the controller 5 and notifying the player of the remaining battery level of the controller 5. Specifically, when a game operation is performed using the controller 5, any one of the plurality of LEDs 34a to 34d is turned on according to the controller type.

  Further, the controller 5 has an imaging information calculation unit 35 (FIG. 6), and a light incident surface 35a of the imaging information calculation unit 35 is provided on the front surface of the housing 31, as shown in FIG. The light incident surface 35a is made of a material that transmits at least infrared light from the markers 6R and 6L.

  Between the first button 32b and the home button 32f on the upper surface of the housing 31, a sound release hole 31a for releasing sound from the speaker 49 (FIG. 5) built in the controller 5 is formed.

  Next, the internal structure of the controller 5 will be described with reference to FIGS. 5 and 6 are diagrams showing the internal structure of the controller 5. FIG. FIG. 5 is a perspective view showing a state in which the upper housing (a part of the housing 31) of the controller 5 is removed. FIG. 6 is a perspective view showing a state in which the lower casing (a part of the housing 31) of the controller 5 is removed. The perspective view shown in FIG. 6 is a perspective view of the substrate 30 shown in FIG.

  In FIG. 5, a substrate 30 is fixed inside the housing 31, and operation buttons 32 a to 32 h, LEDs 34 a to 34 d, an acceleration sensor 37, an antenna 45, and a speaker 49 are provided on the upper main surface of the substrate 30. Etc. are provided. These are connected to a microcomputer (microcomputer) 42 (see FIG. 6) by wiring (not shown) formed on the substrate 30 and the like. In the present embodiment, the acceleration sensor 37 is disposed at a position shifted from the center of the controller 5 with respect to the X-axis direction. This makes it easier to calculate the movement of the controller 5 when the controller 5 is rotated about the Z axis. The acceleration sensor 37 is disposed in front of the center of the controller 5 in the longitudinal direction (Z-axis direction). Further, the controller 5 functions as a wireless controller by the wireless module 44 (FIG. 6) and the antenna 45.

  On the other hand, in FIG. 6, an imaging information calculation unit 35 is provided at the front edge on the lower main surface of the substrate 30. The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41 in order from the front of the controller 5. These members 38 to 41 are respectively attached to the lower main surface of the substrate 30.

  Further, the microcomputer 42 and the vibrator 48 are provided on the lower main surface of the substrate 30. The vibrator 48 is, for example, a vibration motor or a solenoid, and is connected to the microcomputer 42 by wiring formed on the substrate 30 or the like. The controller 48 is vibrated by the operation of the vibrator 48 according to the instruction of the microcomputer 42. As a result, a so-called vibration-compatible game in which the vibration is transmitted to the hand of the player holding the controller 5 can be realized. In the present embodiment, the vibrator 48 is disposed slightly forward of the housing 31. That is, by arranging the vibrator 48 on the end side of the center of the controller 5, the entire controller 5 can be vibrated greatly by the vibration of the vibrator 48. The connector 33 is attached to the rear edge on the lower main surface of the substrate 30. 5 and 6, the controller 5 includes a crystal resonator that generates a basic clock of the microcomputer 42, an amplifier that outputs an audio signal to the speaker 49, and the like.

  The gyro sensor unit 7 is an example of an angular velocity sensor, and includes gyro sensors (gyro sensors 55 and 56 shown in FIG. 7) that detect angular velocities around three axes. The gyro sensor unit 7 is detachably attached to the connector 33 of the controller 5. A plug (plug 53 shown in FIG. 7) that can be connected to the connector 33 is provided at the front end of the gyro sensor unit 7 (end on the Z-axis positive direction side shown in FIG. 3). Further, hooks (not shown) are provided on both sides of the plug 53. In a state where the gyro sensor unit 7 is attached to the controller 5, the plug 53 is connected to the connector 33 and the hook is locked in the locking hole 33 a of the controller 5. Thereby, the controller 5 and the gyro sensor unit 7 are firmly fixed. The gyro sensor unit 7 has a button 51 on a side surface (surface in the X-axis direction shown in FIG. 3). The button 51 is configured such that when the button 51 is pressed, the hook is released from the locked state with respect to the locking hole 33a. Therefore, the gyro sensor unit 7 can be detached from the controller 5 by removing the plug 53 from the connector 33 while pressing the button 51.

  A connector having the same shape as the connector 33 is provided at the rear end of the gyro sensor unit 7. Therefore, other devices that can be attached to the controller 5 (connector 33 thereof) can also be attached to the connector of the gyro sensor unit 7. In FIG. 3, a cover 52 is detachably attached to the connector.

  The shapes of the controller 5 and the gyro sensor unit 7 shown in FIGS. 3 to 6, the shapes of the operation buttons, the number of acceleration sensors and vibrators, and the installation positions are merely examples, and other shapes, numbers, Even in the installation position, the present invention can be realized. In the present embodiment, the imaging direction by the imaging unit is the positive Z-axis direction, but the imaging direction may be any direction. That is, the position of the imaging information calculation unit 35 in the controller 5 (the light incident surface 35a of the imaging information calculation unit 35) does not have to be the front surface of the housing 31, and other surfaces can be used as long as light can be taken in from the outside of the housing 31. May be provided.

  FIG. 7 is a block diagram showing a configuration of the input device 8 (the controller 5 and the gyro sensor unit 7). The controller 5 includes an operation unit 32 (operation buttons 32a to 32i), a connector 33, an imaging information calculation unit 35, a communication unit 36, and an acceleration sensor 37. The controller 5 transmits data indicating the details of the operation performed on the own device to the game apparatus 3 as operation data.

  The operation unit 32 includes the operation buttons 32a to 32i described above, and the operation button data indicating the input state (whether or not each operation button 32a to 32i is pressed) to each operation button 32a to 32i is transmitted to the microcomputer of the communication unit 36. Output to 42.

  The imaging information calculation unit 35 is a system for analyzing the image data captured by the imaging unit, discriminating a region having a high luminance in the image data, and calculating a center of gravity position, a size, and the like of the region. Since the imaging information calculation unit 35 has a sampling period of, for example, about 200 frames / second at the maximum, it can track and analyze even a relatively fast movement of the controller 5.

  The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41. The infrared filter 38 passes only infrared rays from the light incident from the front of the controller 5. The lens 39 collects the infrared light transmitted through the infrared filter 38 and makes it incident on the image sensor 40. The image sensor 40 is a solid-state image sensor such as a CMOS sensor or a CCD sensor, for example, and receives the infrared light collected by the lens 39 and outputs an image signal. Here, the markers 6 </ b> R and 6 </ b> L of the marker unit 6 disposed in the vicinity of the display screen of the television 2 are configured by infrared LEDs that output infrared light toward the front of the television 2. Therefore, by providing the infrared filter 38, the image sensor 40 receives only the infrared light that has passed through the infrared filter 38 and generates image data, so that the images of the markers 6R and 6L can be captured more accurately. Hereinafter, an image captured by the image sensor 40 is referred to as a captured image. Image data generated by the image sensor 40 is processed by the image processing circuit 41. The image processing circuit 41 calculates the position of the imaging target (markers 6R and 6L) in the captured image. The image processing circuit 41 outputs coordinates indicating the calculated position to the microcomputer 42 of the communication unit 36. The coordinate data is transmitted to the game apparatus 3 as operation data by the microcomputer 42. Hereinafter, the coordinates are referred to as “marker coordinates”. Since the marker coordinates change corresponding to the direction (tilt angle) and position of the controller 5 itself, the game apparatus 3 can calculate the direction and position of the controller 5 using the marker coordinates.

  In other embodiments, the controller 5 may not include the image processing circuit 41, and the captured image itself may be transmitted from the controller 5 to the game apparatus 3. At this time, the game apparatus 3 may have a circuit or a program having the same function as the image processing circuit 41, and may calculate the marker coordinates.

  The acceleration sensor 37 detects the acceleration (including gravity acceleration) of the controller 5, that is, detects the force (including gravity) applied to the controller 5. The acceleration sensor 37 detects the value of the acceleration (linear acceleration) in the linear direction along the sensing axis direction among the accelerations applied to the detection unit of the acceleration sensor 37. For example, in the case of a multi-axis acceleration sensor having two or more axes, the component acceleration along each axis is detected as the acceleration applied to the detection unit of the acceleration sensor. For example, the triaxial or biaxial acceleration sensor may be of the type available from Analog Devices, Inc. or ST Microelectronics NV. The acceleration sensor 37 is, for example, a capacitance type acceleration sensor, but other types of acceleration sensors may be used.

  In the present embodiment, the acceleration sensor 37 has a vertical direction (Y-axis direction shown in FIG. 3), a horizontal direction (X-axis direction shown in FIG. 3), and a front-back direction (Z-axis direction shown in FIG. 3) with reference to the controller 5. ) Linear acceleration is detected in each of the three axis directions. Since the acceleration sensor 37 detects acceleration in the linear direction along each axis, the output from the acceleration sensor 37 represents the linear acceleration value of each of the three axes. That is, the detected acceleration is represented as a three-dimensional vector (ax, ay, az) in an XYZ coordinate system (controller coordinate system) set with reference to the input device 8 (controller 5). Hereinafter, a vector having the respective acceleration values related to the three axes detected by the acceleration sensor 37 as components is referred to as an acceleration vector.

  Data indicating the acceleration detected by the acceleration sensor 37 (acceleration data) is output to the communication unit 36. The acceleration detected by the acceleration sensor 37 changes in accordance with the direction (tilt angle) and movement of the controller 5 itself, so that the game apparatus 3 can calculate the direction and movement of the controller 5 using the acceleration data. it can. In the present embodiment, the game apparatus 3 determines the attitude of the controller 5 based on the acceleration data.

  Data (acceleration data) indicating the acceleration (acceleration vector) detected by the acceleration sensor 37 is output to the communication unit 36. In the present embodiment, the acceleration sensor 37 is used as a sensor that outputs data for determining the tilt angle of the controller 5.

  In addition, based on the acceleration signal output from the acceleration sensor 37, a computer such as a processor (for example, the CPU 10) of the game apparatus 3 or a processor (for example, the microcomputer 42) of the controller 5 performs processing, whereby further information regarding the controller 5 is obtained. Those skilled in the art will be able to easily understand from the description of the present specification that can be estimated or calculated (determined). For example, when processing on the computer side is executed on the assumption that the controller 5 on which the acceleration sensor 37 is mounted is stationary (that is, the processing is executed assuming that the acceleration detected by the acceleration sensor is only gravitational acceleration). When the controller 5 is actually stationary, it can be determined whether or not the attitude of the controller 5 is inclined with respect to the direction of gravity based on the detected acceleration. Specifically, whether or not the controller 5 is inclined with respect to the reference depending on whether or not 1G (gravity acceleration) is applied, based on the state in which the detection axis of the acceleration sensor 37 is directed vertically downward. It is possible to know how much it is inclined with respect to the reference according to its size. Further, in the case of the multi-axis acceleration sensor 37, it is possible to know in detail how much the controller 5 is inclined with respect to the direction of gravity by further processing the acceleration signal of each axis. . In this case, the processor may calculate the tilt angle of the controller 5 based on the output from the acceleration sensor 37, or may calculate the tilt direction of the controller 5 without calculating the tilt angle. Good. Thus, by using the acceleration sensor 37 in combination with the processor, the tilt angle or posture of the controller 5 can be determined.

  On the other hand, when it is assumed that the controller 5 is in a dynamic state (a state in which the controller 5 is moved), the acceleration sensor 37 detects an acceleration corresponding to the movement of the controller 5 in addition to the gravitational acceleration. Therefore, the movement direction of the controller 5 can be known by removing the gravitational acceleration component from the detected acceleration by a predetermined process. Even if it is assumed that the controller 5 is in a dynamic state, the direction of gravity is obtained by removing the acceleration component corresponding to the movement of the acceleration sensor from the detected acceleration by a predetermined process. It is possible to know the inclination of the controller 5 with respect to. In another embodiment, the acceleration sensor 37 is a built-in process for performing a predetermined process on the acceleration signal before outputting the acceleration signal detected by the built-in acceleration detection means to the microcomputer 42. An apparatus or other type of dedicated processing apparatus may be provided. A built-in or dedicated processing device converts the acceleration signal into a tilt angle (or other preferred parameter) if, for example, the acceleration sensor 37 is used to detect static acceleration (eg, gravitational acceleration). It may be a thing.

  The communication unit 36 includes a microcomputer 42, a memory 43, a wireless module 44, and an antenna 45. The microcomputer 42 controls the wireless module 44 that wirelessly transmits data acquired by the microcomputer 42 to the game apparatus 3 while using the memory 43 as a storage area when performing processing. The microcomputer 42 is connected to the connector 33. Data transmitted from the gyro sensor unit 7 is input to the microcomputer 42 via the connector 33. Hereinafter, the configuration of the gyro sensor unit 7 will be described.

  The gyro sensor unit 7 includes a plug 53, a microcomputer 54, a 2-axis gyro sensor 55, and a 1-axis gyro sensor 56. As described above, the gyro sensor unit 7 detects angular velocities around three axes (XYZ axes in the present embodiment), and transmits data (angular velocity data) indicating the detected angular velocities to the controller 5.

  The biaxial gyro sensor 55 detects an angular velocity around the X axis and an angular velocity (per unit time) around the Y axis. Further, the uniaxial gyro sensor 56 detects an angular velocity (per unit time) around the Z axis. In this specification, the rotation directions around the XYZ axes are referred to as a roll direction, a pitch direction, and a yaw direction, respectively, with reference to the imaging direction (Z-axis positive direction) of the controller 5. That is, the biaxial gyro sensor 55 detects angular velocities in the roll direction (rotation direction around the X axis) and the pitch direction (rotation direction around the Y axis), and the single axis gyro sensor 56 detects the yaw direction (around the Z axis). Detect the angular velocity in the rotation direction.

  In this embodiment, the 2-axis gyro sensor 55 and the 1-axis gyro sensor 56 are used to detect the angular velocity around the three axes. However, in other embodiments, the angular velocity around the three axes is Any number and combination of gyro sensors may be used as long as they can be detected.

  Further, in the present embodiment, for the purpose of facilitating the calculation in the posture calculation process described later, the three axes where the gyro sensors 55 and 56 detect the angular velocity are the three axes (XYZ) where the acceleration sensor 37 detects the acceleration. Axis). However, in other embodiments, the three axes for detecting the angular velocities by the gyro sensors 56 and 57 and the three axes for detecting the acceleration by the acceleration sensor 37 may not coincide with each other.

  Data indicating the angular velocity detected by each of the gyro sensors 56 and 57 is output to the microcomputer 54. Accordingly, the microcomputer 54 receives data indicating the angular velocity around the three axes of the XYZ axes. The microcomputer 54 transmits data indicating the angular velocities around the three axes as angular velocity data to the controller 5 via the plug 53. Although transmission from the microcomputer 54 to the controller 5 is sequentially performed every predetermined cycle, since the game processing is generally performed in units of 1/60 seconds (one frame time), this time Transmission is preferably performed in the following cycle.

  Returning to the description of the controller 5, the data output from the operation unit 32, the imaging information calculation unit 35, and the acceleration sensor 37 to the microcomputer 42 and the data transmitted from the gyro sensor unit 7 to the microcomputer 42 are temporarily stored. Stored in the memory 43. These data are transmitted to the game apparatus 3 as the operation data. That is, the microcomputer 42 outputs the operation data stored in the memory 43 to the wireless module 44 when the transmission timing to the wireless controller module 19 of the game apparatus 3 arrives. The wireless module 44 modulates a carrier wave of a predetermined frequency with operation data using, for example, Bluetooth (registered trademark) technology, and radiates a weak radio signal from the antenna 45. That is, the operation data is modulated by the wireless module 44 into a weak radio signal and transmitted from the controller 5. The weak radio signal is received by the wireless controller module 19 on the game apparatus 3 side. By demodulating and decoding the received weak radio signal, the game apparatus 3 can acquire operation data. And CPU10 of the game device 3 performs a game process based on the acquired operation data and a game program. Note that the wireless transmission from the communication unit 36 to the wireless controller module 19 is sequentially performed at predetermined intervals, but the game processing is generally performed in units of 1/60 seconds (one frame time). Therefore, it is preferable to perform transmission at a period equal to or shorter than this time. The communication unit 36 of the controller 5 outputs each operation data to the wireless controller module 19 of the game apparatus 3 at a rate of once every 1/200 seconds, for example.

  By using the controller 5, the player can perform an operation of tilting the controller 5 to an arbitrary tilt angle in addition to the conventional general game operation of pressing each operation button. In addition, according to the controller 5, the player can also perform an operation of instructing an arbitrary position on the screen by the controller 5 and an operation of moving the controller 5 itself.

[Outline of posture calculation processing]
Next, with reference to FIGS. 8 to 12, an outline of a posture calculation process for calculating the posture of the input device 8 executed in the game apparatus 3 will be described. In the present embodiment, the game apparatus 3 acquires data (operation data) from the input device 8 including the gyro sensors 55 and 56, the acceleration sensor 37, and the imaging means (imaging element 40). Calculate the posture. In the present embodiment, the input device 8 is configured to include both the acceleration sensor 37 and the image sensor 40. However, in other embodiments, the input device 8 includes only one of the acceleration sensor 37 and the image sensor 40. It may be a configuration.

  The game apparatus 3 includes (1) posture calculation means, (2) first correction means, and (3) second correction means. In the present embodiment, these means are realized by causing a game program (attitude calculation program) executed by the computer (CPU 10) of the game apparatus 3 to function as the means. In other embodiments, part or all of the above means may be realized as a dedicated circuit included in the game apparatus 3.

(1) Attitude calculation means The attitude calculation means calculates the attitude of the input device 8 based on the angular velocity detected by the gyro sensors 55 and 56 (step S4 described later). Any method may be used to calculate the posture from the angular velocity. For example, there is a method of sequentially adding the angular velocity (per unit time) to the initial posture. That is, the current posture can be calculated by integrating the angular velocities sequentially output from the gyro sensors 55 and 56 and calculating the amount of change in posture from the initial state from the integration result. In the following, the posture of the input device 8 calculated from the angular velocity by the posture calculation means is referred to as a “first posture”. However, the posture after the first posture is corrected is also referred to as the first posture.

  Here, the first posture calculated using the angular velocities detected by the gyro sensors 55 and 56 has an error between the actual posture of the input device 8 due to erroneous detection of the gyro sensors 55 and 56. May occur. Therefore, in the present embodiment, the game apparatus 3 corrects the first posture using the acceleration detected by the acceleration sensor 37. Further, the first posture is corrected using an image (captured image) captured by the image sensor 40.

(2) First Correction Unit The first correction unit corrects the first posture based on acceleration data detected by the acceleration sensor 37 (step S5 described later). In the present embodiment, the first correction unit performs correction to bring the first posture closer to the second posture. Here, the second posture refers to a posture determined from the acceleration data, specifically, a posture of the input device 8 when it is assumed that the direction of acceleration indicated by the acceleration data is vertically downward. That is, the posture is calculated on the assumption that the acceleration indicated by the acceleration data is gravitational acceleration. Hereinafter, a correction process (first correction process) by the first correction unit will be described with reference to FIGS. In the present embodiment, first and second processing examples are described as specific methods of the first correction unit, but FIGS. 8 to 10 illustrate the first processing example.

  FIG. 8 is a diagram illustrating correction of the first posture using the second posture. In this embodiment, the posture in the three-dimensional space is actually processed. However, in FIGS. 8 to 10, the posture in the two-dimensional plane will be described for easy understanding of the drawings. A vector G shown in FIG. 8A indicates a vertically downward direction defined in a spatial coordinate system based on a predetermined position in a space where the input device 8 exists, that is, a gravity direction. Also, the vector v1 shown in FIG. 8A is a spatial coordinate of the downward vector of the input device 8 when the controller 5 is in the first posture (that is, the negative Y-axis direction shown in FIGS. 3 to 5). It shows the orientation in the system. When the posture of the input device 8 is in the basic state, the vector indicating the posture matches the vector G. Therefore, the vector v1 corresponds to the first posture in the spatial coordinate system. The first posture can also be expressed as a rotation made by the vector v1 with respect to the vector G, and is shown as an angle θ1 in the two-dimensional FIG. Since the first posture is calculated based on the angular velocity, the vector v1 is calculated by adding the angular velocity to the previous posture and rotating the vector v1. The second posture is calculated based on the acceleration data. A vector v2 shown in FIG. 8A indicates the direction of acceleration indicated by the acceleration data (the direction of acceleration in the view coordinate system). However, the acceleration data is an acceleration applied to the input device 8, and what can be acquired is a vector in a coordinate system based on the input device 8. FIG. 8B shows the relationship between the acceleration sensor axis and the acceleration vector. As shown in FIG. 8B, when the angle formed by the acceleration vector v0 acquired from the acceleration sensor and the Y-axis negative direction of the sensor is θ2, the vector v1 in the spatial coordinate system of FIG. A vector v2 obtained by adding the rotation θ2 to the acceleration vector in the spatial coordinate system. Since the second posture is “the posture of the input device 8 when the acceleration direction indicated by the acceleration data is regarded as being vertically downward” as described above, the rotation of the angle θ2 from the vector v2 to the vector v1 is performed. Becomes the second posture. If the second posture is represented by a downward vector of the input device 8 in the spatial coordinate system as a vector v1, it can be represented as a vector v2 'obtained by rotating the vector G by θ2. In the case of a three-dimensional posture, it can be expressed by a three-dimensional rotation matrix or the like. When the first posture is correctly calculated from the angular velocity and the acceleration data accurately indicates the direction of gravity, the direction of the vector v2 indicating the direction of acceleration is the vertical downward direction of the spatial coordinate system, that is, gravity. Match the direction. That is, when the first posture is not correctly calculated from the angular velocity, or when the acceleration data does not indicate the correct gravity direction, the vector v2 indicating the direction of acceleration and the gravity as shown in FIG. The direction vector G is different. In a situation where the direction indicated by the acceleration data is assumed to coincide with the direction of gravity, such as in a stationary state, the vector v2 is more accurate as data corresponding to the attitude of the input device 8 than the vector v1. It is considered a thing. Even when the input device is not stationary, taking into account the accuracy of the average posture within a certain period of time, the acceleration vector will be close to the direction of gravity on average, so that the error may increase with time. It can be considered more reliable than the posture calculated from the accumulated angular velocity. On the other hand, if the correct posture has been calculated at the previous calculation timing, the posture at the next calculation timing is calculated more accurately using the angular velocity than the acceleration. it is conceivable that. In other words, the posture calculation based on the angular velocity has a smaller error for each timing than the calculation based on the acceleration, but the error increases with time, while the posture calculation based on the acceleration may have a larger error for each timing. Since it can be calculated at each timing, it has a feature that no error is accumulated. Therefore, the first correction unit performs correction in consideration of both the first posture and the second posture.

  The correction by the first correction means is correction that brings the first posture closer to the second posture. That is, the first correction unit performs correction to bring the angle θ1 close to the angle θ2. This may be expressed as a correction that brings the vector v1 closer to the vector v2 '. However, in the calculation process, if the vector v2 is known, the correction is possible even if the vector v2 'itself is not calculated. In the present embodiment, the correction is performed using a vector v3 indicating the correction amount. FIG. 9 is a diagram illustrating a vector v3 indicating the correction amount. A vector v3 illustrated in FIG. 9 is a vector indicating a correction amount for correcting the first posture. Specifically, an angle Δθ formed by the vector v3 with respect to the vector v2 is a correction amount. Although details will be described later, the vector v3 is set between the vector G and the vector v2 (see FIG. 9). By rotating the vector v1 by Δθ, the vector v1 approaches the vector v2 ′ described above.

  The first correction process is performed by rotating the first posture (vector v1) by the correction amount. FIG. 10 is a diagram illustrating a vector indicating the first posture after the correction by the first correction process. As shown in FIG. 10, the corrected first posture (vector v1 ') is obtained by rotating the first posture before correction (vector v1) by an angle Δθ. As a result, the angle θ1 ′ representing the first posture after correction is between the angle θ1 and the angle θ2, and it can be seen that the correction for bringing the angle θ1 closer to the angle θ2 was performed.

  In the first method, the first correction unit performs correction to bring the first posture closer to the second posture, and does not match the corrected first posture with the second posture. This is to prevent the corrected first posture from changing suddenly even when the acceleration data changes suddenly due to erroneous detection or intense operation. However, the first correction unit may perform correction so that the corrected first posture matches the second posture (second method described later). Although the details will be described later, in the first method, the rate at which the first correction unit brings the first posture closer to the second posture is the magnitude of the acceleration indicated by the acceleration data (more specifically, , The difference between the magnitude and the magnitude of gravitational acceleration). However, in other embodiments, the ratio may be a predetermined fixed value.

(3) Second Correction Unit The second correction unit corrects the first posture based on a predetermined image to be captured that is captured by the imaging unit (step S6 described later). Here, in the present embodiment, the predetermined imaging target is the marker unit 6 (infrared LED). In the present embodiment, the second correction unit performs correction to bring the first posture closer to the third posture. The third posture is a posture calculated from the image to be imaged, and specifically, the posture of the input device 8 calculated from the orientation and / or position of the imaged object in the image. Hereinafter, the correction processing (second correction processing) by the second correction means will be described with reference to FIGS. 11 and 12.

  FIG. 11 is a diagram illustrating the correction of the first posture using the third posture. In this embodiment, the posture in the three-dimensional space is actually processed. However, in FIG. 11 and FIG. 12, the posture in the two-dimensional plane will be described for easy understanding of the drawings. A vector v1 shown in FIG. 11 indicates the first posture in the spatial coordinate system. A vector v4 shown in FIG. 11 indicates the third posture in the spatial coordinate system. Since the position and orientation of the marker unit 6 are determined in advance, the orientation of the input device 8 can be relatively calculated based on the orientation and position of the marker in the image. Assuming that the third posture represents the correct posture, when the first posture is correctly calculated from the angular velocity, the vector v1 indicating the first posture indicates the third posture. It matches the vector v4. That is, when the first posture is not correctly calculated from the angular velocity, the vector v1 indicating the first posture and the vector v4 indicating the third posture are different as shown in FIG.

  The second correction process is performed by bringing the first posture (vector v1) closer to the third posture (vector v4) at a predetermined rate. FIG. 12 is a diagram illustrating the first posture after correction by the second correction processing. As shown in FIG. 12, the first posture after correction (vector v1 ′) brings the first posture before correction (vector v1) closer to the third posture (vector v4) at a predetermined rate. Obtained by.

  Here, depending on the orientation and position of the input device 8, the imaging unit may not be able to capture the marker unit 6. In this case, the second correction unit cannot perform the second correction process. Therefore, if the second correction unit performs correction so that the corrected first posture matches the third posture, the second correction processing is performed from a state where the second correction processing cannot be performed. There is a possibility that the first posture may change abruptly when transitioning to a state where it is possible to do so. In this way, if the first posture suddenly changes without conforming to the player's intention, the player will feel uncomfortable with the operation (even if the corrected posture is accurate). In order to prevent such a sudden change, in the present embodiment, correction is performed so that the first posture approaches the third posture at a predetermined rate. As a result, it is possible to prevent the first posture from changing suddenly, thereby preventing the player from feeling uncomfortable with the operation. However, in other embodiments, such as when the input device 8 can be assumed to be used in a posture in which the imaging unit can always image the marker unit 6, the second correction unit is the corrected first unit. You may correct | amend so that this attitude | position may correspond with a 3rd attitude | position.

  In the present embodiment, the game apparatus 3 executes both the first correction process and the second correction process, but in other embodiments, one of the first correction process and the second correction process. May be executed only. In the present embodiment, the game apparatus 3 performs the first correction process first and the second correction process later, but either the first correction process or the second correction process is performed first. You may do it.

  As described above, according to the present embodiment, the attitude of the input device 8 calculated from the angular velocities detected by the gyro sensors 55 and 56 is corrected using the acceleration detected by the acceleration sensor 37, and imaging is performed. Correction is performed using the image captured by the means. Thereby, the error of the attitude calculated from the gyro sensor can be reduced, and the attitude of the input device 8 can be calculated more accurately.

  In addition, from the detection result of the acceleration sensor 37, the rotation around the gravity direction (rotation in the yaw direction) cannot be detected, and therefore the correction by the first correction unit cannot be performed in the yaw direction. However, the correction using the detection result of the acceleration sensor 37 has a feature that the posture of the input device 8 can be any posture (because acceleration can always be detected). On the other hand, if the marker unit 6 is not present in the imaging direction of the input device 8, the marker coordinates are not detected, and therefore the correction by the second correction unit cannot be performed depending on the attitude of the input device 8. However, the correction using the captured image has a feature that the posture (especially the posture in the roll direction) can be accurately calculated. In the present embodiment, the posture of the input device 8 can be calculated more accurately by performing two types of corrections having different features as described above.

[Details of processing in game device 3]
Next, details of processing executed in the game apparatus 3 will be described. First, main data used in the processing in the game apparatus 3 will be described with reference to FIG. FIG. 13 is a diagram illustrating main data stored in the main memory (the external main memory 12 or the internal main memory 11e) of the game apparatus 3. As shown in FIG. 13, a game program 60, operation data 62, and game processing data 67 are stored in the main memory of the game apparatus 3. In addition to the data shown in FIG. 13, the main memory stores data necessary for game processing, such as image data of various objects appearing in the game and data indicating various parameters of the objects.

  A part or all of the game program 60 is read from the optical disc 4 and stored in the main memory at an appropriate timing after the game apparatus 3 is turned on. The game program 60 includes a posture calculation program 61. The attitude calculation program 61 is a program for executing an attitude calculation process for calculating the attitude of the input device 8.

  The operation data 62 is operation data transmitted from the controller 5 to the game apparatus 3. As described above, since the operation data is transmitted from the controller 5 to the game apparatus 3 at a rate of once every 1/200 seconds, the operation data 62 stored in the main memory is updated at this rate. In the present embodiment, only the latest (last acquired) operation data may be stored in the main memory.

  The operation data 62 includes angular velocity data 63, acceleration data 64, marker coordinate data 65, and operation button data 66. The angular velocity data 63 is data indicating angular velocities detected by the gyro sensors 55 and 56 of the gyro sensor unit 7. Here, the angular velocity data 63 indicates the respective angular velocities around the three axes of XYZ shown in FIG. The acceleration data 64 is data indicating the acceleration (acceleration vector) detected by the acceleration sensor 37. Here, the acceleration data 64 represents a three-dimensional acceleration vector Va1 having as components the accelerations in the directions of the three axes of XYZ shown in FIG. In the present embodiment, the magnitude of the acceleration vector Va1 detected by the acceleration sensor 37 while the controller 5 is stationary is “1”. That is, the magnitude of gravitational acceleration detected by the acceleration sensor 37 is “1”.

  The marker coordinate data 65 is data indicating coordinates calculated by the image processing circuit 41 of the imaging information calculation unit 35, that is, the marker coordinates. The marker coordinates are expressed by a two-dimensional coordinate system (x′y ′ coordinate system shown in FIG. 23) for representing a position on a plane corresponding to the captured image. Note that, when the two markers 6R and 6L are imaged by the imaging element 40, two marker coordinates are calculated. On the other hand, if either one of the markers 6R and 6L is not located within the imageable range of the image sensor 40, only one marker is imaged by the image sensor 40, and only one marker coordinate is calculated. Further, when both of the markers 6R and 6L are not located within the image capturing range of the image sensor 40, the marker is not imaged by the image sensor 40, and the marker coordinates are not calculated. Therefore, the marker coordinate data 65 may indicate two marker coordinates, may indicate one marker coordinate, or may indicate that there is no marker coordinate.

  The operation button data 66 is data indicating an input state for each of the operation buttons 32a to 32i.

  The game process data 67 is data used in a game process (FIG. 14) described later. The game processing data 67 includes first posture data 68, acceleration magnitude data 69, correction degree data 70, correction amount vector data 71, correction matrix data 72, roll posture component data 73, yaw posture component data 74, pitch posture component. Data 75 and third attitude data 76 are included. In addition to the data shown in FIG. 13, the game processing data 67 includes various data (data indicating game parameters, etc.) used in the game processing.

上記行列Ｍ１は、所定の基準姿勢から現在の入力装置８の姿勢への回転を表す回転行列である。以下では、第１の姿勢を示す行列Ｍ１を、「第１姿勢行列Ｍ１」と呼ぶ。なお、第１姿勢行列Ｍ１により表される第１の姿勢は、入力装置８が存在する空間の所定位置を基準としたｘｙｚ座標系（上記空間座標系）における姿勢である。ここでは、ｘｙｚ座標系は、入力装置８がマーカ部６の正面に位置することを前提とし、入力装置８の位置からマーカ部６を向く方向をｚ軸正方向とし、鉛直上向き（重力方向の逆方向）をｙ軸正方向とし、入力装置８の位置からマーカ部６を向いた場合の左方向をｘ軸正方向とした座標系であるとする。ここでは、上記所定の基準姿勢は、マーカ部６の正面に位置する入力装置８の撮像方向がマーカ部６の中央を向き、かつ、コントローラ５のボタン面が鉛直上向きとなる姿勢（すなわち、入力装置８を基準としたＸ軸、Ｙ軸、Ｚ軸が、それぞれｘ軸、ｙ軸、ｚ軸の向きと一致する姿勢）であるとする。なお、本実施形態では、行列を用いて第１の姿勢を表現することとしたが、他の実施形態においては、第１の姿勢は、３次のベクトルまたは３つの角度によって表現されてもよい。 The first attitude data 68 is data indicating the first attitude calculated using the angular velocity data 63. In the present embodiment, the first posture is expressed by a 3 × 3 matrix M1 shown in the following equation (1).

The matrix M1 is a rotation matrix representing the rotation from a predetermined reference posture to the current input device 8 posture. Hereinafter, the matrix M1 indicating the first posture is referred to as a “first posture matrix M1”. Note that the first posture represented by the first posture matrix M1 is a posture in the xyz coordinate system (the spatial coordinate system) with reference to a predetermined position in the space where the input device 8 exists. Here, the xyz coordinate system is based on the premise that the input device 8 is located in front of the marker unit 6, and the direction from the input device 8 toward the marker unit 6 is the z-axis positive direction, and is vertically upward (in the gravitational direction). It is assumed that the coordinate system is such that the reverse direction) is the y-axis positive direction and the left direction when facing the marker unit 6 from the position of the input device 8 is the x-axis positive direction. Here, the predetermined reference posture is a posture in which the imaging direction of the input device 8 located in front of the marker unit 6 faces the center of the marker unit 6 and the button surface of the controller 5 is vertically upward (that is, input) Assume that the X axis, the Y axis, and the Z axis with respect to the device 8 are postures that coincide with the directions of the x axis, the y axis, and the z axis, respectively. In the present embodiment, the first posture is expressed using a matrix. However, in other embodiments, the first posture may be expressed by a cubic vector or three angles. .

  The acceleration magnitude data 69 is data indicating the magnitude (length) L of the acceleration vector Va1 indicated by the acceleration data 64.

  The correction degree data 70 is data indicating the degree of correction of the first attitude using the second attitude (correction degree A). The correction degree A is a value that can take a range of 0 ≦ A ≦ C1 (C1 is a predetermined constant of 0 <C1 ≦ 1). Although details will be described later, the corrected first posture becomes closer to the second posture as the correction degree A is larger.

  The correction amount vector data 71 is data indicating a vector indicating a correction amount for correcting the first posture (vector v3 shown in FIG. 9; hereinafter referred to as a correction amount vector). The correction amount vector Vg is calculated based on the vector Va2 representing the acceleration vector Va1 in the xyz coordinate system and the correction degree A.

  The correction matrix data 72 is data indicating a rotation matrix (referred to as a correction matrix) Ma used for correcting the first posture. That is, in the first correction process, the first posture is corrected by multiplying the first posture matrix M1 representing the first posture by the correction matrix Ma. The correction matrix Ma is calculated based on the vector Va2 and the correction amount vector Vg.

  The roll posture component data 73 is data indicating a posture component (roll posture component) M3r related to the roll direction among the posture components included in the third posture calculated from the image to be captured. The yaw posture component data 74 is data indicating the posture component (yaw posture component) M3y related to the yaw direction among the posture components included in the third posture, and the pitch posture component data 75 is the third posture information. Among the posture components included in the posture, this is data indicating a posture component (pitch posture component) M3p related to the pitch direction. Note that the roll direction, yaw direction, and pitch direction referred to here are rotation directions when the imaging direction (Z-axis positive direction) of the input device 8 is used as a reference. In the present embodiment, each of the posture components M3r, M3y, and M3p is represented by a 3 × 3 matrix as in the first posture.

  The third attitude data 76 is data indicating a third attitude calculated from the image to be imaged. In the present embodiment, the third posture is represented by a 3 × 3 matrix M3, like the first posture. Hereinafter, the matrix M3 indicating the third posture is referred to as a “third posture matrix M3”. In the present embodiment, since the marker coordinate data is transmitted as operation data from the input device 8, the third posture matrix M <b> 3 is calculated based on the marker coordinate data 65. Specifically, the third posture matrix M3 is obtained by combining the posture components M3r, M3y, and M3p.

  Next, details of processing performed in the game apparatus 3 will be described with reference to FIGS. FIG. 14 is a main flowchart showing the flow of processing executed in the game apparatus 3. When the power of the game apparatus 3 is turned on, the CPU 10 of the game apparatus 3 executes a startup program stored in a boot ROM (not shown), whereby each unit such as the main memory is initialized. Then, the game program stored in the optical disc 4 is read into the main memory, and the CPU 10 starts executing the game program. The flowchart shown in FIG. 14 is a flowchart showing processing performed after the above processing is completed.

  First, in step S1, the CPU 10 executes an initialization process relating to the game. In this initialization process, the values of various parameters used in the game process are initialized, a virtual game space is constructed, and player objects and other objects are placed at initial positions in the game space. Following step S1, the process of step S2 is executed.

  In step S2, the CPU 10 executes an initial posture setting process. Specifically, a predetermined value is set as the initial posture of the input device 8 in response to the player performing a predetermined operation (for example, an operation of pressing the A button 32d). Here, the reference posture is a posture in which the Z axis is parallel to the vertical direction and the imaging direction of the input device 8 faces the center of the marker unit 6 (the center of the markers 6R and 6L). It is desirable to perform the predetermined operation while holding the input device 8 so that the initial posture becomes the reference posture, but if the input device is close to a stationary state and the marker portion can be imaged, It is possible to calculate the initial posture. When a predetermined operation is performed, the CPU 10 stores data indicating a matrix representing the initial posture in the main memory as first posture data. After the above step S2, the processing loop of steps S3 to S8 is repeatedly executed while the game is executed. Note that one processing loop is executed once per frame time (for example, 1/60 seconds).

  In this embodiment, the initial posture setting process (step S2) is executed only once before the game starts (before the processing loop of steps S3 to S8 is executed). In the embodiment, the initial posture setting process may be executed at an arbitrary timing during the game. That is, the CPU 10 may execute the initial posture setting process in response to the player performing the predetermined operation during the game.

  In step S3, the CPU 10 acquires operation data. That is, operation data transmitted from the controller 5 is received via the wireless controller module 19. Then, angular velocity data, acceleration data, marker coordinate data, and operation button data included in the received operation data are stored in the main memory. Following step S3, the process of step S4 is executed.

  In step S4, the CPU 10 calculates the first posture based on the angular velocity data 63 stored in the main memory. Any method may be used to calculate the attitude of the input device 8 from the angular velocity. However, in the present embodiment, the first attitude is the previous first attitude (calculated in the previous processing loop). (First attitude) and the current angular velocity (the angular velocity acquired in the current processing loop). Specifically, the CPU 10 sets the posture obtained by rotating the previous first posture by unit time at the current angular velocity as the first posture. The previous first posture is indicated by the first posture data 68 stored in the main memory, and the current angular velocity is indicated by the angular velocity data 63 stored in the main memory. Data indicating the posture (3 × 3 matrix) calculated in step S4 is newly stored in the main memory as the first posture data 68. Following step S4, the process of step S5 is executed.

  In step S5, the CPU 10 executes the first correction process described above. The first correction process is a process of correcting the first posture using acceleration data. Hereinafter, the first processing example in the first correction processing will be described with reference to FIG. 15, and the second processing example in the first correction processing will be described with reference to FIGS. 16 to 21.

  FIG. 15 is a flowchart showing the flow of the first processing example of the first correction processing (step S5) shown in FIG. In the first correction process, first, in step S <b> 11, the CPU 10 calculates the magnitude L of the acceleration detected by the acceleration sensor 37. That is, the acceleration data 64 stored in the main memory is read, and the magnitude L is calculated for the acceleration vector Va1 indicated by the acceleration data 64. Data indicating the calculated magnitude L is stored as acceleration magnitude data 69 in the main memory. Following step S11, the process of step S12 is executed.

  In step S12, the CPU 10 determines whether or not the magnitude of the acceleration detected by the acceleration sensor 37 is zero. That is, the acceleration magnitude data 69 stored in the main memory is read, and it is determined whether or not the magnitude L indicated by the acceleration magnitude data 69 is zero. If the determination result of step S12 is negative, the process of step S13 is executed. On the other hand, when the determination result of step S12 is affirmative, the processes of subsequent steps S13 to S21 are skipped, and the CPU 10 ends the first correction process. Thus, in this embodiment, when the magnitude of the acceleration detected by the acceleration sensor 37 is 0, correction using the acceleration is not performed. This means that the direction of gravity cannot be calculated from the detection result of the acceleration sensor 37 when the magnitude of the acceleration is 0, and the subsequent steps S13 to S21 are performed if the magnitude of the acceleration vector is 0. The reason is that it is difficult.

  In step S <b> 13, the CPU 10 normalizes the acceleration vector Va <b> 1 detected by the acceleration sensor 37. That is, the acceleration data 64 stored in the main memory is read, and the acceleration vector Va1 indicated by the acceleration data 64 is corrected so that the magnitude is 1. The CPU 10 stores data indicating the normalized acceleration vector Va1 in the main memory. Following step S13, the process of step S14 is executed.

In step S <b> 14, the CPU 10 calculates the correction degree A indicating the degree of correction of the first posture in the first correction process. The correction degree A is calculated based on the magnitude L of the acceleration vector Va1 before normalization. Specifically, the CPU 10 reads acceleration magnitude data 69 stored in the main memory. Then, using the magnitude L indicated by the acceleration magnitude data 69, the correction degree A is calculated according to the following equation (2).
A = | L−1 | (2)
Data indicating the correction degree A calculated by the above equation (2) is stored as correction degree data 70 in the main memory. The correction degree A calculated by the above equation (2) is a value that is not a final value and is being calculated, and the final correction degree A is obtained by converting the value in the subsequent step S16. A value is obtained. Following step S14, the process of step S15 is executed.

  In step S15, the CPU 10 determines whether or not the correction degree A calculated in step S14 is smaller than a predetermined value R. The predetermined value R is determined in advance and is set to 0.4, for example. Here, as described above, in the present embodiment, the magnitude of the gravitational acceleration detected by the acceleration sensor 37 is “1”, and the correction degree A is the magnitude L of the acceleration vector Va1 and “1”. (The above formula (2)). Therefore, the case where the correction degree A is equal to or greater than the predetermined value R is a case where the magnitude L of the acceleration vector Va1 is separated from the magnitude of the gravitational acceleration by the predetermined value R or more. If the determination result of step S15 is affirmative, the process of step S16 is executed. On the other hand, when the determination result of step S15 is negative, the processes of subsequent steps S16 to S21 are skipped, and the CPU 10 ends the first correction process.

  As described above, in the present embodiment, only when the difference between the acceleration magnitude L detected by the acceleration sensor 37 and the gravitational acceleration magnitude (= 1) is smaller than a predetermined reference (predetermined value R). Correction is performed, and correction is not performed when the difference between the magnitude L and the magnitude of gravitational acceleration is equal to or greater than a predetermined reference. Here, in the state where the input device 8 is moved, in addition to the gravitational acceleration, the acceleration due to inertia caused by the movement of the input device 8 is detected by the acceleration sensor 37, so the magnitude of the detected acceleration vector Va1. L becomes a value different from “1”, and when the input device 8 is moved violently, the size L becomes a value far from “1”. Therefore, the case where the difference between the magnitude L and the magnitude of the gravitational acceleration is equal to or greater than a predetermined reference is estimated to be a case where the input device 8 is moved violently. On the other hand, when the input device 8 is moved violently, the acceleration vector Va1 detected by the acceleration sensor 37 contains many components other than gravitational acceleration (the acceleration component due to inertia), and therefore the acceleration vector Va1. The value of is presumed to be unreliable as a value indicating the direction of gravity. Therefore, the determination process of step S15 determines whether or not the input device 8 is moved violently, in other words, whether or not the value of the acceleration vector Va1 is reliable as a value indicating the direction of gravity. It is processing. In the present embodiment, if the value of the acceleration vector Va1 is not reliable as a value indicating the direction of gravity by the determination process in step S15, no correction is performed, and the value of the acceleration vector Va1 is a value indicating the direction of gravity. Corrections are made only when reliable. As a result, it is possible to prevent the first posture from being correctly corrected as a result of performing the correction on the first posture using the acceleration vector Va1 that is not reliable as a value indicating the direction of gravity.

In step S16, the CPU 10 converts the value of the correction degree A. In the present embodiment, the correction degree A is converted so that the correction degree A becomes closer to 1 as the magnitude L of the detected acceleration vector Va1 is closer to the magnitude of gravitational acceleration. Specifically, the CPU 10 reads the correction degree data 70 stored in the main memory, and converts the correction degree A indicated by the correction degree data 70 according to the following equations (3) to (5).
A2 = 1- (A1 / R) (3)
A3 = A2 × A2 (4)
A4 = A3 × C1 (5)
In the above formulas (3) to (5), the variable A1 is the correction degree before conversion (value indicated by the correction degree data 70 currently stored in the main memory), and the variable A4 is finally converted in step S16. Is the degree of correction. The above expression (3) is an expression for obtaining the corrected degree of correction A2 so that the degree of correction A1 before the conversion becomes closer to 1 as the magnitude of the gravitational acceleration (= 1) is closer. The above equation (4) is an equation for obtaining the corrected degree of correction A3 so that a larger weight is assigned as the correction degree A2 before conversion is closer to 1. The above equation (5) is an equation for adjusting the magnitude of the correction amount. That is, the correction amount of the constant C1 increases as the value of the constant C1 increases. The constant C1 is determined in advance and is set in a range of 0 <C1 ≦ 1 (for example, 0.03). Data indicating the correction degree A4 obtained by the conversion according to the above equations (3) to (5) is stored as new correction degree data 70 in the main memory. Following step S16, the process of step S17 is executed.

  In the present embodiment, the conversions according to the above formulas (3) to (5) are performed. However, in other embodiments, some or all of the conversions in the above formulas (3) to (5) are omitted. May be. However, when the conversion of the above equation (3) is omitted, it is necessary to replace the acceleration vector Va2 and the gravity direction vector (0, -1, 0) in equation (7) used in step S18 described later.

In step S17, the CPU 10 converts the acceleration vector Va1 expressed in the XYZ coordinate system into a value Va2 in the xyz coordinate system. The acceleration vector Va2 in the xyz coordinate system is calculated by converting the normalized acceleration vector Va1 using the first posture matrix M1 representing the first posture obtained in the previous frame. That is, the CPU 10 reads the data of the acceleration vector Va1 (normalized) stored in the main memory and the first attitude data 68 in step S13. Then, using the acceleration vector Va1 and the first posture matrix M1 indicated by the first posture data 68, an acceleration vector Va2 in the xyz coordinate system is calculated. More specifically, normalized acceleration vector Va1 = (nx, ny, nz), each element of the first posture matrix M1 is a variable shown in the above equation (1), and acceleration expressed in the xyz coordinate system If the vector Va2 = (vx, vy, vz), the acceleration vector Va2 is calculated according to the following equation (6).
vx = Xx * nx + Yx * ny + Zx * nz
vy = Xy * nx + Yy * ny + Zy * nz
vz = Xz * nx + Yz * ny + Zz * nz (6)
As shown in the above equation (6), the acceleration vector Va2 can be obtained by rotating the acceleration vector Va1 by the first posture matrix M1 that is a rotation matrix. The acceleration vector Va2 calculated in step S17 is stored in the main memory. Following step S17, the process of step S18 is executed.

In step S18, the CPU 10 calculates the correction amount vector Vg using the acceleration vector Va2 expressed in the xyz coordinate system and the correction degree A. The correction amount vector Vg is calculated using the degree of correction after the conversion in step S16 and a vector (0, -1, 0) indicating the vertical downward direction (gravity direction) of the xyz coordinate system. Specifically, the CPU 10 reads out the correction degree data 70 stored in the main memory, and uses the correction degree A indicated by the correction degree data 70, according to the following equation (7), the correction amount vector Vg = ( gx, gy, gz) is calculated.
gx = (0−vx) × A + vx
gy = (− 1−vy) × A + vy
gz = (0−vz) × A + vz (7)
As shown in the above equation (7), the correction amount vector Vg is a line segment connecting the end point of the acceleration vector Va2 to the end point of the gravity direction vector (0, -1, 0) as A: (1-A). It is a vector whose end point is a point to be internally divided. Accordingly, the correction amount vector Vg is closer to the gravity direction vector as the correction degree A is larger. The CPU 10 stores data indicating the correction amount vector Vg calculated by the above equation (7) as correction amount vector data 71 in the main memory. Following step S18, the process of step S19 is executed.

  In step S19, the CPU 10 normalizes the correction amount vector Vg calculated in step S18. That is, the correction amount vector data 71 stored in the main memory is read, and the vector indicated by the correction amount vector data 71 is normalized. Then, the data indicating the normalized vector is stored as new correction amount vector data 71 in the main memory. The correction amount vector Vg calculated in step S19 corresponds to the vector v3 shown in FIG. Following step S19, the process of step S20 is executed.

  In step S20, the CPU 10 calculates a correction matrix Ma for correcting the first posture. The correction matrix Ma is calculated based on the acceleration vector Va2 expressed in the xyz coordinate system and the correction amount vector Vg normalized in step S19. Specifically, the CPU 10 reads the acceleration vector Va2 and the correction amount vector data 71 stored in the main memory in step S17. Then, a rotation matrix for rotating the acceleration vector Va2 so as to coincide with the correction amount vector Vg is calculated, and the calculated rotation matrix is set as a correction matrix Ma. That is, the correction matrix Ma is a rotation matrix that performs the rotation of the angle Δθ shown in FIG. Data indicating the correction matrix Ma calculated in step S20 is stored in the main memory as correction matrix data 72. Following step S20, the process of step S21 is executed.

  In step S21, the CPU 10 corrects the first posture matrix M1 indicating the first posture with the correction matrix Ma. Specifically, the CPU 10 reads the first attitude data 68 and the correction matrix data 72 stored in the main memory. Then, the first attitude matrix M1 indicated by the first attitude data 68 is converted by the correction matrix Ma indicated by the correction matrix data 72 (the product of the first attitude matrix M1 and the correction matrix Ma is calculated). The converted first posture matrix M1 indicates the corrected first posture. That is, the process of step S21 corresponds to the process of rotating the vector v1 shown in FIG. 10 by the angle Δθ. The CPU 10 stores data indicating the converted first attitude matrix M1 as new first attitude data 68 in the main memory. After step S21, the CPU 10 ends the first correction process.

  As described above, in the first correction process, the correction amount vector Vg is calculated between the acceleration vector detected by the acceleration sensor 37 and the gravity direction vector (vector G shown in FIG. 8) (steps S18 and S19). ), The first posture is corrected by the correction amount (correction matrix Ma. Angle Δθ shown in FIG. 9) represented by the correction amount vector Vg (step S21). As a result, the first attitude calculated from the gyro sensors 55 and 56 (vector v1 or angle θ1 shown in FIG. 8) is corrected to approach the second attitude (angle θ2 shown in FIG. 8) determined by the acceleration sensor 37. It can be performed. By performing such correction, the first posture can be corrected so as to have a more accurate value.

  In the first correction process, as the acceleration vector Va1 is reliable (the smaller the difference between the magnitude L of the acceleration vector Va1 and the magnitude of gravitational acceleration), the value of the correction degree A increases. The first posture is corrected so as to be closer to the second posture. That is, the more reliable the acceleration vector Va1, the larger the correction amount, and the second posture is strongly reflected in the corrected first posture. As described above, according to the present embodiment, the correction amount in the first correction process is determined according to the reliability of the acceleration sensor vector Va1, and therefore the correction amount can be appropriately determined according to the reliability. The attitude of the input device 8 can be calculated more accurately.

  In the present embodiment, the correction amount vector Vg calculated in step S18 is a point that internally divides the line segment connecting the end point of the acceleration vector Va2 to the end point of the gravity direction vector into A: (1-A). Is the vector that approaches the gravity direction vector as the value of the correction degree A increases. In another embodiment, depending on the method of calculating the correction degree A, the correction amount vector Vg is a point that internally divides the end point of the gravity direction vector and the end point of the acceleration vector Va2 into (1-A): A. The end point vector may be a vector that approaches the gravity direction vector as the correction degree A is smaller. At this time, in step S20, a rotation matrix for rotating the correction amount vector Vg so as to coincide with the direction of gravity may be calculated, and the calculated rotation matrix may be used as the correction matrix Ma. This also makes it possible to perform correction as in the present embodiment.

  Next, a second processing example of the first correction processing will be described. In the present embodiment, the game apparatus 3 may adopt the first processing example as the first correction processing, or may adopt the following second processing example.

  As described above, in the present embodiment, when the direction of the acceleration vector is vertically downward, the posture of the input device 8 is set as the second posture, and correction is performed so that the first posture approaches the second posture. (FIGS. 8 to 10, FIG. 15). That is, in order to bring the first posture closer to (or match) the second posture, correction is performed to rotate the first posture so that the acceleration vector and the gravity direction vector are brought closer (matched). . Here, as an example of a method of rotating the first posture, referring to FIGS. 16 and 17, a method of simply rotating the acceleration vector to match the gravity direction vector, that is, rotating the acceleration vector at the shortest distance Let us consider a method of rotating the first posture using a rotation matrix that matches the gravity direction vector.

  FIG. 16 is a diagram illustrating vectors representing the gravity direction, acceleration, and posture in the spatial coordinate system. In FIG. 16, the gravity vector G (= (0, -1, 0)) represents the direction of gravity (vertically downward), and the Z-axis vector M1Z is an input device when the input device 8 is in the first posture. 8 represents the direction of the Z axis, and an acceleration vector Va (corresponding to the vector Va2) represents an acceleration applied to the input device 8. FIG. 17 is a diagram illustrating a state in which the first posture (Z-axis vector M1Z) is rotated by the method of rotating the acceleration vector Va at the shortest distance from the state illustrated in FIG. In the case shown in FIG. 16, when the first posture (Z-axis vector M1Z) is rotated by the above method, as shown in FIG. 17, the Z-axis vector M1Z largely changes before and after correction. As described above, in the method of simply matching the acceleration vector with the gravity direction vector, the predetermined axis (the Z axis in FIG. 16) representing the attitude of the input device 8 may greatly change before and after correction.

  The fact that the predetermined axis representing the posture of the input device 8 changes greatly before and after the correction becomes a problem when, for example, the posture of the object in the virtual space is controlled according to the posture of the input device 8. As an example, consider a game process in which a sword object can be shaken by an operation of shaking the input device 8 by changing the posture of the sword object in the virtual game space according to the posture of the input device 8. In such a game process, for example, if the longitudinal direction of the sword object corresponds to the Z-axis direction of the input device 8, the Z-axis of the input device 8 changes greatly due to the correction so that the longitudinal direction of the sword object The direction will change greatly. Thus, when the longitudinal direction of the object changes greatly, the player feels uncomfortable with the operation (even if the corrected posture is accurate). Therefore, it is preferable to minimize the change due to the correction of the axis corresponding to the longitudinal direction of the object among the axes representing the attitude of the input device 8. Even if the axis is not the axis corresponding to the longitudinal direction of the object, the axis that the player pays more attention to than the other axis (for example, the axis having a more important meaning in the game than the other axis) is changed by the correction. It is preferable to make it as small as possible.

  Therefore, in the second processing example of the first correction processing of the present embodiment, the first posture is corrected so that the change before and after correction is as small as possible with respect to the predetermined axis representing the posture of the input device 8. Hereinafter, a second processing example of the first correction processing will be described with reference to FIGS. In the following description, the case where the predetermined axis that minimizes the change before and after correction is the Z axis of the input device 8 will be described as an example.

  FIG. 18 is a flowchart showing the flow of the second processing example of the first correction processing (step S5) shown in FIG. In the second processing example of the first correction process, first, in step S <b> 11, the CPU 10 calculates the magnitude L of the acceleration detected by the acceleration sensor 37. In step S30, the CPU 10 calculates the difference between the magnitude of the acceleration detected by the acceleration sensor 37 and the magnitude of the gravitational acceleration (here, 1), and the magnitude of the difference is based on the predetermined value K. It is determined whether it is small. That is, when the magnitude of acceleration deviates significantly from the magnitude of gravitational acceleration, at least the posture of the input device 8 (posture estimated from acceleration) is not stable, and the acceleration direction is regarded as gravitational acceleration. Therefore, correction processing is performed only when the magnitude of acceleration is close to the magnitude of gravitational acceleration. The reason for performing step S30 is the same reason as step S15 in the first processing example. If the determination result of step S30 is affirmative, the process of step S31 is executed. On the other hand, when the determination result of step S30 is negative, the processes of subsequent steps S31 to S37 are skipped, and the CPU 10 ends the first correction process.

  In step S31, the CPU 10 calculates a projection acceleration vector Vap. FIG. 19 is a diagram showing a projected acceleration vector and a projected gravity vector (described later). As shown in FIG. 19, the projected acceleration vector Vap is obtained by projecting the acceleration vector Va onto a plane XY perpendicular to the Z axis (Z axis vector M1Z) of the input device 8 in the spatial coordinate system (xyz coordinate system). Is a vector.

The projected acceleration vector Vap can be calculated using the first posture matrix M1 and the acceleration vector (acceleration vector indicated by the acceleration data 64) Va1 detected by the acceleration sensor 37. Specifically, in step S31, the CPU 10 reads the acceleration data 64 and the first attitude data 68 stored in the main memory. Then, using the acceleration vector Va1 indicated by the acceleration data 64 and the first attitude matrix M1 indicated by the first attitude data 68, the projection acceleration vector Vap = (Vax, Vay, Vaz) is calculated according to the following equation (8). To do. That is, if the acceleration vector Va1 = (VX, VY, VZ), each element of the first posture matrix M1 is a variable shown in the above equation (1), and the projection acceleration vector Vap = (Vax, Vay, Vaz), the projection The acceleration vector Vap is calculated according to the following equation (8).
Vax = Xx / VX + Yx / VY
Vay = Xy / VX + Yy / VY
Vaz = Xz · VX + Yz · VY (8)
The above equation (8) means a process of converting the XY component of the acceleration vector Va1 expressed in the controller coordinate system (XYZ coordinate system) into the xyz coordinate system (rotating with the first attitude matrix M1). The projected acceleration vector Vap is a vector on the XY plane expressed in the xyz coordinate system. Therefore, the projected acceleration vector Vap can be obtained by converting a vector (VX, VY, 0) obtained by removing the Z component from the acceleration vector Va1 into the xyz coordinate system as shown in the above equation (8). The CPU 10 normalizes the projection acceleration vector Vap calculated by the above equation (8), and data indicating the normalized projection acceleration vector Vap (projection acceleration data) is stored in the main memory. Following step S31, the process of step S32 is executed.

  In step S32, the CPU 10 calculates a projected gravity vector Gp. As shown in FIG. 19, the projected gravity vector Gp is obtained by projecting the gravity vector G onto the plane XY perpendicular to the Z axis (Z axis vector M1Z) of the input device 8 in the spatial coordinate system (xyz coordinate system). Is a vector.

The projected gravity vector Gp can be calculated by the same method as the projected acceleration vector Vap. That is, the projected gravity vector Gp can be calculated using the first posture matrix M1 and the gravity vector. Here, the gravity vector G = (0, −1, 0) in the spatial coordinate system (xyz coordinate system) can be expressed as (−Xy, −Yy, −Zy) in the controller coordinate system (XYZ coordinate system). . As described above, in step S31, the CPU 10 reads the first attitude data 68 stored in the main memory. Then, using the first posture matrix M1 indicated by the first posture data 68 and the gravity vector G = (− Xy, −Yy, −Zy) in the XYZ coordinate system, the projected gravity vector according to the following equation (9). Gp = (Gx, Gy, Gz) is calculated.
Gx = −Xx · Xy−Yx · Yy
Gy = -Xy.Xy-Yy.Yy
Gz = −Xz · Xy−Yz · Yy (9)
The projected gravity vector Vp can be obtained by converting a vector obtained by removing the Z component from the gravity vector G in the XYZ coordinate system into the xyz coordinate system based on the same idea as the above equation (8). Therefore, the projected gravity vector Vp can be obtained by rotating the XY component of the gravity vector G expressed in the XYZ coordinate system by the first posture matrix M1 as in the above equation (9). The CPU 10 normalizes the projected gravity vector Gp calculated by the above equation (9), and data indicating the normalized projected gravity vector Gp (projected gravity data) is stored in the main memory. Following step S32, the process of step S33 is executed.

  In step S33, the CPU 10 calculates a first transformation matrix mtx1 for performing the first transformation on the first posture. The first transformation matrix mtx1 is a rotation matrix that rotates around the Z axis so that the projected acceleration vector Vap matches the projected gravity vector Gp (see the arrow shown in FIG. 19). Specifically, the CPU 10 reads the projection acceleration data and the projection gravity data stored in the main memory. Then, a rotation matrix for rotating the projection acceleration vector Vap so as to coincide with the projection gravity vector Gp is calculated, and the calculated rotation matrix is set as a first conversion matrix mtx1. Data (first conversion matrix data) indicating the first conversion matrix mtx1 calculated in step S33 is stored in the main memory. Following step S33, the process of step S34 is executed.

  In step S34, the CPU 10 converts the first posture matrix M1 indicating the first posture with the first conversion matrix mtx1. Specifically, the CPU 10 reads the first attitude data 68 and the first transformation matrix data stored in the main memory. Then, the first attitude matrix M1 indicated by the first attitude data 68 is converted by the first conversion matrix mtx1 indicated by the first conversion matrix data (the first conversion matrix from the right side with respect to the first attitude matrix M1). multiply by mtx1). As a result, the first conversion is performed on the first posture. The CPU 10 stores data indicating the converted first attitude matrix M1 as new first attitude data 68 in the main memory. Following step S34, the process of step S35 is executed.

  In step S34, the first posture is transformed by the first transformation matrix. FIG. 20 is a diagram illustrating a state in which the first conversion is performed from the state illustrated in FIG. 19. Here, since the first transformation by the first transformation matrix is rotation around the Z axis, as shown in FIG. 20, the Z axis vector M1Z representing the Z axis of the input device 8 is oriented depending on the first transformation. Does not change. However, although not shown in FIG. 20, the X axis and the Y axis of the input device 8 are rotated around the Z axis by the first conversion, and the first posture is changed.

In step S35, the CPU 10 corrects the acceleration vector Va so as to indicate the acceleration corresponding to the attitude of the input device 8 after being converted in step S34 (applied to the input device 8). The corrected acceleration vector Va ′ can be calculated using the first posture (first posture matrix M1) after being converted in step S34 and the acceleration vector Va1 in the XYZ coordinate system. Specifically, the CPU 10 reads the first attitude data 68 and the acceleration data 64 stored in the main memory, and uses the first attitude matrix M1 and the acceleration vector Va1 to make a correction according to the following equation (10). An acceleration vector Va ′ = (Cx, Cy, Cz) is calculated.
Cx = Xx / VX + Yx / VY + Zx / VZ
Cy = Xy / VX + Yy / VY + Zy / VZ
Cz = Xz.VX + Yz.VY + Zz.VZ (10)
As shown in the above equation (10), the corrected acceleration vector Va ′ is obtained by rotating the acceleration vector Va1 in the XYZ coordinate system by the first posture matrix M1 after being converted in step S34. Can do. As shown in FIG. 20, the corrected acceleration vector Va ′ is obtained by rotating the acceleration vector Va2 in the xyz coordinate system by the first conversion matrix mtx1. That is, the corrected acceleration vector Va ′ can be calculated using the acceleration vector Va2 and the first conversion matrix mtx1. In the second processing example, in order to omit the process of calculating the acceleration vector Va2, the vector Va ′ is calculated by the above equation (10) using the acceleration vector Va1. The CPU 10 normalizes the acceleration vector Va ′ calculated by the above equation (10), and stores data (corrected acceleration data) indicating the normalized acceleration vector Va ′ in the main memory. Following step S35, the process of step S36 is executed.

  In step S36, the CPU 10 calculates a second conversion matrix mtx2 for performing the second conversion on the first posture. FIG. 21 is a diagram illustrating a state in which the second conversion is performed from the state illustrated in FIG. 20. The second transformation matrix mtx2 is a rotation matrix that rotates so that the corrected acceleration vector Va 'in step S35 matches the gravity vector G (see the arrow shown in FIG. 20). Specifically, the CPU 10 reads the corrected acceleration data stored in the main memory. Then, a rotation matrix for rotating the corrected acceleration vector Va ′ so as to coincide with the gravity vector G = (0, −1,0) is calculated, and the calculated rotation matrix is set as a second transformation matrix mtx2. Data indicating the second transformation matrix mtx2 calculated in step S36 (second transformation matrix data) is stored in the main memory. Following step S36, the process of step S37 is executed.

  In step S37, the CPU 10 converts the first posture matrix M1 indicating the first posture with the second conversion matrix mtx2. Specifically, the CPU 10 reads out the first attitude data 68 and the second conversion matrix data stored in the main memory. Then, the first posture matrix M1 indicated by the first posture data 68 is transformed by the second transformation matrix mtx2 indicated by the second transformation matrix data (the product of the first posture matrix M1 and the second transformation matrix mtx2). Is calculated). As a result, the second conversion is performed on the first posture. At this time, as shown in FIG. 21, the direction of the Z-axis vector M1Z is changed by the second transformation. The CPU 10 stores data indicating the converted first attitude matrix M1 as new first attitude data 68 in the main memory. After step S37, the CPU 10 ends the first correction process.

  As described above, according to the second processing example of the first correction process, first, the direction in which the acceleration vector Va is projected onto the plane XY perpendicular to the Z axis with respect to the first posture (projected acceleration vector Vap). And the first transformation that performs rotation about the Z axis to match the direction (projected gravity vector Gp) in which the vertically downward direction (gravity direction) is projected onto the plane XY (step S34, FIG. 20). Further, a second conversion is performed for the first posture, in which rotation is performed so that the direction of the acceleration vector (acceleration vector Va ′) subjected to the first conversion matches the vertical downward direction (step). S37, FIG. 21). As described above, by performing the second conversion after the first conversion, the change in the Z-axis before and after the correction is minimized. Therefore, according to the second processing example, the change in the Z-axis can be reduced as compared with the method (FIG. 17) in which the acceleration vector is simply rotated (at the shortest distance) to coincide with the gravity direction vector. According to this, when the posture of the object is operated by the posture of the input device 8 and the Z axis of the input device 8 is made to correspond to the longitudinal direction of the object in the virtual space, for example, the correction by the first correction process is performed. Even if it is done, the longitudinal direction of the object does not change much. Therefore, the correction can be made inconspicuous for the player (the player can hardly notice the correction), and the player can be prevented from feeling uncomfortable with the correction, thereby providing a game operation with good operability. can do.

  In the above embodiment, the CPU 10 separately executes the first conversion process (step S34) and the second conversion process (step S35). However, in other embodiments, the first and second conversion processes are performed. These conversion processes may be executed together in a single process. That is, in the process illustrated in FIG. 18, instead of executing the process of step S34, the CPU 10 combines the first conversion matrix mtx1 and the second conversion matrix mtx2 in step S37, and the first rotation matrix is combined with the first rotation matrix. The posture may be rotated. Even in this case, the posture of the input device 8 can be corrected by the conversion including the first conversion and the second conversion, and the same effect as in the above embodiment can be obtained.

  Further, in the second processing example, by the conversion composed of the first and second conversions, the change in the orientation before and after correction is minimized with respect to the predetermined axis (Z axis) representing the attitude of the input device 8. The first posture was corrected. Here, in another embodiment, the CPU 10 only needs to minimize the change in orientation before and after correction with respect to a predetermined axis representing the attitude of the input device 8. For example, the CPU 10 corrects the first attitude by the following process. You may make it do. That is, the CPU 10 first rotates the first posture by using the rotation matrix that rotates the acceleration vector by the shortest distance and matches the gravity direction vector (FIG. 17), and then the orientation of the predetermined axis before and after the correction. The first posture may be rotated about the vertically downward direction so that the change is minimized. Also by this, the same effect as the above embodiment can be obtained.

  Further, in the second processing example, correction is performed so that the acceleration vector Va and the gravity vector G coincide with each other without using the correction degree A used in the first processing example. Here, in another embodiment, the correction degree A may be used also in the second processing example. Specifically, in step S33, the CPU 10 calculates a vector whose end point is a point that internally divides the line segment connecting the end point of the projected acceleration vector Vap to the end point of the projected gravity vector Gp into A: (1-A). Then, a rotation matrix that rotates around the Z axis so as to make the projected acceleration vector Vap coincide with the vector may be calculated as the first conversion matrix. In step S36, a vector whose end point is a point dividing the line segment connecting the end point of the corrected acceleration vector Va ′ to the end point of the gravity vector G into A: (1-A) is calculated, and the acceleration What is necessary is just to calculate the rotation matrix which rotates so that vector Va 'may correspond with the said vector as said 2nd conversion matrix. When the correction degree A is used in the second processing example, the same value may be used as the correction degree A for the correction by the first conversion and the correction by the second conversion, or different values may be used. .

  In the second processing example, if the XY plane and the acceleration vector Va are perpendicular to each other, or if the XY plane and the gravity direction vector G are perpendicular to each other, the correction may not be performed correctly. . This is because the projected acceleration vector Vap or the projected gravity vector Gp becomes zero. Therefore, in the above case, the CPU 10 may end the first correction process without executing the correction according to the second process example (that is, the first correction process is not executed). Alternatively, in the above case, the CPU 10 may execute correction according to the first processing example.

  Returning to the description of FIG. 14, in step S <b> 6 subsequent to step S <b> 5, the CPU 10 executes the second correction process described above. The second correction process is a process of correcting the first posture using the marker coordinate data. Hereinafter, the details of the second correction process will be described with reference to FIG.

  FIG. 22 is a flowchart showing the flow of the second correction process (step S6) shown in FIG. In the first correction process, first, in step S <b> 41, the CPU 10 determines whether or not the marker unit 6 is imaged by the imaging unit (imaging device 40) of the input device 8. The determination in step S41 can be made by referring to the marker coordinate data 65 stored in the main memory. Here, when the marker coordinate data 65 indicates two marker coordinates, it is determined that the marker unit 6 is captured, and when the marker coordinate data 65 indicates only one marker coordinate, or there is no marker coordinate. When it shows, it determines with the marker part 6 not being imaged. If the determination result of step S41 is affirmative, the processes of subsequent steps S42 to S47 are executed. On the other hand, when the determination result of step S41 is negative, the processes of the subsequent steps S42 to S47 are skipped, and the CPU 10 ends the second correction process. As described above, when the marker unit 6 is not imaged by the image sensor 40, the attitude of the input device 8 cannot be calculated using data obtained from the image sensor 40. In this case, the second correction is performed. No correction is performed in the process.

  In step S42, the CPU 10 calculates a roll posture component M3r based on the marker coordinate data. The roll posture component M3r is calculated based on the orientation of the marker unit 6 in the captured image, that is, based on the slope of the line connecting the two marker coordinates indicated by the marker coordinate data 65. Hereinafter, an example of a method for calculating the roll posture component M3r will be described with reference to FIG.

  FIG. 23 is a diagram illustrating two-dimensional coordinates corresponding to a captured image. As shown in FIG. 23, in this embodiment, a two-dimensional coordinate system (x′y ′ coordinate system) for representing a position in a captured image has a range of the captured image of −1 ≦ x ′ ≦ 1, − 1 ≦ y ′ ≦ 1. The x′y ′ coordinate system captures images when the input device 8 is in a reference posture (an orientation in which the imaging direction of the input device 8 faces the center of the marker unit 6 and the button surface of the controller 5 is vertically upward). In the image, it is assumed that the vertically downward direction is the positive direction of the y ′ axis and the right direction is the positive direction of the x ′ axis. Further, point P1 and point P2 shown in FIG. 23 indicate the positions of the marker coordinates, and point P3 is a midpoint between point P1 and point P2. A vector v10 shown in FIG. 23 is a vector having a point P1 as a start point and a point P2 as an end point.

上式（１１）によって算出された行列を示すデータは、ロール姿勢成分データ７３としてメインメモリに記憶される。ステップＳ４２の次にステップＳ４３の処理が実行される。 In order to calculate the roll posture component M3r, the CPU 10 first reads the marker coordinate data 65, and calculates the vector v10 from the two marker coordinates indicated by the marker coordinate data 65. Further, a vector (hx, hy) obtained by normalizing the vector v10 is calculated. The vector (hx, hy) is directed in the positive x-axis direction when the input device 8 is in the reference posture, and the direction changes according to the rotation of the input device 8 in the roll direction. Since the vector (hx, hy) corresponds to the posture in the roll direction, the roll posture component M3r can be calculated based on the vector (hx, hy). Specifically, the CPU 10 calculates the roll posture component M3r according to the following equation (11).

Data indicating the matrix calculated by the above equation (11) is stored in the main memory as roll posture component data 73. Following step S42, the process of step S43 is executed.

  In step S43, the CPU 10 calculates a yaw posture component M3y based on the marker coordinate data. The yaw posture component M3y is calculated based on the orientation and position of the marker unit 6 in the captured image. Hereinafter, an example of a method for calculating the yaw posture component M3y will be described with reference to FIG.

  First, the CPU 10 reads the marker coordinate data 65 and calculates the midpoint between the two marker coordinates indicated by the marker coordinate data 65. In the present embodiment, the position of the middle point is used as the position of the marker unit 6. Further, the CPU 10 rotates the calculated midpoint coordinates by the rotation angle related to the roll direction of the input device 8 (in the direction opposite to the rotation direction of the input device 8) around the origin of the x′y ′ coordinate system. Calculated coordinates (px, py) are calculated. In other words, the coordinates of the midpoint are rotated around the origin so that the vector (hx, hy) faces the positive x-axis direction. If the input device 8 is at the same position as the marker unit 6 in the horizontal direction (x-axis direction) (that is, the front position of the marker unit 6), the coordinates (px, py) can be used to calculate the attitude related to the yaw direction.

Next, the CPU 10 based on the coordinates (px, py) after the rotation of the midpoint and the angle (limit angle) θy ′ in the yaw direction when the marker unit 6 is located at the end in the x′-axis direction. Then, the rotation angle θy related to the yaw direction is calculated. Here, the limit angle θy ′ and the x-coordinate value px ′ after the rotation of the midpoint when the limit angle θy1 is reached can be obtained in advance. Therefore, the rotation angle θy related to the yaw direction can be calculated using the fact that the ratio between px and px ′ is equal to the ratio between θy and θy ′. Specifically, the rotation angle θy related to the yaw direction can be calculated by the following equation (12).
θy = px × θy ′ / px ′ (12)
When the horizontal length of the marker unit 6 is ignored, the limit angle θy ′ can be set to ½ of the angle of view of the controller 5 and the value of px ′ can be set to “1”.

上式（１３）によって算出された行列を示すデータが、ヨー姿勢成分データ７４としてメインメモリに記憶される。以上のステップＳ４３の次にステップＳ４４の処理が実行される。 Finally, the CPU 10 calculates a rotation matrix that rotates the angle θy calculated by the above equation (12) as the yaw attitude component M3y. Specifically, the yaw posture component M3y is calculated by the following equation (13).

Data indicating the matrix calculated by the above equation (13) is stored in the main memory as yaw attitude component data 74. Following step S43, the process of step S44 is executed.

  In step S44, the CPU 10 combines the roll posture component M3r and the yaw posture component M3y. That is, the roll attitude component data 73 and the yaw attitude component data 74 are read from the main memory, and the roll attitude component M3r and the yaw attitude component M3y indicated by the data 73 and 74 are integrated. Following step S44, the process of step S45 is executed.

  In step S45, the CPU 10 calculates a pitch posture component M3p based on the first posture. Although different from the processing of the present embodiment, the pitch attitude component M3p can also be calculated based on the y coordinate value of the coordinates (px, py) in the same manner as the yaw attitude component M3y. . However, in the method of calculating the attitude in the yaw direction (pitch direction) using the coordinates (px, py), the input device 8 is at the same position as the marker unit 6 in the horizontal direction (vertical direction in the case of the pitch direction). This method is based on the assumption. In the game system 1 of the present embodiment, it is considered that the player operates the input device 8 at a position substantially in front of the marker unit 6 (TV 2) in the horizontal direction. It is possible to calculate the attitude in the yaw direction by the method of step S43 on the assumption that “they are at the same position”. On the other hand, it is conceivable that the player stands and operates the input device 8 or sits down and operates the input device 8, and the position of the marker unit 6 is also arranged on the upper side of the screen of the television 2. It is also possible to arrange them in For this reason, in the game system 1 of the present embodiment, it is not necessarily assumed that “the input device 8 is in the same position as the marker unit 6 in the vertical direction”. The posture may not be calculated.

上式（１５）によって算出された行列を示すデータが、ピッチ姿勢成分データ７５としてメインメモリに記憶される。以上のステップＳ４５の次にステップＳ４６の処理が実行される。 Therefore, in the present embodiment, the first posture is used as it is for the pitch posture component M3p (thus, in the second correction process, the pitch direction is not corrected). Specifically, the CPU 10 reads the first attitude data 68 from the main memory. Then, using each element of the first posture matrix M1 indicated by the first posture data 68, the rotation angle θp with respect to the pitch direction is calculated according to the following equation (14).
cos (θp) = (Zx × Zx + Zz × Zz) ^1/2
sin (θp) = Zy (14)
Variables Zx, Zy, and Zz in the above equation (14) are elements of the first posture matrix M1 shown in the above equation (1). The first posture matrix M1 used here is the first posture matrix M1 after the first correction processing is performed in the current processing loop. Further, the CPU 10 calculates a matrix of the pitch posture component M3p according to the following equation (15) using cos (θp) and sin (θp) calculated by the above equation (14).

Data indicating the matrix calculated by the above equation (15) is stored in the main memory as pitch attitude component data 75. Following step S45, the process of step S46 is executed.

  In step S46, the CPU 10 calculates a third posture based on the posture components in the roll direction, the yaw direction, and the pitch direction. The third attitude is obtained by further synthesizing the pitch attitude component M3p with the synthesis result of the roll attitude component M3r and the yaw attitude component M3y. Specifically, the CPU 10 reads the pitch attitude component data 75 from the main memory, and integrates the pitch attitude component M3p indicated by the pitch attitude component data 75 in the matrix calculated in step S44. Data indicating the calculated matrix is stored in the main memory as third attitude data 76. Following step S46, the process of step S47 is executed.

In step S47, the CPU 10 corrects the first posture using the third posture. The correction in step S47 is performed by bringing the first posture matrix M1 closer to the third posture matrix M3 at a predetermined ratio (constant C2 below). The CPU 10 reads the first attitude data 68 and the third attitude data 76 from the main memory. Then, using the first attitude matrix M1 indicated by the first attitude data 68 and the third attitude matrix M3 indicated by the third attitude data 76, correction is performed according to the following equation (16).
M1 = (M3-M1 ′) × C2 + M1 ′ (16)
In the above equation (16), the variable M1 ′ is the first posture matrix before correction. The constant C2 is preset in the range of 0 <C2 ≦ 1, and is set to 0.1, for example. Data indicating the corrected first posture matrix M1 calculated by the above equation (16) is stored in the main memory as new first posture data 68. After step S47, the CPU 10 ends the second correction process.

  As described above, in the second correction process, the third posture is calculated from the captured image (marker coordinates), and the first posture is corrected so as to approach the third posture. By this correction, the first posture can be corrected so as to have a more accurate value. In the present embodiment, the third posture is calculated from the captured image only for the posture in the roll direction and the yaw direction. However, as described above, the third posture is calculated from the captured image also in the pitch direction. In other embodiments, the third orientation may be calculated from the captured image in the roll direction, the yaw direction, and the pitch direction. In the second correction process, the third posture may be calculated in at least one of the roll direction, the yaw direction, and the pitch direction. In particular, when the second processing example is adopted in the first correction process (step S7), the second correction process calculates the third posture only in the roll direction, and the first posture only in the roll direction. May be corrected (rotated). In other words, if the correction is made in the yaw direction and the pitch direction, the direction of the Z axis changes, and the change in the direction of the Z axis before and after the correction may become large, so the direction of the Z axis does not change much. When the second processing example is adopted for the purpose, only the correction relating to the roll may be performed.

  Returning to the description of FIG. 14, in step S <b> 7 following step S <b> 6, the CPU 10 executes a game process using the corrected first posture. This game process may be any process as long as the process reflects the first attitude matrix M1 representing the corrected first attitude as an input value in the game result. For example, it may be a process of controlling and displaying an object in the virtual game space so as to have the posture indicated by the first posture matrix M1, or the posture indicated by the first posture matrix M1 and a predetermined posture. It may be a process of controlling and displaying the object so as to move at a speed according to the angle. Following step S7, the process of step S8 is executed.

  In step S8, the CPU 10 determines whether or not to end the game. The determination in step S8 is made based on, for example, whether or not the game has been cleared, whether or not the game is over, and whether or not the player has given an instruction to stop the game. If the determination result of step S8 is negative, the process of step S3 is executed again. Thereafter, the processing loop of steps S3 to S8 is repeatedly executed until it is determined in step S8 that the game is to be ended. On the other hand, if the determination result of step S8 is affirmative, the CPU 10 ends the game process shown in FIG. This is the end of the description of the game process.

  As described above, in the present embodiment, the first posture of the input device 8 is calculated from the angular velocities detected by the gyro sensors 55 and 56 (step S4), and the first posture is subjected to the first correction process (S5). And it corrected by the 2nd correction process (S6). Since the game process is executed using the corrected first attitude (step S7), the CPU 10 can execute the game process based on the accurate attitude of the input device 8. According to this, for example, the posture of the input device 8 can be accurately reflected on the posture of the object in the game space, so that the operability of the game operation can be improved.

[Modification]
In the above embodiment, the case where a three-dimensional posture is calculated using a gyro sensor that detects angular velocities around three axes has been described as an example. However, as illustrated in FIGS. The present invention can also be applied when calculating a posture (rotation angle) on a two-dimensional plane. The posture on the two-dimensional plane may be calculated by detecting an angular velocity about two axes with a two-axis gyro sensor, or calculated by detecting an angular velocity about a predetermined axis with a one-axis gyro sensor. May be.

  In another embodiment, the second correction process may be executed only when it is estimated that the input device 8 is imaging the marker unit 6. Specifically, before executing the second correction process, the CPU 10 determines whether or not the input device 8 (imaging unit) faces a direction in which the marker unit 6 can be imaged. This determination can be performed using the first posture or the second posture. For example, the first posture (or the second posture) may be determined as to whether the imaging direction of the input device 8 is the same or opposite to the direction from the input device 8 to the marker unit 6. In addition, the first posture used for the above determination may be the first posture in which the first and second correction processes are performed in the previous processing loop, or may be calculated in the current processing loop. The first posture in which one correction process is performed may be used.

  As a result of the above determination, when it is determined that the input device 8 faces the direction in which the marker unit 6 can be imaged, the CPU 10 performs the second correction process and faces the direction in which the marker unit 6 can be imaged. If it is determined that there is not, the second correction process is skipped. Note that an object that is not the marker unit 6 (for example, an electric lamp in a room or sunlight outside a window) may be erroneously detected as the marker unit 6, and the third posture is obtained using the marker coordinates obtained by the erroneous detection. When is calculated, correct correction cannot be performed even if the second correction process is performed using the third posture. On the other hand, by performing the above determination process, it is possible to prevent the second correction process from being performed using the third posture calculated from the marker coordinates obtained by the erroneous detection. As a result, the second correction process can be performed more accurately.

  As described above, the present invention is used, for example, as a game device or a game program for performing game processing according to the posture of the input device, for example, for accurately calculating the posture of the input device using a gyro sensor. It is possible.

DESCRIPTION OF SYMBOLS 1 Game system 2 Television 3 Game device 4 Optical disk 5 Controller 6 Marker part 7 Gyro sensor unit 8 Input device 10 CPU
11c GPU
11e Internal main memory 12 External main memory 63 Angular velocity data 64 Acceleration data 65 Marker coordinate data 68 First attitude data 76 Third attitude data

Claims (18)

An attitude calculation apparatus that acquires data from an input device including at least an angular velocity sensor and an acceleration sensor, and calculates an attitude of the input device in a three-dimensional space,
Posture calculating means for acquiring an angular velocity detected by the angular velocity sensor and calculating a new posture by rotating the previous posture in accordance with the acquired angular velocity ;
Acceleration vector calculating means for calculating an acceleration vector indicating the acceleration of the input device based on acceleration data from the acceleration sensor;
First correction means for correcting the orientation of the input device in response to a change in the direction of the acceleration vector so that the direction of the acceleration vector in the space and the vertical downward direction in the space are brought close to or coincide with each other;
Wherein the first correction means, with respect to the posture of the input device, is set for the input device, the change in the uncorrected orientation as the corrected orientation of the predetermined axis you correspond to the attitude of the input device are most An attitude calculation device that corrects the attitude of the input device so as to be smaller.   The first correction means performs a rotation around the predetermined axis so that a direction in which the acceleration vector is projected onto a plane perpendicular to the predetermined axis is close to a direction in which the vertical downward direction is projected onto the plane. 2. The attitude of the input device is corrected by a conversion comprising: a second conversion that performs a rotation that brings the direction of the acceleration vector subjected to the first conversion and the vertical downward direction closer to each other. The posture calculation device described in 1.   The posture calculation apparatus according to claim 1, wherein the first correction unit corrects the posture of the input device so that a direction of the acceleration vector coincides with a vertically downward direction. The first correction means adjusts the amount of correction of the attitude of the input device so that the direction of the acceleration vector and the vertical downward direction in the space are closer as the acceleration vector is closer to the gravitational acceleration. The posture calculation apparatus according to claim 1, wherein the posture calculation device is increased .   The said 1st correction | amendment means correct | amends the attitude | position of the said input device, only when the difference between the magnitude | size of the said acceleration vector and the magnitude | size of gravitational acceleration is smaller than predetermined | prescribed reference | standard. The posture calculation device described in 1. The input device further includes an imaging unit,
6. The posture calculation according to claim 1, further comprising: a second correction unit that further corrects the posture of the input device based on an image of a predetermined imaging target captured by the imaging unit. apparatus.   The posture calculation device according to claim 6, wherein the second correction unit corrects the posture of the input device by adding rotation about the predetermined axis to the posture of the input device.   A game device that performs game processing using the posture corrected by the posture calculation device according to claim 1 as the posture of the input device. An attitude calculation program executed by a computer of an attitude calculation device that acquires data from an input device including at least an angular velocity sensor and an acceleration sensor and calculates an attitude of the input device in a three-dimensional space,
Posture calculating means for acquiring an angular velocity detected by the angular velocity sensor and calculating a new posture by rotating the previous posture in accordance with the acquired angular velocity ;
Acceleration vector calculating means for calculating an acceleration vector indicating the acceleration of the input device based on acceleration data from the acceleration sensor;
The computer is used as first correction means for correcting the orientation of the input device in accordance with a change in the direction of the acceleration vector so that the direction of the acceleration vector in the space and the vertical downward direction in the space are brought close to or coincide with each other. Make it work
Wherein the first correction means, with respect to the posture of the input device, is set for the input device, the change in the uncorrected orientation as the corrected orientation of the predetermined axis you correspond to the attitude of the input device are most A posture calculation program for correcting the posture of the input device so as to be small.   The first correction means performs a rotation around the predetermined axis so that a direction in which the acceleration vector is projected onto a plane perpendicular to the predetermined axis is close to a direction in which the vertical downward direction is projected onto the plane. The posture of the input device is corrected by a conversion comprising: a second conversion that performs a rotation to bring the direction of the acceleration vector subjected to the first conversion and the vertical downward direction closer to each other. The attitude calculation program described in 1.   The attitude calculation program according to claim 9 or 10, wherein the first correction unit corrects the attitude of the input device so that a direction of the acceleration vector coincides with a vertically downward direction. The first correction means adjusts the amount of correction of the attitude of the input device so that the direction of the acceleration vector and the vertical downward direction in the space are closer as the acceleration vector is closer to the gravitational acceleration. The posture calculation program according to claim 9 or 10, wherein the posture calculation program is increased .   The said 1st correction | amendment means correct | amends the attitude | position of the said input device, only when the difference between the magnitude | size of the said acceleration vector and the magnitude | size of gravitational acceleration is smaller than predetermined | prescribed reference | standard. The attitude calculation program described in 1. The input device further includes an imaging unit,
The computer according to any one of claims 9 to 13, further causing the computer to function as a second correction unit that further corrects an attitude of the input device based on an image of a predetermined imaging target captured by the imaging unit. The attitude calculation program described.   The attitude calculation program according to claim 14, wherein the second correction unit corrects the attitude of the input device by adding rotation about the predetermined axis to the attitude of the input device.   A game program for performing a game process using the posture corrected by the posture calculation program according to any one of claims 9 to 15 as the posture of the input device. An attitude calculation system that acquires data from an input device including at least an angular velocity sensor and an acceleration sensor, and calculates an attitude of the input device in a three-dimensional space,
Posture calculating means for acquiring an angular velocity detected by the angular velocity sensor and calculating a new posture by rotating the previous posture in accordance with the acquired angular velocity ;
Acceleration vector calculating means for calculating an acceleration vector indicating the acceleration of the input device based on acceleration data from the acceleration sensor;
First correction means for correcting the orientation of the input device in response to a change in the direction of the acceleration vector so that the direction of the acceleration vector in the space and the vertical downward direction in the space are brought close to or coincide with each other;
Wherein the first correction means, with respect to the posture of the input device, is set for the input device, the change in the uncorrected orientation as the corrected orientation of the predetermined axis you correspond to the attitude of the input device are most An attitude calculation system for correcting the attitude of the input device so as to be smaller. An attitude calculation method executed in an attitude calculation device that acquires data from an input device including at least an angular velocity sensor and an acceleration sensor and calculates the attitude of the input device in a three-dimensional space,
An attitude calculation step of acquiring an angular velocity detected by the angular velocity sensor and calculating a new attitude by rotating the previous attitude according to the acquired angular velocity ;
An acceleration vector calculating step of calculating an acceleration vector indicating the acceleration of the input device based on acceleration data from the acceleration sensor;
A first correction step of correcting the orientation of the input device in response to a change in the direction of the acceleration vector so that the direction of the acceleration vector in the space and the vertical downward direction in the space are brought close to or coincide with each other;
In the first correction step, the orientation calculation unit, with respect to the posture of the input device, is set for the input device, before correction of the predetermined axis you correspond to the attitude of the input device orientation and the corrected An attitude calculation method for correcting the attitude of the input device so that a change in orientation is minimized.

Images