JP5669294B2 - Coordinate calculation apparatus and coordinate calculation program - Google Patents

Description

  The present invention relates to a coordinate calculation device or a coordinate calculation program, and more specifically to a coordinate calculation device or a coordinate calculation program for calculating two-dimensional coordinates.

  Conventionally, game devices that use the attitude of an input device as an input have been considered. For example, Patent Document 1 describes a game device that uses an input device including an acceleration sensor and a gyro sensor. This game device controls a sword possessed by a game character in accordance with the movement of the input device. Specifically, data on the motion of swinging the sword is created based on the output of the acceleration sensor, and data on the posture of the sword is created based on the output of the gyro sensor.

JP 2000-308756 A

  As an example of using the posture of the input device as an input, it is conceivable to calculate the position of the cursor displayed on the screen from the posture. In this way, it is convenient for the user if the position coordinates on the screen can be designated by the attitude of the input device. However, in Patent Document 1, posture data is used only for input by a rough operation such as swinging a sword, and input that requires accuracy such as coordinate input cannot be performed using posture data. .

  Here, it is assumed that the position coordinates on the screen are calculated from the attitude of the input device. In this case, it is preferable that the position coordinates to be calculated indicate a position (on the screen) indicated by the input device. However, even if the posture of the input device is the same, if the position of the input device is different, the position indicated by the input device is also different. Therefore, if the position coordinates are calculated only from the attitude of the input device, the position indicated by the input device cannot be calculated, and appropriate position coordinates cannot be calculated. If a deviation occurs between the calculated position coordinates and the position indicated by the input device, the user may feel uncomfortable with the operation and may feel that the operability is poor. As described above, there is a possibility that appropriate position coordinates cannot be calculated by a method of calculating position coordinates on the screen by simply using the attitude of the input device.

  Therefore, an object of the present invention is to provide a coordinate calculation device or a coordinate calculation program that can appropriately calculate the position on the screen using the attitude of the input device.

  The present invention employs the following configurations (1) to (20) in order to solve the above problems.

(1)
The present invention is a coordinate calculation device that calculates coordinates representing a position on a display screen based on an attitude of an input device. The coordinate calculation apparatus includes a direction acquisition unit, a posture calculation unit, a first coordinate calculation unit, and a correction unit. The direction acquisition means acquires information indicating the direction of the input device when viewed from a predetermined position in the predetermined space. The posture calculation means calculates the posture of the input device in the predetermined space. The first coordinate calculation means calculates first coordinates for determining a position on the display screen based on the attitude of the input device. The correcting unit corrects the first coordinate so that the first coordinate calculated when the direction of the input device acquired by the direction acquiring unit becomes a predetermined direction becomes a predetermined reference value.

In the above description, the “input device” is the input device 8 including the controller 5 and the gyro sensor unit 7 in the embodiment described later. May be. For example, the input device may be only the controller 5.
The above-mentioned “coordinate calculation device” is a game device for executing game processing in the embodiments described later. However, an arbitrary information processing device that calculates the position on the display screen by using the attitude of the input device as an input. It is a concept that includes.
The above-mentioned “information indicating the direction of the input device when viewed from a predetermined position in a predetermined space” is marker coordinate information in the embodiment described later, but calculates the position on the display screen pointed to by the input device. Any information can be used as long as it is possible. Further, the “direction acquisition means” may be one that is acquired by receiving the above information from the outside (input device) of the coordinate calculation device (steps S3 and S51 in the embodiments described later), or the coordinate calculation device. It may be obtained by calculating the above information based on data received from outside.
The “attitude calculation means” may be any means for calculating the attitude of the input device, and the attitude calculation method is not limited. For example, in the embodiment described later, the posture is calculated using outputs of the gyro sensor, the acceleration sensor, and the imaging unit, but the “posture calculating unit” may calculate the posture using only the gyro sensor. Alternatively, the posture may be calculated using only the acceleration sensor.
The “first coordinate” may be any coordinate that can determine the position on the display screen. For example, the first coordinate is a coordinate that represents a position on a plane different from the display screen as in an embodiment described later. Alternatively, it may be coordinates representing the position on the display screen to be calculated. That is, the “first coordinate calculation means” is the CPU 10 that executes the processing (steps S7 and S56) for calculating the coordinates on the virtual plane in the embodiment described later, but the display screen is based on the attitude of the input device. The upper coordinates may be calculated directly.
In the embodiment described later, the “predetermined direction” is a direction in which the input device points to the center of the screen, but is not limited thereto, and may be a direction in which the input device points to the lower left of the screen, for example. The “predetermined reference value” is a value corresponding to the position on the screen to be calculated in the posture in which the input device is in the predetermined direction, and any specific value may be used. For example, in an embodiment to be described later, a value (origin (0, 0) corresponding to a position on the screen (center position of the screen) to be calculated in a posture in which the input device is in a predetermined direction (direction pointing to the screen center). )). Note that the processing in the “correction unit” corresponds to step S42 in the embodiment described later, but when the calculation of the first coordinate and the correction thereof are executed in one calculation (formula (16) described later). ) Corresponds to the process of executing the calculation.

  According to the configuration of (1) above, the first coordinates corresponding to the attitude of the input device at the time when the direction of the input device becomes the predetermined direction is the reference value, and the position on the screen determined from the first coordinates Is a position corresponding to the reference value. That is, according to the configuration of (1) above, by appropriately setting the reference value, it is possible to appropriately set the position on the screen when the position pointed to by the input device is a specific position. For example, when the input device points to the center of the screen, the position on the screen calculated from the attitude of the input device can be set as the center position of the screen. Therefore, according to the configuration of (1) above, the deviation between the position pointed to by the input device and the position on the screen calculated from the posture can be reduced, and the position on the screen can be determined using the posture of the input device. It can be calculated appropriately. In addition, the user can be prevented from feeling uncomfortable due to the deviation.

(2)
The coordinate calculation apparatus may further include second coordinate calculation means for calculating second coordinates on the display screen based on the direction of the input device. At this time, the correcting means causes the position on the display screen determined by the corrected first coordinates to be a position represented by the second coordinates calculated when the direction of the input device becomes a predetermined direction. Make corrections.

  The “second coordinate” represents a position on the display screen, and is calculated based on the marker coordinate in the embodiment described later (step S52), but is calculated based on the direction of the input device. That's fine.

  According to the configuration of (2) above, in a state where the direction of the input device is a predetermined direction, the position on the display screen determined by the corrected first coordinate matches the position represented by the second coordinate. Can be made. Therefore, in the above state, the position on the screen calculated based on the attitude of the input device matches the position calculated based on the direction of the input device. Here, as the position calculated based on the direction of the input device, the position indicated by the input device can be calculated. Therefore, by calculating the position based on the attitude of the input device so as to match the position, the deviation between the position based on the attitude of the input device and the position pointed to by the input device can be reduced. Can be appropriately calculated based on the posture. It is also conceivable that the coordinate calculation device uses both a method for calculating the position based on the attitude of the input device and a method for calculating the position based on the direction of the input device. Even in such a case, according to the configuration of (2) above, the same position is calculated regardless of which method is used in the above state, so the user can switch between the two methods. Can reduce the sense of incongruity.

(3)
The coordinate calculation device corrects according to the difference between the position determined by the first coordinate calculated when the direction of the input device becomes a predetermined direction and the position represented by the second coordinate calculated at the time. A correction amount determining means for determining the amount may be further provided. At this time, the correction means corrects the first coordinate by the correction amount.

  The “correction amount” only needs to be determined so as to be an amount corresponding to the difference, and need not be calculated using the difference itself. The “correction amount” is, for example, the first coordinates calculated at the time when the direction of the input device becomes a predetermined direction (the time when the input device points to the center of the screen) like an offset amount in an embodiment described later. And the difference between the first coordinate (the origin of the X′Y ′ coordinate) corresponding to the position represented by the second coordinate calculated at that time (the center position of the screen).

  According to the configuration of (3) above, by calculating the correction amount according to the difference, it is possible to easily perform correction to match the position determined by the first coordinates with the position represented by the second coordinates. Can do.

(4)
The coordinate calculation apparatus may further include a process execution unit that executes predetermined information processing using a position on the display screen as an input. At this time, the correction amount determining means determines the correction amount in accordance with the direction of the input device becoming the predetermined direction before the predetermined information processing is executed. The first coordinate calculation means calculates the first coordinate while predetermined information processing is repeatedly executed. The correction unit corrects the first coordinate according to the correction amount every time the first coordinate calculation unit calculates the first coordinate.

  The “predetermined information processing” may be, for example, a process of displaying a cursor or a game object at the position of the screen coordinate, or a process of moving the object according to the positional relationship between the object position and the screen coordinate. .

  According to the configuration of (4) above, the coordinate calculation apparatus determines the correction amount before the predetermined information processing is executed. When executing predetermined information processing, the first coordinates are corrected according to a predetermined correction amount. According to this, since the predetermined information processing is executed using the position on the screen corrected to an appropriate value, the user does not feel uncomfortable with the execution result of the predetermined information processing. Usability in information processing can be improved.

(5)
The coordinate calculation apparatus may further include instruction image display means for displaying a predetermined instruction image at a position represented by the second coordinates on the display screen. At this time, when the instruction image falls within a predetermined range on the screen, the correction amount determination means determines the first coordinates calculated when the instruction image falls within the predetermined range, and a predetermined value within the predetermined range. The correction amount is determined based on the second coordinates corresponding to the position.

  The “instruction image” may be an image displayed at the position of the second coordinate in addition to the cursor in the embodiment described later, and may be an image of a character operated by the user in a game, for example.

  According to the configuration of (5) above, when the position of the instruction image (the position indicated by the input device) is within a predetermined range, the correction amount is determined. Therefore, according to the configuration of (5) above, for example, the correction amount can be determined while the user is performing an operation using the instruction image. Therefore, the correction can be performed without causing the user to perform a special operation. The amount can be determined.

(6)
The coordinate calculation apparatus may further include button image display means for displaying a button image for a user to give a predetermined instruction in a predetermined range. At this time, the correction amount determining means executes processing for determining the correction amount only when the button image is displayed.

  In the embodiment described later, the “predetermined instruction” is an instruction for adjustment relating to the cursor, but may be any instruction, for example, the predetermined information processing (for example, a game process using a position on the screen as an input) ) May be an instruction for starting the execution.

  According to the configuration of (6) above, the processing for determining the correction amount is executed when the instruction image enters the button image area. According to this, when the user tries to give an instruction using the button image and the instruction image is matched with the button image area, the correction amount is determined. Therefore, the operation for determining the correction amount can be performed naturally without making the user aware of the operation.

(7)
The first coordinate calculating unit may calculate a coordinate value corresponding to the direction and amount of change from a predetermined posture to the posture calculated by the posture calculating unit as the first coordinate.

  The “predetermined posture” is a posture in which the input device (the Z axis) faces in the horizontal direction in the embodiment described later, but may be an arbitrary posture.

  According to the configuration of (7) above, the position on the screen moves from a predetermined posture in a direction corresponding to the direction in which the input device is rotated by a distance corresponding to the rotation amount. Therefore, the user can easily operate the position on the screen by changing the attitude of the input device.

(8)
The first coordinate calculation means may calculate, as the first coordinate, a coordinate representing a position of an intersection between a line segment extending from a predetermined reference point in a vector direction indicating the attitude of the input device and a predetermined plane. At this time, the correction means moves the coordinate system of the first coordinate so that the first coordinate calculated when the direction of the input device becomes the predetermined direction becomes the predetermined reference value. Correct the coordinates. The coordinate calculation device further includes position calculation means for calculating coordinates representing a position on the display screen based on the first coordinates corrected by the correction means.

The “predetermined reference point”, “vector indicating the attitude of the input device”, and “predetermined plane” are set in a virtual space for calculating the first coordinates (FIG. 18). The “predetermined reference point” represents the position of the input device, for example, and the “predetermined plane” is set to correspond to the screen, for example.
The process performed by the “position calculation unit” corresponds to a process (step S43) of calculating a position on the screen from first coordinates representing a position on a plane in the virtual space in the embodiment described later.

  With configuration (8) above, it is possible to easily calculate the coordinates on the plane (first coordinates) that change according to the attitude of the input device. Further, a fixed amount of correction can be easily performed by changing (moving) the origin of the coordinate system.

(9)
The correcting means may perform correction by moving the coordinate system of the first coordinates on a predetermined plane.

  According to the configuration of (9) above, the correction calculation only needs to add (subtract) a predetermined value to the first coordinates calculated by the first calculation means, and the correction process can be performed easily and at high speed. .

(10)
The correcting means may perform the correction by rotating the coordinate system of the first coordinates around a predetermined reference point.

  According to the configuration of (10) above, the distance from the reference point to the plane of the coordinate system can be made constant regardless of the correction amount (FIG. 26). Therefore, the position on the screen can be calculated more accurately.

(11)
The first coordinate calculation means may calculate a two-dimensional coordinate as the first coordinate. At this time, the correction means corrects only one component of the first coordinate.

  “One component of the first coordinate” is a component (Y ′ component) corresponding to the vertical direction of the screen in the embodiment described later, but may be a component corresponding to the horizontal direction of the screen.

  According to the configuration of (11) above, the position on the screen based on the posture can be corrected only in one direction. Therefore, the present invention can also be applied to the case where correction is performed only in the vertical direction of the screen, for example, as in an embodiment described later.

(12)
The input device may have an imaging unit. At this time, the direction acquisition unit acquires information indicating the position of a predetermined imaging target in the image captured by the imaging unit as information indicating the direction of the input device.

  The “predetermined imaging target” is the marker unit 6 in the embodiment described later, but may be anything as long as the coordinate calculation device can be recognized by the image. In addition, the “position of a predetermined imaging target in an image captured by the imaging unit” is represented by marker coordinates in an embodiment described later.

  According to the configuration of (12) above, the direction of the input device can be easily specified by using the image obtained by the imaging means of the input device.

(13)
The input device may include an acceleration sensor and / or an angular velocity sensor. At this time, the posture calculation means calculates the posture of the input device based on the detection result of the acceleration sensor and / or the angular velocity sensor.

  With configuration (13) above, the attitude of the input device can be easily calculated by using the detection results of the acceleration sensor and / or angular velocity sensor.

(14)
The posture calculation means may calculate the posture of the input device based on the angular velocity detected by the angular velocity sensor and correct the posture based on the acceleration detected by the acceleration sensor.

  According to the configuration of (14) above, the posture can be calculated more accurately by correcting the posture of the input device based on the angular velocity based on the acceleration.

(15)
The input device may have an imaging unit. At this time, the attitude calculation means calculates the attitude of the input device based on the angular velocity detected by the angular velocity sensor, and based on the position of a predetermined imaging target in the image captured by the imaging means, with at least the gravity direction as an axis. The posture is corrected with respect to the rotation performed.

  According to the configuration of (15) above, the attitude can be calculated more accurately by correcting the attitude of the input device based on the angular velocity using information on the image captured by the imaging means.

(16)
In addition, a coordinate calculation device that calculates coordinates representing a position on the display screen based on the attitude of the input device may be configured as follows. That is, the coordinate calculation apparatus may include a direction acquisition unit, a posture calculation unit, a reference determination unit, and a first coordinate calculation unit. The direction acquisition means acquires the direction of the input device when viewed from a predetermined position in the predetermined space. The posture calculation means calculates the posture of the input device in the predetermined space. The reference determining unit determines the posture of the input device at the time when the direction of the input device acquired by the direction acquiring unit becomes a predetermined direction as the reference posture. The first coordinate calculation means sets the first coordinates for determining the position on the display screen to a predetermined reference value when the input device is in the reference attitude, and from the reference attitude, the attitude calculation means Is calculated to be a value corresponding to the direction and amount of change up to the calculated posture.

The process performed by the “reference determination unit” corresponds to the process of determining (storing) the offset amount (step S57) in the embodiment described later, but the reference posture itself may be stored.
In the embodiment described later, the process performed by the “first coordinate calculation unit” is a process of calculating the first coordinate by the first method described later (step S7) and the calculated first coordinate is corrected. This corresponds to the processing (step S42). However, in another embodiment, the process performed by the “first coordinate calculation unit” does not need to include the process of calculating the first coordinate by the first method as in the embodiment described later. The first coordinate may be calculated based on the difference between the posture and the posture calculated by the posture calculation means.

  According to the configuration of (16) above, the reference posture is determined when the direction of the input device becomes a predetermined direction, and the first coordinates are calculated so that the position on the screen in the reference posture becomes a specific value. The Therefore, by appropriately setting the reference value, when the input device is in the reference posture, the position on the screen pointed to by the input device is matched with the calculated position (or the deviation between the two is reduced). be able to. Therefore, the position on the screen can be appropriately calculated using the attitude of the input device, as in the configuration (1). In addition, the user can be prevented from feeling uncomfortable due to the deviation.

(17)
The coordinate calculation device may further include second coordinate calculation means for calculating the second coordinates on the display screen based on the direction of the input device. At this time, the reference determining means sets the reference posture so that the position on the display screen corresponding to the reference value is a position represented by the second coordinates calculated when the direction of the input device becomes a predetermined direction. calculate.

  According to the configuration of (17), as in the configuration of (2), the position on the display screen determined by the first coordinates and the second coordinates in a state where the direction of the input device is a predetermined direction. The position represented by can be matched. Accordingly, the position on the screen can be appropriately calculated based on the posture, as in the configuration (2). Further, even when the method for calculating the position based on the attitude of the input device and the method for calculating the position based on the direction of the input device are used in combination, the user feels uncomfortable when switching between the two methods. be able to.

(18)
The first coordinate may be a coordinate representing a position of an intersection of a line segment extending from a predetermined reference point in a vector direction indicating the attitude of the input device and a predetermined plane. At this time, the reference determining means sets the coordinate system on a predetermined plane so that the first coordinate becomes the reference value when the reference posture is set. The first coordinate calculation means calculates the first coordinates expressed using the coordinate system set by the reference determination means. The coordinate calculation device further includes position calculation means for calculating a position on the display screen based on the first coordinates calculated by the first coordinate calculation means.

  According to the above configuration 18, similarly to the configuration (8), it is possible to easily calculate coordinates on the plane (first coordinates) that change in accordance with the attitude of the input device.

(19)
The reference determining means may set a predetermined plane on a plane fixedly set with respect to the predetermined reference point.

  According to the configuration of (19) above, the coordinate system of the first coordinate is set on one plane regardless of the value of the reference posture, so that the coordinate system can be easily set. .

(20)
The reference determining means may set the predetermined plane so that the distance from the predetermined reference point is constant.

  According to the configuration of (20) above, the distance from the reference point to the plane of the coordinate system is constant regardless of the value of the reference posture, so that the position on the screen can be calculated more accurately.

  The present invention may also be implemented in the form of a coordinate calculation program that causes a computer of an information processing apparatus to function as each of the above means.

  According to the present invention, when the direction of the input device is a predetermined direction, the position on the screen based on the attitude of the input device can be calculated to be an appropriate position that matches the position indicated by the input device. it can. Thereby, the deviation between the position indicated by the input device and the position on the screen calculated from the posture can be reduced, and the user can be prevented from feeling uncomfortable due to the deviation.

External view of game system Functional block diagram of game device The perspective view which shows the external appearance structure of an input device Perspective view showing the external configuration of the controller Diagram showing the internal structure of the controller Diagram showing the internal structure of the controller Block diagram showing the configuration of the input device The figure which shows the vector which shows a 1st attitude | position and a 2nd attitude | position The figure which shows vector v3 which shows correction amount The figure which shows the vector which shows the 1st attitude | position after correction | amendment by a 1st correction process. The figure which shows the vector which shows a 1st attitude | position and a 3rd attitude | position The figure which shows the 1st attitude | position after correction | amendment by a 2nd correction process. The figure which shows the main data memorize | stored in the main memory of a game device Main flowchart showing the flow of processing executed in the game device Flowchart showing the flow of the first correction process (step S5) shown in FIG. Flowchart showing the flow of the second correction process (step S6) shown in FIG. The figure which shows the two-dimensional coordinate corresponding to a captured image The figure which shows the input device and predetermined plane in a predetermined virtual space FIG. 18 is a diagram in which the virtual space shown in FIG. 18 is viewed from the y-axis positive direction side to the negative direction side. The figure which shows the comparison with the 1st method and the 2nd method The figure which shows the main data memorize | stored in the main memory of a game device in the case of employ | adopting a 2nd method. The main flowchart which shows the flow of the process which a game device performs in the case of taking a 2nd method. The flowchart which shows the flow of the offset determination process (step S41) shown in FIG. Figure showing an example of the setting screen Diagram showing how to determine the offset amount The figure which shows the modification of the correction method in a 2nd method.

[Overall configuration of game system]
With reference to FIG. 1, a game system 1 including a game apparatus which is an example of a coordinate calculation apparatus according to an embodiment of the present invention will be described. FIG. 1 is an external view of the game system 1. Hereinafter, the game apparatus and the game program of the present embodiment will be described using a stationary game apparatus as an example. In FIG. 1, the game system 1 includes a television receiver (hereinafter simply referred to as “TV”) 2, a game device 3, an optical disk 4, an input device 8, and a marker unit 6. In the present system, game processing is executed by the game device 3 based on a game operation using the input device 8.

  An optical disk 4 that is an example of an information storage medium that can be used interchangeably with the game apparatus 3 is detachably inserted into the game apparatus 3. The optical disc 4 stores a game program to be executed on the game apparatus 3. An insertion slot for the optical disk 4 is provided on the front surface of the game apparatus 3. The game apparatus 3 executes a game process by reading and executing a game program stored in the optical disc 4 inserted into the insertion slot.

  A television 2 which is an example of a display device is connected to the game apparatus 3 via a connection cord. The television 2 displays a game image obtained as a result of the game process executed in the game device 3. In addition, a marker unit 6 is installed around the screen of the television 2 (upper side of the screen in FIG. 1). The marker unit 6 includes two markers 6R and 6L at both ends thereof. The marker 6R (same for the marker 6L) is specifically one or more infrared LEDs, and outputs infrared light toward the front of the television 2. The marker unit 6 is connected to the game apparatus 3, and the game apparatus 3 can control lighting of each infrared LED included in the marker unit 6.

  The input device 8 gives operation data indicating the content of the operation performed on the own device to the game device 3. In the present embodiment, the input device 8 includes a controller 5 and a gyro sensor unit 7. Although details will be described later, the input device 8 has a configuration in which a gyro sensor unit 7 is detachably connected to the controller 5. The controller 5 and the game apparatus 3 are connected by wireless communication. In the present embodiment, for example, Bluetooth (registered trademark) technology is used for wireless communication between the controller 5 and the game apparatus 3. In other embodiments, the controller 5 and the game apparatus 3 may be connected by wire.

[Internal configuration of game device 3]
Next, the internal configuration of the game apparatus 3 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration of the game apparatus 3. The game apparatus 3 includes a CPU 10, a system LSI 11, an external main memory 12, a ROM / RTC 13, a disk drive 14, an AV-IC 15 and the like.

  The CPU 10 executes a game process by executing a game program stored on the optical disc 4, and functions as a game processor. The CPU 10 is connected to the system LSI 11. In addition to the CPU 10, an external main memory 12, a ROM / RTC 13, a disk drive 14, and an AV-IC 15 are connected to the system LSI 11. The system LSI 11 performs processing such as control of data transfer between components connected thereto, generation of an image to be displayed, and acquisition of data from an external device. The internal configuration of the system LSI will be described later. The volatile external main memory 12 stores a program such as a game program read from the optical disc 4 or a game program read from the flash memory 17, or stores various data. Used as a work area and buffer area. The ROM / RTC 13 includes a ROM (so-called boot ROM) in which a program for starting the game apparatus 3 is incorporated, and a clock circuit (RTC: Real Time Clock) that counts time. The disk drive 14 reads program data, texture data, and the like from the optical disk 4 and writes the read data to an internal main memory 11e or an external main memory 12 described later.

  The system LSI 11 includes an input / output processor (I / O processor) 11a, a GPU (Graphics Processor Unit) 11b, a DSP (Digital Signal Processor) 11c, a VRA 1d, and an internal main memory 11e. Although not shown, these components 11a to 11e are connected to each other by an internal bus.

  The GPU 11b forms part of a drawing unit and generates an image according to a graphics command (drawing command) from the CPU 10. The VRAM 11d stores data (data such as polygon data and texture data) necessary for the GPU 11b to execute the graphics command. When an image is generated, the GPU 11b creates image data using data stored in the VRAM 11d.

  The DSP 11c functions as an audio processor, and generates sound data using sound data and sound waveform (tone color) data stored in the internal main memory 11e and the external main memory 12.

  The image data and audio data generated as described above are read out by the AV-IC 15. The AV-IC 15 outputs the read image data to the television 2 via the AV connector 16, and outputs the read audio data to the speaker 2 a built in the television 2. As a result, an image is displayed on the television 2 and a sound is output from the speaker 2a.

  The input / output processor 11a performs transmission / reception of data to / from components connected to the input / output processor 11a and downloads data from an external device. The input / output processor 11a is connected to the flash memory 17, the wireless communication module 18, the wireless controller module 19, the expansion connector 20, and the memory card connector 21. An antenna 22 is connected to the wireless communication module 18, and an antenna 23 is connected to the wireless controller module 19.

  The input / output processor 11a is connected to the network via the wireless communication module 18 and the antenna 22, and can communicate with other game devices and various servers connected to the network. The input / output processor 11a periodically accesses the flash memory 17 to detect the presence / absence of data that needs to be transmitted to the network. If there is such data, the input / output processor 11a communicates with the network via the wireless communication module 18 and the antenna 22. Send. Further, the input / output processor 11a receives data transmitted from other game devices and data downloaded from the download server via the network, the antenna 22 and the wireless communication module 18, and receives the received data in the flash memory 17. Remember. By executing the game program, the CPU 10 reads out the data stored in the flash memory 17 and uses it in the game program. In the flash memory 17, in addition to data transmitted and received between the game apparatus 3 and other game apparatuses and various servers, save data (game result data or intermediate data) of the game played using the game apparatus 3 May be stored.

  The input / output processor 11a receives operation data transmitted from the controller 5 via the antenna 23 and the wireless controller module 19, and stores (temporarily stores) the data in the buffer area of the internal main memory 11e or the external main memory 12.

  Further, an expansion connector 20 and a memory card connector 21 are connected to the input / output processor 11a. The expansion connector 20 is a connector for an interface such as USB or SCSI, and connects a medium such as an external storage medium, a peripheral device such as another controller, or a wired communication connector. By connecting, communication with the network can be performed instead of the wireless communication module 18. The memory card connector 21 is a connector for connecting an external storage medium such as a memory card. For example, the input / output processor 11a can access an external storage medium via the expansion connector 20 or the memory card connector 21 to store data in the external storage medium or read data from the external storage medium.

  The game apparatus 3 is provided with a power button 24, a reset button 25, and an eject button 26. The power button 24 and the reset button 25 are connected to the system LSI 11. When the power button 24 is turned on, power is supplied to each component of the game apparatus 3 via an AC adapter (not shown). When the reset button 25 is pressed, the system LSI 11 restarts the boot program for the game apparatus 3. The eject button 26 is connected to the disk drive 14. When the eject button 26 is pressed, the optical disk 4 is ejected from the disk drive 14.

[Configuration of Input Device 8]
Next, the input device 8 will be described with reference to FIGS. FIG. 3 is a perspective view showing an external configuration of the input device 8. FIG. 4 is a perspective view showing an external configuration of the controller 5. 3 is a perspective view of the controller 5 as seen from the upper rear side, and FIG. 4 is a perspective view of the controller 5 as seen from the lower front side.

  3 and 4, the controller 5 includes a housing 31 formed by plastic molding, for example. The housing 31 has a substantially rectangular parallelepiped shape whose longitudinal direction is the front-rear direction (the Z-axis direction shown in FIG. 3), and is a size that can be gripped with one hand of an adult or a child as a whole. The player can perform a game operation by pressing a button provided on the controller 5 and moving the controller 5 itself to change its position and posture.

  The housing 31 is provided with a plurality of operation buttons. As shown in FIG. 3, a cross button 32a, a first button 32b, a second button 32c, an A button 32d, a minus button 32e, a home button 32f, a plus button 32g, and a power button 32h are provided on the upper surface of the housing 31. It is done. In the present specification, the upper surface of the housing 31 on which these buttons 32a to 32h are provided may be referred to as a “button surface”. On the other hand, as shown in FIG. 4, a recess is formed on the lower surface of the housing 31, and a B button 32i is provided on the rear inclined surface of the recess. A function corresponding to the game program executed by the game apparatus 3 is appropriately assigned to each of the operation buttons 32a to 32i. The power button 32h is for remotely turning on / off the main body of the game apparatus 3. The home button 32 f and the power button 32 h are embedded in the upper surface of the housing 31. This can prevent the player from pressing the home button 32f or the power button 32h by mistake.

  A connector 33 is provided on the rear surface of the housing 31. The connector 33 is used to connect another device (for example, the gyro sensor unit 7 or another controller) to the controller 5. Further, locking holes 33a are provided on both sides of the connector 33 on the rear surface of the housing 31 in order to prevent the other devices from being easily detached.

  A plurality (four in FIG. 3) of LEDs 34 a to 34 d are provided behind the upper surface of the housing 31. Here, the controller type (number) is assigned to the controller 5 to distinguish it from other main controllers. The LEDs 34a to 34d are used for the purpose of notifying the player of the controller type currently set in the controller 5 and notifying the player of the remaining battery level of the controller 5. Specifically, when a game operation is performed using the controller 5, any one of the plurality of LEDs 34a to 34d is turned on according to the controller type.

  Further, the controller 5 has an imaging information calculation unit 35 (FIG. 6), and a light incident surface 35a of the imaging information calculation unit 35 is provided on the front surface of the housing 31 as shown in FIG. The light incident surface 35a is made of a material that transmits at least infrared light from the markers 6R and 6L.

  Between the first button 32b and the home button 32f on the upper surface of the housing 31, a sound release hole 31a for releasing sound from the speaker 49 (FIG. 5) built in the controller 5 is formed.

  Next, the internal structure of the controller 5 will be described with reference to FIGS. 5 and 6 are diagrams showing the internal structure of the controller 5. FIG. FIG. 5 is a perspective view showing a state in which the upper housing (a part of the housing 31) of the controller 5 is removed. FIG. 6 is a perspective view showing a state in which the lower casing (a part of the housing 31) of the controller 5 is removed. The perspective view shown in FIG. 6 is a perspective view of the substrate 30 shown in FIG.

  In FIG. 5, a substrate 30 is fixed inside the housing 31, and operation buttons 32 a to 32 h, LEDs 34 a to 34 d, an acceleration sensor 37, an antenna 45, and a speaker 49 are provided on the upper main surface of the substrate 30. Etc. are provided. These are connected to a microcomputer (microcomputer) 42 (see FIG. 6) by wiring (not shown) formed on the substrate 30 and the like. In the present embodiment, the acceleration sensor 37 is disposed at a position shifted from the center of the controller 5 with respect to the X-axis direction. This makes it easier to calculate the movement of the controller 5 when the controller 5 is rotated about the Z axis. The acceleration sensor 37 is disposed in front of the center of the controller 5 in the longitudinal direction (Z-axis direction). Further, the controller 5 functions as a wireless controller by the wireless module 44 (FIG. 6) and the antenna 45.

  On the other hand, in FIG. 6, an imaging information calculation unit 35 is provided at the front edge on the lower main surface of the substrate 30. The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41 in order from the front of the controller 5. These members 38 to 41 are respectively attached to the lower main surface of the substrate 30.

  Further, the microcomputer 42 and the vibrator 48 are provided on the lower main surface of the substrate 30. The vibrator 48 is, for example, a vibration motor or a solenoid, and is connected to the microcomputer 42 by wiring formed on the substrate 30 or the like. The controller 48 is vibrated by the operation of the vibrator 48 according to the instruction of the microcomputer 42. As a result, a so-called vibration-compatible game in which the vibration is transmitted to the hand of the player holding the controller 5 can be realized. In the present embodiment, the vibrator 48 is disposed slightly forward of the housing 31. That is, by arranging the vibrator 48 on the end side of the center of the controller 5, the entire controller 5 can be vibrated greatly by the vibration of the vibrator 48. The connector 33 is attached to the rear edge on the lower main surface of the substrate 30. 5 and 6, the controller 5 includes a crystal resonator that generates a basic clock of the microcomputer 42, an amplifier that outputs an audio signal to the speaker 49, and the like.

  Further, the gyro sensor unit 7 includes gyro sensors (gyro sensors 55 and 56 shown in FIG. 7) that detect angular velocities around three axes. The gyro sensor unit 7 is detachably attached to the connector 33 of the controller 5. A plug (plug 53 shown in FIG. 7) that can be connected to the connector 33 is provided at the front end of the gyro sensor unit 7 (end on the Z-axis positive direction side shown in FIG. 3). Further, hooks (not shown) are provided on both sides of the plug 53. In a state where the gyro sensor unit 7 is attached to the controller 5, the plug 53 is connected to the connector 33 and the hook is locked in the locking hole 33 a of the controller 5. Thereby, the controller 5 and the gyro sensor unit 7 are firmly fixed. The gyro sensor unit 7 has a button 51 on a side surface (surface in the X-axis direction shown in FIG. 3). The button 51 is configured such that when the button 51 is pressed, the hook is released from the locked state with respect to the locking hole 33a. Therefore, the gyro sensor unit 7 can be detached from the controller 5 by removing the plug 53 from the connector 33 while pressing the button 51.

  A connector having the same shape as the connector 33 is provided at the rear end of the gyro sensor unit 7. Therefore, other devices that can be attached to the controller 5 (connector 33 thereof) can also be attached to the connector of the gyro sensor unit 7. In FIG. 3, a cover 52 is detachably attached to the connector.

  The shapes of the controller 5 and the gyro sensor unit 7 shown in FIGS. 3 to 6, the shapes of the operation buttons, the number of acceleration sensors and vibrators, and the installation positions are merely examples, and other shapes, numbers, Even in the installation position, the present invention can be realized. In the present embodiment, the imaging direction by the imaging unit is the positive Z-axis direction, but the imaging direction may be any direction. That is, the position of the imaging information calculation unit 35 in the controller 5 (the light incident surface 35a of the imaging information calculation unit 35) does not have to be the front surface of the housing 31, and other surfaces can be used as long as light can be taken in from the outside of the housing 31. May be provided.

  FIG. 7 is a block diagram showing a configuration of the input device 8 (the controller 5 and the gyro sensor unit 7). The controller 5 includes an operation unit 32 (operation buttons 32a to 32i), a connector 33, an imaging information calculation unit 35, a communication unit 36, and an acceleration sensor 37. The controller 5 transmits data indicating the details of the operation performed on the own device to the game apparatus 3 as operation data.

  The operation unit 32 includes the operation buttons 32a to 32i described above, and the operation button data indicating the input state (whether or not each operation button 32a to 32i is pressed) to each operation button 32a to 32i is transmitted to the microcomputer of the communication unit 36. Output to 42.

  The imaging information calculation unit 35 is a system for analyzing the image data captured by the imaging unit, discriminating a region having a high luminance in the image data, and calculating a center of gravity position, a size, and the like of the region. Since the imaging information calculation unit 35 has a sampling period of, for example, about 200 frames / second at the maximum, it can track and analyze even a relatively fast movement of the controller 5.

  The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41. The infrared filter 38 passes only infrared rays from the light incident from the front of the controller 5. The lens 39 collects the infrared light transmitted through the infrared filter 38 and makes it incident on the image sensor 40. The image sensor 40 is a solid-state image sensor such as a CMOS sensor or a CCD sensor, for example, and receives the infrared light collected by the lens 39 and outputs an image signal. Here, the markers 6 </ b> R and 6 </ b> L of the marker unit 6 disposed in the vicinity of the display screen of the television 2 are configured by infrared LEDs that output infrared light toward the front of the television 2. Therefore, by providing the infrared filter 38, the image sensor 40 receives only the infrared light that has passed through the infrared filter 38 and generates image data, so that the images of the markers 6R and 6L can be captured more accurately. Hereinafter, an image captured by the image sensor 40 is referred to as a captured image. Image data generated by the image sensor 40 is processed by the image processing circuit 41. The image processing circuit 41 calculates the position of the imaging target (markers 6R and 6L) in the captured image. The image processing circuit 41 outputs coordinates indicating the calculated position to the microcomputer 42 of the communication unit 36. The coordinate data is transmitted to the game apparatus 3 as operation data by the microcomputer 42. Hereinafter, the coordinates are referred to as “marker coordinates”. Since the marker coordinates change corresponding to the direction (tilt angle) and position of the controller 5 itself, the game apparatus 3 can calculate the direction and position of the controller 5 using the marker coordinates.

  In other embodiments, the controller 5 may not include the image processing circuit 41, and the captured image itself may be transmitted from the controller 5 to the game apparatus 3. At this time, the game apparatus 3 may have a circuit or a program having the same function as the image processing circuit 41, and may calculate the marker coordinates.

  The acceleration sensor 37 detects the acceleration (including gravity acceleration) of the controller 5, that is, detects the force (including gravity) applied to the controller 5. The acceleration sensor 37 detects the value of the acceleration (linear acceleration) in the linear direction along the sensing axis direction among the accelerations applied to the detection unit of the acceleration sensor 37. For example, in the case of a multi-axis acceleration sensor having two or more axes, the component acceleration along each axis is detected as the acceleration applied to the detection unit of the acceleration sensor. For example, the triaxial or biaxial acceleration sensor may be of the type available from Analog Devices, Inc. or ST Microelectronics NV. The acceleration sensor 37 is, for example, a capacitance type acceleration sensor, but other types of acceleration sensors may be used.

  In the present embodiment, the acceleration sensor 37 has a vertical direction (Y-axis direction shown in FIG. 3), a horizontal direction (X-axis direction shown in FIG. 3), and a front-back direction (Z-axis direction shown in FIG. 3) with reference to the controller 5. ) Linear acceleration is detected in each of the three axis directions. Since the acceleration sensor 37 detects acceleration in the linear direction along each axis, the output from the acceleration sensor 37 represents the linear acceleration value of each of the three axes. That is, the detected acceleration is represented as a three-dimensional vector (ax, ay, az) in an XYZ coordinate system (controller coordinate system) set with reference to the input device 8 (controller 5). Hereinafter, a vector having the respective acceleration values related to the three axes detected by the acceleration sensor 37 as components is referred to as an acceleration vector.

  Data indicating the acceleration detected by the acceleration sensor 37 (acceleration data) is output to the communication unit 36. The acceleration detected by the acceleration sensor 37 changes in accordance with the direction (tilt angle) and movement of the controller 5 itself, so that the game apparatus 3 can calculate the direction and movement of the controller 5 using the acceleration data. it can. In the present embodiment, the game apparatus 3 determines the attitude of the controller 5 based on the acceleration data.

  Data (acceleration data) indicating the acceleration (acceleration vector) detected by the acceleration sensor 37 is output to the communication unit 36. In the present embodiment, the acceleration sensor 37 is used as a sensor that outputs data for determining the tilt angle of the controller 5.

  In addition, based on the acceleration signal output from the acceleration sensor 37, a computer such as a processor (for example, the CPU 10) of the game apparatus 3 or a processor (for example, the microcomputer 42) of the controller 5 performs processing, whereby further information regarding the controller 5 is obtained. Those skilled in the art will be able to easily understand from the description of the present specification that can be estimated or calculated (determined). For example, when processing on the computer side is executed on the assumption that the controller 5 on which the acceleration sensor 37 is mounted is stationary (that is, the processing is executed assuming that the acceleration detected by the acceleration sensor is only gravitational acceleration). When the controller 5 is actually stationary, it can be determined whether or not the attitude of the controller 5 is inclined with respect to the direction of gravity based on the detected acceleration. Specifically, whether or not the controller 5 is inclined with respect to the reference depending on whether or not 1G (gravity acceleration) is applied, based on the state in which the detection axis of the acceleration sensor 37 is directed vertically downward. It is possible to know how much it is inclined with respect to the reference according to its size. Further, in the case of the multi-axis acceleration sensor 37, it is possible to know in detail how much the controller 5 is inclined with respect to the direction of gravity by further processing the acceleration signal of each axis. . In this case, the processor may calculate the tilt angle of the controller 5 based on the output from the acceleration sensor 37, or may calculate the tilt direction of the controller 5 without calculating the tilt angle. Good. Thus, by using the acceleration sensor 37 in combination with the processor, the tilt angle or posture of the controller 5 can be determined.

  On the other hand, when it is assumed that the controller 5 is in a dynamic state (a state in which the controller 5 is moved), the acceleration sensor 37 detects an acceleration corresponding to the movement of the controller 5 in addition to the gravitational acceleration. Therefore, the movement direction of the controller 5 can be known by removing the gravitational acceleration component from the detected acceleration by a predetermined process. Even if it is assumed that the controller 5 is in a dynamic state, the direction of gravity is obtained by removing the acceleration component corresponding to the movement of the acceleration sensor from the detected acceleration by a predetermined process. It is possible to know the inclination of the controller 5 with respect to. In another embodiment, the acceleration sensor 37 is a built-in process for performing a predetermined process on the acceleration signal before outputting the acceleration signal detected by the built-in acceleration detection means to the microcomputer 42. An apparatus or other type of dedicated processing apparatus may be provided. A built-in or dedicated processing device converts the acceleration signal into a tilt angle (or other preferred parameter) if, for example, the acceleration sensor 37 is used to detect static acceleration (eg, gravitational acceleration). It may be a thing.

  The communication unit 36 includes a microcomputer 42, a memory 43, a wireless module 44, and an antenna 45. The microcomputer 42 controls the wireless module 44 that wirelessly transmits data acquired by the microcomputer 42 to the game apparatus 3 while using the memory 43 as a storage area when performing processing. The microcomputer 42 is connected to the connector 33. Data transmitted from the gyro sensor unit 7 is input to the microcomputer 42 via the connector 33. Hereinafter, the configuration of the gyro sensor unit 7 will be described.

  The gyro sensor unit 7 includes a plug 53, a microcomputer 54, a 2-axis gyro sensor 55, and a 1-axis gyro sensor 56. As described above, the gyro sensor unit 7 detects angular velocities around the three axes (in this embodiment, the XYZ axes), and transmits data (angular velocity data) indicating the detected angular velocities to the controller 5.

  The biaxial gyro sensor 55 detects an angular velocity around the X axis and an angular velocity (per unit time) around the Y axis. The single axis gyro sensor 56 detects an angular velocity (per unit time) around the Z axis. In this specification, the rotation directions around the XYZ axes are referred to as a roll direction, a pitch direction, and a yaw direction, respectively, with reference to the imaging direction (Z axis positive direction) of the controller 5. That is, the biaxial gyro sensor 55 detects angular velocities in the roll direction (rotation direction around the X axis) and the pitch direction (rotation direction around the Y axis), and the single axis gyro sensor 56 detects the yaw direction (around the Z axis). Detect the angular velocity in the rotation direction.

  In this embodiment, the 2-axis gyro sensor 55 and the 1-axis gyro sensor 56 are used to detect the angular velocity around the three axes. However, in other embodiments, the angular velocity around the three axes is Any number and combination of gyro sensors may be used as long as they can be detected.

  Further, in the present embodiment, for the purpose of facilitating the calculation in the posture calculation process described later, the three axes where the gyro sensors 55 and 56 detect the angular velocity are the three axes (XYZ) where the acceleration sensor 37 detects the acceleration. Axis). However, in other embodiments, the three axes for detecting the angular velocities by the gyro sensors 56 and 57 and the three axes for detecting the acceleration by the acceleration sensor 37 may not coincide with each other.

  Data indicating the angular velocity detected by each of the gyro sensors 56 and 57 is output to the microcomputer 54. Accordingly, the microcomputer 54 receives data indicating the angular velocity around the three axes of the XYZ axes. The microcomputer 54 transmits data indicating the angular velocities around the three axes as angular velocity data to the controller 5 via the plug 53. Although transmission from the microcomputer 54 to the controller 5 is sequentially performed every predetermined cycle, since the game processing is generally performed in units of 1/60 seconds (one frame time), this time Transmission is preferably performed in the following cycle.

  Returning to the description of the controller 5, the data output from the operation unit 32, the imaging information calculation unit 35, and the acceleration sensor 37 to the microcomputer 42 and the data transmitted from the gyro sensor unit 7 to the microcomputer 42 are temporarily stored. Stored in the memory 43. These data are transmitted to the game apparatus 3 as the operation data. That is, the microcomputer 42 outputs the operation data stored in the memory 43 to the wireless module 44 when the transmission timing to the wireless controller module 19 of the game apparatus 3 arrives. The wireless module 44 modulates a carrier wave of a predetermined frequency with operation data using, for example, Bluetooth (registered trademark) technology, and radiates a weak radio signal from the antenna 45. That is, the operation data is modulated by the wireless module 44 into a weak radio signal and transmitted from the controller 5. The weak radio signal is received by the wireless controller module 19 on the game apparatus 3 side. By demodulating and decoding the received weak radio signal, the game apparatus 3 can acquire operation data. And CPU10 of the game device 3 performs a game process based on the acquired operation data and a game program. Note that the wireless transmission from the communication unit 36 to the wireless controller module 19 is sequentially performed at predetermined intervals, but the game processing is generally performed in units of 1/60 seconds (one frame time). Therefore, it is preferable to perform transmission at a period equal to or shorter than this time. The communication unit 36 of the controller 5 outputs each operation data to the wireless controller module 19 of the game apparatus 3 at a rate of once every 1/200 seconds, for example.

  By using the controller 5, the player can perform an operation of tilting the controller 5 to an arbitrary tilt angle in addition to the conventional general game operation of pressing each operation button. In addition, according to the controller 5, the player can also perform an operation of instructing an arbitrary position on the screen by the controller 5 and an operation of moving the controller 5 itself.

[Outline of posture calculation processing]
Next, with reference to FIGS. 8 to 12, an outline of a posture calculation process for calculating the posture of the input device 8 executed in the game apparatus 3 will be described. In the present embodiment, the game apparatus 3 acquires data (operation data) from the input device 8 including the gyro sensors 55 and 56, the acceleration sensor 37, and the imaging means (imaging element 40). Calculate the posture. In the present embodiment, the input device 8 is configured to include both the acceleration sensor 37 and the image sensor 40. However, in other embodiments, the input device 8 includes only one of the acceleration sensor 37 and the image sensor 40. It may be a configuration.

  The game apparatus 3 includes (1) posture calculation means, (2) first correction means, and (3) second correction means. In the present embodiment, these means are realized by causing a game program (coordinate calculation program) executed by the computer (CPU 10) of the game apparatus 3 to function as the means. In other embodiments, part or all of the above means may be realized as a dedicated circuit included in the game apparatus 3.

(1) Attitude calculation means The attitude calculation means calculates the attitude of the input device 8 based on the angular velocity detected by the gyro sensors 55 and 56 (step S4 described later). Any method may be used to calculate the posture from the angular velocity. For example, there is a method of sequentially adding the angular velocity (per unit time) to the initial posture. That is, the current posture can be calculated by integrating the angular velocities sequentially output from the gyro sensors 55 and 56 and calculating the amount of change in posture from the initial state from the integration result. In the following, the posture of the input device 8 calculated from the angular velocity by the posture calculation means is referred to as a “first posture”. However, the posture after the first posture is corrected is also referred to as the first posture.

  Here, the first posture calculated using the angular velocities detected by the gyro sensors 55 and 56 has an error between the actual posture of the input device 8 due to erroneous detection of the gyro sensors 55 and 56. May occur. Therefore, in the present embodiment, the game apparatus 3 corrects the first posture using the acceleration detected by the acceleration sensor 37. Further, the first posture is corrected using an image (captured image) captured by the image sensor 40.

(2) First Correction Unit The first correction unit corrects the first posture based on acceleration data detected by the acceleration sensor 37 (step S5 described later). In the present embodiment, the first correction unit performs correction to bring the first posture closer to the second posture. Here, the second posture refers to a posture determined from the acceleration data, specifically, a posture of the input device 8 when it is assumed that the direction of acceleration indicated by the acceleration data is vertically downward. That is, the posture is calculated on the assumption that the acceleration indicated by the acceleration data is gravitational acceleration. Hereinafter, a correction process (first correction process) by the first correction unit will be described with reference to FIGS.

  FIG. 8 is a diagram illustrating correction of the first posture using the second posture. In this embodiment, the posture in the three-dimensional space is actually processed. However, in FIGS. 8 to 10, the posture in the two-dimensional plane will be described for easy understanding of the drawings. A vector G shown in FIG. 8A indicates a vertically downward direction defined in a spatial coordinate system based on a predetermined position in a space where the input device 8 exists, that is, a gravity direction. Also, the vector v1 shown in FIG. 8A is a spatial coordinate of the downward vector of the input device 8 when the controller 5 is in the first posture (that is, the negative Y-axis direction shown in FIGS. 3 to 5). It shows the orientation in the system. When the posture of the input device 8 is in the basic state, the vector indicating the posture matches the vector G. Therefore, the vector v1 corresponds to the first posture in the spatial coordinate system. The first posture can also be expressed as a rotation made by the vector v1 with respect to the vector G, and is shown as an angle θ1 in the two-dimensional FIG. Since the first posture is calculated based on the angular velocity, the vector v1 is calculated by adding the angular velocity to the previous posture and rotating the vector v1. The second posture is calculated based on the acceleration data. A vector v2 shown in FIG. 8A indicates the direction of acceleration indicated by the acceleration data (the direction of acceleration in the view coordinate system). However, the acceleration data is an acceleration applied to the input device 8, and what can be acquired is a vector in a coordinate system based on the input device 8. FIG. 8B shows the relationship between the acceleration sensor axis and the acceleration vector. As shown in FIG. 8B, when the angle formed by the acceleration vector v0 acquired from the acceleration sensor and the Y-axis negative direction of the sensor is θ2, the vector v1 in the spatial coordinate system of FIG. A vector v2 obtained by adding the rotation θ2 to the acceleration vector in the spatial coordinate system. Since the second posture is “the posture of the input device 8 when the direction of the acceleration indicated by the acceleration data is considered to be vertically downward” as described above, the rotation of the angle θ2 from the vector v2 to the vector v1 is performed. Becomes the second posture. If the second posture is represented by a downward vector of the input device 8 in the spatial coordinate system as a vector v1, it can be represented as a vector v2 'obtained by rotating the vector G by θ2. In the case of a three-dimensional posture, it can be expressed by a three-dimensional rotation matrix or the like. When the first posture is correctly calculated from the angular velocity and the acceleration data accurately indicates the direction of gravity, the direction of the vector v2 indicating the direction of acceleration is the vertical downward direction of the spatial coordinate system, that is, gravity. Match the direction. That is, when the first posture is not correctly calculated from the angular velocity, or when the acceleration data does not indicate the correct gravity direction, the vector v2 indicating the direction of acceleration and the gravity as shown in FIG. The direction vector G is different. In a situation where the direction indicated by the acceleration data is assumed to coincide with the direction of gravity, such as in a stationary state, the vector v2 is more accurate as data corresponding to the attitude of the input device 8 than the vector v1. It is considered a thing. Even when the input device is not stationary, taking into account the accuracy of the average posture within a certain period of time, the acceleration vector will be close to the direction of gravity on average, so that the error may increase with time. It can be considered more reliable than the posture calculated from the accumulated angular velocity. On the other hand, if the correct posture has been calculated at the previous calculation timing, the posture at the next calculation timing is calculated more accurately using the angular velocity than the acceleration. it is conceivable that. In other words, the posture calculation based on the angular velocity has a smaller error for each timing than the calculation based on the acceleration, but the error increases with time, while the posture calculation based on the acceleration may have a larger error for each timing. Since it can be calculated at each timing, it has a feature that no error is accumulated. Therefore, the first correction unit performs correction in consideration of both the first posture and the second posture.

  The correction by the first correction means is correction that brings the first posture closer to the second posture. That is, the first correction unit performs correction to bring the angle θ1 close to the angle θ2. This may be expressed as a correction that brings the vector v1 closer to the vector v2 '. However, in the calculation process, if the vector v2 is known, the correction is possible even if the vector v2 'itself is not calculated. In the present embodiment, the correction is performed using a vector v3 indicating the correction amount. FIG. 9 is a diagram illustrating a vector v3 indicating the correction amount. A vector v3 illustrated in FIG. 9 is a vector indicating a correction amount for correcting the first posture. Specifically, an angle Δθ formed by the vector v3 with respect to the vector v2 is a correction amount. Although details will be described later, the vector v3 is set between the vector G and the vector v2 (see FIG. 9). By rotating the vector v1 by Δθ, the vector v1 approaches the vector v2 ′ described above.

  The first correction process is performed by rotating the first posture (vector v1) by the correction amount. FIG. 10 is a diagram illustrating a vector indicating the first posture after the correction by the first correction process. As shown in FIG. 10, the corrected first posture (vector v1 ') is obtained by rotating the first posture before correction (vector v1) by an angle Δθ. As a result, the angle θ1 ′ representing the first posture after correction is between the angle θ1 and the angle θ2, and it can be seen that the correction for bringing the angle θ1 closer to the angle θ2 was performed.

  In the present embodiment, the first correction unit performs correction to bring the first posture closer to the second posture, and does not match the corrected first posture with the second posture. This is to prevent the corrected first posture from changing suddenly even when the acceleration data changes suddenly due to erroneous detection or intense operation. However, in other embodiments, the first correction unit may perform correction so that the corrected first posture matches the second posture. Although details will be described later, in the present embodiment, the rate at which the first correction unit brings the first posture closer to the second posture is the magnitude of the acceleration indicated by the acceleration data (more specifically, It is determined so as to change according to the difference between the magnitude and the gravitational acceleration. However, in other embodiments, the ratio may be a predetermined fixed value.

(3) Second Correction Unit The second correction unit corrects the first posture based on a predetermined image to be captured that is captured by the imaging unit (step S6 described later). Here, in the present embodiment, the predetermined imaging target is the marker unit 6 (infrared LED). In the present embodiment, the second correction unit performs correction to bring the first posture closer to the third posture. The third posture is a posture calculated from the image to be imaged, and specifically, the posture of the input device 8 calculated from the orientation and / or position of the imaged object in the image. Hereinafter, the correction processing (second correction processing) by the second correction means will be described with reference to FIGS. 11 and 12.

  FIG. 11 is a diagram illustrating the correction of the first posture using the third posture. In this embodiment, the posture in the three-dimensional space is actually processed. However, in FIG. 11 and FIG. 12, the posture in the two-dimensional plane will be described for easy understanding of the drawings. A vector v1 shown in FIG. 11 indicates the first posture in the spatial coordinate system. A vector v4 shown in FIG. 11 indicates the third posture in the spatial coordinate system. Since the position and orientation of the marker unit 6 are determined in advance, the orientation of the input device 8 can be relatively calculated based on the orientation and position of the marker in the image. Assuming that the third posture represents the correct posture, when the first posture is correctly calculated from the angular velocity, the vector v1 indicating the first posture indicates the third posture. It matches the vector v4. That is, when the first posture is not correctly calculated from the angular velocity, the vector v1 indicating the first posture and the vector v4 indicating the third posture are different as shown in FIG.

  The second correction process is performed by bringing the first posture (vector v1) closer to the third posture (vector v4) at a predetermined rate. FIG. 12 is a diagram illustrating the first posture after correction by the second correction processing. As shown in FIG. 12, the first posture after correction (vector v1 ′) brings the first posture before correction (vector v1) closer to the third posture (vector v4) at a predetermined rate. Obtained by.

  Here, depending on the orientation and position of the input device 8, the imaging unit may not be able to capture the marker unit 6. In this case, the second correction unit cannot perform the second correction process. Therefore, if the second correction unit performs correction so that the corrected first posture matches the third posture, the second correction processing is performed from a state where the second correction processing cannot be performed. There is a possibility that the first posture may change abruptly when transitioning to a state where it is possible to do so. Thus, if the first posture suddenly changes without conforming to the player's intention, the player feels uncomfortable with the operation (even if the corrected posture is accurate). In order to prevent such a sudden change, in the present embodiment, correction is performed so that the first posture approaches the third posture at a predetermined rate. As a result, it is possible to prevent the first posture from changing suddenly, thereby preventing the player from feeling uncomfortable with the operation. However, in other embodiments, such as when the input device 8 can be assumed to be used in a posture in which the imaging unit can always image the marker unit 6, the second correction unit is the corrected first unit. You may correct | amend so that this attitude | position may correspond with a 3rd attitude | position.

  In the present embodiment, the game apparatus 3 executes both the first correction process and the second correction process, but in other embodiments, one of the first correction process and the second correction process. May be executed only. In the present embodiment, the game apparatus 3 performs the first correction process first and the second correction process later, but either the first correction process or the second correction process is performed first. You may do it.

  As described above, according to the present embodiment, the attitude of the input device 8 calculated from the angular velocities detected by the gyro sensors 55 and 56 is corrected using the acceleration detected by the acceleration sensor 37, and imaging is performed. Correction is performed using the image captured by the means. Thereby, the error of the attitude calculated from the gyro sensor can be reduced, and the attitude of the input device 8 can be calculated more accurately.

  In addition, from the detection result of the acceleration sensor 37, the rotation around the gravity direction (rotation in the yaw direction) cannot be detected, and therefore the correction by the first correction unit cannot be performed in the yaw direction. However, the correction using the detection result of the acceleration sensor 37 has a feature that the posture of the input device 8 can be any posture (because acceleration can always be detected). On the other hand, if the marker unit 6 is not present in the imaging direction of the input device 8, the marker coordinates are not detected, and therefore the correction by the second correction unit cannot be performed depending on the attitude of the input device 8. However, the correction using the captured image has a feature that the posture (especially the posture in the roll direction) can be accurately calculated. In the present embodiment, the posture of the input device 8 can be calculated more accurately by performing two types of corrections having different features as described above.

[Details of processing in game device 3]
Next, details of processing executed in the game apparatus 3 will be described. First, main data used in the processing in the game apparatus 3 will be described with reference to FIG. FIG. 13 is a diagram illustrating main data stored in the main memory (the external main memory 12 or the internal main memory 11e) of the game apparatus 3. As shown in FIG. 13, a game program 60, operation data 62, and game processing data 67 are stored in the main memory of the game apparatus 3. In addition to the data shown in FIG. 13, the main memory stores data necessary for game processing, such as image data of various objects appearing in the game and data indicating various parameters of the objects.

  A part or all of the game program 60 is read from the optical disc 4 and stored in the main memory at an appropriate timing after the game apparatus 3 is turned on. The game program 60 includes a coordinate calculation program 61. The coordinate calculation program 61 is a program for calculating the attitude of the input device 8 and executing processing for calculating two-dimensional coordinates (on the screen of the television 2) based on the calculated attitude.

  The operation data 62 is operation data transmitted from the controller 5 to the game apparatus 3. As described above, since the operation data is transmitted from the controller 5 to the game apparatus 3 at a rate of once every 1/200 seconds, the operation data 62 stored in the main memory is updated at this rate. In the present embodiment, only the latest (last acquired) operation data may be stored in the main memory.

  The operation data 62 includes angular velocity data 63, acceleration data 64, marker coordinate data 65, and operation button data 66. The angular velocity data 63 is data indicating angular velocities detected by the gyro sensors 55 and 56 of the gyro sensor unit 7. Here, the angular velocity data 63 indicates the respective angular velocities around the three axes of XYZ shown in FIG. The acceleration data 64 is data indicating the acceleration (acceleration vector) detected by the acceleration sensor 37. Here, the acceleration data 64 represents a three-dimensional acceleration vector Va1 having as components the accelerations in the directions of the three axes of XYZ shown in FIG. In the present embodiment, the magnitude of the acceleration vector Va1 detected by the acceleration sensor 37 while the controller 5 is stationary is “1”. That is, the magnitude of gravitational acceleration detected by the acceleration sensor 37 is “1”.

  The marker coordinate data 65 is data indicating coordinates calculated by the image processing circuit 41 of the imaging information calculation unit 35, that is, the marker coordinates. The marker coordinates are expressed in a two-dimensional coordinate system (x′y ′ coordinate system shown in FIG. 17) for representing a position on a plane corresponding to the captured image. Note that, when the two markers 6R and 6L are imaged by the imaging element 40, two marker coordinates are calculated. On the other hand, if either one of the markers 6R and 6L is not located within the imageable range of the image sensor 40, only one marker is imaged by the image sensor 40, and only one marker coordinate is calculated. Further, when both of the markers 6R and 6L are not located within the image capturing range of the image sensor 40, the marker is not imaged by the image sensor 40, and the marker coordinates are not calculated. Therefore, the marker coordinate data 65 may indicate two marker coordinates, may indicate one marker coordinate, or may indicate that there is no marker coordinate.

  The operation button data 66 is data indicating an input state for each of the operation buttons 32a to 32i.

  The game process data 67 is data used in a game process (FIG. 14) described later. The game processing data 67 includes first posture data 68, acceleration magnitude data 69, correction degree data 70, correction amount vector data 71, correction matrix data 72, roll posture component data 73, yaw posture component data 74, pitch posture component. Data 75 and third attitude data 76 are included. In addition to the data shown in FIG. 13, the game processing data 67 includes various data (data indicating game parameters, etc.) used in the game processing.

上記行列Ｍ１は、所定の基準姿勢から現在の入力装置８の姿勢への回転を表す回転行列である。以下では、第１の姿勢を示す行列Ｍ１を、「第１姿勢行列Ｍ１」と呼ぶ。なお、第１姿勢行列Ｍ１により表される第１の姿勢は、入力装置８が存在する空間の所定位置を基準としたｘｙｚ座標系（上記空間座標系）における姿勢である。ここでは、ｘｙｚ座標系は、入力装置８がマーカ部６の正面に位置することを前提とし、入力装置８の位置からマーカ部６を向く方向をｚ軸正方向とし、鉛直上向き（重力方向の逆方向）をｙ軸正方向とし、入力装置８の位置からマーカ部６を向いた場合の左方向をｘ軸正方向とした座標系であるとする。ここでは、上記所定の基準姿勢は、マーカ部６の正面に位置する入力装置８の撮像方向がマーカ部６の中央を向き、かつ、コントローラ５のボタン面が鉛直上向きとなる姿勢（すなわち、入力装置８を基準としたＸ軸、Ｙ軸、Ｚ軸が、それぞれｘ軸、ｙ軸、ｚ軸の向きと一致する姿勢）であるとする。なお、本実施形態では、行列を用いて第１の姿勢を表現することとしたが、他の実施形態においては、第１の姿勢は、３次のベクトルまたは３つの角度によって表現されてもよい。 The first attitude data 68 is data indicating the first attitude calculated using the angular velocity data 63. In the present embodiment, the first posture is expressed by a 3 × 3 matrix M1 shown in the following equation (1).

The matrix M1 is a rotation matrix representing the rotation from a predetermined reference posture to the current input device 8 posture. Hereinafter, the matrix M1 indicating the first posture is referred to as a “first posture matrix M1”. Note that the first posture represented by the first posture matrix M1 is a posture in the xyz coordinate system (the spatial coordinate system) with reference to a predetermined position in the space where the input device 8 exists. Here, the xyz coordinate system is based on the premise that the input device 8 is located in front of the marker unit 6, and the direction from the input device 8 toward the marker unit 6 is the z-axis positive direction, and is vertically upward (in the gravitational direction). It is assumed that the coordinate system is such that the reverse direction) is the y-axis positive direction and the left direction when facing the marker unit 6 from the position of the input device 8 is the x-axis positive direction. Here, the predetermined reference posture is a posture in which the imaging direction of the input device 8 located in front of the marker unit 6 faces the center of the marker unit 6 and the button surface of the controller 5 is vertically upward (that is, input) Assume that the X axis, the Y axis, and the Z axis with respect to the device 8 are postures that coincide with the directions of the x axis, the y axis, and the z axis, respectively. In the present embodiment, the first posture is expressed using a matrix. However, in other embodiments, the first posture may be expressed by a cubic vector or three angles. .

  The acceleration magnitude data 69 is data indicating the magnitude (length) L of the acceleration vector Va1 indicated by the acceleration data 64.

  The correction degree data 70 is data indicating the degree of correction of the first attitude using the second attitude (correction degree A). The correction degree A is a value that can take a range of 0 ≦ A ≦ C1 (C1 is a predetermined constant of 0 <C1 ≦ 1). Although details will be described later, the corrected first posture becomes closer to the second posture as the correction degree A is larger.

  The correction amount vector data 71 is data indicating a vector indicating a correction amount for correcting the first posture (vector v3 shown in FIG. 9; hereinafter referred to as a correction amount vector). The correction amount vector Vg is calculated based on the vector Va2 representing the acceleration vector Va1 in the xyz coordinate system and the correction degree A.

  The correction matrix data 72 is data indicating a rotation matrix (referred to as a correction matrix) Ma used for correcting the first posture. That is, in the first correction process, the first posture is corrected by multiplying the first posture matrix M1 representing the first posture by the correction matrix Ma. The correction matrix Ma is calculated based on the vector Va2 and the correction amount vector Vg.

  The roll posture component data 73 is data indicating a posture component (roll posture component) M3r related to the roll direction among the posture components included in the third posture calculated from the image to be captured. The yaw posture component data 74 is data indicating the posture component (yaw posture component) M3y related to the yaw direction among the posture components included in the third posture, and the pitch posture component data 75 is the third posture information. Among the posture components included in the posture, this is data indicating a posture component (pitch posture component) M3p related to the pitch direction. Note that the roll direction, yaw direction, and pitch direction referred to here are rotation directions when the imaging direction (Z-axis positive direction) of the input device 8 is used as a reference. In the present embodiment, each of the posture components M3r, M3y, and M3p is represented by a 3 × 3 matrix as in the first posture.

  The third attitude data 76 is data indicating a third attitude calculated from the image to be imaged. In the present embodiment, the third posture is represented by a 3 × 3 matrix M3, like the first posture. Hereinafter, the matrix M3 indicating the third posture is referred to as a “third posture matrix M3”. In the present embodiment, since the marker coordinate data is transmitted as operation data from the input device 8, the third posture matrix M <b> 3 is calculated based on the marker coordinate data 65. Specifically, the third posture matrix M3 is obtained by combining the posture components M3r, M3y, and M3p.

  Next, details of processing performed in the game apparatus 3 will be described with reference to FIGS. FIG. 14 is a main flowchart showing the flow of processing executed in the game apparatus 3. When the power of the game apparatus 3 is turned on, the CPU 10 of the game apparatus 3 executes a startup program stored in a boot ROM (not shown), whereby each unit such as the main memory is initialized. Then, the game program stored on the optical disc 4 is read into the main memory, and the CPU 10 starts executing the game program. The flowchart shown in FIG. 14 is a flowchart showing processing performed after the above processing is completed.

  First, in step S1, the CPU 10 executes an initialization process relating to the game. In this initialization process, the values of various parameters used in the game process are initialized, a virtual game space is constructed, and player objects and other objects are placed at initial positions in the game space. Following step S1, the process of step S2 is executed.

  In step S2, the CPU 10 executes an initial posture setting process. Specifically, a predetermined value is set as the initial posture of the input device 8 in response to the player performing a predetermined operation (for example, an operation of pressing the A button 32d). Here, the reference posture is a posture in which the Z axis is parallel to the vertical direction and the imaging direction of the input device 8 faces the center of the marker unit 6 (the center of the markers 6R and 6L). It is desirable to perform the predetermined operation while holding the input device 8 so that the initial posture becomes the reference posture, but if the input device is close to a stationary state and the marker portion can be imaged, It is possible to calculate the initial posture. When a predetermined operation is performed, the CPU 10 stores data indicating a matrix representing the initial posture in the main memory as first posture data. After the above step S2, the processing loop of steps S3 to S9 is repeatedly executed while the game is executed. Note that one processing loop is executed once per frame time (for example, 1/60 seconds).

  In this embodiment, the initial posture setting process (step S2) is executed only once before the game starts (before the processing loop of steps S3 to S9 is executed). In the embodiment, the initial posture setting process may be executed at an arbitrary timing during the game. That is, the CPU 10 may execute the initial posture setting process in response to the player performing the predetermined operation during the game.

  In step S3, the CPU 10 acquires operation data. That is, operation data transmitted from the controller 5 is received via the wireless controller module 19. Then, angular velocity data, acceleration data, marker coordinate data, and operation button data included in the received operation data are stored in the main memory. Following step S3, the process of step S4 is executed.

  In step S4, the CPU 10 calculates the first posture based on the angular velocity data 63 stored in the main memory. Any method may be used to calculate the attitude of the input device 8 from the angular velocity. However, in the present embodiment, the first attitude is the previous first attitude (calculated in the previous processing loop). (First attitude) and the current angular velocity (the angular velocity acquired in the current processing loop). Specifically, the CPU 10 sets the posture obtained by rotating the previous first posture by unit time at the current angular velocity as the first posture. The previous first posture is indicated by the first posture data 68 stored in the main memory, and the current angular velocity is indicated by the angular velocity data 63 stored in the main memory. Data indicating the posture (3 × 3 matrix) calculated in step S4 is newly stored in the main memory as the first posture data 68. Following step S4, the process of step S5 is executed.

  In step S5, the CPU 10 executes the first correction process described above. The first correction process is a process of correcting the first posture using acceleration data. Hereinafter, the details of the first correction process will be described with reference to FIG.

  FIG. 15 is a flowchart showing the flow of the first correction process (step S5) shown in FIG. In the first correction process, first, in step S <b> 11, the CPU 10 calculates the magnitude L of the acceleration detected by the acceleration sensor 37. That is, the acceleration data 64 stored in the main memory is read, and the magnitude L is calculated for the acceleration vector Va1 indicated by the acceleration data 64. Data indicating the calculated magnitude L is stored as acceleration magnitude data 69 in the main memory. Following step S11, the process of step S12 is executed.

  In step S12, the CPU 10 determines whether or not the magnitude of the acceleration detected by the acceleration sensor 37 is zero. That is, the acceleration magnitude data 69 stored in the main memory is read, and it is determined whether or not the magnitude L indicated by the acceleration magnitude data 69 is zero. If the determination result of step S12 is negative, the process of step S13 is executed. On the other hand, when the determination result of step S12 is affirmative, the processes of subsequent steps S13 to S21 are skipped, and the CPU 10 ends the first correction process. Thus, in this embodiment, when the magnitude of the acceleration detected by the acceleration sensor 37 is 0, correction using the acceleration is not performed. This means that the direction of gravity cannot be calculated from the detection result of the acceleration sensor 37 when the magnitude of the acceleration is 0, and the subsequent steps S13 to S21 are performed if the magnitude of the acceleration vector is 0. The reason is that it is difficult.

  In step S <b> 13, the CPU 10 normalizes the acceleration vector Va <b> 1 detected by the acceleration sensor 37. That is, the acceleration data 64 stored in the main memory is read, and the acceleration vector Va1 indicated by the acceleration data 64 is corrected so that the magnitude is 1. The CPU 10 stores data indicating the normalized acceleration vector Va1 in the main memory. Following step S13, the process of step S14 is executed.

In step S <b> 14, the CPU 10 calculates the correction degree A indicating the degree of correction of the first posture in the first correction process. The correction degree A is calculated based on the magnitude L of the acceleration vector Va1 before normalization. Specifically, the CPU 10 reads acceleration magnitude data 69 stored in the main memory. Then, using the magnitude L indicated by the acceleration magnitude data 69, the correction degree A is calculated according to the following equation (2).
A = | L−1 | (2)
Data indicating the correction degree A calculated by the above equation (2) is stored as correction degree data 70 in the main memory. The correction degree A calculated by the above equation (2) is a value that is not a final value and is being calculated, and the final correction degree A is obtained by converting the value in the subsequent step S16. A value is obtained. Following step S14, the process of step S15 is executed.

  In step S15, the CPU 10 determines whether or not the correction degree A calculated in step S14 is smaller than a predetermined value R. The predetermined value R is determined in advance and is set to 0.4, for example. Here, as described above, in the present embodiment, the magnitude of the gravitational acceleration detected by the acceleration sensor 37 is “1”, and the correction degree A is the magnitude L of the acceleration vector Va1 and “1”. (The above formula (2)). Therefore, the case where the correction degree A is equal to or greater than the predetermined value R is a case where the magnitude L of the acceleration vector Va1 is separated from the magnitude of the gravitational acceleration by the predetermined value R or more. If the determination result of step S15 is affirmative, the process of step S16 is executed. On the other hand, when the determination result of step S15 is negative, the processes of subsequent steps S16 to S21 are skipped, and the CPU 10 ends the first correction process.

  As described above, in the present embodiment, only when the difference between the acceleration magnitude L detected by the acceleration sensor 37 and the gravitational acceleration magnitude (= 1) is smaller than a predetermined reference (predetermined value R). Correction is performed, and correction is not performed when the difference between the magnitude L and the magnitude of gravitational acceleration is equal to or greater than a predetermined reference. Here, in the state where the input device 8 is moved, in addition to the gravitational acceleration, the acceleration due to inertia caused by the movement of the input device 8 is detected by the acceleration sensor 37, so the magnitude of the detected acceleration vector Va1. L becomes a value different from “1”, and when the input device 8 is moved violently, the size L becomes a value far from “1”. Therefore, the case where the difference between the magnitude L and the magnitude of the gravitational acceleration is equal to or greater than a predetermined reference is estimated to be a case where the input device 8 is moved violently. On the other hand, when the input device 8 is moved violently, the acceleration vector Va1 detected by the acceleration sensor 37 contains many components other than gravitational acceleration (the acceleration component due to inertia), and therefore the acceleration vector Va1. The value of is presumed to be unreliable as a value indicating the direction of gravity. Therefore, the determination process of step S15 determines whether or not the input device 8 is moved violently, in other words, whether or not the value of the acceleration vector Va1 is reliable as a value indicating the direction of gravity. It is processing. In the present embodiment, if the value of the acceleration vector Va1 is not reliable as a value indicating the direction of gravity by the determination process in step S15, no correction is performed, and the value of the acceleration vector Va1 is a value indicating the direction of gravity. Corrections are made only when reliable. As a result, it is possible to prevent the first posture from being correctly corrected as a result of performing the correction on the first posture using the acceleration vector Va1 that is not reliable as a value indicating the direction of gravity.

In step S16, the CPU 10 converts the value of the correction degree A. In the present embodiment, the correction degree A is converted so that the correction degree A becomes closer to 1 as the magnitude L of the detected acceleration vector Va1 is closer to the magnitude of gravitational acceleration. Specifically, the CPU 10 reads the correction degree data 70 stored in the main memory, and converts the correction degree A indicated by the correction degree data 70 according to the following equations (3) to (5).
A2 = 1- (A1 / R) (3)
A3 = A2 × A2 (4)
A4 = A3 × C1 (5)
In the above formulas (3) to (5), the variable A1 is the correction degree before conversion (value indicated by the correction degree data 70 currently stored in the main memory), and the variable A4 is finally converted in step S16. Is the degree of correction. The above expression (3) is an expression for obtaining the corrected degree of correction A2 so that the degree of correction A1 before the conversion becomes closer to 1 as the magnitude of the gravitational acceleration (= 1) is closer. The above equation (4) is an equation for obtaining the corrected degree of correction A3 so that a larger weight is assigned as the correction degree A2 before conversion is closer to 1. The above equation (5) is an equation for adjusting the magnitude of the correction amount. That is, the correction amount of the constant C1 increases as the value of the constant C1 increases. The constant C1 is determined in advance and is set in a range of 0 <C1 ≦ 1 (for example, 0.03). Data indicating the correction degree A4 obtained by the conversion according to the above equations (3) to (5) is stored as new correction degree data 70 in the main memory. Following step S16, the process of step S17 is executed.

  In the present embodiment, the conversions according to the above formulas (3) to (5) are performed. However, in other embodiments, some or all of the conversions in the above formulas (3) to (5) are omitted. May be. However, when the calculation is such that the value of the correction degree is close to 0 so that the value of the acceleration vector is reliable due to omission of the equation or the like, in the equation (7) used in step S18 described later, the acceleration vector Va2 It is necessary to replace the gravity direction vector (0, -1, 0).

In step S17, the CPU 10 converts the acceleration vector Va1 expressed in the XYZ coordinate system into a value Va2 in the xyz coordinate system. The acceleration vector Va2 in the xyz coordinate system is calculated by converting the normalized acceleration vector Va1 using the first posture matrix M1 representing the first posture obtained in the previous frame. That is, the CPU 10 reads the data of the acceleration vector Va1 (normalized) stored in the main memory and the first attitude data 68 in step S13. Then, using the acceleration vector Va1 and the first posture matrix M1 indicated by the first posture data 68, an acceleration vector Va2 in the xyz coordinate system is calculated. More specifically, normalized acceleration vector Va1 = (nx, ny, nz), each element of the first posture matrix M1 is a variable shown in the above equation (1), and acceleration expressed in the xyz coordinate system If the vector Va2 = (vx, vy, vz), the acceleration vector Va2 is calculated according to the following equation (6).
vx = Xx * nx + Yx * ny + Zx * nz
vy = Xy * nx + Yy * ny + Zy * nz
vz = Xz * nx + Yz * ny + Zz * nz (6)
As shown in the above equation (6), the acceleration vector Va2 can be obtained by rotating the acceleration vector Va1 by the first posture matrix M1 that is a rotation matrix. The acceleration vector Va2 calculated in step S17 is stored in the main memory. Following step S17, the process of step S18 is executed.

In step S18, the CPU 10 calculates the correction amount vector Vg using the acceleration vector Va2 expressed in the xyz coordinate system and the correction degree A. The correction amount vector Vg is calculated using the degree of correction after the conversion in step S16 and a vector (0, -1, 0) indicating the vertical downward direction (gravity direction) of the xyz coordinate system. Specifically, the CPU 10 reads out the correction degree data 70 stored in the main memory, and uses the correction degree A indicated by the correction degree data 70, according to the following equation (7), the correction amount vector Vg = ( gx, gy, gz) is calculated.
gx = (0−vx) × A + vx
gy = (− 1−vy) × A + vy
gz = (0−vz) × A + vz (7)
As shown in the above equation (7), the correction amount vector Vg is a line segment connecting the end point of the acceleration vector Va2 to the end point of the gravity direction vector (0, -1, 0) as A: (1-A). It is a vector whose end point is a point to be internally divided. Accordingly, the correction amount vector Vg is closer to the gravity direction vector as the correction degree A is larger. The CPU 10 stores data indicating the correction amount vector Vg calculated by the above equation (7) as correction amount vector data 71 in the main memory. Following step S18, the process of step S19 is executed.

  In step S19, the CPU 10 normalizes the correction amount vector Vg calculated in step S18. That is, the correction amount vector data 71 stored in the main memory is read, and the vector indicated by the correction amount vector data 71 is normalized. Then, the data indicating the normalized vector is stored as new correction amount vector data 71 in the main memory. The correction amount vector Vg calculated in step S19 corresponds to the vector v3 shown in FIG. Following step S19, the process of step S20 is executed.

  In step S20, the CPU 10 calculates a correction matrix Ma for correcting the first posture. The correction matrix Ma is calculated based on the acceleration vector Va2 expressed in the xyz coordinate system and the correction amount vector Vg normalized in step S19. Specifically, the CPU 10 reads the acceleration vector Va2 and the correction amount vector data 71 stored in the main memory in step S17. Then, a rotation matrix for rotating the acceleration vector Va2 so as to coincide with the correction amount vector Vg is calculated, and the calculated rotation matrix is set as a correction matrix Ma. That is, the correction matrix Ma is a rotation matrix that performs the rotation of the angle Δθ shown in FIG. Data indicating the correction matrix Ma calculated in step S20 is stored in the main memory as correction matrix data 72. Following step S20, the process of step S21 is executed.

  In step S21, the CPU 10 corrects the first posture matrix M1 indicating the first posture with the correction matrix Ma. Specifically, the CPU 10 reads the first attitude data 68 and the correction matrix data 72 stored in the main memory. Then, the first attitude matrix M1 indicated by the first attitude data 68 is converted by the correction matrix Ma indicated by the correction matrix data 72 (the product of the first attitude matrix M1 and the correction matrix Ma is calculated). The converted first posture matrix M1 indicates the corrected first posture. That is, the process of step S21 corresponds to the process of rotating the vector v1 shown in FIG. 10 by the angle Δθ. The CPU 10 stores data indicating the converted first attitude matrix M1 as new first attitude data 68 in the main memory. After step S21, the CPU 10 ends the first correction process.

  As described above, in the first correction process, the correction amount vector Vg is calculated between the acceleration vector detected by the acceleration sensor 37 and the gravity direction vector (vector G shown in FIG. 8) (steps S18 and S19). ), The first posture is corrected by the correction amount (correction matrix Ma. Angle Δθ shown in FIG. 9) represented by the correction amount vector Vg (step S21). As a result, the first attitude calculated from the gyro sensors 55 and 56 (vector v1 or angle θ1 shown in FIG. 8) is corrected to approach the second attitude (angle θ2 shown in FIG. 8) determined by the acceleration sensor 37. It can be performed. By performing such correction, the first posture can be corrected so as to have a more accurate value.

  In the first correction process, as the acceleration vector Va1 is reliable (the smaller the difference between the magnitude L of the acceleration vector Va1 and the magnitude of gravitational acceleration), the value of the correction degree A increases. The first posture is corrected so as to be closer to the second posture. That is, the more reliable the acceleration vector Va1, the larger the correction amount, and the second posture is strongly reflected in the corrected first posture. As described above, according to the present embodiment, the correction amount in the first correction process is determined according to the reliability of the acceleration sensor vector Va1, and therefore the correction amount can be appropriately determined according to the reliability. The attitude of the input device 8 can be calculated more accurately.

  In the present embodiment, the correction amount vector Vg calculated in step S18 is a point that internally divides the line segment connecting the end point of the acceleration vector Va2 to the end point of the gravity direction vector into A: (1-A). Is the vector that approaches the gravity direction vector as the value of the correction degree A increases. In another embodiment, depending on the method of calculating the correction degree A, the correction amount vector Vg is a point that internally divides the end point of the gravity direction vector and the end point of the acceleration vector Va2 into (1-A): A. The end point vector may be a vector that approaches the gravity direction vector as the correction degree A is smaller. At this time, in step S20, a rotation matrix for rotating the correction amount vector Vg so as to coincide with the direction of gravity may be calculated, and the calculated rotation matrix may be used as the correction matrix Ma. This also makes it possible to perform correction as in the present embodiment.

  Returning to the description of FIG. 14, in step S <b> 6 subsequent to step S <b> 5, the CPU 10 executes the second correction process described above. The second correction process is a process of correcting the first posture using the marker coordinate data. Hereinafter, the details of the second correction process will be described with reference to FIG.

  FIG. 16 is a flowchart showing the flow of the second correction process (step S6) shown in FIG. In the first correction process, first, in step S <b> 31, the CPU 10 determines whether or not the marker unit 6 is imaged by the imaging unit (imaging device 40) of the input device 8. The determination in step S31 can be made by referring to the marker coordinate data 65 stored in the main memory. Here, when the marker coordinate data 65 indicates two marker coordinates, it is determined that the marker unit 6 is captured, and when the marker coordinate data 65 indicates only one marker coordinate, or there is no marker coordinate. When it shows, it determines with the marker part 6 not being imaged. If the determination result of step S31 is affirmative, the processes of subsequent steps S32 to S37 are executed. On the other hand, when the determination result of step S31 is negative, the processes of the subsequent steps S32 to S37 are skipped, and the CPU 10 ends the second correction process. As described above, when the marker unit 6 is not imaged by the image sensor 40, the attitude of the input device 8 cannot be calculated using data obtained from the image sensor 40. In this case, the second correction is performed. No correction is performed in the process.

  In step S32, the CPU 10 calculates a roll posture component M3r based on the marker coordinate data. The roll posture component M3r is calculated based on the orientation of the marker unit 6 in the captured image, that is, based on the slope of the line connecting the two marker coordinates indicated by the marker coordinate data 65. Hereinafter, an example of a method for calculating the roll posture component M3r will be described with reference to FIG.

  FIG. 17 is a diagram illustrating two-dimensional coordinates corresponding to a captured image. As shown in FIG. 17, in the present embodiment, a two-dimensional coordinate system (x′y ′ coordinate system) for representing a position in a captured image has a range of the captured image of −1 ≦ x ′ ≦ 1, − 1 ≦ y ′ ≦ 1. The x′y ′ coordinate system captures images when the input device 8 is in a reference posture (an orientation in which the imaging direction of the input device 8 faces the center of the marker unit 6 and the button surface of the controller 5 is vertically upward). In the image, it is assumed that the vertically downward direction is the positive direction of the y ′ axis and the right direction is the positive direction of the x ′ axis. Also, point P1 and point P2 shown in FIG. 17 indicate the positions of the marker coordinates, and point P3 is a midpoint between point P1 and point P2. A vector v10 shown in FIG. 17 is a vector having a point P1 as a start point and a point P2 as an end point.

上式（８）によって算出された行列を示すデータは、ロール姿勢成分データ７３としてメインメモリに記憶される。ステップＳ３２の次にステップＳ３３の処理が実行される。 In order to calculate the roll posture component M3r, the CPU 10 first reads the marker coordinate data 65, and calculates the vector v10 from the two marker coordinates indicated by the marker coordinate data 65. Further, a vector (hx, hy) obtained by normalizing the vector v10 is calculated. The vector (hx, hy) is directed in the positive x-axis direction when the input device 8 is in the reference posture, and the direction changes according to the rotation of the input device 8 in the roll direction. Since the vector (hx, hy) corresponds to the posture in the roll direction, the roll posture component M3r can be calculated based on the vector (hx, hy). Specifically, the CPU 10 calculates the roll posture component M3r according to the following equation (8).

Data indicating the matrix calculated by the above equation (8) is stored in the main memory as roll posture component data 73. Following step S32, the process of step S33 is executed.

  In step S33, the CPU 10 calculates a yaw posture component M3y based on the marker coordinate data. The yaw posture component M3y is calculated based on the orientation and position of the marker unit 6 in the captured image. Hereinafter, an example of a method for calculating the yaw posture component M3y will be described with reference to FIG.

  First, the CPU 10 reads the marker coordinate data 65 and calculates the midpoint between the two marker coordinates indicated by the marker coordinate data 65. In the present embodiment, the position of the middle point is used as the position of the marker unit 6. Further, the CPU 10 rotates the calculated midpoint coordinates by the rotation angle related to the roll direction of the input device 8 (in the direction opposite to the rotation direction of the input device 8) around the origin of the x′y ′ coordinate system. Calculated coordinates (px, py) are calculated. In other words, the coordinates of the midpoint are rotated around the origin so that the vector (hx, hy) faces the positive x-axis direction. If the input device 8 is at the same position as the marker unit 6 in the horizontal direction (x-axis direction) (that is, the front position of the marker unit 6), the coordinates (px, py) can be used to calculate the attitude related to the yaw direction.

Next, the CPU 10 based on the coordinates (px, py) after the rotation of the midpoint and the angle (limit angle) θy ′ in the yaw direction when the marker unit 6 is located at the end in the x′-axis direction. Then, the rotation angle θy related to the yaw direction is calculated. Here, the limit angle θy ′ and the x-coordinate value px ′ after the rotation of the midpoint when the limit angle θy1 is reached can be obtained in advance. Therefore, the rotation angle θy related to the yaw direction can be calculated using the fact that the ratio between px and px ′ is equal to the ratio between θy and θy ′. Specifically, the rotation angle θy related to the yaw direction can be calculated by the following equation (9).
θy = px × θy ′ / px ′ (9)
When the horizontal length of the marker unit 6 is ignored, the limit angle θy ′ can be set to ½ of the angle of view of the controller 5 and the value of px ′ can be set to “1”.

上式（１０）によって算出された行列を示すデータが、ヨー姿勢成分データ７４としてメインメモリに記憶される。以上のステップＳ３３の次にステップＳ３４の処理が実行される。 Finally, the CPU 10 calculates a rotation matrix for performing the rotation of the angle θy calculated by the above equation (9) as the yaw attitude component M3y. Specifically, the yaw posture component M3y is calculated by the following equation (10).

Data indicating the matrix calculated by the above equation (10) is stored in the main memory as yaw attitude component data 74. Following step S33, the process of step S34 is executed.

  In step S34, the CPU 10 combines the roll posture component M3r and the yaw posture component M3y. That is, the roll attitude component data 73 and the yaw attitude component data 74 are read from the main memory, and the roll attitude component M3r and the yaw attitude component M3y indicated by the data 73 and 74 are integrated. Following step S34, the process of step S35 is executed.

  In step S35, the CPU 10 calculates a pitch posture component M3p based on the first posture. Although different from the processing of the present embodiment, the pitch attitude component M3p can also be calculated based on the y coordinate value of the coordinates (px, py) in the same manner as the yaw attitude component M3y. . However, in the method of calculating the attitude in the yaw direction (pitch direction) using the coordinates (px, py), the input device 8 is at the same position as the marker unit 6 in the horizontal direction (vertical direction in the case of the pitch direction). This method is based on the assumption. In the game system 1 of the present embodiment, it is considered that the player operates the input device 8 at a position substantially in front of the marker unit 6 (TV 2) in the horizontal direction. It is possible to calculate the attitude in the yaw direction by the method of step S33 on the assumption that “they are at the same position”. On the other hand, it is conceivable that the player stands and operates the input device 8 or sits down and operates the input device 8, and the position of the marker unit 6 is also arranged on the upper side of the screen of the television 2. It is also possible to arrange them in For this reason, in the game system 1 of the present embodiment, it is not necessarily assumed that “the input device 8 is in the same position as the marker unit 6 in the vertical direction”. The posture may not be calculated.

上式（１２）によって算出された行列を示すデータが、ピッチ姿勢成分データ７５としてメインメモリに記憶される。以上のステップＳ３５の次にステップＳ３６の処理が実行される。 Therefore, in the present embodiment, the first posture is used as it is for the pitch posture component M3p (thus, in the second correction process, the pitch direction is not corrected). Specifically, the CPU 10 reads the first attitude data 68 from the main memory. Then, using each element of the first posture matrix M1 indicated by the first posture data 68, the rotation angle θp with respect to the pitch direction is calculated according to the following equation (11).
cos (θp) = (Zx × Zx + Zz × Zz) ^1/2
sin (θp) = Zy (11)
Variables Zx, Zy, and Zz in the above equation (11) are elements of the first posture matrix M1 shown in the above equation (1). The first posture matrix M1 used here is the first posture matrix M1 after the first correction processing is performed in the current processing loop. Further, the CPU 10 calculates a matrix of the pitch posture component M3p according to the following equation (12) using cos (θp) and sin (θp) calculated by the above equation (11).

Data indicating the matrix calculated by the above equation (12) is stored in the main memory as pitch attitude component data 75. Following step S35, the process of step S36 is executed.

  In step S36, the CPU 10 calculates a third posture based on the posture components in the roll direction, the yaw direction, and the pitch direction. The third attitude is obtained by further synthesizing the pitch attitude component M3p with the synthesis result of the roll attitude component M3r and the yaw attitude component M3y. Specifically, the CPU 10 reads the pitch posture component data 75 from the main memory, and integrates the pitch posture component M3p indicated by the pitch posture component data 75 in the matrix calculated in step S34. Data indicating the calculated matrix is stored in the main memory as third attitude data 76. Following step S36, the process of step S37 is executed.

In step S37, the CPU 10 corrects the first attitude using the third attitude. The correction in step S37 is performed by bringing the first posture matrix M1 closer to the third posture matrix M3 at a predetermined ratio (the following constant C2). The CPU 10 reads the first attitude data 68 and the third attitude data 76 from the main memory. Then, correction is performed according to the following equation (13) using the first posture matrix M1 indicated by the first posture data 68 and the third posture matrix M3 indicated by the third posture data 76.
M1 = (M3-M1 ′) × C2 + M1 ′ (13)
In the above equation (13), the variable M1 ′ is the first posture matrix before correction. The constant C2 is preset in the range of 0 <C2 ≦ 1, and is set to 0.1, for example. Data indicating the corrected first posture matrix M1 calculated by the above equation (13) is stored in the main memory as new first posture data 68. After step S37, the CPU 10 ends the second correction process.

  As described above, in the second correction process, the third posture is calculated from the captured image (marker coordinates), and the first posture is corrected so as to approach the third posture. By this correction, the first posture can be corrected so as to have a more accurate value. In the present embodiment, the third posture is calculated from the captured image only for the posture in the roll direction and the yaw direction. However, as described above, the third posture is calculated from the captured image also in the pitch direction. In other embodiments, the third orientation may be calculated from the captured image in the roll direction, the yaw direction, and the pitch direction. In the second correction process, the third posture may be calculated in at least one of the roll direction, the yaw direction, and the pitch direction.

  Returning to the description of FIG. 14, in step S <b> 7 following step S <b> 6, the CPU 10 calculates two-dimensional coordinates based on the corrected first posture. Although details will be described later, in the present embodiment, the two-dimensional coordinates are used as input by the player in a game process (step S8) described later. Hereinafter, with reference to FIG. 18 and FIG. 19, the detail of step S7 is demonstrated.

  FIG. 18 is a diagram in which an input device and a predetermined plane are virtually arranged in a predetermined virtual space. In the present embodiment, as shown in FIG. 18, the CPU 10 defines a three-dimensional virtual space, and calculates coordinates assuming that the input device 8 and a predetermined plane Q are virtually arranged. The plane Q corresponds to the screen of the television 2. The CPU 10 calculates the coordinates of the position on the plane Q indicated by the direction of the Z-axis (Z-axis vector VZ shown in FIG. 18) of the input device 8 in the virtual space. That is, the coordinates of the intersection R between the line segment obtained by extending the Z-axis vector VZ representing the attitude of the input device 8 in the virtual space and the predetermined plane Q in the virtual space are calculated. The “line segment obtained by extending the Z-axis vector VZ” means a line segment that is parallel to the Z-axis vector VZ and passes through the Z-axis vector VZ. Although details will be described later, in this embodiment, a cursor or the like is displayed at a position on the screen corresponding to the calculated coordinates. Thereby, the position of the cursor on the screen is controlled according to the posture of the input device 8, and the player can move the cursor by an operation for changing the posture of the input device 8.

  As shown in FIG. 18, in this embodiment, the position of the virtual space is expressed by the above-described spatial coordinate system (xyz coordinate system). As a result, the first attitude data 68 can be used as it is as data indicating the attitude of the input device 8, so that calculation in the virtual space can be easily performed. Further, the position on the plane Q is expressed by the X′Y ′ coordinate system. At this time, the plane Q is set to be parallel to the xy plane of the spatial coordinate system, the x axis of the spatial coordinate system and the X ′ axis of the plane Q are parallel, and the y axis of the spatial coordinate system and the Y ′ of the plane Q are set. The X′Y ′ coordinate system is set so that the axes are parallel (FIG. 18).

  In the present embodiment, it is assumed that the player uses the input device 8 at a position substantially in front of the screen of the television 2 and the position of the input device 8 does not change in the virtual space. In other words, the CPU 10 performs the process without changing the position of the input device 8 because only the attitude of the input device 8 changes in the virtual space. As a result, the position on the plane Q (the position of the intersection R) can be uniquely determined from the attitude of the input device 8.

  Next, a method for calculating the position on the plane Q (the position of the intersection R) will be described in detail. FIG. 19 is a view of the virtual space shown in FIG. 18 as viewed from the y-axis positive direction side to the negative direction side. In FIG. 19, the length from a point S representing the position of the input device 8 to a point (projection point) T obtained by projecting the point S onto the plane Q is “L”. The length from the projection point T to the intersection point R with respect to the X ′ component is “Wx”. In the present embodiment, the value of the length L is set by the player at a predetermined timing (for example, timing when game processing is started). The main memory stores data indicating the value of the length L set by the player (referred to as length data).

The Z-axis vector VZ is determined by a first posture matrix M1 that represents the posture of the input device 8 (first posture). That is, the Z-axis vector VZ is a vector (= (Zx, Zy, Zz)) in which each element of the third row of the first posture matrix M1 is in turn an x component, a y component, and a z component. As is clear from FIG. 19, the ratio between the length Wx and the length L is equal to the ratio between the x component (Zx) of the Z-axis vector VZ and the z component (Zz) of the Z-axis vector VZ. Therefore, based on this relationship, the length Wx can be calculated from the x component Zx, the z component Zz, and the known length L of the known vector VZ. The length Wx can be calculated by (14).
Wx = L × Zx / Zz (14)
Similarly to the length Wx of the X ′ component, the length Wy of the Y ′ component from the projection point T to the intersection R can be calculated by the following equation (15).
Wy = L × Zy / Zz (15)

  If the lengths Wx and Wy are obtained, two-dimensional coordinates indicating the position of the intersection point R on the plane Q can be calculated. In the present embodiment, the position of the projection point T is the origin of the X′Y ′ coordinate system. At this time, the two-dimensional coordinates of the intersection R are (Wx, Wy).

  The specific processing in step S7 will be described. First, the CPU 10 reads the first attitude data 68 and the length data from the main memory. Then, Zx, Zy, and Zz included in the first posture matrix M1 indicated by the first posture data 68 and the length L indicated by the length data are substituted into the above equations (14) and (15). To calculate the lengths Wx and Wy. As a result, the two-dimensional coordinates (Wx, Wy) of the intersection R on the plane Q are obtained. The CPU 10 stores data indicating two-dimensional coordinates in the main memory 32. Following step S7, the process of step S8 is executed.

  As described above, according to step S7 described above, a virtual space in which the input device 8 and the predetermined plane Q are arranged is set, and on the plane Q indicated by the direction of the predetermined axis (Z axis) of the input device 8 in the virtual space. 2D coordinates can be obtained from the attitude of the input device 8.

  In the above embodiment, the player can set the length L. Here, as is apparent from the above equations (14) and (15), the value of the calculated two-dimensional coordinates (Wx, Wy) can be changed by adjusting the length L. That is, by adjusting the length L, the amount of change in the two-dimensional coordinates (Wx, Wy) with respect to the change in the direction of the Z-axis vector VZ (that is, the change in the attitude of the input device 8) can be adjusted. it can. Specifically, the amount of change increases as the value of the length L increases. As a result, the cursor moves greatly only by slightly changing the attitude of the input device 8. On the other hand, as the value of the length L becomes smaller, the change amount becomes smaller. As a result, even if the posture of the input device 8 is greatly changed, the cursor moves only slightly. As described above, in the above embodiment, the player himself / herself can adjust the operational feeling of the input device 8 by causing the player to set the length L. For example, the player may set the length L relatively small when it is necessary to finely operate the cursor, and the length L is relatively set when an operation to move the cursor greatly is required. Should be set larger. In another embodiment, the length of L may be a predetermined constant.

  In another embodiment, the length L may be calculated by the game apparatus 10 using a predetermined method. For example, the CPU 10 may calculate an actual distance from the input device 8 to the screen of the television 2 and set the calculated distance as the length L. The actual distance can be calculated using, for example, the length between the two markers 6R and 6L in the captured image captured by the image sensor 40 or the size of the marker 6R or 6L. Furthermore, if the plane Q is set so that the position and size of the plane Q in the virtual space correspond to the position and size of the screen of the television 2 in the real space (for example, the player sets the size of the television screen). The cursor is displayed at a position on the screen of the television 2 corresponding to the two-dimensional coordinates (Wx, Wy). According to this, the cursor can be displayed at a position (on the screen) indicated by the input device 8.

  As a method for obtaining the two-dimensional coordinates using the input device 8, the two-dimensional coordinates can be obtained using the marker coordinates (marker coordinate data 65 shown in FIG. 13) described above. That is, since the marker coordinates change in accordance with the position pointed to by the input device 8, the position on the screen pointed to by the input device 8 can be calculated based on the marker coordinates. Therefore, if the marker portion 6 is present, the two-dimensional coordinates can be calculated using the marker coordinates. However, when the marker unit 6 cannot be used, or in an input device that is not equipped with an image sensor, the two-dimensional coordinates cannot be obtained by photographing with the marker unit 6. The method according to the present embodiment can obtain two-dimensional coordinates even when the imaging device 40 of the input device 8 is not capturing the marker unit 6, and therefore can be used even in an input device without the marker unit or imaging device. Is.

  In the present embodiment, the two-dimensional coordinates are calculated based on the first posture corrected by the first and second correction processes. Here, in another embodiment, the two-dimensional coordinates may be calculated based on the attitude of the input device 8 calculated by an arbitrary method, and the first and second correction processes may not be performed. Good.

  Moreover, in the said embodiment, the aspect which the player uses the Z-axis direction of the input device 8 toward the screen of the television 2 is assumed, and the Z-axis direction of the input device 8 faces the screen (marker part 6) of the television 2. In this case, the plane Q is arranged in the Z-axis direction. However, as described above, since the two-dimensional coordinates can be obtained even if the imaging element 40 does not capture the marker portion 6, the player points the Z-axis direction of the input device 8 in an arbitrary direction. It is possible to use. For example, the CPU 10 may set the plane Q in the Z-axis direction of the input device 8 at the time of the predetermined operation in response to the player performing a predetermined operation (for example, an operation of pressing an operation button). . According to this, since the plane Q can be arranged at an appropriate position according to the orientation of the input device 8 in the virtual space, the input device 8 can be in any orientation at the time of the predetermined operation. Two-dimensional coordinates can be calculated. That is, the player can use the input device 8 in an arbitrary direction.

  In the present embodiment, the game apparatus 3 calculates a two-dimensional coordinate so as to take the origin (0, 0) when the Z-axis direction of the input apparatus 8 faces the horizontal direction (referred to as a first method). )It was adopted. Here, the game apparatus 3 may calculate the two-dimensional coordinates by a second method described later, instead of the first method.

  Returning to the description of FIG. 14, in step S8 following step S7, the CPU 10 executes a game process using the two-dimensional coordinates calculated in step S7. In the present embodiment, as the game process, a process of displaying a cursor at a position on the screen of the television 2 corresponding to the two-dimensional coordinates is executed. Accordingly, the cursor can be displayed at a position corresponding to the posture of the input device 8, and the player can perform an operation of moving the cursor by changing the posture of the input device 8. The game process in step S8 may be any process as long as it is a process that reflects the two-dimensional coordinates as input values in the game result. For example, it may be a process of moving an object in the virtual game space to a position in the game space corresponding to the two-dimensional coordinates, or depending on the size and direction of the two-dimensional vector represented by the two-dimensional coordinates. It may be a process of controlling and displaying the object so as to move at a high speed. Following step S8, the process of step S9 is executed.

  In step S9, the CPU 10 determines whether or not to end the game. The determination in step S9 is made based on, for example, whether or not the game has been cleared, whether or not the game is over, and whether or not the player has given an instruction to stop the game. If the determination result of step S9 is negative, the process of step S3 is executed again. Thereafter, the processing loop of steps S3 to S9 is repeatedly executed until it is determined in step S9 that the game is to be ended. On the other hand, if the determination result of step S9 is affirmative, the CPU 10 ends the game process shown in FIG. This is the end of the description of the game process.

  As described above, in the present embodiment, the first posture of the input device 8 is calculated from the angular velocities detected by the gyro sensors 55 and 56 (step S4), and the first posture is subjected to the first correction process (S5). And it corrected by the 2nd correction process (S6). Since the game process is executed using the corrected first attitude (steps S7 and S8), the CPU 10 can execute the game process based on the accurate attitude of the input device 8. According to this, for example, since the posture of the input device 8 can be accurately reflected on the position of the cursor on the screen, the operability of the game operation can be improved.

[Second method for calculating coordinates]
Hereinafter, the second method for calculating the position on the screen from the attitude of the input device 8 will be described with reference to FIGS. In the above embodiment, the game process may be executed using the following second method instead of the first method described above. In the following description, the coordinates on the plane Q in the virtual space are referred to as “first coordinates”, the coordinates representing the position on the screen calculated from the first coordinates are referred to as “screen coordinates”, and the marker The coordinates (details will be described later) representing the position on the screen calculated from the coordinates are referred to as “second coordinates”.

  FIG. 20 is a diagram showing a comparison between the first method and the second method. 20, in the real space, the input device 8 is positioned below the center of the screen of the television 2, and the position on the screen indicated by the Z-axis direction of the input device 8 (hereinafter referred to as “(input device) indicated position”). ”) Is the center of the screen as an example (see the upper left column in FIG. 20).

  In the first method described above, the coordinate system of the first coordinate (the X′Y ′ coordinate system on the plane Q in the virtual space) always has the projection point T obtained by projecting the position of the input device 8 on the plane Q as the origin. O. That is, the first coordinates are calculated so as to take the value of the origin (0, 0) when the Z-axis direction of the input device 8 is in the horizontal direction (when the Z-axis direction indicates the projection point T). Therefore, as shown in FIG. 20, when the Z-axis direction of the input device 8 is slightly upward, the position R of the first coordinate is above the origin O (see the upper middle column in FIG. 20). As a result, the position U of the screen coordinates calculated from the first coordinates is above the center of the screen (see the upper right column in FIG. 20).

  In the case shown in FIG. 20, in the first method, the Z-axis direction of the input device 8 actually points to the center of the screen (in real space), whereas the calculated position on the screen is from the center. Will also be in an upper position. As described above, in the first method, a deviation occurs between the position indicated by the user using the input device 8 (indicated position of the input device) and the position of the calculated screen coordinates, and the user feels uncomfortable. There was a risk of holding.

  Actually, the situation in which the television 2 is installed (whether it is placed on the floor or on the stand, etc.) and the state of the user who uses the input device 8 (whether the input device 8 is used standing up) It is considered that the height of the input device 8 with respect to the screen varies depending on whether the user is sitting and using. If this height changes, the relationship between the indicated position and orientation of the input device 8 changes, and the orientation of the input device 8 when the indicated position of the input device 8 is at the center of the screen also changes. However, in the first method, since the position of the origin of the X′Y ′ coordinate system is fixed without taking the indicated position of the input device 8 into consideration, the position on the screen cannot be calculated appropriately, and the input The user may feel uncomfortable with the operation using the device 8.

  Therefore, in the second method, the game apparatus 3 uses the first coordinate calculated by the first method when the Z-axis direction of the input device 8 is in the posture pointing to the center of the screen (the indicated position is the screen position). Correction is performed so that a predetermined value (specifically, (0, 0)) is obtained at the center. Specifically, the game apparatus 3 first determines whether or not the Z-axis direction of the input apparatus 8 is in an attitude that points to the center of the screen. This determination can be made by referring to the marker coordinates described above. That is, the game apparatus 3 calculates the coordinates (second coordinates) indicating the indicated position on the screen based on the marker coordinates, and determines whether the second coordinates represent the center (near the center) of the screen. To do.

  As a result of the above determination, when the second coordinate is the center of the screen, the game apparatus 3 determines a correction amount (offset amount) for correcting the first coordinate. As for the offset amount, the position of the screen coordinate corresponding to the first coordinate calculated by the first method when the second coordinate becomes the center of the screen matches the position of the second coordinate. To be determined. Specifically, the offset amount is determined to be a correction amount such that the first coordinate at the time point becomes the origin value (0, 0). Note that determining the offset amount means moving the X′Y ′ coordinate system on the plane Q. In the example of FIG. 20, since the input device 8 faces upward from the horizontal direction, the X′Y ′ coordinate system has a position on the plane Q where the projection point T is the origin (the dotted line shown in FIG. 20). The position is moved so as to move upward (see the lower middle column in FIG. 20). As described above, as a result of setting the first coordinate as the origin value, the position U on the screen corresponding to the first coordinate becomes the center position of the screen (see the upper right column in FIG. 20). Therefore, the indicated position (second coordinate position) of the input device 8 and the calculated position on the screen coincide with each other at the center of the screen. Thus, according to the second method, it is possible to prevent a deviation between the position pointed by the user using the input device 8 and the calculated position, and the user feels uncomfortable. Can be prevented.

  In the present embodiment, the game apparatus 3 corrects the first coordinate only for the Y ′ component of the X′Y ′ coordinate system. That is, the first coordinates (and screen coordinates) are corrected only in the vertical direction of the screen, and are not corrected in the horizontal direction. In the present embodiment, the height (position in the vertical direction) of the input device 8 with respect to the screen cannot be determined only from the attitude of the input device 8, whereas the position of the input device 8 in the horizontal direction with respect to the screen. This is because it can be determined only from the attitude of the input device 8, and the “displacement” as described in FIG. However, in other embodiments, the first coordinates may be corrected in the left-right direction as in the up-down direction.

  Next, with reference to FIGS. 21 to 25, details of processing executed by the game apparatus 3 when the second method is adopted will be described. FIG. 21 is a diagram showing main data stored in the main memory of the game apparatus 3 when the second method is employed. In FIG. 21, the same data as the data shown in FIG. 13 is denoted by the same reference numerals as those in FIG.

  As shown in FIG. 21, in the second method, in addition to the data shown in FIG. 13, first coordinate data 77, second coordinate data 78, offset amount data 79, and screen coordinate data 80 are stored in the main memory. Is done.

  The first coordinate data 77 indicates the first coordinates calculated based on the attitude of the input device 8. That is, the first coordinate data 77 indicates the coordinates of the position on the plane Q in the virtual space.

  The second coordinate data 78 indicates coordinates (second coordinates) representing a position on the screen calculated based on the marker coordinates. The second coordinate represents the position on the screen indicated by the Z-axis direction of the input device 8. The second coordinates ideally represent the position of the intersection of the straight line extending the Z axis of the input device 8 and the screen, but may be any as long as it represents the vicinity of the position of the intersection.

  The offset amount data 79 indicates an offset amount for correcting the first coordinate. In the present embodiment, the coordinate obtained by moving the first coordinate calculated by the first method by the offset amount becomes the corrected first coordinate.

  The screen coordinate data 80 indicates coordinates (screen coordinates) representing a position on the screen calculated based on the first coordinates. In the second method, the screen coordinates are calculated based on the first coordinates after being corrected according to the offset amount.

  FIG. 22 is a main flowchart showing a flow of processing executed by the game apparatus 3 when the second method is adopted. In FIG. 22, the same process steps as those shown in FIG. 14 are denoted by the same step numbers as those in FIG.

  When the second method is adopted, the process of step S41 is executed after the initial posture setting process of step S2. In step S41, the CPU 10 executes an offset determination process. The offset determination process is a process for determining an offset amount for correcting the first coordinates. Hereinafter, the details of the offset determination process will be described with reference to FIG.

  FIG. 23 is a flowchart showing the flow of the offset determination process (step S41) shown in FIG. In the offset determination process, first, in step S51, the CPU 10 acquires operation data. The process of step S51 is the same as the process of step S3 described above. Following step S51, the process of step S52 is executed.

  In step S <b> 52, the CPU 10 calculates second coordinates representing the designated position of the input device 8. The second coordinates are calculated based on the marker coordinates acquired in step S51. That is, the CPU 10 reads the marker coordinate data 65 from the main memory, and calculates the second coordinates based on the marker coordinates. Any method may be used to calculate the second coordinates from the marker coordinates. For example, the coordinates (px, py) calculated in step S33 described above can be calculated from the marker coordinates, and the second coordinates can be calculated based on the coordinates (px, py). Specifically, the positive / negative of the x component of the coordinates (px, py) is reversed so that a predetermined ratio (for example, the length of the captured image in the x-axis direction matches the horizontal length of the screen of the television 2). By performing the scaling by the ratio), it is possible to obtain the indicated position in the horizontal direction of the screen. In addition, the sign of the y component of the coordinates (px, py) is reversed to a predetermined ratio (for example, the ratio of the length of the captured image in the y-axis direction to the length of the screen 2 of the television 2). By performing the scaling, it is possible to obtain a designated position in the vertical direction of the screen. Regarding the vertical direction, it is preferable to appropriately set the offset amount depending on whether the marker unit 6 is installed above or below the television 2. The CPU 10 stores data indicating the calculated second coordinates in the main memory as the second coordinate data 78. Following step S52, the process of step S53 is executed.

  In step S <b> 53, the CPU 10 calculates the first attitude of the input device 8. The processing in step S53 is the same as the processing in steps S4 to S6 described above. In the present embodiment, the process for calculating the first posture is executed for each frame (for each processing loop of steps S51 to S58). However, this process is performed when the determination result of step S54 is positive. It may be executed only in Following step S53, the process of step S54 is executed.

  In step S <b> 54, the CPU 10 displays a predetermined setting screen and a cursor on the screen of the television 2. The setting screen is a screen including a button image for the user to give a predetermined instruction. FIG. 24 is a diagram illustrating an example of the setting screen. In FIG. 24, the predetermined instruction is an instruction for adjusting the cursor speed (sensitivity adjustment), and a button image 81 indicating an instruction for increasing the cursor speed and a button image 82 indicating an instruction for decreasing the cursor speed are displayed. Is done. Note that the cursor to be adjusted here is not the currently displayed cursor 83 but a cursor whose position is determined based on the first coordinates (screen coordinates) used in a later game operation. The contents of the predetermined instruction may be anything, and other adjustments relating to the cursor (for example, adjustment of the followability of the cursor to the operation of the input device 8, adjustment of play set for the cursor, For example, adjustment of a camera shake prevention function), an instruction to end a setting operation on the setting screen, or an instruction to start a game.

  A cursor 83 is displayed on the screen of the television 2 together with the setting screen. That is, the CPU 10 reads the second coordinate data 78 from the main memory and displays the cursor 83 at the position indicated by the second coordinate data 78. Thus, the position of the cursor 83 displayed in step S54 is calculated based on the marker coordinates, and the cursor 83 is controlled by a method based on the marker coordinates. Although not shown in FIG. 23, the CPU 10 is designated when the button image 81 or 82 is designated (a predetermined button of the input device 8 is pressed while the cursor 83 points to the button image). Adjustment processing is performed according to the button image. Following step S54, the process of step S55 is executed.

  Returning to the description of FIG. 23, in step S55, the CPU 10 determines whether or not the cursor 83 is positioned at the center in the vertical direction of the screen. This determination can be made using the second coordinates. Specifically, the CPU 10 reads the second coordinate data 78 from the main memory, and determines whether or not the position indicated by the second coordinate data 78 is within the display area of the button image 81 or 82. That is, the CPU 10 determines whether or not the cursor 83 is positioned within the display area of the button image 81 or 82. If the determination result of step S55 is affirmative, the process of step S56 is executed. On the other hand, if the determination result of step S55 is negative, the processes of steps S56 and S57 are skipped, and the process of step S58 described later is executed.

  In step S55, it is not determined whether or not the second coordinate (cursor 83) is strictly located at the center of the screen, but the second coordinate is located near the center (within the button image display area). It is determined whether or not the coordinates are located. This is because the object of the present invention is to prevent the user from feeling uncomfortable, and it is not necessary to exactly match the indicated position of the input device and the position of the screen coordinates. Further, if it is strictly determined whether or not the second coordinate is located at the center of the screen in step S55, the determination result in step S55 is less likely to be affirmative, and the offset amount may not be determined. Because.

  In step S56, the CPU 10 calculates the first coordinates based on the first posture. The calculation process in step S56 is the same as the calculation process in step S7 described above. That is, the CPU 10 calculates the first coordinates according to the first method based on the first attitude calculated in step S53. The calculated first coordinates are stored in the main memory 32 as first coordinate data 77. Following step S56, the process of step S57 is executed.

  In step S57, the CPU 10 determines an offset amount based on the first coordinates calculated in step S56. FIG. 25 is a diagram illustrating a method for determining an offset amount. Note that the posture of the input device 8 shown in FIG. 25 is the posture at the time when the indicated position of the input device 8 is centered in the vertical direction of the screen (when the determination result in Step S55 is Yes). The origin O is the origin of the X′Y ′ coordinate system in the first method, and is the position of the projection point T described above. Point R indicates the position of the first coordinate calculated in step S56.

  The offset amount is determined so that the Y ′ component of the first coordinate at the time point becomes the origin value (0). That is, with respect to the vertical direction, the position R of the first coordinate is determined to be the position of the origin. Therefore, the offset amount is determined to be a value “−d” obtained by inverting the sign of the value d of the Y ′ component of the first coordinate (see FIG. 25). The CPU 10 reads the first coordinate data 77 from the main memory, and determines a value obtained by inverting the sign of the Y ′ component of the first coordinate indicated by the first coordinate data 77 as the offset amount. Data indicating the determined offset amount is stored in the main memory as offset amount data 79. In the first coordinate correction process (step S42), which will be described later, the first coordinate calculated by the first method is corrected to the coordinate moved by the offset amount. Therefore, it can be said that the process of calculating the offset amount in step S57 is a process of moving the X′Y ′ coordinate system by the offset amount (in the direction in which the sign of the offset amount is reversed). Following step S57, step S58 is executed.

  In step S58, the CPU 10 determines whether or not to end the setting screen. The determination in step S58 is made based on, for example, whether or not the user has given an instruction to end the setting screen. If the determination result of step S58 is negative, the process of step S51 is executed again. Thereafter, the processes of steps S51 to S58 are repeatedly executed until the determination result of step S58 is positive. On the other hand, when the determination result of step S58 is affirmative, the CPU 10 ends the offset determination process.

  As described above, in the offset determination process, the Y ′ component of the first coordinate calculated when the input device 8 points to the center of the screen (with respect to the vertical direction) is set to the origin value (0). An offset amount is determined. Here, in order to determine the offset amount in the offset determination process, a state in which the input device 8 points to the center (with respect to the vertical direction) of the screen must occur. In the present embodiment, in order to cause such a state, button images 81 and 82 for giving a predetermined instruction are displayed in the center (in the vertical direction) (FIG. 24). According to this, since the user naturally operates the input device 8 so as to be in the above state in order to give a predetermined instruction, it is possible to cause the user to perform the offset amount determination operation naturally without making the user aware of it. it can. The predetermined instruction may be any instruction. However, in order to ensure that the process for determining the offset amount is performed reliably, the user must always perform an instruction (for example, a setting screen). An instruction to end or an instruction to start a game is preferable. Further, as in the present embodiment, by giving an instruction regarding the adjustment of the cursor as the predetermined instruction, the offset amount can be determined simultaneously with the adjustment of the cursor. According to this, it is possible to determine the offset amount without giving the user a feeling that “an extra operation is being performed (to determine the offset amount)”.

  Returning to the description of FIG. 22, after the offset determination process of step S <b> 41, the process of step S <b> 3 similar to the above-described embodiment is executed. Further, after step S3, the same processes of steps S4 to S7 as in the above-described embodiment are executed. Note that the process of step S7 is a process of calculating the first coordinates by the first method. In the second method, the process of step S42 is executed after the process of step S7.

  In step S42, the CPU 10 corrects the first coordinates calculated in step S7 according to the offset amount. That is, the CPU 10 reads the first coordinate data 77 and the offset amount data 78 from the main memory, and corrects the first coordinates to a value obtained by adding the offset amount D to the Y ′ component. Data indicating the corrected first coordinate is stored in the main memory as new first coordinate data 77.

In the present embodiment, the first coordinate calculation process (step S7) by the first method and the first coordinate correction process (step S42) are described as separate processes. However, the CPU 10 The two processes may be executed together. That is, in step S7, the CPU 10 may calculate the first coordinates according to the following equation (16) instead of the above equation (15).
Wy = L × Zy / Zz + D (16)
In the above equation (16), the variable D indicates an offset amount and is a value (−d) obtained by inverting the sign of the Y ′ component (d) of the first coordinate calculated in step S56. When the above equation (16) is used, the first coordinates can be calculated in a form including correction processing.

  Following step S42, the processes of steps S43 and S44 are executed. The process of steps S43 and S44 is the same as the process of step S8 described above, but in FIG. 22, the process of step S8 is divided into two and will be described as steps S43 and S44.

  In step S <b> 43, the CPU 10 calculates coordinates (screen coordinates) representing the position on the screen of the television 2. The screen coordinates are calculated based on the first coordinates corrected in step S42. That is, the CPU 10 reads the first coordinate data 77 from the main memory, and calculates screen coordinates based on the first coordinates indicated by the first coordinate data 77. The screen coordinates are calculated to be the center of the screen when the first coordinates are the origin value (0, 0). The screen coordinate is a direction in which the direction from the center of the screen to the screen coordinate corresponds to the direction from the origin to the first coordinate, and the length from the center of the screen to the screen coordinate is the first coordinate from the origin. It is calculated to be a length corresponding to the length up to. Therefore, in a state where the Z axis of the input device 8 is directed to the center of the screen, the screen coordinates represent the center of the screen (the screen coordinates coincide with the designated position), and the direction of the Z axis of the input device 8 is determined from this state. The screen coordinates move in the direction corresponding to the changed direction. Data indicating the screen coordinates calculated as described above is stored in the main memory as screen coordinate data 80. Following step S43, the process of step S44 is executed.

  In step S44, the CPU 10 performs a game process using the screen coordinates calculated in step S43. The game process in step S43 may be any process as long as the process is based on the screen coordinates. For example, a process of displaying a cursor or a game object at the position of the screen coordinates may be performed, or a process of moving the object according to the positional relationship between the position of the object and the screen coordinates may be performed. When any image is displayed on the screen coordinates, the user can visually recognize the position of the screen coordinates. Therefore, the first method has a high possibility of having the above-mentioned uncomfortable feeling. Therefore, the second method is particularly effective in such a case. After step S44, the process of step S9 similar to that in the above embodiment is executed. This is the end of the description of the second method.

  As described above, in the second method, when the game apparatus 3 displays the setting screen, the game apparatus 3 detects a state in which the input device 8 points to the center with respect to the vertical direction of the screen (step 54). An offset amount is determined using the first coordinates at (step S57). In the subsequent game processing, the first coordinates calculated by the first method are corrected according to the offset amount (step S42). Therefore, in a state where the input device 8 points to the center in the vertical direction of the screen, the Y ′ component of the first coordinate is 0, and as a result, the screen coordinate represents the center position of the screen in the vertical direction. Become. Therefore, according to the second method, when the user points to the center of the screen with the input device 8, the screen coordinates represent the center of the screen. It can be calculated appropriately.

  In the above embodiment, when the designated position is the center of the screen, the position of the screen coordinates matches the designated position. On the other hand, when the designated position is another position, the position of the screen coordinates does not necessarily match the designated position depending on the length L or the like. This is because if the position of the screen coordinates and the designated position coincide with each other at the center of the screen, the user will hardly feel any sense of incongruity, and the position of the screen coordinates differs from the designated position at the other positions. This is because there are almost no disadvantages. Further, it is considered that the merit of determining the screen coordinates in accordance with the attitude of the input device 8 (the merit that the degree of change of the screen coordinates with respect to the attitude change can be appropriately set) is larger than the above demerit.

  When the posture at the time when the input device 8 points to the center of the screen is set as the reference posture, the processing for determining the origin at the time, in other words, the posture of the input device 8 when the first coordinate takes the origin. It can be said that the process of step S57 which is a process to determine is a "process to determine the reference posture". The direction from the origin to the corrected first coordinate corresponds to the direction of change (rotation direction) from the reference posture to the current posture, and the distance from the origin to the corrected first coordinate is This corresponds to the amount of change (rotation amount) from the reference posture to the current posture. Therefore, the process of calculating the corrected first coordinate (step S7 and step S742) is “the first coordinate so as to be a value corresponding to the direction and amount of change from the reference posture to the current posture. It can be said that this is a process of calculating “. As described above, in the second method, (a) the process of determining the reference posture at the time when the input device 8 points to the center of the screen, and (b) the direction and amount of change from the reference posture to the current posture. A process of calculating the corresponding first coordinates (and thus the screen coordinates) is executed. Therefore, the second method calculates the screen coordinates so that the reference posture always has a specific value (in this embodiment, the value at the center of the screen) by the processes (a) and (b). This prevents the user from feeling uncomfortable.

[Modification]
(Modification regarding the second method)
In the above-described embodiment, in the process before the start of the game (steps S2 and S3), the cursor is not displayed according to the attitude of the input device 8 (first cursor) but is displayed at the second coordinate position based on the marker coordinates. A cursor (second cursor) is used. Thus, the game apparatus 3 may use two types of cursors with different calculation methods. If the two types of cursors do not match at any position on the screen (the state of the input device 8 does not match when the positions of the two types of cursors match), the cursor moves from one cursor to the other. At the time of switching, the user feels that the operation method of the cursor is completely changed, so that the user may feel uncomfortable and feel that the operability is poor. On the other hand, in the above embodiment, the two types of cursors match at the center of the screen (the state of the input device 8 when the cursor is positioned at the center of the screen matches). Therefore, there is little discomfort when switching between the two types of cursors, and the user can be prevented from being confused with respect to the cursor operation.

  Moreover, in the said embodiment, since the offset amount regarding a 1st cursor is determined when a user performs operation using the said 2nd cursor, the behavior of the 2nd cursor currently displayed corresponds to the determination of an offset amount. There is no change. Therefore, the game apparatus 3 can determine the offset amount without causing the user to feel uncomfortable due to a sudden change in the behavior of the cursor.

  In other embodiments, the game apparatus 3 does not necessarily use the second cursor when determining the offset amount. For example, the game apparatus 3 displays a message “Please point the controller at the center of the screen and press the A button” (the cursor is not displayed), and when the A button is pressed, the processing of steps S56 and S57 is executed. You may make it do. Even in this case, the screen coordinates are located at the center of the screen in a state in which the user feels that “the input device 8 points to the center of the screen”. Can be reduced. In addition, the game apparatus 3 does not need to display a screen that suggests determining the offset amount, and may determine the offset amount when, for example, a game image is displayed.

  In the above embodiment, the game apparatus 3 executes the process of determining the offset amount (step S57) only before the process of using the screen coordinates as an input (step S44). Here, in another embodiment, when the process using the screen coordinates as an input is repeatedly executed, the process of determining the offset amount (again) may be executed between the times when the process is repeatedly executed. For example, in the case of the above embodiment, the process for determining the offset amount may be executed while the process loop of steps S3 to S9 is repeatedly executed during the game. For example, the game apparatus 3 may execute a process for determining the offset amount at a timing when the setting screen is displayed during the game, may execute the process at a predetermined time interval, or may be instructed by the user. The processing may be executed at the timing. As described above, when the state of the user holding the input device changes during the game by re-determining the offset amount during the game (for example, when the game is standing and playing but sitting in the middle of the game) However, an appropriate offset amount can be determined again. Note that when the process for re-determining the offset amount is executed, the offset amount changes, thereby causing a change in screen coordinates not intended by the user, and the user may not be able to perform the game operation well. For this reason, the game apparatus 3 does not execute the process of re-determining the offset amount as much as possible during the period in which the process using the screen coordinates as an input is being performed (for example, the period in which the game operation is performed using the cursor). Is preferred. Further, when executing the processing, it is preferable to execute it in a scene where the influence of the change in the screen coordinates is small (for example, during a game pause or during the display of a menu screen).

  In the above embodiment, the game apparatus 3 calculates the first coordinates from the attitude of the input device 8 using the plane Q set in the virtual space, and calculates the screen coordinates from the first coordinates. Here, the method for calculating the screen coordinates from the attitude of the input device 8 may be any method as long as the position of the screen coordinates changes according to the attitude. For example, the game apparatus 3 may directly calculate the screen coordinates from the attitude of the input device 8 without calculating the first coordinates (in other words, the first coordinates are coordinates representing a position on the screen). May be.) Specifically, the game apparatus 3 sets the screen coordinates to be the center position of the screen when the input device 8 is in a predetermined reference posture, and has a size proportional to the rotation angle from the reference posture. Thus, the screen coordinates may be calculated so as to represent the position moved in the rotation direction from the reference posture (the screen coordinates at the center position). Further, for example, the game apparatus 3 may directly calculate the screen coordinates from the attitude of the input device 8 by setting the X′Y ′ coordinates so as to coincide with the coordinate system of the screen coordinates.

  In the above embodiment, the game apparatus 3 indirectly corrects the screen coordinates by correcting the first coordinates using the offset amount. However, in other embodiments, the game coordinates are changed to the screen coordinates. You may make it correct | amend directly. Specifically, the game apparatus 3 may calculate screen coordinates from the first coordinates in step S56, and may calculate an offset amount related to the screen coordinates in step S57. At this time, in steps S42 and S43, the game apparatus 3 calculates screen coordinates from the first coordinates calculated by the first method, and corrects the calculated screen coordinates according to the offset amount, thereby finally What is necessary is just to obtain the correct screen coordinates.

  In the above embodiment, the game apparatus 3 sets the offset amount with respect to the first coordinate, that is, corrects the first coordinate by moving the X′Y ′ coordinate system on the plane Q. (FIGS. 20 and 25). Here, in another embodiment, the first coordinate may be corrected by rotating the X′Y ′ coordinate system around the position of the input device 8 in the virtual space. FIG. 26 is a diagram illustrating a modification of the correction method in the second method. FIG. 26 is a side view of the plane Q in the virtual space. The plane Q indicated by the dotted line is the same plane as the plane Q used in the first method, and the plane Q ′ indicated by the solid line is corrected. This is the later plane Q. In the modification shown in FIG. 26, the CPU 10 executes the following processing instead of the processing in steps S56 and S57 shown in FIG. That is, the CPU 10 sets the corrected X′Y ′ coordinate system based on the attitude of the input device 8 (first attitude M1). The X′Y ′ coordinate system is set on a plane Q ′ that is the origin at a position separated by a length L in the direction of the Z-axis vector VZ of the input device 8 and that is perpendicular to the Z-axis vector VZ. Further, during the game, the CPU 10 calculates the first coordinates represented by the X′Y ′ coordinate system on the plane Q ′ instead of the processing of steps S7 and S42 shown in FIG. According to the modified example, since the distance from the position of the input device 8 in the virtual space is constant, the first coordinates (screen coordinates) can be calculated more accurately. For example, when the screen of the television 2 is actually installed tilted from the vertical direction (for example, when the television 2 is installed on the ceiling with the screen facing downward from the horizontal), the screen coordinates are calculated more accurately. can do. On the other hand, in the correction method in the above embodiment, the X′Y ′ coordinate system is set in a plane fixed with respect to the position of the input device 8, so that the correction calculation is simplified. Therefore, when the situation where the screen of the television 2 is installed with a large inclination from the vertical direction is not assumed, or when priority is given to reducing the processing load of the CPU 10, it is preferable to employ the correction method in the above embodiment.

(Modification regarding imaging means)
In the above-described embodiment, the game apparatus 3 uses a captured image (marker coordinates) obtained by an imaging unit included in the input apparatus 8 in order to detect that the input apparatus 8 is in a direction toward the center of the screen. Here, as long as the game apparatus 3 can know the direction of the input apparatus 8 when viewed from a predetermined position in the space, the game apparatus 3 may perform the detection using other information instead of the captured image. . For example, when a camera is installed around the television 2, the game apparatus 3 may perform the detection using a captured image obtained by capturing the input device 8 with the camera. Any method may be used for the detection. For example, a device that generates a signal such as a radio wave or an ultrasonic wave is installed around the input device 8. In this case, the direction of the input device 8 when viewed from a predetermined position in the space may be detected by detecting the signal.

(Modification regarding attitude calculation of input device 8)
In the above embodiment, the game apparatus 3 calculates the attitude of the input device 8 using the detection results obtained by the gyro sensors 55 and 56, the acceleration sensor 37, and the image sensor 40 (steps S3 to S6). Here, the method for calculating the attitude of the input device 8 may be any method. For example, the attitude of the input device 8 may be calculated using only the detection result of the gyro sensor, or may be calculated using only the detection result of the acceleration sensor.

  In the above-described embodiment, the case where a three-dimensional posture is calculated using a gyro sensor that detects angular velocities around three axes has been described as an example. However, the present invention is also applied to a case where a two-dimensional posture is calculated. It is possible to apply. For example, if it is premised that the rotation in the roll direction is not performed, the two-dimensional posture related to the rotation in the pitch direction and the yaw direction may be calculated by detecting an angular velocity about two axes with a two-axis gyro sensor. . Then, the screen coordinates may be calculated from the two-dimensional posture.

  In another embodiment, the second correction process may be executed only when it is estimated that the input device 8 is imaging the marker unit 6. Specifically, before executing the second correction process, the CPU 10 determines whether or not the input device 8 (imaging unit) faces a direction in which the marker unit 6 can be imaged. This determination can be performed using the first posture or the second posture. For example, the first posture (or the second posture) may be determined as to whether the imaging direction of the input device 8 is the same or opposite to the direction from the input device 8 to the marker unit 6. In addition, the first posture used for the above determination may be the first posture in which the first and second correction processes are performed in the previous processing loop, or may be calculated in the current processing loop. The first posture in which one correction process is performed may be used.

  As a result of the above determination, when it is determined that the input device 8 faces the direction in which the marker unit 6 can be imaged, the CPU 10 performs the second correction process and faces the direction in which the marker unit 6 can be imaged. If it is determined that there is not, the second correction process is skipped. Note that an object that is not the marker unit 6 (for example, an electric lamp in a room or sunlight outside a window) may be erroneously detected as the marker unit 6, and the third posture is obtained using the marker coordinates obtained by the erroneous detection. When is calculated, correct correction cannot be performed even if the second correction process is performed using the third posture. On the other hand, by performing the above determination process, it is possible to prevent the second correction process from being performed using the third posture calculated from the marker coordinates obtained by the erroneous detection. As a result, the second correction process can be performed more accurately.

  As described above, the present invention, for the purpose of appropriately calculating the position on the screen using the attitude of the input device, etc., for example, a game device or a game program for performing a game process using the calculated position as an input It can be used as

DESCRIPTION OF SYMBOLS 1 Game system 2 Television 3 Game device 4 Optical disk 5 Controller 6 Marker part 7 Gyro sensor unit 8 Input device 10 CPU
11c GPU
11e internal main memory 12 external main memory 63 angular velocity data 64 acceleration data 65 marker coordinate data 68 first posture data 76 third posture data 77 first coordinate data 78 second coordinate data 79 offset amount data 80 screen coordinate data

Claims (23)

A coordinate calculation device that calculates coordinates representing a position on a display screen based on an attitude of an input device,
Direction acquisition means for acquiring information indicating the direction of the input device when viewed from a predetermined position in a predetermined space;
Attitude calculating means for calculating the attitude of the input device in the predetermined space;
First coordinate calculating means for calculating a first coordinate for determining a position on the display screen based on an attitude of the input device;
Second coordinate calculation means for calculating second coordinates on the display screen based on the direction of the input device acquired by the direction acquisition means;
The difference between the position determined by the first coordinates calculated when the direction of the input device acquired by the direction acquisition means becomes a predetermined direction and the position represented by the second coordinates calculated at the time Correction amount determining means for determining a correction amount according to
A coordinate calculation apparatus comprising correction means for correcting the first coordinate by the correction amount. Processing execution means for executing predetermined information processing in which the position on the display screen is used as an input;
The correction amount determining means determines the correction amount in response to the direction of the input device becoming a predetermined direction before the predetermined information processing is executed,
The first coordinate calculation means calculates first coordinates while the predetermined information processing is repeatedly executed,
The coordinate calculation apparatus according to claim 1, wherein the correction unit corrects the first coordinate according to the correction amount every time the first coordinate calculation unit calculates the first coordinate. An instruction image display means for displaying a predetermined instruction image at a position represented by the second coordinates on the display screen;
When the instruction image falls within a predetermined range on the screen, the correction amount determination means includes a first coordinate calculated when the instruction image enters the predetermined range, and a predetermined value within the predetermined range. The coordinate calculation apparatus according to claim 1, wherein the correction amount is determined based on a second coordinate corresponding to the position. Button image display means for displaying a button image for a user to give a predetermined instruction in the predetermined range;
The coordinate calculation apparatus according to claim 3, wherein the correction amount determination unit executes the process of determining the correction amount only when the button image is displayed.   The first coordinate calculation unit calculates a coordinate value corresponding to a change direction and a change amount from a predetermined posture to the posture calculated by the posture calculation unit as a first coordinate. The coordinate calculation apparatus according to any one of the above. The first coordinate calculation means calculates, as the first coordinate, a coordinate representing a position of an intersection between a line segment extending from a predetermined reference point in a vector direction indicating the attitude of the input device and a predetermined plane;
The correction means moves the coordinate system of the first coordinate so that the first coordinate calculated when the direction of the input device becomes the predetermined direction becomes the predetermined reference value. Correct the coordinates,
6. The position calculation unit according to claim 1, further comprising a position calculation unit that calculates a coordinate representing a position on the display screen based on the first coordinate corrected by the correction unit. Coordinate calculation device.   The coordinate calculation apparatus according to claim 6, wherein the correction unit performs correction by moving a coordinate system of a first coordinate on the predetermined plane.   The coordinate calculation apparatus according to claim 6, wherein the correction unit performs correction by rotating a coordinate system of a first coordinate around the predetermined reference point. The first coordinate calculating means calculates a two-dimensional coordinate as the first coordinate;
The coordinate calculation apparatus according to claim 1, wherein the correction unit corrects only one component of the first coordinate. The input device has an imaging means,
10. The information acquisition apparatus according to claim 1, wherein the direction acquisition unit acquires information indicating a position of a predetermined imaging target in an image captured by the imaging unit as information indicating the direction of the input device. The coordinate calculation apparatus described in 1. The input device includes an acceleration sensor and / or an angular velocity sensor,
The coordinate calculation apparatus according to any one of claims 1 to 10, wherein the attitude calculation unit calculates an attitude of the input device based on a detection result of an acceleration sensor and / or an angular velocity sensor.   The coordinate calculation apparatus according to claim 11, wherein the attitude calculation unit calculates an attitude of the input device based on an angular velocity detected by the angular velocity sensor and corrects the attitude based on an acceleration detected by the acceleration sensor. . The input device has an imaging means,
The posture calculation means calculates the posture of the input device based on the angular velocity detected by the angular velocity sensor, and based on the position of a predetermined imaging target in the image captured by the imaging means, with at least the gravity direction as an axis. The coordinate calculation apparatus according to claim 11, wherein the posture is corrected with respect to the rotation performed. A coordinate calculation device that calculates coordinates representing a position on a display screen based on an attitude of an input device,
Direction acquisition means for acquiring the direction of the input device when viewed from a predetermined position in a predetermined space;
Attitude calculating means for calculating the attitude of the input device in the predetermined space;
Reference determining means for determining, as a reference posture, the posture of the input device when the direction of the input device acquired by the direction acquiring device becomes a predetermined direction;
The posture calculation means calculates the first coordinates for determining the position on the display screen so as to be a predetermined reference value when the input device assumes the reference posture, and from the reference posture. A first coordinate calculation means for calculating a value corresponding to the direction and amount of change up to the posture;
A second coordinate calculating means for calculating a second coordinate on the display screen based on a direction of the input device;
The reference determination unit is a coordinate calculation device that determines the posture of the input device at the time when the second coordinate becomes a predetermined value as the reference posture. The first coordinate is a coordinate representing a position of an intersection of a line segment extending from a predetermined reference point in the direction of a vector indicating the attitude of the input device and a predetermined plane;
The reference determining means sets a coordinate system on the predetermined plane so that the first coordinate becomes the reference value when the reference posture is set;
The first coordinate calculation means calculates first coordinates expressed using the coordinate system set by the reference determination means,
The coordinate calculation apparatus according to claim 14 , further comprising a position calculation unit that calculates a position on the display screen based on the first coordinate calculated by the first coordinate calculation unit. The coordinate calculation apparatus according to claim 15 , wherein the reference determination unit sets the predetermined plane on a plane fixedly set with respect to the reference point. The coordinate calculation apparatus according to claim 15 , wherein the reference determination unit sets the predetermined plane so that a distance from the reference point is constant. A coordinate calculation program executed in a computer of a coordinate calculation device that calculates coordinates representing a position on a display screen based on an attitude of an input device,
Direction acquisition means for acquiring information indicating the direction of the input device when viewed from a predetermined position in a predetermined space;
Attitude calculating means for calculating the attitude of the input device in the predetermined space;
First coordinate calculating means for calculating a first coordinate for determining a position on the display screen based on an attitude of the input device;
Second coordinate calculation means for calculating second coordinates on the display screen based on the direction of the input device acquired by the direction acquisition means;
The difference between the position determined by the first coordinates calculated when the direction of the input device acquired by the direction acquisition means becomes a predetermined direction and the position represented by the second coordinates calculated at the time Correction amount determining means for determining a correction amount according to
A coordinate calculation program for causing the computer to function as correction means for correcting the first coordinate by the correction amount. A coordinate calculation program executed in a computer of a coordinate calculation device that calculates coordinates representing a position on a display screen based on an attitude of an input device,
Direction acquisition means for acquiring the direction of the input device when viewed from a predetermined position in a predetermined space;
Attitude calculating means for calculating the attitude of the input device in the predetermined space;
Reference determining means for determining, as a reference posture, the posture of the input device when the direction of the input device acquired by the direction acquiring device becomes a predetermined direction;
The posture calculation means calculates the first coordinates for determining the position on the display screen so as to be a predetermined reference value when the input device assumes the reference posture, and from the reference posture. First coordinate calculation means for calculating a value corresponding to the direction and amount of change up to the posture;
Causing the computer to function as second coordinate calculating means for calculating second coordinates on the display screen based on the direction of the input device;
The reference calculation unit is a coordinate calculation program for determining, as the reference posture, the posture of the input device when the second coordinate becomes a predetermined value. A coordinate calculation system for calculating coordinates representing a position on a display screen based on an attitude of an input device,
Direction acquisition means for acquiring information indicating the direction of the input device when viewed from a predetermined position in a predetermined space;
Attitude calculating means for calculating the attitude of the input device in the predetermined space;
First coordinate calculating means for calculating a first coordinate for determining a position on the display screen based on an attitude of the input device;
Second coordinate calculation means for calculating second coordinates on the display screen based on the direction of the input device acquired by the direction acquisition means;
The difference between the position determined by the first coordinates calculated when the direction of the input device acquired by the direction acquisition means becomes a predetermined direction and the position represented by the second coordinates calculated at the time Correction amount determining means for determining a correction amount according to
A coordinate calculation system comprising correction means for correcting the first coordinates by the correction amount. A coordinate calculation system for calculating coordinates representing a position on a display screen based on an attitude of an input device,
Direction acquisition means for acquiring the direction of the input device when viewed from a predetermined position in a predetermined space;
Attitude calculating means for calculating the attitude of the input device in the predetermined space;
Reference determining means for determining, as a reference posture, the posture of the input device when the direction of the input device acquired by the direction acquiring device becomes a predetermined direction;
The posture calculation means calculates the first coordinates for determining the position on the display screen so as to be a predetermined reference value when the input device assumes the reference posture, and from the reference posture. A first coordinate calculation means for calculating a value corresponding to the direction and amount of change up to the posture;
A second coordinate calculating means for calculating a second coordinate on the display screen based on a direction of the input device;
The reference determination unit is a coordinate calculation system that determines, as the reference posture, the posture of the input device when the second coordinate becomes a predetermined value. A coordinate calculation method executed by a coordinate calculation system for calculating coordinates representing a position on a display screen based on an attitude of an input device,
A direction acquisition step of acquiring information representing the direction of the input device when viewed from a predetermined position in a predetermined space;
An attitude calculating step for calculating an attitude of the input device in the predetermined space;
A first coordinate calculating step for calculating a first coordinate for determining a position on the display screen based on an attitude of the input device;
A second coordinate calculating step of calculating a second coordinate on the display screen based on the direction of the input device acquired in the direction acquiring step;
The difference between the position determined by the first coordinates calculated when the direction of the input device acquired in the direction acquisition step becomes a predetermined direction and the position represented by the second coordinates calculated at the time A correction amount determination step for determining a correction amount according to
A coordinate calculation method comprising a correction step of correcting the first coordinate by the correction amount. A coordinate calculation method executed by a coordinate calculation system for calculating coordinates representing a position on a display screen based on an attitude of an input device,
A direction acquisition step of acquiring the direction of the input device when viewed from a predetermined position in a predetermined space;
An attitude calculating step for calculating an attitude of the input device in the predetermined space;
A reference determining step for determining, as a reference posture, the posture of the input device when the direction of the input device acquired by the direction acquiring step is a predetermined direction;
The first coordinate for determining the position on the display screen is calculated by the posture calculation step so as to have a predetermined reference value when the input device assumes the reference posture and from the reference posture. A first coordinate calculation step for calculating a value corresponding to the direction of change and the amount of change up to the determined posture,
A second coordinate calculating step of calculating a second coordinate on the display screen based on a direction of the input device;
In the reference determination step, a coordinate calculation method of determining, as the reference posture, the posture of the input device when the second coordinates become a predetermined value.

Images