JP2011138295A - Apparatus, program and system for processing information and display range control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To control a display range to be displayed at a display device by using the output of a gyro-sensor. <P>SOLUTION: An information processing apparatus inputs an angular rate detected by a gyro-sensor included in an input device, and displays an image on a display device. The information processing apparatus calculates an orientation of the input device based on the angular rate. Then, the information processing apparatus calculates coordinates at an intersection between a line extending from a predetermined position in a predetermined space toward a vector representing the orientation and a predetermined plane within the predetermined space. A display range of a display target that is to be displayed on the display device is controlled based on the coordinates. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an information processing device, an information processing program, an information processing system, and a display range control method, and more specifically, information for controlling a display range displayed on a display device using a detection result of a gyro sensor as an input. The present invention relates to a processing device, an information processing program, an information processing system, and a display range control method.

  Conventionally, a game apparatus using an acceleration sensor and a gyro sensor has been considered. For example, Patent Document 1 describes a game device that uses an input control device including an acceleration sensor and a gyro sensor. This game device controls a sword possessed by a game character in accordance with the movement of an input control device. Specifically, data on the motion of swinging the sword is created based on the output of the acceleration sensor, and data on the posture of the sword is created based on the output of the gyro sensor.

JP 2000-308756 A

  In a system using an input device including a gyro sensor, an operation for moving the input device itself can be used as an input. Therefore, it is convenient for the user if various operations such as an operation for designating a position on the display screen and an operation for scrolling an image can be performed using such an input device. However, in Patent Document 1, the output of the gyro sensor is only used for controlling the posture of the object (sword) in the virtual space, and the output of the gyro sensor cannot be used for various operations.

  Therefore, an object of the present invention is to provide an information processing device, an information processing program, an information processing system, and a display range control method capable of controlling a display range displayed on a display device using an output of a gyro sensor. It is to be.

  The present invention employs the following configurations (1) to (13) in order to solve the above problems.

(1)
The present invention is an information processing apparatus that inputs an angular velocity detected by a gyro sensor included in an input device and displays an image on a display device. The information processing apparatus includes an attitude calculation unit, a coordinate calculation unit, and a display control unit. The attitude calculation unit calculates the attitude of the input device based on the angular velocity. The coordinate calculation unit calculates the coordinates of the intersection of a line segment extending from a predetermined position in the predetermined space in the direction of the vector representing the posture and the predetermined plane in the predetermined space. A display control part controls the display range which should be displayed on a display apparatus among display objects based on a coordinate.

The “input device” is not limited to a device including an acceleration sensor or the like like the input device 8 in an embodiment described later, and is a concept including an arbitrary input unit including a gyro sensor.
In the embodiment described later, the “information processing apparatus” is a game apparatus for executing a game process, but includes a concept including an arbitrary information processing apparatus that displays an image on a display device using an angular velocity of the input device as an input. It is. In the embodiment described later, the game apparatus 3 and the input apparatus 8 are separate bodies. However, for example, when the information processing apparatus is a portable type, the information processing apparatus and the input apparatus are integrated. Also good.
The “posture calculation unit” may be any unit that calculates the posture of the input device based on the angular velocity, and the posture calculation method is not limited. For example, in the embodiment described later, the posture is calculated using outputs of the gyro sensor, the acceleration sensor, and the imaging unit. However, the “posture calculation unit” may calculate the posture using only the gyro sensor.
The “coordinate calculator” calculates a position (two-dimensional coordinates) on a predetermined plane from the attitude of the input device. The “coordinate calculation unit” may execute step S7 in the embodiment described later, or may execute steps S81 and S82 in addition to step S7.
The “display control unit” may be anything as long as it controls the display range based on the coordinates calculated by the coordinate calculation unit. The “display control unit” may control the display range using a virtual camera set in a three-dimensional virtual space, for example, as in the embodiments described later and the configuration of (2) below. The display range may be set and controlled on the two-dimensional plane that is the display target.

  According to the configuration of (1) above, the coordinate calculation unit can obtain two-dimensional coordinates corresponding to the posture from the posture of the input device. Therefore, by controlling the display range based on the two-dimensional coordinates in the display control unit, the display range can be controlled using the attitude of the input device. Thus, according to the configuration of (1) above, the display range displayed on the display device can be controlled using the output of the gyro sensor.

(2)
The display control unit may control the display range by changing the orientation and / or position of the virtual camera based on the coordinates in the virtual space where the display target is arranged.

  According to the configuration (2), the display range can be easily controlled by controlling the orientation and / or position of the virtual camera.

(3)
The information processing apparatus may further input operation information indicating a direction instruction operation on an operation unit provided in the input device. At this time, the display control unit controls the display range by changing the orientation of the virtual camera based on the coordinates and changing the position of the virtual camera based on the operation information.

  The “operation means” is a concept including an arbitrary operation device provided in the input device, and is a concept including a stick, a touch pad, and the like in addition to a cross button in an embodiment described later.

  According to the configuration of (3) above, the user can change the orientation of the virtual camera by an operation that changes the attitude of the input device, and can also change the position of the virtual camera by an operation on the operation means provided in the input device. Can do. Therefore, the user can perform two types of input with one hand holding the input device, and can change the display range by two types of methods, so that the display range can be operated more freely. That is, according to the configuration of (3) above, the degree of freedom of operation of the display range can be improved and the operability of the operation on the display range can be improved.

(4)
The display control unit may change the display range when the coordinates indicate a position outside the predetermined area on the predetermined plane.

  In the embodiment described later, the “predetermined area” is a predetermined area Ar corresponding to the screen of the display device (television 2), but any position / shape as long as it is an area on a predetermined plane. Also good.

  According to the configuration of (4) above, the display range is changed by changing the attitude of the input device from the attitude facing the predetermined reference direction (the attitude of the input device when the coordinates are inside the predetermined area). Can do.

(5)
The information processing apparatus may further include a screen position calculation unit that calculates the position on the screen based on the coordinates so that the predetermined area corresponds to the area of the screen of the display device.

  According to the configuration of (5) above, the predetermined area corresponds to the area of the screen of the display device. Therefore, the “posture toward the predetermined reference direction” is an attitude in which the input device faces the screen. Therefore, the display range can be changed by turning the input device away from the screen, and the user can change the display range with an intuitive and easy-to-understand operation.

(6)
The information processing apparatus may further include an instruction image display unit and a process execution unit. The instruction image display unit displays a predetermined instruction image at the position on the screen calculated by the screen position calculation unit. The process execution unit executes predetermined information processing based on the position of the instruction image.

The “instruction image” is a concept including an arbitrary image displayed at a position on the screen calculated by the screen position calculation unit in addition to the cursor in the embodiment described later. That is, the “instruction image display unit” is not limited to the one that displays the cursor as in step S44 in the embodiment described later, and may be one that displays the locus of the position on the screen, for example.
In the embodiment described later, the “process execution unit” is the CPU 10 that executes the game process based on the cursor position (step S45) as the “predetermined information process”. As long as it executes processing using the position on the screen calculated by as an input.

According to the configuration of (6) above, in a state where the input device is facing the screen, the predetermined processing is executed by the processing execution unit, and the input device is not facing the screen (outside the screen). The display range is controlled by the display control unit.
Therefore, the user can perform an instruction operation using the instruction image by operating the input device toward the screen, and can move the display range by pointing the input device to the outside of the screen. According to this, since the display range can be moved by an operation that is very intuitive and easy to understand even during an instruction operation using an instruction image, the operability of the instruction operation using the instruction image can be improved.

(7)
The display control unit may determine the direction in which the display range is changed according to the direction from the predetermined area to the position indicated by the coordinates.

  According to the configuration of (7) above, the display range is changed from a posture facing the predetermined reference direction (a posture of the input device when the coordinates are inside the predetermined region) to a direction corresponding to the direction of changing the posture of the input device. Can be changed. According to this, since the display range can be moved in the direction in which the input device is directed, the user can change the display range with an intuitive and easy-to-understand operation.

(8)
The display control unit may determine the degree to which the display range is changed based on the degree to which the position indicated by the coordinates is away from the predetermined area.

The “degree of changing the display range” is a concept including at least the speed of the display range (the amount of movement per unit time) and the amount of movement of the display range.
The above “degree to which the position indicated by the coordinates is away from the predetermined area” is expressed as the length of the protrusion vector Vd in the embodiment described later, but is not limited to this, and the distance from the coordinate position to the predetermined area May be used as long as it represents information. The “degree” may be expressed as, for example, the length from the coordinate position to the center position of the predetermined area, or the side closest to the position of the coordinate among the edges of the predetermined area from the coordinate position. It may be expressed as the length to the midpoint.

  According to the configuration of (8) above, the degree to which the display range is changed is determined according to the amount by which the attitude of the input device is changed from the attitude facing the predetermined reference direction. According to this, since the user can move the display range largely (or quickly) if the direction of the input device is greatly changed, the user can change the display range by an intuitive and easy-to-understand operation. .

(9)
The display control unit may determine the degree to which the display range is changed based on the speed of the coordinates when the coordinates are located outside the predetermined area.

  The above "determining based on the coordinate speed" means determining the degree by using the coordinate speed as it is, and using information calculated from the coordinate speed (using the coordinate speed processed) And) determining the above degree. For example, the display control unit may determine the degree using a scroll component speed S in an embodiment described later, or may determine the degree using a maximum value of the speed in a predetermined period.

  According to the configuration of (9) above, the degree of changing the display range is determined in accordance with the speed at which the posture of the input device is changed from the posture facing the predetermined reference direction. According to this, since the user can move the display range large (or fast) if the direction of the input device is changed quickly, the user can change the display range with an intuitive and easy-to-understand operation. .

(10)
The display control unit may determine the speed at which the display range is changed using at least the maximum value of the speed of the coordinates in a predetermined period after the coordinates are outside the predetermined area.

  The “predetermined period after the coordinates appear outside the predetermined area” may be a period during which the coordinates are outside the predetermined area, as in the embodiment described later, It may be a period until a predetermined time elapses after going outside.

  According to the configuration of (10) above, since the maximum value in the predetermined period is reflected in the moving speed of the display range, the movement of the display range can be continued even when the input device is stopped. Therefore, even when the display range needs to be moved greatly because the display target is very large, the user can easily move the display range.

(11)
The display control unit may change the display range based on the coordinates in a period specified by the user using the input device.

  With configuration (11) above, the display range changes only during the period specified by the user, so that it is possible to prevent the display range from being changed by mistake.

(12)
The display control unit may move the display range in an amount corresponding to the coordinate movement amount in a direction corresponding to the coordinate movement direction.

  According to the configuration of (12) above, since the moving direction and moving amount of the display range are determined according to the moving direction and moving amount of the coordinates, the user can change the display range with an intuitive and easy-to-understand operation.

(13)
The display control unit moves the display range in a direction corresponding to the direction from the predetermined reference position to the position indicated by the coordinates on the predetermined plane at a speed corresponding to the distance from the predetermined reference position to the position indicated by the coordinates. Also good.

  According to the configuration of (13) above, the display range is changed from a posture facing the predetermined reference direction (an posture of the input device when the coordinates are the predetermined reference position) to a direction corresponding to the direction in which the posture of the input device is changed. Can be changed. According to this, since the display range can be moved in the direction in which the input device is directed, the user can change the display range with an intuitive and easy-to-understand operation. Further, the display range can be moved at a speed corresponding to the amount by which the posture of the input device is changed from the posture facing the reference direction. Therefore, since the user can move the display range even when the input device is stationary, the display range can be changed even when the display range needs to be moved greatly because the display target is very large. It can be moved easily.

  The present invention may also be implemented in the form of an information processing program that causes a computer of the information processing apparatus to function as means equivalent to the above-described units. Further, the present invention may be implemented in the form of an information processing system including one or more information processing apparatuses including the above-described units. At this time, the one or more information processing apparatuses may perform direct communication via wired or wireless communication, or may perform communication via a network. Furthermore, the present invention may be implemented in the form of a display range control method performed by each of the above units.

  According to the present invention, a two-dimensional coordinate corresponding to the posture is obtained from the posture of the input device calculated from the output of the gyro sensor, and the display range is controlled based on the two-dimensional coordinate. The display range displayed on the display device can be controlled using the output.

External view of game system Functional block diagram of game device The perspective view which shows the external appearance structure of an input device Perspective view showing the external configuration of the controller Diagram showing the internal structure of the controller Diagram showing the internal structure of the controller Block diagram showing the configuration of the input device The figure which shows the vector which shows a 1st attitude | position and a 2nd attitude | position The figure which shows vector v3 which shows correction amount The figure which shows the vector which shows the 1st attitude | position after correction | amendment by a 1st correction process. The figure which shows the vector which shows a 1st attitude | position and a 3rd attitude | position The figure which shows the 1st attitude | position after correction | amendment by a 2nd correction process. The figure which shows the main data memorize | stored in the main memory of a game device Main flowchart showing the flow of processing executed in the game device Flowchart showing the flow of the first correction process (step S5) shown in FIG. Flowchart showing the flow of the second correction process (step S6) shown in FIG. The figure which shows the two-dimensional coordinate corresponding to a captured image The figure which shows the input device and predetermined plane in a predetermined virtual space FIG. 18 is a diagram in which the virtual space shown in FIG. 18 is viewed from the y-axis positive direction side to the negative direction side. 14 is a flowchart showing the flow of the display control process (step S8) shown in FIG. The figure which shows an example of the relationship between the attitude | position of an input device, and the display range determined by camera direction control processing 20 is a flowchart showing the flow of the camera orientation control process (step S41) shown in FIG. The figure which shows the plane Q arrange | positioned in 1st virtual space The figure which shows the virtual space where the display object and the virtual camera are arranged The figure which shows the example of the 1st-4th control method regarding position control of a virtual camera. The flowchart which shows the flow of the camera position control process (step S42) in the case of employ | adopting a 1st control method. The flowchart which shows the flow of the camera position control process (step S42) in the case of employ | adopting a 2nd control method. The flowchart which shows the flow of the camera position control process (step S42) in the case of employ | adopting a 3rd control method. The figure which shows the data for game processing memorize | stored in the main memory of a game device in the case of employ | adopting a 4th control method. The flowchart which shows the flow of the camera position control process (step S42) in the case of employ | adopting a 4th control method. The figure which shows the scroll component speed S in the plane Q The figure which shows the comparison with the 1st method and the 2nd method The figure which shows the main data memorize | stored in the main memory of a game device in the case of employ | adopting a 2nd method. The main flowchart which shows the flow of the process which a game device performs in the case of taking a 2nd method. 34 is a flowchart showing the flow of the offset determination process (step S41) shown in FIG. Figure showing an example of the setting screen Diagram showing how to determine the offset amount The figure which shows the modification of the correction method in a 2nd method.

[Overall configuration of game system]
With reference to FIG. 1, a game system 1 including a game apparatus which is an example of a coordinate calculation apparatus according to an embodiment of the present invention will be described. FIG. 1 is an external view of the game system 1. Hereinafter, the game apparatus and the game program of the present embodiment will be described using a stationary game apparatus as an example. In FIG. 1, the game system 1 includes a television receiver (hereinafter simply referred to as “TV”) 2, a game device 3, an optical disk 4, an input device 8, and a marker unit 6. In the present system, game processing is executed by the game device 3 based on a game operation using the input device 8.

  An optical disk 4 that is an example of an information storage medium that can be used interchangeably with the game apparatus 3 is detachably inserted into the game apparatus 3. The optical disc 4 stores a game program to be executed on the game apparatus 3. An insertion slot for the optical disk 4 is provided on the front surface of the game apparatus 3. The game apparatus 3 executes a game process by reading and executing a game program stored in the optical disc 4 inserted into the insertion slot.

  A television 2 which is an example of a display device is connected to the game apparatus 3 via a connection cord. The television 2 displays a game image obtained as a result of the game process executed in the game device 3. In addition, a marker unit 6 is installed around the screen of the television 2 (upper side of the screen in FIG. 1). The marker unit 6 includes two markers 6R and 6L at both ends thereof. The marker 6R (same for the marker 6L) is specifically one or more infrared LEDs, and outputs infrared light toward the front of the television 2. The marker unit 6 is connected to the game apparatus 3, and the game apparatus 3 can control lighting of each infrared LED included in the marker unit 6.

  The input device 8 gives operation data indicating the content of the operation performed on the own device to the game device 3. In the present embodiment, the input device 8 includes a controller 5 and a gyro sensor unit 7. Although details will be described later, the input device 8 has a configuration in which a gyro sensor unit 7 is detachably connected to the controller 5. The controller 5 and the game apparatus 3 are connected by wireless communication. In the present embodiment, for example, Bluetooth (registered trademark) technology is used for wireless communication between the controller 5 and the game apparatus 3. In other embodiments, the controller 5 and the game apparatus 3 may be connected by wire.

[Internal configuration of game device 3]
Next, the internal configuration of the game apparatus 3 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration of the game apparatus 3. The game apparatus 3 includes a CPU 10, a system LSI 11, an external main memory 12, a ROM / RTC 13, a disk drive 14, an AV-IC 15 and the like.

  The CPU 10 executes a game process by executing a game program stored on the optical disc 4, and functions as a game processor. The CPU 10 is connected to the system LSI 11. In addition to the CPU 10, an external main memory 12, a ROM / RTC 13, a disk drive 14, and an AV-IC 15 are connected to the system LSI 11. The system LSI 11 performs processing such as control of data transfer between components connected thereto, generation of an image to be displayed, and acquisition of data from an external device. The internal configuration of the system LSI will be described later. The volatile external main memory 12 stores a program such as a game program read from the optical disc 4 or a game program read from the flash memory 17, or stores various data. Used as a work area and buffer area. The ROM / RTC 13 includes a ROM (so-called boot ROM) in which a program for starting the game apparatus 3 is incorporated, and a clock circuit (RTC: Real Time Clock) that counts time. The disk drive 14 reads program data, texture data, and the like from the optical disk 4 and writes the read data to an internal main memory 11e or an external main memory 12 described later.

  The system LSI 11 includes an input / output processor (I / O processor) 11a, a GPU (Graphics Processor Unit) 11b, a DSP (Digital Signal Processor) 11c, a VRA 1d, and an internal main memory 11e. Although not shown, these components 11a to 11e are connected to each other by an internal bus.

  The GPU 11b forms part of a drawing unit and generates an image according to a graphics command (drawing command) from the CPU 10. The VRAM 11d stores data (data such as polygon data and texture data) necessary for the GPU 11b to execute the graphics command. When an image is generated, the GPU 11b creates image data using data stored in the VRAM 11d.

  The DSP 11c functions as an audio processor, and generates sound data using sound data and sound waveform (tone color) data stored in the internal main memory 11e and the external main memory 12.

  The image data and audio data generated as described above are read out by the AV-IC 15. The AV-IC 15 outputs the read image data to the television 2 via the AV connector 16, and outputs the read audio data to the speaker 2 a built in the television 2. As a result, an image is displayed on the television 2 and a sound is output from the speaker 2a.

  The input / output processor 11a performs transmission / reception of data to / from components connected to the input / output processor 11a and downloads data from an external device. The input / output processor 11a is connected to the flash memory 17, the wireless communication module 18, the wireless controller module 19, the expansion connector 20, and the memory card connector 21. An antenna 22 is connected to the wireless communication module 18, and an antenna 23 is connected to the wireless controller module 19.

  The input / output processor 11a is connected to the network via the wireless communication module 18 and the antenna 22, and can communicate with other game devices and various servers connected to the network. The input / output processor 11a periodically accesses the flash memory 17 to detect the presence / absence of data that needs to be transmitted to the network. If there is such data, the input / output processor 11a communicates with the network via the wireless communication module 18 and the antenna 22. Send. Further, the input / output processor 11a receives data transmitted from other game devices and data downloaded from the download server via the network, the antenna 22 and the wireless communication module 18, and receives the received data in the flash memory 17. Remember. By executing the game program, the CPU 10 reads out the data stored in the flash memory 17 and uses it in the game program. In the flash memory 17, in addition to data transmitted and received between the game apparatus 3 and other game apparatuses and various servers, save data (game result data or intermediate data) of the game played using the game apparatus 3 May be stored.

  The input / output processor 11a receives operation data transmitted from the controller 5 via the antenna 23 and the wireless controller module 19, and stores (temporarily stores) the data in the buffer area of the internal main memory 11e or the external main memory 12.

  Further, an expansion connector 20 and a memory card connector 21 are connected to the input / output processor 11a. The expansion connector 20 is a connector for an interface such as USB or SCSI, and connects a medium such as an external storage medium, a peripheral device such as another controller, or a wired communication connector. By connecting, communication with the network can be performed instead of the wireless communication module 18. The memory card connector 21 is a connector for connecting an external storage medium such as a memory card. For example, the input / output processor 11a can access an external storage medium via the expansion connector 20 or the memory card connector 21 to store data in the external storage medium or read data from the external storage medium.

  The game apparatus 3 is provided with a power button 24, a reset button 25, and an eject button 26. The power button 24 and the reset button 25 are connected to the system LSI 11. When the power button 24 is turned on, power is supplied to each component of the game apparatus 3 via an AC adapter (not shown). When the reset button 25 is pressed, the system LSI 11 restarts the boot program for the game apparatus 3. The eject button 26 is connected to the disk drive 14. When the eject button 26 is pressed, the optical disk 4 is ejected from the disk drive 14.

[Configuration of Input Device 8]
Next, the input device 8 will be described with reference to FIGS. FIG. 3 is a perspective view showing an external configuration of the input device 8. FIG. 4 is a perspective view showing an external configuration of the controller 5. 3 is a perspective view of the controller 5 as seen from the upper rear side, and FIG. 4 is a perspective view of the controller 5 as seen from the lower front side.

  3 and 4, the controller 5 includes a housing 31 formed by plastic molding, for example. The housing 31 has a substantially rectangular parallelepiped shape whose longitudinal direction is the front-rear direction (the Z-axis direction shown in FIG. 3), and is a size that can be gripped with one hand of an adult or a child as a whole. The player can perform a game operation by pressing a button provided on the controller 5 and moving the controller 5 itself to change its position and posture.

  The housing 31 is provided with a plurality of operation buttons. As shown in FIG. 3, a cross button 32a, a first button 32b, a second button 32c, an A button 32d, a minus button 32e, a home button 32f, a plus button 32g, and a power button 32h are provided on the upper surface of the housing 31. It is done. In the present specification, the upper surface of the housing 31 on which these buttons 32a to 32h are provided may be referred to as a “button surface”. On the other hand, as shown in FIG. 4, a recess is formed on the lower surface of the housing 31, and a B button 32i is provided on the rear inclined surface of the recess. A function corresponding to the game program executed by the game apparatus 3 is appropriately assigned to each of the operation buttons 32a to 32i. The power button 32h is for remotely turning on / off the main body of the game apparatus 3. The home button 32 f and the power button 32 h are embedded in the upper surface of the housing 31. This can prevent the player from pressing the home button 32f or the power button 32h by mistake.

  A connector 33 is provided on the rear surface of the housing 31. The connector 33 is used to connect another device (for example, the gyro sensor unit 7 or another controller) to the controller 5. Further, locking holes 33a are provided on both sides of the connector 33 on the rear surface of the housing 31 in order to prevent the other devices from being easily detached.

  A plurality (four in FIG. 3) of LEDs 34 a to 34 d are provided behind the upper surface of the housing 31. Here, the controller type (number) is assigned to the controller 5 to distinguish it from other main controllers. The LEDs 34a to 34d are used for the purpose of notifying the player of the controller type currently set in the controller 5 and notifying the player of the remaining battery level of the controller 5. Specifically, when a game operation is performed using the controller 5, any one of the plurality of LEDs 34a to 34d is turned on according to the controller type.

  Further, the controller 5 has an imaging information calculation unit 35 (FIG. 6), and a light incident surface 35a of the imaging information calculation unit 35 is provided on the front surface of the housing 31, as shown in FIG. The light incident surface 35a is made of a material that transmits at least infrared light from the markers 6R and 6L.

  Between the first button 32b and the home button 32f on the upper surface of the housing 31, a sound release hole 31a for releasing sound from the speaker 49 (FIG. 5) built in the controller 5 is formed.

  Next, the internal structure of the controller 5 will be described with reference to FIGS. 5 and 6 are diagrams showing the internal structure of the controller 5. FIG. FIG. 5 is a perspective view showing a state in which the upper housing (a part of the housing 31) of the controller 5 is removed. FIG. 6 is a perspective view showing a state in which the lower casing (a part of the housing 31) of the controller 5 is removed. The perspective view shown in FIG. 6 is a perspective view of the substrate 30 shown in FIG.

  In FIG. 5, a substrate 30 is fixed inside the housing 31, and operation buttons 32 a to 32 h, LEDs 34 a to 34 d, an acceleration sensor 37, an antenna 45, and a speaker 49 are provided on the upper main surface of the substrate 30. Etc. are provided. These are connected to a microcomputer (microcomputer) 42 (see FIG. 6) by wiring (not shown) formed on the substrate 30 and the like. In the present embodiment, the acceleration sensor 37 is disposed at a position shifted from the center of the controller 5 with respect to the X-axis direction. This makes it easier to calculate the movement of the controller 5 when the controller 5 is rotated about the Z axis. The acceleration sensor 37 is disposed in front of the center of the controller 5 in the longitudinal direction (Z-axis direction). Further, the controller 5 functions as a wireless controller by the wireless module 44 (FIG. 6) and the antenna 45.

  On the other hand, in FIG. 6, an imaging information calculation unit 35 is provided at the front edge on the lower main surface of the substrate 30. The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41 in order from the front of the controller 5. These members 38 to 41 are respectively attached to the lower main surface of the substrate 30.

  Further, the microcomputer 42 and the vibrator 48 are provided on the lower main surface of the substrate 30. The vibrator 48 is, for example, a vibration motor or a solenoid, and is connected to the microcomputer 42 by wiring formed on the substrate 30 or the like. The controller 48 is vibrated by the operation of the vibrator 48 according to the instruction of the microcomputer 42. As a result, a so-called vibration-compatible game in which the vibration is transmitted to the hand of the player holding the controller 5 can be realized. In the present embodiment, the vibrator 48 is disposed slightly forward of the housing 31. That is, by arranging the vibrator 48 on the end side of the center of the controller 5, the entire controller 5 can be vibrated greatly by the vibration of the vibrator 48. The connector 33 is attached to the rear edge on the lower main surface of the substrate 30. 5 and 6, the controller 5 includes a crystal resonator that generates a basic clock of the microcomputer 42, an amplifier that outputs an audio signal to the speaker 49, and the like.

  Further, the gyro sensor unit 7 includes gyro sensors (gyro sensors 55 and 56 shown in FIG. 7) that detect angular velocities around three axes. The gyro sensor unit 7 is detachably attached to the connector 33 of the controller 5. A plug (plug 53 shown in FIG. 7) that can be connected to the connector 33 is provided at the front end of the gyro sensor unit 7 (end on the Z-axis positive direction side shown in FIG. 3). Further, hooks (not shown) are provided on both sides of the plug 53. In a state where the gyro sensor unit 7 is attached to the controller 5, the plug 53 is connected to the connector 33 and the hook is locked in the locking hole 33 a of the controller 5. Thereby, the controller 5 and the gyro sensor unit 7 are firmly fixed. The gyro sensor unit 7 has a button 51 on a side surface (surface in the X-axis direction shown in FIG. 3). The button 51 is configured such that when the button 51 is pressed, the hook is released from the locked state with respect to the locking hole 33a. Therefore, the gyro sensor unit 7 can be detached from the controller 5 by removing the plug 53 from the connector 33 while pressing the button 51.

  A connector having the same shape as the connector 33 is provided at the rear end of the gyro sensor unit 7. Therefore, other devices that can be attached to the controller 5 (connector 33 thereof) can also be attached to the connector of the gyro sensor unit 7. In FIG. 3, a cover 52 is detachably attached to the connector.

  The shapes of the controller 5 and the gyro sensor unit 7 shown in FIGS. 3 to 6, the shapes of the operation buttons, the number of acceleration sensors and vibrators, and the installation positions are merely examples, and other shapes, numbers, Even in the installation position, the present invention can be realized. In the present embodiment, the imaging direction by the imaging unit is the positive Z-axis direction, but the imaging direction may be any direction. That is, the position of the imaging information calculation unit 35 in the controller 5 (the light incident surface 35a of the imaging information calculation unit 35) does not have to be the front surface of the housing 31, and other surfaces can be used as long as light can be taken in from the outside of the housing 31. May be provided.

  FIG. 7 is a block diagram showing a configuration of the input device 8 (the controller 5 and the gyro sensor unit 7). The controller 5 includes an operation unit 32 (operation buttons 32a to 32i), a connector 33, an imaging information calculation unit 35, a communication unit 36, and an acceleration sensor 37. The controller 5 transmits data indicating the details of the operation performed on the own device to the game apparatus 3 as operation data.

  The operation unit 32 includes the operation buttons 32a to 32i described above, and the operation button data indicating the input state (whether or not each operation button 32a to 32i is pressed) to each operation button 32a to 32i is transmitted to the microcomputer of the communication unit 36. Output to 42.

  The imaging information calculation unit 35 is a system for analyzing the image data captured by the imaging unit, discriminating a region having a high luminance in the image data, and calculating a center of gravity position, a size, and the like of the region. Since the imaging information calculation unit 35 has a sampling period of, for example, about 200 frames / second at the maximum, it can track and analyze even a relatively fast movement of the controller 5.

  The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41. The infrared filter 38 passes only infrared rays from the light incident from the front of the controller 5. The lens 39 collects the infrared light transmitted through the infrared filter 38 and makes it incident on the image sensor 40. The image sensor 40 is a solid-state image sensor such as a CMOS sensor or a CCD sensor, for example, and receives the infrared light collected by the lens 39 and outputs an image signal. Here, the markers 6 </ b> R and 6 </ b> L of the marker unit 6 disposed in the vicinity of the display screen of the television 2 are configured by infrared LEDs that output infrared light toward the front of the television 2. Therefore, by providing the infrared filter 38, the image sensor 40 receives only the infrared light that has passed through the infrared filter 38 and generates image data, so that the images of the markers 6R and 6L can be captured more accurately. Hereinafter, an image captured by the image sensor 40 is referred to as a captured image. Image data generated by the image sensor 40 is processed by the image processing circuit 41. The image processing circuit 41 calculates the position of the imaging target (markers 6R and 6L) in the captured image. The image processing circuit 41 outputs coordinates indicating the calculated position to the microcomputer 42 of the communication unit 36. The coordinate data is transmitted to the game apparatus 3 as operation data by the microcomputer 42. Hereinafter, the coordinates are referred to as “marker coordinates”. Since the marker coordinates change corresponding to the direction (tilt angle) and position of the controller 5 itself, the game apparatus 3 can calculate the direction and position of the controller 5 using the marker coordinates.

  In other embodiments, the controller 5 may not include the image processing circuit 41, and the captured image itself may be transmitted from the controller 5 to the game apparatus 3. At this time, the game apparatus 3 may have a circuit or a program having the same function as the image processing circuit 41, and may calculate the marker coordinates.

  The acceleration sensor 37 detects the acceleration (including gravity acceleration) of the controller 5, that is, detects the force (including gravity) applied to the controller 5. The acceleration sensor 37 detects the value of the acceleration (linear acceleration) in the linear direction along the sensing axis direction among the accelerations applied to the detection unit of the acceleration sensor 37. For example, in the case of a multi-axis acceleration sensor having two or more axes, component acceleration along each axis is detected as the acceleration applied to the detection unit of the acceleration sensor. For example, the triaxial or biaxial acceleration sensor may be of the type available from Analog Devices, Inc. or ST Microelectronics NV. The acceleration sensor 37 is, for example, a capacitance type acceleration sensor, but other types of acceleration sensors may be used.

  In the present embodiment, the acceleration sensor 37 has a vertical direction (Y-axis direction shown in FIG. 3), a horizontal direction (X-axis direction shown in FIG. 3), and a front-back direction (Z-axis direction shown in FIG. 3) with reference to the controller 5. ) Linear acceleration is detected in each of the three axis directions. Since the acceleration sensor 37 detects acceleration in the linear direction along each axis, the output from the acceleration sensor 37 represents the linear acceleration value of each of the three axes. That is, the detected acceleration is represented as a three-dimensional vector (ax, ay, az) in an XYZ coordinate system (controller coordinate system) set with reference to the input device 8 (controller 5). Hereinafter, a vector having the respective acceleration values related to the three axes detected by the acceleration sensor 37 as components is referred to as an acceleration vector.

  Data indicating the acceleration detected by the acceleration sensor 37 (acceleration data) is output to the communication unit 36. The acceleration detected by the acceleration sensor 37 changes in accordance with the direction (tilt angle) and movement of the controller 5 itself, so the game apparatus 3 calculates the direction and movement of the controller 5 using the acquired acceleration data. can do. In the present embodiment, the game apparatus 3 calculates the attitude, tilt angle, and the like of the controller 5 based on the acquired acceleration data.

  In addition, based on the acceleration signal output from the acceleration sensor 37, a computer such as a processor (for example, the CPU 10) of the game apparatus 3 or a processor (for example, the microcomputer 42) of the controller 5 performs processing, whereby further information regarding the controller 5 is obtained. Those skilled in the art will be able to easily understand from the description of the present specification that can be estimated or calculated (determined). For example, when processing on the computer side is executed on the assumption that the controller 5 on which the acceleration sensor 37 is mounted is stationary (that is, the processing is executed assuming that the acceleration detected by the acceleration sensor is only gravitational acceleration). When the controller 5 is actually stationary, it can be determined whether or not the attitude of the controller 5 is inclined with respect to the direction of gravity based on the detected acceleration. Specifically, whether or not the controller 5 is inclined with respect to the reference depending on whether or not 1G (gravity acceleration) is applied, based on the state in which the detection axis of the acceleration sensor 37 is directed vertically downward. It is possible to know how much it is inclined with respect to the reference according to its size. Further, in the case of the multi-axis acceleration sensor 37, it is possible to know in detail how much the controller 5 is inclined with respect to the direction of gravity by further processing the acceleration signal of each axis. . In this case, the processor may calculate the tilt angle of the controller 5 based on the output from the acceleration sensor 37, or may calculate the tilt direction of the controller 5 without calculating the tilt angle. Good. Thus, by using the acceleration sensor 37 in combination with the processor, the tilt angle or posture of the controller 5 can be determined.

  On the other hand, when it is assumed that the controller 5 is in a dynamic state (a state in which the controller 5 is moved), the acceleration sensor 37 detects an acceleration corresponding to the movement of the controller 5 in addition to the gravitational acceleration. Therefore, the movement direction of the controller 5 can be known by removing the gravitational acceleration component from the detected acceleration by a predetermined process. Even if it is assumed that the controller 5 is in a dynamic state, the direction of gravity is obtained by removing the acceleration component corresponding to the movement of the acceleration sensor from the detected acceleration by a predetermined process. It is possible to know the inclination of the controller 5 with respect to. In another embodiment, the acceleration sensor 37 is a built-in process for performing a predetermined process on the acceleration signal before outputting the acceleration signal detected by the built-in acceleration detection means to the microcomputer 42. An apparatus or other type of dedicated processing apparatus may be provided. A built-in or dedicated processing device converts the acceleration signal into a tilt angle (or other preferred parameter) if, for example, the acceleration sensor 37 is used to detect static acceleration (eg, gravitational acceleration). It may be a thing.

  The communication unit 36 includes a microcomputer 42, a memory 43, a wireless module 44, and an antenna 45. The microcomputer 42 controls the wireless module 44 that wirelessly transmits data acquired by the microcomputer 42 to the game apparatus 3 while using the memory 43 as a storage area when performing processing. The microcomputer 42 is connected to the connector 33. Data transmitted from the gyro sensor unit 7 is input to the microcomputer 42 via the connector 33. Hereinafter, the configuration of the gyro sensor unit 7 will be described.

  The gyro sensor unit 7 includes a plug 53, a microcomputer 54, a 2-axis gyro sensor 55, and a 1-axis gyro sensor 56. As described above, the gyro sensor unit 7 detects angular velocities around the three axes (in this embodiment, the XYZ axes), and transmits data (angular velocity data) indicating the detected angular velocities to the controller 5.

  The biaxial gyro sensor 55 detects an angular velocity around the X axis and an angular velocity (per unit time) around the Z axis. Further, the uniaxial gyro sensor 56 detects an angular velocity (per unit time) around the Y axis. In this specification, the rotation directions around the XYZ axes are referred to as a pitch direction, a yaw direction, and a roll direction, respectively, with reference to the imaging direction (Z-axis positive direction) of the controller 5. That is, the biaxial gyro sensor 55 detects angular velocities in the pitch direction (rotation direction around the X axis) and the roll direction (rotation direction around the Z axis), and the single axis gyro sensor 56 detects the yaw direction (around the Y axis). Detect the angular velocity in the rotation direction.

  In this embodiment, the 2-axis gyro sensor 55 and the 1-axis gyro sensor 56 are used to detect the angular velocity around the three axes. However, in other embodiments, the angular velocity around the three axes is Any number and combination of gyro sensors may be used as long as they can be detected.

  Further, in the present embodiment, for the purpose of facilitating the calculation in the posture calculation process described later, the three axes where the gyro sensors 55 and 56 detect the angular velocity are the three axes (XYZ) where the acceleration sensor 37 detects the acceleration. Axis). However, in other embodiments, the three axes for detecting the angular velocities by the gyro sensors 56 and 57 and the three axes for detecting the acceleration by the acceleration sensor 37 may not coincide with each other.

  Data indicating the angular velocity detected by each of the gyro sensors 56 and 57 is output to the microcomputer 54. Accordingly, the microcomputer 54 receives data indicating the angular velocity around the three axes of the XYZ axes. The microcomputer 54 transmits data indicating the angular velocities around the three axes as angular velocity data to the controller 5 via the plug 53. Although transmission from the microcomputer 54 to the controller 5 is sequentially performed every predetermined cycle, since the game processing is generally performed in units of 1/60 seconds (one frame time), this time Transmission is preferably performed in the following cycle.

  Returning to the description of the controller 5, the data output from the operation unit 32, the imaging information calculation unit 35, and the acceleration sensor 37 to the microcomputer 42 and the data transmitted from the gyro sensor unit 7 to the microcomputer 42 are temporarily stored. Stored in the memory 43. These data are transmitted to the game apparatus 3 as the operation data. That is, the microcomputer 42 outputs the operation data stored in the memory 43 to the wireless module 44 when the transmission timing to the wireless controller module 19 of the game apparatus 3 arrives. The wireless module 44 modulates a carrier wave of a predetermined frequency with operation data using, for example, Bluetooth (registered trademark) technology, and radiates a weak radio signal from the antenna 45. That is, the operation data is modulated by the wireless module 44 into a weak radio signal and transmitted from the controller 5. The weak radio signal is received by the wireless controller module 19 on the game apparatus 3 side. By demodulating and decoding the received weak radio signal, the game apparatus 3 can acquire operation data. And CPU10 of the game device 3 performs a game process based on the acquired operation data and a game program. Note that the wireless transmission from the communication unit 36 to the wireless controller module 19 is sequentially performed at predetermined intervals, but the game processing is generally performed in units of 1/60 seconds (one frame time). Therefore, it is preferable to perform transmission at a period equal to or shorter than this time. The communication unit 36 of the controller 5 outputs each operation data to the wireless controller module 19 of the game apparatus 3 at a rate of once every 1/200 seconds, for example.

  By using the controller 5, the player can perform an operation of tilting the controller 5 to an arbitrary tilt angle in addition to the conventional general game operation of pressing each operation button. In addition, according to the controller 5, the player can also perform an operation of instructing an arbitrary position on the screen by the controller 5 and an operation of moving the controller 5 itself.

[Outline of posture calculation processing]
Next, with reference to FIGS. 8 to 12, an outline of a posture calculation process for calculating the posture of the input device 8 executed in the game apparatus 3 will be described. In the present embodiment, the game apparatus 3 acquires data (operation data) from the input device 8 including the gyro sensors 55 and 56, the acceleration sensor 37, and the imaging means (imaging element 40). Calculate the posture. In the present embodiment, the input device 8 is configured to include both the acceleration sensor 37 and the image sensor 40. However, in other embodiments, the input device 8 includes only one of the acceleration sensor 37 and the image sensor 40. It may be a configuration.

  The game apparatus 3 includes (1) posture calculation means, (2) first correction means, and (3) second correction means. In the present embodiment, these means are realized by causing a game program (coordinate calculation program) executed by the computer (CPU 10) of the game apparatus 3 to function as the means. In other embodiments, part or all of the above means may be realized as a dedicated circuit included in the game apparatus 3.

(1) Attitude calculation means The attitude calculation means calculates the attitude of the input device 8 based on the angular velocity detected by the gyro sensors 55 and 56 (step S4 described later). Any method may be used to calculate the posture from the angular velocity. For example, there is a method of sequentially adding the angular velocity (per unit time) to the initial posture. That is, the current posture can be calculated by integrating the angular velocities sequentially output from the gyro sensors 55 and 56 and calculating the amount of change in posture from the initial state from the integration result. In the following, the posture of the input device 8 calculated from the angular velocity by the posture calculation means is referred to as a “first posture”. However, the posture after the first posture is corrected is also referred to as the first posture.

  Here, the first posture calculated using the angular velocities detected by the gyro sensors 55 and 56 has an error between the actual posture of the input device 8 due to erroneous detection of the gyro sensors 55 and 56. May occur. Therefore, in the present embodiment, the game apparatus 3 corrects the first posture using the acceleration detected by the acceleration sensor 37. Further, the first posture is corrected using an image (captured image) captured by the image sensor 40.

(2) First Correction Unit The first correction unit corrects the first posture based on acceleration data detected by the acceleration sensor 37 (step S5 described later). In the present embodiment, the first correction unit performs correction to bring the first posture closer to the second posture. Here, the second posture refers to a posture determined from the acceleration data, specifically, a posture of the input device 8 when it is assumed that the direction of acceleration indicated by the acceleration data is vertically downward. That is, the posture is calculated on the assumption that the acceleration indicated by the acceleration data is gravitational acceleration. Hereinafter, a correction process (first correction process) by the first correction unit will be described with reference to FIGS.

  FIG. 8 is a diagram illustrating correction of the first posture using the second posture. In this embodiment, the posture in the three-dimensional space is actually processed. However, in FIGS. 8 to 10, the posture in the two-dimensional plane will be described for easy understanding of the drawings. A vector G shown in FIG. 8A indicates a vertically downward direction defined in a spatial coordinate system based on a predetermined position in a space where the input device 8 exists, that is, a gravity direction. Also, the vector v1 shown in FIG. 8A is a spatial coordinate of the downward vector of the input device 8 when the controller 5 is in the first posture (that is, the negative Y-axis direction shown in FIGS. 3 to 5). It shows the orientation in the system. When the posture of the input device 8 is in the basic state, the vector indicating the posture matches the vector G. Therefore, the vector v1 corresponds to the first posture in the spatial coordinate system. The first posture can also be expressed as a rotation made by the vector v1 with respect to the vector G, and is shown as an angle θ1 in the two-dimensional FIG. Since the first posture is calculated based on the angular velocity, the vector v1 is calculated by adding the angular velocity to the previous posture and rotating the vector v1. The second posture is calculated based on the acceleration data. A vector v2 shown in FIG. 8A indicates the direction of acceleration indicated by the acceleration data (the direction of acceleration in the view coordinate system). However, the acceleration data is an acceleration applied to the input device 8, and what can be acquired is a vector in a coordinate system based on the input device 8. FIG. 8B shows the relationship between the acceleration sensor axis and the acceleration vector. As shown in FIG. 8B, when the angle formed by the acceleration vector v0 acquired from the acceleration sensor and the Y-axis negative direction of the sensor is θ2, the vector v1 in the spatial coordinate system of FIG. A vector v2 obtained by adding the rotation θ2 to the acceleration vector in the spatial coordinate system. Since the second posture is “the posture of the input device 8 when the acceleration direction indicated by the acceleration data is regarded as being vertically downward” as described above, the rotation of the angle θ2 from the vector v2 to the vector v1 is performed. Becomes the second posture. If the second posture is represented by a downward vector of the input device 8 in the spatial coordinate system as a vector v1, it can be represented as a vector v2 'obtained by rotating the vector G by θ2. In the case of a three-dimensional posture, it can be expressed by a three-dimensional rotation matrix or the like. When the first posture is correctly calculated from the angular velocity and the acceleration data accurately indicates the direction of gravity, the direction of the vector v2 indicating the direction of acceleration is the vertical downward direction of the spatial coordinate system, that is, gravity. Match the direction. That is, when the first posture is not correctly calculated from the angular velocity, or when the acceleration data does not indicate the correct gravity direction, the vector v2 indicating the direction of acceleration and the gravity as shown in FIG. The direction vector G is different. In a situation where the direction indicated by the acceleration data is assumed to coincide with the direction of gravity, such as in a stationary state, the vector v2 is more accurate as data corresponding to the attitude of the input device 8 than the vector v1. It is considered a thing. Even when the input device is not stationary, taking into account the accuracy of the average posture within a certain period of time, the acceleration vector will be close to the direction of gravity on average, so that the error may increase with time. It can be considered more reliable than the posture calculated from the accumulated angular velocity. On the other hand, if the correct posture has been calculated at the previous calculation timing, the posture at the next calculation timing is calculated more accurately using the angular velocity than the acceleration. it is conceivable that. In other words, the posture calculation based on the angular velocity has a smaller error for each timing than the calculation based on the acceleration, but the error increases with time, while the posture calculation based on the acceleration may have a larger error for each timing. Since it can be calculated at each timing, it has a feature that no error is accumulated. Therefore, the first correction unit performs correction in consideration of both the first posture and the second posture.

  The correction by the first correction means is correction that brings the first posture closer to the second posture. That is, the first correction unit performs correction to bring the angle θ1 close to the angle θ2. This may be expressed as a correction that brings the vector v1 closer to the vector v2 '. However, in the calculation process, if the vector v2 is known, the correction is possible even if the vector v2 'itself is not calculated. In the present embodiment, the correction is performed using a vector v3 indicating the correction amount. FIG. 9 is a diagram illustrating a vector v3 indicating the correction amount. A vector v3 illustrated in FIG. 9 is a vector indicating a correction amount for correcting the first posture. Specifically, an angle Δθ formed by the vector v3 with respect to the vector v2 is a correction amount. Although details will be described later, the vector v3 is set between the vector G and the vector v2 (see FIG. 9). By rotating the vector v1 by Δθ, the vector v1 approaches the vector v2 ′ described above.

  The first correction process is performed by rotating the first posture (vector v1) by the correction amount. FIG. 10 is a diagram illustrating a vector indicating the first posture after the correction by the first correction process. As shown in FIG. 10, the corrected first posture (vector v1 ') is obtained by rotating the first posture before correction (vector v1) by an angle Δθ. As a result, the angle θ1 ′ representing the first posture after correction is between the angle θ1 and the angle θ2, and it can be seen that the correction for bringing the angle θ1 closer to the angle θ2 was performed.

  In the present embodiment, the first correction unit performs correction to bring the first posture closer to the second posture, and does not match the corrected first posture with the second posture. This is to prevent the corrected first posture from changing suddenly even when the acceleration data changes suddenly due to erroneous detection or intense operation. However, in other embodiments, the first correction unit may perform correction so that the corrected first posture matches the second posture. Although details will be described later, in the present embodiment, the rate at which the first correction unit brings the first posture closer to the second posture is the magnitude of the acceleration indicated by the acceleration data (more specifically, It is determined so as to change according to the difference between the magnitude and the gravitational acceleration. However, in other embodiments, the ratio may be a predetermined fixed value.

(3) Second Correction Unit The second correction unit corrects the first posture based on a predetermined image to be captured that is captured by the imaging unit (step S6 described later). Here, in the present embodiment, the predetermined imaging target is the marker unit 6 (infrared LED). In the present embodiment, the second correction unit performs correction to bring the first posture closer to the third posture. The third posture is a posture calculated from the image to be imaged, and specifically, the posture of the input device 8 calculated from the orientation and / or position of the imaged object in the image. Hereinafter, the correction processing (second correction processing) by the second correction means will be described with reference to FIGS. 11 and 12.

  FIG. 11 is a diagram illustrating the correction of the first posture using the third posture. In this embodiment, the posture in the three-dimensional space is actually processed. However, in FIG. 11 and FIG. 12, the posture in the two-dimensional plane will be described for easy understanding of the drawings. A vector v1 shown in FIG. 11 indicates the first posture in the spatial coordinate system. A vector v4 shown in FIG. 11 indicates the third posture in the spatial coordinate system. Since the position and orientation of the marker unit 6 are determined in advance, the orientation of the input device 8 can be relatively calculated based on the orientation and position of the marker in the image. Assuming that the third posture represents the correct posture, when the first posture is correctly calculated from the angular velocity, the vector v1 indicating the first posture indicates the third posture. It matches the vector v4. That is, when the first posture is not correctly calculated from the angular velocity, the vector v1 indicating the first posture and the vector v4 indicating the third posture are different as shown in FIG.

  The second correction process is performed by bringing the first posture (vector v1) closer to the third posture (vector v4) at a predetermined rate. FIG. 12 is a diagram illustrating the first posture after correction by the second correction processing. As shown in FIG. 12, the first posture after correction (vector v1 ′) brings the first posture before correction (vector v1) closer to the third posture (vector v4) at a predetermined rate. Obtained by.

  Here, depending on the orientation and position of the input device 8, the imaging unit may not be able to capture the marker unit 6. In this case, the second correction unit cannot perform the second correction process. Therefore, if the second correction unit performs correction so that the corrected first posture matches the third posture, the second correction processing is performed from a state where the second correction processing cannot be performed. There is a possibility that the first posture may change abruptly when transitioning to a state where it is possible to do so. Thus, if the first posture suddenly changes without conforming to the player's intention, the player feels uncomfortable with the operation (even if the corrected posture is accurate). In order to prevent such a sudden change, in the present embodiment, correction is performed so that the first posture approaches the third posture at a predetermined rate. As a result, it is possible to prevent the first posture from changing suddenly, thereby preventing the player from feeling uncomfortable with the operation. However, in other embodiments, such as when the input device 8 can be assumed to be used in a posture in which the imaging unit can always image the marker unit 6, the second correction unit is the corrected first unit. You may correct | amend so that this attitude | position may correspond with a 3rd attitude | position.

  In the present embodiment, the game apparatus 3 executes both the first correction process and the second correction process, but in other embodiments, one of the first correction process and the second correction process. May be executed only. In the present embodiment, the game apparatus 3 performs the first correction process first and the second correction process later, but either the first correction process or the second correction process is performed first. You may do it.

  As described above, according to the present embodiment, the attitude of the input device 8 calculated from the angular velocities detected by the gyro sensors 55 and 56 is corrected using the acceleration detected by the acceleration sensor 37, and imaging is performed. Correction is performed using the image captured by the means. Thereby, the error of the attitude calculated from the gyro sensor can be reduced, and the attitude of the input device 8 can be calculated more accurately.

  In addition, from the detection result of the acceleration sensor 37, the rotation around the gravity direction (rotation in the yaw direction) cannot be detected, and therefore the correction by the first correction unit cannot be performed in the yaw direction. However, the correction using the detection result of the acceleration sensor 37 has a feature that the posture of the input device 8 can be any posture (because acceleration can always be detected). On the other hand, if the marker unit 6 is not present in the imaging direction of the input device 8, the marker coordinates are not detected, and therefore the correction by the second correction unit cannot be performed depending on the attitude of the input device 8. However, the correction using the captured image has a feature that the posture (especially the posture in the roll direction) can be accurately calculated. In the present embodiment, the posture of the input device 8 can be calculated more accurately by performing two types of corrections having different features as described above.

[Details of processing in game device 3]
Next, details of processing executed in the game apparatus 3 will be described. First, main data used in the processing in the game apparatus 3 will be described with reference to FIG. FIG. 13 is a diagram illustrating main data stored in the main memory (the external main memory 12 or the internal main memory 11e) of the game apparatus 3. As shown in FIG. 13, a game program 60, operation data 62, and game processing data 67 are stored in the main memory of the game apparatus 3. In addition to the data shown in FIG. 13, the main memory stores data necessary for game processing, such as image data of various objects appearing in the game and data indicating various parameters of the objects.

  A part or all of the game program 60 is read from the optical disc 4 and stored in the main memory at an appropriate timing after the game apparatus 3 is turned on. The game program 60 includes a coordinate calculation program 61. The coordinate calculation program 61 is a program for calculating the attitude of the input device 8 and executing processing for calculating two-dimensional coordinates (on the screen of the television 2) based on the calculated attitude.

  The operation data 62 is operation data transmitted from the controller 5 to the game apparatus 3. As described above, since the operation data is transmitted from the controller 5 to the game apparatus 3 at a rate of once every 1/200 seconds, the operation data 62 stored in the main memory is updated at this rate. In the present embodiment, only the latest (last acquired) operation data may be stored in the main memory.

  The operation data 62 includes angular velocity data 63, acceleration data 64, marker coordinate data 65, and operation button data 66. The angular velocity data 63 is data indicating angular velocities detected by the gyro sensors 55 and 56 of the gyro sensor unit 7. Here, the angular velocity data 63 indicates the respective angular velocities around the three axes of XYZ shown in FIG. The acceleration data 64 is data indicating the acceleration (acceleration vector) detected by the acceleration sensor 37. Here, the acceleration data 64 represents a three-dimensional acceleration vector Va1 having as components the accelerations in the directions of the three axes of XYZ shown in FIG. In the present embodiment, the magnitude of the acceleration vector Va1 detected by the acceleration sensor 37 while the controller 5 is stationary is “1”. That is, the magnitude of gravitational acceleration detected by the acceleration sensor 37 is “1”.

  The marker coordinate data 65 is data indicating coordinates calculated by the image processing circuit 41 of the imaging information calculation unit 35, that is, the marker coordinates. The marker coordinates are expressed in a two-dimensional coordinate system (x′y ′ coordinate system shown in FIG. 17) for representing a position on a plane corresponding to the captured image. Note that, when the two markers 6R and 6L are imaged by the imaging element 40, two marker coordinates are calculated. On the other hand, if either one of the markers 6R and 6L is not located within the imageable range of the image sensor 40, only one marker is imaged by the image sensor 40, and only one marker coordinate is calculated. Further, when both of the markers 6R and 6L are not located within the image capturing range of the image sensor 40, the marker is not imaged by the image sensor 40, and the marker coordinates are not calculated. Therefore, the marker coordinate data 65 may indicate two marker coordinates, may indicate one marker coordinate, or may indicate that there is no marker coordinate.

  The operation button data 66 is data indicating an input state for each of the operation buttons 32a to 32i.

  The game process data 67 is data used in a game process (FIG. 14) described later. The game processing data 67 includes first posture data 68, acceleration magnitude data 69, correction degree data 70, correction amount vector data 71, correction matrix data 72, roll posture component data 73, yaw posture component data 74, pitch posture component. Data 75, third attitude data 76, virtual plane coordinate data 77, protrusion vector data 78, camera orientation data 79, camera position data 80, and cursor position data 81 are included. In addition to the data shown in FIG. 13, the game processing data 67 includes various data (data indicating game parameters, etc.) used in the game processing.

上記行列Ｍ１は、所定の基準姿勢から現在の入力装置８の姿勢への回転を表す回転行列である。以下では、第１の姿勢を示す行列Ｍ１を、「第１姿勢行列Ｍ１」と呼ぶ。なお、第１姿勢行列Ｍ１により表される第１の姿勢は、入力装置８が存在する空間の所定位置を基準としたｘｙｚ座標系（上記空間座標系）における姿勢である。ここでは、ｘｙｚ座標系は、入力装置８がマーカ部６の正面に位置することを前提とし、入力装置８の位置からマーカ部６を向く方向をｚ軸正方向とし、鉛直上向き（重力方向の逆方向）をｙ軸正方向とし、入力装置８の位置からマーカ部６を向いた場合の左方向をｘ軸正方向とした座標系であるとする。ここでは、上記所定の基準姿勢は、マーカ部６の正面に位置する入力装置８の撮像方向がマーカ部６の中央を向き、かつ、コントローラ５のボタン面が鉛直上向きとなる姿勢（すなわち、入力装置８を基準としたＸ軸、Ｙ軸、Ｚ軸が、それぞれｘ軸、ｙ軸、ｚ軸の向きと一致する姿勢）であるとする。なお、本実施形態では、行列を用いて第１の姿勢を表現することとしたが、他の実施形態においては、第１の姿勢は、３次のベクトルまたは３つの角度によって表現されてもよい。 The first attitude data 68 is data indicating the first attitude calculated using the angular velocity data 63. In the present embodiment, the first posture is expressed by a 3 × 3 matrix M1 shown in the following equation (1).

The matrix M1 is a rotation matrix representing the rotation from a predetermined reference posture to the current input device 8 posture. Hereinafter, the matrix M1 indicating the first posture is referred to as a “first posture matrix M1”. Note that the first posture represented by the first posture matrix M1 is a posture in the xyz coordinate system (the spatial coordinate system) with reference to a predetermined position in the space where the input device 8 exists. Here, the xyz coordinate system is based on the premise that the input device 8 is located in front of the marker unit 6, and the direction from the input device 8 toward the marker unit 6 is the z-axis positive direction, and is vertically upward (in the gravitational direction). It is assumed that the coordinate system is such that the reverse direction) is the y-axis positive direction and the left direction when facing the marker unit 6 from the position of the input device 8 is the x-axis positive direction. Here, the predetermined reference posture is a posture in which the imaging direction of the input device 8 located in front of the marker unit 6 faces the center of the marker unit 6 and the button surface of the controller 5 is vertically upward (that is, input) Assume that the X axis, the Y axis, and the Z axis with respect to the device 8 are postures that coincide with the directions of the x axis, the y axis, and the z axis, respectively. In the present embodiment, the first posture is expressed using a matrix. However, in other embodiments, the first posture may be expressed by a cubic vector or three angles. .

  The acceleration magnitude data 69 is data indicating the magnitude (length) L of the acceleration vector Va1 indicated by the acceleration data 64.

  The correction degree data 70 is data indicating the degree of correction of the first attitude using the second attitude (correction degree A). The correction degree A is a value that can take a range of 0 ≦ A ≦ C1 (C1 is a predetermined constant of 0 <C1 ≦ 1). Although details will be described later, the corrected first posture becomes closer to the second posture as the correction degree A is larger.

  The correction amount vector data 71 is data indicating a vector indicating a correction amount for correcting the first posture (vector v3 shown in FIG. 9; hereinafter referred to as a correction amount vector). The correction amount vector Vg is calculated based on the vector Va2 representing the acceleration vector Va1 in the xyz coordinate system and the correction degree A.

  The correction matrix data 72 is data indicating a rotation matrix (referred to as a correction matrix) Ma used for correcting the first posture. That is, in the first correction process, the first posture is corrected by multiplying the first posture matrix M1 representing the first posture by the correction matrix Ma. The correction matrix Ma is calculated based on the vector Va2 and the correction amount vector Vg.

  The roll posture component data 73 is data indicating a posture component (roll posture component) M3r related to the roll direction among the posture components included in the third posture calculated from the image to be captured. The yaw posture component data 74 is data indicating the posture component (yaw posture component) M3y related to the yaw direction among the posture components included in the third posture, and the pitch posture component data 75 is the third posture information. Among the posture components included in the posture, this is data indicating a posture component (pitch posture component) M3p related to the pitch direction. Note that the roll direction, yaw direction, and pitch direction referred to here are rotation directions when the imaging direction (Z-axis positive direction) of the input device 8 is used as a reference. In the present embodiment, each of the posture components M3r, M3y, and M3p is represented by a 3 × 3 matrix as in the first posture.

  The third attitude data 76 is data indicating a third attitude calculated from the image to be imaged. In the present embodiment, the third posture is represented by a 3 × 3 matrix M3, like the first posture. Hereinafter, the matrix M3 indicating the third posture is referred to as a “third posture matrix M3”. In the present embodiment, since the marker coordinate data is transmitted as operation data from the input device 8, the third posture matrix M <b> 3 is calculated based on the marker coordinate data 65. Specifically, the third posture matrix M3 is obtained by combining the posture components M3r, M3y, and M3p.

  The virtual plane coordinate data 77 is data indicating two-dimensional coordinates (referred to as “virtual plane coordinates”) indicating a position on a predetermined plane (plane Q shown in FIG. 18) in a predetermined virtual space. The virtual plane coordinates are calculated based on the attitude of the input device 8. In the present embodiment, the virtual plane coordinates are used for the game process, and the display range displayed on the screen of the television 2 among the display targets (game map, web page, etc.) is determined based on the virtual plane coordinates. .

  The protrusion vector data 78 is data indicating a vector (referred to as an “extrusion vector”) indicating the direction and amount in which the virtual plane coordinates protrude from a predetermined area (area Ar shown in FIG. 23) on the plane Q ( (See FIG. 23). In the present embodiment, the display range to be displayed on the screen is determined based on the protrusion vector.

  The camera orientation data 79 is data indicating the orientation (posture) of the virtual camera. The camera position data 80 is data indicating the position of the virtual camera in the virtual space. Here, in this embodiment, a display target and a virtual camera are set in a virtual space (different from the virtual space in which the plane Q is arranged), and an image is generated by a method using the virtual camera. Therefore, the display range displayed on the screen of the television 2 is determined by the position and orientation of the virtual camera. In the present embodiment, the orientation (posture) of the virtual camera is represented by a 3 × 3 rotation matrix as in the first posture of the input device 8. However, in other embodiments, the orientation of the virtual camera may be represented by a cubic vector or three angles.

  The cursor position data 81 is data indicating the position of the cursor on the screen of the television 2. In the present embodiment, coordinates (screen coordinates) representing a position on the screen are calculated based on the virtual surface coordinates, and a cursor is displayed at the position. That is, the cursor position data 81 indicates screen coordinates calculated based on the virtual plane coordinates.

  Next, details of processing performed in the game apparatus 3 will be described with reference to FIGS. FIG. 14 is a main flowchart showing the flow of processing executed in the game apparatus 3. When the power of the game apparatus 3 is turned on, the CPU 10 of the game apparatus 3 executes a startup program stored in a boot ROM (not shown), whereby each unit such as the main memory is initialized. Then, the game program stored in the optical disc 4 is read into the main memory, and the CPU 10 starts executing the game program. The flowchart shown in FIG. 14 is a flowchart showing processing performed after the above processing is completed.

  First, in step S1, the CPU 10 executes an initialization process relating to the game. In this initialization process, the values of various parameters used in the game process are initialized, a virtual game space is constructed, and player objects and other objects are placed at initial positions in the game space. Following step S1, the process of step S2 is executed.

  In step S2, the CPU 10 executes an initial posture setting process. Specifically, a predetermined value is set as the initial posture of the input device 8 in response to the player performing a predetermined operation (for example, an operation of pressing the A button 32d). Here, the reference posture is a posture in which the Z axis is parallel to the vertical direction and the imaging direction of the input device 8 faces the center of the marker unit 6 (the center of the markers 6R and 6L). It is desirable to perform the predetermined operation while holding the input device 8 so that the initial posture becomes the reference posture, but if the input device is close to a stationary state and the marker portion can be imaged, It is possible to calculate the initial posture. When a predetermined operation is performed, the CPU 10 stores data indicating the matrix representing the initial posture in the main memory as first posture data. After the above step S2, the processing loop of steps S3 to S9 is repeatedly executed while the game is executed. Note that one processing loop is executed once per frame time (for example, 1/60 seconds).

  In this embodiment, the initial posture setting process (step S2) is executed only once before the game starts (before the processing loop of steps S3 to S9 is executed). In the embodiment, the initial posture setting process may be executed at an arbitrary timing during the game. That is, the CPU 10 may execute the initial posture setting process in response to the player performing the predetermined operation during the game.

  In step S3, the CPU 10 acquires operation data. That is, operation data transmitted from the controller 5 is received via the wireless controller module 19. Then, angular velocity data, acceleration data, marker coordinate data, and operation button data included in the received operation data are stored in the main memory. Following step S3, the process of step S4 is executed.

  In step S4, the CPU 10 calculates the first posture based on the angular velocity data 63 stored in the main memory. Any method may be used to calculate the attitude of the input device 8 from the angular velocity. However, in the present embodiment, the first attitude is the previous first attitude (calculated in the previous processing loop). (First attitude) and the current angular velocity (the angular velocity acquired in the current processing loop). Specifically, the CPU 10 sets the posture obtained by rotating the previous first posture by unit time at the current angular velocity as the first posture. The previous first posture is indicated by the first posture data 68 stored in the main memory, and the current angular velocity is indicated by the angular velocity data 63 stored in the main memory. Data indicating the posture (3 × 3 matrix) calculated in step S4 is newly stored in the main memory as the first posture data 68. Following step S4, the process of step S5 is executed.

  In step S5, the CPU 10 executes the first correction process described above. The first correction process is a process of correcting the first posture using acceleration data. Hereinafter, the details of the first correction process will be described with reference to FIG.

  FIG. 15 is a flowchart showing the flow of the first correction process (step S5) shown in FIG. In the first correction process, first, in step S <b> 11, the CPU 10 calculates the magnitude L of the acceleration detected by the acceleration sensor 37. That is, the acceleration data 64 stored in the main memory is read, and the magnitude L is calculated for the acceleration vector Va1 indicated by the acceleration data 64. Data indicating the calculated magnitude L is stored as acceleration magnitude data 69 in the main memory. Following step S11, the process of step S12 is executed.

  In step S12, the CPU 10 determines whether or not the magnitude of the acceleration detected by the acceleration sensor 37 is zero. That is, the acceleration magnitude data 69 stored in the main memory is read, and it is determined whether or not the magnitude L indicated by the acceleration magnitude data 69 is zero. If the determination result of step S12 is negative, the process of step S13 is executed. On the other hand, when the determination result of step S12 is affirmative, the processes of subsequent steps S13 to S21 are skipped, and the CPU 10 ends the first correction process. Thus, in this embodiment, when the magnitude of the acceleration detected by the acceleration sensor 37 is 0, correction using the acceleration is not performed. This means that the direction of gravity cannot be calculated from the detection result of the acceleration sensor 37 when the magnitude of the acceleration is 0, and the subsequent steps S13 to S21 are performed if the magnitude of the acceleration vector is 0. The reason is that it is difficult.

  In step S <b> 13, the CPU 10 normalizes the acceleration vector Va <b> 1 detected by the acceleration sensor 37. That is, the acceleration data 64 stored in the main memory is read, and the acceleration vector Va1 indicated by the acceleration data 64 is corrected so that the magnitude is 1. The CPU 10 stores data indicating the normalized acceleration vector Va1 in the main memory. Following step S13, the process of step S14 is executed.

In step S <b> 14, the CPU 10 calculates the correction degree A indicating the degree of correction of the first posture in the first correction process. The correction degree A is calculated based on the magnitude L of the acceleration vector Va1 before normalization. Specifically, the CPU 10 reads acceleration magnitude data 69 stored in the main memory. Then, using the magnitude L indicated by the acceleration magnitude data 69, the correction degree A is calculated according to the following equation (2).
A = | L−1 | (2)
Data indicating the correction degree A calculated by the above equation (2) is stored as correction degree data 70 in the main memory. The correction degree A calculated by the above equation (2) is a value that is not a final value and is being calculated, and the final correction degree A is obtained by converting the value in the subsequent step S16. A value is obtained. Following step S14, the process of step S15 is executed.

  In step S15, the CPU 10 determines whether or not the correction degree A calculated in step S14 is smaller than a predetermined value R. The predetermined value R is determined in advance and is set to 0.4, for example. Here, as described above, in the present embodiment, the magnitude of the gravitational acceleration detected by the acceleration sensor 37 is “1”, and the correction degree A is the magnitude L of the acceleration vector Va1 and “1”. (The above formula (2)). Therefore, the case where the correction degree A is equal to or greater than the predetermined value R is a case where the magnitude L of the acceleration vector Va1 is separated from the magnitude of the gravitational acceleration by the predetermined value R or more. If the determination result of step S15 is affirmative, the process of step S16 is executed. On the other hand, when the determination result of step S15 is negative, the processes of subsequent steps S16 to S21 are skipped, and the CPU 10 ends the first correction process.

  As described above, in the present embodiment, only when the difference between the acceleration magnitude L detected by the acceleration sensor 37 and the gravitational acceleration magnitude (= 1) is smaller than a predetermined reference (predetermined value R). Correction is performed, and correction is not performed when the difference between the magnitude L and the magnitude of gravitational acceleration is equal to or greater than a predetermined reference. Here, in the state where the input device 8 is moved, in addition to the gravitational acceleration, the acceleration due to inertia caused by the movement of the input device 8 is detected by the acceleration sensor 37, so the magnitude of the detected acceleration vector Va1. L becomes a value different from “1”, and when the input device 8 is moved violently, the size L becomes a value far from “1”. Therefore, the case where the difference between the magnitude L and the magnitude of the gravitational acceleration is equal to or greater than a predetermined reference is estimated to be a case where the input device 8 is moved violently. On the other hand, when the input device 8 is moved violently, the acceleration vector Va1 detected by the acceleration sensor 37 contains many components other than gravitational acceleration (the acceleration component due to inertia), and therefore the acceleration vector Va1. The value of is presumed to be unreliable as a value indicating the direction of gravity. Therefore, the determination process of step S15 determines whether or not the input device 8 is moved violently, in other words, whether or not the value of the acceleration vector Va1 is reliable as a value indicating the direction of gravity. It is processing. In the present embodiment, if the value of the acceleration vector Va1 is not reliable as a value indicating the direction of gravity by the determination process in step S15, no correction is performed, and the value of the acceleration vector Va1 is a value indicating the direction of gravity. Corrections are made only when reliable. As a result, it is possible to prevent the first posture from being correctly corrected as a result of performing the correction on the first posture using the acceleration vector Va1 that is not reliable as a value indicating the direction of gravity.

In step S16, the CPU 10 converts the value of the correction degree A. In the present embodiment, the correction degree A is converted so that the correction degree A becomes closer to 1 as the magnitude L of the detected acceleration vector Va1 is closer to the magnitude of gravitational acceleration. Specifically, the CPU 10 reads the correction degree data 70 stored in the main memory, and converts the correction degree A indicated by the correction degree data 70 according to the following equations (3) to (5).
A2 = 1- (A1 / R) (3)
A3 = A2 × A2 (4)
A4 = A3 × C1 (5)
In the above formulas (3) to (5), the variable A1 is the correction degree before conversion (value indicated by the correction degree data 70 currently stored in the main memory), and the variable A4 is finally converted in step S16. Is the degree of correction. The above expression (3) is an expression for obtaining the corrected degree of correction A2 so that the degree of correction A1 before the conversion becomes closer to 1 as the magnitude of the gravitational acceleration (= 1) is closer. The above equation (4) is an equation for obtaining the corrected degree of correction A3 so that a larger weight is assigned as the correction degree A2 before conversion is closer to 1. The above equation (5) is an equation for adjusting the magnitude of the correction amount. That is, the correction amount of the constant C1 increases as the value of the constant C1 increases. The constant C1 is determined in advance and is set in a range of 0 <C1 ≦ 1 (for example, 0.03). Data indicating the correction degree A4 obtained by the conversion according to the above equations (3) to (5) is stored as new correction degree data 70 in the main memory. Following step S16, the process of step S17 is executed.

  In the present embodiment, the conversions according to the above formulas (3) to (5) are performed. However, in other embodiments, some or all of the conversions in the above formulas (3) to (5) are omitted. May be. However, when the calculation is such that the value of the correction degree is close to 0 so that the value of the acceleration vector is reliable due to omission of the equation or the like, in the equation (7) used in step S18 described later, It is necessary to replace the gravity direction vector (0, -1, 0).

In step S17, the CPU 10 converts the acceleration vector Va1 expressed in the XYZ coordinate system into a value Va2 in the xyz coordinate system. The acceleration vector Va2 in the xyz coordinate system is calculated by converting the normalized acceleration vector Va1 using the first posture matrix M1 representing the first posture obtained in the previous frame. That is, the CPU 10 reads the data of the acceleration vector Va1 (normalized) stored in the main memory in step S13 and the first attitude data 68. Then, using the acceleration vector Va1 and the first posture matrix M1 indicated by the first posture data 68, an acceleration vector Va2 in the xyz coordinate system is calculated. More specifically, normalized acceleration vector Va1 = (nx, ny, nz), each element of the first posture matrix M1 is a variable shown in the above equation (1), and acceleration expressed in the xyz coordinate system If the vector Va2 = (vx, vy, vz), the acceleration vector Va2 is calculated according to the following equation (6).
vx = Xx * nx + Yx * ny + Zx * nz
vy = Xy * nx + Yy * ny + Zy * nz
vz = Xz * nx + Yz * ny + Zz * nz (6)
As shown in the above equation (6), the acceleration vector Va2 can be obtained by rotating the acceleration vector Va1 by the first posture matrix M1 that is a rotation matrix. The acceleration vector Va2 calculated in step S17 is stored in the main memory. Following step S17, the process of step S18 is executed.

In step S18, the CPU 10 calculates the correction amount vector Vg using the acceleration vector Va2 expressed in the xyz coordinate system and the correction degree A. The correction amount vector Vg is calculated using the degree of correction after the conversion in step S16 and a vector (0, -1, 0) indicating the vertical downward direction (gravity direction) of the xyz coordinate system. Specifically, the CPU 10 reads out the correction degree data 70 stored in the main memory, and uses the correction degree A indicated by the correction degree data 70, according to the following equation (7), the correction amount vector Vg = ( gx, gy, gz) is calculated.
gx = (0−vx) × A + vx
gy = (− 1−vy) × A + vy
gz = (0−vz) × A + vz (7)
As shown in the above equation (7), the correction amount vector Vg is a line segment connecting the end point of the acceleration vector Va2 to the end point of the gravity direction vector (0, -1, 0) as A: (1-A). It is a vector whose end point is a point to be internally divided. Accordingly, the correction amount vector Vg is closer to the gravity direction vector as the correction degree A is larger. The CPU 10 stores data indicating the correction amount vector Vg calculated by the above equation (7) as correction amount vector data 71 in the main memory. Following step S18, the process of step S19 is executed.

  In step S19, the CPU 10 normalizes the correction amount vector Vg calculated in step S18. That is, the correction amount vector data 71 stored in the main memory is read, and the vector indicated by the correction amount vector data 71 is normalized. Then, the data indicating the normalized vector is stored as new correction amount vector data 71 in the main memory. The correction amount vector Vg calculated in step S19 corresponds to the vector v3 shown in FIG. Following step S19, the process of step S20 is executed.

  In step S20, the CPU 10 calculates a correction matrix Ma for correcting the first posture. The correction matrix Ma is calculated based on the acceleration vector Va2 expressed in the xyz coordinate system and the correction amount vector Vg normalized in step S19. Specifically, the CPU 10 reads the acceleration vector Va2 and the correction amount vector data 71 stored in the main memory in step S17. Then, a rotation matrix for rotating the acceleration vector Va2 so as to coincide with the correction amount vector Vg is calculated, and the calculated rotation matrix is set as a correction matrix Ma. That is, the correction matrix Ma is a rotation matrix that performs the rotation of the angle Δθ shown in FIG. Data indicating the correction matrix Ma calculated in step S20 is stored in the main memory as correction matrix data 72. Following step S20, the process of step S21 is executed.

  In step S21, the CPU 10 corrects the first posture matrix M1 indicating the first posture with the correction matrix Ma. Specifically, the CPU 10 reads the first attitude data 68 and the correction matrix data 72 stored in the main memory. Then, the first attitude matrix M1 indicated by the first attitude data 68 is converted by the correction matrix Ma indicated by the correction matrix data 72 (the product of the first attitude matrix M1 and the correction matrix Ma is calculated). The converted first posture matrix M1 indicates the corrected first posture. That is, the process of step S21 corresponds to the process of rotating the vector v1 shown in FIG. 10 by the angle Δθ. The CPU 10 stores data indicating the converted first attitude matrix M1 as new first attitude data 68 in the main memory. After step S21, the CPU 10 ends the first correction process.

  As described above, in the first correction process, the correction amount vector Vg is calculated between the acceleration vector detected by the acceleration sensor 37 and the gravity direction vector (vector G shown in FIG. 8) (steps S18 and S19). ), The first posture is corrected by the correction amount (correction matrix Ma. Angle Δθ shown in FIG. 9) represented by the correction amount vector Vg (step S21). As a result, the first attitude calculated from the gyro sensors 55 and 56 (vector v1 or angle θ1 shown in FIG. 8) is corrected to approach the second attitude (angle θ2 shown in FIG. 8) determined by the acceleration sensor 37. It can be performed. By performing such correction, the first posture can be corrected so as to have a more accurate value.

  In the first correction process, as the acceleration vector Va1 is reliable (the smaller the difference between the magnitude L of the acceleration vector Va1 and the magnitude of gravitational acceleration), the value of the correction degree A increases. The first posture is corrected so as to be closer to the second posture. That is, the more reliable the acceleration vector Va1, the larger the correction amount, and the second posture is strongly reflected in the corrected first posture. As described above, according to the present embodiment, the correction amount in the first correction process is determined according to the reliability of the acceleration sensor vector Va1, and therefore the correction amount can be appropriately determined according to the reliability. The attitude of the input device 8 can be calculated more accurately.

  In the present embodiment, the correction amount vector Vg calculated in step S18 is a point that internally divides the line segment connecting the end point of the acceleration vector Va2 to the end point of the gravity direction vector into A: (1-A). Is the vector that approaches the gravity direction vector as the value of the correction degree A increases. In another embodiment, depending on the method of calculating the correction degree A, the correction amount vector Vg is a point that internally divides the end point of the gravity direction vector and the end point of the acceleration vector Va2 into (1-A): A. The end point vector may be a vector that approaches the gravity direction vector as the correction degree A is smaller. At this time, in step S20, a rotation matrix for rotating the correction amount vector Vg so as to coincide with the direction of gravity may be calculated, and the calculated rotation matrix may be used as the correction matrix Ma. This also makes it possible to perform correction as in the present embodiment.

  Returning to the description of FIG. 14, in step S <b> 6 subsequent to step S <b> 5, the CPU 10 executes the second correction process described above. The second correction process is a process of correcting the first posture using the marker coordinate data. Hereinafter, the details of the second correction process will be described with reference to FIG.

  FIG. 16 is a flowchart showing the flow of the second correction process (step S6) shown in FIG. In the first correction process, first, in step S <b> 31, the CPU 10 determines whether or not the marker unit 6 is imaged by the imaging unit (imaging device 40) of the input device 8. The determination in step S31 can be made by referring to the marker coordinate data 65 stored in the main memory. Here, when the marker coordinate data 65 indicates two marker coordinates, it is determined that the marker unit 6 is captured, and when the marker coordinate data 65 indicates only one marker coordinate, or there is no marker coordinate. When it shows, it determines with the marker part 6 not being imaged. If the determination result of step S31 is affirmative, the processes of subsequent steps S32 to S37 are executed. On the other hand, when the determination result of step S31 is negative, the processes of the subsequent steps S32 to S37 are skipped, and the CPU 10 ends the second correction process. As described above, when the marker unit 6 is not imaged by the image sensor 40, the attitude of the input device 8 cannot be calculated using data obtained from the image sensor 40. In this case, the second correction is performed. No correction is performed in the process.

  In step S32, the CPU 10 calculates a roll posture component M3r based on the marker coordinate data. The roll posture component M3r is calculated based on the orientation of the marker unit 6 in the captured image, that is, based on the slope of the line connecting the two marker coordinates indicated by the marker coordinate data 65. Hereinafter, an example of a method for calculating the roll posture component M3r will be described with reference to FIG.

  FIG. 17 is a diagram illustrating two-dimensional coordinates corresponding to a captured image. As shown in FIG. 17, in the present embodiment, a two-dimensional coordinate system (x′y ′ coordinate system) for representing a position in a captured image has a range of the captured image of −1 ≦ x ′ ≦ 1, − 1 ≦ y ′ ≦ 1. The x′y ′ coordinate system captures an image when the input device 8 is in a reference posture (an orientation in which the imaging direction of the input device 8 faces the center of the marker unit 6 and the button surface of the controller 5 is vertically upward). In the image, it is assumed that the vertically downward direction is the positive direction of the y ′ axis and the right direction is the positive direction of the x ′ axis. Also, point P1 and point P2 shown in FIG. 17 indicate the positions of the marker coordinates, and point P3 is a midpoint between point P1 and point P2. A vector v10 shown in FIG. 17 is a vector having a point P1 as a start point and a point P2 as an end point.

上式（８）によって算出された行列を示すデータは、ロール姿勢成分データ７３としてメインメモリに記憶される。ステップＳ３２の次にステップＳ３３の処理が実行される。 In order to calculate the roll posture component M3r, the CPU 10 first reads the marker coordinate data 65, and calculates the vector v10 from the two marker coordinates indicated by the marker coordinate data 65. Further, a vector (hx, hy) obtained by normalizing the vector v10 is calculated. The vector (hx, hy) is directed in the positive x-axis direction when the input device 8 is in the reference posture, and the direction changes according to the rotation of the input device 8 in the roll direction. Since the vector (hx, hy) corresponds to the posture in the roll direction, the roll posture component M3r can be calculated based on the vector (hx, hy). Specifically, the CPU 10 calculates the roll posture component M3r according to the following equation (8).

Data indicating the matrix calculated by the above equation (8) is stored in the main memory as roll posture component data 73. Following step S32, the process of step S33 is executed.

  In step S33, the CPU 10 calculates a yaw posture component M3y based on the marker coordinate data. The yaw posture component M3y is calculated based on the orientation and position of the marker unit 6 in the captured image. Hereinafter, an example of a method for calculating the yaw posture component M3y will be described with reference to FIG.

  First, the CPU 10 reads the marker coordinate data 65 and calculates the midpoint between the two marker coordinates indicated by the marker coordinate data 65. In the present embodiment, the position of the middle point is used as the position of the marker unit 6. Further, the CPU 10 rotates the calculated midpoint coordinates by the rotation angle related to the roll direction of the input device 8 (in the direction opposite to the rotation direction of the input device 8) around the origin of the x′y ′ coordinate system. Calculated coordinates (px, py) are calculated. In other words, the coordinates of the midpoint are rotated around the origin so that the vector (hx, hy) faces the positive x-axis direction. If the input device 8 is at the same position as the marker unit 6 in the horizontal direction (x-axis direction) (that is, the front position of the marker unit 6), the coordinates (px, py) can be used to calculate the attitude related to the yaw direction.

Next, the CPU 10 based on the coordinates (px, py) after the rotation of the midpoint and the angle (limit angle) θy ′ in the yaw direction when the marker unit 6 is located at the end in the x′-axis direction. Then, the rotation angle θy related to the yaw direction is calculated. Here, the limit angle θy ′ and the x-coordinate value px ′ after the rotation of the midpoint when the limit angle θy1 is reached can be obtained in advance. Therefore, the rotation angle θy related to the yaw direction can be calculated using the fact that the ratio between px and px ′ is equal to the ratio between θy and θy ′. Specifically, the rotation angle θy related to the yaw direction can be calculated by the following equation (9).
θy = px × θy ′ / px ′ (9)
When the horizontal length of the marker unit 6 is ignored, the limit angle θy ′ can be set to ½ of the angle of view of the controller 5 and the value of px ′ can be set to “1”.

上式（１０）によって算出された行列を示すデータが、ヨー姿勢成分データ７４としてメインメモリに記憶される。以上のステップＳ３３の次にステップＳ３４の処理が実行される。 Finally, the CPU 10 calculates a rotation matrix for performing the rotation of the angle θy calculated by the above equation (9) as the yaw attitude component M3y. Specifically, the yaw posture component M3y is calculated by the following equation (10).

Data indicating the matrix calculated by the above equation (10) is stored in the main memory as yaw attitude component data 74. Following step S33, the process of step S34 is executed.

  In step S34, the CPU 10 combines the roll posture component M3r and the yaw posture component M3y. That is, the roll attitude component data 73 and the yaw attitude component data 74 are read from the main memory, and the roll attitude component M3r and the yaw attitude component M3y indicated by the data 73 and 74 are integrated. Following step S34, the process of step S35 is executed.

  In step S35, the CPU 10 calculates a pitch posture component M3p based on the first posture. Although different from the processing of the present embodiment, the pitch attitude component M3p can also be calculated based on the y coordinate value of the coordinates (px, py) in the same manner as the yaw attitude component M3y. . However, in the method of calculating the attitude in the yaw direction (pitch direction) using the coordinates (px, py), the input device 8 is at the same position as the marker unit 6 in the horizontal direction (vertical direction in the case of the pitch direction). This method is based on the assumption. In the game system 1 of the present embodiment, it is considered that the player operates the input device 8 at a position substantially in front of the marker unit 6 (TV 2) in the horizontal direction. It is possible to calculate the attitude in the yaw direction by the method of step S33 on the assumption that “they are at the same position”. On the other hand, it is conceivable that the player stands and operates the input device 8 or sits down and operates the input device 8, and the position of the marker unit 6 is also arranged on the upper side of the screen of the television 2. It is also possible to arrange them in For this reason, in the game system 1 of the present embodiment, it is not necessarily assumed that “the input device 8 is at the same position as the marker unit 6 in the vertical direction”. Therefore, the coordinates in the pitch direction are determined using the coordinates (px, py). The posture may not be calculated.

上式（１２）によって算出された行列を示すデータが、ピッチ姿勢成分データ７５としてメインメモリに記憶される。以上のステップＳ３５の次にステップＳ３６の処理が実行される。 Therefore, in the present embodiment, the first posture is used as it is for the pitch posture component M3p (thus, in the second correction process, the pitch direction is not corrected). Specifically, the CPU 10 reads the first attitude data 68 from the main memory. Then, using each element of the first posture matrix M1 indicated by the first posture data 68, the rotation angle θp with respect to the pitch direction is calculated according to the following equation (11).
cos (θp) = (Zx × Zx + Zz × Zz) ^1/2
sin (θp) = Zy (11)
Variables Zx, Zy, and Zz in the above equation (11) are elements of the first posture matrix M1 shown in the above equation (1). The first posture matrix M1 used here is the first posture matrix M1 after the first correction processing is performed in the current processing loop. Further, the CPU 10 calculates a matrix of the pitch posture component M3p according to the following equation (12) using cos (θp) and sin (θp) calculated by the above equation (11).

Data indicating the matrix calculated by the above equation (12) is stored in the main memory as pitch attitude component data 75. Following step S35, the process of step S36 is executed.

  In step S36, the CPU 10 calculates a third posture based on the posture components in the roll direction, the yaw direction, and the pitch direction. The third attitude is obtained by further synthesizing the pitch attitude component M3p with the synthesis result of the roll attitude component M3r and the yaw attitude component M3y. Specifically, the CPU 10 reads the pitch posture component data 75 from the main memory, and integrates the pitch posture component M3p indicated by the pitch posture component data 75 in the matrix calculated in step S34. Data indicating the calculated matrix is stored in the main memory as third attitude data 76. Following step S36, the process of step S37 is executed.

In step S37, the CPU 10 corrects the first attitude using the third attitude. The correction in step S37 is performed by bringing the first posture matrix M1 closer to the third posture matrix M3 at a predetermined ratio (the following constant C2). The CPU 10 reads the first attitude data 68 and the third attitude data 76 from the main memory. Then, correction is performed according to the following equation (13) using the first posture matrix M1 indicated by the first posture data 68 and the third posture matrix M3 indicated by the third posture data 76.
M1 = (M3-M1 ′) × C2 + M1 ′ (13)
In the above equation (13), the variable M1 ′ is the first posture matrix before correction. The constant C2 is preset in the range of 0 <C2 ≦ 1, and is set to 0.1, for example. Data indicating the corrected first posture matrix M1 calculated by the above equation (13) is stored in the main memory as new first posture data 68. After step S37, the CPU 10 ends the second correction process.

  As described above, in the second correction process, the third posture is calculated from the captured image (marker coordinates), and the first posture is corrected so as to approach the third posture. By this correction, the first posture can be corrected so as to have a more accurate value. In the present embodiment, the third posture is calculated from the captured image only for the posture in the roll direction and the yaw direction. However, as described above, the third posture is calculated from the captured image also in the pitch direction. In other embodiments, the third orientation may be calculated from the captured image in the roll direction, the yaw direction, and the pitch direction. In the second correction process, the third posture may be calculated in at least one of the roll direction, the yaw direction, and the pitch direction.

  Returning to the description of FIG. 14, in step S <b> 7 subsequent to step S <b> 6, the CPU 10 calculates two-dimensional coordinates (virtual plane coordinates) based on the corrected first posture. Although details will be described later, in the present embodiment, the two-dimensional coordinates are used as input by the player in a game process (step S8) described later. Hereinafter, with reference to FIG. 18 and FIG. 19, the detail of step S7 is demonstrated.

  FIG. 18 is a diagram in which an input device and a predetermined plane are virtually arranged in a predetermined virtual space. In the present embodiment, as shown in FIG. 18, the CPU 10 defines a three-dimensional virtual space, and calculates coordinates assuming that the input device 8 and a predetermined plane Q are virtually arranged. The plane Q corresponds to the screen of the television 2. The CPU 10 calculates the coordinates of the position on the plane Q indicated by the direction of the Z-axis (Z-axis vector VZ shown in FIG. 18) of the input device 8 in the virtual space. That is, the coordinates of the intersection R between the line segment obtained by extending the Z-axis vector VZ representing the attitude of the input device 8 in the virtual space and the predetermined plane Q in the virtual space are calculated. The “line segment obtained by extending the Z-axis vector VZ” means a line segment that is parallel to the Z-axis vector VZ and passes through the Z-axis vector VZ. Although details will be described later, in this embodiment, a cursor or the like is displayed at a position on the screen corresponding to the calculated coordinates. Thereby, the position of the cursor on the screen is controlled according to the posture of the input device 8, and the player can move the cursor by an operation for changing the posture of the input device 8.

  As shown in FIG. 18, in this embodiment, the position of the virtual space is expressed by the above-described spatial coordinate system (xyz coordinate system). As a result, the first attitude data 68 can be used as it is as data indicating the attitude of the input device 8, so that calculation in the virtual space can be easily performed. Further, the position on the plane Q is expressed by the X′Y ′ coordinate system. At this time, the plane Q is set to be parallel to the xy plane of the spatial coordinate system, the x axis of the spatial coordinate system and the X ′ axis of the plane Q are parallel, and the y axis of the spatial coordinate system and the Y ′ of the plane Q are set. The X′Y ′ coordinate system is set so that the axes are parallel (FIG. 18).

  In the present embodiment, it is assumed that the player uses the input device 8 at a position substantially in front of the screen of the television 2 and the position of the input device 8 does not change in the virtual space. In other words, the CPU 10 performs the process without changing the position of the input device 8 because only the attitude of the input device 8 changes in the virtual space. As a result, the position on the plane Q (the position of the intersection R) can be uniquely determined from the attitude of the input device 8.

  Next, a method for calculating the position on the plane Q (the position of the intersection R) will be described in detail. FIG. 19 is a view of the virtual space shown in FIG. 18 as viewed from the y-axis positive direction side to the negative direction side. In FIG. 19, the length from a point S representing the position of the input device 8 to a point (projection point) T obtained by projecting the point S onto the plane Q is “L”. The length from the projection point T to the intersection point R with respect to the X ′ component is “Wx”. In the present embodiment, the value of the length L is set by the player at a predetermined timing (for example, timing when game processing is started). The main memory stores data indicating the value of the length L set by the player (referred to as length data).

The Z-axis vector VZ is determined by a first posture matrix M1 that represents the posture of the input device 8 (first posture). That is, the Z-axis vector VZ is a vector (= (Zx, Zy, Zz)) in which each element of the third row of the first posture matrix M1 is in turn an x component, a y component, and a z component. As is clear from FIG. 19, the ratio between the length Wx and the length L is equal to the ratio between the x component (Zx) of the Z-axis vector VZ and the z component (Zz) of the Z-axis vector VZ. Therefore, based on this relationship, the length Wx can be calculated from the x component Zx, the z component Zz, and the known length L of the known vector VZ. The length Wx can be calculated by (14).
Wx = L × Zx / Zz (14)
Similarly to the length Wx of the X ′ component, the length Wy of the Y ′ component from the projection point T to the intersection R can be calculated by the following equation (15).
Wy = L × Zy / Zz (15)

  If the lengths Wx and Wy are obtained, two-dimensional coordinates indicating the position of the intersection point R on the plane Q can be calculated. In the present embodiment, the position of the projection point T is the origin of the X′Y ′ coordinate system. At this time, the two-dimensional coordinates of the intersection R are (Wx, Wy).

  The specific processing in step S7 will be described. First, the CPU 10 reads the first attitude data 68 and the length data from the main memory. Then, Zx, Zy, and Zz included in the first posture matrix M1 indicated by the first posture data 68 and the length L indicated by the length data are substituted into the above equations (14) and (15). To calculate the lengths Wx and Wy. As a result, the two-dimensional coordinates (Wx, Wy) of the intersection R on the plane Q are obtained. The CPU 10 stores data indicating the two-dimensional coordinates in the main memory 32 as the virtual plane coordinate data 77. Following step S7, the process of step S8 is executed.

  As described above, according to step S7 described above, a virtual space in which the input device 8 and the predetermined plane Q are arranged is set, and on the plane Q indicated by the direction of the predetermined axis (Z axis) of the input device 8 in the virtual space. 2D coordinates can be obtained from the attitude of the input device 8.

  In the above embodiment, the player can set the length L. Here, as is apparent from the above equations (14) and (15), the value of the calculated two-dimensional coordinates (Wx, Wy) can be changed by adjusting the length L. That is, by adjusting the length L, the amount of change in the two-dimensional coordinates (Wx, Wy) with respect to the change in the direction of the Z-axis vector VZ (that is, the change in the attitude of the input device 8) can be adjusted. it can. Specifically, the amount of change increases as the value of the length L increases. As a result, the cursor moves greatly only by slightly changing the attitude of the input device 8. On the other hand, as the value of the length L becomes smaller, the change amount becomes smaller. As a result, even if the posture of the input device 8 is greatly changed, the cursor moves only slightly. As described above, in the above embodiment, the player himself / herself can adjust the operational feeling of the input device 8 by causing the player to set the length L. For example, the player may set the length L relatively small when it is necessary to finely operate the cursor, and the length L is relatively set when an operation to move the cursor greatly is required. Should be set larger. In another embodiment, the length of L may be a predetermined constant.

  In another embodiment, the length L may be calculated by the game apparatus 10 using a predetermined method. For example, the CPU 10 may calculate an actual distance from the input device 8 to the screen of the television 2 and set the calculated distance as the length L. The actual distance can be calculated using, for example, the length between the two markers 6R and 6L in the captured image captured by the image sensor 40 or the size of the marker 6R or 6L. Furthermore, if the plane Q is set so that the position and size of the plane Q in the virtual space correspond to the position and size of the screen of the television 2 in the real space (for example, the player sets the size of the television screen). The cursor is displayed at a position on the screen of the television 2 corresponding to the two-dimensional coordinates (Wx, Wy). According to this, the cursor can be displayed at a position (on the screen) indicated by the input device 8.

  As a method for obtaining the two-dimensional coordinates using the input device 8, the two-dimensional coordinates can be obtained using the marker coordinates (marker coordinate data 65 shown in FIG. 13) described above. That is, since the marker coordinates change in accordance with the position pointed to by the input device 8, the position on the screen pointed to by the input device 8 can be calculated based on the marker coordinates. Therefore, if the marker portion 6 is present, the two-dimensional coordinates can be calculated using the marker coordinates. However, when the marker unit 6 cannot be used, or in an input device that is not equipped with an image sensor, the two-dimensional coordinates cannot be obtained by photographing with the marker unit 6. The method according to the present embodiment can obtain two-dimensional coordinates even when the imaging device 40 of the input device 8 is not capturing the marker unit 6, and therefore can be used even in an input device without the marker unit or imaging device. Is.

  In the present embodiment, the two-dimensional coordinates are calculated based on the first posture corrected by the first and second correction processes. Here, in another embodiment, the two-dimensional coordinates may be calculated based on the attitude of the input device 8 calculated by an arbitrary method, and the first and second correction processes may not be performed. Good.

  Moreover, in the said embodiment, the aspect which the player uses the Z-axis direction of the input device 8 toward the screen of the television 2 is assumed, and the Z-axis direction of the input device 8 faces the screen (marker part 6) of the television 2. In this case, the plane Q is arranged in the Z-axis direction. However, as described above, since the two-dimensional coordinates can be obtained even if the imaging element 40 does not capture the marker portion 6, the player points the Z-axis direction of the input device 8 in an arbitrary direction. It is possible to use. For example, the CPU 10 may set the plane Q in the Z-axis direction of the input device 8 at the time of the predetermined operation in response to the player performing a predetermined operation (for example, an operation of pressing an operation button). . According to this, since the plane Q can be arranged at an appropriate position according to the orientation of the input device 8 in the virtual space, the input device 8 can be in any orientation at the time of the predetermined operation. Two-dimensional coordinates can be calculated. That is, the player can use the input device 8 in an arbitrary direction.

  In the present embodiment, the game apparatus 3 calculates a two-dimensional coordinate so as to take the origin (0, 0) when the Z-axis direction of the input apparatus 8 faces the horizontal direction (referred to as a first method). )It was adopted. Here, the game apparatus 3 may calculate the two-dimensional coordinates by a second method described later, instead of the first method.

  Returning to the description of FIG. 14, in step S <b> 8 following step S <b> 7, the CPU 10 performs display control processing using the two-dimensional coordinates. The display control process is a process of controlling the display range to be displayed on the display device (television 2) among the display objects based on the two-dimensional coordinates calculated in step S7. The display control process is a process for scrolling the display target based on the two-dimensional coordinates. That is, in this embodiment, the user can scroll the screen displayed on the television 2 by an operation of changing the attitude of the input device 8. The display target may be anything, such as a field formed in the game space (including objects arranged on the field), a map representing the game space, or the like. It may be a Web page or an electronic program guide.

  In the present embodiment, the CPU 10 arranges the display target and the virtual camera in the virtual space, and changes the orientation and / or position of the virtual camera in the virtual space based on the virtual plane coordinates. Control shall be performed. Note that the virtual space in which the display target is arranged is a space different from the virtual space in which the two-dimensional coordinates calculated in step S7 exist. Hereinafter, for the purpose of distinguishing between the two, the virtual space in which the two-dimensional coordinates exist is referred to as “first virtual space”, and the virtual space in which the display target is arranged is referred to as “second virtual space”. is there.

  The display range control method may be any method as long as the display range is changed according to the virtual plane coordinates. For example, the CPU 10 may move the display target instead of the virtual camera. The display range control method may not be a method of calculating the display range using a virtual camera in a three-dimensional space. For example, the CPU 10 may set a display range on a two-dimensional plane that is a display target, and change the display range according to the virtual plane coordinates.

  Hereinafter, the details of the display control process will be described with reference to FIG. In the following description of the display control process, the two-dimensional coordinates calculated in step S7 are referred to as “virtual plane coordinates” for the purpose of distinguishing them from coordinates on the display screen (screen coordinates).

  FIG. 20 is a flowchart showing the flow of the display control process (step S8) shown in FIG. In the display control process in the present embodiment, first, in step S41, a camera orientation control process is executed. The camera orientation control process is a process for controlling the orientation of the virtual camera based on the virtual plane coordinates.

  FIG. 21 is a diagram illustrating an example of the relationship between the attitude of the input device 8 and the display range determined by the camera orientation control process. In FIG. 21, when the input device 8 is in the reference posture (here, the posture in which the Z axis of the input device 8 faces the screen of the television 2), a display range is set such that a part of the calendar is displayed on the screen. (See the upper part of FIG. 21). Here, when the input device 8 is turned to the right from the reference posture and the position pointed to by the input device 8 is removed to the right side of the screen (see the middle stage in FIG. 21), the display range changes to the right (image displayed on the screen). Moves to the left, and the line of sight tilts to the left and right with respect to the calendar). That is, in the above case, the CPU 10 controls the orientation of the virtual camera so that the display range moves to the right. When the input device 8 is directed upward from the reference posture and the position pointed to by the input device 8 is removed from the upper side of the screen (see the lower row in FIG. 21), the display range changes upward (the image moves downward, The line of sight is tilted up and down with respect to the calendar). As described above, according to the camera orientation control processing in the present embodiment, the user can control the virtual camera so that the orientation of the input device 8 and the orientation of the virtual camera correspond to each other. The direction and amount of scrolling the image can be manipulated by the direction and amount removed from the screen of the television 2. Details of the camera orientation control process will be described below.

  FIG. 22 is a flowchart showing the flow of the camera orientation control process (step S41) shown in FIG. In the camera orientation control process, first, in step S46, the CPU 10 determines whether or not the virtual plane coordinates are outside a predetermined area. Here, the predetermined area is an area set on the plane Q arranged in the first virtual space. FIG. 23 is a diagram illustrating the plane Q arranged in the first virtual space. In FIG. 23, a region Ar represented by a one-dot chain line is the predetermined region. Here, CPU10 calculates the position (cursor position mentioned later) on the screen of the television 2 based on a virtual surface coordinate (step S43 mentioned later). The position on the screen is calculated so that the predetermined area Ar corresponds to the area of the screen of the television 2. For example, when the virtual plane coordinates are located at the edge of the predetermined area Ar, the position of the edge of the screen is calculated as the position on the screen. That is, in the present embodiment, the predetermined area Ar is an area corresponding to the area of the screen of the television 2 on the plane Q. The predetermined area Ar preferably corresponds to the area of the screen of the television 2, but the edge of the predetermined area Ar does not have to correspond exactly to the edge of the area of the screen. Therefore, the predetermined area Ar is not limited to a rectangle, and may be an ellipse or a circle, for example. In other embodiments, the predetermined area Ar may be set in any manner.

  In the present embodiment, it is assumed that the predetermined area Ar is predetermined. However, the predetermined area Ar may be variable according to a user instruction or a game situation. For example, the CPU 10 determines the predetermined area Ar according to the length L (see FIG. 19) from the input device 8 to the plane Q in the first virtual space so that the predetermined area Ar corresponds to the area of the screen of the television 2. May be changed.

  The CPU 10 reads the data indicating the predetermined area Ar and the virtual plane coordinate data 77 that are determined in advance from the main memory, and determines the positional relationship between the virtual plane coordinates and the predetermined area Ar to perform the determination in step S46. If the determination result of step S46 is affirmative, the processes of steps S47 and S48 are executed. On the other hand, when the determination result of step S46 is negative, a process of step S49 described later is executed.

  In step S47, the CPU 10 calculates a protrusion vector. The protrusion vector is a vector representing the direction and amount in which the virtual surface coordinates protrude from the predetermined area Ar on the plane Q. Hereinafter, with reference to FIG. 23, a method of calculating the protrusion vector will be described. In FIG. 23, a point R is a position indicated by the virtual plane coordinates. The point P10 is the projection point T described above, and is the origin of the X′Y ′ coordinate system on the plane Q. At this time, the protrusion vector Vd is a vector having the intersection P11 between the line segment connecting the point P10 to the point R and the predetermined area Ar as the start point and the point R as the end point. Therefore, the protrusion vector Vd can be calculated based on the formula representing the predetermined area Ar and the virtual plane coordinates. Specifically, the CPU 10 reads data indicating the predetermined area Ar and the virtual plane coordinate data 77 from the main memory, and calculates the protrusion vector Vd based on the expression indicating the predetermined area Ar and the virtual plane coordinates. Data indicating the calculated protrusion vector Vd is stored in the main memory as the protrusion vector data 78. Following step S47, the process of step S48 is executed.

  In addition, the protrusion vector should just represent the direction and quantity which the virtual surface coordinate protrudes from the predetermined area | region Ar, and the calculation method of an protrusion vector is not restricted above. For example, the protrusion vector may be calculated so that the midpoint of the side closest to the virtual plane coordinate among the four sides of the predetermined area Ar is the start point (and the virtual plane coordinate point R is the end point). .

  In step S48, the CPU 10 controls the direction of the virtual camera based on the protrusion vector Vd. In the present embodiment, the direction of the virtual camera is such that the display range moves in a direction corresponding to the direction of the protruding vector Vd, and the display range moves by an amount corresponding to the size of the protruding vector Vd. Be controlled. Details of step S48 will be described below.

  First, the virtual space (second virtual space) in which the display target and the virtual camera are arranged will be described. FIG. 24 is a diagram illustrating a virtual space in which a display target and a virtual camera are arranged. In the present embodiment, the second virtual space is represented by an x ″ y ″ z ″ coordinate system, which is displayed when viewed from the front position of the planar display object 92. This is a coordinate system in which the direction toward the object 92 is the forward direction, the backward direction is the z "-axis positive direction, the upward direction is the y" -axis positive direction, and the right direction is the x "-axis positive direction.

The orientation (posture) of the virtual camera 91 is represented by a 3 × 3 rotation matrix, like the first posture of the input device 8. In this matrix, each component of the following three vectors when the line-of-sight direction of the virtual camera 91 is the forward direction is used as each element of the matrix.
-Vector CX = (CXx, CXy, CXz) pointing to the right (corresponding to the right of the screen)
-Vector CY = (CYx, CYy, CYz) pointing upward (corresponding to the upward direction of the screen)
-Vector CZ = (CZx, CZy, CZZ) facing backward
The matrix is a rotation matrix representing the rotation from a predetermined reference posture to the current virtual camera 91 posture. In the present embodiment, the predetermined reference posture is a posture in which the three vectors CX, CY, and CZ coincide with the directions of the x ″ axis, the y ″ axis, and the z ″ axis, respectively. In such a case, the line-of-sight direction of the virtual camera 91 faces in a direction perpendicular to the plane of the display object 92.

In step S48, the CPU 10 first calculates an angle (rotation angle) at which the virtual camera 91 should be rotated from the reference posture based on the protrusion vector Vd. In the present embodiment, the rotation angle rX around the y ″ axis of the virtual camera 91 is calculated according to the X ′ component dx of the protrusion vector Vd, and x ″ of the virtual camera 91 according to the Y ′ component dy of the protrusion vector Vd. A rotation angle rY about the axis is calculated. Specifically, the CPU 10 reads the protrusion vector data 78 from the main memory, and calculates the rotation angles rX and rY according to the following equation (16).
rX = dx × K1
rY = dy × K1 (16)
In the above equation (16), the constant K1 is predetermined. If the constant K1 is increased, the rotation angle of the virtual camera 91 is increased.

上式（１７）で算出された行列Ｍｃを示すデータは、カメラ向きデータ７９としてメインメモリに記憶される。以上のステップＳ４８の後、ＣＰＵ１０はカメラ向き制御処理を終了する。 Next, in step S48, the CPU 10 calculates the attitude of the virtual camera 91 rotated by the rotation angle from the reference attitude. Specifically, the matrix Mc representing the posture of the virtual camera 91 after rotation is calculated according to the following equation (17).

Data indicating the matrix Mc calculated by the above equation (17) is stored in the main memory as camera orientation data 79. After step S48, the CPU 10 ends the camera orientation control process.

  According to the above step S48, the change direction of the posture of the virtual camera 91 (direction changing from the reference posture) is determined according to the direction of the protrusion vector Vd, and the change amount of the posture of the virtual camera 91 (change from the reference posture). The amount, that is, the rotation angle is determined according to the magnitude of the protrusion vector Vd. Accordingly, the direction in which the display range changes (corresponding to the change direction of the attitude of the virtual camera 91) is determined according to the direction from the predetermined area Ar to the position indicated by the virtual plane coordinates (the direction of the protrusion vector Vd). Further, the degree to which the display range changes (corresponding to the amount of change in the attitude of the virtual camera) is determined according to the degree to which the position indicated by the virtual plane coordinates is away from the predetermined area Ar (the magnitude of the protrusion vector Vd). The

  In the process of step S48, only the orientation (posture) of the virtual camera is changed. However, in other embodiments, the orientation and position of the virtual camera may be changed. For example, when the virtual camera that is the reference posture is turned to the right, the CPU 10 may change only the posture of the virtual camera 91, or the posture of the virtual camera 91 is turned to the right and the position is moved to the right. Also good.

  On the other hand, in step S49, the CPU 10 controls the virtual camera 91 to be in the reference posture. The matrix representing the reference posture is a matrix when the variables rX and rY indicating the rotation angle are set to “0” in the above equation (17). The CPU 10 stores data indicating a matrix representing the reference posture in the main memory as camera orientation data 79. After step S48, the CPU 10 ends the camera orientation control process.

  According to the camera orientation control process described above, when the virtual plane coordinates are within the predetermined area Ar, the virtual camera 91 is fixed to the reference posture (step S49), so the display range does not move. Thus, in the present embodiment, the process of moving the display range (steps S47 and S48) is executed when the virtual plane coordinates indicate a position outside the predetermined area Ar on the plane Q. On the other hand, when the virtual plane coordinates are out of the predetermined area Ar, the orientation of the virtual camera 91 is controlled according to the direction in which the virtual plane coordinates are out of the predetermined area Ar and the degree to which the virtual plane coordinates are out. The display range moves according to the degree. Here, in this embodiment, the case where the virtual plane coordinates are within the predetermined area Ar is a case where the input device 8 faces the screen of the television 2, that is, the input device 8 points to a position on the screen. is there. Further, the case where the virtual plane coordinates are outside the predetermined area Ar is a case where the direction of the input device 8 deviates from the screen of the television 2, that is, the position pointed to by the input device 8 deviates from the screen. Therefore, according to the camera orientation control process, the user can move the display range by operating the input device 8 so that the orientation of the input device 8 is removed from the screen of the television 2 (see FIG. 21).

  In the camera orientation control process, the CPU 10 determines the amount of movement of the display range based on the degree to which the virtual plane coordinates are away from the predetermined area Ar. Here, in another embodiment, the CPU 10 may determine the speed of the display range (movement amount per unit time) based on the above degree. That is, when the virtual plane coordinates go out of the predetermined area Ar, the CPU 10 may rotate the virtual camera at an angular velocity corresponding to the above degree. According to this, even if the user stops the input device 8 in a state in which the direction of the input device 8 is removed from the screen of the television 2, the display range can be changed by rotating the virtual camera.

  Returning to FIG. 20, in step S42, the CPU 10 executes a camera position control process. The camera position control process is a process for controlling the position of the virtual camera based on the virtual plane coordinates. Hereinafter, the camera position control process will be described with reference to FIGS.

  Although various control methods in the camera position control process can be considered, four types of control methods will be described below as an example. FIG. 25 is a diagram illustrating examples of first to fourth control methods relating to position control of the virtual camera. In FIG. 25, the CPU 10 displays the cursor 93 at the position of the screen coordinates calculated from the virtual plane coordinates.

  The first control method is a method of controlling the position of the virtual camera in accordance with the movement amount and movement direction of the virtual plane coordinates. In the first control method, the virtual camera is moved in an amount corresponding to the moving amount of the virtual surface coordinates (in a predetermined period) and in a direction corresponding to the moving direction of the virtual surface coordinates. In the present embodiment, it is assumed that the virtual camera is moved by the first control method only during a period designated by the user (for example, a period during which the user presses a predetermined button of the input device 8). . According to the first control method, as shown in the upper part of FIG. 25, the user presses a predetermined button at a certain point in time, and moves the cursor 93 to the left while pressing the predetermined button (virtual plane coordinates). Is moved in the negative direction of the X ′ axis), the image displayed on the screen is moved together with the cursor 93 (that is, the display range is moved to the right). At this time, in this embodiment, the position of the virtual camera is controlled so that the position of the display target pointed by the cursor 93 (the position of the character “D” in FIG. 25) does not change before and after the movement of the display range. That is, in the present embodiment, the display range can be moved while the position of the display target pointed by the cursor 93 is fixed.

  The second control method is a method of continuously moving the virtual camera in a direction corresponding to the direction from the predetermined reference position to the virtual plane coordinates. In the second control method, the moving direction of the virtual camera is controlled according to the direction from the reference position to the position indicated by the virtual plane coordinates. In the present embodiment, the moving speed of the virtual camera is controlled according to the distance from the reference position to the position indicated by the virtual plane coordinates (see arrow 94 shown in FIG. 25). The predetermined reference position is, for example, the position of the virtual plane coordinates when the user presses (or starts to press) a predetermined button of the input device 8. However, the predetermined reference position may be any position, for example, a predetermined position. According to the second control method, when the user presses a predetermined button at a certain time and then moves the cursor 93 to the right, the display range continuously moves to the right (see the upper part of FIG. 25). ). In the present embodiment, the moving speed increases as the length increases. In the present embodiment, as in the first control method, the virtual camera is moved by the second control method only during a period specified by the user.

  The third and fourth control methods are methods of moving the camera when the virtual plane coordinates are outside the predetermined area Ar. In the present embodiment, the virtual camera is moved in a direction corresponding to the direction from the predetermined area Ar to the position indicated by the virtual plane coordinates. According to the third and fourth control methods, when the position pointed to by the input device 8 is moved from the inside of the screen to the outside of the screen, the display range moves in the direction in which the position goes out of the screen. For example, as shown in the lower part of FIG. 25, when the position indicated by the input device 8 is moved to the right side of the screen, the display range moves to the right.

  Note that the difference between the third control method and the fourth control method is in the method of calculating the moving speed of the virtual camera. That is, in the third control method, the moving speed of the virtual camera is calculated based on the degree to which the virtual plane coordinates are away from the predetermined area Ar. On the other hand, in the fourth control method, the moving speed of the virtual camera is calculated based on the speed of the virtual plane coordinates while the virtual plane coordinates are located outside the predetermined area Ar.

  Next, the details of the first control method will be described with reference to FIG. FIG. 26 is a flowchart showing the flow of the camera position control process (step S42) when the first control method is adopted. In this case, first, in step S51, the CPU 10 determines whether or not the current time is a predetermined period for moving the camera position. The predetermined period is a period specified by the user. In the present embodiment, the predetermined period is a period during which the B button 32i of the input device 8 is pressed. That is, the CPU 10 refers to the operation button data 66 stored in the main memory, and determines whether or not the B button 32i is pressed. If the determination result of step S51 is affirmative, the processes of steps S52 and S53 are executed. On the other hand, if the determination result of step S51 is negative, the CPU 10 ends the camera position control process. That is, when the B button 32i is not pressed, the position of the virtual camera is not moved. As described above, in the present embodiment, the CPU 10 moves the display range based on the virtual plane coordinates only during a period designated by the user using the input device 8.

  In step S52, the CPU 10 calculates the movement amount / movement direction of the virtual plane coordinates. This movement amount / movement direction means the speed of the virtual surface coordinates, and is calculated as a difference between the virtual surface coordinates in the current processing loop of steps S3 to S9 and the virtual surface coordinates in the previous processing loop. That is, the CPU 10 calculates a velocity vector having the previous virtual plane coordinate as the start point and the current virtual plane coordinate as the end point. Following step S52, the process of step S53 is executed.

In step S53, the CPU 10 moves the virtual camera according to the movement amount / movement direction of the virtual plane coordinates. In the present embodiment, the CPU 10 prevents the position of the display target indicated by the cursor 93 from changing before and after the movement of the virtual camera. Specifically, first, a vector (movement vector) Cv representing the amount and direction of movement of the virtual camera to be moved is calculated. When the velocity vector calculated in step S52 is (Vrx, Vry), the movement vector Cv = (Cvx, Cvy) is calculated according to the following equation (18).
Cvx = Vrx
Cvy = Vry (18)
As shown in the above equation (18), here, the X ′ axis in the plane Q of the first virtual space and the x ″ axis in the second virtual space correspond to each other in the plane Q of the first virtual space. The virtual camera is moved so that the Y ′ axis corresponds to the y ″ axis in the second virtual space. Here, the X′Y ′ coordinate system in the plane Q and the x ″ y ″ z ″ coordinate system in the second virtual space have the same size (the length 1 in the X′Y ′ coordinate system is x ″). y "z" coordinate system).

  Once the movement vector Cv is calculated, the virtual camera is then moved according to the movement vector Cv. That is, the CPU 10 reads the camera position data 80 from the main memory, and calculates a position obtained by moving the position indicated by the camera position data by the movement vector Cv. The calculated position becomes the position of the virtual camera after movement. Data indicating the calculated position is stored in the main memory as new camera position data 80. After step S53 described above, the CPU 10 ends the camera position control process.

  According to the processing of steps S51 to S53, the virtual camera is moved by the movement amount / movement direction corresponding to the movement amount / movement direction of the virtual plane coordinates. Therefore, according to the first control method, the user can move the display range in the direction in which the attitude of the input device 8 is changed by an amount corresponding to the changed angle (FIG. 25). In the first control method, the amount of change of the input device 8 corresponds to the amount of movement of the display range. Therefore, when the first control method is adopted, the user can easily control the amount of movement of the display range. It is. Therefore, the first control method is particularly effective when the amount of movement of the display range needs to be finely manipulated and the position of the display range needs to be manipulated with high accuracy.

  Further, in the present embodiment, the position of the display target pointed to by the cursor 93 does not change by the above processing, so that the display range can be moved without changing the target pointed to by the cursor 93 according to the above processing. In other embodiments, the position of the display target indicated by the cursor 93 may change before and after the movement of the virtual camera.

  Further, in the present embodiment, the CPU 10 performs the virtual camera movement process only during the predetermined period specified by the user, so the virtual camera is not moved during a period other than the predetermined period. Therefore, although details will be described later, only the position of the cursor on the screen is controlled in accordance with an operation on the input device 8 during a period other than the predetermined period. Therefore, according to the present embodiment, the operation on the input device 8 can be used for both control of the virtual camera (display range movement control) and control of the cursor. Since the user can perform both the movement of the cursor and the movement of the display range using the input device 8, according to the present embodiment, an operation method that is very easy to operate can be provided.

  In step S51, the predetermined period is a period during which the B button 32i of the input device 8 is pressed. In other embodiments, the predetermined period may be a period specified by the user. For example, the predetermined period is a period from when an operation is performed by the user until a further operation is performed thereafter (for example, a period from when the B button 32i is pressed until the next B button 32i is pressed) ), Or a period until a predetermined time elapses after a certain operation is performed by the user.

  Next, the details of the second control method will be described with reference to FIG. FIG. 27 is a flowchart showing the flow of the camera position control process (step S42) when the second control method is adopted. In this case, first, in step S55, the CPU 10 determines whether or not the current time is a predetermined period for moving the camera position. The process in step S55 is the same as that in step S51. If the determination result of step S55 is affirmative, the processes of steps S56 and S57 are executed. On the other hand, when the determination result of step S55 is negative, the CPU 10 ends the camera position control process.

  In step S56, the CPU 10 determines whether or not the predetermined period has started. The determination in step S56 can be made by referring to the determination result in step S55 in the previous processing loop. That is, if the determination result of step S55 in the previous processing loop is negative, it can be determined that the predetermined period has started in the current processing loop. If the determination result of step S56 is affirmative, the process of step S57 is executed. On the other hand, when the determination result of step S56 is negative, the process of step S57 is skipped and the process of step S58 is executed.

  In step S57, the CPU 10 stores the virtual plane coordinates calculated in step S7 in the main memory as coordinates indicating a predetermined reference position. Thus, in the present embodiment, the position of the virtual plane coordinates at the time when the predetermined period starts is the predetermined reference position. In another embodiment, the predetermined reference position may be determined in advance. For example, the position of the projection point T described above may be set as the predetermined reference position. Following step S57, the process of step S58 is executed.

  In step S58, the CPU 10 calculates a difference between the predetermined reference position and the position indicated by the virtual plane coordinates. Specifically, the CPU 10 reads data indicating the predetermined reference position stored in step S57 and the virtual plane coordinate data 77 from the main memory. Then, the difference is calculated by subtracting a coordinate value representing a predetermined reference position from the value of the virtual surface coordinates. Following step S58, the process of step S59 is executed.

In step S59, the CPU 10 moves the virtual camera according to the difference calculated in step S58. Specifically, first, a movement vector Cv similar to that in step S53 is calculated. In the second control method, the movement vector Cv = (Cvx, Cvy) is calculated according to the following equation (19).
Cvx = Δx × K2
Cvy = Δy × K2 (19)
In the above equation (19), the variable Δx is the X ′ component of the difference, and the variable Δy is the Y ′ component of the difference. The constant K2 is predetermined. If the constant K2 is increased, the moving speed of the virtual camera is increased. According to the above equation (19), if the difference becomes too large, the movement vector Cv becomes too large, and as a result, the display range may move too quickly. Therefore, in another embodiment, the CPU 10 may provide an upper limit value for the magnitude of the movement vector Cv and correct the movement vector Cv so as not to exceed the upper limit value.

  Once the movement vector Cv is calculated, the virtual camera is then moved according to the movement vector Cv. The method for calculating the position after movement of the virtual camera based on the movement vector Cv is the same as in step S53 in step S59. Data indicating the calculated position of the virtual camera after movement is stored in the main memory as new camera position data 80. After the above step S59, the CPU 10 ends the camera position control process.

  According to the processing of steps S55 to S59, the virtual camera is moved by the movement amount / movement direction according to the difference between the predetermined reference position and the virtual plane coordinates. Therefore, according to the second control method, the user moves from the reference posture (of the input device 8) corresponding to the reference position to the direction in which the posture of the input device 8 is changed at a speed corresponding to the changed amount. The display range can be moved (FIG. 25). In the second control method, the user can move the display range even when the input device 8 is stationary as long as the user is tilted from the reference posture. Therefore, the second control method is particularly effective when the display range needs to be moved greatly because the display target is very large.

  In this embodiment, also in the second control method, as in the first control method, the CPU 10 performs the movement process of the virtual camera only for the predetermined period specified by the user. Therefore, even when the second control method is employed, the user can perform both the movement of the cursor and the movement of the display range by using the input device 8, and an operation method that is easy to operate can be provided. it can.

  Next, details of the third control method will be described with reference to FIG. FIG. 28 is a flowchart showing the flow of the camera position control process (step S42) when the third control method is adopted. In this case, first, in step S61, the CPU 10 determines whether or not the virtual plane coordinates are outside the predetermined area Ar. The process in step S61 is the same as the process in step S46 described above. If the determination result of step S61 is affirmative, the process of step S62 is executed. When the determination result of step S61 is negative, the CPU 10 ends the camera position control process. That is, also in the third control method, as in the virtual camera orientation control process (steps S46 to S48), the virtual camera is controlled to move the display range only when the virtual plane coordinates are outside the predetermined area Ar. To do.

In step S62, the CPU 10 calculates a protrusion vector Vd. The process of step S62 is the same as the process of step S47. In step S63 subsequent to step S62, the CPU 10 controls the position of the virtual camera based on the protrusion vector Vd. In the third control method, the virtual camera is moved in a direction corresponding to the direction of the protrusion vector Vd at a speed (movement amount) corresponding to the magnitude of the protrusion vector Vd. Specifically, the CPU 10 calculates the movement vector Cv = (Cvx, Cvy) according to the following equation (20).
Cvx = dx × K3
Cvy = dy × K3 (20)
In the above equation (20), the variables dx and dy are the X ′ component and the Y ′ component of the protrusion vector Vd. The constant K3 is determined in advance. If the constant K3 is increased, the moving speed of the virtual camera is increased.

  After calculating the movement vector Cv, the CPU 10 moves the virtual camera according to the movement vector Cv. The method for calculating the position after movement of the virtual camera based on the movement vector Cv is the same as in step S53 and the like. Data indicating the calculated position of the virtual camera after movement is stored in the main memory as new camera position data 80. After step S63, the CPU 10 ends the camera position control process.

  As described above, according to the processes in steps S61 to S63, when the virtual plane coordinates are outside the predetermined area Ar, the virtual camera moves in a direction corresponding to the direction from the predetermined area Ar to the virtual plane coordinates. Is done. Therefore, also in the third control method, as in the virtual camera direction control process (steps S46 to S48), the user operates the input device 8 so as to remove the direction of the input device 8 from the screen of the television 2, thereby setting the display range. It can be moved (see FIG. 25).

  Further, according to the processing of steps S61 to S63, the movement amount of the virtual camera (the magnitude of the movement vector Cv) is determined according to the distance from the predetermined area Ar to the virtual plane coordinates (the magnitude of the protrusion vector Vd). Is done. Therefore, since the virtual camera moves even if the virtual plane coordinates are stopped, the user can move the display range even when the input device 8 is not moved (if the orientation of the input device 8 is off the screen). Can do. Therefore, like the second control method, the third control method is particularly effective when the display range needs to be moved greatly because the display target is very large.

  In the third control method in the present embodiment, the CPU 10 determines the speed of the display range (the amount of movement per unit time) based on the degree to which the virtual plane coordinates are away from the predetermined area Ar. Here, in another embodiment, the CPU 10 may determine the amount of movement of the display range based on the above degree. That is, the CPU 10 moves the display range by an amount corresponding to the above degree when the virtual plane coordinates go out of the predetermined area Ar, and stops the display range when the virtual plane coordinates stop. Good.

  Next, details of the fourth control method will be described with reference to FIGS. 29 to 31. FIG. 29 is a diagram showing game processing data 67 stored in the main memory of the game apparatus 3 when the fourth control method is employed. In FIG. 29, the description of the data shown in FIG. 13 is omitted, but the data shown in FIG. 13 is stored in the main memory even when the fourth control method is adopted.

  As shown in FIG. 29, when the fourth control method is adopted, the game processing data 67 includes scroll component speed data 83, maximum speed data 84, and minimum speed data 85 in addition to the data shown in FIG. Further includes.

  The scroll component speed data 83 is data indicating the scroll component speed S of the virtual plane coordinates. Here, the scroll component speed S of the virtual surface coordinates represents the magnitude of the component (referred to as a scroll component) in the direction from the predetermined area Ar to the virtual surface coordinates among the speeds of the virtual surface coordinates (see FIG. 31). . Although details will be described later, in this embodiment, the direction from the predetermined area Ar to the virtual plane coordinates corresponds to the moving direction of the display range, that is, the scroll direction of the image. Therefore, hereinafter, the direction from the predetermined area Ar to the virtual plane coordinates is referred to as “scroll direction”.

  The maximum velocity data 84 indicates the maximum value of the scroll component velocity S during the period when the virtual plane coordinates are outside the predetermined area Ar. Further, the minimum speed value data 85 indicates the minimum value of the scroll component speed S in the period. Note that these maximum and minimum values are erased (reset) when the virtual plane coordinates enter the inside of the predetermined area Ar.

  FIG. 30 is a flowchart showing the flow of the camera position control process (step S42) when the fourth control method is adopted. In this case, first, in step S71, the CPU 10 determines whether or not the virtual plane coordinates are outside the predetermined area Ar. The process of step S71 is the same as the process of step S46 described above. If the determination result of step S71 is negative, the process of step S72 is executed. On the other hand, when the determination result of step S71 is affirmative, the process of step S73 described later is executed.

  In step S72, the CPU 10 resets the maximum value Smax and the minimum value Smin of the scroll component speed. Specifically, the CPU 10 stores data indicating “0” as the maximum speed value data 84 in the main memory, and stores data indicating “0” as the minimum speed value data 85 in the main memory. In addition, when step S72 is executed also in the previous processing loop (when the values indicated by the maximum speed data 84 and the minimum speed data 85 are already “0”), step S72 in the current processing loop. This process may be omitted.

  After step S72, the CPU 10 ends the camera position control process. That is, when the virtual plane coordinates are within the predetermined area Ar, the process of controlling the position of the virtual camera (step S78) is not executed, and the display range does not move. As described above, also in the fourth control method, the virtual camera is controlled to display the display range only when the virtual plane coordinates are outside the predetermined area Ar, as in the virtual camera orientation control process (steps S46 to S48). Move.

  On the other hand, in step S73, the CPU 10 calculates the scroll component speed S. The scroll component speed S is calculated based on the position of the virtual plane coordinates, the speed of the virtual plane coordinates, and the predetermined area Ar. Hereinafter, a method for calculating the scroll component speed S will be described with reference to FIG.

  FIG. 31 is a diagram showing the scroll component speed S in the plane Q. As shown in FIG. In FIG. 31, vector Vn represents the scroll direction, and vector Vr represents the speed of the virtual plane coordinates. In step S73, first, the scroll direction vector Vn is calculated. The scroll direction vector Vn is a unit vector that points in the direction from the point P10 (the projection point T and the origin of the X′Y ′ coordinate system) to the point R represented by the virtual plane coordinates. The scroll direction vector Vn only needs to represent the direction of the virtual plane coordinates with respect to the predetermined area Ar. For example, in another embodiment, the scroll direction vector Vn may be calculated so as to represent the direction from the midpoint of the side closest to the virtual plane coordinate among the four sides of the predetermined area Ar to the virtual plane coordinate. .

Once the scroll direction vector Vn is calculated, the scroll component speed S is then calculated. As shown in FIG. 31, the scroll component speed S is the magnitude of the component in the scroll direction (the direction of the vector Vn) out of the virtual surface coordinate speed (the vector Vr). That is, assuming that the scroll direction vector Vn = (nx, ny) and the virtual surface coordinate speed Vr = (Vrx, Vry), the scroll component speed S is calculated according to the following equation (21).
S = nx × Vrx + ny × Vry (21)
Data indicating the scroll component speed S calculated according to the above equation (21) is stored in the main memory as scroll component speed data 83. As is clear from FIG. 31 and the above equation (21), the scroll component speed S becomes a positive value when the virtual plane coordinate moves in a direction away from the predetermined area Ar, and the virtual plane coordinate is the predetermined area. A negative value is obtained when moving in a direction approaching Ar. Following step S73, the process of step S74 is executed.

  In step S74, the CPU 10 determines whether or not the scroll component speed S calculated in step S73 has exceeded the maximum value Smax. That is, the CPU 10 reads the scroll component speed data 83 and the maximum speed data 84 from the main memory, and compares the scroll component speed S and the maximum value Smax. If the determination result of step S73 is affirmative, the process of step S75 is executed. On the other hand, when the determination result of step S73 is negative, the process of step S75 is skipped and the process of step S76 is executed.

  In step S75, the CPU 10 updates the maximum value Smax of the scroll component speed S. That is, the maximum value Smax is updated to the value of the scroll component speed S calculated in the current processing loop. Specifically, the CPU 10 stores data indicating the same value as the scroll component speed data 83 as new speed maximum value data 84 in the main memory. Following step S75, the process of step S76 is executed.

  In step S76, the CPU 10 determines whether or not the scroll component speed S calculated in step S73 is less than the minimum value Smin. That is, the CPU 10 reads the scroll component speed data 83 and the speed minimum value data 85 from the main memory, and compares the scroll component speed S and the minimum value Smin. If the determination result of step S76 is affirmative, the process of step S77 is executed. On the other hand, when the determination result of step S76 is negative, the process of step S77 is skipped and the process of step S78 is executed.

  In step S77, the CPU 10 updates the minimum value Smin of the scroll component speed S. That is, the minimum value Smin is updated to the value of the scroll component speed S calculated in the current processing loop. Specifically, the CPU 10 stores data indicating the same value as the scroll component speed data 83 in the main memory as new speed minimum value data 85. Following step S77, the process of step S78 is executed.

In step S78, the CPU 10 moves the virtual camera based on the maximum value Smax and the minimum value Smin. In the fourth control method, the virtual camera is moved in a direction corresponding to the scroll direction at a speed (movement amount) corresponding to the sum of the maximum value Smax and the minimum value Smin. That is, in the fourth control method, the movement vector Cv = (Cvx, Cvy) is calculated according to the following equation (22).
Cvx = nx × (Smax + Smin)
Cvy = ny × (Smax + Smin) (22)
As is apparent from the above equation (22), the movement vector Cv has a size corresponding to the sum of the maximum value Smax and the minimum value Smin, with a direction corresponding to the scroll direction vector Vn = (nx, ny). In other embodiments, the direction of the movement vector Cv may coincide with the movement direction of the virtual plane coordinates. However, when the direction of the movement vector Cv is the movement direction of the virtual plane coordinates, the direction of the input device 8 with respect to the screen does not match the movement direction of the display range, and the user may feel uncomfortable. On the other hand, according to the present embodiment, the display range is moved in a direction corresponding to the direction of the virtual plane coordinates with respect to the predetermined area Ar, thereby matching the direction of the input device 8 with respect to the screen and the moving direction of the display range. Therefore, operability can be improved.

  After calculating the movement vector Cv, the CPU 10 moves the virtual camera according to the movement vector Cv. The method for calculating the position of the virtual camera after the movement based on the movement vector Cv is the same as in steps S53 and S59. Data indicating the calculated position of the virtual camera after movement is stored in the main memory as new camera position data 80. After step S78, the CPU 10 ends the camera position control process.

  As described above, according to the processes in steps S71 to S78, when the virtual plane coordinates are outside the predetermined area Ar, the virtual camera moves in a direction corresponding to the direction from the predetermined area Ar to the virtual plane coordinates. Is done. Therefore, in the fourth control method, similarly to the virtual camera orientation control process (steps S46 to S48), the user operates the input device 8 so as to remove the orientation of the input device 8 from the screen of the television 2, thereby displaying the display range. Can be moved (see FIG. 25).

  According to the above equation (22) in step S78, when the sum is a negative value, the movement vector Cv is opposite to the scroll direction vector Vn, so the sum is a positive value. The moving direction of the display range is opposite to the case. That is, when the sum is a negative value, the direction of the input device 8 (the direction when the direction toward the television 2 is the front direction) and the moving direction of the display range are reversed. For example, while the user points the input device 8 toward the right side of the screen, the display range moves to the left. Therefore, when the sum is a negative value, the user may feel uncomfortable with the operation. Therefore, in step S78, the CPU 10 may set the movement vector Cv to “0” when the sum is a negative value. That is, when the sum is a negative value, the display range may be stopped. According to this, since the direction of the input device 8 and the moving direction of the display range are not reversed, it is possible to prevent the user from feeling uncomfortable with the operation.

  Further, according to the processing in steps S71 to S78, the movement amount of the virtual camera (the magnitude of the movement vector Cv) is not the virtual plane coordinate speed itself, but the maximum value Smax and the minimum value Smin of the scroll component speed S. And decided according to the sum. Therefore, since the virtual camera moves even if the virtual plane coordinates are stopped, the user can move the display range even when the input device 8 is not moved (if the orientation of the input device 8 is off the screen). Can do. Therefore, like the second control method, the fourth control method is particularly effective when the display range needs to be moved greatly because the display target is very large.

  In the processes of steps S71 to S78, it is determined on the two-dimensional plane of the X′Y ′ coordinate system whether or not the virtual plane coordinates are outside the predetermined area Ar. Here, in another embodiment, the determination may be performed separately for the X ′ axis and the Y ′ axis. For example, when the virtual plane coordinates are outside the predetermined area Ar with respect to the X′-axis direction and within the predetermined area Ar with respect to the X′-axis direction, the CPU 10 determines the direction (x The virtual camera may be moved only in the “axis direction” and may not be moved in the direction corresponding to the Y′-axis direction (y ”-axis direction).

  Here, in the third control method, the moving speed of the display range is a speed corresponding to the distance from the predetermined area Ar to the virtual plane coordinates. For this reason, at the time when the movement of the display range is started (that is, the time immediately after the virtual plane coordinate goes out of the predetermined area Ar), the moving speed is always a value close to zero. Accordingly, when the user moves the input device 8 quickly, a gap is generated between the speed of movement of the input device 8 and the moving speed of the display range. There is a risk of embrace.

  On the other hand, according to the fourth control method, when the movement of the display range is started, the movement speed of the display range is approximately the speed of the virtual plane coordinates at that time. That is, when moving the display range, if the user moves the input device 8 quickly, the display range also moves fast. If the user moves the input device 8 slowly, the display range also moves slowly. Therefore, there is no gap between the speed of movement of the input device 8 and the moving speed of the display range, so that the user does not feel uncomfortable when starting to move the display range. Thus, according to the fourth control method, an operation method with better operability can be provided. In particular, as in the present embodiment, when the cursor is displayed at a position on the screen according to the virtual plane coordinates (step S43 to be described later), the movement speed of the cursor before the start of movement of the display range, and after the movement starts. The moving speed of the display range at becomes equal. Therefore, the switching from the state in which the cursor moves on the screen to the state in which the display range moves can be smoothly felt without discomfort for the user, and the user can have a good operational feeling.

  In the third control method, when the user wants to move the display range at a low speed, the input device 8 must be stationary with the posture in which the virtual plane coordinates are slightly outside the circumference of the predetermined area Ar. It is difficult to operate. On the other hand, according to the fourth control method, when the user wants to move the display range at a low speed, the user only has to change the posture of the input device 8 slowly, and the display range can be easily reduced in speed. It can be moved with.

  In the fourth control method in the present embodiment, the CPU 10 determines the speed of the display range based on the scroll component speed S. Here, in another embodiment, the CPU 10 may determine the amount of movement of the display range based on the scroll component speed S. That is, when the virtual plane coordinates go out of the predetermined area Ar, the CPU 10 moves the display range by an amount corresponding to the sum of the maximum value Smax and the minimum value Smin of the scroll component speed S, and the virtual plane coordinates are stopped. In such a case, the display range may be stopped.

  Further, in the fourth control method in the present embodiment, the CPU 10 determines the degree of changing the display range (the moving speed of the display range) based on the scroll component speed S. Here, in another embodiment, the CPU 10 determines the above degree by any method as long as it is determined based on the speed of the virtual surface coordinates when the virtual surface coordinates are located outside the predetermined area Ar. May be determined. For example, the degree may be determined according to the maximum value Smax of the scroll component speed S. The degree may be determined according to the scroll component speed S or according to the speed of the virtual plane coordinates (at this time, when the virtual plane coordinates are stopped, the display range is stopped).

  In the fourth control method according to the present embodiment, the CPU 10 determines the speed of the display range according to the maximum value and the minimum value of the scroll component speed S during the period when the virtual plane coordinates are outside the predetermined area Ar. did. Here, the period for determining the maximum value and the minimum value of the scroll component speed S does not have to be the entire period in which the virtual plane coordinates are outside the predetermined area Ar, but only a part of the period. There may be. For example, the CPU 10 may determine the maximum value and the minimum value from the scroll component speed S in a certain period (predetermined) after the virtual surface coordinates are outside the predetermined area Ar. This also prevents the user from feeling uncomfortable when starting the movement of the display range because there is no gap between the speed of movement of the input device 8 and the moving speed of the display range, as in the above embodiment. Can do.

  As described above, four control methods have been described as examples of the camera position control process in step S42. However, the method for controlling the position of the virtual camera is not limited to these control methods. The camera position control process may be any method that controls the position of the virtual camera using the virtual plane coordinates.

  In the above embodiment, both the orientation and position of the virtual camera are controlled (steps S41 and S42). In other embodiments, only one of the orientation and position of the virtual camera is controlled. You may make it do. That is, the CPU 10 may execute only one of steps S41 and S42.

  In the above-described embodiment, in the camera orientation control process (step S41), the CPU 10 controls the orientation of the virtual camera based on the degree to which the virtual plane coordinates are away from the predetermined area Ar. The orientation of the virtual camera may be controlled based on other information related to the virtual plane coordinates. The orientation of the virtual camera may be controlled according to the amount and direction of movement of the virtual plane coordinates, for example, as in the first control method, or a predetermined reference, as in the second control method. It may be controlled according to the difference from the position to the position indicated by the virtual surface coordinates, or the virtual surface coordinates in the case where the virtual surface coordinates are located outside the predetermined area Ar as in the fourth control method. It may be controlled based on the speed. That is, in another embodiment, the display range is changed by controlling the orientation of the virtual camera instead of controlling the position of the virtual camera in the first, second, and fourth control methods. You may do it.

  In the above embodiment, the CPU 10 controls the display range using only the attitude (virtual plane coordinates) of the input device 8, but in other embodiments, the CPU 10 includes information on the attitude of the input device 8. In addition, the display range may be controlled using information obtained by other input methods. For example, the CPU 10 may control the display range by further using operation data related to the cross button 32 a of the input device 8. Specifically, in the display control process shown in FIG. 20, instead of the camera position control process in step S42, a process for controlling the position of the virtual camera according to a direction instruction using the cross button 32a may be executed. More specifically, in the above processing, the CPU 10 may control the position of the virtual camera so that the display range moves in the direction according to the pressed part of the upper, lower, left, and right parts of the cross button 32a. Good. According to this, the user can change the position of the virtual camera by using the cross button 32 a and can change the direction of the virtual camera by changing the direction of the input device 8. That is, since the display range can be changed by two kinds of methods, the display range can be operated more freely and the operability can be improved.

  The input method used for controlling the display range together with the input method using the attitude of the input device 8 is not limited to the input method using the cross button 32a, and may be another input method. However, an input method using an operating means provided in the input device 8 is preferable. This is because the user can perform both operations for changing the posture of the input device 8 and operations for the operation means with one hand. For example, the input device 8 may include an operation unit such as a stick or a touch pad instead of the cross button 32a, and the CPU 10 may control the display range in accordance with a direction instruction to these operation units. In another embodiment, the CPU 10 may control the position of the virtual camera based on the virtual plane coordinates, and may control the orientation of the virtual camera according to a direction instruction using the operation unit.

  Returning to the description of FIG. 20, in step S43, the CPU 10 calculates a position (cursor position) at which the cursor is displayed on the screen of the television 2 based on the virtual plane coordinates. In the present embodiment, the cursor position is calculated so that the predetermined area Ar on the plane Q in the first virtual space corresponds to the area of the screen of the television 2. That is, the CPU 10 calculates the cursor position so that the positional relationship of the virtual plane coordinates with respect to the predetermined region Ar is the same as the positional relationship of the cursor position with respect to the screen region. When the virtual surface coordinates are outside the predetermined area Ar, the CPU 10 specifies the position closest to the virtual surface coordinates in the circumference of the predetermined area Ar, and determines the position of the edge of the screen corresponding to the position. You may calculate as a cursor position. Data indicating the calculated cursor position is stored in the main memory as cursor position data 81. Following step S43, the process of step S44 is executed.

  In step S44, the CPU 10 generates an image to be displayed to be displayed on the television 2 based on the position and orientation of the virtual camera. Specifically, the CPU 10 generates a display target image by performing conversion processing such as perspective projection conversion on the display target with the position of the virtual camera as the viewpoint position and the direction of the virtual camera as the viewing direction. As a result, an image representing a part of the display target is generated, and the display range is determined using the virtual camera. The image generated in step S44 is displayed on the screen of the television 2.

  Further, the CPU 10 displays the cursor image at the cursor position so as to overlap the display target image. Since the cursor is displayed at a position corresponding to the posture of the input device 8, the player can perform an operation of moving the cursor by changing the posture of the input device 8. If the virtual plane coordinates are outside the predetermined area Ar, the CPU 10 may or may not display the cursor. For example, when the second to fourth control methods are employed in the process of step S42, the CPU 10 moves the cursor to the position of the edge of the screen so that the user can easily recognize the moving direction of the display range. You may make it display. On the other hand, when the first control method is adopted in the process of step S42, when the cursor is displayed at the edge of the screen when the virtual plane coordinates are outside the predetermined area Ar, the position of the display target indicated by the cursor Will change. Therefore, in this case, the CPU 10 may not display the cursor (ie, the cursor appears to have gone out of the screen). When the second control method is adopted in the process of step S42, the CPU 10 represents a difference between the reference position and the virtual plane coordinates so that the user in the moving direction of the display range can easily recognize. (Arrow 94 shown in FIG. 25) may be displayed. Following step S44, the process of step S45 is executed.

  In step S45, the CPU 10 executes a game process based on the cursor position. For example, when a predetermined operation (for example, an operation of pressing a predetermined button of the input device 8) is performed in a state where a button image representing a predetermined instruction is pointed by the cursor, the CPU 10 The game process according to the above is executed. The game process in step S45 may be any process as long as the cursor position is used as a user input. For example, the game process may be a process of drawing a line along a locus along which the cursor position has moved. Through step S45, the user can use the input device 8 as a pointing device for designating a position on the screen.

  In other embodiments, the game process in step S45 may be any process as long as the two-dimensional coordinates are reflected in the game result as an input value. For example, it may be a process of moving an object in the virtual game space to a position in the game space corresponding to the two-dimensional coordinates, or depending on the size and direction of the two-dimensional vector represented by the two-dimensional coordinates. It may be a process of controlling and displaying the object so as to move at a high speed. After step S45, the CPU 10 ends the display control process shown in FIG.

  In step S9 following the display control process (step S8), the CPU 10 determines whether or not to end the game. The determination in step S9 is made based on, for example, whether or not the game has been cleared, whether or not the game is over, and whether or not the player has given an instruction to stop the game. If the determination result of step S9 is negative, the process of step S3 is executed again. Thereafter, the processing loop of steps S3 to S9 is repeatedly executed until it is determined in step S9 that the game is to be ended. On the other hand, if the determination result of step S9 is affirmative, the CPU 10 ends the game process shown in FIG. This is the end of the description of the game process.

  As described above, in the present embodiment, the first posture of the input device 8 is calculated from the angular velocities detected by the gyro sensors 55 and 56 (step S4), and the first posture is subjected to the first correction process (S5). And it corrected by the 2nd correction process (S6). Since the game process is executed using the corrected first attitude (steps S7 and S8), the CPU 10 can execute the game process based on the accurate attitude of the input device 8. According to this, for example, since the posture of the input device 8 can be accurately reflected on the position of the cursor on the screen, the operability of the game operation can be improved.

[Second method for calculating coordinates]
Hereinafter, a second method for calculating the position on the screen from the attitude of the input device 8 will be described with reference to FIGS. In the above embodiment, the game process may be executed using the following second method instead of the first method described above. In the following description, the coordinates on the plane Q in the virtual space are referred to as “first coordinates”, the coordinates representing the position on the screen calculated from the first coordinates are referred to as “screen coordinates”, and the marker The coordinates (details will be described later) representing the position on the screen calculated from the coordinates are referred to as “second coordinates”.

  FIG. 32 is a diagram showing a comparison between the first method and the second method. 32, in real space, the input device 8 is positioned below the center of the screen of the television 2, and the position on the screen indicated by the Z-axis direction of the input device 8 (hereinafter referred to as “(input device) indicated position”). ”) Becomes the center of the screen as an example (see the upper left column in FIG. 32).

  In the first method described above, the coordinate system of the first coordinate (the X′Y ′ coordinate system on the plane Q in the virtual space) always has the projection point T obtained by projecting the position of the input device 8 on the plane Q as the origin. O. That is, the first coordinates are calculated so as to take the value of the origin (0, 0) when the Z-axis direction of the input device 8 is in the horizontal direction (when the Z-axis direction indicates the projection point T). Therefore, as shown in FIG. 32, when the Z-axis direction of the input device 8 is slightly upward, the position R of the first coordinate is above the origin O (see the upper middle column in FIG. 32). As a result, the position U of the screen coordinates calculated from the first coordinates is above the center of the screen (see the upper right column in FIG. 32).

  In the case shown in FIG. 32, in the first method, the Z-axis direction of the input device 8 actually points to the center of the screen (in real space), whereas the calculated position on the screen is from the center. Will also be in an upper position. As described above, in the first method, a deviation occurs between the position indicated by the user using the input device 8 (indicated position of the input device) and the position of the calculated screen coordinates, and the user feels uncomfortable. There was a risk of holding.

  Actually, the situation in which the television 2 is installed (whether it is placed on the floor or on the stand, etc.) and the state of the user who uses the input device 8 (whether the input device 8 is used standing up) It is considered that the height of the input device 8 with respect to the screen varies depending on whether the user is sitting and using. If this height changes, the relationship between the indicated position and orientation of the input device 8 changes, and the orientation of the input device 8 when the indicated position of the input device 8 is at the center of the screen also changes. However, in the first method, since the position of the origin of the X′Y ′ coordinate system is fixed without taking the indicated position of the input device 8 into consideration, the position on the screen cannot be calculated appropriately, and the input The user may feel uncomfortable with the operation using the device 8.

  Therefore, in the second method, the game apparatus 3 uses the first coordinate calculated by the first method when the Z-axis direction of the input device 8 is in the posture pointing to the center of the screen (the indicated position is the screen position). Correction is performed so that a predetermined value (specifically, (0, 0)) is obtained at the center. Specifically, the game apparatus 3 first determines whether or not the Z-axis direction of the input apparatus 8 is in an attitude that points to the center of the screen. This determination can be made by referring to the marker coordinates described above. That is, the game apparatus 3 calculates the coordinates (second coordinates) indicating the indicated position on the screen based on the marker coordinates, and determines whether the second coordinates represent the center (near the center) of the screen. To do.

  As a result of the above determination, when the second coordinate is the center of the screen, the game apparatus 3 determines a correction amount (offset amount) for correcting the first coordinate. As for the offset amount, the position of the screen coordinate corresponding to the first coordinate calculated by the first method when the second coordinate becomes the center of the screen matches the position of the second coordinate. To be determined. Specifically, the offset amount is determined to be a correction amount such that the first coordinate at the time point becomes the origin value (0, 0). Note that determining the offset amount means moving the X′Y ′ coordinate system on the plane Q. In the example of FIG. 32, since the input device 8 faces upward from the horizontal direction, the X′Y ′ coordinate system has a position on the plane Q where the projection point T is the origin (the dotted line shown in FIG. 32). The position is moved so as to move upward (see the lower middle column in FIG. 32). As described above, as a result of setting the first coordinate as the origin value, the position U on the screen corresponding to the first coordinate becomes the center position of the screen (see the upper right column in FIG. 32). Therefore, the indicated position (second coordinate position) of the input device 8 and the calculated position on the screen coincide with each other at the center of the screen. Thus, according to the second method, it is possible to prevent a deviation between the position pointed by the user using the input device 8 and the calculated position, and the user feels uncomfortable. Can be prevented.

  In the present embodiment, the game apparatus 3 corrects the first coordinate only for the Y ′ component of the X′Y ′ coordinate system. That is, the first coordinates (and screen coordinates) are corrected only in the vertical direction of the screen, and are not corrected in the horizontal direction. In the present embodiment, the height (position in the vertical direction) of the input device 8 with respect to the screen cannot be determined only from the attitude of the input device 8, whereas the position of the input device 8 in the horizontal direction with respect to the screen. This is because it can be determined only from the attitude of the input device 8 and the “displacement” as described in FIG. 32 does not occur in the left-right direction. However, in other embodiments, the first coordinates may be corrected in the left-right direction as in the up-down direction.

  Next, with reference to FIG. 33 to FIG. 37, details of processing executed by the game apparatus 3 when the second method is adopted will be described. FIG. 33 is a diagram showing main data stored in the main memory of the game apparatus 3 when the second method is adopted. In FIG. 33, the same data as the data shown in FIG. 13 is denoted by the same reference numerals as those in FIG.

  As shown in FIG. 33, in the second method, in addition to the data shown in FIG. 13, first coordinate data 87, second coordinate data 88, offset amount data 89, and screen coordinate data 90 are stored in the main memory. Is done.

  The first coordinate data 87 indicates the first coordinates calculated based on the attitude of the input device 8. That is, the first coordinate data 87 indicates the coordinates of the position on the plane Q in the virtual space.

  The second coordinate data 88 indicates coordinates (second coordinates) representing a position on the screen calculated based on the marker coordinates. The second coordinate represents the position on the screen indicated by the Z-axis direction of the input device 8. The second coordinates ideally represent the position of the intersection of the straight line extending the Z axis of the input device 8 and the screen, but may be any as long as it represents the vicinity of the position of the intersection.

  The offset amount data 79 indicates an offset amount for correcting the first coordinate. In the present embodiment, the coordinate obtained by moving the first coordinate calculated by the first method by the offset amount becomes the corrected first coordinate.

  The screen coordinate data 90 indicates coordinates (screen coordinates) representing a position on the screen calculated based on the first coordinates. In the second method, the screen coordinates are calculated based on the first coordinates after being corrected according to the offset amount.

  FIG. 34 is a main flowchart showing a flow of processing executed by the game apparatus 3 when the second method is adopted. In FIG. 34, the same processing steps as the processing steps shown in FIG. 14 are denoted by the same step numbers as in FIG.

  When the second method is adopted, the process of step S41 is executed after the initial posture setting process of step S2. In step S81, the CPU 10 executes an offset determination process. The offset determination process is a process for determining an offset amount for correcting the first coordinates. Hereinafter, the details of the offset determination process will be described with reference to FIG.

  FIG. 35 is a flowchart showing the flow of the offset determination process (step S81) shown in FIG. In the offset determination process, first, in step S91, the CPU 10 acquires operation data. The process of step S91 is the same as the process of step S3 described above. Following step S91, the process of step S92 is executed.

  In step S <b> 92, the CPU 10 calculates second coordinates representing the designated position of the input device 8. The second coordinates are calculated based on the marker coordinates acquired in step S91. That is, the CPU 10 reads the marker coordinate data 65 from the main memory, and calculates the second coordinates based on the marker coordinates. Any method may be used to calculate the second coordinates from the marker coordinates. For example, the coordinates (px, py) calculated in step S33 described above can be calculated from the marker coordinates, and the second coordinates can be calculated based on the coordinates (px, py). Specifically, the positive / negative of the x component of the coordinates (px, py) is reversed so that a predetermined ratio (for example, the length of the captured image in the x-axis direction matches the horizontal length of the screen of the television 2). By performing the scaling by the ratio), it is possible to obtain the indicated position in the horizontal direction of the screen. In addition, the sign of the y component of the coordinates (px, py) is reversed to a predetermined ratio (for example, the ratio of the length of the captured image in the y-axis direction to the length of the screen 2 of the television 2). By performing the scaling, it is possible to obtain a designated position in the vertical direction of the screen. Regarding the vertical direction, it is preferable to appropriately set the offset amount depending on whether the marker unit 6 is installed above or below the television 2. The CPU 10 stores data indicating the calculated second coordinates in the main memory as second coordinate data 88. Following step S92, the process of step S93 is executed.

  In step S <b> 93, the CPU 10 calculates the first posture of the input device 8. The process of step S93 is the same as the process of steps S4 to S6 described above. In the present embodiment, the process for calculating the first posture is executed for each frame (for each process loop of steps S91 to S98). However, this process is performed when the determination result of step S94 is positive. It may be executed only in Following step S93, the process of step S94 is executed.

  In step S <b> 94, the CPU 10 displays a predetermined setting screen and a cursor on the screen of the television 2. The setting screen is a screen including a button image for the user to give a predetermined instruction. FIG. 36 is a diagram illustrating an example of the setting screen. In FIG. 36, the predetermined instruction is an instruction for adjusting the cursor speed (sensitivity adjustment), and a button image 101 indicating an instruction for increasing the cursor speed and a button image 102 indicating an instruction for decreasing the cursor speed are displayed. Is done. Note that the cursor to be adjusted here is not the currently displayed cursor 103 but a cursor whose position is determined based on the first coordinates (screen coordinates) used in a later game operation. The contents of the predetermined instruction may be anything, and other adjustments relating to the cursor (for example, adjustment of the followability of the cursor to the operation of the input device 8, adjustment of play set for the cursor, For example, adjustment of a camera shake prevention function), an instruction to end a setting operation on the setting screen, or an instruction to start a game.

  A cursor 103 is displayed on the screen of the television 2 together with the setting screen. That is, the CPU 10 reads the second coordinate data 88 from the main memory, and displays the cursor 103 at the position indicated by the second coordinate data 78. Thus, the position of the cursor 103 displayed in step S94 is calculated based on the marker coordinates, and the cursor 103 is controlled by a method based on the marker coordinates. Although not shown in FIG. 35, the CPU 10 is designated when the button image 101 or 102 is designated (a predetermined button of the input device 8 is pressed while the cursor 103 points to the button image). Adjustment processing is performed according to the button image. Following step S94, the process of step S95 is executed.

  Returning to the description of FIG. 35, in step S95, the CPU 10 determines whether or not the cursor 103 is positioned at the center in the vertical direction of the screen. This determination can be made using the second coordinates. Specifically, the CPU 10 reads the second coordinate data 88 from the main memory, and determines whether the position indicated by the second coordinate data 88 is within the display area of the button image 101 or 102. That is, the CPU 10 determines whether or not the cursor 103 is positioned within the display area of the button image 101 or 102. If the determination result of step S95 is affirmative, the process of step S96 is executed. On the other hand, if the determination result of step S95 is negative, the processes of steps S96 and S97 are skipped, and the process of step S98 described later is executed.

  In step S95, it is not determined whether or not the second coordinate (cursor 103) is strictly located at the center of the screen, but the second coordinate (in the display area of the button image) is located near the center. It is determined whether or not the coordinates are located. This is because the object of the present invention is to prevent the user from feeling uncomfortable, and it is not necessary to exactly match the indicated position of the input device and the position of the screen coordinates. Further, if it is strictly determined whether or not the second coordinate is positioned at the center of the screen in step S95, the determination result in step S95 is less likely to be affirmative, and the offset amount may not be determined. Because.

  In step S96, the CPU 10 calculates the first coordinates based on the first posture. The calculation process in step S96 is the same as the calculation process in step S7 described above. That is, the CPU 10 calculates the first coordinates according to the first method based on the first attitude calculated in step S93. The calculated first coordinates are stored in the main memory 32 as first coordinate data 87. Following step S96, the process of step S97 is performed.

  In step S97, the CPU 10 determines an offset amount based on the first coordinates calculated in step S96. FIG. 37 is a diagram illustrating a method for determining an offset amount. The posture of the input device 8 shown in FIG. 37 is the posture at the time when the indicated position of the input device 8 becomes the center in the vertical direction of the screen (when the determination result in Step S95 is Yes). The origin O is the origin of the X′Y ′ coordinate system in the first method, and is the position of the projection point T described above. Point R indicates the position of the first coordinate calculated in step S96.

  The offset amount is determined so that the Y ′ component of the first coordinate at the time point becomes the origin value (0). That is, with respect to the vertical direction, the position R of the first coordinate is determined to be the position of the origin. Therefore, the offset amount is determined to be a value “−d” obtained by inverting the sign of the value d of the Y ′ component of the first coordinate (see FIG. 37). The CPU 10 reads the first coordinate data 87 from the main memory, and determines a value obtained by inverting the sign of the Y ′ component of the first coordinate indicated by the first coordinate data 87 as the offset amount. Data indicating the determined offset amount is stored in the main memory as offset amount data 89. In the first coordinate correction process (step S82), which will be described later, the first coordinate calculated by the first method is corrected to the coordinate moved by the offset amount. Therefore, it can be said that the process of calculating the offset amount in step S97 is a process of moving the X′Y ′ coordinate system by the offset amount (in the direction in which the sign of the offset amount is reversed). Following step S97, step S98 is executed.

  In step S98, the CPU 10 determines whether or not to end the setting screen. The determination in step S98 is made based on, for example, whether or not the user has given an instruction to end the setting screen. If the determination result of step S98 is negative, the process of step S91 is executed again. Thereafter, the processes of steps S91 to S98 are repeatedly executed until the determination result of step S98 is positive. On the other hand, when the determination result of step S98 is affirmative, the CPU 10 ends the offset determination process.

  As described above, in the offset determination process, the Y ′ component of the first coordinate calculated when the input device 8 points to the center of the screen (with respect to the vertical direction) is set to the origin value (0). An offset amount is determined. Here, in order to determine the offset amount in the offset determination process, a state in which the input device 8 points to the center (with respect to the vertical direction) of the screen must occur. In the present embodiment, in order to generate such a state, button images 101 and 102 for giving a predetermined instruction are displayed in the center (with respect to the vertical direction) (FIG. 36). According to this, since the user naturally operates the input device 8 so as to be in the above state in order to give a predetermined instruction, it is possible to cause the user to perform the offset amount determination operation naturally without making the user aware of it. it can. The predetermined instruction may be any instruction. However, in order to ensure that the process for determining the offset amount is performed reliably, the user must always perform an instruction (for example, a setting screen). An instruction to end or an instruction to start a game is preferable. Further, as in the present embodiment, by giving an instruction regarding the adjustment of the cursor as the predetermined instruction, the offset amount can be determined simultaneously with the adjustment of the cursor. According to this, it is possible to determine the offset amount without giving the user a feeling that “an extra operation is being performed (to determine the offset amount)”.

  Returning to the description of FIG. 34, the process of step S3 similar to the above-described embodiment is executed after the offset determination process of step S81. Further, after step S3, the same processes of steps S4 to S7 as in the above-described embodiment are executed. Note that the process of step S7 is a process of calculating the first coordinates by the first method. In the second method, the process of step S82 is executed after the process of step S7.

  In step S82, the CPU 10 corrects the first coordinates calculated in step S7 according to the offset amount. That is, the CPU 10 reads the first coordinate data 87 and the offset amount data 78 from the main memory, and corrects the first coordinate to a value obtained by adding the offset amount D to the Y ′ component. Data indicating the corrected first coordinates is stored as new first coordinate data 87 in the main memory.

In the present embodiment, the first coordinate calculation process (step S7) by the first method and the first coordinate correction process (step S82) are described as separate processes. However, the CPU 10 The two processes may be executed together. That is, in step S7, the CPU 10 may calculate the first coordinates according to the following equation (23) instead of the above equation (15).
Wy = L × Zy / Zz + D (23)
In the above equation (23), the variable D indicates an offset amount, and is a value (−d) obtained by inverting the sign of the Y ′ component (d) of the first coordinate calculated in step S96. When the above equation (23) is used, the first coordinates can be calculated in a form including correction processing.

  Following step S82, the processes of steps S83 and S84 are executed. The process of steps S83 and S84 is the same as the process of step S8 described above, but in FIG. 34, the process of step S8 is divided into two and described as steps S83 and S84.

  In step S <b> 83, the CPU 10 calculates coordinates (screen coordinates) representing the position on the screen of the television 2. The screen coordinates are calculated based on the first coordinates corrected in step S82. That is, the CPU 10 reads the first coordinate data 87 from the main memory, and calculates the screen coordinates based on the first coordinates indicated by the first coordinate data 87. The screen coordinates are calculated to be the center of the screen when the first coordinates are the origin value (0, 0). The screen coordinate is a direction in which the direction from the center of the screen to the screen coordinate corresponds to the direction from the origin to the first coordinate, and the length from the center of the screen to the screen coordinate is the first coordinate from the origin. It is calculated to be a length corresponding to the length up to. Therefore, in a state where the Z axis of the input device 8 is directed to the center of the screen, the screen coordinates represent the center of the screen (the screen coordinates coincide with the designated position), and the direction of the Z axis of the input device 8 is determined from this state. The screen coordinates move in the direction corresponding to the changed direction. Data indicating the screen coordinates calculated as described above is stored in the main memory as screen coordinate data 90. Following step S83, the process of step S84 is executed.

  In step S84, the CPU 10 performs a game process using the screen coordinates calculated in step S83. The game process in step S84 may be any process as long as the process is based on the screen coordinates. For example, a process of displaying a cursor or a game object at the position of the screen coordinates may be performed, or a process of moving the object according to the positional relationship between the position of the object and the screen coordinates may be performed. Further, the game process in step S84 may be the display control process in step S8. At this time, in the display control process, the display range may be controlled based on the first coordinates (virtual plane coordinates) corrected in step S82. That is, in the processes of steps S41 and S42, the orientation and position of the virtual camera may be controlled based on the first coordinates corrected in step S82. In step S43, the position of the screen coordinates calculated in step S83 is set as the cursor position. When any image is displayed on the screen coordinates, the user can visually recognize the position of the screen coordinates. Therefore, the first method has a high possibility of having the above-mentioned uncomfortable feeling. Therefore, the second method is particularly effective in such a case. Following step S84, the process of step S9 similar to that in the above embodiment is executed. This is the end of the description of the second method.

  As described above, in the second method, when the game apparatus 3 displays the setting screen, the game apparatus 3 detects a state in which the input device 8 points to the center with respect to the vertical direction of the screen (step 54). The offset amount is determined using the first coordinates at (step S97). In the subsequent game processing, the first coordinates calculated by the first method are corrected according to the offset amount (step S82). Therefore, in a state where the input device 8 points to the center in the vertical direction of the screen, the Y ′ component of the first coordinate is 0, and as a result, the screen coordinate represents the center position of the screen in the vertical direction. Become. Therefore, according to the second method, when the user points to the center of the screen with the input device 8, the screen coordinates represent the center of the screen. It can be calculated appropriately.

  In the above embodiment, when the designated position is the center of the screen, the position of the screen coordinates matches the designated position. On the other hand, when the designated position is another position, the position of the screen coordinates does not necessarily match the designated position depending on the length L or the like. This is because if the position of the screen coordinates and the designated position coincide with each other at the center of the screen, it is considered that the user hardly feels a sense of incongruity, and the position of the screen coordinates and the designated position are different at the other positions. This is because there are almost no disadvantages. Further, it is considered that the merit of determining the screen coordinates in accordance with the attitude of the input device 8 (the merit that the degree of change of the screen coordinates with respect to the attitude change can be appropriately set) is larger than the above demerit.

  When the posture at the time when the input device 8 points to the center of the screen is set as the reference posture, the processing for determining the origin at the time, in other words, the posture of the input device 8 when the first coordinate takes the origin. It can be said that the process of step S97 which is the process of determining is a "process of determining the reference posture". The direction from the origin to the corrected first coordinate corresponds to the direction of change (rotation direction) from the reference posture to the current posture, and the distance from the origin to the corrected first coordinate is This corresponds to the amount of change (rotation amount) from the reference posture to the current posture. Therefore, the process of calculating the corrected first coordinate (step S7 and step S742) is “the first coordinate so as to be a value corresponding to the direction and amount of change from the reference posture to the current posture. It can be said that this is a process of calculating “. As described above, in the second method, (a) the process of determining the reference posture at the time when the input device 8 points to the center of the screen, and (b) the direction and amount of change from the reference posture to the current posture. A process of calculating the corresponding first coordinates (and thus the screen coordinates) is executed. Therefore, the second method calculates the screen coordinates so that the reference posture always has a specific value (in this embodiment, the value at the center of the screen) by the processes (a) and (b). This prevents the user from feeling uncomfortable.

[Modification]
(Modification regarding the second method)
In the above-described embodiment, in the process before the start of the game (steps S2 and S3), the cursor is not displayed according to the attitude of the input device 8 (first cursor) but is displayed at the second coordinate position based on the marker coordinates. A cursor (second cursor) is used. Thus, the game apparatus 3 may use two types of cursors with different calculation methods. If the two types of cursors do not match at any position on the screen (the state of the input device 8 does not match when the positions of the two types of cursors match), the cursor moves from one cursor to the other. At the time of switching, the user feels that the operation method of the cursor is completely changed, so that the user may feel uncomfortable and feel that the operability is poor. On the other hand, in the above embodiment, the two types of cursors match at the center of the screen (the state of the input device 8 when the cursor is positioned at the center of the screen matches). Therefore, there is little discomfort when switching between the two types of cursors, and the user can be prevented from being confused with respect to the cursor operation.

  Moreover, in the said embodiment, since the offset amount regarding a 1st cursor is determined when a user performs operation using the said 2nd cursor, the behavior of the 2nd cursor currently displayed corresponds to the determination of an offset amount. There is no change. Therefore, the game apparatus 3 can determine the offset amount without causing the user to feel uncomfortable due to a sudden change in the behavior of the cursor.

  In other embodiments, the game apparatus 3 does not necessarily use the second cursor when determining the offset amount. For example, the game apparatus 3 displays a message “Please point the controller at the center of the screen and press the A button” (the cursor is not displayed), and when the A button is pressed, execute the processes of steps S96 and S97. You may make it do. Even in this case, the screen coordinates are located at the center of the screen in a state in which the user feels that “the input device 8 points to the center of the screen”. Can be reduced. In addition, the game apparatus 3 does not need to display a screen that suggests determining the offset amount, and may determine the offset amount when, for example, a game image is displayed.

  In the above embodiment, the game apparatus 3 executes the process of determining the offset amount (step S97) only before the process of using the screen coordinates as an input (step S84). Here, in another embodiment, when the process using the screen coordinates as an input is repeatedly executed, the process of determining the offset amount (again) may be executed between the times when the process is repeatedly executed. For example, in the case of the above embodiment, the process for determining the offset amount may be executed while the process loop of steps S3 to S9 is repeatedly executed during the game. For example, the game apparatus 3 may execute a process for determining the offset amount at a timing when the setting screen is displayed during the game, may execute the process at a predetermined time interval, or may be instructed by the user. The processing may be executed at the timing. As described above, when the state of the user holding the input device changes during the game by re-determining the offset amount during the game (for example, when the game is standing and playing but sitting in the middle of the game) However, an appropriate offset amount can be determined again. Note that when the process for re-determining the offset amount is executed, the offset amount changes, thereby causing a change in screen coordinates not intended by the user, and the user may not be able to perform the game operation well. For this reason, the game apparatus 3 does not execute the process of re-determining the offset amount as much as possible during the period in which the process using the screen coordinates as an input is being performed (for example, the period in which the game operation is performed using the cursor). Is preferred. Further, when executing the processing, it is preferable to execute it in a scene where the influence of the change in the screen coordinates is small (for example, during a game pause or during the display of a menu screen).

  In the above embodiment, the game apparatus 3 calculates the first coordinates from the attitude of the input device 8 using the plane Q set in the virtual space, and calculates the screen coordinates from the first coordinates. Here, the method for calculating the screen coordinates from the attitude of the input device 8 may be any method as long as the position of the screen coordinates changes according to the attitude. For example, the game apparatus 3 may directly calculate the screen coordinates from the attitude of the input device 8 without calculating the first coordinates (in other words, the first coordinates are coordinates representing a position on the screen). May be.) Specifically, the game apparatus 3 sets the screen coordinates to be the center position of the screen when the input device 8 is in a predetermined reference posture, and has a size proportional to the rotation angle from the reference posture. Thus, the screen coordinates may be calculated so as to represent the position moved in the rotation direction from the reference posture (the screen coordinates at the center position). Further, for example, the game apparatus 3 may directly calculate the screen coordinates from the attitude of the input device 8 by setting the X′Y ′ coordinates so as to coincide with the coordinate system of the screen coordinates.

  In the above embodiment, the game apparatus 3 indirectly corrects the screen coordinates by correcting the first coordinates using the offset amount. However, in other embodiments, the game coordinates are changed to the screen coordinates. You may make it correct | amend directly. Specifically, the game apparatus 3 may further calculate screen coordinates from the first coordinates in step S96, and may calculate an offset amount related to the screen coordinates in step S97. At this time, in steps S82 and S83, the game apparatus 3 calculates screen coordinates from the first coordinates calculated by the first method, and corrects the calculated screen coordinates according to the offset amount, thereby finally What is necessary is just to obtain a screen coordinate.

  In the above embodiment, the game apparatus 3 sets the offset amount with respect to the first coordinate, that is, corrects the first coordinate by moving the X′Y ′ coordinate system on the plane Q. (FIGS. 32 and 37). Here, in another embodiment, the first coordinate may be corrected by rotating the X′Y ′ coordinate system around the position of the input device 8 in the virtual space. FIG. 38 is a diagram showing a modification of the correction method in the second method. FIG. 38 is a side view of the plane Q in the virtual space. The plane Q indicated by the dotted line is the same plane as the plane Q used in the first method, and the plane Q ′ indicated by the solid line is corrected. This is the later plane Q. In the modification shown in FIG. 38, the CPU 10 executes the following processing instead of the processing in steps S96 and S97 shown in FIG. That is, the CPU 10 sets the corrected X′Y ′ coordinate system based on the attitude of the input device 8 (first attitude M1). The X′Y ′ coordinate system is set on a plane Q ′ that is the origin at a position separated by a length L in the direction of the Z-axis vector VZ of the input device 8 and that is perpendicular to the Z-axis vector VZ. Further, during the game, the CPU 10 calculates the first coordinates represented by the X′Y ′ coordinate system on the plane Q ′ instead of the processing of steps S7 and S82 shown in FIG. According to the modified example, since the distance from the position of the input device 8 in the virtual space is constant, the first coordinates (screen coordinates) can be calculated more accurately. For example, when the screen of the television 2 is actually installed tilted from the vertical direction (for example, when the television 2 is installed on the ceiling with the screen facing downward from the horizontal), the screen coordinates are calculated more accurately. can do. On the other hand, in the correction method in the above embodiment, the X′Y ′ coordinate system is set in a plane fixed with respect to the position of the input device 8, so that the correction calculation is simplified. Therefore, when the situation where the screen of the television 2 is installed with a large inclination from the vertical direction is not assumed, or when priority is given to reducing the processing load of the CPU 10, it is preferable to employ the correction method in the above embodiment.

(Modification regarding imaging means)
In the above-described embodiment, the game apparatus 3 uses a captured image (marker coordinates) obtained by an imaging unit included in the input apparatus 8 in order to detect that the input apparatus 8 is in a direction toward the center of the screen. Here, as long as the game apparatus 3 can know the direction of the input apparatus 8 when viewed from a predetermined position in the space, the game apparatus 3 may perform the detection using other information instead of the captured image. . For example, when a camera is installed around the television 2, the game apparatus 3 may perform the detection using a captured image obtained by capturing the input device 8 with the camera. Any method may be used for the detection. For example, a device that generates a signal such as a radio wave or an ultrasonic wave is installed around the input device 8. In this case, the direction of the input device 8 when viewed from a predetermined position in the space may be detected by detecting the signal.

(Modification regarding attitude calculation of input device 8)
In the above embodiment, the game apparatus 3 calculates the attitude of the input device 8 using the detection results obtained by the gyro sensors 55 and 56, the acceleration sensor 37, and the image sensor 40 (steps S3 to S6). Here, the method for calculating the attitude of the input device 8 may be any method. For example, the attitude of the input device 8 may be calculated using only the detection result of the gyro sensor, or may be calculated using only the detection result of the acceleration sensor.

  In the above-described embodiment, the case where a three-dimensional posture is calculated using a gyro sensor that detects angular velocities around three axes has been described as an example. However, the present invention is also applied to a case where a two-dimensional posture is calculated. It is possible to apply. For example, if it is premised that the rotation in the roll direction is not performed, the two-dimensional posture related to the rotation in the pitch direction and the yaw direction may be calculated by detecting an angular velocity about two axes with a two-axis gyro sensor. . Then, the screen coordinates may be calculated from the two-dimensional posture.

  In another embodiment, the second correction process may be executed only when it is estimated that the input device 8 is imaging the marker unit 6. Specifically, before executing the second correction process, the CPU 10 determines whether or not the input device 8 (imaging unit) faces a direction in which the marker unit 6 can be imaged. This determination can be performed using the first posture or the second posture. For example, the first posture (or the second posture) may be determined as to whether the imaging direction of the input device 8 is the same or opposite to the direction from the input device 8 to the marker unit 6. In addition, the first posture used for the above determination may be the first posture in which the first and second correction processes are performed in the previous processing loop, or may be calculated in the current processing loop. The first posture in which one correction process is performed may be used.

  As a result of the above determination, when it is determined that the input device 8 faces the direction in which the marker unit 6 can be imaged, the CPU 10 performs the second correction process and faces the direction in which the marker unit 6 can be imaged. If it is determined that there is not, the second correction process is skipped. Note that an object that is not the marker unit 6 (for example, an electric lamp in a room or sunlight outside a window) may be erroneously detected as the marker unit 6, and the third posture is obtained using the marker coordinates obtained by the erroneous detection. When is calculated, correct correction cannot be performed even if the second correction process is performed using the third posture. On the other hand, by performing the above determination process, it is possible to prevent the second correction process from being performed using the third posture calculated from the marker coordinates obtained by the erroneous detection. As a result, the second correction process can be performed more accurately.

  As described above, the present invention is an information processing apparatus for displaying a Web page or an electronic program guide on a display device for the purpose of controlling the display range displayed on the display device using the output of the gyro sensor. It can also be used as a game device for displaying game images on a display device.

DESCRIPTION OF SYMBOLS 1 Game system 2 Television 3 Game device 4 Optical disk 5 Controller 6 Marker part 7 Gyro sensor unit 8 Input device 10 CPU
11c GPU
11e Internal main memory 12 External main memory 63 Angular velocity data 64 Acceleration data 65 Marker coordinate data 68 First attitude data 77 Virtual plane coordinate data 78 Projection vector data 79 Camera orientation data 80 Camera position data 81 Cursor position data 91 Virtual camera 92 Display target

Claims (28)

An information processing device that inputs an angular velocity detected by a gyro sensor included in an input device and displays an image on a display device,
An attitude calculation unit that calculates an attitude of the input device based on the angular velocity;
A coordinate calculation unit that calculates the coordinates of the intersection of a line segment extending from a predetermined position in a predetermined space in the direction of the vector representing the posture and a predetermined plane in the predetermined space;
An information processing apparatus comprising: a display control unit that controls a display range to be displayed on the display device among display targets based on the coordinates.   The information processing apparatus according to claim 1, wherein the display control unit controls the display range by changing a direction and / or position of a virtual camera based on the coordinates in a virtual space in which the display target is arranged. . The information processing apparatus further inputs operation information indicating a direction instruction operation on an operation unit provided in the input device,
The information processing according to claim 2, wherein the display control unit controls the display range by changing a direction of the virtual camera based on the coordinates and changing a position of the virtual camera based on the operation information. apparatus.   The information processing apparatus according to any one of claims 1 to 3, wherein the display control unit changes the display range when the coordinates indicate a position outside a predetermined region on the predetermined plane.   The information processing apparatus according to claim 4, further comprising: a screen position calculation unit that calculates a position on the screen based on the coordinates so that the predetermined area corresponds to a screen area of the display device. An instruction image display unit for displaying a predetermined instruction image at a position on the screen calculated by the screen position calculation unit;
The information processing apparatus according to claim 5, further comprising: a process execution unit that executes predetermined information processing based on a position of the instruction image.   The information processing according to any one of claims 4 to 6, wherein the display control unit determines a direction in which the display range is changed in accordance with a direction from the predetermined region to a position indicated by the coordinates. apparatus.   8. The display control unit according to claim 4, wherein the display control unit determines a degree of changing the display range based on a degree that a position indicated by the coordinates is away from the predetermined region. Information processing device.   The said display control part determines the degree to which the said display range is changed based on the speed of the said coordinate in case the said coordinate is located outside the said predetermined area | region, The any one of Claim 4-7 The information processing apparatus described in 1.   10. The display control unit according to claim 9, wherein the display control unit determines a speed at which the display range is changed using at least a maximum value of the speed of the coordinates in a predetermined period after the coordinates are outside the predetermined area. Information processing device.   The information processing apparatus according to any one of claims 1 to 10, wherein the display control unit changes the display range based on the coordinates in a period specified by a user using the input device.   The information processing apparatus according to claim 11, wherein the display control unit moves the display range in a direction according to a movement direction of the coordinates by an amount according to a movement amount of the coordinates.   The display control unit displays the display in a direction according to a direction from a predetermined reference position on the predetermined plane to a position indicated by the coordinates at a speed according to a distance from the predetermined reference position to the position indicated by the coordinates. The information processing apparatus according to claim 11, wherein the range is moved. An information processing program that is executed in a computer of an information processing device that inputs an angular velocity detected by a gyro sensor included in the input device and displays an image on a display device,
Attitude calculating means for calculating the attitude of the input device based on the angular velocity;
Coordinate calculating means for calculating the coordinates of the intersection of a line segment extending from a predetermined position in a predetermined space in the direction of the vector representing the posture and a predetermined plane in the predetermined space;
An information processing program for causing the computer to function as display control means for controlling a display range to be displayed on the display device among display objects based on the coordinates.   The information processing program according to claim 14, wherein the display control means controls the display range by changing a direction and / or position of a virtual camera based on the coordinates in a virtual space in which the display target is arranged. . The information processing apparatus further inputs operation information indicating a direction instruction operation on an operation unit provided in the input device,
The information processing according to claim 15, wherein the display control unit controls the display range by changing an orientation of the virtual camera based on the coordinates and changing a position of the virtual camera based on the operation information. program.   The information processing program according to any one of claims 14 to 16, wherein the display control unit changes the display range when the coordinates indicate a position outside a predetermined region on the predetermined plane.   18. The information according to claim 17, further causing the computer to function as a screen position calculating unit that calculates a position on the screen based on the coordinates so that the predetermined area corresponds to a screen area of the display device. Processing program. Instruction image display means for displaying a predetermined instruction image at the position on the screen calculated by the screen position calculation means;
The information processing program according to claim 18, further causing the computer to function as processing execution means for executing predetermined information processing based on a position of the instruction image.   The information processing according to any one of claims 17 to 19, wherein the display control unit determines a direction in which the display range is changed according to a direction from the predetermined area to a position indicated by the coordinates. program.   The display control unit according to any one of claims 17 to 20, wherein the display control unit determines a degree of changing the display range based on a degree that a position indicated by the coordinates is away from the predetermined area. Information processing program.   The said display control means determines the degree to which the said display range is changed based on the speed of the said coordinate in case the said coordinate is located outside the said predetermined area | region. Information processing program described in 1.   The said display control means determines the speed | rate which changes the said display range using at least the maximum value of the speed | velocity | rate of the said coordinate in the predetermined period after the said coordinate comes out of the said predetermined area | region. Information processing program.   The information processing program according to any one of claims 14 to 23, wherein the display control unit changes the display range based on the coordinates in a period specified by a user using the input device.   25. The information processing program according to claim 24, wherein the display control unit moves the display range by an amount corresponding to a movement amount of the coordinates in a direction corresponding to a movement direction of the coordinates.   The display control means displays the display in a direction corresponding to a direction from a predetermined reference position on the predetermined plane to a position indicated by the coordinates at a speed corresponding to a distance from the predetermined reference position to the position indicated by the coordinates. The information processing program according to claim 24, wherein the range is moved. An information processing system for inputting an angular velocity detected by a gyro sensor included in an input device and displaying an image on a display device,
An attitude calculation unit that calculates an attitude of the input device based on the angular velocity;
A coordinate calculation unit that calculates the coordinates of the intersection of a line segment extending from a predetermined position in a predetermined space in the direction of the vector representing the posture and a predetermined plane in the predetermined space;
An information processing system comprising: a display control unit that controls a display range to be displayed on the display device among display objects based on the coordinates. A display range control method for inputting an angular velocity detected by a gyro sensor included in an input device and displaying an image on a display device,
An attitude calculating step for calculating an attitude of the input device based on the angular velocity;
A coordinate calculation step for calculating the coordinates of the intersection of a line segment extending from a predetermined position in a predetermined space in the direction of the vector representing the posture and a predetermined plane in the predetermined space;
A display range control method comprising: a display control step of controlling a display range to be displayed on the display device among display targets based on the coordinates.

Images