JP6555831B2 - Information processing program, information processing system, information processing apparatus, and information processing method - Google Patents

Description

  The present invention relates to an information processing system in which a device is mounted on another device and used.

  Conventionally, there is an information processing system in which a device (for example, a controller) is mounted on another device and used in a game application (for example, see Patent Document 1). For example, in a game system, a user may play a game using an operation device in which a core unit as a controller is attached to a gun-type subunit.

JP 2007-301000 A

  Conventionally, in one game application, a controller is mounted on one type of additional device corresponding to the game application. That is, since one additional device is used for one application, there is little variation in operation using the additional device, and there is room for improvement in terms of interest.

  Therefore, an object of the present invention is to improve interest in a game application in which an additional device is used.

  In order to solve the above problems, the present invention employs the following configurations (1) to (20).

(1)
An example of the present invention is an information processing program executed by a computer of a game system that executes a predetermined game application. The game system includes a plurality of types of sub devices and a main device that can be attached to and detached from any of the plurality of types of sub devices. The game program causes the computer to function as a determination unit and a game process execution unit. The discriminating unit discriminates the type of the sub device to which the main device is attached among the plurality of types of sub devices during execution of the predetermined game application. The game process execution means executes a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application. The game process execution means executes a game process corresponding to the type of the sub device determined by the determination means.

  According to the configuration of (1) above, the type of the sub device to which the main device is attached is determined during the execution of the game application, and the game process corresponding to the determined type of the sub device is executed. According to this, the user can play a game by switching the sub device to which the main device is mounted while the game application is being executed. Thereby, the game system can provide a new game experience to the user, and can improve the interest in the game application in which the sub device is used.

(2)
The main device may include an imaging device. The discriminating unit may discriminate the type of the sub device to which the main device is attached based on the captured image captured by the imaging device.

  According to the configuration of (2) above, the configuration of the sub device can be simplified.

(3)
Each of the plurality of sub-devices may include a detected portion that can be imaged by the imaging device of the main device mounted on the sub-device. The discriminating means is mounted on the main device based on at least one of the position, shape, and orientation of the first detected portion image showing at least a part of the detected portion included in the captured image. Determine the type of sub-device.

  According to the configuration of (3) above, the configuration of the sub device can be simplified.

(4)
At least any one of the plurality of sub-devices may further include a movable part that moves in response to an operation by a user. When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. It may change. The game process execution means indicates at least a part of the detected part included in the captured image, and at least of the position, shape, and orientation of the first detected part image and / or the second detected part image Based on one, the user's operation may be detected, and a game process based on the detected user's operation may be executed.

  According to the configuration of (4) above, the game system can detect a user operation performed on the sub device in addition to determining the type of the sub device.

(5)
The detected part may be provided at a position in the movable part that can be imaged by the imaging device provided in the main device mounted on a sub-device provided with the movable part. The relative position and / or the relative posture between the main device and the detected part may change by moving the detected part in response to the movable part being moved in response to the user's operation.

  According to the configuration of (5) above, the game system can easily detect the relative position and / or relative orientation between the main device and the detected part from the captured image by providing the detected part in the movable part. Can do.

(6)
The main device may be attachable to the movable part. When the movable unit is moved in response to the user's operation, the main device mounted on the movable unit is moved, so that the relative position and / or relative posture between the main device and the detected unit is changed. It may change.

  According to the configuration of (6) above, the game system can easily determine the relative position and / or relative orientation between the main device and the detected portion from the captured image by adopting a configuration in which the main device can be attached to the movable portion. Can be detected.

(7)
The game process execution means may generate a different game image according to the determined type of the sub device.

  According to the configuration of (7) above, since the game image changes according to the type of the sub device to which the main device is attached, the game system can make the user easily recognize the attachment state of the main device. it can.

(8)
The game process execution means may control an object arranged in the virtual space based on a game operation using the main device and / or a sub device to which the main device is attached. The game process execution means may change the object according to the determined type of the sub device.

  With configuration (8) above, the user can change the object to be operated by changing the sub device on which the main device is mounted. Thereby, the game system can provide the user with a new game operation.

(9)
The object may be an object representing a vehicle on which the game character can board.

  According to the configuration of (9) above, the user can change the vehicle object to be operated by changing the sub device on which the main device is mounted, and the game system provides a highly entertaining game. Can do.

(10)
The main device may include a plurality of types of operation detection units for detecting user operations on the main device. When it is determined that the main device is attached to a predetermined type of sub-device, the game processing means is detected by the operation detection unit according to the determined type of the sub-device among a plurality of types of operation detection units. A game process may be executed based on the result.

  According to the configuration of (10) above, the game system can execute the game process based on a detection result by an appropriate operation detection unit according to the type of the sub device to which the main device is attached. In addition, when the user switches the sub device on which the main device is mounted, the user's convenience is improved because it is possible to reduce the trouble of changing the game setting (for example, the setting relating to the operation detection unit).

(11)
The main device may include an operation detection unit for detecting a user operation on the main device. The game processing means may determine the content of the game processing executed according to the detection result by the operation detection unit based on the determined type of the sub device.

  According to the configuration of (11) above, the game system appropriately sets the content of the game process executed according to the detection result by the operation detection unit according to the type of the sub device to which the main device is attached. Can do. In addition, when the user switches the sub device on which the main device is mounted, the user's convenience is improved because the time and effort for changing the game settings is reduced.

(12)
The game processing means may control the position and / or posture of the virtual camera arranged in the virtual space based on the determined type of the sub device.

  According to the configuration of (12) above, the game system can provide the user with a game image viewed from an appropriate viewpoint according to the game situation. In addition, when the user switches the sub apparatus to which the main apparatus is mounted, the user's convenience is improved because the effort for changing the settings of the virtual camera can be reduced.

(13)
The determination unit may further determine that the main device is not attached to any of the plurality of sub-devices.

  According to the configuration of (13) above, the game system can detect a state in which the main device is not attached to the sub device.

(14)
When it is determined that the main device is not attached to any of the plurality of sub devices, the game process execution means is different from the case where the main device is attached to any of the plurality of sub devices. Game processing may be executed.

  According to the configuration of (14) above, the user recognizes whether or not the main device is attached to the sub device in addition to the type of the sub device to which the main device is attached by looking at the game image. Can do. According to this, the game system can make a user recognize the more detailed mounting state of the main device.

(15)
The game system may further include a storage unit. The information processing program may further cause the computer to function as storage control means. When it is determined that the main device is attached to a predetermined type of sub device, the storage control means stores device orientation information related to the orientation of the main device with respect to the predetermined type of sub device in the storage unit. The game process execution means may execute the game process based on a direction operation by the user with respect to the main device. The game processing execution means determines that the main device is not attached to any of the plurality of sub devices from the attached state where the main device is determined to be attached to the predetermined type of sub device. When the state changes to the non-wearing state, the relationship between the direction specified by the direction operation and the content of the game process executed in response to the operation in the direction is stored in the storage unit in the wearing state. You may determine based on the apparatus orientation information memorize | stored.

  According to the configuration of (15) above, the game system can perform game processing in consideration of the orientation of the main device in the mounted state before changing to the non-mounted state.

(16)
The detected part included in at least one of the plurality of sub-devices may include a first portion and a second portion. The information processing program may further cause the computer to function as recognition means. The recognition means recognizes the first part and recognizes the second part based on the captured image. The determination unit may determine the type of the sub apparatus on which the main apparatus is mounted based on at least one of the position, shape, and orientation of the first partial image that indicates the first part. The discriminating means includes a first state in which the first part is recognized and the second part is not recognized, and a second state in which the first part and the second part are recognized as states of the main device mounted on the sub device. It may be determined.

  According to the configuration of (16) above, the game system can estimate the degree of attachment of the main device to the sub device.

(17)
The game process execution means executes the first game process when it is determined that the main device is in the first state, and the first game process when the main device is determined as the second state. May execute a different second game process.

  According to the configuration of (17) above, different game processes are executed in the first state and the second state, and different game results are produced (for example, different game images are displayed). It is possible to make the user recognize the degree of attachment to the sub-device.

(18)
The detected part included in at least one of the plurality of sub-devices may include a first portion and a second portion. The information processing program may further cause the computer to function as recognition means. The recognition means recognizes the first part and recognizes the second part based on the captured image. The determination unit may determine the type of the sub apparatus on which the main apparatus is mounted based on at least one of the position, shape, and orientation of the first partial image that indicates the first part. The determining means may determine the degree to which the main device is attached to the sub device based on the recognition result of the first part and the recognition result of the second part.

  According to the configuration of (18) above, the game system can estimate the degree of attachment of the main device to the sub device.

(19)
The game process execution means may execute a game process according to the degree to which the main device is attached to the sub device.

  According to the configuration of (19) above, game processing is executed according to the degree to which the main device is attached to the sub device, and a game result according to the degree is produced (for example, an image according to the attachment degree is displayed). Therefore, the game system can make the user recognize the degree of attachment of the main device to the sub device.

(20)
At least one of the plurality of sub-devices may further include a movable unit that moves in response to an operation by the user. The relative position between the main device attached to the sub apparatus including the movable part and the detected part of the sub apparatus may be changed by moving the movable part in accordance with a user operation. The game process execution means detects the first type of user operation based on at least one of the position, shape, and orientation of the first partial image indicating the first part, and the detected first type A game process based on the above operation may be executed. The game process execution means detects the user's second type operation based on at least one of the position, shape, and orientation of the second partial image indicating the second part, and the detected second type A game process based on the above operation may be executed.

  With configuration (20) above, the user performs some game operations even when the user is determined to be in the first state (for example, when the main device is being attached to the sub device). be able to. Thereby, the game system can improve the operability of the operation using the sub device.

  In addition, another example of the present invention may be an information processing apparatus or an information processing system including each means in the above (1) to (20). Another example of the present invention may be an information processing method executed in the information processing apparatus (or information processing system) in the above (1) to (20).

  ADVANTAGE OF THE INVENTION According to this invention, the interest property in the game application in which an additional apparatus (namely, sub apparatus) is used can be improved.

The figure which shows an example of the state which mounted | wore the main body apparatus with the left controller and the right controller The figure which shows an example of the state which each removed the left controller and the right controller from the main body apparatus Six views showing an example of the main unit Six views showing an example of the left controller Six-sided view showing an example of the right controller Block diagram showing an example of the internal configuration of the main unit Block diagram showing an example of the internal configuration of the main unit, the left controller and the right controller The figure which shows the relationship between the combination of a controller and an additional operation apparatus, and the object used as operation object The figure which shows an example of a 1st addition operation apparatus and a right controller The figure which looked at the 1st addition operation device shown in Drawing 9 from the front side The figure which shows an example of the internal structure of a housing | casing The figure which shows an example of the captured image imaged by the infrared imaging part, when the handle | steering_wheel of a 1st additional operation apparatus will be in a reference state. The figure which shows an example of the captured image imaged by the infrared imaging part, when the handle | steering_wheel of a 1st additional operation apparatus will be in the state rotated from the reference | standard state. The figure which shows an example of the 1st additional operation apparatus in the state in which the lowering operation with respect to the left lever was performed The figure which shows an example of the captured image imaged with the infrared imaging part in the state in which the lowering operation with respect to the left lever was performed The figure which shows an example of the game image displayed when a right controller is mounted | worn with a 1st additional operation apparatus. The figure which shows an example of a 2nd additional operation apparatus and a right controller The figure which looked at the 2nd additional operation device shown in Drawing 17 from the upper part The figure which shows an example of the inside of a holding part in the state in which the button of the holding part was pressed The figure which shows an example of the game image displayed when a right controller is mounted | worn with a 2nd additional operation apparatus. The figure which shows an example of the game image displayed when it changes from a 2nd mounting state to a non-wearing state. The figure which shows an example of the setting method of the front direction of a right controller in the case of shifting to a non-wearing state from a 2nd wearing state. The figure which shows an example of a 3rd additional operation apparatus and a right controller The figure which shows an example of an internal structure of a 3rd additional operation apparatus. The figure which shows an example of the game image displayed when a right controller is mounted | worn with a 3rd additional operation apparatus. The figure which shows an example in the case of becoming a partial recognition state and the case of becoming a full recognition state The figure which shows an example of the various data used for the information processing in a game system The flowchart which shows an example of the flow of the information processing performed by information processing apparatus FIG. 28 is a sub-flowchart showing an example of a detailed flow of the wearing state determination process in step S2 shown in FIG.

[1. Hardware configuration of main unit and each controller]
Hereinafter, a game system according to an example of the present embodiment will be described. An example of the game system 1 in the present embodiment includes a main device (information processing device; functions as a game device main body in the present embodiment) 2, a left controller 3, and a right controller 4. The main body device 2 is detachably attachable to the left controller 3 and the right controller 4. That is, the game system 1 can be used as an apparatus in which the left controller 3 and the right controller 4 are respectively attached to the main body apparatus 2 and integrated. The game system 1 can also use the main body device 2, the left controller 3, and the right controller 4 as separate bodies (see FIG. 2). Although details will be described later, the game system 1 includes an attachment to which each controller 3 or 4 can be attached. Below, the hardware configuration of the main body device 2 and the controllers 3 and 4 will be described first, and then the configuration of the attachment and the processing of the game system 1 when using the attachment will be described.

  FIG. 1 is a diagram illustrating an example of a state in which the left controller 3 and the right controller 4 are mounted on the main body device 2. As shown in FIG. 1, the left controller 3 and the right controller 4 are each attached to and integrated with the main body device 2. The main device 2 is a device that executes various processes (for example, game processes) in the game system 1. The main device 2 includes a display 12. The left controller 3 and the right controller 4 are devices including an operation unit for a user to input.

  FIG. 2 is a diagram illustrating an example of a state in which the left controller 3 and the right controller 4 are removed from the main body device 2. As shown in FIGS. 1 and 2, the left controller 3 and the right controller 4 are detachable from the main body device 2. Hereinafter, the left controller 3 and the right controller 4 may be collectively referred to as “controller”.

  FIG. 3 is a six-sided view illustrating an example of the main device 2. As shown in FIG. 3, the main device 2 includes a substantially plate-shaped housing 11. In the present embodiment, the main surface of the housing 11 (in other words, the surface on the front side, that is, the surface on which the display 12 is provided) is generally rectangular.

  The shape and size of the housing 11 are arbitrary. As an example, the housing 11 may be a portable size. Further, the main body device 2 alone or the integrated device in which the left controller 3 and the right controller 4 are attached to the main body device 2 may be a portable device. Further, the main body device 2 or the integrated device may be a handheld device. Further, the main body device 2 or the integrated device may be a portable device.

  As shown in FIG. 3, the main device 2 includes a display 12 provided on the main surface of the housing 11. The display 12 displays an image generated by the main device 2. In the present embodiment, the display 12 is a liquid crystal display (LCD). However, the display 12 may be any type of display device.

  The main device 2 also includes a touch panel 13 on the screen of the display 12. In the present embodiment, the touch panel 13 is a type capable of multi-touch input (for example, a capacitance type). However, the touch panel 13 may be of any type, and may be of a type capable of single touch input (for example, a resistance film type).

  The main device 2 includes a speaker (that is, the speaker 88 shown in FIG. 6) inside the housing 11. As shown in FIG. 3, speaker holes 11 a and 11 b are formed in the main surface of the housing 11. And the output sound of the speaker 88 is output from these speaker holes 11a and 11b, respectively.

  The main device 2 includes a left terminal 17 that is a terminal for the main device 2 to perform wired communication with the left controller 3, and a right terminal 21 for the main device 2 to perform wired communication with the right controller 4.

  As shown in FIG. 3, the main body device 2 includes a slot 23. The slot 23 is provided on the upper side surface of the housing 11. The slot 23 has a shape in which a predetermined type of storage medium can be mounted. The predetermined type of storage medium is, for example, a storage medium (for example, a dedicated memory card) dedicated to the game system 1 and the same type of information processing apparatus. The predetermined type of storage medium stores, for example, data (eg, application save data) used by the main device 2 and / or a program (eg, application program) executed by the main device 2. Used to do. The main device 2 also includes a power button 28.

  The main device 2 includes a lower terminal 27. The lower terminal 27 is a terminal for the main device 2 to communicate with the cradle. In the present embodiment, the lower terminal 27 is a USB connector (more specifically, a female connector). When the integrated device or the main device 2 alone is placed on the cradle, the game system 1 can display an image generated and output by the main device 2 on the stationary monitor. Further, in the present embodiment, the cradle has a function of charging the mounted integrated device or the main body device 2 alone. The cradle has a function of a hub device (specifically, a USB hub).

  FIG. 4 is a six-sided view illustrating an example of the left controller 3. As shown in FIG. 4, the left controller 3 includes a housing 31. In the present embodiment, the housing 31 has a vertically long shape, that is, a shape that is long in the vertical direction (that is, the y-axis direction shown in FIGS. 1 and 4). The left controller 3 can also be held in a vertically long orientation when removed from the main unit 2. The housing 31 has a shape and size that can be gripped with one hand, particularly the left hand, when gripped in a vertically long orientation. Further, the left controller 3 can be held in a landscape orientation. When the left controller 3 is gripped in a landscape orientation, it may be gripped with both hands.

  The left controller 3 includes an analog stick 32. As shown in FIG. 4, the analog stick 32 is provided on the main surface of the housing 31. The analog stick 32 can be used as a direction input unit capable of inputting a direction. By tilting the analog stick 32, the user can input a direction corresponding to the tilting direction (and an input corresponding to the tilted angle). Note that the left controller 3 may include a cross key or a slide stick capable of slide input, as a direction input unit, instead of the analog stick. In the present embodiment, an input for pressing the analog stick 32 is possible.

  The left controller 3 includes various operation buttons. The left controller 3 includes four operation buttons 33 to 36 (specifically, a right direction button 33, a down direction button 34, an up direction button 35, and a left direction button 36) on the main surface of the housing 31. Further, the left controller 3 includes a recording button 37 and a − (minus) button 47. The left controller 3 includes a first L button 38 and a ZL button 39 on the upper left of the side surface of the housing 31. In addition, the left controller 3 includes a second L button 43 and a second R button 44 on a side surface of the housing 31 that is attached when the housing 31 is attached to the main body device 2. These operation buttons are used to give instructions according to various programs (for example, OS programs and application programs) executed by the main body device 2.

  The left controller 3 also includes a terminal 42 for the left controller 3 to perform wired communication with the main body device 2.

  FIG. 5 is a six-sided view illustrating an example of the right controller 4. As shown in FIG. 5, the right controller 4 includes a housing 51. In the present embodiment, the housing 51 has a vertically long shape, that is, a shape that is long in the vertical direction. The right controller 4 can also be gripped in a portrait orientation when removed from the main device 2. The housing 51 has a shape and size that can be gripped with one hand, particularly the right hand, when gripped in a portrait orientation. The right controller 4 can also be held in a landscape orientation. When the right controller 4 is gripped in a landscape orientation, it may be gripped by both hands.

  Similar to the left controller 3, the right controller 4 includes an analog stick 52 as a direction input unit. In the present embodiment, the analog stick 52 has the same configuration as the analog stick 32 of the left controller 3. Further, the right controller 4 may include a cross key or a slide stick capable of slide input, instead of the analog stick. Similarly to the left controller 3, the right controller 4 has four operation buttons 53 to 56 (specifically, an A button 53, a B button 54, an X button 55, and a Y button 56) on the main surface of the housing 51. Is provided. Further, the right controller 4 includes a + (plus) button 57 and a home button 58. The right controller 4 includes a first R button 60 and a ZR button 61 on the upper right side of the side surface of the housing 51. The right controller 4 includes a second L button 65 and a second R button 66 as in the left controller 3.

  A window portion 68 is provided on the lower side surface of the housing 51. As will be described in detail later, the right controller 4 includes an infrared imaging unit 123 and an infrared light emitting unit 124 disposed inside the housing 51. The infrared imaging unit 123 images the periphery of the right controller 4 through the window unit 68 with the downward direction of the right controller 4 (the negative y-axis direction shown in FIG. 5) as the imaging direction. The infrared light emitting unit 124 has a predetermined range centered on the downward direction of the right controller 4 (the negative y-axis direction shown in FIG. 5) as an irradiation range, and the imaging target captured by the infrared imaging unit 123 via the window unit 68. Irradiate with infrared light. The window unit 68 is for protecting the lens of the camera of the infrared imaging unit 123, the light emitter of the infrared light emitting unit 124, and the like, and emits light of a wavelength detected by the camera or light emitted by the light emitter. A transparent material (for example, a transparent material) is used. Note that the window 68 may be a hole formed in the housing 51. In the present embodiment, the infrared imaging unit 123 itself has a filter member that suppresses transmission of light having a wavelength other than light detected by the camera (in this embodiment, infrared light). However, in other embodiments, the window 68 may have a filter function.

  Although details will be described later, the right controller 4 includes an NFC communication unit 122. The NFC communication unit 122 performs near field communication based on NFC (Near Field Communication) standards. The NFC communication unit 122 includes an antenna 122a used for short-range wireless communication and a circuit (for example, an NFC chip) that generates a signal (radio wave) to be transmitted from the antenna 122a. Note that the NFC communication unit 122 may perform short-range wireless communication by arbitrary short-range communication (also referred to as non-contact communication) instead of short-range wireless communication based on the NFC standard. Here, the NFC standard can be used for proximity communication (non-contact communication), and “an arbitrary proximity communication may be used to perform short-range wireless communication” means that the proximity according to the NFC standard is used. It is intended that short-range wireless communication may be performed by other proximity communication excluding communication.

  The right controller 4 includes a terminal 64 for the right controller 4 to perform wired communication with the main body device 2.

  FIG. 6 is a block diagram illustrating an example of the internal configuration of the main device 2. The main unit 2 includes the components 81 to 91, 97, and 98 shown in FIG. 6 in addition to the configuration shown in FIG. Some of these components 81 to 91, 97, and 98 may be mounted on the electronic circuit board as electronic components and housed in the housing 11.

  The main device 2 includes a processor 81. The processor 81 is an information processing unit that executes various types of information processing executed in the main device 2, and may be composed of, for example, only a CPU (Central Processing Unit), a CPU function, or a GPU (Graphics Processing Unit). ) May be composed of SoC (System-on-a-chip) including a plurality of functions such as functions. The processor 81 executes an information processing program (for example, a game program) stored in a storage unit (specifically, an internal storage medium such as the flash memory 84 or an external storage medium attached to the slot 23). As a result, various types of information processing are executed.

  The main device 2 includes a flash memory 84 and a DRAM (Dynamic Random Access Memory) 85 as an example of an internal storage medium built therein. The flash memory 84 and the DRAM 85 are connected to the processor 81. The flash memory 84 is a memory mainly used for storing various data (may be programs) stored in the main device 2. The DRAM 85 is a memory used for temporarily storing various data used in information processing.

  The main device 2 includes a slot interface (hereinafter abbreviated as “I / F”) 91. The slot I / F 91 is connected to the processor 81. The slot I / F 91 is connected to the slot 23 and reads and writes data from and to a predetermined type of storage medium (for example, a dedicated memory card) attached to the slot 23 in accordance with an instruction from the processor 81.

  The processor 81 appropriately reads and writes data between the flash memory 84, the DRAM 85, and each storage medium, and executes the above information processing.

  The main device 2 includes a network communication unit 82. The network communication unit 82 is connected to the processor 81. The network communication unit 82 performs communication (specifically, wireless communication) with an external device via a network. In the present embodiment, the network communication unit 82 communicates with an external device by connecting to a wireless LAN by a method based on the Wi-Fi standard as a first communication mode. Moreover, the network communication part 82 performs radio | wireless communication between the other main body apparatuses 2 of the same kind with a predetermined | prescribed communication system (For example, communication by an original protocol, infrared communication) as a 2nd communication mode. The wireless communication according to the second communication mode can be wirelessly communicated with other main body devices 2 arranged in the closed local network area, and directly communicates between the plurality of main body devices 2. This realizes a function that enables so-called “local communication” in which data is transmitted and received.

  The main device 2 includes a controller communication unit 83. The controller communication unit 83 is connected to the processor 81. The controller communication unit 83 performs wireless communication with the left controller 3 and / or the right controller 4. The communication method between the main device 2 and the left controller 3 and the right controller 4 is arbitrary, but in this embodiment, the controller communication unit 83 communicates with the left controller 3 and between the right controller 4 and the Bluetooth ( Performs communication according to the registered trademark standard.

  The processor 81 is connected to the left terminal 17, the right terminal 21, and the lower terminal 27 described above. When performing wired communication with the left controller 3, the processor 81 transmits data to the left controller 3 through the left terminal 17 and receives operation data from the left controller 3 through the left terminal 17. Further, when performing wired communication with the right controller 4, the processor 81 transmits data to the right controller 4 via the right terminal 21 and receives operation data from the right controller 4 via the right terminal 21. Further, the processor 81 transmits data to the cradle via the lower terminal 27 when communicating with the cradle. Thus, in the present embodiment, the main device 2 can perform both wired communication and wireless communication with the left controller 3 and the right controller 4, respectively. When the left controller 3 and the right controller 4 are mounted on the main body device 2 or the single body device 2 is mounted on the cradle, the main body device 2 receives data (for example, image data or sound) via the cradle. Data) can be output to a stationary monitor or the like.

  Here, the main device 2 can perform communication simultaneously with the plurality of left controllers 3 (in other words, in parallel). Further, the main body device 2 can perform communication simultaneously with the plurality of right controllers 4 (in other words, in parallel). Accordingly, a plurality of users can simultaneously input to the main device 2 using the set of the left controller 3 and the right controller 4 respectively. As an example, the first user performs input to the main body device 2 using the first set of the left controller 3 and the right controller 4, and at the same time, the second user uses the second set of the left controller 3 and the right controller 4. Thus, it becomes possible to input to the main device 2.

  The main device 2 includes a touch panel controller 86 that is a circuit that controls the touch panel 13. The touch panel controller 86 is connected between the touch panel 13 and the processor 81. The touch panel controller 86 generates data indicating, for example, a position where touch input is performed based on a signal from the touch panel 13 and outputs the data to the processor 81.

  The display 12 is connected to the processor 81. The processor 81 displays the generated image and / or the image acquired from the outside (for example, by executing the above information processing) on the display 12.

  The main device 2 includes a codec circuit 87 and a speaker (specifically, a left speaker and a right speaker) 88. The codec circuit 87 is connected to the speaker 88 and the audio input / output terminal 25 and to the processor 81. The codec circuit 87 is a circuit that controls input / output of audio data to / from the speaker 88 and the audio input / output terminal 25.

  The main body device 2 includes an acceleration sensor 89. In the present embodiment, the acceleration sensor 89 detects the magnitude of acceleration along predetermined three axis directions (for example, the xyz axis shown in FIG. 1). Note that the acceleration sensor 89 may detect an acceleration in a uniaxial direction or a biaxial direction.

  The main body device 2 includes an angular velocity sensor 90. In the present embodiment, the angular velocity sensor 90 detects angular velocities about predetermined three axes (for example, the xyz axis shown in FIG. 1). Note that the angular velocity sensor 90 may detect an angular velocity around one axis or around two axes.

  The acceleration sensor 89 and the angular velocity sensor 90 are connected to the processor 81, and the detection results of the acceleration sensor 89 and the angular velocity sensor 90 are output to the processor 81. The processor 81 can calculate information related to the movement and / or posture of the main device 2 based on the detection results of the acceleration sensor 89 and the angular velocity sensor 90.

  The main device 2 includes a power control unit 97 and a battery 98. The power control unit 97 is connected to the battery 98 and the processor 81. Although not shown, the power control unit 97 is connected to each part of the main body device 2 (specifically, each part that receives power supply from the battery 98, the left terminal 17, and the right terminal 21). The power control unit 97 controls power supply from the battery 98 to each of the above parts based on a command from the processor 81.

  The battery 98 is connected to the lower terminal 27. When an external charging device (for example, a cradle) is connected to the lower terminal 27 and electric power is supplied to the main body device 2 via the lower terminal 27, the supplied electric power is charged to the battery 98.

  FIG. 7 is a block diagram illustrating an example of an internal configuration of the main body device 2, the left controller 3, and the right controller 4. The details of the internal configuration related to the main unit 2 are shown in FIG. 6 and are not shown in FIG.

  The left controller 3 includes a communication control unit 101 that performs communication with the main device 2. As shown in FIG. 7, the communication control unit 101 is connected to each component including the terminal 42. In the present embodiment, the communication control unit 101 can communicate with the main device 2 by both wired communication via the terminal 42 and wireless communication not via the terminal 42. The communication control unit 101 controls a communication method performed by the left controller 3 for the main body device 2. That is, when the left controller 3 is attached to the main body device 2, the communication control unit 101 communicates with the main body device 2 via the terminal 42. Further, when the left controller 3 is disconnected from the main body device 2, the communication control unit 101 performs wireless communication with the main body device 2 (specifically, the controller communication unit 83). Wireless communication between the controller communication unit 83 and the communication control unit 101 is performed, for example, according to the Bluetooth (registered trademark) standard.

  The left controller 3 includes a memory 102 such as a flash memory. The communication control unit 101 is configured by, for example, a microcomputer (also referred to as a microprocessor), and executes various processes by executing firmware stored in the memory 102.

  The left controller 3 includes buttons 103 (specifically, buttons 33 to 39, 43, 44, and 47). The left controller 3 includes an analog stick 32 (described as “stick” in FIG. 7). Each button 103 and the analog stick 32 repeatedly outputs information related to operations performed on itself to the communication control unit 101 at an appropriate timing.

  The left controller 3 includes an inertial sensor. Specifically, the left controller 3 includes an acceleration sensor 104. The left controller 3 includes an angular velocity sensor 105. In the present embodiment, the acceleration sensor 104 detects the magnitude of acceleration along predetermined three axes (for example, the xyz axis shown in FIG. 4). Note that the acceleration sensor 104 may detect an acceleration in a uniaxial direction or a biaxial direction. In the present embodiment, the angular velocity sensor 105 detects angular velocities about predetermined three axes (for example, the xyz axis shown in FIG. 4). The angular velocity sensor 105 may detect an angular velocity around one axis or around two axes. The acceleration sensor 104 and the angular velocity sensor 105 are each connected to the communication control unit 101. The detection results of the acceleration sensor 104 and the angular velocity sensor 105 are repeatedly output to the communication control unit 101 at appropriate timing.

  The communication control unit 101 receives information related to input (specifically, information related to an operation or a detection result by a sensor) from each input unit (specifically, each button 103, analog stick 32, and each sensor 104 and 105). To get. The communication control unit 101 transmits operation data including the acquired information (or information obtained by performing predetermined processing on the acquired information) to the main body device 2. The operation data is repeatedly transmitted at a rate of once every predetermined time. Note that the interval at which the information related to the input is transmitted to the main device 2 may or may not be the same for each input unit.

  By transmitting the operation data to the main device 2, the main device 2 can obtain an input made to the left controller 3. That is, the main body device 2 can determine the operation on each button 103 and the analog stick 32 based on the operation data. Further, the main device 2 can calculate information related to the movement and / or posture of the left controller 3 based on operation data (specifically, detection results of the acceleration sensor 104 and the angular velocity sensor 105).

  The left controller 3 includes a vibrator 107 for notifying the user by vibration. In the present embodiment, the vibrator 107 is controlled by a command from the main device 2. That is, when the communication control unit 101 receives the command from the main device 2, the communication control unit 101 drives the vibrator 107 according to the command. Here, the left controller 3 includes a codec unit 106. Upon receiving the command, the communication control unit 101 outputs a control signal corresponding to the command to the codec unit 106. The codec unit 106 generates a drive signal for driving the vibrator 107 from the control signal from the communication control unit 101 and supplies the drive signal to the vibrator 107. As a result, the vibrator 107 operates.

  More specifically, the vibrator 107 is a linear vibration motor. Since a linear vibration motor is driven in a predetermined direction in accordance with an input voltage, unlike a normal motor that rotates, the linear vibration motor can be vibrated with an amplitude and a frequency in accordance with the waveform of the input voltage. In the present embodiment, the vibration control signal transmitted from the main device 2 to the left controller 3 may be a digital signal that represents a frequency and an amplitude for each unit time. In another embodiment, information indicating the waveform itself may be transmitted from the main device 2, but the amount of communication data can be reduced by transmitting only the amplitude and frequency. In order to further reduce the data amount, only the difference from the previous value may be transmitted instead of the numerical values of the amplitude and frequency at that time. In this case, the codec unit 106 converts the digital signal indicating the amplitude and frequency values acquired from the communication control unit 101 into an analog voltage waveform, and inputs the voltage in accordance with the waveform, thereby causing the vibrator 107 to operate. Drive. Therefore, the main body device 2 can control the amplitude and frequency at which the vibrator 107 is vibrated at that time by changing the amplitude and frequency to be transmitted every unit time. The amplitude and frequency transmitted from the main device 2 to the left controller 3 are not limited to one, and two or more may be transmitted. In that case, the codec unit 106 can generate a waveform of a voltage for controlling the vibrator 107 by synthesizing the waveforms indicated by the received plurality of amplitudes and frequencies.

  The left controller 3 includes a power supply unit 108. In the present embodiment, the power supply unit 108 includes a battery and a power control circuit. Although not shown, the power control circuit is connected to the battery and is connected to each part of the left controller 3 (specifically, each part that receives power from the battery).

  As illustrated in FIG. 7, the right controller 4 includes a communication control unit 111 that performs communication with the main body device 2. The right controller 4 includes a memory 112 connected to the communication control unit 111. The communication control unit 111 is connected to each component including the terminal 64. The communication control unit 111 and the memory 112 have the same functions as the communication control unit 101 and the memory 102 of the left controller 3. Therefore, the communication control unit 111 communicates with the main body device 2 both in wired communication via the terminal 64 and wireless communication not through the terminal 64 (specifically, communication in accordance with the Bluetooth (registered trademark) standard). Communication is possible, and the right controller 4 controls a communication method performed with respect to the main body device 2.

  The right controller 4 includes input units similar to the input units of the left controller 3. Specifically, each button 113, the analog stick 52, and an inertial sensor (acceleration sensor 114 and angular velocity sensor 115) are provided. Each of these input units has the same function as each input unit of the left controller 3 and operates in the same manner.

  The right controller 4 includes a vibrator 117 and a codec unit 116. The vibrator 117 and the codec unit 116 operate in the same manner as the vibrator 107 and the codec unit 106 of the left controller 3. That is, the communication control unit 111 operates the vibrator 117 using the codec unit 116 in accordance with a command from the main body device 2.

  The right controller 4 includes an NFC communication unit 122 that performs short-range wireless communication based on the NFC standard. The NFC communication unit 122 has a so-called NFC reader / writer function. Here, short-range wireless communication in this specification refers to the other device (here, the device close to the antenna 122a) by radio waves from one device (here, the right controller 4) (for example, by electromagnetic induction). A communication method for generating electromotive force is included. The other device can operate by the generated electromotive force, and may or may not have a power source. The NFC communication unit 122 communicates with the communication target when the right controller 4 (antenna 122a) and the communication target are close to each other (typically, when the distance between the two is less than ten centimeters). It becomes possible. The communication target is an arbitrary device capable of short-range wireless communication with the NFC communication unit 122, for example, an NFC tag or a storage medium having an NFC tag function. However, the communication target may be another device having an NFC card emulation function.

  The right controller 4 includes an infrared imaging unit 123. The infrared imaging unit 123 includes an infrared camera that captures an image around the right controller 4. As an example, the main device 2 and / or the right controller 4 calculates captured information (for example, information related to the luminance of a plurality of blocks obtained by dividing at least a part of the entire captured image). Based on this information, the surrounding change of the right controller 4 is determined. In addition, the infrared imaging unit 123 may perform imaging with ambient light, but in the present embodiment, the infrared imaging unit 123 includes an infrared light emitting unit 124 that irradiates infrared rays. For example, the infrared light emitting unit 124 emits infrared rays in synchronization with the timing at which the infrared camera captures an image. Then, the infrared light irradiated by the infrared light emitting unit 124 is reflected by the imaging target, and the reflected infrared light is received by the infrared camera, whereby an infrared image is acquired. Thereby, the infrared imaging unit 123 can obtain a clearer infrared image. The infrared imaging unit 123 and the infrared light emitting unit 124 may be provided in the right controller 4 as separate devices, or may be provided in the right controller 4 as a single device provided in the same package. It may be provided. In this embodiment, an infrared imaging unit 123 having an infrared camera is used. However, in other embodiments, a visible light camera (a camera using a visible light image sensor) is used instead of the infrared camera as an imaging unit. ) May be used.

  The right controller 4 includes a processing unit 121. The processing unit 121 is connected to the communication control unit 111. The processing unit 121 is connected to the NFC communication unit 122, the infrared imaging unit 123, and the infrared light emitting unit 124. The processing unit 121 executes management processing for the NFC communication unit 122 in response to a command from the main device 2. For example, the processing unit 121 controls the operation of the NFC communication unit 122 in accordance with a command from the main device 2. Further, the processing unit 121 controls the activation of the NFC communication unit 122, and controls the operation (specifically, reading and writing) of the NFC communication unit 122 with respect to a communication target (for example, an NFC tag). In addition, the processing unit 121 receives information to be transmitted to the communication target via the communication control unit 111 from the main body device 2 and passes the information to the NFC communication unit 122, or receives information received from the communication target to the NFC communication unit 122. Or transmitted to the main device 2 via the communication control unit 111.

  The processing unit 121 includes a CPU, a memory, and the like, and performs a predetermined program (for example, image processing and various calculations) stored in a storage device (not illustrated) provided in the right controller 4 (for example, a nonvolatile memory). Management processing for the infrared imaging unit 123 is executed in accordance with a command from the main body device 2. For example, the processing unit 121 causes the infrared imaging unit 123 to perform an imaging operation, and acquires and / or calculates information based on the imaging result (information on the captured image or information calculated from the information). Or transmitted to the main device 2 via the communication control unit 111. In addition, the processing unit 121 executes management processing for the infrared light emitting unit 124 in accordance with a command from the main device 2. For example, the processing unit 121 controls the light emission of the infrared light emitting unit 124 in accordance with a command from the main device 2. Note that the memory used when the processing unit 121 performs processing may be provided in the processing unit 121 or the memory 112.

  The right controller 4 includes a power supply unit 118. The power supply unit 118 has the same function as the power supply unit 108 of the left controller 3 and operates in the same manner.

[2. Overview of attachment and game processing using it]
In the present embodiment, the game system 1 includes multiple types (here, three types) of additional operation devices as peripheral devices used together with the main body device 2 and the controllers 3 and 4 described above. The additional operation device is a peripheral device for mounting and using the controller in the additional operation device. The controller (here, the right controller 4) is detachably attached to the additional operation device. Although details will be described later, in this embodiment, the user (in other words, the player) attaches the controller to the additional operation device, and performs a game operation using the additional operation device to which the controller is attached. As described above, in this embodiment, a game operation is performed using the controller as the main device, the additional operation device as the sub device, and the sub device equipped with the main device.

  In the present embodiment, the game system 1 includes three types of additional operation devices (specifically, first to third additional operation devices 6 to 8 shown in FIG. 8). In other embodiments, the number of additional operation devices included in the game system 1 may be an arbitrary number of two or more.

  In the present embodiment, the user plays a game using a plurality of types (three types here) of additional operation devices while one game application is being executed. Note that although the content of the game executed by the game application is arbitrary, in the present embodiment, the player character operated by the user can drive various vehicles and move in the game space. The user can change the additional operation device to which the controller is attached during the execution of the game application (more specifically, during the game play). Further, the game system 1 executes different game processes depending on the type of the additional operation device to which the controller is attached. Although details will be described later, in the present embodiment, the vehicle object driven by the player character changes according to the type of the additional operation device to which the controller is attached. That is, in the present embodiment, an object that is a target operated by the user changes according to the type of the additional operation device to which the controller is attached.

  FIG. 8 is a diagram illustrating a relationship between a combination of a controller and an additional operation device and an object to be operated. As shown in FIG. 8, in the present embodiment, the right controller 4 can be attached to any one of the first to third additional operation devices 6 to 8.

  In the present embodiment, the right controller 4 is used as a controller to be attached to the additional operation device. This is because in this embodiment, the type of the additional operation device attached to the right controller 4 is determined based on the imaging result of the infrared imaging unit 123 provided in the right controller 4. Details of the method for determining the type of the additional operation device will be described later.

  As shown in FIG. 8, in a state where the right controller 4 is mounted on the first additional operation device 6 (hereinafter referred to as “first mounting state”), the car object 201 driven by the player character is in the game space. Appear. When the right controller 4 is in the first wearing state, the user operates the automobile object 201 using the first additional operation device 6 to which the right controller 4 is attached. That is, when the game system 1 determines that the right controller 4 is attached to the first additional operation device 6, the game system 1 places the player character and the car object 201 in the game space. Then, the game system 1 controls the movement of the automobile object 201 based on an operation on the first additional operation device 6 to which the right controller 4 is attached.

  Also, as shown in FIG. 8, in a state where the right controller 4 is attached to the second additional operation device 7 (hereinafter referred to as “second attachment state”), the airplane object 202 driven by the player character is a game. Appears in space. When the right controller 4 is in the second wearing state, the user operates the airplane object 202 using the second additional operation device 7 to which the right controller 4 is attached. That is, when the game system 1 determines that the right controller 4 is attached to the second additional operation device 7, the game system 1 places the player character and the airplane object 202 in the game space. Then, the game system 1 controls the operation of the airplane object 202 based on an operation on the second additional operation device 7 to which the right controller 4 is attached.

  Further, as shown in FIG. 8, in a state where the right controller 4 is mounted on the third additional operation device 8 (hereinafter referred to as “third mounting state”), the ship object 203 driven by the player character is a game. Appears in space. When the right controller 4 is in the third attachment state, the user operates the ship object 203 using the third additional operation device 8 to which the right controller 4 is attached. That is, when the game system 1 determines that the right controller 4 is attached to the third additional operation device 8, the game system 1 places the player character and the ship object 203 in the game space. Then, the game system 1 controls the operation of the ship object 203 based on an operation on the third additional operation device 8 to which the right controller 4 is attached.

  Although not shown, the player character does not board the vehicle object when the right controller 4 is not attached to any additional operation device (hereinafter referred to as “non-attached state”). When the right controller 4 is not attached, the player operates a player character that is not on the vehicle object. That is, when the game system 1 determines that the right controller 4 is not attached to any additional operation device, the game system 1 places a player character that is not on the vehicle object in the game space. Then, the game system 1 controls the action of the player character based on an operation on the right controller 4. In other embodiments, the game system 1 may not allow the user to perform an operation on the player character (in other words, an operation using the right controller 4) in the non-wearing state.

  In this embodiment, even if the mounting state of the right controller 4 changes, the game system 1 executes a process for progressing the game (in other words, progress of the game) without interruption. That is, the game system 1 allows the game to proceed seamlessly even when the wearing state of the right controller 4 changes. For example, when the user tries to replace the right controller 4 attached to the first additional operation device 6 with the second additional operation device 7, the attachment state of the right controller 4 is changed from the first attachment state to the non-attachment state. Then, it shifts to the second wearing state. In this case, the game system 1 continues the process for advancing the game without interrupting the progress of the game (for example, without pausing the process in response to the non-wearing state). Run. That is, in a period in which the wearing state of the right controller 4 shifts from the first wearing state to the second wearing state through the non-wearing state, the game system 1 continuously accepts an operation by the user and responds to the operation. The game process is continuously executed. As described above, in the present embodiment, the operation of mounting the right controller 4 on the additional operation device by causing the game to proceed even during the period in which the user changes the mounting state of the right controller 4 is part of the game operation. It is possible to make the user recognize. Accordingly, it is possible to provide the user with a new game experience in which the game operation is performed while switching the additional operation device to which the right controller 4 is attached during the game play.

  Note that in another embodiment, the game system 1 may temporarily stop the process for advancing the game in response to the right controller 4 being in a non-wearing state. At this time, the game system 1 may resume processing for advancing the game in response to the right controller 4 being attached to any of the additional operation devices.

  In another embodiment, the game system 1 may confirm the switching of the vehicle object to the user when the vehicle object is switched in response to a change in the mounting state of the right controller 4. For example, in the above case, the game system 1 may display a message such as “Do you want to switch vehicle objects?” And accept an instruction input for switching. At this time, the progress of the game may be paused or may not be paused.

  As described above, in the present embodiment, the user can switch the object to be operated by switching the wearing state of the right controller 4 during the game. Therefore, according to the present embodiment, it is possible to provide the user with a new game experience of playing the game while replacing the additional operation device to which the controller is mounted according to the situation during the game, which is highly interesting. A game can be offered.

  Hereinafter, the configuration of each additional operation device 6 to 8 and the processing executed when the right controller 4 is attached to each additional operation device 6 to 8 will be described for each additional operation device.

[2-1. First additional operation device]
[2-1-1. Configuration of first additional operation device]
With reference to FIGS. 9 to 16, the game process when the first additional operation device 6 and the right controller 4 are mounted on the first additional operation device 6 will be described. FIG. 9 is a diagram illustrating an example of the first additional operation device and the right controller. 10 is a view of the first additional operating device shown in FIG. 9 as viewed from the front side (that is, the Z-axis negative direction side shown in FIG. 9).

  As shown in FIG. 9, the first additional operation device 6 includes a housing 601, a handle 602, a left lever 604, and a right lever 605. The first additional operation device 6 is an operation device capable of operating the handle 602 and the levers 604 and 605. The first additional operation device 6 is an operation device simulating a vehicle handle, and is an operation device for operating the vehicle object 201 using the handle 602 and the levers 604 and 605.

  Here, although the material of each additional operation device 6-8 is arbitrary, in this embodiment, the material of each component (however, except the marker mentioned later) of each additional operation device 6-8 is a cardboard. The additional operation devices 6 to 8 may be provided to the user in a state before assembly, and the user may assemble the additional operation devices 6 to 8. By making the material of each additional operation device 6-8 into corrugated board, an additional operation device can be manufactured at low cost. Further, by letting the user assemble the additional operation device, it is possible to provide the user with the pleasure of assembling the operation device used in the game.

  The handle 602 shown in FIG. 9 is connected to the front surface of the housing 601 (that is, the surface on the Z axis negative direction side). The handle 602 is rotatably connected to the housing 601. Specifically, the handle 602 includes an annular portion having an annular shape and a shaft portion that extends from one end of the annular portion to the other end through the center of the annular portion. The handle 602 is rotatable with respect to the housing 601 with the vicinity of the center of the annular portion as a rotation axis. With the above configuration, the user can perform an operation of rotating the handle 602 (hereinafter referred to as “rotating operation”) (FIG. 10).

  In the present embodiment, an insertion hole 603 is provided at the position of the rotation shaft of the handle 602 in the shaft portion. The insertion hole 603 is provided so as to extend along the rotation axis (in other words, along the direction perpendicular to the rotation surface of the handle). The insertion hole 603 is provided for mounting the right controller 4. That is, as shown in FIG. 9, the right controller 4 is attached to the first additional operation device 6 by inserting the right controller 4 into the insertion hole 603. In the present embodiment, the user inserts the right controller 4 into the insertion hole 603 from the lower side (that is, the side on which the window 68 of the infrared imaging unit 123 is provided), whereby the first controller 4 is moved to the first controller 4. The additional operation device 6 can be correctly attached. Although not shown, a bottom surface on which the right controller 4 inserted into the insertion hole 603 abuts is provided behind the insertion hole 603 in the shaft portion. Therefore, when the right controller 4 inserted into the insertion hole 603 is inserted to a certain depth, the right controller 4 stops by contacting the bottom surface.

  Although not shown, a hole is provided at a position facing the window portion 68 of the right controller 4 attached to the first additional operation device 6 on the bottom surface provided in the back of the insertion hole 603. Therefore, the infrared imaging unit 123 of the right controller 4 attached to the first additional operation device 6 can image the inside of the housing 601 through the hole on the bottom surface.

  Each lever 604 and 605 shown in FIG. 9 is provided so as to protrude from the side surface of the housing 601. Specifically, the left lever 604 is provided on the left upper side of the housing 601 (that is, the X axis negative direction side and the Y axis positive direction side shown in FIG. 9). The right lever 605 is provided on the side surface on the upper right side of the housing 601 (that is, the X axis positive direction side and the Y axis positive direction side shown in FIG. 9). Each lever 604 and 605 is operably connected to the housing 601 in the vertical direction. Hereinafter, the detailed configuration of each lever 604 and 605 will be described.

  FIG. 11 is a diagram illustrating an example of the internal configuration of the housing 601. 11 is a view of the inside of the housing 601 with the front surface of the housing 601 (that is, the surface to which the handle 602 is connected) removed. As shown in FIGS. 9 and 11, a left lever hole 606 is provided on the upper left side surface of the housing 601. A right lever hole 607 is provided on the upper right side surface of the housing 601. Here, each lever 604 and 605 is a rod-shaped member. As shown in FIG. 11, one end of the left lever 604 is rotatably connected to the housing 601 inside the housing 601, and the other end of the left lever 604 is connected via a left lever hole 606. , Provided so as to protrude outside the housing 601. Also, as shown in FIG. 11, one end of the right lever 605 is rotatably connected to the housing 601 inside the housing 601, and the other end of the right lever 605 has a right lever hole 607. And is provided so as to protrude outside the housing 601. The direction of the rotation axis of each lever 604 and 605 is substantially the same as the direction of the rotation axis of the handle 602 (in other words, parallel).

  The left lever hole 606 is provided in such a size that the left lever 604 can rotate (in other words, the end of the left lever 604 protruding to the outside of the housing 601 can move in the vertical direction). It is done. The right lever hole 607 is sized so that the right lever 605 can rotate (in other words, the end of the right lever 605 protruding outside the housing 601 can move in the vertical direction). Is provided. Further, each lever 604 and 605 is urged by an urging member (not shown) in a direction in which an end protruding outside the housing 601 moves upward. Accordingly, when the levers 604 and 605 are not operated by the user, the levers 604 and 605 are brought into contact with the upper edges of the lever holes 606 and 607 and stopped as shown in FIGS. With the configuration described above, the user can perform an operation of lowering the levers 604 and 605 (hereinafter referred to as “lowering operation”) (FIG. 10).

[2-1-2. Processing based on marker]
In the present embodiment, a marker that can be imaged by the infrared imaging unit 123 of the right controller 4 is provided in each of the additional operation devices 6 to 8. In the present embodiment, the game system 1 images a marker by the infrared imaging unit 123 of the right controller 4 attached to the additional operation device, and detects the marker from the captured image. Then, the game system 1 determines the type of the additional operation device based on the marker detection result, or performs an operation performed on the additional operation device (specifically, in the case of the first additional operation device 6, Rotating operation and lowering operation). Details of the discrimination process using the marker will be described below.

  As illustrated in FIG. 11, the housing 601 has a position where the infrared imaging unit 123 of the right controller 4 attached to the first additional operation device 6 can capture an image (in other words, the infrared imaging unit 123. The imaging surface 611 is provided within the imaging range. A first basic marker 612 is provided on the imaging surface 611. In the present specification, a marker used to determine the type of the additional operation device to which the right controller 4 is attached is referred to as a “basic marker”. The first basic marker 612 is a basic marker provided in the first additional operation device 6 and is used to determine that the right controller 4 is attached to the first additional operation device 6.

  Here, the marker is made of a material (and / or color) that the infrared imaging unit 123 can distinguish from other parts (that is, parts other than the marker) on the inner wall surface of the casing of the additional operation device. Is done. The marker may be a figure drawn on the inner surface of the casing, a picture, a combination of a figure and a picture, or a component attached to the inner side of the casing. The marker may be made of a material having, for example, retroreflective characteristics. According to this, since the infrared light from the infrared light emitting unit 124 is reflected more toward the infrared imaging unit 123, the infrared imaging unit 123 can more easily recognize the marker. Further, it is easy to recognize even a marker at a position away from the infrared imaging unit 123. Moreover, you may take the structure which provides the infrared light emission part 124 in a housing | casing as a marker instead of a retroreflection material.

  In the present embodiment, the first basic marker 612 has a shape composed of four square portions (FIG. 11). Although details will be described later, the basic markers provided in the additional operation devices 6 to 8 have different shapes. In the example illustrated in FIG. 11, the first basic marker 612 has a shape in which one of the four square portions forming the first basic marker 612 is larger than the other three squares. Therefore, when the game system 1 detects the basic marker in the image captured by the infrared imaging unit 123 of the right controller 4, the game system 1 determines the type of the additional operation device to which the right controller 4 is attached based on the detected shape of the basic marker. Can be determined. For example, the game system 1 can determine that the right controller 4 is attached to the first additional operation device 6 when the first basic marker 612 is detected in the captured image. In the present specification, the “marker shape” is a concept including the size of the marker. That is, it can be said that two markers having different sizes have different shapes.

  In the present embodiment, the first basic marker 612 is used to detect a rotation operation on the handle 602. Hereinafter, a method for detecting a rotation operation will be described with reference to FIGS.

  FIG. 12 is a diagram illustrating an example of a captured image captured by the infrared imaging unit 123 when the handle 602 of the first additional operation device 6 is in a reference state (specifically, a state where the shaft portion is horizontal). It is. FIG. 13 is a diagram illustrating an example of a captured image captured by the infrared imaging unit 123 when the handle 602 of the first additional operation device 6 is rotated from the reference state. The first basic marker image 622 shown in FIGS. 12 and 13 is an image showing the first basic marker 612 in the captured image.

  When the handle 602 is rotated by the rotation operation, the right controller 4 attached to the handle 602 is also rotated. Therefore, the direction of the first basic marker image 622 included in the captured image changes according to the rotation angle of the handle 602. Therefore, the game system 1 can calculate the rotation angle of the handle 602 based on the orientation of the first basic marker image 622 in the captured image, and can detect the rotation operation based on the rotation angle. Specifically, the game system 1 detects whether or not a rotation operation has been performed and / or the amount of rotation by which the rotation operation has been performed. For example, in the example shown in FIGS. 12 and 13, the first basic marker image 622 shown in FIG. 13 is arranged in a direction rotated 90 ° to the left with respect to the first basic marker image 622 shown in FIG. . Therefore, when the captured image shown in FIG. 13 is acquired from the right controller 4, the game system 1 can determine that the handle 602 is rotated 90 ° to the right from the reference state.

  As described above, the basic marker used to determine the type of the additional operation device on which the right controller 4 is mounted may be used to detect an operation on the additional operation device.

  As shown in FIG. 11, the position of the left lever 604 inside the housing 601 is a position where the infrared imaging unit 123 of the right controller 4 attached to the first additional operation device 6 can capture an image. A left lever marker 613 is provided. The left lever marker 613 is used to detect an operation on the left lever 604. Hereinafter, a method for detecting a rotation operation will be described with reference to FIGS. 11, 12, 14, and 15.

  FIG. 14 is a diagram illustrating an example of the first additional operation device 6 in a state where the lowering operation with respect to the left lever 604 is performed. FIG. 15 is a diagram illustrating an example of a captured image captured by the infrared imaging unit 123 in a state where the lowering operation with respect to the left lever 604 is performed. 11 described above is a diagram illustrating an example of the first additional operation device 6 in a state where the lowering operation with respect to the left lever 604 is not performed, and FIG. 12 is captured by the infrared imaging unit 123 in this state. An example of the captured image is shown. Note that the left lever marker image 623 shown in FIGS. 12 and 15 is an image showing the left lever marker 613 in the captured image.

  When the left lever 604 is moved downward by the lowering operation, the position and orientation of the left lever marker 613 viewed from the infrared imaging unit 123 of the right controller 4 attached to the first additional operation device 6 are lowered with respect to the left lever 604. The position changes from the position and orientation of the left lever marker 613 when no operation is performed. Therefore, the position and orientation of the left lever marker image 623 in the captured image change between a state where the lowering operation is not performed on the left lever 604 (FIG. 12) and a state where the lowering operation is performed (FIG. 15). Specifically, the position of the left lever marker image 623 when the lowering operation is performed on the left lever 604 is lower left than the position of the left lever marker image 623 when the lowering operation is not performed on the left lever 604. It is moving toward. Therefore, when the captured image shown in FIG. 15 is acquired from the right controller 4, the game system 1 can determine that the lowering operation for the left lever 604 is being performed.

  In the present embodiment, the right controller 4 attached to the first additional operation device 6 rotates according to the rotation of the handle 602. Therefore, the game system 1 calculates the position and orientation with reference to the first basic marker image 622 as the position and orientation of the left lever marker image 623 in the captured image, and the left lever 604 based on the calculated position and orientation. The lowering operation is detected.

  Further, as shown in FIG. 11, a right lever marker 614 is provided at a position inside the housing 601 in the right lever 605. The right lever marker 614 is used to detect an operation on the right lever 605. The detection of the lowering operation on the right lever 605 can be performed in the same manner as the detection of the lowering operation on the left lever 604. That is, the game system 1 calculates the position and orientation of the right lever marker image 624 in the captured image (specifically, the position and orientation with reference to the first basic marker image 622), and the calculated position and orientation are calculated. Based on this, the lowering operation for the right lever 605 is detected.

[2-1-3. Game processing when right controller 4 is attached to first additional operation device 6]
FIG. 16 is a diagram illustrating an example of a game image displayed when the right controller 4 is attached to the first additional operation device 6. When the right controller 4 is attached to the first additional operation device 6, the game system 1 causes the right controller 4 to use the first additional operation device 6 based on the captured image acquired from the infrared imaging unit 123 of the right controller 4. It is discriminated that it is attached to. At this time, as shown in FIG. 16, the game system 1 displays a game image representing a state in which the player character 200 has boarded the car object 201 on the display device. In the present embodiment, the display device on which the game image is displayed may be the display 12 described above or a stationary monitor.

  When it is determined that the right controller 4 is attached to the first additional operation device 6, the game system 1 detects each operation (specifically, a rotation operation and a lowering operation) on the first additional operation device 6. Then, the operation of the automobile object 201 is controlled in accordance with these operations performed by the user. In addition, the method of controlling the operation | movement of the motor vehicle object 201 according to a user's operation is arbitrary. For example, in the present embodiment, the game system 1 controls the traveling direction of the automobile object 201 in accordance with a rotation operation on the handle 602. In addition, the game system 1 changes the gear of the automobile object 201 according to the lowering operation with respect to the left lever 604, and fires a bullet from the automobile object 201 according to the lowering operation with respect to the right lever 605.

  As described above, the user can play the game in which the car object 201 appears in the game space and the car object 201 is operated by attaching the right controller 4 to the first additional operation device 6.

[2-2. Second additional operation device]
[2-2-1. Configuration of second additional operation device]
With reference to FIGS. 17 to 22, the game process when the second additional operation device 7 and the right controller 4 are mounted on the second additional operation device 7 will be described. FIG. 17 is a diagram illustrating an example of the second additional operation device and the right controller. FIG. 18 is a view of the second additional operating device 7 shown in FIG. 17 as viewed from above (that is, the Y axis positive direction side shown in FIG. 17).

  As shown in FIG. 17, the second additional operation device 7 includes a pedestal portion 701, a grip portion 702, and a button 704. The second additional operation device 7 is an operation device capable of operating the grip portion 702 and the button 704. The second additional operation device 7 is an operation device imitating an airplane control stick, and is an operation apparatus for operating the airplane object 202 using a grip 702 such as a control stick.

  In the second additional operation device 7 shown in FIG. 17, the grip portion 702 is connected to the pedestal portion 701. The grip portion 702 can be tilted with respect to the pedestal portion 701. In FIG. 17, four front, rear, left, and right arrows indicating the tilting direction of the gripping portion 702 are shown. However, the gripping portion 702 is in any direction from a reference state in which the longitudinal direction of the gripping portion 702 is vertical. It is possible to lean on.

  As shown in FIG. 17, the grip portion 702 has an insertion hole 703 on the upper surface. The insertion hole 703 is provided so as to extend along the longitudinal direction of the grip portion 702. The insertion hole 703 is provided for mounting the right controller 4. That is, as shown in FIG. 17, the right controller 4 is attached to the second additional operation device 7 by inserting the right controller 4 into the insertion hole 703. In the present embodiment, the user can correctly attach the right controller 4 to the second additional operation device 7 by inserting the right controller 4 into the insertion hole 703 from below. As described above, the upper portion of the grip portion 702 is formed in a cylindrical shape, and the right controller 4 can be inserted into the cylindrical portion. Moreover, as shown in FIG. 17, a groove is provided on the side surface of the cylindrical portion. When the right controller 4 is inserted into the grip portion 702, the analog stick 52 is inserted along the groove, and the analog stick 52 is brought into contact with the end of the groove, whereby the right controller 4 is inserted into the grip portion 702. Is stopped.

  As shown in FIG. 18, the insertion hole 703 of the grip portion 702 is connected to an internal cavity in the grip portion 702. Therefore, the infrared imaging unit 123 of the right controller 4 inserted into the insertion hole 703 and attached to the grip unit 702 can image the inside of the grip unit 702.

  As shown in FIG. 17, the button 704 is provided on the grip portion 702, specifically, on the side surface of the upper part of the cylinder of the grip portion 702. The button 704 is provided so as to be pressed from above. As shown in FIG. 18, a hole 705 is provided on a side surface on which the button 704 is provided among the side surfaces of the cylindrical portion of the grip portion 702. A part of the button 704 is provided so as to protrude from the hole 705 (FIG. 18). Therefore, the part of the button 704 can be imaged by the infrared imaging unit 123 of the right controller 4 attached to the second additional operation device 7.

  With the above configuration, the user can perform an operation of tilting the grip portion 702 (hereinafter referred to as “tilting operation”) and an operation of pressing the button 704 (hereinafter referred to as “pressing operation”). There is (FIG. 17).

[2-2-2. Processing based on marker]
In the second additional operation device 7, as in the first additional operation device 6, a marker that can be imaged by the infrared imaging unit 123 is provided inside the additional operation device. The game system 1 detects a marker from an image captured by the infrared imaging unit 123 of the right controller 4 attached to the second additional operation device 7. Then, the game system 1 determines that the right controller 4 is attached to the second additional operation device 7 based on the marker detection result, or performs an operation (specifically) performed on the second additional operation device 7. Specifically, a pressing operation) is detected.

  As shown in FIG. 18, the second basic marker 711 is provided on the bottom surface inside the grip portion 702 at a position where the infrared imaging unit 123 of the right controller 4 attached to the second additional operation device 7 can capture images. It is done. The shape of the second basic marker 711 is arbitrary, but the second basic marker 711 has a shape different from that of the first basic marker 612 described above and a third basic marker 811 described later. In the example illustrated in FIG. 18, the second basic marker 711 has a shape in which one square among the four square portions forming the second basic marker 711 is smaller than the other three squares.

  As with the first additional operation device 6, the game system 1 also determines the type of the additional operation device to which the right controller 4 is attached, using the basic marker. That is, when the game system 1 detects the second basic marker 711 in the captured image acquired from the right controller 4, the game system 1 determines that the right controller 4 is attached to the second additional operation device 7.

  As illustrated in FIG. 18, a button marker 712 is provided at a position where the infrared imaging unit 123 of the right controller 4 mounted on the second additional operation device 7 can capture an image. Specifically, the button marker 712 is provided in a portion of the button 704 that protrudes from the hole 705 of the grip portion 702. The button marker 712 is used for detecting a pressing operation on the button 704.

  FIG. 19 is a diagram illustrating an example of the inside of the gripping unit in a state where the button of the gripping unit is pressed. FIG. 18 is a diagram illustrating an example of the inside of the gripping unit in a state where the button of the gripping unit is not pressed. The position of the button marker 712 as viewed from the infrared imaging unit 123 of the right controller 4 attached to the first additional operation device 6 is the state in which the button 704 of the grip unit 702 is not pressed (FIG. 18) and the position of the grip unit 702. This is different from the state where the button 704 is pressed (FIG. 19). Specifically, the position of the button marker 712 when the button 704 is pressed is more rightward than the position of the button marker 712 when the button 704 is not pressed (in other words, closer to the second basic marker 711). Direction). Therefore, the game system 1 performs an operation on the button 704 based on the position of the image indicating the button marker 712 in the captured image captured by the infrared imaging unit 123 of the right controller 4 attached to the first additional operation device 6. Can be detected.

  Further, the tilting operation with respect to the grip portion 702 is detected based on the detection result of the acceleration sensor of the right controller 4. That is, the game system 1 calculates the attitude of the right controller 4 based on the acceleration detected by the acceleration sensor 89 provided in the right controller 4. Then, the game system 1 calculates the direction and amount of tilt of the right controller 4 and the grip portion 702 with respect to the above-described reference state based on the calculated posture. Thereby, the game system 1 can detect a tilting operation with respect to the grip portion 702.

  As described above, the game system 1 may detect an operation on the additional operation device using the marker, or may detect the operation using the operation data from the right controller 4 attached to the additional operation device. Good. The game system 1 may detect an operation on the additional operation device using both the marker and the operation data.

[2-2-3. Game processing when marker is detected]
FIG. 20 is a diagram illustrating an example of a game image displayed when the right controller 4 is attached to the second additional operation device 7. When the right controller 4 is attached to the second additional operation device 7, the game system 1 causes the right controller 4 to use the second additional operation device 7 based on the captured image acquired from the infrared imaging unit 123 of the right controller 4. It is discriminated that it is attached to. At this time, as shown in FIG. 20, the game system 1 displays a game image representing a state in which the player character 200 has boarded the airplane object 202 on the display device.

  When it is determined that the right controller 4 is attached to the second additional operation device 7, the game system 1 detects each operation (specifically, tilting operation and pressing operation) on the second additional operation device 7. The operation of the airplane object 202 is controlled in accordance with these operations performed by the user. Note that a method for controlling the operation of the airplane object 202 in accordance with a user operation is arbitrary. For example, in the present embodiment, the game system 1 controls the traveling direction of the airplane object 202 according to the tilting operation with respect to the grip portion 702. In addition, the game system 1 fires a bullet from the airplane object 202 in response to a pressing operation on the button 704.

  As described above, by attaching the right controller 4 to the second additional operation device 7, the user can cause the airplane object 202 to appear in the game space and play a game for operating the airplane object 202.

[2-2-4. Processing when transitioning from the second wearing state to the non-wearing state]
As described above, when the right controller 4 is not mounted on any of the additional operation devices during the game by the game application, the player character 200 is placed in the game space without being boarded on the vehicle object. Note that the game system 1 determines that the right controller 4 is not attached to any additional operation device when no basic marker is detected from the captured image acquired from the right controller 4.

  Here, in the present embodiment, when the wearing state of the right controller 4 changes from the second wearing state to the non-wearing state, the player character 200 is equipped with a parachute from the state of boarding the airplane object 202. To change.

  FIG. 21 is a diagram illustrating an example of a game image displayed when the second wearing state is changed to the non-wearing state. When the wearing state related to the right controller 4 changes from the second wearing state to the non-wearing state, the player character 200 is placed in the air without boarding the airplane object 202. Therefore, as shown in FIG. It becomes equipped and slowly falls to the ground. After landing on the ground, the player character 200 moves on the ground in response to an operation on the right controller 4.

  In a state where the player character 200 is equipped with a parachute, the user can control the movement of the player character 200 by operating the right controller 4. Specifically, the game system 1 controls the falling direction of the player character 200 so as to drop in a direction according to the posture of the right controller 4. For example, in response to an operation of tilting the right controller 4 forward from the reference state, the player character 200 falls while moving forward in the game space, and in response to an operation of tilting the right controller 4 to the right from the reference state, the player character 200. Falls while moving to the right in the game space. The reference state of the right controller 4 in the non-mounted state is a state in which the upward direction of the right controller 4 (that is, the y-axis positive direction) is vertically upward. Note that the posture of the right controller 4 in the reference state in the non-wearing state is substantially the same as the posture of the right controller 4 when the grip portion 702 is in the reference state in the second wearing state.

  As described above, in the present embodiment, when the player character 200 is equipped with a parachute, the player character 200 is operated by tilting the right controller 4 in the same manner as when the player character 200 is riding on the airplane object 202. 200 is controlled.

  Here, in order to detect a game operation (that is, an operation of tilting the right controller 4) in a state where the player character 200 is equipped with a parachute, the game system 1 moves forward (in other words, the right controller 4). Which side of the four sides is treated as the front side). Note that the process of setting the front direction of the right controller 4 includes the direction of the operation performed by the user on the right controller 4 (here, the direction in which the right controller 4 is tilted) and the operation in that direction. It can be said that this is a process of setting a relationship with the content of the game process executed in response to the above (here, the moving direction in the process of moving the player character). In the present embodiment, the game system 1 sets the front direction of the right controller 4 by the following method.

  FIG. 22 is a diagram illustrating an example of a forward setting method of the right controller 4 when the second mounting state is shifted to the non-mounting state. First, as shown in FIG. 22, the right controller 4 can be attached to the second additional operation device 7 in two orientations. That is, as a user's operation mode using the right controller 4 and the second additional operation device 7, two operation modes, a first operation mode and a second operation mode, can be considered. In the first operation mode, the main controller mounting surface of the right controller 4 is directed to the second additional operation device 7 in the first direction facing the front direction of the second additional operation device 7 (that is, the Z-axis positive direction shown in FIG. 17). This is a mode in which the operation is performed in the state of being mounted. Further, the second operation mode is such that the main controller mounting surface of the right controller 4 is in the second direction in which the second additional operation device 7 faces in the rear direction (that is, the Z-axis negative direction shown in FIG. 17). 7 is a state in which the operation is performed in a state where the device is attached to the device 7. The “main body mounting surface” is a surface facing the main body device 2 when the right controller 4 is mounted on the main body device 2, in other words, a surface on the x-axis positive direction side shown in FIG. It is.

  In the present embodiment, the game system 1 is in the direction in which the right controller 4 is mounted on the second additional operation device 7 in the second mounting state (whether the direction is the first direction or the second direction). In other words, whether it is the first operation mode or the second operation mode) is stored. That is, when the game system 1 is operated in the first operation mode in the second wearing state, the game system 1 stores orientation information indicating the first direction, and is operated in the second operation mode in the second wearing state. Is stored, orientation information indicating the second orientation is stored.

  When the game device 1 changes from the second wearing state to the non-wearing state, the game system 1 sets the forward direction of the right controller 4 in the non-wearing state according to the orientation information stored in the second wearing state. Specifically, when the stored orientation information indicates the first orientation, the game system 1 determines the direction in which the main body mounting surface faces (that is, the x-axis positive direction shown in FIG. 5) of the right controller 4. Set as forward. In this case, the game system 1 calculates the posture of the right controller 4 with the direction in which the main body mounting surface faces as the front direction, and detects an operation of tilting the right controller 4 (see FIG. 22). Therefore, in this case, when the right controller 4 is tilted so that the main body mounting surface is tilted forward, the game system 1 detects an operation of tilting the right controller 4 forward. As a result, the player character 200 falls while moving forward in the game space. That is, in the above case, the operation of tilting the right controller 4 so that the main body mounting surface tilts forward is handled as an input for instructing the front direction in both the second mounting state and the non-mounting state.

  On the other hand, when the stored orientation information indicates the second orientation, in a non-wearing state after the second wearing state, the game system 1 is in a direction opposite to the direction in which the main body mounting surface faces (that is, 5 is set as the forward direction of the right controller 4. In this case, the game system 1 calculates the posture of the right controller 4 with the direction opposite to the direction in which the main body mounting surface faces as the front direction, and detects an operation of tilting the right controller 4 (see FIG. 22). Therefore, in this case, when the right controller 4 is tilted so that the main body mounting surface is tilted forward, the game system 1 detects an operation of tilting the right controller 4 backward. As a result, the player character 200 falls while moving backward in the game space. That is, in the above case, the operation of tilting the right controller 4 so that the main body mounting surface tilts forward is handled as an input for instructing the backward direction in both the second mounting state and the non-mounting state.

  As described above, in the present embodiment, when the right controller 4 is operated in the first operation mode in the second mounting state, the right controller 4 with the main body mounting surface side as the front side in the subsequent non-mounting state. The tilting operation is detected. On the other hand, when the right controller 4 is operated in the second operation mode in the second mounting state, the tilting operation of the right controller 4 is detected with the main body mounting surface side as the rear side in the subsequent non-mounting state. The Therefore, according to the present embodiment, the user can perform a game operation without changing the direction of the right controller 4 in the second wearing state and the subsequent non-wearing state. In other words, the user instructs the same direction by tilting the right controller 4 in the same direction in the tilting operation of the right controller 4 in the second wearing state and the tilting operation of the right controller 4 in the non-wearing state. be able to. Thereby, the operability of the right controller 4 when the second mounting state is changed to the non-mounting state can be improved.

  As described above, in the present embodiment, the game system 1 determines that the right controller 4 is attached to a predetermined additional operation device (in this embodiment, the second additional operation device 7). Orientation information relating to the orientation of the right controller with respect to a predetermined additional operation device is stored. Further, the game system 1 executes a game process based on a direction operation by the user with respect to the right controller 4. When the game system 1 changes from the second wearing state to the non-wearing state, the game process 1 is executed in response to the direction specified by the direction operation and the operation in the direction being performed. The relationship with the content is determined based on the orientation information stored in the second wearing state (that is, the front direction of the right controller 4 is set). As described above, the operability of the operation in the non-wearing state is improved by determining the relationship in the non-wearing state in consideration of the direction of the right controller 4 in the wearing state before the non-wearing state. Can do.

  In other embodiments, the direction operation on the right controller 4 in the non-mounted state is not limited to the tilt operation as in the present embodiment, and may be any operation that specifies the direction. For example, in another embodiment, the direction operation on the right controller 4 in the non-wearing state may be an operation on an analog stick. That is, the game system 1 may control the direction in which the player character 200 falls in response to an operation on the analog stick in the non-wearing state. Specifically, when the orientation information stored in the second attachment state indicates the first orientation, the game system 1 performs an operation of tilting the analog stick 52 in the direction in which the main body attachment surface faces in the forward direction. It may be detected as an instruction to instruct, and the player character 200 may be dropped while moving forward. On the other hand, when the orientation information indicates the second orientation, the game system 1 detects an operation of tilting the analog stick 52 in the direction in which the main body mounting surface faces as an operation of instructing the backward direction, and the player character 200 May be dropped while moving backward.

[2-3. Third additional operation device]
[2-3-1. Configuration of third additional operation device]
With reference to FIG. 23 to FIG. 25, the game processing when the third additional operation device 8 and the right controller are attached to the third additional operation device 8 will be described. FIG. 23 is a diagram illustrating an example of the third additional operation device and the right controller.

  As shown in FIG. 23, the third additional operation device 8 includes a housing 801, a left handle 802, and a mounted portion 803. Although not shown, the third additional operation device 8 includes a right handle on a side surface opposite to the left handle 802 (that is, the X positive direction side shown in FIG. 23). The third additional operation device 8 is an operation device capable of operating the left and right handles. In the present embodiment, the third additional operation device 8 is an operation device for operating the ship object 203 using each handle.

  In the third additional operation device 8 shown in FIG. 23, the left handle 802 is connected to the left side surface of the housing 801 (that is, the X-axis negative direction side surface shown in FIG. 23). The left handle 802 is rotatably connected to the housing 801. Specifically, the left handle 802 includes an annular portion having an annular shape and a shaft portion that extends from one end of the annular portion to the other end through the center of the annular portion. The left handle 802 is rotatable with respect to the housing 801 with the vicinity of the center of the annular portion as a rotation axis. With the above configuration, the user can perform an operation of rotating the left handle 802 (hereinafter, referred to as “left rotation operation”) (FIG. 23).

  Although not shown, the right handle is connected to the right side surface of the housing 801 (that is, the surface on the X axis positive direction side shown in FIG. 23). The right handle is rotatably connected to the housing 801. Specifically, the right handle has the same shape as the left handle 802 and can rotate relative to the housing 801 with the vicinity of the center of the annular portion of the right handle as a rotation axis. With the above configuration, the user can perform an operation of rotating the right handle (hereinafter referred to as “right rotation operation”).

  The mounted portion 803 is provided on the upper surface of the housing 801 (that is, the surface on the Y axis positive direction side shown in FIG. 23). The mounted portion 803 has a cylindrical shape, and the right controller 4 can be mounted inside itself. Further, as shown in FIG. 23, a groove is provided on the side surface of the cylindrical portion of the mounted portion 803. When the right controller 4 is inserted into the mounted portion 803, the analog stick 52 is inserted along the groove, and the analog stick 52 comes into contact with the end of the groove, whereby the mounted portion 803 of the right controller 4 is contacted. Insertion for is stopped.

  The lower end of the mounted portion 803 (that is, the end on the Y axis negative direction side shown in FIG. 23) is open. In the housing 801, a hole is provided at a position facing the opening at the lower end of the mounted portion 803. Therefore, the infrared imaging unit 123 of the right controller 4 mounted on the mounted portion 803 of the third additional operation device 8 can image the inside of the housing 801 through the opening at the lower end of the mounted portion 803. It is.

[2-3-2. Processing based on marker]
Similarly to the first and second additional operation devices 6 and 7 in the third additional operation device 8, a marker that can be imaged by the infrared imaging unit 123 is provided inside the additional operation device. The game system 1 detects a marker from an image captured by the infrared imaging unit 123 of the right controller 4 attached to the third additional operation device 8. Then, the game system 1 determines that the right controller 4 is attached to the third additional operation device 8 based on the marker detection result, or performs an operation (specifically) performed on the third additional operation device 8. Specifically, a left rotation operation and a right rotation operation) are detected.

  FIG. 24 is a diagram illustrating an example of the internal configuration of the third additional operation device. FIG. 24 is a view of the inside of the housing 801 with the top surface of the housing 801 (that is, the surface on which the mounted portion 803 is provided) removed.

  As shown in FIG. 24, a third basic marker 811 is provided inside the housing 801 at a position where the infrared imaging unit 123 of the right controller 4 attached to the third additional operation device 8 can capture images. The shape of the third basic marker 811 is arbitrary, but has a shape different from the first basic marker 612 and the second basic marker 711 described above. In the example shown in FIG. 24, the third basic marker 811 has a shape composed of one rectangle and two squares.

  As with the first additional operation device 6 and the second additional operation device 7, the game system 1 also uses the basic marker to determine the type of additional operation device to which the right controller 4 is attached. . That is, when the game system 1 detects the third basic marker 811 in the captured image acquired from the right controller 4, the game system 1 determines that the right controller 4 is attached to the third additional operation device 8.

  As shown in FIG. 24, the third additional operation device 8 includes a first left gear 805 and a second left gear 806 inside the housing 801. The first left gear 805 is connected to the left handle 802, and rotates on the same rotation axis as the left handle 802 according to the rotation of the left handle 802 (see the arrow shown in FIG. 24). The second left gear 806 is provided to be rotatable with respect to the housing 801. The second left gear 806 is arranged to mesh with the first left gear 805, and rotates on a rotation axis perpendicular to the rotation axis of the first left gear 805 in accordance with the rotation of the first left gear 805 (FIG. 24). See the arrow). With the above configuration, the second left gear 806 rotates according to the rotation of the left handle 802.

  As illustrated in FIG. 24, the second left gear 806 is provided with a left handle marker 812 at a position where the infrared imaging unit 123 of the right controller 4 attached to the third additional operation device 8 can capture an image. In the present embodiment, the left handle marker 812 includes a first partial marker provided near the center of the second left gear 806 and a second partial marker provided near the periphery of the second left gear 806. The shape of the left handle marker 812 may be any shape that can identify the rotation of the second left gear 806.

  In the present embodiment, when the second left gear 806 rotates in response to the rotation of the left handle 802, the direction of the left handle marker 812 (in other words, the direction from the first partial marker to the second partial marker) changes. . Therefore, the game system 1 can calculate the rotation angle of the left handle 802 based on the direction of the left handle marker 812, and can detect an operation on the left handle. Therefore, the game system 1 calculates the rotation angle of the left handle 802 based on the orientation of the image indicating the left handle marker 812 in the captured image acquired from the right controller 4 attached to the third additional operation device 8. The left rotation operation can be detected based on the rotation angle. Specifically, the game system 1 detects the rotation angle of the left handle 802.

  As shown in FIG. 24, the third additional operation device 8 includes a first right gear 807 and a second right gear 808 inside the housing 801. The first right gear 807 is connected to the right handle, and rotates on the same rotation axis as the right handle according to the rotation of the right handle (see the arrow shown in FIG. 24). The second right gear 808 is provided so as to be rotatable with respect to the housing 801. The second right gear 808 is arranged to mesh with the first right gear 807, and rotates on a rotation axis perpendicular to the rotation axis of the first right gear 807 in accordance with the rotation of the first right gear 807 (FIG. 24). See the arrow). With the above configuration, the second right gear 808 rotates according to the rotation of the right handle.

  The second right gear 808 is provided with a right handle marker 813 at a position where the infrared imaging unit 123 of the right controller 4 attached to the third additional operation device 8 can capture an image. In the present embodiment, the right handle marker 813 includes a third partial marker provided near the center of the second right gear 808 and a fourth partial marker provided near the circumference of the second right gear 808.

  With the above configuration, the game system 1 can detect the right rotation operation in the same manner as the left rotation operation. That is, the game system 1 can calculate the rotation angle of the right handle based on the orientation of the image indicating the right handle marker 813 in the captured image acquired from the right controller 4 attached to the third additional operation device 8. And right-handed rotation operation can be detected.

[2-3-3. Game processing when marker is detected]
FIG. 25 is a diagram illustrating an example of a game image displayed when the right controller 4 is attached to the third additional operation device 8. When the right controller 4 is attached to the third additional operation device 8, the game system 1 causes the right controller 4 to use the third additional operation device 8 based on the captured image acquired from the infrared imaging unit 123 of the right controller 4. It is discriminated that it is attached to. At this time, as shown in FIG. 25, the game system 1 displays a game image representing a state in which the player character 200 has boarded the ship object 203 on the display device.

  When it is determined that the right controller 4 is attached to the third additional operation device 8, the game system 1 performs each operation on the third additional operation device 8 (specifically, left rotation operation and right rotation operation). And the operation of the ship object 203 is controlled in accordance with these operations performed by the user. A method for controlling the operation of the ship object 203 according to the user's operation is arbitrary. For example, in the present embodiment, the game system 1 controls the direction of the left screw of the ship object 203 in accordance with the left rotation operation with respect to the left handle 802. Further, the game system 1 controls the direction of the right screw of the ship object 203 according to the right rotation operation with respect to the right handle. The game system 1 controls the traveling direction of the ship object 203 based on the direction of the left screw and the direction of the right screw.

  As described above, the user can play the game in which the ship object 203 appears in the game space and the ship object 203 is operated by attaching the right controller 4 to the third additional operation device 8.

[2-4. Summary of processing based on markers]
As described above, in the present embodiment, the game system 1 determines the type of the additional operation device to which the right controller 4 is attached based on the captured image captured by the infrared imaging unit 123. According to the present embodiment, since the game system 1 performs determination based on an image captured by the imaging device, for example, it is possible to determine the type of the additional operation device without adding an electrical configuration to the additional operation device. Can do. Thereby, the configuration of the additional operation device can be simplified.

  Here, in the present embodiment, each of the additional operation devices 6 to 8 includes a marker as an example of a detected portion that can be imaged by the infrared imaging unit 123 of the right controller 4 attached to the additional operation device. . The game system 1 has a shape of a first detected portion image (an image showing a basic marker in the present embodiment) showing at least a part of the detected portion (a basic marker in the present embodiment) included in the captured image. Based on the above, the type of the additional operation device to which the right controller 4 is attached (that is, one of the first to third additional devices 6 to 8) is determined. As described above, the game system 1 can accurately determine the type of the additional operation device based on the captured image obtained by imaging the detected portion provided in the additional operation device.

  In the present embodiment, the additional operation device includes a plurality of detected units (that is, markers). However, in other embodiments, the additional operation device may include one detected unit. For example, the additional operation device may include only the basic marker.

  In the embodiment, the type of the additional operation device is determined based on the shape of the image indicating the basic marker in the captured image. Here, the basic marker is not limited to the shape of the basic marker, and a certain basic marker is also determined by the position (specifically, the position in the captured image) or the direction (specifically, the direction on the captured image) of the captured basic marker. It is possible to distinguish from other basic markers. Therefore, in another embodiment, the game system 1 may determine the type of the additional operation device based on at least one of the position, shape, and orientation of the image indicating the basic marker. Therefore, the basic marker provided in the additional operation device is compared with (a) the position of the right controller 4 attached to the additional operation device with respect to the infrared imaging unit 123, compared with the basic marker provided in another type of additional operation device, (B) The direction with respect to the infrared imaging unit 123 and (c) at least one of the shapes may be different.

  Moreover, in this embodiment, each additional operation apparatus 6-8 is provided with the movable part which moves according to operation by a user, respectively. Specifically, the first additional operation device 6 includes a handle 602 and levers 604 and 605 as a movable portion, and the second additional operation device 7 includes a button 704 as a movable portion, and a third additional operation device. 8 includes a second left gear 806 and a second right gear 808 as movable parts. Note that the movable part may be one that the user moves directly (that is, the user touches and moves the movable part) like the handle 602. Further, the movable part moves indirectly by the user's operation like the second left gear 806 (that is, the movable part moves when the user moves an operation part (for example, the left handle 802) different from the movable part). ).

  In this embodiment, the relative position between the right controller 4 mounted on the additional operation device including the movable portion and the detected portion of the additional operation device by moving the movable portion according to a user operation. And / or relative posture changes. The game system 1 shows at least a part of the detected part included in the captured image, and the first detected part image and / or the second detected part image (the second detected part image is the first detected part). Based on at least one of the position and orientation (different from the image) (may be based on the shape of the first detected portion image and / or the second detected portion image as described later) An operation is detected, and a game process based on the detected user operation is executed. Here, in the first additional operation device 6, the “second detected portion image” is the first basic marker image 622, the left lever marker image 623, and the right lever marker image 624, and the second additional operation. In the device 7, the image shows the button marker 712, and in the third additional operation device 8, the image shows the left handle marker 812 and the image showing the right handle marker 813. As described above, the second detected portion image may be a basic marker image, or may be a marker image different from the basic marker image. As described above, based on the position and / or orientation of the marker in the captured image, the user's operation can be detected with high accuracy. For example, the game system 1 can also determine the type of the additional operation device and detect the user's operation based on one captured image, so that the processing efficiency can be improved.

  Note that, in the present embodiment, the game system 1 is operated by the user based on the marker position (for example, the position of the left lever marker image 623) or the direction (for example, the direction of the first basic marker image 622) in the captured image. Was detected. Here, in other embodiments, the game system 1 may detect a user operation based on the shape of the marker in addition to (or instead of) the position and / or orientation of the marker. For example, when the left lever 604 is rotatable about the longitudinal direction, markers having different shapes may be provided on the four side surfaces of the left lever 604 that is a quadrangular prism. At this time, the game system 1 can determine which of the side surfaces of the left lever 604 faces the infrared imaging unit 123 of the right controller 4 based on the shape of the marker in the captured image. Thereby, the game system 1 can detect the user's operation of rotating the left lever 604 about the longitudinal direction of the left lever 604.

  Note that the relative position and / or relative posture between the right controller 4 and the detected part are the lever markers 613 and 614 in the first additional operation device 6, the button markers 712 in the second additional operation device 7, and the third addition. It may be changed by moving a movable part provided with a detected part like the second left gear 806 and the second right gear 808 in the operating device 8. Further, the relative position and / or the relative posture may be changed by moving a movable part to which the right controller 4 is attached, like the handle 602 in the first additional operation device 6. Further, the relative position and / or the relative posture changes when both the movable part provided with the detected part and the movable part attached with the right controller 4 move like the first additional operation device 6. May be. According to the above, it is possible to detect the change in the relative position and / or the relative posture by detecting the detected part, and it is possible to accurately detect the user operation.

  In the present embodiment, the game system 1 has no right controller 4 attached to any of the additional operation devices 6 to 8 in addition to the type of additional operation device to which the right controller 4 is attached. It is further determined. Specifically, the game system 1 repeatedly acquires images captured by the infrared imaging unit 123 from the right controller 4 during execution of the game application. If no basic marker is detected from the captured image, the game system 1 determines that the right controller 4 is not attached to any additional operation device. According to this, the game system 1 can determine whether or not the right controller 4 is attached to the additional operation device, in addition to determining the type of the additional operation device to which the right controller 4 is attached. Therefore, the mounting state of the right controller 4 can be determined in more detail.

[2-5. Summary of game processing according to wearing state]
As described above, in the present embodiment, a game process corresponding to switching the type of the additional operation device to which the right controller 4 is attached is executed. Specifically, the game system 1 executes the following game process with different process contents depending on the type of the additional operation device.

  In the present embodiment, the game system 1 generates different game images according to the type of the additional operation device that is determined to be attached with the right controller 4 (FIGS. 16, 20, and 25). Specifically, the game system 1 converts the game image showing the vehicle object controlled based on the game operation using the right controller 4 and the additional operation device attached thereto to the determined type of the additional operation device. Change accordingly. According to the above, the user can change the object operated by the additional operation device by attaching the right controller 4 to the additional operation device. Therefore, according to the present embodiment, the game system 1 can provide the user with a new operation feeling that the object to be operated is changed by changing the additional operation device on which the controller is mounted.

  In other embodiments, the content of the game image that changes depending on the type of the additional operation device is arbitrary. In the present embodiment, the object to be operated by the user changes depending on the type of the additional operation device. However, in other embodiments, the object that is not the user's operation target (for example, the clothes of the player character and / or Or the item etc. which it has) may change according to the kind of additional operation apparatus. In addition, the game system 1 may change the whole object (for example, the player character may be switched) or a part of the object (for example, the clothes of the player character, And / or the items held etc.) may be changed. The change of the object may be a change of only the shape of the object, a change of only the color, or a change of the shape and color. Further, for example, the game image may represent a two-dimensional game world, and the game system 1 may change the image representing the two-dimensional object according to the type of the additional operation device.

  Further, in another embodiment, the game system 1 generates a game sound (or in place of the game image) together with the game image (or instead of the game image) according to the type of the additional operation device that is determined to be equipped with the right controller 4. For example, you may make it change BGM output during a game.

  In the present embodiment, when it is determined that the right controller 4 is not attached to any of the additional operation devices, the game system 1 is attached to any of the additional operation devices. A game process different from that performed is executed. More specifically, the game system 1 generates different game images in the above two cases (FIGS. 20 and 21). According to this, in addition to changing the type of the additional operation device on which the right controller 4 is mounted, the user can also change the display of the game depending on whether the right controller 4 is mounted on the additional operation device. it can.

  Further, in the present embodiment, the game system 1 determines the position and / or posture of the virtual camera arranged in the virtual space based on the type of the additional operation device that is determined to have the right controller 4 attached. Control. In the present embodiment, the position and orientation of the virtual camera change according to the type of the additional operation device to which the right controller 4 is attached. Specifically, in this embodiment, the virtual camera is arranged behind the user's operation target in a posture facing the operation target (FIGS. 16, 20, and 25). Here, when the right controller 4 is attached to the first additional operation device 6, the position and orientation of the virtual camera are set so that the vehicle object 201 to be operated is displayed slightly below the center of the screen. (FIG. 16). When the right controller 4 is attached to the third additional operation device 8 or when the right controller 4 is not attached to any additional operation device, the right controller 4 is attached to the first additional operation device 6. The virtual camera is set as in the case of On the other hand, when the right controller 4 is attached to the second additional operation device 7, the position and orientation of the virtual camera are set so that the airplane object 202 that is the operation target is displayed in the center of the screen. (FIG. 20). Thus, by setting the virtual camera according to the type of the additional operation device to which the right controller 4 is attached, the game system 1 provides the user with a game image viewed from an appropriate viewpoint according to the game situation. be able to.

  Moreover, in this embodiment, the game system 1 changes the input used for a game process according to the kind of additional operation apparatus determined that the right controller 4 is mounted | worn. That is, in the present embodiment, the right controller 4 includes a plurality of types of operation detection units for detecting a user operation on the right controller 4. Specifically, the right controller 4 includes buttons 113, an analog stick 52, an acceleration sensor 114, and an angular velocity sensor 115 as operation detection units. Here, when it is determined that the right controller 4 is attached to a predetermined type of additional operation device (in this embodiment, the second additional operation device 7), the game system 1 determines a plurality of types of operation detection units. Among them, the game processing is executed based on the detection result by the operation detection unit (acceleration sensor 114 in the present embodiment) corresponding to the determined type of the sub device. Therefore, according to the present embodiment, the game system 1 can execute the game process based on the detection result by the appropriate operation detection unit according to the type of the additional operation device to which the right controller 4 is attached. For example, when the right controller 4 is attached to the second additional operation device 7, the game system 1 can detect the tilting operation on the grip portion 702 based on the detection result by the acceleration sensor 114, and the acceleration The game process can be executed using the detection result by the sensor 114 (in other words, the content of the tilting operation calculated based on the detection result) as a user input.

  In other embodiments, the game system 1 sets the content of the game process executed in accordance with the detection result by the operation detection unit to the type of the additional operation device that is determined to be equipped with the right controller 4. It may be changed accordingly. For example, the game system 1 may change the game operation detected based on the detection result by the operation detection unit according to the type of the additional operation device, or the game detected based on the detection result by the operation detection unit You may change the game process performed according to operation according to the kind of additional operation apparatus.

  Specifically, in another embodiment, the game system 1 may detect a rotation operation on the handle 602 based on a detection result by the acceleration sensor 114 when the right controller 4 is in the first wearing state. Good. At this time, the game system 1 calculates the posture of the right controller 4 related to the rotation about the axis parallel to the longitudinal direction of the right controller 4 based on the detection result by the acceleration sensor 114, and executes the game process based on the posture. To do. On the other hand, in the second wearing state, the game system 1 calculates the posture of the right controller 4 regarding the rotation about the two axes perpendicular to the longitudinal direction of the right controller 4 based on the detection result by the acceleration sensor 114. The game process corresponding to the posture is executed. As described above, when the game system 1 executes the game process according to the posture based on the detection result by the acceleration sensor 114, the posture about the different axis depends on the type of the additional operation device to which the right controller 4 is attached. May be calculated. As a result, the content of the game process executed in accordance with the detection result differs depending on the type of the additional operation device on which the right controller 4 is attached.

  In another embodiment, the operation detection unit used for the game process is not limited to the acceleration sensor 114 but may be each button 113, the analog stick 52, the angular velocity sensor 115, and the like. For example, in the first wearing state, the game system 1 executes a game process for moving the virtual camera in accordance with the detection result by the analog stick 52 (in other words, input to the analog stick 52), and the second wearing state. In, game processing for changing the attitude of the airplane object 202 in accordance with the detection result by the analog stick 52 may be executed. As described above, the operation detected based on the detection result by the operation detection unit may be an operation on the operation unit included in the right controller 4 or an operation of moving the right controller 4.

[2-6. Processing during installation]
As described above, in the present embodiment, the game system 1 detects the mounting state and the game operation of the right controller 4 by imaging the marker of the additional operation device by the infrared imaging unit 123 of the right controller 4. Here, in a state in which the right controller 4 is being attached to the additional operation device, the game system 1 recognizes (in other words, detects) only some of the plurality of markers of the additional operation device. There is a case. In such a case, in this embodiment, the game system 1 detects some game operations and notifies the user that the right controller 4 is being installed. Hereinafter, with reference to FIG. 26, a process when the right controller 4 is being installed will be described.

  FIG. 26 is a diagram illustrating an example of a partial recognition state and a full recognition state. In FIG. 26, the case where the right controller 4 is attached to the first additional operation device 6 will be described as an example. As shown in FIG. 26, in a state where the right controller 4 is inserted halfway into the first additional operating device 6 (the upper column in the left column shown in FIG. 26), the infrared imaging unit 123 Only one basic marker 612 is imaged, and the other markers (that is, the lever markers 613 and 614) cannot be imaged.

  Here, in this embodiment, in each of the additional operation devices 6 to 8, the distance from the insertion axis to the basic marker when the right controller 4 is inserted into the additional operation device is the distance between the insertion axis and the other marker. It is set smaller than the distance up to. That is, the basic marker is arranged at a position near the insertion axis, and the other markers are arranged at a position far from the insertion axis. The “insertion axis” is a straight line that is parallel to the direction in which the right controller 4 is inserted into the additional operation device and passes through the position of the right controller 4. Therefore, in the present embodiment, in the state where the right controller 4 is being installed, the visual field of the infrared imaging unit 123 is determined by the inner wall of the cylindrical portion (that is, the portion into which the right controller 4 is inserted) in the additional operation device. As a result, only the basic marker may be imaged by the infrared imaging unit 123.

  As described above, when only the first basic marker 612 is imaged by the infrared imaging unit 123, the captured image includes the first basic marker image 622, and other marker images (that is, each lever marker image 623). And 624) are not included (the upper column of the middle row shown in FIG. 26). At this time, the basic marker is recognized based on the captured image, and the other markers are not recognized. In the following, a state in which the basic marker is recognized based on the image captured by the infrared imaging unit 123 and the other markers are not recognized is referred to as a “partially recognized state” (upper row shown in FIG. 26). . Further, a state in which both the basic marker and the other markers are recognized based on the image captured by the infrared imaging unit 123 is referred to as an “all recognition state” (lower row shown in FIG. 26).

  Since the first basic marker 612 is detected in the partial recognition state, the game system 1 executes a game process based on the first basic marker 612. Specifically, in the partial recognition state, the game system 1 determines that the right controller 4 is attached to the first additional operation device 6 and causes the car object 201 to appear in the game space. And the game system 1 detects the above-mentioned rotation operation, and controls operation | movement of the motor vehicle object 201 according to rotation operation. Note that in the partially recognized state, the lever markers 613 and 614 are not detected, so the game system 1 cannot detect the lowering operation described above, and controls the movement of the car object 201 in accordance with the lowering operation. Do not do.

  As described above, in the partially recognized state, the user cannot perform some game operations (that is, the lowering operation in the case of the first additional operation device 6). Therefore, the game system 1 notifies the user that some game operations are performed. That is, in the partial recognition state, the game system 1 displays a game image different from the game image displayed in the full recognition state on the display device. In the present embodiment, the game system 1 displays an image representing the appearance of smoke from the car object 201 (upper column in the right column in FIG. 26). Thus, the user can recognize that the right controller 4 cannot be correctly attached, and the game system 1 can prompt the user to correctly attach the right controller 4.

  On the other hand, in the state where the right controller 4 is inserted all the way into the first additional operating device 6 (that is, correctly inserted) (the lower column in the left column shown in FIG. 26), the infrared The imaging unit 123 is in a state in which both the first basic marker 612 and the lever markers 613 and 614 can be imaged. Therefore, in this state, the captured image includes the first basic marker image 622 and other marker images (that is, the lever marker images 623 and 624) (the lower column in the center column shown in FIG. 26), The game system 1 can recognize the first basic marker 612 and the lever markers 613 and 614. Therefore, the game system 1 executes both the game process based on the first basic marker 612 and the game process based on the lever markers 613 and 614. That is, the game system 1 displays a game image including a normal car object 201 (that is, a car object 201 that does not emit smoke) (lower column in the right column shown in FIG. 26). In addition, the game system 1 detects the rotation operation and the lowering operation described above, and controls the operation of the automobile object 201 according to the rotation operation and the lowering operation.

  In FIG. 26, the case where the right controller 4 is attached to the first additional operation device 6 has been described as an example, but the case where the right controller 4 is attached to another additional operation device is also partially recognized. In the game system 1, the game system 1 executes a part of the game processes executed in the entire recognition state.

  Specifically, in the partial recognition state when the right controller 4 is attached to the second additional operation device 7, the game system 1 determines that the right controller 4 is attached to the second additional operation device 7. The airplane object 202 appears in the game space. Then, the game system 1 detects the above-described tilting operation, and controls the operation of the airplane object 202 according to the tilting operation. This is because when the right controller 4 is attached to the second additional operation device 7, the tilting operation is detected without being based on the marker, so that the game system 1 can detect the tilting operation even in a partially recognized state. . In addition, the game system 1 displays an image representing the appearance of smoke from the airplane object 202 on the display device.

  Further, in the partially recognized state when the right controller 4 is attached to the third additional operation device 8, the game system 1 determines that the right controller 4 is attached to the third additional operation device 8, and the game space The ship object 203 appears at. At this time, since there is no detectable user operation, the game system 1 does not control the operation of the ship object 203 in accordance with the user operation. Thus, the game system 1 does not need to detect a game operation in a partially recognized state with respect to all types of additional operation devices. In addition, the game system 1 displays an image representing the appearance of smoke from the ship object 203 on the display device.

  As described above, in the present embodiment, the detected portion (that is, the marker) included in the additional operation device includes the first portion (that is, the basic marker) and the second portion (that is, other markers other than the basic marker). ). The game system 1 recognizes the first part included in the captured image and recognizes the second part included in the captured image. The game system 1 determines the type of the additional operation device to which the right controller 4 is attached based on at least one of the position, shape, and orientation of the first partial image indicating the first part. In addition, the game system 1 recognizes the first part as a mounted state of the right controller mounted on the additional operation device, the partial recognition state where the second part is not recognized, and the first part and the second part. Discriminating all recognition states. In other words, based on the recognition result of the first part (that is, whether or not the first part is recognized) and the recognition result of the second part, the game system 1 determines that the right controller 4 is Determine the degree of wearing. In addition, the “degree to which the right controller 4 is attached to the additional operation device” is, in other words, the degree indicating how much the right controller 4 is attached to the additional operation device. In the present embodiment, whether the right controller 4 is in the middle of being attached to the additional operation device or whether the right controller 4 is fully attached to the additional operation device is shown. Based on the above, the game system 1 can determine whether the right controller 4 is correctly attached to the additional operation device or whether the right controller 4 is being attached.

  Further, when the game system 1 determines that the wearing state of the right controller 4 is a partially recognized state, the game system 1 performs a first game process (that is, a process of displaying an image representing a state in which smoke is emitted from the vehicle object). And when it is determined that the right controller 4 is in the fully recognized state, a second game process different from the first game process (that is, an image representing a vehicle object that does not emit smoke) is displayed. Process). In other words, the game system 1 executes a game process according to the degree to which the right controller 4 is attached to the additional operation device. According to this, the user can easily recognize whether or not the right controller 4 is correctly attached to the additional operation device.

  Note that, in the present embodiment, the game system 1 performs a first type of user operation (for example, a rotation operation on the first additional operation device 6) based on at least one of the position and orientation of the first partial image. ) And a game process based on the detected first type operation is executed. In addition, the game system 1 detects and detects a second type of user operation (for example, a lowering operation on the first additional operation device 6) based on at least one of the position and orientation of the second partial image. A game process based on the second type of operation is performed. Accordingly, the user can perform some game operations even when the right controller 4 is being attached to the additional operation device. Therefore, in this embodiment, the game system 1 can improve the operability of the game operation using the additional operation device.

  As described above, in the present embodiment, the additional operation device is formed of cardboard and is assembled by the user. Therefore, the user may not be able to assemble the additional operation device well, and at this time, the user may not be able to smoothly attach (in other words, insert) the right controller 4 to the additional operation device. . In this regard, according to the present embodiment, even if the user cannot smoothly attach the right controller 4 to the additional operation device and is partially recognized, the game system 1 By notifying the user, the user can be prompted to correctly attach the right controller 4.

  In the present embodiment, for example, even if a part of the operation unit of the additional operation device (for example, the levers 604 and 605 in the first additional operation device 6) is not damaged, the game system 1 An operation on an operation unit other than the some operation units can be detected.

[3. Specific example of processing in game system]
Next, a specific example of information processing in the game system 1 will be described with reference to FIGS.

[3-1. Data used for information processing]
FIG. 27 is a diagram illustrating an example of various data used for information processing in the game system 1. Various types of data shown in FIG. 27 are stored in a storage medium (for example, a storage medium mounted in the slot 23, the flash memory 84, and / or the DRAM 85) accessible by the main device 2.

  As shown in FIG. 27, the main device 2 stores a game program 301. The game program 301 is a game application program for executing the game in the present embodiment, and is stored in, for example, a storage medium attached to the slot 23 or the flash memory 84.

  As shown in FIG. 27, the main device 2 stores operation data 302, wearing state data 303, recognition state data 304, orientation data 305, forward data 306, and object data 307. These data (in other words, information) 302 to 307 are acquired or generated and used in information processing (FIGS. 28 and 29) described later.

  The operation data 302 is operation data received from the controller. In the present embodiment, the operation data 302 includes data of an image captured by the infrared imaging unit 123. In other embodiments, the operation data 302 includes information obtained by performing some processing on the captured image and / or information obtained from the captured image (for example, information indicating the position and orientation of the marker). May be included. When main device 2 receives operation data from a plurality of controllers, main device 2 stores operation data 302 for each controller.

  The wearing state data 303 indicates the wearing state of the right controller 4. Specifically, the wearing state data 303 indicates one of the first to third wearing states and the non-wearing state.

  The recognition state data 304 indicates a marker recognition state in the game system 1. Specifically, the recognition state data 304 indicates the above-described partial recognition state or full recognition state.

  The direction data 305 indicates the above-described direction information, and specifically indicates the first direction or the second direction.

  The forward data 306 indicates the forward direction of the right controller 4 described above set in the non-wearing state. In the present embodiment, the front direction data 306 indicates a direction in which the main body mounting surface faces or a direction opposite to the direction.

  The object data 307 indicates information related to objects (that is, player characters and vehicle objects) that are operation targets of the user. Specifically, the object data 307 includes data indicating whether or not the player character is driving the vehicle object, and when the player character is driving the vehicle object. The object data 307 includes data of various parameters indicating the state of the player character or vehicle object (for example, position and orientation in the game space).

  FIG. 28 is a flowchart illustrating an example of the flow of information processing executed by the information processing apparatus. Note that the series of processing shown in FIG. 28 is started in response to a predetermined condition being satisfied after the game program 301 is started (for example, an instruction to play the game is given by the user). The

  In the present embodiment, it is assumed that the processor 81 (in other words, the computer) of the main device 2 executes the process of each step shown in FIGS. 28 and 29 by executing the game program 301. However, in another embodiment, a part of the processes in the above steps may be executed by a processor (for example, a dedicated circuit) other than the processor 81. Further, when the main device 2 can communicate with the server, a part of the processing of each step shown in FIGS. 28 and 29 may be executed in the server. In addition, when the controller includes an information processing unit (for example, a processor), a part of the processing of each step illustrated in FIGS. 28 and 29 may be executed in the controller. In addition, the processing of each step shown in FIG. 28 and FIG. 29 is merely an example, and if the same result is obtained, the processing order of each step may be changed. In addition to the processing of each step Another process may be performed (or alternatively).

  Further, the processor 81 of the main device 2 executes the processing of each step shown in FIGS. 28 and 29 using the memory (specifically, the flash memory 84 and / or the DRAM 85). That is, the processor 81 stores the data obtained in each processing step in the memory, and when using the data in the subsequent processing steps, reads the data from the memory and uses it.

  In step S1 shown in FIG. 28, the processor 81 acquires operation data. Since the right controller 4 repeatedly transmits the operation data to the main device 2, the main device 2 sequentially receives the operation data and stores it in the memory as the operation data 302. The operation data 302 may include a predetermined number of operation data in order from the latest. In step S1, the processor 81 reads the operation data 302 from the memory. Following step S1, the process of step S2 is executed.

  In step S2, the processor 81 executes a mounting state determination process. The mounting state determination process is a process for determining a state related to mounting of the right controller 4 on the additional operation device. Hereinafter, with reference to FIG. 29, the details of the wearing state determination processing will be described.

  FIG. 29 is a sub-flowchart showing an example of a detailed flow of the wearing state determination process in step S2 shown in FIG. In the mounting state determination process, first, in step S21, the processor 81 detects a marker image. Specifically, the processor 81 detects a marker image included in the captured image from the captured image data included in the operation data 302 stored in the memory.

  A method for detecting the marker image from the captured image is arbitrary. In the present embodiment, in the captured image, the marker image appears as a region with high luminance. Therefore, the processor 81 extracts a high luminance area from the captured image and detects the extracted area as a marker image area. Here, in a state where the right controller 4 is being attached to the additional operation device, the captured image may include a noise image in addition to the marker image. The noise image is generated when infrared light from the infrared light emitting unit 124 is diffusely reflected on the inner wall of the cylindrical portion of the additional operation device and received by the infrared imaging unit 123. Since such a noise image is larger than the marker image, the processor 81 determines that an area having a predetermined size or larger among the high luminance areas extracted from the captured image is a noise image, and the marker image. As not detected. Note that the processor 81 may determine the marker image and the noise image based on the luminance and / or shape of the high luminance region.

  As described above, in step S21, a marker image is detected in a state where the right controller 4 is attached to the additional operation device. When a plurality of marker images are included in the captured image, the processor 81 detects each marker image (that is, the basic marker image and other marker images). Further, only the basic marker image may be detected in a state where the right controller 4 is in the process of being attached to the additional operation device. On the other hand, when the right controller 4 is not attached to the additional operation device, the marker image is not detected. Following step S21, the process of step S22 is executed.

In step S22, the processor 81 determines whether or not the basic marker is recognized based on the detection result in step S21. In the present embodiment, the processor 81 makes a determination based on the current detection result and the past detection result. Specifically, the processor 81 calculates a determination value L (t) according to the following equation (1).
L (t) = p · Lc + (1−p) · L (t−1) (1)
In the above equation (1), the variable Lc is a numerical value representing the current detection result. Specifically, the variable Lc is 1 when a basic marker is detected, and 0 when it is not detected. The variable L (t−1) is a determination value calculated in the previous process of step S22. The constant p is preset in the range of 0 <p ≦ 1. As the constant p increases, the degree to which the past detection result (that is, L (t−1)) is reflected in the determination value L (t) increases.

  When the calculated determination value L (t) is larger than the predetermined threshold, the processor 81 determines that the basic marker has been recognized. On the other hand, when the calculated determination value L (t) is equal to or less than the predetermined threshold, the processor 81 determines that the basic marker is not recognized. If the determination result of step S22 is negative, the process of step S23 is executed. On the other hand, when the determination result of step S22 is affirmative, the process of step S24 is executed.

  A method for determining whether or not the marker is recognized based on the detection result of the marker image is arbitrary. For example, in another embodiment, the processor 81 may perform the determination based only on the latest detection result in step S21 (in other words, the coefficient p is set to 1). Further, for example, in another embodiment, it may be determined that the basic marker image is recognized when the basic marker image is detected a predetermined number of times in succession in step S21.

  In step S23, the processor 81 determines that the state of the right controller 4 is a non-wearing state. That is, the processor 81 stores the wearing state data 303 indicating the non-wearing state in the memory. After step S23, the processor 81 ends the wearing state determination process.

  On the other hand, in step S24, the processor 81 determines the type of the additional operation device to which the right controller 4 is attached based on the recognized basic marker. That is, as described in “[2-1-2. Processing Based on Marker]”, the first to third basic markers of each of the additional operation devices 6 to 8 have different shapes. By identifying the shape of the recognized basic marker, the type of the additional operation device is determined. In addition, the processor 81 stores mounting state data 303 indicating the determination result (that is, one of the first to third additional operation devices) in the memory. Following step S24, the process of step S25 is executed.

  In step S25, the processor 81 determines whether or not a marker image other than the basic marker image is detected in step S21. For example, when only the basic marker image is detected in step S21 (including the case where the above-described noise image is detected), the processor 81 determines that no other marker image is detected. If the determination result of step S25 is affirmative, the process of step S26 is executed. On the other hand, when the determination result of step S25 is negative, a process of step S28 described later is executed.

  In step S26, the processor 81 determines whether or not the right controller 4 is stationary. Here, in a state where the right controller 4 is being attached to the additional operation device, the right controller 4 is moving, and in this state, the position of the marker image in the captured image changes due to the movement of the right controller 4. The operation may not be detected correctly based on the marker. Therefore, in the present embodiment, when the right controller 4 is stationary, it is determined that the right controller 4 is in a fully recognized state, and in this case, in order to detect an operation based on the marker, step S26. Judgment is made. That is, in this embodiment, when both the determination results in steps S25 and S26 are affirmative, the game system 1 recognizes another marker image.

  In the present embodiment, the determination in step S26 is performed based on the recognized basic marker. That is, the basic marker image detected in the captured image has not changed (for example, the amount of movement from the previously detected basic marker image is less than or equal to a predetermined value, and the amount of change in size is less than or equal to a predetermined value. State) continues for a predetermined period, the processor 81 determines that the right controller 4 is stationary. On the other hand, when the basic marker image detected in the captured image does not change for a predetermined period, the processor 81 determines that the right controller 4 is not stationary. In other embodiments, the determination in step S26 may be performed based on the detection result of the acceleration sensor 114 and / or the angular velocity sensor 115 of the right controller 4 in addition to (or instead of) the basic marker. Good. If the determination result of step S26 is affirmative, the process of step S27 is executed. On the other hand, when the determination result of step S26 is negative, a process of step S28 described later is executed.

  In step S <b> 27, the processor 81 determines that it is in the full recognition state. That is, the processor 81 stores recognition state data 304 indicating all recognition states in the memory. Following step S27, a process of step S29 described later is executed.

  On the other hand, in step S28, the processor 81 determines that the partial recognition state is set. That is, the processor 81 stores recognition state data 304 indicating a partial recognition state in the memory. Following step S28, the process of step S29 is executed.

  In step S29, the processor 81 calculates the position and orientation on the captured image of the marker image used for detecting the operation. The “marker image used for detecting the operation” is the first basic marker image 622 and the marker images 623 and 624 in the first wearing state, and the marker of the button marker 712 in the second wearing state. It is an image, and in the third wearing state, it is each image of each handle marker 812 and 813. The processor 81 stores data indicating the calculated position and orientation in the memory for each marker image. If it is determined that the right controller 4 is in a partially detected state (step S28), the processor 81 does not calculate the position and orientation of the marker image when there is no marker image used for operation detection. After step S29, the processor 81 ends the wearing state determination process. After completing the wearing state determination process, the process of step S3 shown in FIG. 28 is executed.

  In step S3, the processor 81 determines whether or not the mounting state of the right controller 4 has changed. Specifically, the processor 81 performs the determination in step S3 based on whether or not the contents of the mounting state data are updated in step S2. If the determination result of step S3 is affirmative, the process of step S4 is executed. On the other hand, when the determination result of step S3 is negative, a process of step S9 described later is executed.

  In step S4, the processor 81 changes the vehicle object that the player character 200 drives based on the wearing state of the right controller 4. That is, the processor 81 updates the data indicating the vehicle object that the player character 200 drives in the object data 307 stored in the memory. Specifically, when the right controller 4 is in the first wearing state, data indicating the car object 201 is stored in the memory, and when the right controller 4 is in the second wearing state, the plane object 202 is indicated. When the data is stored in the memory and the right controller 4 is in the third mounting state, the data indicating the ship object 203 is stored in the memory, and the process of step S5 is executed after step S4.

  In step S5, the processor 81 determines whether or not the right controller 4 is in the second mounting state. If the determination result of step S5 is affirmative, the process of step S6 is executed. On the other hand, when the determination result of step S5 is negative, the process of step S7 is executed.

  In step S <b> 6, the processor 81 stores orientation information indicating the orientation when the right controller 4 is attached to the second additional operation device 7. Specifically, the processor 81 stores orientation data indicating the first orientation or the second orientation in the memory. The processor 81 can determine the orientation when the right controller 4 is attached to the second additional operation device 7 based on the orientation of the basic marker image detected at the time of attachment. Following step S6, a process of step S9 described later is executed.

  In step S7, the processor 81 determines whether or not the right controller 4 has changed from the second mounting state to the non-mounting state. This determination is made based on whether or not the mounting state data stored in the memory has been updated from the content indicating the second mounting state to the content indicating the non-wearing state. If the determination result of step S7 is affirmative, the process of step S8 is executed. On the other hand, when the determination result of step S7 is negative, the process of step S9 is executed.

  In step S8, the processor 81 sets the front direction of the right controller 4 based on the orientation data stored in the memory. Specifically, the processor 81 sets the forward direction of the right controller 4 by the method described in “[2-2-4. Processing when Transitioning from Second Mounting State to Non-Mounting State]” above. . The processor 81 stores data indicating the set forward direction in the memory as the forward data 306. Following step S8, the process of step S9 is executed.

In step S <b> 9, the processor 81 detects a user operation according to the mounting state and the recognition state of the right controller 4. In the present embodiment, operations detected according to the mounting state and the recognition state of the right controller 4 are as follows.
When in the first wearing state and in the full recognition state: Rotation operation and lowering operation. When in the first wearing state and in the partial recognition state: Rotating operation and in the second wearing state. In the full recognition state: Tilt operation and push-down operation / second wearing state, in partial recognition state: Tilt operation / third wearing state, full recognition state: Left-side rotation operation and right-side rotation operation / third attachment state, partially recognized state: none / non-attachment state: operation on right controller 4 The processor 81 performs the above operation based on operation data Detect each operation. At this time, the processor 81 identifies the above-described operation detection unit as necessary according to the mounting state. In the present embodiment, when the wearing state of the right controller 4 is the second wearing state, the acceleration sensor 114 is specified. Then, the processor 81 detects an operation based on the captured image. When the operation detection unit is specified, the processor 81 detects the operation based on the detection result of the operation detection unit. In addition, the specific detection method of each said operation is "[2-1-2. Process based on marker]", "[2-2-2. Process based on marker]", and "[2-3. -2. Processing based on marker] ”. In the attached state, the detected operation differs depending on the state of the player character 200 (that is, whether the player character is standing on the ground or is equipped with a parachute in the air). Also good. For example, when the player character 200 is standing on the ground, the processor 81 detects an operation on the analog stick 52 of the right controller 4 and when the player character 200 is equipped with a parachute in the air. The processor 81 detects an operation for changing the posture of the right controller 4. Following step S9, the process of step S10 is executed.

  In step S10, the processor 81 controls the operation target (that is, the player character 200 or the vehicle object) according to the wearing state of the right controller 4 according to the detected operation. That is, in the first wearing state, the processor 81 controls the car object 201 according to the rotation operation and the lowering operation. In the second wearing state, the processor 81 controls the airplane object 202 according to the tilting operation and the pressing operation. In the third mounting state, the processor 81 controls the ship object 203 according to the left rotation operation and the right rotation operation. In the non-wearing state, the processor 81 controls the player character 200 in accordance with an operation on the right controller 4. Following step S10, the process of step S11 is executed.

  In step S <b> 11, the processor 81 sets a virtual camera according to the mounting state of the right controller 4. That is, the processor 81 sets the virtual camera so as to have a position and posture according to the wearing state in accordance with the method described in “[2-5. Summary of game processing according to wearing state]”. Following step S11, the process of step S12 is executed.

  In the series of processes in steps 9 to S11, the mounting state and the recognition state of the right controller 4 can be known by referring to the mounting state data 303 and the recognition state data 304 stored in the memory.

  In step S12, the processor 81 generates a game image based on the virtual camera, and displays the generated game image on the display device. That is, the processor 81 displays on the display device a game image showing the game space as viewed from the virtual camera set in step S11 and including the object whose operation is controlled by the processing in step S10. Note that when the recognition state of the right controller 4 is a partial recognition state, the processor 81 displays a game image representing the appearance of smoke from the vehicle object. Following step S12, the process of step S13 is executed.

  In step S13, the processor 81 determines whether or not to end the game. The processor 81 determines to end the game, for example, when a predetermined end condition (for example, the player character has reached the goal) is satisfied, or when an input indicating an end instruction is performed by the user. To do. If the determination result of step S13 is negative, the process of step S1 is executed again. The processor 81 repeatedly executes the processing loop of steps S1 to S13 until it is determined in step S13 that the game is to be ended. On the other hand, if the determination result of step S13 is affirmative, the processor 81 ends the information processing shown in FIG.

[4. Effects and modifications of this embodiment]
According to the embodiment described above, a game program that is an example of an information processing device is a main program that can be attached to and detached from any of a plurality of types of sub-devices (ie, additional operation devices) and a plurality of types of sub-devices. The computer (that is, the processor 81) of the game system 1 including the device (that is, the right controller 4) is caused to function as the following means.
A discriminating means for discriminating the type of the sub device to which the main device is mounted among the plural types of sub devices during execution of the predetermined game application (step S24).
-Game process execution means for executing a game process based on a game operation using the main device and / or a sub-device on which the main device is mounted during execution of a predetermined game application (steps S9 to S12)
The game process execution means executes a game process according to the determined type of the sub device.

  The “computer” is one computer (specifically, the processor 81) in the above-described embodiment, but may be a plurality of computers in another embodiment. The “computer” may be, for example, (a plurality of) computers provided in a plurality of devices. More specifically, the “computer” includes the processor 81 of the main device 2 and information processing included in the controller. It may be constituted by a part.

  The above-mentioned “game operation using the main device and / or the sub device to which the main device is attached” may be an operation on the main device (for example, an operation on a button provided on the right controller 4), The operation may be an operation on the sub device to which the main device is mounted (for example, an operation on an operation unit provided in the sub device), or an operation on both of them (for example, a portion of the sub device in which the main device is mounted) May be an operation of moving.

  According to the above-described embodiment, the game system 1 determines the type of the sub device to which the main device is attached during execution of the game application, and executes the game process according to the type of the sub device. According to this, the user can play the game while switching the sub device to which the main device is mounted, for example, during the game. Thereby, the game system 1 can provide the user with a new game experience of performing a game operation while switching the sub-device during game play, for example, and can provide a highly entertaining game.

  In the above-described embodiment, the game system 1 determines the type of the sub device to which the main device is attached based on the captured image captured by the imaging device (specifically, the infrared imaging unit 123). Here, in other embodiments, the type of the sub device may be determined by a method other than the method using the imaging device. For example, when the main device and the sub device are electrically connected by a connector or a cable, the game system 1 determines the type of the sub device based on a signal received by the main device from the sub device via the connector or cable. It may be determined. Further, for example, when performing short-range wireless communication (for example, communication according to the NFC standard) between the main device and the sub device, the game system 1 causes the sub device to move from the sub device to the main device. The type of the sub device may be determined based on the received signal.

  In the above embodiment, the main device is the right controller 4 that functions as an operating device by itself, but in other embodiments, the main device is a device that does not function as an operating device by itself. May be. For example, the main apparatus may include the infrared imaging unit 123 and may not include an operation detection unit such as a button or an acceleration sensor.

  The “plural types of sub-devices” have differences that are determined to be different types by determination using a predetermined method. Specifically, in the present embodiment, a plurality of additional operation devices (that is, the first to third additional operation devices 6 to 8) having different basic markers are “plural types of sub devices”. In another embodiment, the game system 1 may include a plurality of additional operation devices having the same shape as a plurality of types of sub-devices, each including a plurality of additional operation devices each having a different basic marker. Good. In another embodiment, the game system 1 handles a plurality of additional operation devices having different shapes from each other as a plurality of additional operation devices having the same basic marker as the same type of additional operation device. May be.

  In the embodiment described above, one right controller 4 is used as a controller that is attached to the additional operation device. Here, in another embodiment, a plurality of controllers may be used by being attached to the additional operation device in one game application. For example, in another embodiment, the right controller 4 is attached to the additional operation device and the left controller 3 is attached to the other additional operation device, and these two operation devices are used for game operations in the game application. May be. At this time, since the left controller 3 does not have a function of detecting a marker, the game system 1 may detect a game operation on the left controller 3 based on a detection result by the acceleration sensor and / or the angular velocity sensor. In another embodiment, the right controller 4 and the left controller 3 may be simultaneously attached to one additional operation device.

  In other embodiments, the right controller 4 may not be directly attached to the additional operation device, but may be indirectly attached to the additional operation device. For example, in another embodiment, the right controller 4 may be attached to the holding device, and the holding device to which the right controller 4 is attached may be attached to any of the additional operation devices.

  The content of the game process in the “game process according to the determined type of sub-device” is arbitrary. In the embodiment described above, as the game process, the game system 1 receives a different operation depending on the type of the sub device, and controls the operation of the object according to the operation (steps S9 and S10). A process of setting a virtual camera at a different position and orientation according to the type (step S11) and a process of generating a different game image according to the type of the sub device (step S12) are executed. In another embodiment, the game process is a game parameter (for example, a parameter indicating strength, quickness, physical strength, or the like) set for an object to be operated by the user, The game process may be executed using game parameters set to different values depending on the type. At this time, the game image may be the same regardless of the type of the sub device.

  As described above, the above embodiment can be used for, for example, a game system or a game program in order to improve interest in a game application in which an additional device (that is, a sub device) is used.

DESCRIPTION OF SYMBOLS 1 Game system 2 Main body apparatus 3 Left controller 4 Right controller 6 1st additional operation apparatus 7 2nd additional operation apparatus 8 3rd additional operation apparatus 81 Processor 114 Acceleration sensor 115 Angular velocity sensor 123 Infrared imaging part 200 Player character 201 Automobile object 202 Airplane object 203 Ship object 602 Handle 604 Left lever 605 Right lever 612 First basic marker 613 Left lever marker 614 Right lever marker 702 Grasping part 704 Button 711 Second basic marker 712 Button marker 802 Left handle 812 Left handle marker 813 Right handle marker

Claims (47)

An information processing program that is a game system including a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and that is executed by a computer of the game system that executes a predetermined game application Because
A discriminating means for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
During execution of the predetermined game application, the computer functions as a game process execution means for executing a game process based on a game operation using the main apparatus and / or the sub apparatus to which the main apparatus is attached. Let
The game process execution means executes a game process according to the type of the sub device determined by the determination means,
The main device includes an imaging device,
The determining means determines the type of the sub device on which the main device is mounted based on a captured image captured by the imaging device,
At least one of the plurality of types of sub-devices,
A detected portion that can be imaged by the imaging device of the main device mounted on the sub-device;
A movable part that moves in response to an operation by the user,
When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
The game process execution means performs an operation of the user based on at least one of a position, shape, and orientation of a first detected portion image indicating at least a part of the detected portion included in the captured image. An information processing program that detects and executes a game process based on the detected user operation. Each of the plurality of types of sub-devices includes a detected portion that can be imaged by the imaging device of the main device mounted on the sub-device,
The discriminating unit is mounted with the main device based on at least one of the position, shape, and orientation of the first detected part image and / or the second detected part image included in the captured image. The information processing program according to claim 1 , wherein the information processing apparatus determines a type of the sub device. The detected part is a position in the movable part, and is provided at a position that can be imaged by the imaging device provided in the main device mounted on a sub-device provided with the movable part,
2. The relative position and / or relative attitude between the main device and the detected part changes as the detected part moves in response to the movable part being moved in response to an operation of the user. Or the information processing program of Claim 2 . The main device can be attached to the movable part,
When the movable unit is moved in response to the user's operation, the main device mounted on the movable unit is moved, so that the relative position and / or relative posture between the main device and the detected unit is changed. The information processing program according to claim 1 or 2, which changes.   An information processing program that is a game system including a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and that is executed by a computer of the game system that executes a predetermined game application Because
  A discriminating means for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
  During execution of the predetermined game application, the computer functions as a game process execution means for executing a game process based on a game operation using the main apparatus and / or the sub apparatus to which the main apparatus is attached. Let
  The game process execution means executes a game process according to the type of the sub device determined by the determination means,
  The main device includes an imaging device,
  At least one of the plurality of types of sub-devices,
    A detected portion that can be imaged by the imaging device of the main device mounted on the sub-device;
    A movable part that moves in response to an operation by the user,
  When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
  The game process execution means is based on at least one of the position, shape, and orientation of a first detected part image that indicates at least a part of the detected part included in the captured image captured by the imaging device. , Detecting the user's operation, executing a game process based on the detected user's operation,
  The main device can be attached to the movable part,
  When the movable unit is moved in response to the user's operation, the main device mounted on the movable unit is moved, so that the relative position and / or relative posture between the main device and the detected unit is changed. An information processing program that changes.   The information processing program according to claim 1, wherein the game process execution unit generates a different game image according to the determined type of the sub device. The game process execution means includes
Based on a game operation using the main device and / or the sub device to which the main device is attached, controls an object placed in a virtual space,
The information processing program according to claim 6, wherein the object is changed according to the determined type of the sub device.   The information processing program according to claim 7, wherein the object is an object representing a vehicle on which a game character can board. The main device includes a plurality of types of operation detection units for detecting a user operation on the main device.
When it is determined that the main device is attached to a predetermined type of sub-device, the game processing means, according to the type of the determined sub-device among the plurality of types of the operation detection units. The information processing program according to any one of claims 1 to 8, wherein the game process is executed based on a detection result by the operation detection unit. The main device includes an operation detection unit for detecting a user operation on the main device,
10. The game processing unit according to claim 1, wherein the game processing unit determines a content of a game process to be executed according to a detection result by the operation detection unit based on the determined type of the sub device. Information processing program according to item.   The game processing means controls the position and / or posture of a virtual camera arranged in a virtual space based on the determined type of the sub-device, according to any one of claims 1 to 10. The information processing program described.   The information processing program according to any one of claims 1 to 11, wherein the determination unit further determines that the main device is not attached to any of the plurality of types of sub-devices.   When it is determined that the main device is not attached to any of the plurality of types of sub-devices, the game process execution means is connected to any of the plurality of types of sub-devices. The information processing program according to claim 12, wherein the information processing program executes a game process different from a case where the game process is performed. The game system further includes a storage unit,
The information processing program includes:
When it is determined that the main device is attached to a predetermined type of sub-device, the storage control unit stores device orientation information related to the orientation of the main device with respect to the predetermined type of sub-device in the storage unit. Make the computer work better,
The game process execution means includes
A game process is executed based on a direction operation by the user with respect to the main device,
Non-attached when it is determined that the main device is not attached to any of the plurality of types of sub-devices from the attached state where it is determined that the main device is attached to the predetermined type of sub-devices When the state is changed, the relationship between the direction specified by the direction operation and the content of the game process executed in response to the operation in the direction is stored in the storage unit in the mounted state. The information processing program according to claim 12 or 13, which is determined based on the stored device orientation information. The detected portion included in at least one of the plurality of types of sub-devices includes a first portion and a second portion;
The information processing program includes:
Recognizing the first part based on the captured image, causing the computer to further function as a recognition means for recognizing the second part;
The discrimination means includes
Based on at least one of the position, shape, and orientation of the first partial image indicating the first portion, the type of the sub device to which the main device is attached is determined,
As a state of the main device mounted on the sub-device, the first state in which the first part is recognized and the second part is not recognized, and the second state in which the first part and the second part are recognized The information processing program according to any one of claims 2 to 5, wherein:   The game process execution means executes a first game process when it is determined that the main device is in the first state, and when it is determined that the main device is in the second state, The information processing program according to claim 15, wherein a second game process different from the first game process is executed. The detected portion included in at least one of the plurality of types of sub-devices includes a first portion and a second portion;
The information processing program includes:
Recognizing the first part based on the captured image, causing the computer to further function as a recognition means for recognizing the second part;
The discrimination means includes
Based on at least one of the position, shape, and orientation of the first partial image indicating the first portion, the type of the sub device to which the main device is attached is determined,
Based on the recognition result of the recognition result and the second portion of the first portion, wherein the main device to determine the degree to be attached to the sub-device, any one of the preceding claims 2 Information processing program described in 1.   The information processing program according to claim 17, wherein the game process execution unit executes a game process according to a degree to which the main device is attached to the sub device. At least one of the plurality of types of sub-devices further includes a movable part that moves in response to an operation by a user,
When the movable part is moved according to the user's operation, the relative position between the main device attached to the sub apparatus including the movable part and the detected part of the sub apparatus changes,
The game process execution means includes
The first type operation of the user is detected based on at least one of the position, shape, and orientation of the first partial image showing the first part, and based on the detected first type operation. Run the game process
The second type operation of the user is detected based on at least one of the position, shape, and orientation of the second partial image indicating the second part, and based on the detected second type operation The information processing program according to any one of claims 15 to 18, which executes a game process. An information processing program that is a game system including a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and that is executed by a computer of the game system that executes a predetermined game application Because
A discriminating means for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
During execution of the predetermined game application, the computer functions as a game process execution means for executing a game process based on a game operation using the main apparatus and / or the sub apparatus to which the main apparatus is attached. Let
The information processing program, wherein the game process execution means controls the position and / or orientation of a virtual camera arranged in a virtual space based on the type of the sub device determined by the determination means. The main device includes an imaging device,
21. The information processing program according to claim 20, wherein the determination unit determines a type of a sub device to which the main device is attached based on a captured image captured by the imaging device. Each of the plurality of types of sub-devices includes a detected portion that can be imaged by the imaging device of the main device mounted on the sub-device,
The discriminating means is mounted on the main device based on at least one of the position, shape, and orientation of the first detected portion image showing at least a part of the detected portion included in the captured image. The information processing program according to claim 21, wherein the information processing program determines the type of the sub device. At least one of the plurality of types of sub-devices further includes a movable part that moves in response to an operation by a user,
When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
The game process execution means includes at least one of positions, shapes, and orientations of the first detected part image and / or the second detected part image showing at least a part of the detected part included in the captured image. The information processing program according to claim 22, wherein an operation of the user is detected based on one and a game process based on the detected user operation is executed. The detected part is a position in the movable part, and is provided at a position that can be imaged by the imaging device provided in the main device mounted on a sub-device provided with the movable part,
24. The relative position and / or relative attitude between the main device and the detected part is changed by moving the detected part in response to the movable part being moved in response to an operation of the user. Information processing program described in 1. The main device can be attached to the movable part,
When the movable unit is moved in response to the user's operation, the main device mounted on the movable unit is moved, so that the relative position and / or relative posture between the main device and the detected unit is changed. The information processing program according to claim 23, wherein the information processing program changes.   The information processing program according to any one of claims 20 to 25, wherein the game process execution unit generates a different game image according to the determined type of the sub device. The game process execution means includes
Based on a game operation using the main device and / or the sub device to which the main device is attached, controls an object placed in a virtual space,
27. The information processing program according to claim 26, wherein the object is changed according to the determined type of the sub device.   The information processing program according to claim 27, wherein the object is an object representing a vehicle on which a game character can board. The main device includes a plurality of types of operation detection units for detecting a user operation on the main device.
When it is determined that the main device is attached to a predetermined type of sub-device, the game processing means, according to the type of the determined sub-device among the plurality of types of the operation detection units. The information processing program according to any one of claims 20 to 28, wherein game processing is executed based on a detection result by the operation detection unit. The main device includes an operation detection unit for detecting a user operation on the main device,
30. The game processing unit according to claim 20, wherein the game processing unit determines the content of the game process to be executed according to the detection result by the operation detection unit based on the determined type of the sub device. Information processing program according to item.   The information processing program according to any one of claims 20 to 30, wherein the determination unit further determines that the main device is not attached to any of the plurality of types of sub-devices.   When it is determined that the main device is not attached to any of the plurality of types of sub-devices, the game process execution means is connected to any of the plurality of types of sub-devices. 32. The information processing program according to claim 31, wherein the information processing program executes a game process different from that of the game process. The game system further includes a storage unit,
The information processing program includes:
When it is determined that the main device is attached to a predetermined type of sub-device, the storage control unit stores device orientation information related to the orientation of the main device with respect to the predetermined type of sub-device in the storage unit. Make the computer work better,
The game process execution means includes
A game process is executed based on a direction operation by the user with respect to the main device,
Non-attached when it is determined that the main device is not attached to any of the plurality of types of sub-devices from the attached state where it is determined that the main device is attached to the predetermined type of sub-devices When the state is changed, the relationship between the direction specified by the direction operation and the content of the game process executed in response to the operation in the direction is stored in the storage unit in the mounted state. The information processing program according to claim 31, wherein the information processing program is determined based on the stored device orientation information. The detected portion included in at least one of the plurality of types of sub-devices includes a first portion and a second portion;
The information processing program includes:
Recognizing the first part based on the captured image, causing the computer to further function as a recognition means for recognizing the second part;
The discrimination means includes
Based on at least one of the position, shape, and orientation of the first partial image indicating the first portion, the type of the sub device to which the main device is attached is determined,
As a state of the main device mounted on the sub-device, the first state in which the first part is recognized and the second part is not recognized, and the second state in which the first part and the second part are recognized The information processing program according to any one of claims 22 to 25, wherein:   The game process execution means executes a first game process when it is determined that the main device is in the first state, and when it is determined that the main device is in the second state, The information processing program according to claim 34, wherein a second game process different from the first game process is executed. The detected portion included in at least one of the plurality of types of sub-devices includes a first portion and a second portion;
The information processing program includes:
Recognizing the first part based on the captured image, causing the computer to further function as a recognition means for recognizing the second part;
The discrimination means includes
Based on at least one of the position, shape, and orientation of the first partial image indicating the first portion, the type of the sub device to which the main device is attached is determined,
26. The degree of mounting of the main device on the sub device is determined based on the recognition result of the first portion and the recognition result of the second portion. Information processing program described in 1.   37. The information processing program according to claim 36, wherein the game process execution means executes a game process according to a degree to which the main device is attached to the sub device. At least one of the plurality of types of sub-devices further includes a movable part that moves in response to an operation by a user,
When the movable part is moved according to the user's operation, the relative position between the main device attached to the sub apparatus including the movable part and the detected part of the sub apparatus changes,
The game process execution means includes
The first type operation of the user is detected based on at least one of the position, shape, and orientation of the first partial image showing the first part, and based on the detected first type operation. Run the game process
The second type operation of the user is detected based on at least one of the position, shape, and orientation of the second partial image indicating the second part, and based on the detected second type operation The information processing program according to any one of claims 34 to 37, wherein game processing is executed. An information processing system that includes a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
A discriminating unit for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
A game process execution unit configured to execute a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
The game process execution unit executes a game process according to the type of the sub device determined by the determination unit;
The main device includes an imaging device,
The determination unit determines the type of the sub device on which the main device is mounted based on a captured image captured by the imaging device,
At least one of the plurality of types of sub-devices,
A detected portion that can be imaged by the imaging device of the main device mounted on the sub-device;
A movable part that moves in response to an operation by the user,
When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
The game process execution unit detects an operation of the user based on at least one of a position, a shape, and an orientation of a detected portion image that indicates at least a part of the detected portion included in the captured image. An information processing system that executes game processing based on the detected user operation.   An information processing system that includes a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
  A discriminating unit for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
  A game process execution unit configured to execute a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
  The game process execution unit executes a game process according to the type of the sub device determined by the determination unit;
  The main device includes an imaging device,
  At least one of the plurality of types of sub-devices,
    A detected portion that can be imaged by the imaging device of the main device mounted on the sub-device;
    A movable part that moves in response to an operation by the user,
  When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
  The game process execution unit is based on at least one of a position, a shape, and an orientation of a detected portion image that indicates at least a part of the detected portion included in a captured image captured by the imaging device. Detecting a user operation, executing a game process based on the detected user operation,
  The main device can be attached to the movable part,
  When the movable unit is moved in response to the user's operation, the main device mounted on the movable unit is moved, so that the relative position and / or relative posture between the main device and the detected unit is changed. A changing information processing system. An information processing system that includes a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
A discriminating unit for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
A game process execution unit configured to execute a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
The information processing system, wherein the game process execution unit controls a position and / or posture of a virtual camera arranged in a virtual space based on the type of the sub device determined by the determination unit . An information processing apparatus included in a game system including a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
A discriminating unit for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
A game process execution unit configured to execute a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
The game process execution unit executes a game process according to the type of the sub device determined by the determination unit;
The main device includes an imaging device,
The determination unit determines the type of the sub device on which the main device is mounted based on a captured image captured by the imaging device,
At least one of the plurality of types of sub-devices,
A detected portion that can be imaged by the imaging device of the main device mounted on the sub-device;
A movable part that moves in response to an operation by the user,
When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
The game process execution unit detects an operation of the user based on at least one of a position, a shape, and an orientation of a detected portion image that indicates at least a part of the detected portion included in the captured image. An information processing apparatus that executes a game process based on the detected user operation.   An information processing apparatus included in a game system including a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
  A discriminating unit for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
  A game process execution unit configured to execute a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
  The game process execution unit executes a game process according to the type of the sub device determined by the determination unit;
  The main device includes an imaging device,
  At least one of the plurality of types of sub-devices,
    A detected portion that can be imaged by the imaging device of the main device mounted on the sub-device;
    A movable part that moves in response to an operation by the user,
  When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
  The game process execution unit is based on at least one of a position, a shape, and an orientation of a detected portion image that indicates at least a part of the detected portion included in a captured image captured by the imaging device. Detecting a user operation, executing a game process based on the detected user operation,
  The main device can be attached to the movable part,
  When the movable unit is moved in response to the user's operation, the main device mounted on the movable unit is moved, so that the relative position and / or relative posture between the main device and the detected unit is changed. An information processing device that changes. An information processing apparatus included in a game system including a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
A discriminating unit for discriminating the type of sub-device to which the main device is mounted among the plurality of types of sub-devices during execution of the predetermined game application;
A game process execution unit configured to execute a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
The game processing executing unit, based on the type of the sub-device which is determined by the determination unit, to control the position and / or orientation of the virtual camera placed in the virtual space, the information processing apparatus. An information processing method that is executed in an information processing system that includes a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
A determination step of determining a type of a sub device to which the main device is mounted among the plurality of types of sub devices during execution of the predetermined game application;
A game process execution step for executing a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
In the game process execution step, a game process corresponding to the type of the sub-device determined in the determination step is executed,
The main device includes an imaging device,
In the determination step, based on a captured image captured by the imaging device, the type of the sub device to which the main device is attached is determined,
At least one of the plurality of types of sub-devices,
A detected portion that can be imaged by the imaging device of the main device mounted on the sub-device;
A movable part that moves in response to an operation by the user,
When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
In the game process execution step, an operation of the user is detected based on at least one of a position, a shape, and an orientation of a detected part image indicating at least a part of the detected part included in the captured image. And an information processing method for executing a game process based on the detected user operation.   An information processing method that is executed in an information processing system that includes a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
  A determination step of determining a type of a sub device to which the main device is mounted among the plurality of types of sub devices during execution of the predetermined game application;
  A game process execution step for executing a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
  In the game process execution step, a game process corresponding to the type of the sub-device determined in the determination step is executed,
  The main device includes an imaging device,
  At least one of the plurality of types of sub-devices,
    A detected portion that can be imaged by the imaging device of the main device mounted on the sub-device;
    A movable part that moves in response to an operation by the user,
  When the movable portion is moved in accordance with the user's operation, the relative position and / or relative posture between the main device mounted on the sub device including the movable portion and the detected portion of the sub device is changed. Change,
  In the game process execution step, based on at least one of the position, shape, and orientation of the detected portion image that indicates at least a part of the detected portion included in the captured image captured by the imaging device, Detecting the user's operation, executing a game process based on the detected user's operation,
  The main device can be attached to the movable part,
  When the movable unit is moved in response to the user's operation, the main device mounted on the movable unit is moved, so that the relative position and / or relative posture between the main device and the detected unit is changed. Information processing methods that change. An information processing method that is executed in an information processing system that includes a plurality of types of sub-devices and a main device that can be attached to and detached from any of the plurality of types of sub-devices, and executes a predetermined game application,
A determination step of determining a type of a sub device to which the main device is mounted among the plurality of types of sub devices during execution of the predetermined game application;
A game process execution step for executing a game process based on a game operation using the main device and / or the sub device to which the main device is mounted during execution of the predetermined game application;
An information processing method for controlling the position and / or orientation of a virtual camera arranged in a virtual space based on the type of the sub-device determined in the determination step in the game process execution step.

Images