JP2008134991A - Input method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a new input method and its related technology, which can give an input by using the imaging result of an object. <P>SOLUTION: Input devices 3L, 3R attached to the respective palm sides of right and left hands are photographed by an image sensor 54 (input state) and the photographed results are processed to make cursors 70L, 70R interlock with the movement of the input devices 3L, 3R. A player moves the cursor 70L or 70R to a blinking fixed position object 100 and clenches one hand. The input device 3L or 3R attached to the clenched hand is not photographed (non-input state). When the player performs these operations, another fixed position object 100 blinks, and thereby the player performs similar operation. The processing is repeated. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an input method using an imaging result of a subject and related technology.

  The golf game system disclosed in Patent Document 1 by the applicant of the present application includes a game machine and a golf club type input device, and an imaging unit is housed inside the housing of the game machine. It is composed of an infrared light emitting diode or the like. Infrared light is intermittently irradiated to a predetermined range in front of the imaging unit by the infrared light emitting diode, and therefore the image sensor intermittently applies a reflector provided in the golf club type input device moving within the range. Take a picture. By processing such a strobe image of the reflector, the speed of the input device serving as an input of the game machine is calculated. In this way, an input can be given in real time to a computer or game machine using a stroboscope.

JP 2004-85524 A

  An object of the present invention is to provide a novel input method and related technology capable of giving an input by using an imaging result of a subject.

  According to the first aspect of the present invention, the input method includes a step of displaying a target image on a display device, a step of displaying a cursor on the display device, a step of imaging a subject worn on the palm of the user, The step of detecting the movement of the subject based on the image of the subject obtained by the imaging, the step of moving the cursor according to the detected movement of the subject, and the cursor being the target image And a step of giving a predetermined change to the target image and / or the cursor when an image of the subject is not obtained immediately after the image is overlaid on the cursor.

  According to this configuration, the user can change the target image and / or the cursor by moving the cursor with the palm open, overlaying the target image, and clenching the hand to hide the subject. it can. That is, the user can change the target image and / or the cursor by moving the cursor to the target image and clenching the hand. Therefore, the user can give an input by an operation as if holding the target image.

  In the input method, the target image appears from a non-display state at an arbitrary position or any one of a plurality of predetermined positions.

  In the input method, the target image may be statically displayed in advance at an arbitrary position or a predetermined position.

  In the above input method, the target image may be plural, and may be statically displayed in advance at a plurality of arbitrary positions or a plurality of predetermined positions.

  In the input method, the target image may be a moving moving image.

  According to the second aspect of the present invention, the input method is based on the step of displaying the first object on the display device, the step of imaging a plurality of subjects, and the images of the plurality of subjects obtained by the imaging. Detecting the positions of the plurality of subjects, and changing the form of the first object according to the detected positional relationship between the plurality of subjects.

  According to this structure, the user can change a 1st object to the form according to those positional relationship by moving a some to-be-photographed object.

  The input method includes a step of displaying a second object on the display device, and a step of changing the second object when the first object and the second object satisfy a predetermined positional relationship; Further included.

  According to this configuration, the user can change the first object by moving a plurality of subjects and accordingly change the second object.

  According to a third aspect of the present invention, the training method includes the steps of imaging a subject, detecting the movement of the subject based on the image of the subject obtained by the imaging, and detecting the detected A condition that the cursor displayed on the screen of the display device is moved according to the movement of the subject, a step where a stationary or moving target appears on the screen, and at least the cursor overlaps the target And the step of giving a predetermined change to the target, and the step of repeating the step of causing the change and the step of giving the change, and the position where the target appears is newly set each time.

  According to this configuration, the player can operate the cursor by moving the subject. Therefore, it is possible to instruct the player to place the cursor on the stationary target that appears as soon as possible. By repeating the action based on such an instruction, the player can perform training on the ability (a kind of moving object recognition) of how quickly he can react to stationary targets that appear one after another.

  Further, it is possible to cause the player to move the cursor as quickly as possible to the target that moves around as much as possible. As a result, agility and ability to read ahead can be trained.

  Here, the appearance of a stationary target includes not only the appearance of an object representing the target but also the appearance of the object by a change in form such as color and shape, and combinations thereof. In addition, the predetermined change includes a change in form and disappearance of a target.

  In this training method, when a light is applied to the reflector as the subject, and an image based on the light irradiated to the reflector and reflected by the reflector is obtained in the step of capturing, A step of determining that the subject is detected as being in the first input state, and an image based on the light which is irradiated on the reflector and reflected by the reflector is not obtained in the step of capturing. And determining that the subject is in the second input state because the subject has not been detected. In the step of giving the predetermined change, the cursor is placed on the target in the first input state. When the second input state is determined from the overlapped state, the predetermined change is given to the target.

  According to this configuration, when the player images the reflector, the first input state is determined, and when the player is not imaged, the second input state is determined. Then, if the player simply superimposes the cursor on the target in the first input state, the selection is not confirmed, but the selection is confirmed only after the second input state. Therefore, in order to give a predetermined change to the target, the player has to change the state of the reflector from being picked up to not picked up. Along with this, the player moves according to such a change in the state. As a result, in addition to the above-described ability training, the player can be exercised, and health maintenance / promotion and rehabilitation can be supported.

  According to a fourth aspect of the present invention, the training method includes the steps of imaging the subject, detecting the movement of the subject based on the image of the subject obtained by the imaging, and detecting the detected The method includes a step of moving a cursor displayed on a screen of a display device in accordance with a movement of a subject, and a step of causing an object moving around on the screen to appear on the screen.

  According to this configuration, the player can operate the cursor by moving the subject. Therefore, it is possible to instruct the player to move the cursor so that the moving object does not collide. The player can perform training on the ability (a kind of moving object prediction) of how to avoid an object skillfully by repeating an action based on such an instruction.

  The training method further includes causing a new object that moves around the screen to appear on the screen according to a predetermined condition. According to this configuration, since objects can appear one after another, the difficulty level can be improved.

  In this training method, the appearance position of the object is newly set each time. According to this configuration, since the appearance position of the object is not constant, the difficulty level can be further improved.

  The training method further includes a step of determining whether or not the cursor has collided with the object. According to this configuration, since it is known that the cursor has collided with the object, when the cursor collides with the object, it can be notified to the player by video and / or audio.

  According to a fifth aspect of the present invention, the training method includes the steps of imaging a subject, detecting the movement of the subject based on the image of the subject obtained by the imaging, and detecting the detected Moving the cursor displayed on the screen of the display device in accordance with the movement of the subject; displaying the plurality of targets assigned with an order on the screen; and overlapping the cursor with at least the target according to the order. And subjecting the target to a predetermined change.

  According to this configuration, the player can operate the cursor by moving the subject. Therefore, it is possible to instruct the player to place the cursor on the displayed target as soon as possible according to the order. By repeating the action based on such an instruction, the player can perform training on the ability (a kind of instantaneous recognition) of how quickly the target can be recognized in order.

  Here, the predetermined change includes a change in form and disappearance of the target.

  In this training method, numbers are attached to the targets, and the plurality of numbers attached to the plurality of targets are continuous integers. According to this configuration, it is possible to instruct the player to place the cursor on the target as soon as possible in accordance with the order of the attached numbers. On the other hand, in the above training method, a number may be attached to the target, and the plurality of numbers attached to the plurality of targets may be discontinuous and different integers. According to this configuration, since the numbers assigned to the targets are discontinuous and different integers, the difficulty level can be improved.

  The training method includes the steps of irradiating a reflector as the subject with light, and when the image based on the light that is reflected on the reflector and reflected by the reflector is obtained in the step of capturing, A step of determining that the subject is detected as being in the first input state, and an image based on the light which is irradiated on the reflector and reflected by the reflector is not obtained in the step of capturing. And determining that the subject is in the second input state because the subject has not been detected, and in the step of giving the predetermined change, the target is applied to the target in the order in the first input state. When the second input state is determined from the state where the cursor overlaps, the predetermined change is given to the target.

  According to this configuration, when the player images the reflector, the first input state is determined, and when the player is not imaged, the second input state is determined. Then, if the player simply superimposes the cursor on the target in the first input state, the selection is not confirmed, but the selection is confirmed only after the second input state. Therefore, in order to give a predetermined change to the target, the player has to change the state of the reflector from being picked up to not picked up. Along with this, the player moves according to such a change in the state. As a result, in addition to the above-described ability training, the player can be exercised, and health maintenance / promotion and rehabilitation can be supported.

  According to a sixth aspect of the present invention, the training method includes the steps of imaging the subject, detecting the movement of the subject based on the image of the subject obtained by the imaging, and detecting the detected Moving the cursor displayed on the screen of the display device according to the movement of the subject; displaying a plurality of stationary objects at a plurality of predetermined positions on the screen; and A step of displaying a marker at a position where the object is displayed for a predetermined time; and a step of repeating the step of displaying the marker. In the step of repeating, the position of displaying the marker includes the plurality of positions. A new one is selected from the fixed position objects each time.

  According to this configuration, the player can operate the cursor by moving the subject. Therefore, it is possible to instruct the player to specify the position of the marker displayed at a predetermined time with the cursor. By allowing the player to perform such a specified action after the step of repeating the step of displaying the marker, the player can be trained in short-term memory (a kind of short-term memory).

  In this training method, in the repeating step, the plurality of markers that are sequentially displayed for the predetermined time each belong to one of a plurality of forms. According to this configuration, since it is possible to instruct the player on the form of the marker to be specified, the difficulty level can be improved. On the other hand, in the training method, in the step of displaying, a predetermined number and a plurality of the markers can be simultaneously displayed at the positions of different fixed position objects for the predetermined time. According to this configuration, since it is possible to instruct the player to specify a plurality of markers, the difficulty level can be improved. Further, in this training method, in the step of displaying, the predetermined number and the plurality of markers displayed simultaneously at different positions of the fixed-position objects belong to one of a plurality of forms, respectively. You can also. According to this configuration, since it is possible to instruct the player on the forms of a plurality of markers to be specified, the difficulty level can be further improved.

  The training method includes a step of outputting a video and / or sound instructing to specify a position of the marker displayed at a predetermined time, and the fixed-position object in which the marker instructed to specify is located And a first determination step of determining whether or not the cursors overlap.

  According to this configuration, since it is known whether or not the cursor is placed at the correct position, it is possible to notify the player of the correctness by video and / or audio.

  In this training method, when a light is applied to the reflector as the subject, and an image based on the light irradiated to the reflector and reflected by the reflector is obtained in the step of capturing, A step of determining that the subject is detected as being in a first input state, and an image based on the light irradiated on the reflector and reflected by the reflector is not obtained in the step of capturing. And determining that the subject is in a second input state because the subject has not been detected, and moving the cursor in the first input state to the fixed-position object where the marker instructed to specify is located. And a second determination step of determining whether or not the second input state has been reached from the state in which the two have overlapped.

  According to this configuration, when the player images the reflector, the first input state is determined, and when the player is not imaged, the second input state is determined. The first input state can be the selected state, and the second input state can be the selected confirmed state. Therefore, if the player simply superimposes the cursor on the fixed position object in the first input state, the selection is not confirmed, but the selection is confirmed only after the second input state. For this reason, in order to confirm the selection, the player has to change the state where the reflector is imaged to the state where the image is not imaged. Along with this, the player moves according to such a change in the state. As a result, in addition to the above-described ability training, the player can be exercised, and health maintenance / promotion and rehabilitation can be supported.

  Further, in the training method, when it is determined in the second determination step that the second input state has been established, a step of storing which fixed position object the cursor is over, and the fixed position From the step of displaying the predetermined object at a predetermined position on the screen in a region other than the region where the object is displayed, and from the state where the cursor overlaps the predetermined object in the first input state, the second A third determination step for determining whether or not the input state has been reached, and when it is determined in the third determination step that the second input state has been reached, refer to the information stored in the step stored. And a fourth determination step of determining whether or not the information matches the instruction by the step of outputting.

  According to this configuration, whether or not the instruction matches the instruction when the cursor is overlaid on a predetermined object located in a region other than the fixed-position object and is changed from the first input state to the second input state. Since the determination is performed, it is possible to prevent the determination as to whether or not the instruction is matched based on the selection state that the player does not intend as much as possible.

  In the training methods according to the third to sixth embodiments, the cursor includes a transparent color or a translucent color. According to this configuration, the player can also visually recognize what is displayed in the back of the cursor, and can improve visibility.

  In the training method according to the third aspect, the fifth aspect, and the sixth aspect, in the step of imaging, the two reflectors mounted on the left and right hands of the player are imaged and moved. In the step, one of the two cursors is moved according to the movement of one of the reflectors, and the other cursor is moved according to the movement of the other reflector. According to this configuration, since the player uses both hands, the player can perform an exercise of moving both hands in a balanced manner.

  In the training method according to the third, fifth, and sixth aspects, in the step of imaging, the reflector attached to the player's hand is imaged. According to this configuration, the player can perform hand exercise. In these training methods, in the step of imaging, the reflector mounted on the palm side of the player is imaged. According to this configuration, the player can open the hand to capture the image of the reflector to be in the first input state, and can hold the hand and conceal the reflector to be in the second input state. In this case, open hands (Janken's "Pa") and clenching (Janken's "Guu") are repeated. Inevitably, such an operation is performed by extending the arm forward to some extent or completely. Therefore, it is possible to train nerves and muscles that depend on actions that are not often performed in daily life. On the other hand, in these training methods, in the step of imaging, the reflector mounted on the back side of the player's hand can be imaged. According to this configuration, the player captures the reflector with the back side of the hand facing the imaging device to be in the first input state, hides the reflector with the palm side facing the imaging device, and is in the second input state. can do. In this case, the hand is rotated about the elbow as a fulcrum and the arm as an axis. Inevitably, such an operation is performed by extending the arm forward to some extent or completely. Therefore, it is possible to train nerves and muscles that depend on actions that are not often performed in daily life. In addition, the player holds the hand and holds the fist toward the imaging device to image the reflector to enter the first input state, and hides the fist from the imaging device (for example, aligning the finger backs of the left and right hands). It can be in an input state. Therefore, it is possible to train nerves and muscles that depend on actions that are not often performed in daily life.

  According to a seventh aspect of the present invention, the training method includes the steps of irradiating a reflector as a subject attached to a player's hand with light, imaging the reflector, and the imaging obtained by the imaging. Detecting a movement of the reflector based on an image of the reflector, moving a cursor displayed on a screen of a display device according to the detected movement of the reflector, and the reflector Determining that the subject is detected as being in the first input state when an image based on the light that is irradiated on the reflector and reflected by the reflector is obtained in the step of capturing, and the reflection When the image based on the light irradiated onto the body and reflected by the reflector is not obtained in the step of capturing, the subject is not detected and the second input state is set. Determining, according to a computer program, along with the cursor, and displaying an image on the screen.

  According to this configuration, when the player images the reflector, the first input state is determined, and when the player is not imaged, the second input state is determined. Therefore, by displaying an image that causes the player to repeatedly perform the first input state and the second input state by the computer program, the player can be in a state in which the reflector is imaged and a state in which the reflector is not imaged. Must be created repeatedly. Along with this, the player moves according to such a change in the state. As a result, the player can exercise and can support health maintenance / promotion and rehabilitation.

  In this training method, in the step of displaying, the video is related to a predetermined human ability.

  According to this configuration, since the image displayed on the screen is related to a predetermined human ability, the state in which the reflector is imaged and the state in which the reflector is not imaged in relation to this image is given to the player. By repeating the above, it is possible to perform training with a predetermined human ability.

  Here, the predetermined ability is a series of sensory organs such as eyes → sensory nerves → brain → motor nerves → parts of the body such as hands, such as dynamic body recognition ability, dynamic body prediction ability, instantaneous recognition ability, and short-term memory ability. Includes transmission and storage functions.

  According to an eighth aspect of the present invention, the training method includes the steps of imaging the subject, detecting the movement of the subject based on the image of the subject obtained by the imaging, and detecting the detected Corresponding to the language information by voice and / or video, moving the cursor displayed on the screen of the display device according to the movement of the subject, presenting language information to the player by voice and / or video Presenting the first sensory information and the second sensory information that conflicts with the language information to the player.

  According to this configuration, it is possible to cause the player to refine functions such as action selection and interference elimination using the Stroop effect. Here, the “Stroop effect” is a phenomenon in which sensory information (for example, color) and language information (for example, characters) interfere with each other in a human information processing process.

  According to a ninth aspect of the present invention, the training method includes a step of imaging a subject, a step of detecting a movement of the subject based on an image of the subject obtained by the imaging, and the detected Moving the cursor displayed on the screen of the display device in accordance with the movement of the subject; moving the predetermined object displayed on the screen toward the cursor according to a predetermined rule; Displaying a target on the screen, determining whether or not the predetermined object has overlapped a predetermined area including the target, and displaying a moving avoidance object to be avoided by the predetermined object on the screen; Whether or not the predetermined object has touched the avoidance object Comprising determining the.

  According to this configuration, when a predetermined object comes into contact with the avoidance object, if the player is given a disadvantage such as deduction, the cursor is prevented from contacting the avoidance object with the avoidance object. Can be operated to guide a predetermined object to the target. As a result, the player must operate the cursor while predicting the movement of the avoidance object, and can train the ability to read ahead.

  According to a tenth aspect of the present invention, the training method includes the steps of imaging a subject, detecting the movement of the subject based on the image of the subject obtained by the imaging, and detecting the detected Controlling the movement of the response object displayed on the screen of the display device in accordance with the movement of the subject; causing a plurality of types of moving objects to appear and move on the screen; the moving object and the response object; Changing the moving direction of the moving object when the positional relationship satisfies a predetermined condition, and each of the plurality of types of moving objects presents different information to the player. In the training method, a plurality of types of objects corresponding to the plurality of types are displayed on the screen. Further comprising the step of displaying.

  According to this configuration, it is possible to cause the player to perform an action of operating the response object and returning the moving object to an object of the same type as the information presented by the moving object. As a result, it is necessary for the player to instantaneously determine the content of information presented by the moving moving object and to return the moving object to an appropriate object among a plurality of types of objects by operating the response object. For this reason, a player's instantaneous judgment power can be trained. Here, the information presented by the moving object includes sensory information such as color and language information such as characters.

  According to an eleventh aspect of the present invention, the training method corresponds to a step of imaging a plurality of subjects, and a step of detecting a motion of the subject based on the images of the subjects obtained by the imaging. Displaying a plurality of response objects responding to the detected movement of the subject, simultaneously displaying a plurality of routes to each corresponding destination, and moving on the route toward the destination The method includes a step of displaying a moving object and a step of changing the plurality of paths to be displayed simultaneously according to a predetermined rule, wherein the response object is arranged at the corresponding arrival point.

  According to this configuration, it is possible to cause the player to perform an action of changing the response object at the timing when the moving object reaches the response object. As a result, the player needs to change the response object at an appropriate timing in accordance with the moving object that moves one after another, so that the judgment power and the ability to read ahead can be trained. Moreover, if a moving object appears and moves in accordance with music, it is possible to train a sense of rhythm.

  According to a twelfth aspect of the present invention, the training method includes the steps of imaging a subject, detecting the movement of the subject based on the image of the subject obtained by the imaging, and detecting the detected Moving the cursor displayed on the screen of the display device according to the movement of the subject, displaying N objects (N is a natural number of 2 or more) indicating different information on the screen, and the N After the step of making the information indicated by the objects invisible and the step of making the information invisible not visible, the step of exchanging a predetermined number of the objects among the N objects, and the exchanging the positions After the step, the information indicated by any one of the N objects is And Shimesuru step, said object indicating the explicit the information demonstrating the step includes a step of determining whether or not selected by the cursor.

  According to this configuration, the information indicated by the N objects can be stored in the player, and the object indicating the specified information can be selected by the cursor after the object is shuffled. As a result, the player must store the arrangement of the N objects and the information indicated by them, and keep track of the replacement of the objects. For this reason, a player's memory can be trained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and the description thereof is incorporated.

  FIG. 1 is a block diagram showing the overall configuration of a training system according to an embodiment of the present invention. As shown in FIG. 1, the training system includes an information processing device 1, input devices 3 </ b> L and 3 </ b> R, and a television monitor 5. Here, when it is not necessary to distinguish between the input devices 3L and 3R, they are referred to as the input device 3.

  FIG. 2 is a perspective view of the input device 3 of FIG. As shown in FIG. 2, the input device 3 is formed by passing a belt 19 on the bottom surface side of the transparent body 17 and fixing the belt 19 inside the transparent body 17. The retroreflective sheet 15 is attached over the entire inner surface of the transparent body 17 (excluding the bottom surface side). A method of using the input device 3 will be described later.

  Here, when it is necessary to distinguish between the input devices 3L and 3R, the transparent body 17 and the retroreflective sheet 15 of the input device 3L are referred to as the transparent body 17L and the retroreflective sheet 15L, respectively, and the input device 3R is transparent. The body 17 and the retroreflective sheet 15 are referred to as a transparent body 17R and a retroreflective sheet 15R, respectively.

  Returning to FIG. 1, the information processing apparatus 1 is connected to the television monitor 5 by the AV cable 7. Further, although not shown, the information processing apparatus 1 is supplied with a power supply voltage by an AC adapter or a battery. A power switch (not shown) is provided on the back surface of the information processing apparatus 1.

  The information processing apparatus 1 is provided with an infrared filter 20 that transmits only infrared light on the front side thereof, and further, four infrared light emitting diodes 9 that generate infrared light are exposed so as to surround the infrared filter 20. is doing. An image sensor 54 described later is disposed on the back side of the infrared filter 20.

  The four infrared light emitting diodes 9 emit infrared light intermittently. The infrared light from the infrared light emitting diode 9 is reflected by the retroreflective sheet 15 attached to the input device 3 and input to the image sensor 54 provided on the back side of the infrared filter 20. In this way, the input device 3 is photographed by the image sensor 54.

  When infrared light is irradiated intermittently, photographing processing by the image sensor 54 is performed even when infrared light is not irradiated. The information processing apparatus 1 obtains the difference between the image signal at the time of infrared light irradiation and the image signal at the time of non-irradiation of the input device 3 moved by the player, and based on the difference signal DI (difference image DI). The position of the input device 3 (that is, the retroreflective sheet 15) is calculated.

  Thus, by obtaining the difference, noise due to light other than the reflected light from the retroreflective sheet 15 can be removed as much as possible, and the retroreflective sheet 15 can be detected with high accuracy.

  FIG. 3 is an explanatory diagram showing an example of a usage state of the input devices 3L and 3R in FIG. As shown in FIGS. 1 and 3, the player puts the input device 3 by passing the middle finger through the belt 19 in FIG. 2. In this case, the transparent body 17 and the retroreflective sheet 15 are placed on the palm side. As shown in FIG. 1, when the player opens his hand toward the information processing apparatus 1, that is, toward the image sensor 54, the transparent body 17, that is, the retroreflective sheet 15 appears. Taken. On the other hand, when the transparent body 17 is clamped, the transparent body 17, that is, the retroreflective sheet 15 is hidden in the hand and is not photographed by the image sensor 54. Therefore, the player can control the input to the information processing apparatus 1 by causing the retroreflective sheet 15 to be photographed or not photographed by opening and closing the hand. In the present embodiment, the case where the retroreflective sheet 15 is photographed is referred to as an input state (sometimes referred to as a first input state), and the state where the retroreflective sheet 15 is not photographed is sometimes referred to as a non-input state (second input state). .)

  FIG. 4 is a diagram showing an electrical configuration of the information processing apparatus 1 of FIG. As illustrated in FIG. 4, the information processing apparatus 1 includes a multimedia processor 50, an image sensor 54, an infrared light emitting diode 9, an external memory 52, and a bus 56. The external memory 52 includes a ROM, a RAM, and / or a flash memory that is necessary according to the system specifications.

  The multimedia processor 50 can access the external memory 52 through the bus 56. Therefore, the multimedia processor 50 can execute the program stored in the external memory 52 and can read and process the data stored in the external memory 52. The external memory 52 stores in advance a program for performing various processes such as control of various screens described later, position detection of the retroreflective sheets 15L and 15R, and determination of an input state and a non-input state, image data, audio data, and the like. Is done.

  Although not shown, the multimedia processor includes a central processing unit (hereinafter referred to as “CPU”), a graphics processing unit (hereinafter referred to as “GPU”), and a sound processing unit (hereinafter referred to as “SPU”). ), A geometry engine (hereinafter referred to as “GE”), an external interface block, a main RAM, an A / D converter (hereinafter referred to as “ADC”), and the like.

  The CPU executes programs stored in the external memory 52 to perform various calculations and control of the entire system. As processing of the CPU related to graphics processing, a program stored in the external memory 52 is executed, and parameters for enlargement / reduction, rotation, and / or translation of each object, viewpoint coordinates (camera coordinates), and line-of-sight vector are calculated. Perform calculations. Here, a unit composed of one or a plurality of polygons or sprites and applied with the same transformation of enlargement / reduction, rotation, and translation is referred to as an “object”.

  The GPU generates a three-dimensional image composed of polygons and sprites in real time and converts it into an analog composite video signal. The SPU generates PCM (pulse code modulation) waveform data, amplitude data, and main volume data, and analog-multiplies them to generate an analog audio signal. The GE performs a geometric operation for displaying a three-dimensional image. Specifically, the GE performs operations such as matrix product, vector affine transformation, vector orthogonal transformation, perspective projection transformation, vertex brightness / polygon brightness calculation (vector inner product), and polygon back surface culling processing (vector outer product).

  The external interface block is an interface with peripheral devices (in this embodiment, the image sensor 54 and the infrared light emitting diode 9), and includes a 24-channel programmable digital input / output (I / O) port. The ADC is connected to four-channel analog input ports, and converts analog signals input from the analog input device (in this embodiment, the image sensor 54) into digital signals through these ADCs. The main RAM is used as a CPU work area, a variable storage area, a virtual storage mechanism management area, and the like.

  The input devices 3L and 3R are irradiated with the infrared light of the infrared light emitting diode 9, and the infrared light is reflected by the retroreflective sheets 15L and 15R. The reflected light from the retroreflective sheets 15L and 15R is photographed by the image sensor 54. Therefore, the image sensor 54 outputs image signals including the retroreflective sheets 15L and 15R. As described above, the multimedia processor 50 intermittently blinks the infrared light-emitting diode 9 for strobe photography, so that an image signal when the infrared light is extinguished is also output. These analog image signals from the image sensor 54 are converted into digital data by an ADC built in the multimedia processor 50.

  The multimedia processor 50 generates the difference signal DI (difference image DI) from the digital image signal input from the image sensor 54 via the ADC, and based on this, input / non-input by the input devices 3L and 3R. In addition, the position of the input devices 3L and 3R is detected, calculation, graphic processing, sound processing, etc. are executed, and a video signal and an audio signal are output. The video signal and the audio signal are given to the television monitor 5 through the AV cable 7, and accordingly, an image is displayed on the television monitor 5, and sound is output from a speaker (not shown).

  As will be described later, the multimedia processor 50 controls the movement of the cursors 70L and 70R according to the detected positions of the input devices 3L and 3R. That is, the multimedia processor 50 extracts the images of the retroreflective sheets 15L and 15R from the difference image DI, and calculates the coordinates of the respective points of interest on the difference image DI. Then, the multimedia processor 50 obtains the positions of the two points of interest on the screen of the television monitor 5 by converting the coordinates on the difference image DI of the two points of interest into screen coordinates. The multimedia processor 50 displays the cursors 70L and 70R at the positions on the screen of these two attention points (corresponding to the retroreflective sheets 15L and 15R). The screen coordinate system is a coordinate system used when displaying an image on the television monitor 5.

  Next, the processing contents by the multimedia processor 50 will be described by exemplifying screens in various modes executed by the training system of the present embodiment.

  [Flash catch mode]

  FIG. 5 is a view showing an example of a training screen in the flash catch mode by the training system of FIG. Referring to FIG. 5A, the multimedia processor 50 displays a screen including 20 fixed position objects 100 arranged in a grid (4 rows × 5 columns) on the television monitor 5.

  Further, the multimedia processor 50 displays the cursors 70L and 70R on the television monitor 5. The multimedia processor 50 links the cursor 70L to the movement of the retroreflective sheet 15L photographed by the image sensor 54, and links the cursor 70R to the movement of the retroreflective sheet 15R photographed by the image sensor 54. Since the colors of the cursors 70L and 70R are translucent colors, the player can check the fixed position object 100 even if the cursors 70L and 70R overlap the fixed position object 100. Of course, the colors of the cursors 70L and 70R can be made transparent. Thus, the visibility of the player can be improved by making the cursors 70L and 70R translucent or transparent.

  In a state where the player opens the left hand and the retroreflective sheet 15L is detected, that is, in the input state of the left hand, a cursor 70L in a form imitating the left hand in the opened state is displayed. Similarly, in the state where the player opens the right hand and the retroreflective sheet 15R is detected, that is, in the input state of the right hand, the cursor 70R in a form imitating the opened right hand is displayed. Further, as shown in FIG. 5B, in a state where the player closes the left hand and the retroreflective sheet 15L is not detected, that is, in the non-input state of the left hand, a cursor 70L simulating the closed left hand is displayed. Is done. Although not shown, in the state where the player closes the right hand and the retroreflective sheet 15R is not detected, that is, in the non-input state of the right hand, the cursor 70R in a form imitating the closed right hand is displayed. Thus, the player can recognize input / non-input by the form of the cursors 70L and 70R.

  When playing, the multimedia processor 50 randomly selects one fixed position object 100 from the twenty fixed position objects 100 and blinks the selected fixed position object 100. In the figure, blinking is indicated by hatching. In FIG. 5A, the fixed position object 100 of 3 rows and 2 columns is blinking, and in FIG. 5B, the fixed position object 100 of 1 row and 5 columns is displayed. It is flashing. The blinking home position object 100 may be called a target.

  In the input state, the player moves the cursor 70L or 70R to the blinking home position object 100. Then, the player places the cursor 70L or 70R on the blinking fixed position object 100 to make it non-input state. Then, the multimedia processor 50 newly selects the fixed position object 100 and blinks it. The multimedia processor 50 detects input / non-input of the player and repeats such processing for a predetermined time. Even if the player moves the cursor 70L or 70R to the non-flashing fixed position object 100 and puts it in the non-input state, the multimedia processor 50 maintains the flashing fixed position object 100 at that time, and sets a new fixed position object 100. The position object 100 is not blinked.

  The movement of the cursor 70L or 70R to the non-input state by moving the cursor 70L or 70R in the input state to the blinking fixed position object 100 may be called holding the target. The player reacts quickly to the target and tries to grasp as many targets as possible within a predetermined time.

  Here, in order to move the cursor 70L, the player must open the left hand to enter the input state. This is because the position cannot be specified unless the retroreflective sheet 15L can be detected. Therefore, when the player changes from the input state to the non-input state, the non-input state cursor 70L is displayed stationary at the position of the last cursor 70L in the input state. The same applies to the cursor 70R.

  As described above, in this mode, the player detects flashing as soon as possible and tries to grasp the target as soon as possible. Therefore, in this mode, the player can perform training on the ability (a kind of moving object recognition) of how quickly the target can appear one after another.

  [Ball escape mode]

  FIG. 6 is an exemplary view of a training screen in the ball escape mode by the training system of FIG. Referring to FIG. 6A, the multimedia processor 50 displays an object 102 that moves around on the screen of the television monitor 5. One object 102 appears first, and one more object is added after a predetermined time has elapsed. Thus, new objects 102 are added one after another as the predetermined time elapses. As shown in FIG. 6B, the multimedia processor 50 emits the newly added object 102 from the exit port object 104 in the direction indicated by the arrow object 106. The position of the exit port object 104 and the direction of the arrow object 106 are randomly selected and displayed in an arbitrary direction at an arbitrary position on the screen. That is, they are not in a fixed position and a fixed direction. The multimedia processor 50 performs such processing for movement and appearance of the object 102 for a predetermined time.

  The cursors 70L and 70R are the same as in the flash catch mode. However, in the ball escape mode, the player always operates the cursors 70L and 70R in the input state. The player attempts to avoid collision with the object 102 by operating the cursors 70L and 70R. Over time, the number of objects 102 increases, making it difficult to avoid. When the cursor 70L or 70R collides with the object 102, the multimedia processor 50 gives an effect to the object 102 and outputs a sound effect. The multimedia processor 50 counts the number of collisions and displays the result after a predetermined time.

  As described above, in this mode, the player operates the cursors 70L and 70R so as not to collide with the moving object 102. Therefore, in this mode, it is possible to perform training of the ability (a kind of moving object prediction) of how well the object 102 can be avoided. In addition, since the objects 102 appear one after another, the difficulty level can be gradually increased according to the number of the objects 102. Furthermore, since the position of the exit port object 104 and the direction of the arrow object 106 are random and the appearance position of the object 102 is not constant, the difficulty can be further improved.

  [First number catch mode]

  FIG. 7 is an exemplary view of a training screen in the first number catch mode by the training system of FIG. Referring to FIG. 7A, the multimedia processor 50 displays 20 stationary targets 110 with consecutive numbers (integers) 1 to 20 on the television monitor 5. The cursors 70L and 70R are the same as in the flash catch mode.

  The player operates the cursors 70L and 70R in the input state, overlaps the target 110, and enters the non-input state. In this specification, such an operation may be called holding the target 110. The player tries to grasp the target 110 as soon as possible, starting with the target 110 with a small value. As shown in FIG. 7B, when the target 110 is correctly grasped in order from the smallest number, the target 110 is extinguished. If the target 110 is not grasped in the correct order, the number of times is counted as a player failure.

  The multimedia processor 50 displays 20 targets 110 and starts counting time from the time when the player is notified of the start of play. Then, when the player grasps the target 110 with the numeral 20, the time counting is stopped and the result is displayed. Therefore, the player can grasp the time required to hold all the targets 110. The multimedia processor 50 also displays the number of failures along with the time count result.

  Referring to FIG. 7C, the multimedia processor 50 replaces the screen of FIG. 7A with ten targets 110 with discontinuous and different numbers (integers) attached to the television monitor 5. It can also be displayed. In this case as well, as in the case of FIG. 7A, the player tries to grasp the target 110 as soon as possible, starting with the target 110 with a smaller value. When the target 110 is correctly grasped in order from the smallest number, the target 110 is extinguished. If the target 110 is not grasped in the correct order, the number of times is counted as a player failure. In addition, the time count and the like are the same as in the case of FIG. Since the numbers assigned to the target 110 are discontinuous, the case of FIG. 7C is more difficult than the case of FIG.

  As described above, in this mode, the player attempts to grasp the target 110 as soon as possible in order from the smallest number. Therefore, in this mode, it is possible to perform training on the ability (a kind of instantaneous recognition) of how quickly the target 110 can be recognized in order.

  [Short-term memory mode]

  FIG. 8 is an exemplary diagram of a training screen in the short-term memory mode by the training system of FIG. Referring to FIG. 8A, the multimedia processor 50 displays a screen including 16 fixed position objects 120 arranged in a grid (4 rows × 4 columns) on the television monitor 5. The cursors 70L and 70R are the same as in the flash catch mode.

  First, the multimedia processor 50 sets any two fixed-position objects 120 to the first color (for example, red and a hatched portion that rises to the right) at the same time for a predetermined time (for example, 1 second). The fixed-position object 120 is set to the second color (for example, blue, a slanting portion with a lower right), and any two fixed-position objects 120 are set to the third color (for example, yellow, the crossed-hatched portion) (question) . The first to third colors may be called markers. Note that the standard color of the fixed position object 120 is white.

  Then, after a predetermined time has elapsed, the multimedia processor 50, the two newly selected fixed position objects 120 to which the first color is added and the two newly selected fixed position objects 120 to which the second color is added. The first to third colors are assigned simultaneously to the two newly selected fixed-position objects 120 to which the third color is to be added and for a predetermined time (question).

  The multimedia processor 50 repeats the above questions a predetermined number of times. After completing all the questions, the multimedia processor 50 displays the instruction unit 122 as shown in FIG. The instruction unit 122 determines the position of the fixed-position object 120 in which the color marker displayed on the instruction unit 122 is located last (that is, the last color marker displayed on the instruction unit 122 is displayed). Instructs the player to show. In addition, the number next to the marker displayed on the instruction unit 122 indicates the number N of markers of that color that are attached in one question. In the example of FIG. 8B, the instruction unit 122 indicates that the position of the home position object 120 where the last two (N = 2) markers of the third color (cross hatched portion) are located is indicated. Is directed against.

  The player moves the cursor 70L or 70R to the fixed position object 120 in which the marker instructed by the instruction unit 122 is finally located in the input state to make it non-input state (selection). Then, the multimedia processor 50 changes the color of the fixed position object 120 where the cursor 70L or 70R overlaps to a predetermined color (for example, black). When the player finishes selecting the number of fixed position objects 120 indicated by the instruction unit 122 (2 in the example in the figure), the player moves the cursor 70L or 70R in the input state on the determination button 124 displayed on the upper right of the screen. To enter the non-input state (selection confirmed). Then, the multimedia processor 50 determines whether or not the position of the fixed-position object 120 selected by the player completely matches the position of the marker of the color indicated by the instruction unit 122 that the multimedia processor 50 gave the last question. Judging. “Complete” means that when the number indicated by the instruction unit 122 is N, the position of the N fixed-position objects 120 selected by the player is the color indicated by the instruction unit 122 that the multimedia processor 50 has given the last question. This means that the positions of the N markers completely coincide with each other.

  If they do not match completely, the multimedia processor 50 determines that the operation has failed and ends the process. On the other hand, when they completely match, the multimedia processor 50 determines that the operation is successful and moves to the next stage. In each stage, a predetermined number of questions and instructions after the completion are given. In this embodiment, each time the stage is cleared, the questions are difficult and the difficulty level is high.

  Note that the player can redo the selection until the cursor 70L or 70R is moved to the enter button 124 to enter the non-input state (until the selection is confirmed). In this case, when the player puts the cursor 70L or 70R on the selected fixed-position object 120 and puts it in the non-input state, the selection of the fixed-position object 120 is released.

  In addition, moving the cursor 70L or 70R to the fixed position object 120 or the determination button 124 in the input state to make the non-input state may be referred to as holding the fixed position object 120 or the determination button 124.

  Here, the difficulty level will be described. The degree of difficulty can be adjusted by adjusting the number of markers and the number of colors that appear in one question. The difficulty level can also be adjusted by changing the color and / or number of markers between a predetermined number of questions. For example, only two markers of the first color can be displayed in one question, two markers of the second color can be displayed in the next question, and these questions can be repeated alternately.

  As described above, in this mode, the player tries to memorize the color and position of the marker that has been given the last question, and after all the questions have been given, the last position of the marker of the indicated color is set to the cursor 70L or 70R. Do the pointing by. Therefore, in this mode, it is possible to train the player's short-term memory (a kind of short-term memory). Further, the selection is confirmed only when the player holds the determination button 124. Therefore, it can be prevented as much as possible that the correct / incorrect determination is made based on the selection state not intended by the player.

  [Second number catch mode]

  The contents of the second number catch mode are the same as the contents of the first number catch mode. The second number catch mode has a feature in the adjustment of the difficulty level as compared with the first number catch mode. Hereinafter, different points will be mainly described.

  FIG. 9 is an exemplary view of a training screen in the second number catch mode by the training system of FIG. Referring to FIG. 9, multimedia processor 50 displays board 11 divided into 6 × 6 square blocks on television monitor 5. However, in the area corresponding to the 2 × 2 square block at the center of the board 11, a timer 13 indicating the remaining time until the end of play (for example, countdown from 30 seconds) is displayed.

  The multimedia processor 50 displays discontinuous and different numbers (integers) on a predetermined number of square blocks arbitrarily selected on the board 11. At this time, two types of large and small are prepared as the sizes of images representing numbers. If the size of the image representing the number is the same, the difficulty level becomes low. Therefore, the difficulty level can be adjusted by preparing two types. Of course, more than three types can be prepared. Further, the number may be displayed by rotating clockwise or counterclockwise by a predetermined angle. If the numbers are displayed correctly without rotation, the difficulty level becomes low. Therefore, the difficulty level can be adjusted by rotating and displaying the numbers.

  In this mode, since the player plays with one hand, the multimedia processor 50 displays only one cursor 70L. In the example shown in the figure, since the cursor 70L corresponding to the left hand is displayed, the player operates the cursor 70L with the left hand. Since the right hand is played before or after the left hand, the cursor 70R corresponding to the right hand is displayed.

  In this mode as well as in the first number catch mode, the player tries to grasp the number as soon as possible, starting from the smallest value. If it is grasped correctly in order from the smallest number, the number will disappear. If it is not grasped in the correct order, the player is notified by voice (for example, a buzzer sound), and the wrongly grasped number is enclosed in a blue frame and the correct number is enclosed in a red frame. Inform.

  As described above, in this mode, the player tries to grip as soon as possible in order from the smallest number. Therefore, in this mode, it is possible to train the ability (a kind of instantaneous recognition) of how quickly numbers can be recognized in order.

  [Third number catch mode]

  In the third number catch mode, the second number catch mode is performed simultaneously by two players.

  FIG. 10 is a view showing an example of a training screen in the third number catch mode by the training system of FIG. Referring to FIG. 10, multimedia processor 50 displays board 11 on television monitor 5. This board 11 is divided into right and left at the center, and includes a board 22-1 for the first player and a board 22-2 for the second player.

  Further, the multimedia processor 50 displays discontinuous and different numbers (integers) on a predetermined number of square blocks arbitrarily selected on the board 22-1. At the same time, the multimedia processor 50 displays the same number as the number displayed on the board 22-1 in the corresponding square block on the board 22-2. Completely identical refers to the same numerical value, size, and form (rotation).

  Further, in this mode, the multimedia processor 50 includes a cursor 24-1 linked to the input device 3L worn by the first player, for example, on the right hand, and a cursor 24-- linked to the input device 3R worn, for example, on the right hand, by the second player. 2 is displayed.

  In this mode as well, as in the second number catch mode, the first and second players try to grasp the numbers as soon as possible, starting with the numbers with the smallest values. Note that both start at the same time and follow the timer 13. Also, if the correct number is held in order from the smallest number, that number is erased. If it is not grasped in the correct order, the player is notified by voice (for example, a buzzer sound), and the wrongly grasped number is enclosed in a blue frame and the correct number is enclosed in a red frame. Inform.

  As described above, in this mode, the player tries to grip as soon as possible in order from the smallest number. Therefore, in this mode, it is possible to train the ability (a kind of instantaneous recognition) of how quickly numbers can be recognized in order. Moreover, since two players compete, it is possible to improve the player's concentration and motivation.

  [Butterfly catch mode]

  FIG. 11 is an illustration of a training screen in butterfly catch mode by the training system of FIG. Referring to FIG. 11, multimedia processor 50 displays three types of butterfly objects 29, 31, and 33 that fly around on television monitor 5. The sizes of the butterfly objects 29, 31 and 33 are large, small and medium, respectively. In particular, the butterfly object 33 is controlled to move more quickly than the other butterfly objects 29 and 31.

  In this mode, since the player plays with one hand, the multimedia processor 50 displays only one cursor 70R. In the example in the figure, the cursor 70R corresponding to the right hand is displayed, so the player operates the cursor 70R with the right hand. Since the left hand is played before or after the right hand is played, a cursor 70L corresponding to the left hand is displayed at that time.

  In this mode, the player attempts to grasp as many butterfly objects 29, 31 and 33 as possible in time (for example, 30 seconds). For time reference, the multimedia processor 50 displays a timer 27 (eg, a countdown from 30 seconds). Further, the multimedia processor 50 displays the acquisition number display unit 35. The number-of-acquisition display section 35 indicates how many butterfly objects have been grasped (acquired).

  In this mode, three levels are prepared. In the second and third levels, the butterfly objects 29, 31 and 33 are controlled to leave the cursor 70R (70L) when the distance from the cursor 70R (70L) reaches a predetermined distance. In this case, the butterfly objects 29, 31 and 33 are farther apart on the third level than on the second level. In the first level, the position of the cursor 70R (70L) is not considered in the trajectory of the butterfly objects 29, 31 and 33. The multimedia processor 50 controls the butterfly objects 29, 31 and 33.

  As described above, in this mode, the player tries to grip as many of the moving butterfly objects 29, 31, and 33 as possible. Thus, this mode allows training of agility and readability.

  [Stroop mode]

  The Stroop mode is a mode using a Stroop effect. The Stroop effect is a phenomenon in which sensory information (for example, color) and language information (for example, characters) interfere with each other in a human information processing process. For example, when you ask a question that answers the color name, the color name of the word “Midori” written in red ink is better than the color name of the word “Midori” written in green ink. It means that it takes more time to answer (red).

  12 (a) and 12 (b) are examples of training screens in the Stroop mode by the training system of FIG. With reference to FIGS. 12A and 12B, the multimedia processor 50 displays a hexagonal language information presentation unit 39 on the television monitor 5. The language information presentation unit 39 displays either “picture” or “voice”. The multimedia processor 50 displays sensory information presentation units 37T, 37B, 37L, and 37R on the top, bottom, left, and right of the language information presentation unit 39, and makes one of these colors different from the other three colors. Further, the multimedia processor 50 outputs one of the voices “we”, “sita”, “hidari”, or “migi”. However, a sound indicating a direction different from the direction in which the sensory information presenting unit having only one color is located is output.

  As shown in FIG. 12A, when the character “picture” is displayed on the language information presentation unit 39, the player operates the cursor 70R to provide four sensory information presentation units 37T, 37B, 37L, and 37R. Among them, try to grasp the sensory information display part with different colors as quickly as possible. In this case, if the sensory information presenting unit in the direction indicated by the voice is grasped, the operation fails.

  Six countdown objects 41 are displayed around the language information presentation unit 39, and the color of one countdown object 41 changes whenever a predetermined time elapses. In this case, starting from the countdown object 41 at the upper right, the color changes clockwise, and the play ends when the color of the countdown object 41 at the upper left changes.

  Further, as shown in FIG. 12B, when the character “voice” is displayed on the language information presentation unit 39, the player operates the cursor 70R to provide four sensory information presentation units 37T, 37B, and 37L. And 37R, an attempt is made to grasp the sensory information display unit located in the direction indicated by the sound output from the multimedia processor 50 as quickly as possible. In this case, if a sensory information presenting unit having only one color is gripped, a failure occurs.

  The multimedia processor 50 repeats the display as described above while changing the characters of the language information presentation unit 39 and the colors of the sensory information presentation units 37T, 37B, 37L, and 37R, and causes the player to play with the right hand. When the play with the right hand is completed, the cursor 70L corresponding to the left hand is displayed instead of the cursor 70R so that the play with the left hand is performed.

  As described above, in this mode, play using the Stroop effect can be performed. As a result, it is possible to train functions such as player action selection and interference rejection.

  [Panel escape mode]

  FIG. 13 is an exemplary view of a training screen in the panel escape mode by the training system of FIG. Referring to FIG. 13, the multimedia processor 50 displays a board 43 divided into 15 × 15 square blocks on the television monitor 5. In addition, the multimedia processor 50 displays a cursor 45 that is linked to the movement of the input device 3R, and a square slave object 47 that moves toward the cursor 45. In this case, the slave object 47 moves with one square block of the board 43 as one unit. However, the slave object 47 moves in the horizontal or vertical direction and does not move in an oblique manner. Furthermore, the multimedia processor 50 displays the target 49 on any of the square blocks of the board 43.

  The player tries to superimpose the slave object 47 on the target 49 by operating the cursor 45. When the slave object 47 is overlaid on the target 49, the multimedia processor 50 erases the target 49 and displays the new target 49 in another square block.

  In this case, the multimedia processor 50 causes the avoidance object 60H (for example, red) to appear from the left and right edges of the board 43, and moves the avoidance object 60H toward the other edge in the horizontal direction. In addition, the multimedia processor 50 causes the avoidance object 60V (for example, blue) to appear from the upper and lower edges of the board 43, and moves the avoidance object 60V in the vertical direction toward the other edge. When the avoidance objects 60H and 60V appear, an arrow indicating the traveling direction is written therein.

  When the slave object 47 is overlaid on the target 49, the multimedia processor 50 adds one point. When the slave object 47 touches the avoidance object 60H or 60V, one point is subtracted. . Therefore, the player needs to guide the slave object 47 to the target 49 by operating the cursor 45 while predicting the movement of the avoidance objects 60H and 60V. For this reason, training of the ability to read the player's tip can be performed.

  [Balance line mode]

  FIG. 14 is a view showing an example of a training screen in the balance line mode by the training system of FIG. Referring to FIG. 14, multimedia processor 50 displays bars 66L and 66R, bar-shaped operation object 63, and cursors 69L and 69R on television monitor 5. The multimedia processor 50 links the cursors 69L and 69R to the movements of the input devices 3L and 3R, respectively. Further, the multimedia processor 50 moves the left end of the operation object 63 up and down along the bar 66L according to the movement of the cursor 69L. On the other hand, the multimedia processor 50 moves the right end of the operation object 63 up and down along the bar 66R according to the movement of the cursor 69R. Although the operation object 63 is linear, since it expands and contracts, the operation object 63 can be made horizontal or slanted at an arbitrary angle by operating the cursors 69L and 69R.

  Further, the multimedia processor 50 causes the ball objects 65R, 65B, 65Y, and 65G to appear one after another from the center of the upper end of the screen in an arbitrary order and drop downward. The colors of the ball objects 65R, 65B, 65Y, and 65G are red, blue, yellow, and green, respectively. Corresponding to the ball objects 65R, 65B, 65Y and 65G, rectangular tray objects 75R, 75B, 75Y and 75G are displayed. The colors of the saucer objects 75R, 75B, 75Y, and 75G are red, blue, yellow, and green, respectively.

  The player operates the operation object 63 with the cursors 69L and 69R, hits the falling ball objects 65R, 65B, 65Y and 65G, and hits the corresponding saucer objects 75R, 75B, 75Y and 75G of the same color. Try. At this time, if the ball objects 65R, 65B, 65Y, and 65G can be hit back to the corresponding saucer objects 75R, 75B, 75Y, and 75G of the same color, 0 points, 2 points, 1 point, and 3 points are respectively obtained. Is added.

  As described above, it is necessary for the player to instantly determine the color of the falling ball objects 65R, 65B, 65Y, and 65G and to return to the appropriate tray objects 75R, 75B, 75Y, and 75G. For this reason, a player's instantaneous judgment power can be trained.

  [Rhythm ball mode]

  FIG. 15 is a view showing an example of a training screen in the rhythm ball mode by the training system of FIG. Referring to FIG. 15, multimedia processor 50 displays a path area 76 composed of 20 × 20 dots on television monitor 5. The multimedia processor 50 can change the shapes of the right path 78L and the left path 78R by changing the color of the dots in the path area 76.

  Regardless of the shape of the paths 78L and 78R, the paths 78L and 78R are set to extend to the receiving objects 72L and 72R displayed on the lower edge of the path area 76, respectively. The positions of the receiving objects 72L and 72R are fixed. In addition, the multimedia processor 50 displays cursors 69L and 69R. The receiving objects 72L and 72R contract in response to detection of the non-input state from the input state of the input devices 3L and 3R, respectively, and expand again in response to detection of the input state from the non-input state.

  The multimedia processor 50 causes the ball objects 74L and 74R to appear from the upper edge of the screen, respectively, and moves toward the receiving objects 72L and 72R along the paths 78L and 78R, respectively. In this case, the appearance timing and interval of the ball objects 74L and 74R are set according to the music to be output.

  The player operates the receiving objects 72L and 72R with the input devices 3L and 3R, and contracts the receiving objects 72L and 72R at the timing when the ball objects 74L and 74R reach the receiving objects 72L and 72R. Since the player needs to contract the receiving objects 72L and 72R at an appropriate timing in accordance with the ball objects 74L and 74R that move one after another, the player can train the judgment power and the ability to read ahead. In addition, since the ball objects 74L and 74R appear and move according to music, it is possible to train a sense of rhythm.

  [Shuffle memory mode]

  FIG. 16A to FIG. 16C are examples of training screens in the shuffle memory mode by the training system of FIG. Referring to FIG. 16A, the multimedia processor 50 displays five panels 86-1 to 86-5 on the television monitor 5. Different graphics are written on the panels 86-1 to 86-5. Simultaneously with the display of such panels 86-1 to 86-5, when the timer 80 starts counting down (for example, from 15 seconds) and the count becomes 0, as shown in FIG. 86-1 to 86-5 are turned over and the OK object 103 disappears. On the other hand, even before the timer 80 reaches 0, when the player operates the cursor 70R interlocked with the movement of the input device 3L and holds the OK object 103, as shown in FIG. 86-5 is turned over and the OK object 103 disappears. During the display of the screen of FIG. 16A, the player tries to memorize which figure is written on which panel 86-1 to 86-5.

  When the panels 86-1 to 86-5 are turned over, the multimedia processor 50 shuffles the turned-up panels 86-1 to 86-5 for the number of times indicated in the number-of-times explicit object 82. In the example shown in the figure, the panels 86-1 and 86-5 are exchanged, and the panels 86-3 and 86-4 are exchanged (double shuffle).

  After the end of the shuffle, the multimedia processor 50 displays the question object 91 as shown in FIG. In the question object 91, a graphic written on any one of the panels 86-1 to 86-5 is written. The player operates the cursor 70R to hold the panel 86-1, 86-2, 86-3 or 86-5 on which the same graphic as the graphic written on the question object 91 is written. If you can hold the appropriate panel, it will be correct, otherwise it will be incorrect.

  The display of the screens shown in FIGS. 16A to 16C is repeatedly performed while changing the figure written on the question object 91 and the number of shuffles. As the number of shuffles increases, memory becomes difficult and the difficulty level increases. Also, increasing the number of panels increases the difficulty level, and decreasing the number decreases the difficulty level.

  As described above, the player must memorize the arrangement of the panels 86-1 to 86-5 and the graphic described there within a predetermined time, and memorize the replacement of the panels. I must. For this reason, a player's memory can be trained.

  Next, a selection screen for mode selection will be described.

  FIG. 17 is a view showing an example of a mode selection screen by the training system of FIG. Referring to FIG. 17A, the multimedia processor 50 displays a selection screen including direction buttons 130L and 130R, a selection target 132, and cursors 70L and 70R on the television monitor 5. The cursors 70L and 70R are the same as in the flash catch mode.

  As shown in FIG. 17B, when the player moves the cursor 70L or 70R onto the direction button 130R in the input state to make it non-input, the multimedia processor 50 rotates the selection target 132 in the right direction, The selection target 132 on the left is displayed. On the other hand, when the player moves the cursor 70L or 70R to the non-input state by moving the cursor 70L or 70R in the input state, the multimedia processor 50 rotates the selection target 132 to the left and displays the selection target 132 on the right. To do. The player displays a desired selection target 132 by such an operation. When the player moves the cursor 70L or 70R onto the selection target 132 in the input state and puts it into the non-input state, the multimedia processor 50 executes a mode corresponding to the selection target 132.

  Here, moving the cursor 70L or 70R to the direction button 130L or 130R or the selection target 132 in the input state to make it non-input may be referred to as holding the direction button 130L or 130R or the selection target 132.

  By the way, there is a mode for playing one hand at a time, such as the second number catch mode. In this case, it may be determined in advance which of the left and right hands is used first, but the player can also select. The left / right selection screen in this case will be described.

  FIG. 18 is a view showing an example of a left / right selection screen by the training system of FIG. Referring to FIG. 18, the left / right selection screen includes a left hand object 88L that imitates the left hand and a right hand object 88R that imitates the right hand. In addition, cursors 70L and 70R that are linked to the movements of the input devices 3L and 3R are displayed.

  The player operates one of the cursors 70L or 70R to hold either the left hand object 88L or the right hand object 88R. Then, the cursor 70L or 70R corresponding to the grasped object 88L or 88R is displayed on the training screen. Therefore, the player can play with a desired hand.

  Next, another mounting method of the input device 3 will be described.

  FIG. 19 is a diagram showing another example of mounting the input device 3 of FIG. Referring to FIG. 19, the player puts the input device 3 with his / her middle finger passing through belt 19 in FIG. 2. In this case, the transparent body 17 and the retroreflective sheet 15 are placed on the back of the hand. This is different from the case of FIG. Accordingly, when the player turns the back of the hand toward the information processing apparatus 1, that is, the image sensor 54, the transparent body 17, that is, the retroreflective sheet 15 appears, and the retroreflective sheet 15 is photographed (input state). On the other hand, when the palm is turned to the image sensor 54, the transparent body 17, that is, the retroreflective sheet 15 is hidden and is not photographed by the image sensor 54 (non-input state). Therefore, the player performs input / non-input control on the information processing device 1 by rotating the hand around the elbow as a fulcrum and the arm as an axis so as to cause the retroreflective sheet 15 to be taken or not taken. be able to. In each of the above modes, the player can be made to perform input / non-input by such a method.

  FIG. 20 is a diagram showing still another example of mounting the input device 3 of FIG. Referring to FIG. 20, the player puts the input device 3 with his / her middle finger passing through belt 19 in FIG. In this case, the transparent body 17 and the retroreflective sheet 15 are placed on the back of the hand. Then, the player clasps his hand. Therefore, when the player directs the fist toward the information processing apparatus 1, that is, the image sensor 54, the transparent body 17, that is, the retroreflective sheet 15 appears, and the retroreflective sheet 15 is photographed (input state). On the other hand, when the fist is hidden from the image sensor 54 (for example, the finger backs of the left and right hands are aligned), the transparent body 17, that is, the retroreflective sheet 15 is not photographed by the image sensor 54 (non-input state). Therefore, the player can perform input / non-input control on the information processing apparatus 1 by changing the direction of the fist so that the retroreflective sheet 15 is taken or not taken. In each of the above modes, the player can be made to perform input / non-input by such a method.

  As described above, according to the present embodiment, when the player images the retroreflective sheet 15, it is determined as an input state, and when the retroreflective sheet 15 is not captured, it is determined as a non-input state. Therefore, by displaying an image (for example, FIGS. 5, 7, and 8) that causes the player to repeatedly perform the input state and the non-input state, the player images the retroreflective sheet 15. It is necessary to repeatedly create a state and a state where no image is captured. Along with this, the player moves according to such a change in the state. As a result, the player can exercise and can support health maintenance / promotion and rehabilitation. Further, since the player uses both hands, the player can perform an exercise of moving both hands in a balanced manner.

  In addition, when the video displayed on the television monitor 5 is related to a predetermined human ability (for example, FIG. 5 to FIG. 8), the predetermined human ability training can be performed. Here, the predetermined ability is a series of sensory organs such as eyes → sensory nerves → brain → motor nerves → parts of the body such as hands, such as dynamic body recognition ability, dynamic body prediction ability, instantaneous recognition ability, and short-term memory ability. Includes transmission and storage functions.

  In this case, the retroreflective sheet 15 mounted on the palm side of the player is imaged in relation to the mounting example of FIG. Accordingly, the player can open the hand and take an image of the retroreflective sheet 15 to enter the input state, and hold the hand to hide the retroreflective sheet 15 and enter the non-input state. In this case, open hands (Janken's "Pa") and clenching (Janken's "Guu") are repeated. Inevitably, such an operation is performed by extending the arm forward to some extent or completely. Therefore, it is possible to train nerves and muscles that depend on actions that are not often performed in daily life.

  Further, in this case, the retroreflective sheet 15 mounted on the back of the player's hand is imaged in relation to the mounting example of FIG. Accordingly, the player may enter the input state by imaging the retroreflective sheet 15 with the back side of the hand facing the image sensor 54, and hide the retroreflective sheet 15 with the palm side facing the image sensor 54 to enter the non-input state. it can. In this case, the hand is rotated about the elbow as a fulcrum and the arm as an axis. Inevitably, such an operation is performed by extending the arm forward to some extent or completely. Therefore, it is possible to train nerves and muscles that depend on actions that are not often performed in daily life.

  Furthermore, in relation to the wearing example of FIG. 11, in this case, the retroreflective sheet 15 attached to the back of the hand with the hand clamped is imaged. Therefore, the player points the fist toward the image sensor 54 and images the retroreflective sheet 15 to be in the input state, and hides the fist from the image sensor 54 (for example, by aligning the finger backs of the left and right hands) to the non-input state. Can do. Therefore, it is possible to train nerves and muscles that depend on actions that are not often performed in daily life.

  Next, detection processing and left / right determination processing of the retroreflective sheets 15L and 15R will be described with reference to the drawings.

  FIG. 21 is an explanatory diagram of detection processing of the retroreflective sheets 15L and 15R. FIG. 21 shows a difference image (32 × 32 pixels) based on difference image data generated from image data when infrared light is emitted and when the light is turned off. In the figure, a small square represents one pixel. The upper left corner is the origin of the XY coordinate axes.

  This image includes two regions 251 and 253 having large luminance values. Regions 251 and 253 are retroreflective sheets 15L and 15R. However, at this time, it cannot be determined which region corresponds to which retroreflective sheet.

  First, the multimedia processor 50 scans the difference image data from X = 0 to X = 31 with Y = 0 as the starting point, and then increments Y to make the difference from X = 0 to X = 31. Scan image data. Such processing is performed up to Y = 31, and the difference image data of 32 × 32 pixels is scanned to obtain the upper end position minY, lower end position maxY, left end position minX, and right end position maxX of the pixel data larger than the threshold ThL.

  Next, the multimedia processor 50 executes a scan in the positive direction of the X axis with the coordinates (minX, minY) as a starting point, and first calculates a distance LT to a pixel exceeding the threshold ThL. Further, the multimedia processor 50 executes a scan in the negative direction of the X axis starting from the coordinates (maxX, minY), and first calculates a distance RT to a pixel exceeding the threshold ThL. Further, the multimedia processor 50 performs a scan in the positive direction of the X axis with the coordinates (minX, maxY) as a starting point, and first calculates a distance LB to a pixel exceeding the threshold ThL. Further, the multimedia processor 50 executes a scan in the negative direction of the X axis starting from the coordinates (maxX, maxY), and first calculates a distance RB to a pixel exceeding the threshold ThL.

  The multimedia processor 50 sets the coordinates (maxX, minY) as the first extraction point when the distance LT> RT, and sets the coordinates (minX, minY) as the first extraction point when the distance LT ≦ RT. In addition, when the distance LB> RB, the multimedia processor 50 sets the coordinates (maxX, maxY) as the second extraction point, and when the distance LB ≦ RB, sets the coordinates (minX, maxY) as the second extraction point. .

  FIG. 22 is an explanatory diagram of the left / right determination process. FIG. 22 shows the position TPL2 of the retroreflective sheet 15L of the previous time (one video frame before) and the position TPL1 of the previous time (2 video frames before), the position TPR2 of the retroreflective sheet 15R of the previous time (1 video frame before), and A position TPR1 two times before (one video frame before) is shown. Positions TPL1, TPL2, TPR1, and TPR2 are positions on the difference image based on the difference image data.

  The multimedia processor 50 calculates a velocity vector VL starting from the position TPL1 and ending at the position TPL2. Then, the end point of the velocity vector VL starting from the position TPL2 is set as the predicted position TPLp of the retroreflective sheet 15L. On the other hand, the multimedia processor 50 calculates a velocity vector VR starting from the position TPR1 and ending at the position TPR2. Then, the end point of the velocity vector VR starting from the position TPR2 is set as the predicted position TPRp of the retroreflective sheet 15R.

  The multimedia processor 50 includes a distance LD1 between the first extraction point TPN1 and the predicted position TPLp, a distance RD1 between the first extraction point TPN1 and the predicted position TPRp, a distance LD2 between the second extraction point TPN2 and the predicted position TPLp, and A distance RD2 between the second extraction point TPN2 and the predicted position TPRp is obtained.

  If the distance LD1> RD1, the multimedia processor 50 sets the first extraction point TPN1 as the current position of the retroreflective sheet 15R. If the distance LD1 ≦ RD1, the multimedia processor 50 sets the first extraction point TPN1 as the current position of the retroreflective sheet 15L. And Further, if the distance LD2> RD2, the multimedia processor 50 sets the second extraction point TPN2 as the current position of the retroreflective sheet 15R, and if the distance LD2 ≦ RD2, sets the second extraction point TPN2 as the current position of the retroreflective sheet 15L. The position of

  Thus, since the left and right are assigned to the first extraction point TPN1 and the second extraction point TPN2 based on the left and right predicted positions TPLp and TPRp, the left and right of the retroreflective sheet 15L and the retroreflective sheet 15R are switched. However (even if crossed), the multimedia processor 50 can accurately recognize each of the retroreflective sheets 15L and 15R on the difference image based on the difference image data.

  Next, the flow of processing by the multimedia processor 50 will be described using a flowchart.

  FIG. 23 is a flowchart showing an example of the overall processing flow by the multimedia processor 50 of FIG. Referring to FIG. 23, in step S1, multimedia processor 50 performs initial setting of the system such as initialization of various variables (including flags and counters). In step S <b> 3, the multimedia processor 50 executes processing according to the application program stored in the external memory 52. In step S5, the multimedia processor 50 stands by until an interrupt due to the video synchronization signal occurs. That is, the multimedia processor 50 returns to the same step S5 when the interrupt due to the video synchronization signal has not occurred, and proceeds to step S7 when the interrupt due to the video synchronization signal has occurred. For example, the interruption by the video synchronization signal occurs every 1/60 seconds. In synchronization with this interruption, in steps S7 and S9, the multimedia processor 50 updates the image displayed on the television monitor 5 and reproduces the sound. Then, the multimedia processor 50 returns to step S3.

  The application program that controls the processing in step S3 includes a plurality of programs. The plurality of programs include a program that executes processing shown in the flowchart below.

  FIG. 24 is a flowchart showing an example of the flow of the main part of the processing by the application program in step S3 of FIG. Referring to FIG. 24, in step S21, the multimedia processor 50 executes the photographing process of the retroreflective sheets 15L and 15R. In step S23, the multimedia processor 50 detects the images of the retroreflective sheets 15L and 15R based on the image obtained in step S21. In step S25, the multimedia processor 50 determines whether or not a process of appropriately grasping the target has been performed. In step S27, the multimedia processor 50 executes control of video displayed on the television monitor 5 and other necessary calculations.

  Here, the target means an object to be grasped by the player. In the above example, the fixed position objects 100 and 120, the target 110, the numbers in FIGS. 9 and 10, the butterfly objects 29, 31 and 33, 37, and the senses. Information presentation units 37T, 37B, 37L and 37R, ball objects 74L and 74R, 84, panels 86-1 to 86-5, an OK object 103, and direction buttons 130L and 130R.

  In addition, the grasping of the target means that the player moves the cursor to the target in the input state and puts it into the non-input state.

  FIG. 25 is a flowchart showing an example of the flow of the photographing process in step S21 of FIG. As shown in FIG. 25, in step S41, the multimedia processor 50 turns on the infrared light emitting diode 9. In step S43, the multimedia processor 50 acquires image data when the infrared light is turned on from the image sensor 54, and stores it in the main RAM.

  Here, in this embodiment, a CMOS image sensor of 32 pixels × 32 pixels is used as an example of the image sensor 54. Therefore, pixel data of 32 pixels × 32 pixels is output from the image sensor 54 as image data. This pixel data is converted into digital data by an A / D converter and stored as an element of a two-dimensional array P1 [X] [Y] on the main RAM.

  In step S45, the multimedia processor 50 turns off the infrared light emitting diode 9. In step S47, the multimedia processor 50 acquires image data (32 pixel × 32 pixel pixel data) when the infrared light is extinguished from the image sensor 54, and stores it in the main RAM. In this case, the pixel data is stored as an element of a two-dimensional array P2 [X] [Y] on the main RAM.

  The flash photography is performed as described above. Here, in the two-dimensional coordinate system representing the position of each pixel constituting the image by the image sensor 54, the horizontal direction is the X axis and the vertical direction is the Y axis. The origin O is the upper left corner of the image. In this embodiment, since the image sensor 54 of 32 pixels × 32 pixels is used, X = 0 to 31 and Y = 0 to 31. This also applies to the difference image DI. The pixel data is a luminance value.

  FIG. 26 is a flowchart showing an example of the flow of the sheet detection process in step S23 of FIG. As shown in FIG. 26, in step S61, the multimedia processor 50 causes the pixel data P1 [X] [Y] when emitting infrared light and the pixel data P2 [X] [Y] when turning off infrared light. And is substituted into the array Dif [X] [Y]. In step S63, the multimedia processor 50 proceeds to step S65 when the difference of 32 × 32 pixels is calculated, and proceeds to step S61 otherwise. As described above, the multimedia processor 50 repeats the process of step S61 to generate difference image data between the image data when the infrared light is emitted and the image data when the infrared light is turned off. Thus, by obtaining the difference image data (difference image), noise due to light other than the reflected light from the retroreflective sheets 15L and 15R can be removed as much as possible, and the retroreflective sheets 15L and 15R can be detected with high accuracy.

  In step S65, the multimedia processor 50 executes the left and right upper and lower end (minX, maxX, minY, maxY) detection processing described in FIG.

  FIG. 27 is a flowchart showing an example of the flow of left / right upper / lower end detection processing in step S65 of FIG. As shown in FIG. 27, in step S91, the multimedia processor 50 assigns “0” to “X”, “Y”, “maxX”, “maxY”, “k”, and “SN”. Also, the multimedia processor 50 substitutes “31” for “minX” and “minY”.

  In step S93, the multimedia processor 50 compares the elements of the array Dif [X] [Y] with a predetermined threshold ThL. In step S95, the multimedia processor 50 proceeds to step S97 if the element of the array Dif [X] [Y] is greater than the predetermined threshold ThL, and proceeds to step S121 if it is less than or equal to the predetermined threshold ThL.

  The processes in steps S93 and S95 are processes for detecting whether or not the retroreflective sheets 15L and 15R are photographed. When the retroreflective sheets 15L and 15R are photographed, the luminance values of pixels corresponding to the retroreflective sheets 15L and 15R increase on the difference image. For this reason, the brightness value is distinguished by the threshold value ThL, and a pixel having a brightness value larger than the threshold value ThL is recognized as part of the photographed retroreflective sheets 15L and 15R.

  In step S97, the multimedia processor 50 increments the count value k by one. In step S99, the multimedia processor 50 determines whether or not the count value k is “1”. If k = 1, the process proceeds to step S101. Otherwise, the process proceeds to step S103.

  In step S101, the multimedia processor 50 substitutes the current Y coordinate for the minimum Y coordinate minY. In other words, the scan repeats the process of starting from (X, Y) = (0, 0), performing X = 0 to 31, incrementing Y, and again performing X = 0 to 31. (Refer to Steps S121 to S129 described later) “Y” of the array Dif [X] [Y] having an element (that is, a pixel) that first exceeds the threshold ThL is the minimum Y coordinate minY.

  In step S103, the multimedia processor 50 compares the current maximum Y coordinate maxY with the current Y coordinate. In step S105, the multimedia processor 50 proceeds to step S107 when the current Y coordinate is larger than the current maximum Y coordinate maxY, and proceeds to step S109 otherwise. In step S107, the multimedia processor 50 substitutes the current Y coordinate for the maximum Y coordinate maxY.

  In step S109, the multimedia processor 50 compares the current minimum X coordinate minX with the current X coordinate. In step S111, the multimedia processor 50 proceeds to step S113 if the current X coordinate is smaller than the current minimum X coordinate minX, and proceeds to step S115 otherwise. In step S113, the multimedia processor 50 substitutes the current X coordinate for the minimum X coordinate minX.

  In step S115, the multimedia processor 50 compares the current maximum X coordinate maxX with the current X coordinate. In step S117, the multimedia processor 50 proceeds to step S119 if the current X coordinate is larger than the current maximum X coordinate maxX, and proceeds to step S121 otherwise. In step S119, the multimedia processor 50 substitutes the current X coordinate for the maximum X coordinate maxX.

  In step S121, the multimedia processor 50 increments “X” by one. In step S123, the multimedia processor 50 proceeds to step S125 when X = 32 (that is, when the processing for one row of pixels of the difference image is completed), and proceeds to step S93 otherwise.

  In step S125, the multimedia processor 50 substitutes “0” for “X”. In step S127, the multimedia processor 50 increments “Y” by one. The processes in steps S125 and S127 are executed to advance the process for the next line because the process for one line of the difference image has been completed.

  In step S129, the multimedia processor 50 proceeds to step S131 when Y = 32 (that is, when processing of 32 × 32 pixels of the difference image is completed), and proceeds to step S93 otherwise.

  By repeating the above steps S93 to S129, when Y = 32, the minimum X coordinate minX, the maximum X coordinate maxX, the minimum Y coordinate minY, and the maximum Y coordinate maxY are all determined.

  In step S131, it is determined whether or not “maxX”, “maxY”, “minX”, and “minY” remain the initial values. If they are the initial values, the process proceeds to step S133, and otherwise returns. The fact that “maxX”, “maxY”, “minX”, and “minY” are all initial values means that all pixels of the difference image are equal to or less than the threshold ThL, and the retroreflective sheets 15L and 15R are not photographed. To do. Accordingly, in step S133, the multimedia processor 50 substitutes “01” (indicating that the number of photographed retroreflective sheets is zero) in the sheet number flag SN indicating the number of photographed retroreflective sheets. To do. In step S135, the multimedia processor 50 turns off the left open flag OL and the right open flag OR, and proceeds to step S77 in FIG. The left open flag OL and the right open flag OR are flags indicating that the retroreflective sheets 15L and 15R are photographed, respectively.

  Returning to FIG. 26, in step S67, the multimedia processor 50 determines the two point positions (first extraction point (Xtp [0], Ytp [0]) and second extraction point (Xtp [1]) described in FIG. ], Ytp [1])) are executed.

  FIG. 28 is a flowchart showing an example of the flow of the two-point position determination process in step S67 of FIG. As shown in FIG. 28, the multimedia processor 50 substitutes “0” for “M” in step S151, and repeats the processing from step S153 to step S185. Here, as shown in step S153, Ytb = minY in the first loop, and Ytb = maxY in the second loop. In step S155, the multimedia processor 50 starts scanning with the coordinates (minX, Ytb) as a starting point.

  In step S157, the multimedia processor 50 substitutes “0” for the count value Cl. In step S159, the multimedia processor 50 compares the difference data Dif [X] [Y] with the threshold value ThL. If the difference data is larger than the threshold value, the multimedia processor 50 proceeds to step S165. Otherwise, the multimedia processor 50 proceeds to step S161. move on. In step S161, the multimedia processor 50 increments the count value Cl by one. In step S163, the multimedia processor 50 increments the coordinate X by one, and proceeds to step S159.

  The count value Cl when it is determined in step S159 that Dif [X] [Y]> ThL corresponds to the distance LT or LB in FIG. In step S155, when Ytb = minY, Cl = LT, and when Ytb = maxY, Cl = LB.

  In step S165, the multimedia processor 50 starts scanning using the coordinates (maxX, Ytb) as a starting point. In step S167, the multimedia processor 50 substitutes “0” for the count value Cr. In step S169, the multimedia processor 50 compares the difference data Dif [X] [Y] with the threshold value ThL. If the difference data is larger than the threshold value, the multimedia processor 50 proceeds to step S175. Otherwise, the multimedia processor 50 proceeds to step S171. move on. In step S171, the multimedia processor 50 increments the count value Cr by one. In step S173, the multimedia processor 50 decrements the coordinate X by one, and proceeds to step S169.

  The count value Cr at the time when it is determined in step S169 that Dif [X] [Y]> ThL corresponds to the distance RT or RB in FIG. In step S165, when Ytb = minY, Cr = RT, and when Ytb = maxY, Cr = RB.

  In step 175, the multimedia processor 50 compares the distances Cl and Cr. If the distance Cl is greater than the distance Cr in step S177, the process proceeds to step S179, and otherwise, the process proceeds to step S181.

  In step S181, “minX” is substituted for “Xtp [M]” and “Ytb” is substituted for “Ytp [M]”. On the other hand, in step S179, “maxX” is substituted for “Xtp [M]” and “Ytb” is substituted for “Ytp [M]”.

  Here, the coordinates (Xtp [0], Ytp [0]) are the coordinates of the first extraction point described in FIG. 21, and the coordinates (Xtp [1], Ytp [1]) are the coordinates of the second extraction point. Coordinates.

  In step S183, the multimedia processor 50 increments “M” by one, and proceeds to step S185. When the loop from step S153 to step S185 ends, the process returns.

  Returning to FIG. 26, in step S69, the multimedia processor 50 executes a process of assigning left and right to the first extraction point and the second extraction point.

  FIG. 29 is a flowchart showing an example of the left / right determination process in step S69 of FIG. The position of the retroreflective sheet 15L is called a left extraction point, and the position of the retroreflective sheet 15R is called a right extraction point.

  As shown in FIG. 29, in step S201, the multimedia processor 50 determines the position (Xnl, Ynl) of the current left extraction point from the position (XL, YL) of the previous left extraction point (XL, YL). Predict. In step S203, the multimedia processor 50 predicts the position (Xnr, Ynr) of the current right extraction point from the past (previous and previous) right extraction point positions (XR, YR). Here, the left extracted point (Xnl, Ynl) corresponds to the predicted position TPLp in FIG. 22, and the position of the right extracted point (Xnr, Ynr) corresponds to the predicted position TPRp.

  In step S205, the multimedia processor 50 substitutes “0” for “M”. In step S207, the multimedia processor 50 calculates a distance Dl between the predicted position (Xnl, Ynl) and the extraction point (Xtp [M], Ytp [M]). In step S209, the multimedia processor 50 calculates a distance Dr between the predicted position (Xnr, Ynr) and the extraction point (Xtp [M], Ytp [M]).

  Here, the extraction points (Xtp [0], Ytp [0]) are the first extraction points obtained by the routine of FIG. 28, and the extraction points (Xtp [1], Ytp [1]) are This is the second extraction point obtained by the routine. When M = 0, the distance Dl corresponds to the distance LD1 in FIG. 22, and the distance Dr corresponds to the distance RD1. When M = 1, the distance Dl corresponds to the distance LD2 in FIG. 22, and the distance Dr corresponds to the distance RD2.

  In step S211, the multimedia processor 50 compares the distance Dl with the distance Dr. In step S213, the multimedia processor 50 proceeds to step S215 if Dl> Dr, otherwise proceeds to step S217.

  In step S217, the multimedia processor 50 sets the position (XL, YL) of the current left extraction point as coordinates (Xtp [M], Ytp [M]).

  On the other hand, in step S215, the multimedia processor 50 sets the position (XR, YR) of the current right extraction point as coordinates (Xtp [M], Ytp [M]).

  In step S219, the multimedia processor 50 increments “M” by one. In step S221, the multimedia processor 50 determines whether or not M = 2. If M = 2, the process returns, otherwise the process proceeds to step S207.

  Returning to FIG. 26, in step S71, the multimedia processor 50 calculates the midpoint (XM, YM) of the left extraction point (XL, YL) and the right extraction point (XR, YR).

  In step S73, the multimedia processor 50 executes processing for determining the number of detected retroreflective sheets.

  FIG. 30 is a flowchart showing an example of the detection sheet number determination process in step S73 of FIG. Referring to FIG. 30, in step S241, multimedia processor 50 compares Y coordinate YL of the left extraction point with Y coordinate YR of the right extraction point. In step S243, the multimedia processor 50 proceeds to step S251 if the Y coordinate YL is large, otherwise proceeds to step S247.

  In step S251, the multimedia processor 50 scans the difference image along the diagonal line that rises to the right. On the other hand, in step S247, the multimedia processor 50 scans the difference image along the diagonal line descending to the right.

  In step S253, when the multimedia processor 50 detects a pixel having a threshold value ThL or less as a result of the scan in step S247 or S251, the multimedia processor 50 considers that the two retroreflective sheets 15L and 15R have been photographed, and proceeds to step S255. If no pixel equal to or less than the threshold ThL is detected, it is considered that only one of the retroreflective sheets 15L and 15R has been detected, and the process proceeds to step S259.

  In step S255, the multimedia processor 50 sets “11” (indicating that the two retroreflective sheets 15L and 15R have been photographed) to the sheet number flag SN. In step S257, the multimedia processor 50 turns on the left open flag OL and the right open flag OR. On the other hand, in step S259, the multimedia processor 50 sets “10” (indicating that only one of the retroreflective sheets 15L and 15R has been captured) to the sheet number flag SN.

  Returning to FIG. 26, in step S75, when only one of the retroreflective sheets 15L and 15R is photographed, the multimedia processor 50 identifies which one is photographed (one point identifying process). ).

  FIG. 31 is a flowchart showing an example of the flow of the one-point specifying process in step S75 of FIG. Referring to FIG. 31, in step S271, the multimedia processor 50 determines whether or not the sheet number flag SN is “10”, that is, only one of the retroreflective sheets 15L and 15R has been photographed. If YES, the process proceeds to step S273 to identify the left or right retroreflective sheet. Otherwise, the process returns.

  In step S273, the multimedia processor 50 calculates the distance Dlm between the predicted position (Xnl, Ynl) and the midpoint (XM, YM). In step S275, the multimedia processor 50 calculates a distance Drm between the predicted position (Xnr, Ynr) and the midpoint (XM, YM). The midpoint (XM, YM) is obtained in step S71 of FIG. The predicted position (Xnl, Ynl) is obtained in step S201 of FIG. The predicted position (Xnr, Ynr) is obtained in step S203 in FIG.

  In step S277, the multimedia processor 50 compares the distance Dlm with the distance Drm. In step S279, the multimedia processor 50 proceeds to step S281 if Dlm> Drm, otherwise proceeds to step S285.

  In step S281, the multimedia processor 50 sets the position (XL, YL) of the current left extraction point as coordinates (0, 0), and sets the position (XR, YR) of the current right extraction point as coordinates (XM, YM). In step S283, the multimedia processor 50 turns off the left open flag OL, turns on the right open flag OR, and returns.

  On the other hand, in step S285, the multimedia processor 50 sets the current position of the left extracted point (XL, YL) as coordinates (XM, YM) and the current position of the right extracted point (XR, YR) as coordinates (X 0,0). In step S287, the multimedia processor 50 turns on the left open flag OL, turns off the right open flag OR, and returns.

  Returning to FIG. 26, in step S77, the multimedia processor 50 converts the left extracted point (XL, YL) and the right extracted point (XR, YR) into screen coordinates and returns. In the case of continuing to step S135 in FIG. 27, the multimedia processor 50 converts the left extraction point (XL, YL) and the right extraction point (XR, YR) to (0, 0), respectively, and converts them into screen coordinates.

  Next, details of the catch determination process in step S25 of FIG. 24 will be described.

  FIG. 32 is a flowchart showing an exemplary flow of the catch determination process in step S25 of FIG. Referring to FIG. 32, in step S301, the multimedia processor 50 turns off the left grip trigger and the right grip trigger. The left grip trigger is a flag that is turned on when the cursor 70L moves to the target in the input state and is in a non-input state (corresponding to the player gripping the left hand wearing the input device 3L). The right grip trigger is a flag that is turned on when the cursor 70R moves to the target in the input state and is in a non-input state (corresponding to the player gripping the right hand wearing the input device 3R).

  In step S303, the multimedia processor 50 determines whether or not the left open flag OL has transitioned from on to off, and if so, proceeds to step S305 assuming that the player has clasped the left hand, otherwise The process proceeds to step S309. In step S305, the multimedia processor 50 determines whether or not the immediately preceding left extraction point is located on the target. If so, the multimedia processor 50 considers that the target has been grasped and proceeds to step S307. In step S307, the multimedia processor 50 turns on the left grip trigger.

  In step S309, the multimedia processor 50 determines whether or not the right open flag OR has transitioned from on to off, and if so, proceeds to step S311 assuming that the player has clasped the right hand, otherwise returns. To do. In step S311, the multimedia processor 50 determines whether or not the immediately preceding right extraction point is located on the target. If so, the multimedia processor 50 considers that the target has been grasped and proceeds to step S313. In step S313, the multimedia processor 50 turns on the right grip trigger and returns.

  Next, details of the video control process in step S27 of FIG. 24 will be described.

  FIG. 33 is a flowchart showing an example of the flow of the video control process in step S27 of FIG. The process shown in this flowchart is a process when the modes of FIGS. 5, 7, 8, 9, 10, 10, 12, and 16 are executed.

  Referring to FIG. 33, in step S331, the multimedia processor 50 refers to the left grip trigger and the right grip trigger, and determines whether or not the process of properly grasping the target is performed according to the corresponding mode. Judge and evaluate. In step S333, the multimedia processor 50 controls the target displayed on the television monitor 5 or the target to be displayed (setting of coordinates and image data, etc.). The targets are, for example, fixed position objects 100 and 120, target 110, numbers in FIGS. 9 and 10, butterfly objects 29, 31 and 33, 37, sensory information presentation units 37T, 37B, 37L and 37R, and ball objects 74L and 74R. , 84, panels 86-1 to 86-5, OK object 103, and direction buttons 130L and 130R. Note that a change can be given to the target according to the result of step S331, and a change can be given to the target when it is determined that the target has been grasped.

  In step S335, the multimedia processor 50 sets the coordinates of the cursor 70L at a position corresponding to the left extracted point (XL, YL) converted into the screen coordinates, and the right extracted point (XR, Y) converted into the screen coordinates. The coordinates of the cursor 70R are set at a position corresponding to YR). The multimedia processor 50 sets the image data of the cursor 70L with the palm open when the left open flag OL is on, and the cursor with the hand clasped when the left open flag OL is off. 70L of image data is set. Further, the multimedia processor 50 sets the image data of the cursor 70R with the palm open when the right open flag OR is on, and the cursor with the hand clasped when the right open flag OR is off. 70R image data is set.

  In step S337, the multimedia processor 50 controls the effect image (setting of coordinates and image data, etc.) according to the result of step S331. In step S339, the multimedia processor 50 controls images other than those described above (such as setting coordinates and image data). For example, the images to be controlled are the background image and the images 122, 124, 13, 27, 35, 41, 39, and 91.

  As described above, according to the present embodiment, the user moves the cursor with the palm open, overlaps the target, and claws the hand so as to hide the retroreflective sheet. A change can be applied to the cursor. That is, the user can change the target and / or cursor by moving the cursor to the target and clenching the hand. Therefore, the user can give an input by an operation as if holding the target.

  FIG. 34 is a flowchart showing another example of the flow of the video control process in step S27 of FIG. The process shown in this flowchart is a process when the mode of FIG. 14 is executed. In this case, step S25 in FIG. 24 may be skipped.

  Referring to FIG. 34, in step S351, the multimedia processor 50 determines whether or not the operation object 63 has hit the ball objects (65R, 65B, 65Y and 65G). In step S353, the multimedia processor 50 calculates the vertical position and inclination of the operation object 63 based on the left extraction point (XL, YL) and the right extraction point (XR, YR), and the operation object 63 Control the form and movement (setting of coordinates and image data, etc.).

  In step S355, the multimedia processor 50 controls the appearance and movement of the ball object (setting of coordinates and image data, etc.). In this case, referring to the determination result of step S351, when the operation object 63 hits the ball object, the ball object is moved at a direction and speed according to the inclination and speed of the operation object 63 at that time.

  In step S357, the multimedia processor 50 controls the effect image (such as setting of coordinates and image data) based on the result of step S351. In step S359, the multimedia processor 50 controls images other than those described above (setting of coordinates and image data, etc.). For example, the images to be controlled are the background image and the images 75R, 75B, 75Y, 75G, 69L, 69R, and the like.

  As described above, according to the present embodiment, the user can change the operation object 63 to a form corresponding to the positional relationship by moving the two input devices 3L and 3R. . Further, the user can change the operation object 63 by moving the two input devices 3L and 3R, and accordingly change the ball object.

  The present invention is not limited to the above-described embodiment, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

  (1) Transparency of the transparent body 17 includes translucency and colored transparency.

  (2) In the above, an example in which the middle finger is passed through the input device 3 has been described. However, the number of fingers to be inserted and the number thereof are not limited thereto.

  (3) The shape of the input device is not limited to the shape of the input device 3 described above. Further, a weight having a predetermined weight can be incorporated in the input device so that the player can move his / her hand in a loaded state.

  (4) Instead of attaching a reflecting member such as the retroreflective sheet 15 to the input device 3, a self-luminous device such as an infrared light emitting diode can be attached. In this case, the information processing apparatus 1 does not require the infrared light emitting diode 9. Further, without using an input device, a subject can be photographed by an imaging device such as an image sensor or CCD, and the motion can be detected by analyzing the image.

  (5) In relation to the flash catch mode, the number and arrangement of the fixed position objects 100 are not limited to those in FIG. Further, it is possible to display only the marker without displaying the fixed position object 100. Furthermore, the selection can be confirmed only by placing the cursor 70L or 70R on the target. That is, the player only needs to move the cursors 70L and 70R in the input state, and does not need to be in the non-input state.

  (6) In relation to the ball escape mode, the speed of the object 102 may be variable or constant. The object 102 can also appear from a fixed position.

  (7) Although numbers are assigned to the targets 110 in relation to the number catch mode, the targets 110 are not limited to this as long as there is an order. For example, alphabets or katakana can be added. Furthermore, the selection can be confirmed only by placing the cursor 70L or 70R on the target 110. That is, the player only needs to move the cursors 70L and 70R in the input state, and does not need to be in the non-input state.

  (8) In relation to the short-term memory mode, the number and arrangement of the fixed-position objects 120 are not limited to those shown in FIG. Moreover, as described above, various appearance modes of the marker can be adopted. Furthermore, in the above description, the player is instructed to indicate the position of the marker that has been given the last question. Furthermore, the selection can also be confirmed by moving the cursor 70L or 70R to the home position object 120 and bringing it into the non-input state without displaying the enter button 124. Furthermore, the selection can be confirmed only by moving the cursor 70L or 70R to the home position object 120.

  (9) In the above description, the cursor such as the cursors 70L and 70R is superimposed on the object such as the fixed-position object 100, and the non-input state is performed. It is also possible to perform an operation that only overlaps an object.

  (10) In the above, several modes for playing one hand at a time have been exemplified, but it is also possible to display two cursors linked to the operation and play using both hands.

  (11) In the butterfly catch mode, the butterfly objects 29, 31 and 33 are moved around. However, the shape and form of the butterfly objects 29, 31 and 33 can be arbitrarily set.

  (12) In the balance line mode, the moving object 66L and the like and the tray object 75L and the like are colored to identify the colors. However, the characters and the like can be identified by attaching characters or the like thereto. Moreover, you may write both of them. Furthermore, a Stroop effect may be added.

  (13) In the shuffle memory mode, the object to be stored is a figure, but the present invention is not limited to this, and any object such as a number, a character, a color, and a drawing can be targeted.

It is a block diagram which shows the whole structure of the training system by embodiment of this invention. It is a perspective view of the input device 3L or 3R of FIG. It is a figure which shows the state which mounted | wore the left and right hands, respectively with the input devices 3L and 3R of FIG. It is a figure which shows the electrical constitution of the information processing apparatus 1 of FIG. It is an illustration figure of the training screen in the flash catch mode by the training system of FIG. It is an illustration figure of the training screen in the ball escape mode by the training system of FIG. It is an illustration figure of the training screen in the 1st number catch mode by the training system of FIG. It is an illustration figure of the training screen in the short-term memory mode by the training system of FIG. It is an illustration figure of the training screen in the 2nd number catch mode by the training system of FIG. It is an illustration figure of the training screen in the 3rd number catch mode by the training system of FIG. It is an illustration figure of the training screen in the butterfly catch mode by the training system of FIG. It is an illustration figure of the training screen in Stroop mode by the training system of FIG. It is an illustration figure of the training screen in the panel escape mode by the training system of FIG. It is an illustration figure of the training screen in the balance line mode by the training system of FIG. It is an illustration figure of the training screen in the rhythm ball mode by the training system of FIG. It is an illustration figure of the training screen in the shuffle memory mode by the training system of FIG. It is an illustration figure of the mode selection screen by the training system of FIG. It is an illustration figure of the right-and-left selection screen by the training system of FIG. It is a figure which shows the other example of mounting | wearing of the input devices 3L and 3R of FIG. It is a figure which shows the further mounting example of the input devices 3L and 3R of FIG. It is explanatory drawing of the detection process of the retroreflection sheets 15L and 15R. It is explanatory drawing of a left-right determination process. 5 is a flowchart showing an example of the overall processing flow by the multimedia processor 50 of FIG. 4. It is a flowchart which shows an example of the flow of the principal part of the process by the application program in step S3 of FIG. It is a flowchart which shows an example of the flow of the imaging | photography process in step S21 of FIG. It is a flowchart which shows an example of the flow of the sheet | seat detection process in step S23 of FIG. It is a flowchart which shows an example of the flow of the left-right upper / lower end detection process in step S65 of FIG. It is a flowchart which shows an example of the flow of 2 point | piece position determination processing in step S67 of FIG. It is a flowchart which shows an example of the flow of the left-right determination process in step S69 of FIG. It is a flowchart which shows an example of the flow of the detection sheet number determination process in step S73 of FIG. It is a flowchart which shows an example of the flow of the one point specific process in step S75 of FIG. It is a flowchart which shows an example of the flow of the catch determination process in step S25 of FIG. It is a flowchart which shows an example of the flow of the video control process in step S27 of FIG. It is a flowchart which shows the other example of the flow of the image | video control process in step S27 of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus 3, 3L, 3R ... Input device, 5 ... Television monitor, 9 ... Infrared light emitting diode, 15, 15L, 15R ... Retroreflective sheet, 17, 17L, 17R ... Transparent body, 50 ... Multi Media processor, 52 ... external memory, 54 ... image sensor, 56 ... bus, 70L, 70R ... cursor, 100,120 ... fixed position object, 102 ... object, 104 ... outlet object, 106 ... arrow object, 110 ... target, 122: Instruction unit, 124: Determination button, 130L, 130R: Direction button, 132: Selection target.

Claims (7)

Displaying a target image on a display device;
Displaying a cursor on the display device;
Imaging a subject worn on the palm of the user;
Detecting the movement of the subject based on the image of the subject obtained by the imaging;
Moving the cursor in response to the detected movement of the subject;
And a step of giving a predetermined change to the target image and / or the cursor when an image of the subject is not obtained immediately after the cursor overlaps the target image.   The input method according to claim 1, wherein the target image appears from a non-display state at an arbitrary position or any one of a plurality of predetermined positions.   The input method according to claim 1, wherein the target image is statically displayed in advance at an arbitrary position or a predetermined position.   The input method according to claim 1, wherein there are a plurality of target images, and the target images are statically displayed in advance at a plurality of arbitrary positions or a plurality of predetermined positions.   The input method according to claim 1, wherein the target image is a moving moving image. Displaying a first object on a display device;
Imaging a plurality of subjects;
Detecting the positions of the plurality of subjects based on the images of the plurality of subjects obtained by the imaging;
Changing the form of the first object in accordance with the detected positional relationship between the plurality of subjects. Displaying a second object on the display device;
The input method according to claim 6, further comprising: changing the second object when the first object and the second object satisfy a predetermined positional relationship.