JP2010233671A - Program, information storage medium and game device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a game in which a player easily feels the sense of affinity with a player character. <P>SOLUTION: The whole body image of a player 2 during game play is photographed at all times in an image sensor module 1227, a human body silhouette 30 is extracted from a player image, and the coordinates of a head feature point 40 equivalent to the head position of the player 2, a left feature point 42 equivalent to a right hand position and a right feature point 44 equivalent to a left hand position are acquired from there. When the left feature point 42 and the right feature point 44 are present within a determination area 50, the player character 6 is automatically controlled so that a gun 8 is turned to the position indicated by a game controller 1230. When determining the state in which the player 2 exits from the determination area 50, extends hands and makes a certain pose, the posture parameter value of the arm joint of the player character 6 is calculated based on the position coordinates of the left feature point 42 and the right feature point 44, and operation control is performed so as to imitate the movement of the arm of the player 2. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a program or the like for causing a computer to display and control a predetermined character to control the progress of a game.

Currently, in many commercially available video games, a player operates a player character by operating a lever, a button switch, or the like. This is the same whether the game device is a home game device that includes a game controller in which a lever or a button switch is integrated, or an arcade game device.
On the other hand, a technique for controlling the player character by detecting the motion of the player with an acceleration sensor or the like provided in the game controller (see Non-Patent Document 1, for example), and determining a hand gesture by photographing the player's hand A technique for inputting a game operation by doing so is known (for example, see Patent Document 1).

JP 2001-246161 A

Software for the Nintendo game device “Wii” “Wii Sports” manual, 2006

  Now, in video games, how close the player is to the player character is an important factor for enjoying the sense of immersion in the game. For this reason, the player character is designed to be familiar, or designed to be a long-awaited hero image.

  However, no matter how much the player character's design is devised, the operation method has not changed. That is, in the conventional technology, even if the motion detection of the player by the acceleration sensor is used or the detection of the hand gesture is used, the motion prepared in advance corresponding to the operation input is operated in the same manner as the operation with the lever or the button switch. According to the above, the player character was controlled in motion. For this reason, the player has a certain sense of limit to the familiarity of the player character.

  The present invention has been made in view of such circumstances, and it is an object of the present invention to realize a game in which a player can more easily get a sense of familiarity with a player character.

A first invention for solving the above problems is a computer (for example, the home game apparatus of FIG. 1) provided with a digital image capturing device (for example, the image sensor module 1227 of FIG. 1) for shooting the player. 1200) is a program for generating a game space image in which a character model is arranged and executing a given game,
Character movement control means for controlling the movement of the character model (for example, the processing unit 200 in FIG. 8, the game calculation unit 210, the player character movement control unit 214, step S18 in FIG. 14),
Virtual camera control means (for example, the processing unit 200, the game calculation unit 210, the virtual camera control unit 216, FIG. 8), which arranges the virtual camera at a position away from the character model by a predetermined distance and controls movement according to the movement of the character model. Step S18 in FIG.
Image generation means for generating an image of the game space including the character model based on the virtual camera (for example, the processing unit 200, the image generation unit 260 in FIG. 8, step S14 in FIG. 14),
Model behavior control means for controlling the behavior of the character model based on the player image photographed by the photographing device (for example, the processing unit 200 in FIG. 8, the game calculation unit 210, the player character behavior control unit 224, the steps in FIG. 14). A program for causing the computer to function as S24).

  According to another aspect, there is provided a game device that includes a digital photographing device for photographing a player, generates an image of a game space in which a character model is arranged, and executes a predetermined game. Character movement control means for movement control, virtual camera control means for placing a virtual camera at a predetermined distance from the character model and controlling movement according to the movement of the character model, and the character model based on the virtual camera It is possible to configure a game apparatus including image generation means for generating an image of the game space and model behavior control means for controlling the behavior of the character model based on a player image photographed by the photographing apparatus. .

  According to the first aspect, the player who operates the game can be photographed by the photographing device, and the behavior of the character model appearing in the game can be controlled based on the photographed player image. Compared to conventional character control by lever or button operation from a game controller, character control using player motion detection by an acceleration sensor built in the game controller, or character control using hand gesture detection, A sense of familiarity with the character to be operated can be enhanced, and a sense of completeness in the game can be enhanced.

  In particular, for example, by configuring the player image so that at least the player's body is photographed, the character can be operated based on the player's overall movements and poses rather than the player's body part. Therefore, it is possible to remarkably enhance the closeness between the player and the character operated by the player.

According to a second aspect of the present invention, the character model has left and right arms, and the right arm of the player when the player faces the display device is the right arm of the character model viewed from the virtual camera, A left-right relationship setting means (for example, FIG. 8) that sets the correspondence between the left and right of the player image and the left and right of the character model so that the left arm corresponds to the left arm of the character model viewed from the virtual camera. Processing unit 200, game calculation unit 210, step S10 in FIG. 14),
Arm part determination means for determining the left arm part and the right arm part in the player image photographed by the photographing device (for example, the processing unit 200, the game calculation unit 210, the feature point information acquisition unit 220, FIG. 14). In step S6), the computer is further functioned,
The model behavior control means includes arm control means for controlling the left arm and the right arm of the character model based on the position change of the left or right arm portion of the corresponding player image, respectively, according to the correspondence relationship. It is a program of the 1st invention for making a computer function.

  According to the second invention, the same effects as in the first invention can be obtained, and the right arm and the left arm of the character that can be operated by the player can be correctly associated with the player's right arm and the left arm, and the motion can be controlled.

  In a third aspect of the present invention, the game is a game in which the character model battles an enemy character, and when the character model receives a partial arm destruction attack, the portion disappears and the arm control Means for causing the computer to function so as to control the existing left arm and right arm of the character model based on the position change of the left or right arm portion of the corresponding player image, respectively, according to the correspondence relationship. 2 is a program of the invention.

  According to the third invention, the same effect as the second invention is achieved, and even if the arm of the character model is partially destroyed and is not displayed, when a part remains Control can be performed based on the position change of the left or right arm portion of the player image.

According to a fourth aspect of the present invention, there is provided player left / right movement detecting means (for example, the processing unit 200, FIG. 8) for detecting that the player image in the player image photographed by the photographing device has moved to the left or right. The computer is caused to function as the game calculation unit 210, the gesture determination unit 222, and step S22 in FIG.
In accordance with the correspondence, the model behavior control means controls the attitude of the character model so as to incline in a direction corresponding to the direction detected by the player left-right movement detection means (for example, the processing unit 200, FIG. 8). A program according to the second or third aspect of the invention for causing the computer to function so as to have a game calculation unit 210, a player character behavior control unit 224, and step S24 in FIG.

  According to the fourth invention, the same effect as the second or third invention is achieved, and when the player image in the player image is detected to move leftward or rightward, The attitude can be controlled so that the model is tilted in a direction corresponding to the detected direction.

  According to a fifth aspect of the present invention, the left-right relationship setting means reverses the left-right correspondence relationship depending on whether the orientation of the virtual camera faces the front side or the back side of the character model. For example, the program according to any one of the second to fourth inventions for causing the computer to function so as to include the processing unit 200 of FIG. 8, the game calculation unit 210, and step S10 of FIG.

  According to the fifth invention, the same effects as in any of the second to fourth inventions can be obtained, and left and right depending on whether the orientation of the virtual camera faces the front side or the back side of the character model. And the right and left in the player image can be correctly associated with the left and right in the character model.

  According to a sixth aspect of the invention, the model behavior control means causes the character model to bend based on a change in the vertical length of the player image in the player image shot by the shooting device. Any one of the first to fifth inventions for causing the computer to function so as to include means (for example, the processing unit 200 in FIG. 8, the game calculation unit 210, the player character behavior control unit 224, and step S24 in FIG. 14). It is a program.

  According to the sixth invention, the same effect as any of the first to fifth inventions can be obtained, and the player can detect the squatting action and cause the character model to bend.

According to a seventh aspect of the invention, the model behavior control means controls the behavior of the upper body of the character model based on the player image photographed by the photographing device (for example, the processing unit 200 of FIG. A calculation unit 210, a player character behavior control unit 224, step S24 in FIG.
Lower body behavior control means for controlling the behavior of the lower body of the character model based on movement control by the character movement control means (for example, the processing unit 200, the game calculation unit 210, the player character movement control unit 214 in FIG. A program according to any one of the first to sixth inventions for causing the computer to function so as to include step S18).

  According to the seventh invention, the same effect as any of the first to sixth inventions can be obtained, and the character model can be controlled separately for the upper body and the lower body. Therefore, a configuration is possible in which the upper body performs motion control based on the player image while automatically controlling the character model to move to a predetermined position. For example, in a so-called rail shooting game, it is a suitable technical element for realizing motion control of a character model based on a player image.

  According to an eighth aspect of the present invention, there is provided a program for causing the computer to function as a character model setting unit that selects or generates the character model from a plurality of types of character models based on the player image in the player image. A program according to any one of the first to seventh inventions.

  According to the eighth aspect, it is possible to select or generate and set from a plurality of types of character models based on the player image in the player image. Therefore, the player character can be varied.

  According to a ninth aspect of the invention, the character model setting means sets a color of the character model based on the color of the player image in the player image (for example, the processing unit 200 and the game calculation unit 210 in FIG. 8). FIG. 15 is a program of an eighth invention for causing the computer to function so as to have steps S60 to S62) of FIG.

  According to the ninth aspect, the same effect as the eighth aspect can be obtained, and the color of the character model can be set based on the color of the clothes of the player. In particular, if the color of the character model is set to a color similar to the color of the player image, it is possible to further enhance the familiarity that the player has in the character model.

  According to a tenth aspect of the present invention, the character model setting means sets the shape of the character model based on the contour shape of the player image in the player image (for example, the processing unit 200 of FIG. 8, game calculation) 15 is a program of the eighth or ninth invention for causing the computer to function so as to have the unit 210, steps S54 to S58) of FIG.

  According to the tenth invention, the same effect as in the eighth or ninth invention can be obtained, and the body shape of the character model can be made to correspond to the shape of the player's figure. In particular, when the player's body shape is estimated from the player image and the body shape of the character model is changed in accordance with the estimated body shape, the familiarity of the player with the character model can be further enhanced.

  The eleventh invention is a computer-readable information storage medium storing the program of any one of the first to tenth inventions. The “information storage medium” mentioned here includes, for example, a magnetic disk, an optical disk, an IC memory, and the like. According to the eleventh invention, by causing a computer to read and execute the program of any one of the first to tenth inventions, the computer has the same effect as any one of the first to tenth inventions. It can be demonstrated.

The figure which shows the system structural example of a household game device. The front view and side view which show the structural example of a game controller. The figure which shows the play style example. The figure which shows the example of a game screen. The conceptual diagram explaining the setting principle of a player character. The conceptual diagram explaining the principle of action control of a player character. The figure which shows the example of gesture and a pose input. The functional block diagram which shows a function structural example. The figure which shows the data structural example of game space setting data. The figure which shows the data structural example of player character initial setting data. The figure which shows the data structural example of gesture dictionary data. The figure which shows the data structural example of player character status data. The figure which shows the data structural example of the setting data corresponding to right and left. The flowchart for demonstrating the flow of the main processes. The flowchart for demonstrating the flow of a model matching process. The flowchart for demonstrating the flow of a player character behavior control process. The flowchart following FIG. The figure explaining the concept of the inversion silhouette display as a substitute of the player character in 2nd Embodiment. The figure explaining the example of control at the time of the damage expression when a player receives damage in a reverse silhouette display state. The figure explaining the example of control which makes a player character appear by transformation operation. The flowchart for demonstrating the flow of the player character behavior control process (player character behavior control process B) in 2nd Embodiment. The flowchart following FIG. The flowchart for demonstrating the flow of a game screen production | generation display process. The figure which shows the structural example of the gun shooting game apparatus which is a modification of a game device.

[First Embodiment]
As a first embodiment to which the present invention is applied, an example in which a TPS game (Third Person Shooting Game) is executed in a consumer game device including an image sensor will be described.

  FIG. 1 is a diagram for explaining a configuration example of a consumer game device according to the present embodiment. As shown in the figure, the consumer game device 1200 includes a game device main body 1201, a video monitor 1220, a game controller 1230, an optical signal output device 1226, and an image sensor module 1227.

  The game apparatus main body 1201 includes, for example, a control unit 1210 on which a CPU, an image processing LSI, an IC memory, and the like are mounted, and information storage medium reading devices 1206 and 1208 such as an optical disk 1202 and a memory card 1204. Then, the home game device 1200 reads the game program and various setting data from the optical disk 1202 and the memory card 1204, and the control unit 1210 executes various game operations based on operation inputs made by the game controller 1230. Run video games.

The control unit 1210 includes various microprocessors such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and DSP (Digital Signal Processor), and electrical and electronic devices such as an ASIC (Application Specific Integrated Circuit) and an IC memory. Each part of the game apparatus 1200 is controlled.
In addition, the control unit 1210 includes a communication device 1212 that is wired or wirelessly connected to the communication line 1 such as the Internet, a LAN (Local Area Network), and a WAN (Wide Area Network), and realizes data communication with an external device. Prepare. Also, a short-range wireless communication module 1214 is provided, and data transmission / reception is realized with a plurality of game controllers 1230 via short-range wireless. As a short-range wireless format, for example, Bluetooth (registered trademark), UWB (ultra-wide band wireless), wireless LAN, or the like can be applied as appropriate.

  Then, the control unit 1210 generates a game screen and game sound based on the operation input signal received from the game controller 1230 and executes the video game. A video signal or sound signal based on the generated game screen or game sound is output to a video monitor 1220 (including a display monitor, a television capable of external input such as an audio signal and a video signal) connected by a cable 1209. . The video monitor 1220 is provided with an image display device 1222 for displaying a video image and a speaker 1224 for outputting sound. The player emits sound from the speaker 1224 while watching the game screen displayed on the image display device 1222. Play the game while listening to the game sound.

  FIG. 2 is an external view showing an example of the game controller 1230 used in the present embodiment, and is (1) a front view (= top view) and (2) a right side view as seen from the right when viewed from the front. The game controller 1230 has a bar shape having a substantially square section that is chamfered, and the player operates by gripping one hand in the manner of gripping the bar.

  The game controller 1230 connects various input devices and output devices around a built-in controller control unit 1260 with a local bus circuit realized by, for example, an IIC (Inter-Integrated Circuit) bus. To control input / output between devices.

  Specifically, for example, as an input device for switches, operation buttons 1240a, 1240b, and 1240c are provided from the lower end portion on the front side (upper surface side) to the center portion. Further, the front part on the front side is provided with direction input keys 1242 that can individually input the vertical and horizontal directions by pressing the four corners of the cross, and the center part on the back side is provided with a trigger 1246.

  As other input devices, an acceleration sensor 1248 for detecting the movement and inclination of the game controller 1230 and an image sensor 1256 are provided.

  In the acceleration sensor 1248, the longitudinal direction of the game controller 1230 (upward in the drawing as shown in FIG. 2 (1)) is the positive direction of the Z axis, and the right direction toward the front (right in the drawing as shown in FIG. 2 (1)). ) Is the positive direction of the X-axis, and the acceleration in the three orthogonal directions with the Y-axis positive direction as the front direction toward the front (the left direction in FIG. 2) is detected. An operation input signal including the corresponding information is output to the controller control unit 1260.

  The image sensor 1256 is realized by mounting a filter that transmits only infrared light in front of an image sensor such as a CCD sensor or a CMOS sensor. The image sensor 1256 is provided at the tip of the game controller 1230 to photograph the front side in the longitudinal direction. The image signal is output to the controller control unit 1260.

  In addition, the game controller 1230 in this embodiment includes a speaker 1252. The speaker 1252 generates sound according to the sound output signal output from the controller control unit 1260, and emits the sound toward the front side.

  The controller control unit 1260 performs wireless communication with various microchips such as a bus controller IC that controls data communication in a CPU and a local bus circuit, electronic components such as an IC memory, and the short-range wireless communication module 1214 of the game apparatus main body 1201. The short-range wireless communication module 1262 to be realized is mounted.

  Then, the controller control unit 1260 generates an operation input signal based on signals transmitted from various input devices via the local bus circuit, and the generated operation input signal is transmitted to the game apparatus main body 1201 by the short-range wireless communication module 1262. Send. When the short-range wireless communication module 1262 receives an output signal sent from the game apparatus main body 1201, the control signal is generated and sent to the output device associated with the received output signal. The electric power required by the controller control unit 1260 and each part is supplied from a battery 1270 built in a battery chamber recessed in the back side of the game controller 1230.

  The optical signal output device 1226 includes a plurality of point light sources that emit infrared light. The optical signal output device 1226 is installed at a predetermined position (for example, the upper part) near the video monitor 1220, and can project infrared light forward of the video monitor 1220.

  The image sensor module 1227 is, for example, a lens, a mechanical shutter, a shutter driver, a photoelectric conversion element such as a CCD image sensor module or a CMOS image sensor module, a digital signal processor (DSP) that reads an amount of charge from the photoelectric conversion element and generates image data, It is realized with an IC memory or the like.

FIG. 3 is a diagram showing a game play style in the present embodiment.
The player 2 holds the game controller 1230, stands in front of the video monitor 1220, and holds the tip of the game controller 1230 facing the video monitor 1220.

  When the tip of the game controller 1230 is directed to the video monitor 1220, the optical signal output device 1226 can be photographed by the image sensor 1256 of the game controller 1230. The control unit 1210 receives image data captured by the image sensor 1256 from the game controller 1230, moves the pointer 4 to the image display device 1222 based on the position of each point light source of infrared light in the image, A function as a pointing device is realized.

  The image sensor module 1227 is installed, for example, on the top of the video monitor 1220 so that the whole body of the player 2 in front of the video monitor 1220 can be photographed from the front, and generates image data of the photographed player image W2 to generate a control unit. 1210 can be output.

FIG. 4 is a view showing an example of a game screen displayed on the video monitor 1220.
The video game of this embodiment is a so-called TPS.
That is, the control unit 1210 arranges background objects such as walls, stairs, and passages in the three-dimensional virtual space to form the game space, and the player character 6 model or NPC (NPC) operated by the player in the game space. A model of the enemy character 10 which is a non-playable character is arranged, and a game space image obtained by photographing the surroundings of the player character 6 with a virtual camera with a third-person viewpoint is rendered. Further, the rendered game space image further includes a hit point gauge 20 indicating the hit point value of the player character 6, an orientation display 22 indicating the front direction of the player character 6 in the game space, and a situation around the player character 6. An information display such as a so-called radar display 24 to be displayed is synthesized and displayed to generate a game image W4 and displayed on the video monitor 1220.

The player character 6 possesses the gun 8, and when the player operates the trigger 1246 of the game controller 1230 and inputs an attack operation, the player character 6 is controlled to shoot toward the display position of the pointer 4. .
The enemy character 10 can be defeated by attacking the enemy character 10 and setting the hit point of the enemy character 10 to “0”. On the other hand, when the player character 6 is attacked by the enemy character 10, the hit point of the player character 6 is subtracted, and when the hit point becomes “0”, the player character 6 becomes unable to fight and the game is over. .

  In the game of this embodiment, the player character 6 is automatically controlled to move to a predetermined battle position prepared in the game space, and when all the enemy characters 10 appearing there are defeated, the player character 6 is automatically moved to the next battle position. It is a so-called rail shooting type game in which the game progresses while being controlled to move. If the player character 6 can be operated well to survive the battle at each battle position, the game is cleared.

[Description of how to set the player character model]
FIG. 5 is a conceptual diagram for explaining the setting of the display model of the player character 6 in the present embodiment.
In the present embodiment, the player's body shape and clothing color elements can be reflected in the model of the player character 6 prior to the start of the game. Therefore, prior to the start of the game, a predetermined play preparation guide is displayed on the screen of the video monitor 1220 and an image for calibration is acquired by the image sensor module 1227 while giving an instruction to the player 2.

  More specifically, in the play preparation guide, first, the player instructs the video monitor 1220 not to stand for a certain period of time, and the first calibration image C1 of only the background is captured by the image sensor module 1227 during that time. Next, the player is instructed to open both hands facing the front to the image sensor module 1227, and the second calibration image C2 is photographed.

  If the first calibration image C1 and the second calibration image C2 are acquired, the control unit 1210 extracts the human body silhouette 30 of the player 2 from the difference between the two images. Since the human body silhouette 30 is obtained from the standing front image of the player 2, the lateral width Lb at the intermediate height is regarded as the body shape estimation parameter value of the player 2, and the body shape of the player character 6 is determined according to the lateral width Lb. change. For example, the width Lb is compared with a predetermined standard, and is classified into three types of skinny, standard, and thick, and the size of the circumference of the player character 6 is changed according to the classification result, and the applied player character model is used in the game. To do.

  In the example on the left side of FIG. 5, since the horizontal width Lb of the human body silhouette 30a is determined to be standard, the player character 6a has a standard figure. On the other hand, in the example on the right side of FIG. 5, since the horizontal width Lb of the human body silhouette 30b is determined to be thick, the player character 6b is adjusted so that the waistline is thick. Not only the waistline but also the head, arms and legs can be adjusted. Further, the body shape of the player character 6 may be changed based on the ratio between the expansion and the horizontal width LB.

  Further, when the human body silhouette 30 is extracted, the control unit 1210 statistically processes the color information of the pixels in the region of the human body silhouette 30 and determines the representative color of the human body silhouette 30 from a predetermined option. As the representative color, for example, a color that matches the hue of the most occupied pixel in the human body silhouette 30 is selected from six colors of red, orange, yellow, blue, green, and purple plus white and black. Then, texture data having a color similar to the representative color is selected from a plurality of texture data prepared in advance, and the selected texture data is used as application texture data used in the game.

  In this way, the player character 6 is set so as to have the characteristics of the body shape and clothing color of the player 2, so that the player feels close to the player character 6.

[Description of player character motion control method]
FIG. 6 is a conceptual diagram for explaining a motion control method for the player character 6 in the present embodiment. A player character in a conventional TPS is based on an operation input from a game controller, and performs an attack operation or an avoidance operation according to motion data prepared in advance corresponding to the operation input from the game controller. Operation control (behavior control) is performed. However, in this embodiment, the player character 6 is controlled based on an operation input from the game controller 1230 in part as in the past, but in part, the operation (behavior) of the player 2 captured by the image sensor module 1227 is controlled. Motion control is performed so that the player character 6 is imitated.

  Specifically, the control unit 1210 extracts the human body silhouette 30 from the second calibration image C2, and extracts the central upward convex portion and the convex portions at the left and right end portions from the extracted human body silhouette 30. Then, the extracted center upward convex portion is recognized as the head feature point 40 corresponding to the head of the player 2 and the coordinates of the representative point (for example, the tip position or the center position of the corresponding portion) in the screen coordinate system are acquired. . Similarly, a convex portion in the left direction toward the screen of the human body silhouette 30 is identified as a left feature point 42, and a convex portion in the right direction toward the screen is identified as a right feature point 44.

  During the game play, the state of the player 2 is always captured by the image sensor module 1227, and the head feature point 40, the left feature point 42, and the right are captured for each captured frame (for example, the player images W2, W3,...). The coordinates of the feature point 44 are continuously acquired. The coordinates of the head feature point 40, the left feature point 42, and the right feature point 44 serve as basic information for controlling the movement of the player character 6 so as to imitate the movement of the player 2 photographed by the image sensor module 1227.

  It should be noted here that the information of each feature point obtained from the player images W2, W3,... Is two-dimensional position information in the screen coordinate system, and is an image taken by one image sensor module 1227. Since the position information in the front-rear direction of each feature point cannot be obtained only from, the three-dimensional operation cannot be performed. However, it can be said that the position of the hand in the front-rear direction of the body is generally within the same range in the pose with the hand extended to the left and right. In other words, in a pose with the hands extended to the left and right, a prescribed value (for example, the front and back positions of the player character 6 in front of the chest) is applied as the front and rear direction positions of the left feature point 42 and the right feature point 44, and the inverse kinematic Even if the player character 6 is controlled by obtaining the posture parameters of each joint of the arm by the tick processing, it can be said that it is sufficient to imitate the movement of the arm of the player 2.

  On the other hand, from the viewpoint of consistency in the shooting game, the player character 6 basically needs to hold the gun 8 in front of the body or obliquely forward and point it in the direction of the pointer 4. If, for example, the movement of the actual arm of the player 2 indicated within the narrow range of the game screen by the game controller 1230 is controlled so that the player character 6 is always imitated as it is, the direction of the muzzle of the gun 8 is the direction of the pointer 4. The inconvenience is that the bullets fly diagonally from the muzzle that is not facing the target.

  Therefore, in this embodiment, depending on how far the left feature point 42 and the right feature point 44 are separated from the head feature point 40 in the left-right direction, whether the pose that the player 2 is currently doing is a pose that spreads the arm. Determine whether the pose is aimed with a gun. In the example of FIG. 6, a rectangular determination area 50 centered on the left and right of the head feature point 40 is set. If the left feature point 42 and the right feature point 44 are within the determination area 50, the pose and determination area holding the gun If it is outside 50, it indicates that it is determined that the pose is to spread arms.

If it is determined that the “pose holding a gun”, the upper body 6 u of the player character 6 is automatically controlled so that the gun 8 is directed to the display position of the pointer 4. If it is determined that the arm is a pose that spreads the arms, the left feature point 42 and the right feature point are determined from the vertical and horizontal two-dimensional coordinate values of the left feature point 42 and the right feature point 44 and a predetermined value that determines the front-back direction. 44 three-dimensional information is set in a pseudo manner, and posture parameter values of the arm joints are calculated by inverse kinematics processing. Then, the action of the player 2 is imitated on the upper body 6 u of the player character 6.
At this time, which arm the posture parameter calculated based on the position coordinate of which feature point is applied to depends on whether the virtual camera is shooting the player character 6 from behind or from the front. Of course, in the former case, the left and right sides are opposite, and in the latter case, the left and right sides are the same.

  In the TPS of the present embodiment, since the player character 6 is assumed to be a so-called rail trailing type in which the player character 6 is automatically moved to a predetermined battle position and battles there, the lower body model 6d of the player character 6 is automatically controlled. Further, even when the player character 6 is moved in response to a moving operation input from the game controller 1230 as in the known FPS or TPS, even if the rail model is not a rail trailing type, the lower body model 6d is similarly changed by the player 2 It is preferable not to imitate the operation but to automatically control the operation (behavior) of the lower body model 6d in accordance with the moving operation from the game controller 1230.

In the present embodiment, gesture input is made possible by using the points where the coordinates of the human body silhouette 30 and the feature points 40, 42, 44 can be acquired from the player images W2, W3,.
For example, as shown in the player image W2 → W4 in FIG. 7, when a downward movement of the head feature point 40 exceeding the reference value is detected, or the vertical length of the human body silhouette 30 has reached a predetermined determination reference value. In this case, it can be determined that a bending gesture has been input, and a bending avoiding motion that avoids an enemy attack by bending to the player character 6 can be performed.

  Similarly, as shown in the player image W2 → W6 in FIG. 7, when the movement of the head feature point 40 exceeding the reference value in the left-right direction is detected, or when the human body silhouette 30 exceeds the reference value, When it is detected that the robot is tilted to the right, it is determined that a lateral avoidance gesture in the left-right direction has been input, and an avoidance motion that avoids an attack by tilting the body of the player character 6 in the corresponding lateral direction is executed. Can do. In the case of avoidance in the horizontal direction, the player character 6 is tilted in the direction opposite to the moving direction on the player image under the condition that the virtual camera captures the player character 6 from behind. That is, when the player 2 tilts his body to the right on the captured player image, the player character 6 is controlled to tilt to the left and perform an avoidance exercise.

  Further, as shown in the player image W2 → W8 in FIG. 7, the input by the gesture is not limited to the movement action of the player character 6, but for example, the relative positional relationship of the head feature point 40, the left feature point 42, and the right feature point 44. Is determined to correspond to the relative positional relationship in a predetermined pose, or when a pattern matching determination is made that the shape of the human body silhouette is similar to a predetermined dictionary pattern, control is performed so that a special attack (SP attack) is activated. You can also.

  In this embodiment, the operation input by gestures and poses is described as only these three types. However, by setting a plurality of types of gestures to be determined, not only attack techniques but also defense techniques, recovery techniques, and ammunition supplements as appropriate. Of course, it is possible to assign actions such as.

[Description of functional block]
Next, a functional configuration example for executing the above-described game will be described.
FIG. 8 is a functional block diagram illustrating an example of a functional configuration according to the present embodiment. As shown in the figure, in the present embodiment, the operation input unit 100, the player photographing unit 110, the processing unit 200, the optical signal generation unit 330, the sound output unit 350, the image display unit 360, and the communication unit 370. And a storage unit 500.

  The operation input unit 100 outputs an operation input signal to the processing unit 200 in accordance with various operation inputs made by the player. In the present embodiment, an optical signal light receiving unit 102 and a communication unit 104 are included.

  The operation input unit 100 includes a button switch, a joystick, a touch pad, a trackball, a multi-axis detection type acceleration sensor having two or more detection axes, or a single axis detection type acceleration sensor unit combined with different detection axis directions, It can be realized by a multi-directional detection type inclination sensor that enables at least two detection directions or a single direction detection type inclination sensor unit combined with different detection directions. In the present embodiment, the game controller 1230 in FIG. 2 corresponds to the operation input unit 100.

The optical signal light receiving unit 102 is a device that receives an optical signal projected from the optical signal generating unit 330 and converts it into an electrical signal, thereby converting it into image data. For example, a CCD sensor, a CMOS sensor, a lens, an optical filter It is realized by a DSP (digital signal processor) that performs signal conversion processing. The image sensor 1256 in FIG. 2 and the controller control unit 1260 for performing signal processing correspond to this.
The optical signal generator 330 is realized by a light emitting element such as an LED, a filter that transmits only light of a specific frequency, and the like, and an optical signal indicating a reference position for using the game controller 1230 as a pointing device in a specific direction. Flood light. The optical signal output device 1226 in FIG. 1 corresponds to this.

  The communication unit 104 realizes data communication between the game controller 1230 and the game apparatus main body 1201. In this embodiment, the short-range wireless communication module 1262 in FIG.

  The player photographing unit 110 photographs the state of the player 2 and converts it into an electrical signal, generates digital image data, and outputs the digital image data to the processing unit 200. For example, it is realized by a lens, a mechanical shutter, a shutter driver, a photoelectric conversion element such as a CCD image sensor or a CMOS image sensor, a digital signal processor (DSP) that reads out the charge amount from the photoelectric conversion element and generates image data, an IC memory, or the like. In FIG. 1, the image sensor module 1227 corresponds to this.

  The processing unit 200 is realized by electronic components such as a microprocessor such as a CPU and a GPU, an ASIC (application-specific integrated circuit), and an IC memory, and includes the operation input unit 100 and the storage unit 500. Input / output data to / from each functional unit. Then, various arithmetic processes are executed based on a predetermined program and data and an operation input signal from the operation input unit 100 to control the operation of the game apparatus. In FIG. 1, the control unit 1210 corresponds to this. The processing unit 200 in this embodiment includes a game calculation unit 210, a sound generation unit 250, an image generation unit 260, and a communication control unit 270.

  The game calculation unit 210 executes various processes related to the progress of the game. For example, (1) a process for forming a game space by placing a background model in a three-dimensional virtual space, and (2) a model used in a game of a player character 6 that resembles the player's body shape and clothing color based on the player image. (3) Processing for placing models of the player character 6 and the enemy character 10 in the game space, (4) Automatic motion control processing for the enemy character 10, (5) Collision determination processing between models, (6 ) Attack hit determination, (7) damage calculation process, and (8) damage reflection process.

  The game calculation unit 210 of this embodiment includes a pointer control unit 212, a player character movement control unit 214, a virtual camera control unit 216, a human body silhouette extraction unit 218, a feature point information acquisition unit 220, and a gesture determination. Unit 222 and player character behavior control unit 224.

  The pointer controller 212 controls the display of the pointer 4 in the game screen based on an image obtained by photographing the optical signal emitted from the optical signal generator 330 received by the optical signal receiver 102.

  The player character movement control unit 214 automatically controls the movement of the player character 6. Since the TPS of this embodiment is a rail trailing type, a plurality of battle positions determined in advance in the game space while controlling the motion of the lower half model 6d of the player character 6 (see FIG. 6) with a predetermined walking / running motion. The movement is controlled step by step according to the determined order.

  The virtual camera control unit 216 controls the movement of the virtual camera as a third person viewpoint. Specifically, the position and orientation of the virtual camera are controlled according to information defining the relative position and shooting direction with respect to the player character 6 for each game stage.

  The human body silhouette extraction unit 218 extracts the human body silhouette 30 from the player image photographed by the player photographing unit 110. The extraction technique can be realized by using a known silhouette extraction technique, a contour line extraction technique, or the like.

  The feature point information acquisition unit 220 extracts feature points such as the head, left and right hands, and joint positions of the player 2 from the player image photographed by the player photographing unit 110, and from the player image constantly photographed during game play. The coordinate value of each feature point is always calculated. For example, it is possible to appropriately use a technique for detecting a part of a human face or hand that has been photographed by image processing or a technique for detecting a part of a human body in preprocessing of a gesture detection technique.

  The gesture determination unit 222 detects a predetermined gesture from the player image photographed by the player photographing unit 110. For example, a known gesture input technique such as a pattern recognition technique for matching the entire shape of the human body silhouette with predetermined dictionary data can be used as appropriate. It can also be realized by detecting changes in the size and inclination of the human body silhouette.

  The player character behavior control unit 224 controls the behavior of the upper body model 6 a of the player character 6. That is, when a predetermined gesture is detected by the gesture determination unit 222, operation control is performed with a motion previously associated with the detected gesture. In addition, when the positions of the left feature point 42 and the right feature point 44 acquired by the feature point information acquisition unit 220 are outside the predetermined determination area 50, the two-dimensional vertical and horizontal two-dimensional positions obtained by image processing of the player image A pseudo three-dimensional coordinate value of each feature point is formed from the coordinates and predetermined front and rear coordinate values. Then, the posture parameter value of the arm joint is reversely calculated by the inverse kinematics process, and the player character 6 is controlled based on the obtained posture parameter value, and the control of imitating the motion of the upper body of the player 2 is performed.

  The sound generation unit 250 is realized by a processor such as a digital signal processor (DSP) or a voice synthesis IC, or an audio codec capable of playing back an audio file, for example. Based on the processing result of the game calculation unit 210, sound effects and BGM related to the game Then, sound signals of various operation sounds are generated and output to the sound output unit 350.

  The sound output unit 350 is realized by a device that outputs sound effects, BGM, and the like based on the sound signal input from the sound generation unit 250. In FIG. 1, the speaker 1224 corresponds to this.

  The image generation unit 260 is realized by, for example, a processor such as a GPU (Graphics Processing Unit) or a digital signal processor (DSP), a program such as a video signal IC or a video codec, an IC memory for a drawing frame such as a frame buffer, or the like. The image generation unit 260 generates one game image in one frame time (for example, 1/60 second) based on the processing result by the game calculation unit 210, and outputs an image signal of the generated game image to the image display unit 360. To do.

  The image display unit 360 displays various game images based on the image signal input from the image generation unit 260. For example, it can be realized by an image display device such as a flat panel display, a cathode ray tube (CRT), a projector, or a head mounted display. In FIG. 1, the image display device 1222 corresponds to this.

  The communication control unit 270 executes data processing related to data communication, and realizes data exchange with an external device via the communication unit 370.

  The communication unit 370 is connected to the communication line 1 to realize communication. For example, a wireless communication device, a modem, a TA (terminal adapter), a wired communication cable jack, a control circuit, and the like are realized, and the communication device 1212 and the short-range wireless communication module 1214 in FIG.

  The storage unit 500 stores a system program for realizing various functions for causing the processing unit 200 to control the home game device 1200 in an integrated manner, a game program necessary for executing the game, various data, and the like. . Further, it is used as a work area of the processing unit 200, and temporarily stores calculation results executed by the processing unit 200 according to various programs, input data input from the operation input unit 100, and the like. This function is realized by, for example, an IC memory such as a RAM or a ROM, a magnetic disk such as a hard disk, an optical disk such as a CD-ROM, DVD, or BD (Blueray Disc (registered trademark)). In FIG. 1, the IC memory, the optical disk 1402, and the memory card 1404 mounted on the control unit 1210 correspond to this.

  In the present embodiment, the storage unit 500 stores a system program 501 and a game program 502. The system program 501 is a program for realizing the basic functions of the consumer game device 1200, and includes various types of firmware that can be used by calling each function unit on the application software side. The game program 502 is software for realizing a function as the game calculation unit 210 by being read and executed by the processing unit 200.

The storage unit 500 stores game space setting data 504, player character initial setting data 510, and gesture dictionary data 530 as data prepared in advance. Furthermore, player character status data 540, feature point data 550, left / right correspondence setting data 552, and player image temporary storage data 560 are stored as data that is generated and updated as needed as the game is prepared and progressed. .
In addition, data (for example, timer values and counters) necessary for executing processing related to the progress of the game, control information such as the position and posture of the virtual camera, position information and hit point information of each enemy character 10, etc. Control information and the like are also stored as appropriate in the same manner as known TPS.

The game space setting data 504 stores information for forming a game space in the three-dimensional virtual space and arranging the enemy character 10 as a game stage.
For example, as shown in FIG. 9, a plurality of stage data sets 504a collected for each game stage are stored. One stage data set 504a includes (1) background object data 504b that defines the arrangement position and posture of a background object for arranging walls, stairs, passages, and the like, and (2) a polygon of the enemy character 10 that appears. Enemy character setting data 504c for storing models, textures, motion data, hit point initial values, etc., (3) enemy character arrangement data 504d for storing arrangement position coordinates for each enemy character, and (4) a plurality of items in the game stage. Battle position setting data 504e that stores the battle positions and their transit order in association with each other, and (5) information on the relative position, shooting direction, shooting angle of view, etc. of the virtual camera and the player character 6 on the game stage. Virtual camera shooting condition data 504f to be stored.

  In this embodiment, the virtual camera shooting condition data 504f is set based on a third-person viewpoint for shooting from the diagonally upper rear side of the player character 6, but depending on the game stage, the front of the player character 6 is copied, It may be set so as to realize a situation in which an enemy approaches from behind the player character 6.

  The player character initial setting data 510 is data relating to polygon model data for forming a display model for the player character 6, texture data, motion data, hit point initial values, and a display model for items such as the gun 8 possessed. Is stored. In this embodiment, for example, as shown in FIG. 10, skeleton model data 511, polygon model data 512, a texture data library 514, and a motion data library 516 are included.

  The polygon model data 512 includes an upper body polygon model 512a and a lower body polygon model 512b. At the time of display, these two polygon models are connected at a predetermined position (for example, waist belt position) and are treated as a polygon model of the whole body of the player character 6.

  The texture data library 514 stores a plurality of texture data 514b applied to the polygon model of the player character 6 for each selection condition 514a. In the present embodiment, the selection condition 514a is statistical color information (for example, a hue with a high content rate) obtained from color information of pixels included in the human body silhouette 30 extracted from the second calibration image C2 before the game starts. A combination thereof can be set as appropriate. In this embodiment, the obtained statistical color information is associated with texture data arranged in a similar color.

  The motion data library 516 stores various motion data 516b of the player character 6 in association with the operation command 516a, and attached effect data 516c for realizing effect display and the like performed when the motion is activated.

  For example, as shown in FIG. 11, the gesture dictionary data 530 stores a plurality of gesture pattern data 530a serving as dictionary data for detecting a specific gesture from a player image in association with an operation command 530b. The gesture pattern data 530a can be set as appropriate according to the gesture determination method. For example, in the case where the gesture is determined by the shape of the human body silhouette 30, the pattern matching dictionary data may be used.

  The player character status data 540 stores information on various parameters necessary for controlling the player character 6 while the game is in progress. For example, as shown in FIG. 12, it includes applied player character model data 542, posture parameter data 544 indicating a posture based on a joint angle, hit points 546, and motion control data 548. Of course, other than these can be stored as appropriate.

  The applied player character model data 542 includes applied polygon model data 542a that stores polygon model data changed to reflect the shape of the player 2 based on a copy of the polygon model data 512 of the player character initial setting data 510, and the player character. And applied texture data 542b for storing texture data 514b selected from the texture data library 514 of the initial setting data 510.

The posture parameter data 544 stores posture parameters of each joint of the skeleton model 511 of the player character 6.
The motion control data 548 stores information for managing the currently executed motion control.

  The feature point data 550 stores information on each feature point obtained from the player image. For example, information such as a coordinate value in the image coordinate system of the player image and a relative position coordinate value with the head feature point 40 as a reference point is stored.

  The left / right correspondence setting data 552 is information for setting the correspondence between the left and right in the player image at that time and the left and right in the player character 6. On the stage where the virtual camera shoots from behind the player character 6, “right” and “left” are set in the player character direction 552b corresponding to “left” and “right” in the player image direction 552a, respectively. 6, “left” and “right” are set in the player character direction 552b corresponding to “left” and “right” in the player image direction 552a, respectively.

  The player image temporary storage data 560 can temporarily store a predetermined number of image data of player images for a plurality of frames. The stored image data is sequentially deleted from the oldest and new image data is stored.

[Description of process flow]
Next, the flow of processing in this embodiment will be described. A series of processing described here is realized by the processing unit 200 reading and executing the game program 502.

  FIG. 14 is a flowchart for explaining the main processing flow in the present embodiment. First, the processing unit 200 starts shooting a player image by the image sensor module 1227 (step S2), and executes model matching processing (step S4).

  FIG. 15 is a flowchart for explaining the flow of the model matching process. The processing unit 200 performs an operation guide display for instructing the player not to enter the shooting range on the image display device 1222 and a live view display of the image shot by the image sensor module 1227 to perform the first calibration. The image C1 is photographed (step S50; see FIG. 5). Similarly, the player is instructed to enter the center of the shooting range and take a predetermined pose with both hands left and right in front of the image sensor module 1227 in the standing position, and the second calibration image C2 is shot ( Step S52).

  Next, the human body silhouette 30 in the second calibration image C2 is extracted from the difference between the first calibration image C1 and the second calibration image C2 (step S54), and the body shape of the player 2 is extracted from the extracted human body silhouette 30. The lateral width Lb (see FIG. 5) is calculated as a parameter value for estimation (step S56).

  Then, the body width classification of the player 2 is determined by comparing the horizontal width Lb with a predetermined reference value, the size of the trunk of the player character 6 is changed according to the determined body classification, and the changed polygon model is applied to the applied polygon model. Data 542a is set (step S58). It should be noted that the generation of the applied player character model may be configured such that a polygon model is prepared in advance for each body shape classification and any one is selected and used instead of deforming the polygon model of the player character 6.

  Next, the processing unit 200 performs statistical processing on the color information of the pixels included in the extracted human body silhouette 30 to obtain statistical color information that causes the player 2 to image clothing (step S60), and corresponds to the statistical color information. The texture data 514b associated with the selection condition 514a of the texture data library 514 to be used is set as the applied texture data 542b (step S62), and the model matching process is terminated.

  Returning to the flowchart of FIG. 14, the processing unit 200 recognizes the head feature point 40, the left feature point 42, and the right feature point 44 from the image of the player 2 based on the second calibration image C2 (step S6). ), A trace process for constantly calculating the information of each feature point for each frame photographed by the image sensor module 1227 and storing it in the feature point data 550 is started (step S8).

  Next, the processing unit 200 randomly selects the game stage data set 504a of the game stage to be used this time from the game space setting data 504, and refers to the virtual camera shooting condition data 504f at the selected stage, so that the virtual camera is the player character. The left / right correspondence setting data 552 is set according to whether 6 is photographed from the rear or the front (step S10). That is, when the player 2 faces the image display device 1222, the right arm of the player is on the right arm of the player character 6 viewed from the virtual camera, and the left arm is on the left side of the player character 6 viewed from the virtual camera. The correspondence between the left and right of the player image and the left and right of the player character 6 is set according to the relative shooting position of the virtual camera with respect to the player character 6 so as to correspond to the arm.

Then, a game space is formed in the three-dimensional virtual space according to the stage data set 504a of the selected game stage, and the enemy character 10, the player character 6, and the virtual camera are arranged (step S12).
Next, generation of a game image is started (step S14), and the game is started (step S16).

If the game is started, the processing unit 200 repeatedly executes the processes in steps S18 to S34 in a control cycle corresponding to the refresh rate of the image display device 1222 until a predetermined game end condition is satisfied.
Specifically, first, referring to the stage data set 504a of the game stage currently being played in the game space setting data 504, the movement control of the player character 6 and the virtual camera is controlled according to the setting of the battle position setting data 504e. This is executed for the number of minutes (step S18). When moving the player character 6, the processing unit 200 controls the motion of the lower body model 6d so that the player character 6 walks and runs.

  Next, the processing unit 200 automatically controls the action of the enemy character 10 which is an NPC (step S20), and refers to the gesture dictionary data 530 to detect a specific gesture associated with the operation command from the player image ( Step S22). Then, a player character behavior control process for controlling the behavior of the upper body of the player character 6 is executed (step S24).

16 to 17 are flowcharts for explaining the flow of the player character behavior control process in the present embodiment.
In this process, first, the processing unit 200 determines whether or not there is a motion currently being executed (step S80). If there is no motion being executed (NO in step S80) and the gesture corresponding to the SP (special) attack operation command is detected by the gesture detection process in step S22 (YES in step S84), the processing unit Reference numeral 200 refers to the motion data library 516 of the player character initial setting data 510 and executes effect display according to the attached effect data 516c associated with the SP attack operation command 516a (step S86). Then, the motion control of the player character 6 according to the corresponding motion data 516b is started (step S88), and the player character behavior control process is ended. The motion control at this time may be configured to control the lower body model 6d of the player character 6 together or may be configured to control only the upper body model 6u.

  When there is no motion being executed and a gesture corresponding to the right avoidance or left avoidance operation command is detected by the gesture detection process in step S22 (YES in step S90), the processing unit 200 sets the left / right correspondence setting data 552. Referring to the left and right sides of the player screen of the side avoidance gesture detected depending on whether the virtual camera is shooting the player character 6 from the front or from the back in the game stage being played at that time The lateral avoidance motion control of the player character 6 is started so as to perform an avoidance action in the same direction as the direction or in the opposite direction (step S92), and the player character behavior control process is ended.

  If there is no motion being executed and a gesture corresponding to the bending avoidance operation command is detected (YES in step S94), the processing unit 200 starts the bending avoiding motion control of the player character 6 (step S96). Then, the player character behavior control process is terminated.

  Here, the types of gesture input that can be performed are limited to SP attack, side avoidance, and bend avoidance. However, when other gesture inputs are possible, the same applies to other gesture inputs. Processing shall be executed.

  When no gesture input is detected (NO in step S94), the processing unit 200 executes a damage determination process for the player character (step S98). That is, the hit determination of the attack of the enemy character 10 and the damage value due to the attack of the hit enemy are calculated.

  If the determined damage is a large damage due to a special attack by the enemy character 10 (YES in step S100) and the amount of damage received this time satisfies a predetermined site breakage condition (YES in step S102), the processing unit Reference numeral 200 denotes a random selection of any part from the arm parts such as the hand, the forearm, the upper arm, and the shoulder, and predetermined selection candidate parts such as the ears, and the selected part is set to be non-displayed ( Step S104). Further, an effect that the part is damaged is executed (step S106), and the motion control is started so that the player character 6 performs a damaging motion that falls upon receiving predetermined damage (step S108), and the player character behavior control process is ended. To do. Of course, if the amount of damage received this time does not satisfy the predetermined part damage condition, the part selection and non-display setting process and the effect display process related to part damage are skipped. In addition, the operation of other parts that remain in the display state without being selected as damaged parts is controlled as before.

  On the other hand, if the determined damage is not a large damage due to the special attack by the enemy character 10 (NO in step S100), the processing unit 200 is out of the determination area 50 of the left feature point 42 and the right feature point 44. (Step S120).

  When the left feature point 42 and / or the right feature point 44 is outside the determination region 50 (see FIG. 6) (FIG. 17; YES in step S120), the processing unit 200 performs an operation in which the player spreads his / her hand. Judge that Then, referring to the left / right correspondence setting data 552, the joint posture parameters of the arm corresponding to the left feature point 42 and / or the right feature point 44 exiting the determination area 50 are calculated by inverse kinematics (step S122). Then, the movement control of the arm is executed according to the calculated posture parameter (step S124), and the player character behavior control process is terminated. As a result, when the player 2 spreads his arm, the player character 6 moves to imitate it.

  Here, when the part is damaged, the damaged part is only hidden, so that the operation is controlled as before. In addition, when there exists a damaged part and the movement of the part is felt small, you may change operation control as follows. That is, for example, if the arm part beyond the elbow of the arm is damaged and the upper arm part remains, the left end point 42 / right feature corresponding to the tip of the remaining upper arm part and the arm Operation is controlled by resetting the points 44 so as to be associated with each other. In this case, the remaining upper arm portion moves greatly in response to a large movement of the left feature point 42 / right feature point 44 in the player image.

  On the other hand, when the left feature point 42 and / or the right feature point 44 are present in the determination region 50 (NO in step S120), the processing unit 200 puts the player 2 in a “pose with a gun”. The player character 6 controls to hold the gun 8 so that the muzzle of the gun 8 is directed toward the position in the game space indicated by the marker 4 (step S126), and the player character behavior control process is performed. finish.

  If there is a motion that is already being executed when the player character behavior control process is executed (YES in step S80), the processing unit 200 sets the current motion for the current control cycle (1). (Only for the frame) and execute the player character behavior control process.

When the player character behavior control process is completed, the process returns to the flowchart of FIG.
Next, when there is an input of an attack operation of the player character 6 (any one of an attack operation from the game controller 1230 and an attack operation by a gesture) (YES in step S26), the processing unit 200 determines that the enemy character 10 The damage determination process is executed (step S28), and the determined damage is reflected in the hit enemy character 10 (step S30).

  Next, the processing unit 200 determines whether or not the current game situation satisfies a predetermined game end condition (step S34). For example, in this embodiment, when the current hit point of the player character 6 is “0” or when all the enemy characters 10 are defeated, it is determined that the game is over, but detailed conditions can be set as appropriate.

  If the game end condition is not satisfied (NO in step S34), the process proceeds to step S18. If the game end condition is satisfied (YES in step S34), a game end process is executed (step S36), and if the current hit point of the player character 6 is “0”, a game over effect is displayed. When all the enemy characters 10 are defeated, a game clear effect display is performed, and the series of processes is terminated.

  As described above, according to the present embodiment, the player character 6 resembling the body shape and clothing color of the player 2 appears, and the player character 6 shoots toward the position of the marker 4 using the game controller 1230 as a pointing device. Can be realized.

  Moreover, the player character 6 is automatically controlled to hold the gun 8 toward the position of the marker 4 when the player 2 is posing to hold the game controller 1230 in front of the chest. When the player character 6 poses to spread his arms, the movement of the player character 6 is controlled so as to imitate the movement of the player 2's arm. Like the known TPS, the player character 6 imitates the action of the player 2 not only by controlling the action of the player character 6 by an operation input from the game controller 1230, but the player 2 embraces the player character 6. A sense of closeness is enhanced, and a sense of completeness and immersion in the game can be enhanced.

  In addition, using the detection of the pose of the player 2, operation input by gestures is also possible. The gesture operation input in the present embodiment can use the position and movement of a feature point such as a hand in addition to detecting the position where the body of the player 2 is captured in the player image. Not limited to this, various operation commands such as special attack (for example, magic activation), hit point recovery, bullet replenishment, etc. can be assigned.

  Then, by including an element that expands the hand in the gesture corresponding to the operation command, if the player 2 makes a gesture like an action associated with the activation of a technique in the special effects hero program, the player character 6 in the game screen also It mimics and realizes an unprecedented operational feeling such as invoking the corresponding action.

  In this embodiment, the upper character polygon model 512a of the player character initial setting data 510 is deformed in order to make the player character 6 resemble the body shape of the player. However, the present invention is not limited to this. For example, a plurality of polygon model data 512 are prepared according to the condition of the width Lb (see FIG. 5) that is the body shape estimation parameter value, and the game calculation unit 210 performs the model matching process in step S58 (see FIG. 15). A configuration may be adopted in which any one of the prepared polygon model data 512 is selected.

  Similarly, a configuration in which a plurality of polygon model data 512 is prepared and any one is selected according to a parameter obtained from the player image can be applied to the statistical color information of the human body silhouette range name. In this case, in the case of a player with red clothing, it can be changed to a rabbit design, and the player with blue clothing can be changed to a player character with a horse design.

[Second Embodiment]
Next, a second embodiment to which the present invention is applied will be described. This embodiment is basically realized by the same technology as that of the first embodiment. However, the player character 6 is not always displayed in the game screen, but basically, the human body silhouette of the player 2 is reversed left and right. The difference is that the player character 6 can be displayed when the silhouette is displayed and transformed into the player character 6 by a predetermined transformation operation input.
Hereinafter, the same reference numerals are given to the same components as those in the first embodiment, and the description thereof will be omitted, and only the differences from the first embodiment will be described.

FIG. 18 is a diagram for explaining the concept of reverse silhouette display as an alternative to the player character 6 in the present embodiment. In the present embodiment, while the game is being executed, the player image W12 including the whole body (or at least the upper body) of the player 2 is always photographed by the image sensor module 1227, and the human body silhouette 30 is extracted for each photographed frame. Then, a part 32 corresponding to the upper body part is cut out from the extracted human body silhouette 30, and if the virtual camera is set for a game stage where the player character is photographed from behind, the cut out part is subjected to left-right inversion processing to generate a determination silhouette 60. And display it at the center of the bottom of the game screen. Further, if the game stage is set so that the virtual camera shoots the player character from the front, the cut-out part is displayed as the inverted silhouette 60 at the center of the lower end of the game screen without being reversed horizontally.
The color of the inverted silhouette 60 is set to the same color as the statistical color information of the human body silhouette 30 in the same manner as the selection of the texture data applied to the player character 6.

  Since the inverted silhouette 60 is a horizontally reversed silhouette image, the game screen can be seen as if the player's back view is there. For players, they feel the illusion that they are in the game world.

  Further, although the player character 6 is not displayed on the game screen, it is arranged in the game space in a non-displayed state, and an attack hit determination from the enemy character 10 is performed on the player character 6 in the non-displayed state. Based on the determination.

  If it is determined that the non-displayed player character 6 has received a special attack from the enemy character 10, as shown in FIG. 19, a model for damage similar to a human body with a reverse silhouette 60 prepared in advance ( Of course, the display is replaced with a relative position so that only the upper body is captured with respect to the virtual camera), and the damage model 62 is caused to perform a predetermined motion. As a result, nothing has happened in the player image W18 actually captured by the image sensor module 1227, but it can be produced as if the player of the inverted silhouette 60 was flashed in the game screen W16. Of course, when the done motion by the damage model 62 is completed, the display is returned to the original inverted silhouette 60.

  Note that the display / non-display setting of the inverted silhouette 60 is managed by providing a silhouette display flag in the storage unit 500 and operating the flag. Similarly, the display / non-display setting of the damage model 62 is managed by providing a damage model display flag in the storage unit 500 and performing flag operation.

  Then, as shown in FIG. 20, in this embodiment, the changeable gauge 26 is controlled to be stored according to the progress of the game play time, and can be changed to the player character 6 when the specified value is stored as in the game screen W20. Become. The transformation operation is input when the player 2 makes a predetermined transformation gesture or takes a transformation pose.

  When the transformation operation input is detected, the player character 6 is displayed instead of the inverted silhouette 60 and the damage model 62 as in the game screen W24. When the player character 6 is transformed, an avoidance operation input or a special attack operation input by gesture input in the first embodiment can be performed, and higher operability and advantageous battle can be achieved than before the transformation. However, the transformation is set to be automatically canceled at a certain time. When the transformation possible time is counted down from the transformation start time and the transformation possible time becomes “0”, the inverted silhouette is automatically substituted for the player character 6. 60 is displayed and the transformation is solved.

  Whether or not the device is in the transformation state is managed by the transformation flag in the storage unit 500. The points displayed on the gauge 26 by the transformable gauge 26 are stored and managed in the storage unit 500 as transformable points.

FIG. 21 is a flowchart for explaining the flow of the player character behavior control process (player character behavior control process B) in the present embodiment.
In the same process, when there is no motion being executed (NO in step S80), the processing unit 200 determines whether or not it is currently being transformed (step S150).

If there is a transformation operation input (NO in step S150) in a state where transformation is not in progress (YES in step S152), the processing unit 200 determines whether the transformable gauge 26 has accumulated to a specified value. If the specified value is stored (YES in step S154), the transformable gauge 26 is consumed (step S156), and the display setting of the inverted silhouette 60 is performed by operating the silhouette display flag provided in the storage unit 500. Is set to display OFF, the transformation flag is set to “1 (transformation state)”, and the model of the player character 6 is set to the display state (step S158). Further, a predetermined transformation effect display (for example, an effect display in which the display position of the inversion silhouette 60 is wrapped with dazzling rainbow light) is started (step S160).
Of course, steps S156 to S160 are skipped if there is no transformation operation input even in a state where transformation is not being performed (NO in step S152), or if there is a transformation operation input and the transformable gauge 26 is not stored up to the specified value. .

  On the other hand, in the state of transformation (YES in step S150), the processing unit 200 counts down the possible transformation time for the current control cycle (step S162), and as a result, the transformation possible time becomes “0” ( In step S164, the silhouette display flag provided in the storage unit 500 is operated to set the display setting of the inverted silhouette 60 to display ON, and the transformation flag is set to “0 (untransformed state)”. 6 model is set to a non-display state, and transformation is solved (step S166).

  If the possible transformation time is not “0” and the transformation state is still continued (NO in step S164), the processing unit 200 determines whether or not there is an operation input by a gesture or pose other than transformation (step S180). Then, if there is an operation input by a gesture or pose other than transformation (YES in step S180), motion control of the upper body model 6u of the player character 6 is started with a motion corresponding to the input gesture (step S182). That is, steps S180 to S182 correspond to steps S84 to S96 (see FIG. 16) of the first embodiment.

  On the other hand, if there is no operation input by a gesture or a pose in the transformed state (NO in step S180), the processing unit 200 moves the upper body of the player character 6 so as to open the muzzle of the gun 8 to the place in the game space indicated by the marker 4. The operation is controlled (step S184).

  Any one of processing related to transformation (steps S154 to S160), processing related to transformation cancellation (step S166), motion control of the player character 6 in response to a gesture input (step S182), and posture control of the gun 8 (step S184) is executed. If so, the processing unit 200 next executes a damage determination process for the player character (FIG. 22; step S200). If it is determined that the enemy character 10 has received large damage due to a special attack (YES in step S202), and if it is not currently being transformed (NO in step S204), the processing unit 200 sets the display of the inverted silhouette 60. Is set to OFF, the display of the damage model 62 is set to ON (step S206), the damage model 62 is placed at the display position of the inverted silhouette 60 (step S208), and the damage model 62 is used for the damage with a predetermined burned motion. The motion control of the model 62 is started (step S210), and the player character behavior control process B is ended.

  On the other hand, if it is determined that the enemy character 10 has received a large damage due to a special attack (YES in step S202) and currently being transformed (YES in step S204), motion control of the player character 6 based on the struck motion is performed. Start and end the player character behavior control process B.

  When the player character behavior control process B is executed and there is a motion that is already being executed (YES in step S80), the processing unit 200 continuously executes the currently executed motion for the current control cycle (step S80). S126). If it is not currently being transformed (NO in step S220) and the motion has been finished (YES in step S222), the display of the inverted silhouette 60 is set to ON and the display of the damage model 62 is set to OFF (step). S224), the player character behavior control process B is terminated.

  FIG. 23 is a flowchart for explaining the flow of the game screen generation display process in the present embodiment. This process is a process executed instead of step S14 of the first embodiment. Specifically, when the display setting of the inverted silhouette 60 is ON (step S250 ON), the processing unit 200 renders a game space image in which the player character 6 is not displayed (step S252), and displays various information on this. Are synthesized (step S254).

  If the shooting position of the virtual camera on the currently executed game stage is set to shoot the player character 6 from behind (“from behind” in step S256), the upper body part of the human body silhouette 30 is cut out and reversed left and right. This is processed into the inverted silhouette 60 (step S258). On the other hand, if the virtual camera is set to shoot the player character 6 from the front ("from the front" in step S256), the processing unit 200 cuts out the upper body part of the human body silhouette 30 and flips it as it is without flipping left and right. The silhouette 60 is set (step S260).

  If the inverted silhouette 60 is generated, the processing unit 200 then sets the color of the inverted silhouette 60 so as to be a similar color based on the statistical color information of the human body silhouette 30 (step S262). Then, the inverted silhouette 60 is combined with the previously generated game space image at a predetermined position to generate a game image for one frame in the current control cycle (step S264), and this is displayed on the image display device 1222. (Step S290), the game screen generation display process is terminated.

  On the other hand, when the display setting of the reverse silhouette 60 is OFF and the display setting of the player character 6 is ON (OFF in step S250 → ON in S280), the game space image displaying the player character 6 is rendered (step S250). In step S282, a variety of information displays are combined with this to generate a game image in the current control cycle (step S288), which is displayed on the image display device 1222 (step S290), and the game screen generation / display processing ends.

  If the display setting of the player character 6 is OFF and the display setting of the damage model 62 is ON (step S284 ON), the game space image displaying the damage model 62 is rendered (step S286). Various information displays are combined with this to generate a game image in the current control cycle (step S288), which is displayed on the image display device 1222 (step S290), and the game screen generation / display process ends.

  As described above, according to the present embodiment, it is possible to realize a TPS that has the same effects as the first embodiment and displays the inverted silhouette 60 that is reversed left and right from an image obtained by shooting the player 2 from the front.

[Modification]
As described above, the first to second embodiments to which the present invention is applied have been described. However, the embodiments of the present invention are not limited to these configurations, and additions, omissions, and changes of components may be appropriately made. it can.

  For example, the game device is not limited to the home game device 1200 but may be a professional gun shooting game device 1100 as shown in FIG. The gun shooting game apparatus 1100 can be realized by adding an image sensor module 1227 to a known game apparatus used, for example, with a work name “Time Crisis 4” manufactured by NAMCO BANDAI Games. Specifically, the gun shooting game device 1100 includes a gun-type controller 1130 that simulates a gun, an image display device 1122, a speaker 1124, a coin detection sensor 1144 that detects coins inserted from a coin insertion slot 1142, and The apparatus main body 1101 includes a control unit 1150.

  The control unit 1150 corresponds to a control board of the game apparatus, and includes various microprocessors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a VRAM, and the like. Various IC memories such as a RAM and a flash memory 1152 are mounted. In addition, a so-called I / F circuit (interface circuit) such as a communication device 1154, an amplifier circuit for outputting an audio signal to the speaker 1124, and a signal input / output circuit with the gun-type controller 1130 and the coin detection sensor 1144 is mounted. Each element mounted on the control unit 1150 is electrically connected via a bus circuit so that data can be read and written and signals can be transmitted and received.

  The flash memory 1152 stores programs and various setting data necessary for executing various arithmetic processes related to game play. When the coin detection sensor 1144 detects the insertion of a predetermined amount of coins, the control unit 1150 reads the program and data from the flash memory 1152 and temporarily stores them in the IC memory to be mounted. Then, the read program is executed, a game image is generated and displayed on the image display device 1122, and a game sound is generated and emitted from the speaker 1124.

The player stands in front of the screen of the image display device 1122 and holds the gun-type controller 1130 so as to aim at the screen. In the game screen, an aim 6 indicating the target and the position aimed at by the gun-type controller 1130 is displayed. Therefore, the gun-type controller 1130 is held so that the aim 6 is aligned with an arbitrary target in the game screen, and the trigger You can enjoy shooting games by pulling and shooting.
The program and various setting data necessary for executing the game on the gun shooting game device 1100 are read from the flash memory 1152, but the communication device 1154 is connected to the communication line 1 and downloaded from an external device. It is also good.

  Moreover, although the example which performs TPS was given in the said embodiment, if it is a game by a 3rd person viewpoint, not only a gun shooting game but the game of other genres, such as RPG, may be sufficient.

2 player 4 pointer 6 player character 8 gun 30 human body silhouette 40 head feature point 42 left feature point 44 right feature point 50 determination area 60 inversion silhouette 62 damage model 100 operation input unit 102 optical signal light receiving unit 110 player photographing unit 200 processing Unit 210 game calculation unit 212 pointer control unit 214 player character movement control unit 216 virtual camera control unit 218 human body silhouette extraction unit 220 feature point information acquisition unit 222 gesture determination unit 224 player character behavior control unit 500 storage unit 502 game program 504 game space Setting data 510 Character initial setting data 530 Gesture dictionary data 550 Feature point data 552 Left / right correspondence setting data 1200 Home game device 1201 Device main body 12 DESCRIPTION OF SYMBOLS 10 Control unit 1220 Video monitor 1222 Image display apparatus 1226 Optical signal output apparatus 1227 Image sensor module 1230 Game controller

Claims (12)

A program for causing a computer including a digital photographing device for photographing a player to generate an image of a game space in which a character model is arranged and to execute a given game,
Character movement control means for controlling movement of the character model;
Virtual camera control means for arranging a virtual camera at a position away from the character model by a predetermined distance and controlling movement according to the movement of the character model;
Image generating means for generating an image of the game space including the character model based on the virtual camera;
Model behavior control means for controlling behavior of the character model based on a player image photographed by the photographing device;
A program for causing the computer to function as The character model has left and right arms,
When the player faces the display device, the right arm of the player corresponds to the right arm of the character model viewed from the virtual camera, and the left arm corresponds to the left arm of the character model viewed from the virtual camera Left and right relationship setting means for setting a correspondence relationship between the left and right of the player image and the left and right of the character model,
Arm part determination means for determining a left arm part and a right arm part in the player image photographed by the photographing device;
Further functioning the computer as
The model behavior control means includes arm control means for controlling the left arm and the right arm of the character model based on the position change of the left or right arm portion of the corresponding player image, respectively, according to the correspondence relationship. Make the computer work,
A program according to claim 1 for. The game is a game in which the character model battles enemy characters,
When the character model is subjected to a partial part destruction attack of the arm, the part disappears,
The arm control means causes the computer to function so as to control the existing left arm and right arm of the character model based on the position change of the left or right arm portion of the corresponding player image according to the correspondence relationship. ,
A program according to claim 2 for. Causing the computer to function as player left / right movement detecting means for detecting that a player image in the player image photographed by the photographing device has moved to either the left or right side;
The model behavior control means causes the computer to function so as to have an inclination posture control means for controlling the posture so that the character model is inclined in a direction corresponding to a detection direction by the player left-right movement detection means according to the correspondence relationship.
The program of Claim 2 or 3 for.   The computer includes the correspondence relationship inversion means for inverting the correspondence relationship between left and right depending on whether the orientation of the virtual camera faces the front side or the back side of the character model. The program as described in any one of Claims 2-4 for functioning.   The model behavior control means includes a bending action control means for causing the character model to bend based on a change in a vertical length of a player image in a player image shot by the shooting apparatus. The program as described in any one of Claims 1-5 for functioning a computer. The model behavior control means is
Upper body behavior control means for controlling behavior of the upper body of the character model based on a player image photographed by the photographing device;
Lower body behavior control means for controlling behavior of the lower body of the character model based on movement control by the character movement control means;
The program according to any one of claims 1 to 6 for causing the computer to function so as to include   8. The computer according to claim 1, wherein the computer functions as character model setting means for selecting or generating the character model from a plurality of types of character models based on the player image in the player image. The program as described in any one.   The program according to claim 8, wherein the computer functions so that the character model setting unit includes a coloring unit that sets a color of the character model based on a color of the player image in the player image.   10. The computer according to claim 8, wherein the character model setting means causes the computer to function so as to have body shape setting means for setting a body shape of the character model based on a contour shape of the player image in the player image. Program.   The computer-readable information storage medium which memorize | stored the program as described in any one of Claims 1-10. A game device that includes a digital photographing device for photographing a player, generates an image of a game space in which a character model is arranged, and executes a predetermined game.
Character movement control means for controlling movement of the character model;
Virtual camera control means for disposing a virtual camera at a predetermined distance from the character model and controlling movement according to the movement of the character model;
Image generating means for generating an image of the game space including the character model based on the virtual camera;
Model behavior control means for controlling the behavior of the character model based on a player image photographed by the photographing device;
A game device comprising:

Images