JP2021005181A - Information processing system, information processing method and computer program - Google Patents

Abstract

To provide a system for improving a user's sense of immersion into a virtual space.SOLUTION: An information processing system for generating a moving image including an animation of a character object generated on the basis of the behavior of a user is provided with one or more computer processors. The one or more computer processors include: a reception function for receiving an input of behavior data based on the behavior of the user by executing a computer readable instruction; and a processing function for generating moving image data including the character object which performs a behavior following the behavior data. The processing function generates moving image data for making the character object perform expression following preliminarily prepared second behavior data in place of movement following first behavior data in the case that the reception function receives the first behavior data based on a specific behavior of the user.SELECTED DRAWING: Figure 9

Description

The present invention relates to information processing systems, information processing methods and computer programs.

In a virtual reality game (hereinafter referred to as "VR game"), a user can immerse himself in the game by wearing a head-mounted display (hereinafter referred to as HMD) as a display device on his head. Generally, the operation of the game in the VR game is performed by the operation device held by the user (see Patent Document 1), and the character object in the moving image to be displayed on the HMD based on the user's action input by the operation device. An animation of (player character) is generated.

Further, there is also known a video distribution system that generates an animation of a character object (avatar) generated based on a user's behavior and distributes a video including the animation.

Non-Patent Document 1 discloses a moving image distribution system that generates an animation of a virtual idol based on motion capture data representing the movement of an actor and delivers a moving image including the animation live.

In the generation of a conventional VR game video or a virtual idol video, behavior data such as user operation data and user movements are faithfully reflected in the behavior of the character object in the video.

However, if all of the above behavior data is reflected in the behavior of the character object, there is a problem that even awkward behavior of the user is reflected and the quality of the generated moving image is lowered.

Further, if all of the above behavior data is reflected in the behavior of the character object, there is a problem that the user's feeling of immersion in the VR space is impaired.

These problems have a problem that not only the user's motivation to act is reduced, but also the viewing motivation of other users and viewing users who see this video is reduced.

Japanese Unexamined Patent Publication No. 2017-217196

Yusuke Hasegawa, Dawn of a new era of virtual idols PROJECT MariA, CG WORLD, Japan, Bone Digital Co., Ltd., November 10, 2017, Volume 231 and pp. 74-79

One object of the present invention is to provide technical improvements that solve or alleviate at least some of the problems of the prior art described above. One of the more specific objects of the present invention is to improve the quality of the generated moving image. Further, one of the more specific objects of the present invention is to improve the user's feeling of immersion in the VR space. Further, one of the more specific objects of the present invention is to improve the user's motivation to act and also to improve the viewing motivation of other users and viewing users who see this moving image.

Other objectives of the disclosure herein will become apparent by reference to the entire specification. The invention disclosed in the present specification may solve the problem grasped from the description of the mode for carrying out the invention of the present specification in place of or in addition to the above-mentioned problem. Good.

The information processing system of the present invention is an information processing system that generates a moving image including an animation of a character object generated based on a user's behavior, and includes one or a plurality of computer processors, wherein the one or a plurality of computer processors are provided. By executing a computer-readable command, a reception function that accepts input of action data based on the user's behavior and a processing function that generates video data including a character object that performs an action according to the behavior data are realized and processed. The function is that when the reception function receives the first action data based on the user's specific action, the character object is expressed according to the second action data prepared in advance instead of the action according to the first action data. It is characterized in that the moving image data in which the above is performed is generated.

The processing function can display a state display indicating to the user that the moving image data in which the character object is expressed according to the second action data is generated.

When the reception function receives the third action data based on the user's specified action, the processing function replaces the action according to the first action data received after the reception function receives the third action data. It is possible to generate moving image data in which the character object is expressed according to the second action data.

The processing function can display the input guide of the first action data to the user.

When the reception function receives the third action data based on the user's specified action, the processing function displays the input guide of the first action data to the user, and the reception function is the first after the input guide is displayed. When one action data is received, it is possible to generate moving image data in which the character object expresses according to the second action data prepared in advance instead of the action according to the first action data.

When the action data started to be accepted by the reception function satisfies the first condition, the processing function uses the action data as the first action data and replaces the character object with the action according to the first action data in advance. Generates video data that is expressed according to the prepared second action data, and if the action data received by the reception function does not meet the first condition, the character object follows the action according to the first action data. It is possible to generate video data in which an action is performed.

The processing function determines the achievement rate by comparing the behavior data with the model behavior data, and when the achievement rate exceeds a predetermined value, it can be determined that the behavior data satisfies the first condition.

The processing function is divided into a case where the first action data received by the reception function satisfies the second condition and a case where the first action data received by the reception function does not satisfy the second condition. The effects of expressions that follow behavioral data can be different.

The processing function compares the behavior data with the model behavior data to determine the accuracy rate within a predetermined time, and when the accuracy rate exceeds a predetermined value, the first behavior data satisfies the second condition. It can be judged that it was.

The processing function can display a status display indicating the accuracy rate to the user.

The information processing method of the present invention is an information processing method in an information processing system that generates a moving image including an animation of a character object generated based on a user's behavior, and is applied to one or more computer processors to the user's behavior. The reception step that accepts the input of the behavior data based on the behavior data and the processing step that generates the video data including the character object that performs the behavior according to the behavior data are executed. In the processing step, the first step based on the specific behavior of the user in the reception step When the action data is received, the character object is characterized in that it generates moving image data in which the character object is expressed according to the second action data prepared in advance instead of the action according to the first action data.

The computer program of the present invention is a computer program for generating a moving image including an animation of a character object generated based on a user's behavior, and causes one or a plurality of computer processors to generate behavior data based on the user's behavior. The reception function that accepts input and the processing function that generates video data including character objects that perform actions according to the action data are realized, and the processing function accepts the first action data based on the user's specific action. In this case, the character object is characterized in that it generates moving image data in which an expression according to a second action data prepared in advance is performed instead of an action according to the first action data.

According to the present invention, it is possible to provide technical improvements that solve or alleviate at least a part of the problems of the prior art described above. Specifically, according to the information processing system, the information processing method, and the computer program of the present invention, the quality of the generated moving image can be improved. Further, according to the present invention, it is possible to improve the user's feeling of immersion in the VR space. Furthermore, according to the information processing system, the information processing method, and the computer program of the present invention, it is possible to improve the user's motivation to act and also to improve the viewing motivation of other users and viewing users who see this moving image. Can be done.

It is a system block diagram which shows an example of the information processing system which concerns on embodiment of this invention. It is a system block diagram which shows an example of the information processing system which concerns on embodiment of this invention. It is a conceptual diagram which shows an example of the display screen which concerns on embodiment of this invention. It is a conceptual diagram which shows an example of the display screen which concerns on embodiment of this invention. It is a conceptual diagram which shows an example of the display screen which concerns on embodiment of this invention. It is a conceptual diagram which shows the concept of achievement rate which concerns on embodiment of this invention. It is a conceptual diagram which shows the concept of the accuracy rate which concerns on embodiment of this invention. It is a conceptual diagram which shows an example of the display screen which concerns on embodiment of this invention. It is a block diagram of a game processing system. It is a figure which shows the coordinate system used for specifying the position and orientation of a player's head. It is a figure which shows the frequency spectrum. It is a figure which shows the text data. It is a flowchart which shows the process when the voice input information is input. It is a perspective view which shows the state of drawing in a three-dimensional virtual space with a pen or stick type operation device. It is a figure which shows the virtual space where a locus object is drawn. It is a figure which shows the locus object. It is a flowchart which shows the process when a locus object is input as input information. It is a figure which shows the state before the additional object is drawn with respect to the weapon (sword) object. It is a figure which shows the state which the additional object is drawn with respect to the weapon (sword) object. It is a figure which shows the hardware configuration of a game processing apparatus.

An example of the information processing system according to the embodiment of the present invention will be described.

The information processing system according to the embodiment of the present invention is an information processing system that generates a moving image including an animation of a character object generated based on a user's behavior, and includes one or a plurality of computer processors.

FIG. 1 is an example of an information processing system according to an embodiment of the present invention, and shows a configuration of a VR game system. As shown in FIG. 1, the VR game system 100A of the present invention includes a server device 200 and a plurality of player terminals 300 connected to the server device 200 via a network 600.

In the VR game system 100A, it is assumed that the character objects of other players can be displayed on each other on the plurality of player terminals 300.

In addition, the method of displaying one's character object may be a first-person viewpoint or a third-person viewpoint. Such switching of viewpoints can be performed by a user operation.

Similarly, FIG. 2 is an example of an information processing system according to an embodiment of the present invention, and shows a configuration of a moving image distribution system. As shown in FIG. 2, the moving image distribution system 100B of the present invention includes a server device 200, a distribution user terminal 400 and a viewing user terminal 500 connected to the server device 200 via a network 600.

In the video distribution system 100B, the number of distribution user terminals 400 may be plural. In this case, a plurality of distribution users may co-star to generate one moving image.

The video including the animation of the character object generated based on the user's behavior is a video displayed on the screen of a VR game performed by one or more players, a video displayed on the screen of the distribution user terminal, and one or more. Includes videos for distribution displayed on the screen of the viewing user's terminal.

Then, the one or more computer processors realize the reception function and the processing function by executing a computer-readable instruction.

The reception function accepts input of behavior data based on the user's behavior.

The user's behavior includes the user moving a body or facial expression and / or the user uttering a word or sound.

The behavior data can be, for example, data output from a sensor (controller or the like) worn or carried by the user, and / or information based on the analysis result of an image of the user's behavior.

The processing function generates moving image data of a moving image including a character object that performs an action according to the above action data.

The character object that performs an action according to the action data means a character object that reflects the action of the user.

Further, the moving image including this character object may include a background image or another object in addition to the character object.

Then, when the reception function receives the first action data based on the user's specific action, the processing function causes the character object to perform the second action prepared in advance instead of the action according to the first action data. It is characterized by generating moving image data in which expressions are performed according to the data.

The first behavioral data is based on the user's specific behavior. The specific behavior of the user is a predetermined behavior, and is distinguished from the above-mentioned user behavior (behavior other than the predetermined behavior).

Examples of predetermined actions include actions such as "waving down the hand three times" or "saying" magic activation "". Such an action may be stored in a predetermined storage device as an initial setting, or may be freely determined / selected by the user. Such behavior can be specifically determined using the value of a predetermined sensor of the controller worn or carried by the user. As another example of the predetermined action, a pattern such as a magic circle may be drawn with a wand (action), a spell may be cast (voice), a sword may be swung (action), or a setting operation may be performed. The cane or sword may be the one actually held by the user or a fictitious one on the VR screen.

In the case of casting a spell (voice), set the spell in advance so that even if the spell is composed of several clauses, it can be judged by the first few letters (about 3 to 5 letters as an example). Can be left. Such information is referred to as voice reference information, and details will be described later.

The second action data is prepared in advance in place of the action according to the first action data, and relates to the expression performed by the character object.

The expression according to the second action data is displayed for other users, and may be displayed in another expression on the screen of the input player (user) in the case of a third-party viewpoint.

Examples of expressions performed by the character object include expressions such as "activating a special skill" or "activating magic". Such representations may include auditory or visual effects in addition to the animation of the character object.

For example, even if the user draws a magic circle with an unfamiliar action as the first action, the magic circle is drawn with a sharp action as an expression according to the second action data on the character object, and powerful magic is drawn. It is possible to make an expression that activates.

According to the above configuration, it is possible to provide a technical improvement that solves or alleviates at least a part of the problems of the prior art described above.

Specifically, according to the above configuration, even if the behavior of the user who is the performer is awkward, the character object can be made to perform a good-looking behavior prepared in advance in place of the behavior. The quality of the generated video can be maintained or improved.

Further, according to the above configuration, even if the behavior of the user who is the performer is awkward, the character object can be made to perform a good-looking behavior prepared in advance in place of the behavior, so that the user's VR can be performed. It is possible to improve the immersive feeling in the space.

Regarding the input determination of the specific action described above, for example, after the operation of the specific action that is the first action data is completed, the determination process is completed with the lapse of a predetermined time, and the second action data. You can activate expressions that follow. In addition, if a specific action that becomes another first action data is performed before the end time, the end determination can be performed again.

That is, in the information processing system of the present invention, the processing function can have an input determination means. Specifically, the input determination means determines whether or not the motion input of the specific action data exceeds a predetermined time. By this determination, it is determined that the action is completed when the predetermined time is exceeded after the operation of the specific action data is completed. For example, after the end of the action of the specific action that is the first action data, the determination process is completed and the expression according to the second action data can be activated after a lapse of a predetermined time.

In addition, the input determination means may perform the input determination process of the other specific action again when another specific action is performed after the operation of the specific action is completed and before the predetermined time is exceeded. it can.

Further, by improving the quality of the generated moving image, it is possible to improve the user's motivation to act and also to improve the viewing motivation of other users and viewing users who see this moving image.

The processing function in the embodiment of the present invention can display a status display to the user.

The state display indicates that the moving image data in which the character object is expressed according to the second action data is generated.

FIG. 3 is an example of a screen when the character object is made to perform an action according to the first action data, and FIG. 4 is a screen when the character object is made to perform an expression according to the second action data. This is an example. In FIGS. 3 and 4, character C1 on which the user's behavior is reflected and character C2 on which the behavior of another user is reflected are displayed on the screen. In FIG. 4, the position and size of the character C2 are changed from those in FIG. Further, in FIG. 4, the character C1 is expressed to activate a special move while dancing with a ribbon as a weapon. In FIG. 4, a predetermined mark M1 such as a reproduction mark is displayed on the upper right of the screen as an example of the state display. The mode of this state display is not limited to this.

According to such a configuration, the user can clearly recognize whether the character object is performing an action according to the action data based on its own action or an expression according to the second action data.

When the processing function in the embodiment of the present invention receives the third action data, the processing function causes the character object to perform the first action instead of the action according to the first action data received after the reception function receives the third action data. It is possible to generate moving image data in which expressions are performed according to the second behavior data.

The third behavior data is based on the user's prescribed behavior.

The prescribed action can be, for example, an action such as "waving the right hand three times". Such behavior can be specifically defined using the values of a predetermined sensor of the controller worn or carried by the user. Alternatively, using the buttons provided on the controller, inputs such as "press the A button once" and "press and hold the X button" can be set as the prescribed actions.

The third behavior data is a signal before the user performs a specific behavior. Therefore, it is preferable to specify a simple action rather than the first action data. Specifically, a simple action is an action that can be completed in a shorter time or less movement than a specific action.

Further, the second behavior data may be selected by the combination of the third behavior data and the specific behavior. For example, regarding the default action using controller input as described above, by pressing one button and inputting "fire" by voice, an attack with a gun, which is a weapon, is activated, and while pressing the other button, "fire" is activated. It may be assumed that a magical attack is activated by performing voice input.

According to such a configuration, the condition regarding the accuracy of the user's behavior for recognizing the user's behavior as a specific behavior can be relaxed as compared with the case where the third behavior data is not available.

The processing function in the embodiment of the present invention can display the input guide of the first action data to the user.

The input guide is a sample of the first behavioral data.

Specifically, as shown in FIG. 5, the input guide G1 is displayed at a predetermined position on the screen. The predetermined position and the size outside the input can be appropriately set according to the progress of the game.

According to such a configuration, the condition regarding the accuracy of the user's behavior for the user's behavior to be recognized as a specific behavior can be relaxed as compared with the case where the input guide is not provided.

The processing function in the embodiment of the present invention can display the input guide of the first action data to the user when the reception function receives the third action data based on the action specified by the user.

Then, when the reception function receives the first action data after the input guide is displayed, the character object is made to perform an expression according to the second action data prepared in advance instead of the action according to the first action data. Video data can be generated.

For example, when a user casts a spell as the third action data, a guide for drawing a magic circle can be displayed as an input guide, and the action of drawing the magic circle by the user can be used as the first action data. As described above, the judgment related to the spell casting at this time can be set in advance so that the judgment can be made by the first few letters.

According to such a configuration, the condition regarding the accuracy of the user's behavior for the user's behavior to be recognized as a specific behavior can be relaxed as compared with the case where the third behavior data and the input guide are not provided. it can.

When the action data started to be accepted by the reception function satisfies the first condition, the processing function according to the embodiment of the present invention uses the action data as the first action data and follows the character object according to the first action data. Instead of the action, it is possible to generate moving image data in which the expression according to the second action data prepared in advance is performed.

Then, when the action data received by the reception function does not satisfy the first condition, the processing function can generate moving image data in which the character object is made to perform an action according to the action according to the first action data.

The period until it is determined that the first condition is satisfied (the period during which the input of the first action data is waiting) is also included in the case where the first condition is not satisfied, and the character object is the first. It may be made to perform an action according to an action according to the action data.

Alternatively, the character object may be made to perform an expression according to another fourth action data until it is determined that the first condition is satisfied. The fourth action data preferably represents, for example, a motion of accumulating power, a motion of chanting a spell, etc., and the reception function may be applied after receiving the third action data. it can.

The expression according to the fourth action data is displayed for other users, and may be displayed in another expression on the screen of the input player (user) in the case of a third-party viewpoint.

According to such a configuration, it is possible to generate a high-quality moving image without stopping the action of the character object in the moving image.

Subsequently, the first condition described above will be described in detail.
The processing function in the embodiment of the present invention determines the achievement rate by comparing the behavior data with the model behavior data, and determines that the behavior data satisfies the first condition when the achievement rate exceeds a predetermined value. can do.

The model behavior data may be stored in a predetermined storage device as an initial setting, or may be determined / selected by the user.

The achievement rate can be a numerical value indicating "how far it has been achieved". For example, as shown in FIG. 6, when there is data for drawing a heart-shaped figure as model action data, 100% when all the heart-shaped figures are drawn, 50% when half is drawn, and so on. In addition, the achievement rate is determined.

Further, for example, the achievement rate when the action data is voice data can be determined based on the reference frequency spectrum 51 or the text data described later. Details will be described later.

The case where the achievement rate exceeds a predetermined value can be, for example, a case where it exceeds 50%, but is not limited to this. For example, the difficulty level of the game may be changed according to the user by changing a predetermined value of the achievement rate.

In addition, a time limit may be set to determine the satisfaction of the first condition. By changing the time limit, the difficulty level of the game can be changed according to the user.

Further, the processing function includes a case where the first action data received by the reception function satisfies the second condition and a case where the first action data received by the reception function does not satisfy the second condition. The effect of the expression according to the second behavior data can be different.

The effect of the expression according to the second action data is, for example, when the expression according to the second action data is the activation of recovery magic, the recovery of physical strength is the effect.

To make the effects different may mean, for example, different strengths of the effects or different effects themselves.

Subsequently, the second condition will be described in detail.
The processing function in the embodiment of the present invention determines the accuracy rate by comparing the behavior data with the model behavior data, and when the accuracy rate exceeds a predetermined value, the first behavior data satisfies the second condition. It can be judged that it is satisfied.

The accuracy rate can be a numerical value representing "how it was done". For example, as shown in FIG. 7, when there is data for drawing a heart-shaped figure as model behavior data, 100% when all the heart-shaped figures match, and the ratio when there is a deviation part. The accuracy rate is determined, such as the corresponding numerical value.

The case where the accuracy rate exceeds a predetermined value can be, for example, a case where it exceeds 50%, but is not limited to this. For example, the difficulty level of the game may be changed according to the user by changing a predetermined value of the accuracy rate.

In addition, a time limit may be set to determine the satisfaction of the second condition. By changing the time limit, the difficulty level of the game can be changed according to the user.

Then, the processing function can display a status display indicating the achievement rate and / or the accuracy rate to the user.

FIG. 8 is an example of a screen in which a status display M2 indicating an achievement rate and / or an accuracy rate is displayed at a predetermined position on the screen. As described above, in FIG. 8, characters such as "achievement rate 50%" are displayed at the lower left of the screen as an example of the state display, but the mode of this state display is not limited to this. For example, the brilliance of the trajectory of the heart drawn by the user may be changed according to the achievement rate or the accuracy rate, and this may be used as a state display.

The conditions for satisfying these achievement rates and accuracy rates may change depending on the skill. Specifically, the guide (determination of accuracy rate) becomes loose, and the required achievement rate / accuracy rate fluctuates. As an example, character skills owned by character objects (eg, auxiliary effects of characters such as swordsmanship I and water magic I), armor skills (eg, auxiliary effects of fire magic with weapons with fire attributes), job skills (Example: In the case of a swordsman, the achievement rate of swordsmanship skill / improvement of accuracy rate, the achievement rate of magic skill / decrease of accuracy rate) can be set.

Subsequently, the game processing system to which the present invention is applied will be described with reference to the drawings. As shown in FIG. 9, the game processing system to which the present invention is applied includes a head-mounted display (hereinafter, HMD) 10, a game processing device 20, and an operating device 30. The HMD 10 and the game processing device 20 communicate with each other by wire or wirelessly. In addition, wired or wireless communication is also performed between the operating device 30 and the HMD 10 and the game processing device 20.

The HMD 10 is attached to the head of the player P as shown in FIG. 14, and the operating device 30 is operated by the player P. The game processing device 20 displays a virtual space image of the game on the HMD 10 from a viewpoint according to the position and orientation of the HMD 10, and based on the information detected by the HMD 10 and the operating device 30, the progress of the game performed in the virtual space is performed. to manage. One HMD 10 is associated with one operating device 30. Therefore, the HMD 10 and the operating device 30 associated with each other are used by the same player. The game processing system may have a plurality of pairs of the HMD 10 and the operating device 30, and in the case of a plurality of pairs, a plurality of people can play a game at the same time.

[HMD]
The HMD 10 includes an HMD control unit 11, a sensor 12, a display 13, a speaker 14, and a microphone 15. The HMD 10 outputs various information to the game processing device 20, and also inputs various information from the game processing device 20.

The sensor 12 detects the position and orientation of the HMD 10 in the real space. The sensor 12 includes at least one such as a gyro sensor, an acceleration sensor, and a geomagnetic sensor. For example, the sensor 12 may further include a line-of-sight sensor to directly detect the direction in which the player's line of sight is directed (line-of-sight direction).

The HMD control unit 11 calculates the orientation (viewpoint and line-of-sight direction) of the head of the player P using the detection result of the sensor 12. The HMD control unit 11 may calculate the position of the head in addition to the direction of the head of the player P. When calculating the direction of the head, the HMD control unit 11 calculates it as an angle around each axis. When calculating the position of the head, the HMD control unit 11 calculates the position in the three axial directions. For example, as shown in FIG. 10, the three axes are an X-axis along the horizontal direction, a Y-axis along the vertical direction, and a Z-axis along the depth direction. The angles around the axis are pitch θp indicating the rotation direction centered on the X axis, yaw θy indicating the rotation direction centered on the Y axis, and roll θr indicating the rotation direction centered on the Z axis.

The HMD control unit 11 includes an input / output interface and transmits the direction of the head of the player P to the game processing device 20. When the position of the head of the player P of the HMD control unit 11 is calculated, the position of the head is transmitted to the game processing device 20. The game processing device 20 calculates the position and orientation of the player P in the virtual space V (the viewpoint of the camera and the line-of-sight direction of the camera) from the position and orientation of the head of the player P.

The display 13 is an example of a display device. The display 13 outputs a parallax image so that the player P can visually recognize a three-dimensional image with a sense of depth. The game processing device 20 extracts image data for outputting a parallax image based on the position and orientation of the player P in the virtual space V. The game processing device 20 transmits the extracted image data to the HMD 10. The HMD control unit 11 uses the signal received from the game processing device 20 to display the image of the virtual space V on the display 13.

The speaker 14 outputs sound effects, sounds, and game sounds such as music to be played in the background of the parallax image. The game processing device 20 grasps the progress of the game and generates sound data for outputting a sound suitable for the progress of the game. The game processing device 20 transmits the generated sound data to the HMD 10. The HMD control unit 11 outputs the audio signal received from the game processing device 20 to the speaker 14.

The microphone 15 converts the voice of the player P into an electric signal and generates a voice signal. In the microphone 15, the player P can input the voice input information which is the operation instruction of the game by voice. The game processing device 20 generates operation information based on voice input information, and accepts operations based on the operation information in the game processing. For example, voices such as attack methods (punch, kick, missile launch, etc.), defense methods (hide, escape, lower, barrier, etc.) and spells necessary for the progress of the game are input.

[Operating device]
The operation device 30 is configured to be able to communicate with the HMD 10 associated with the operation device 30 by wire or wirelessly, and the player P holds the operation device 30 in his hand, or the player P holds the operation device 30 in his hand or arm. It is carried by the player P by being attached to the player P. The operation device 30 is, for example, a controller dedicated to a game and a smart device such as a smartphone. Further, the operating device 30 is a pen or stick type controller, and by detecting the locus of the pen or stick type controller, drawing can be performed in the three-dimensional virtual space displayed by the HMD 10. Such an operation device 30 includes an information processing unit 31, a tracking information detection unit 32, and an operation unit 33.

The information processing unit 31 includes a processor, a memory, a communication interface, and the like. The information processing unit 31 outputs a signal corresponding to the detection result of the sensor 12 and a signal corresponding to the operation to the operation unit 33 to the HMD 10. The HMD 10 outputs a signal corresponding to the signal input from the operation device 30 to the game processing device 20.

The tracking information detection unit 32 includes a device that detects information for calculating the position and orientation of the operation device 30. The position of the operating device 30 corresponds to the position of the hand of the player P. The tracking information detection unit 32 includes, for example, at least one of a gyro sensor, an acceleration sensor, and a geomagnetic sensor. The position of the operating device 30 calculated from the detection result of the tracking information detection unit 32 may be a position relative to the initial position of the operating device 30, or relative to the position of the HMD 10 associated with the operating device 30. It may be an absolute position in the real space.

The operation unit 33 is, for example, a button type that the player P pushes and operates, or a lever type that moves up, down, left, and right, and is configured to be operable without the player P visually observing. Further, the operation unit 33 may be a touch panel. The operation device 30 directly communicates with the game processing device 20 without going through the HMD 10, and outputs a signal according to the detection result of the tracking information detection unit 32 and a signal according to the operation to the operation unit 33. It may be output to 20. In this case, the association between the HMD 10 and the operation device 30 may be managed by storing the association of the identification information between the HMD 10 and the operation device 30 in the game processing device 20.

For example, the tracking information detection unit 32 includes a position detection device for detecting information mainly used for calculating the position of the HMD 10. Specifically, the position detection device can detect information for calculating the orientation of the HMD 10. The position detection device may be used to calculate the position in addition to the orientation of the HMD 10.

Further, in addition to the tracking information detection unit 32, a tracking auxiliary device 36 may be provided. For example, one of the position detection device and the tracking assist device 36 is a light emitting device such as a laser device or an LED that emits infrared light or visible light, and the other of the position detection device and the tracking assist device 36 is. It is a light receiving device such as a camera that receives light corresponding to the above light emitting device. Then, which of the plurality of light emitting devices having the predetermined arrangement receives the light from the light receiving device, and which of the plurality of light receiving devices having the predetermined arrangement receives the light from the light emitting device. Information such as whether the light was received, the direction in which the light is emitted from the light emitting device, the time required for the light emitted from the light emitting device to reach the light receiving device, etc. is detected, and based on this information, the position of the HMD 10 and the position The orientation is calculated. When the tracking auxiliary device 36 is a light receiving device, or when the tracking auxiliary device 36 is a light emitting device and the game processing device 20 controls the light emission, the tracking auxiliary device 36 directly with the game processing device 20. Alternatively, it may be configured so that communication is possible via the HMD 10.

It should be noted that the tracking information detection is performed so that the position and orientation of the HMD 10 can be calculated by using only the detection result of the tracking information detection unit 32 included in the HMD 10 without using the tracking auxiliary device 36 installed outside the HMD 10. The unit 32 may be configured. The position of the HMD 10 calculated from the detection result of the tracking information detection unit 32 may be a position relative to the initial position of the HMD 10, or a predetermined position located in the real space, that is, when the player P plays the game. It may be an absolute position in a three-dimensional space of the size of.

[Game processing device]
The game processing device 20 is composed of various hardware elements such as a central processing unit, a memory, and an integrated circuit for a specific application. The game processing device 20 includes a control unit 21 and a storage unit 22, which are examples of control devices. The game processing device 20 and the HMD 10 may have a relationship in which one HMD 10 is connected to one game processing device 20, or a plurality of HMD 10s are connected to one game processing device 20. It may be a relationship.

The control unit 21 causes the hardware element to execute the game processing program, and causes the progress management unit 21A, the scene determination unit 21B, the voice recognition unit 21C, the first determination unit 21D, the first action control unit 21E, the drawing unit 21F, and the first. 2 Functions as a determination unit 21G and a second action control unit 21H. The storage unit 22 stores various information used for the progress of the game. The information stored in the storage unit 22 includes image information 22A, object information 22B, and game sound information 22C. In addition, the game processing program is installed in the storage unit 22 from an external recording medium such as a network, an optical disk, a hard disk, or a USB memory.

The image information 22A is an example of virtual space information, and is information for displaying an image of the virtual space V as shown in FIG. The image of the virtual space V includes an image that is a background of the virtual space V and an image of an object existing in the virtual space V. The objects existing in the virtual space V include the objects existing only in the virtual space V.

An object is a unit element of an image to be displayed on the display 13. Objects include, for example, player characters, enemy characters, ally characters, and game characters such as neutral characters. An object is one of game media such as weapons, armor, magic, vehicles, and game items such as buildings, mobile phones, smartphones, and bulletin boards. For example, the game medium may be used in the game or may be used outside the game. The game medium is data that can be changed according to the game. That is, the game medium is electronic data used in the game, and is data that can be acquired, owned, used, managed, exchanged, synthesized, strengthened, sold, discarded, or donated by the user in the game. For example, the game medium includes image data and moving image data representing cards, virtual currencies, tickets, characters, avatars, and the like. In addition, the game medium is level information, status information, parameter information (physical strength value, attack power, magical power, etc.), ability information (skills, abilities, spells, etc.) that are embodied in cards, virtual currencies, tickets, characters, avatars, etc. , Job, etc.) contains arbitrary media data. Objects also include indexes such as text, icons, and maps.

The object information 22B includes the identifier of the object, the type of the object, and the position of the object in the virtual space V. The position of the object is the position of the object in the virtual space V. The position of the object and the like are updated by the control unit 21 as the game progresses.

The game sound information 22C is an example of virtual space information, and is information for causing the HMD 10 to output a game sound. The progress management unit 21A extracts information for outputting the game sound to be recognized by the player P from the game sound information 22C, and transmits the extracted information to the HMD 10.

The progress management unit 21A manages the progress of the game. The scene determination unit 21B determines whether or not the current scene is an input scene in which input information as operation information can be input while the game is progressing by the progress management unit 21A. For example, the scene determination unit 21B determines whether or not the scene can be input by a push button as input information. In addition, the scene determination unit 21B determines whether or not the scene is capable of voice input information as operation information. Further, the scene determination unit 21B determines whether or not the scene can input the locus of the movement (gesture) of the hand holding the operation device 30 of the player P as operation information.

〔Voice input〕
The voice recognition unit 21C recognizes the voice input information input through the microphone 15. The voice recognition unit 21C generates a frequency spectrum of the voice input period based on the recognized voice input information. The frequency spectrum can be calculated by Fourier transforming the voice spectrum with the horizontal axis as the time axis and the vertical axis as the intensity in the voice input period. The voice recognition unit 21C generates, for example, a frequency spectrum in which the horizontal axis is the frequency and the vertical axis is the intensity in the voice input period. In addition, the voice recognition unit 21C generates text data corresponding to the voice signal based on the voice recognition result and the text conversion dictionary.

The first determination unit 21D compares the voice-input voice input information with the voice reference information, and calculates the degree of coincidence between the voice input information and the voice reference information. The voice reference information is a reference spectrum for determination held by the first determination unit 21D in a storage unit such as a memory. The voice reference information is an example of the above-mentioned model behavior data.

As shown in FIG. 11, for example, when there is an input scene in which the spell of "Abracadabra" is input as voice input information, the first determination unit 21D uses the reference frequency spectrum 51 of the spell of "Abracadabra" as voice reference information. It is held in advance. The first determination unit 21D compares the frequency spectrum 52 of the spell of "Abracadabra" voice-input as voice input information with the reference frequency spectrum 51 of the voice reference information, and calculates the degree of agreement. The frequency spectrum 52 of the voice input information and the reference frequency spectrum 51 of the voice reference information have a higher degree of coincidence as the overlap of the spectra increases, and a lower degree of coincidence as the overlap of the spectra decreases. The "matching degree" referred to here has the same meaning as the "accuracy rate" described above.

The voice input information may be a voice spectrum in which the horizontal axis is the time axis and the vertical axis is the intensity. In this case, the first determination unit 21D holds the voice spectrum as reference information corresponding to the voice spectrum.

Further, as shown in FIG. 12, the first determination unit 21D recognizes the voice input information by voice, converts it into text data, and text data “Abra Cadabra” converted from the voice input information and text as voice reference information. Compare with the data "Abra Kadabra" and calculate the degree of agreement. For example, if the text data as the voice input information and the text data as the voice reference information are both "Abraco doubler", the degree of matching is 100%, and if the voice input information is "Abraco doubler" with one character difference. , The degree of agreement is 86%.

The calculation of the degree of agreement in the text data may be weighted according to the order of the characters in the voice reference information. For example, if the first letter of a word is different, the degree of matching should be lowered even if the same one letter is different. The difference in sound at the beginning of a word tends to sound different as a whole spell in terms of name, but it often sounds the same in terms of name as a whole even if the sound of the letters located in the middle or at the end of the word is different. .. For example, even if the same one letter is different, "Obraka dabra" has a different first letter, so the degree of matching is lower than that of "Abraco dabra" with different intermediate letters.

When converting the voice input information into text data, if the voice input information includes a silent section, a blank (space) may be added at a position corresponding to the silent section. In addition, when English is used instead of Japanese, a grammar dictionary or the like may be referred to and a blank (space) may be added between words. Then, the degree of agreement in the text data may be calculated including a blank (space).

Then, when the degree of agreement is equal to or greater than a predetermined value, the character object is made to perform an expression according to the second action data prepared in advance instead of the action (utterance) according to the voice input information input through the microphone 15.

For example, in the voice reference information "Abda Kadabra", as expressions according to the second action data, the action of "drawing a magic circle with a wand held in the right hand" and the visual sense of "magical energy being emitted toward an enemy" and Auditory effects and the like are stored in association with each other, and a moving image including a character object that expresses these expressions can be generated.

Further, as another embodiment of the present invention, in addition to the above configuration, the first action control unit 21E may control the action effect according to the degree of coincidence between the voice input information and the voice reference information. For example, as the first action control method, the first action control unit 21E controls that the higher the degree of coincidence, the higher the effect of the action, and the lower the degree of coincidence, the lower the effect of the action. Further, the first action control unit 21E holds a threshold value for the degree of coincidence, and when the degree of coincidence exceeds the threshold value, the effect of the action is made higher than usual. A high degree of agreement can be associated with an accurate or effective attack. The "matching degree" referred to here has the same meaning as the "accuracy rate" described above.

On the contrary, as the second action control method, the first action control unit 21E controls that the higher the degree of matching is, the lower the effect of the action is, and the lower the degree of matching is, the higher the action effect is. You may. Further, the first action control unit 21E holds a threshold value for the degree of coincidence, and when the degree of coincidence exceeds the threshold value, the effect of the action is made lower than usual. The degree of agreement also has a high correlation with the degree of skill of the player P, and when the degree of agreement is high, the degree of skill tends to be high. Therefore, the higher the degree of match, the lower the effect of the action and make the game difficult, and the lower the degree of match, the higher the effect of the action. As a result, the entertainment of the game can be enhanced.

Further, when the character level of the user exceeds the threshold level, the first action control method may be switched to the second action control method. As a result, for a user who continuously plays the game, when a beginner who has just started the game, the first action control method is to facilitate the game and to control the second action when the threshold value exceeds a predetermined level. By using the method, the game can be made difficult and even an expert can enjoy the game in the same way.

Further, the degree of agreement may be set with a threshold value for expressing the action effect. In this case, the first action control unit 21E exerts the action effect only when the degree of coincidence exceeds the threshold value. Then, the first action control unit 21E changes the action effect according to the degree of agreement when the threshold value is exceeded. Then, when the degree of agreement is less than the threshold value, the spell or attack is judged to be a failure (the action effect does not appear).

Next, a case where the operation information is input by voice will be described with reference to FIG. In step S1, the scene determination unit 21B determines whether the current scene is an input scene in which audio input information can be input. For example, in the case of a shooting game or a fighting game, it is determined whether the battle scene is such that various operation information of attack or defense is input. Also, in the case of a role-playing game, it is determined whether or not the scene is for inputting a spell, for example. In the case of an input scene capable of voice input, the scene determination unit 21B displays, for example, a voice input capable display in the virtual space.

When the player P inputs voice to be the operation information into the microphone 15, the voice recognition unit 21C converts the voice into electroacoustic to generate voice input information in step S2. Then, the voice recognition unit 21C generates a frequency spectrum of the voice input period.

In step S3, the first determination unit 21D compares the frequency spectrum 52 as the voice input information with the reference frequency spectrum 51 of the voice reference information, and in step S4, the frequency spectrum 52 of the voice input information and the reference frequency of the voice reference information. The degree of agreement with the spectrum 51 is calculated. In addition, the first determination unit 21D recognizes the voice information as the voice input information by voice, converts it into text data, and calculates the degree of matching in the text data.

Then, in step S5, when the above-mentioned degree of agreement satisfies the condition, the character object is made to perform the expression according to the second action data prepared in advance, and the moving image data is generated.

Further, the first action control unit 21E may control the action effect according to the degree of coincidence between the voice information and the voice reference information (not shown). For example, the first action control unit 21E controls that the higher the degree of coincidence, the higher the effect of the action, and the lower the degree of coincidence, the lower the effect of the action. Also, when the degree of coincidence exceeds the threshold value, the effect of the action is made higher than usual. Further, conversely, the first action control unit 21E controls that the higher the degree of coincidence, the lower the effect of the action, and the lower the degree of coincidence, the higher the effect of the action. Also, when the degree of coincidence exceeds the threshold value, the effect of the action is made lower than usual.

Further, the achievement rate can be determined based on the reference frequency spectrum 51 or the text data as well as the degree of agreement.

[Operation trajectory input]
As shown in FIG. 14, when the player P performs a specific gesture operation such as drawing a picture or writing a character by using an operation device 30 such as a pen or a stick type controller, the drawing unit 21F is in the space. The locus of the operating device 30 is detected. In FIG. 14, the operating device 30 such as a pen or stick type controller is moved in a spiral shape.

The locus of the operating device 30 can be detected by a sensor or a camera.

Also, when drawing a picture or character without displaying the drawing reference object described later, the plane to be input (which may be 3D because it actually moves) is not shown, unlike the case of tracing the guide. It is preferable to perform the adjustment process.

As an example, in such an adjustment process, the 3D coordinates which are the loci of the operating device 30 can be mapped on an appropriate 2D surface, and then a picture or a character drawn by the player P can be specified by using a known recognition technique. ..

Then, as shown in FIG. 15, the drawing unit 21F draws the locus object 61 corresponding to the locus in the three-dimensional virtual space V displayed by the HMD 10.

Further, as shown in FIG. 16, the drawing unit 21F displays the drawing reference object 63 as drawing reference information as a model when drawing the locus object 61 in the three-dimensional virtual space V. The drawing reference information is stored in a storage unit such as a memory. The drawing reference information is an example of the above-mentioned model behavior data.

The locus object 61 and the drawing reference object 63 here are displayed as if they are floating in the three-dimensional virtual space V. In this case, the operation device 30 is moved so as to trace the drawing reference object 63. As a result, the locus objects 61 are sequentially displayed in the three-dimensional virtual space V so as to trace the drawing reference object 63.

The drawing reference object 63 is not a spiral object, but a symbol object such as a magic circle composed of patterns and characters (for example, a star such as a pentagram or a hexagram), which is sometimes used to use magic. It may be a type polygon). The magic circle or the like can be drawn on the floor or in the air where the player P stands in the three-dimensional virtual space V. It may also be a three-dimensional object such as a cake. Further, the drawing reference object 63 may be like a frame of a coloring book, and the drawing reference object 63 may be colored so as not to protrude from the frame.

The second determination unit 21G compares the locus object 61 with the drawing reference object 63, and calculates the degree of coincidence between the locus object 61 and the drawing reference object 63. The drawing reference object 63 serves as a correct answer (model) when inputting drawing input information preset according to a story or the like, and is an abstract figure, a realistic figure, a spell, an attack name, or the like. It may be a character indicating a defense name or the like. Further, the locus of the operation device 30 when the player P makes a gesture may be used. For example, in the drawing reference object 63, when the player P swings the sword from top to bottom, the locus of the operating device 30 has a straight line or an arc line shape up and down, and when the player P swings a baseball bat, It will have an almost horizontal circular shape.

For example, in the second determination unit 21G, the larger the overlap between the locus object 61 and the drawing reference object 63, the higher the degree of coincidence, and the less the overlap, the lower the degree of coincidence. As an example, in the overlap between the locus object 61 and the drawing reference object 63, match / mismatch is detected in pixel units, and the ratio of matching pixels is calculated as the degree of match. Alternatively, a match / mismatch is detected for each pixel block obtained by dividing the image, and the ratio of matching pixels is calculated as the degree of match.

Then, when the degree of agreement is equal to or higher than a predetermined value, the character object is made to perform an expression according to the second action data prepared in advance instead of the action according to the miracle input through the operation device 30.

Further, as another embodiment of the present invention, in addition to the above configuration, the second action control unit 21H may control the action effect according to the degree of coincidence between the locus object 61 and the drawing reference object 63. .. The second action control unit 21H calculates the degree of agreement as a result of comparing the locus object 61 and the drawing reference object 63. Then, the second action control unit 21H controls the action effect according to the degree of coincidence. For example, as the first action control method, the second action control unit 21H controls that the higher the degree of coincidence, the higher the effect of the action, and the lower the degree of coincidence, the lower the effect of the action. Further, the second action control unit 21H holds a threshold value for the degree of coincidence, and enhances the effect of the action when the degree of coincidence exceeds the threshold value. A high degree of agreement can be associated with an accurate or effective attack.

On the contrary, as the second action control method, the second action control unit 21H controls that the higher the degree of coincidence, the lower the effect of the action, and the lower the degree of match, the higher the action effect. You may. Since the degree of matching is highly correlated with the skill level of the player P, the higher the degree of matching, the lower the effect of the action, and the lower the degree of matching, the higher the action effect, thereby enhancing the entertainment of the game. Because it can be done.

Further, when the character level of the user exceeds the threshold level, the first action control method may be switched to the second action control method. As a result, for a user who continuously plays the game, when a beginner who has just started the game, the first action control method is to facilitate the game and to control the second action when the threshold value exceeds a predetermined level. By using the method, the game can be made difficult and even an expert can enjoy the game in the same way.

Next, a case where input information is input by an operating device 30 such as a pen or a stick type controller will be described with reference to FIG. In step S11, the scene determination unit 21B determines whether the current scene is an input scene in which drawing input information can be input. Here, the scene determination unit 21B determines whether or not the scene can be input with drawing input information by the operation device 30 such as a pen or a stick type controller. For example, in the case of a shooting game or a fighting game, it is determined whether or not, for example, an attack or defense operation locus (movement locus of the operation device 30) is a battle scene in which operation information is input. Further, in the case of a role-playing game, for example, it is determined whether or not the motion locus (movement locus of the operation device 30) when casting a spell is input as drawing input information. In the case of an input scene in which the motion locus can be input, the scene determination unit 21B displays that the motion locus can be input in the virtual space.

In step S12, the tracking information detection unit 32 of the operating device 30 recognizes and calculates the position and orientation of the operating device 30. The drawing unit 21F draws the position on the time axis of the operating device 30 in the virtual space V as an operation locus. In step S13, the second determination unit 21G reads the drawing reference object 63. Here, the drawing reference object 63 may be displayed in the virtual space V so that the player P can trace it when drawing the locus object 61, or in any area in the virtual space V. It may or may not be displayed as a model. In step S14, the second determination unit 21G compares the locus object 61 with the drawing reference object 63, and in step S15, calculates the degree of coincidence between the locus object 61 and the drawing reference object 63.

Then, in step S16, when the above-mentioned degree of agreement satisfies the condition, the character object is made to perform the expression according to the second action data prepared in advance, and the moving image data is generated.

Further, the second action control unit 21H may control the action effect according to the degree of coincidence between the locus object 61 and the drawing reference object 63 (not shown). For example, the second action control unit 21H controls that the higher the degree of coincidence, the higher the effect of the action, and the lower the degree of coincidence, the lower the effect of the action. Also, when the degree of coincidence exceeds the threshold value, the effect of the action is made higher than usual. Further, conversely, the second action control unit 21H controls that the higher the degree of coincidence, the lower the effect of the action, and the lower the degree of coincidence, the higher the effect of the action. Also, when the degree of coincidence exceeds the threshold value, the effect of the action is made lower than usual.

Further, the drawing unit 21F may draw an additional object 73 as a locus object on the weapon object 72 as a game medium held by the character object 71 as a game medium with an operation device 30 such as a pen or a stick type controller. it can. FIG. 18 shows before the additional object 73 is drawn for the weapon (sword) object 72, and FIG. 19 shows after the additional object 73 is drawn for the weapon (sword) object 72. For example, the weapon object 72 is selected by the operation device 30 such as the stick type controller, and after the selection, the operation device 30 such as the stick type controller is regarded as a brush or a pen for the weapon object 72, and the drawing operation is performed. As a result, the additional object 73 is drawn as if the sword is burning with respect to the weapon object 72. The additional object 73 here is an effect object indicating the effect of the weapon. As a result, the additional object 73 is associated with the weapon object 72, and the weapon object 72 is displayed with the additional object 73 added. As a result, the weapon object 72 can be arranged according to the preference of the player P. Further, the weapon object 72 may have an increased attack power or the like by adding an additional object 73. Such a mode can be performed in a scene different from the process of FIG. 17, and the scene determination unit 21B determines the scene.

The additional object 73 here may function as a locus object 61. That is, the drawing unit 21F draws the additional object 73 by performing an operation of drawing the sword as if a flame is emitted from the operating device 30. In this case, the ideal (model) flame shape is the drawing reference object 63, and the second determination unit 21G compares the locus object 61, which is also the additional object 73, with the drawing reference object 63, and calculates the degree of agreement. To do. Then, the action effect is controlled by the second action control unit 21H. Along with this, the trajectory object 61 is displayed as an additional object 73 associated with the weapon object 72.

[Hardware configuration]
The hardware configuration of the game processing device 20 of the present embodiment will be described with reference to FIG.

FIG. 20 is a diagram showing an example of the game processing device 20. The game processing device 20 is a desktop personal computer, a notebook personal computer, a personal digital assistant (PDA), a server, a game-dedicated machine for home or business use, a computer of another type, or the like. The connection relationship between the configuration shown in FIG. 20 and each configuration is an example.

The game processing device 20 has a processor 201 and a memory 202. The game processing device 20 also includes a north bridge 203, a south bridge 204, and a storage device 205. The processor 201, the memory 202, and the bridges 203 and 204 are connected to each other by using various buses 206. The processor 201 controls the progress of the game and outputs an image to the HMD 10 or the like connected to the north bridge 203. The game processing device 20 may have a plurality of processors, or may have a plurality of memories.

The memory 202 is connected to the north bridge 203 and outputs information regarding game processing to the processor 201. For example, the memory 202 is a volatile storage device or a non-volatile storage device. Further, the memory 202 is a storage medium such as a magnetic disk or an optical disk.

The storage device 205 is connected to the south bridge 204 and outputs information regarding game processing to the processor 201. For example, the storage device 205 is a hard disk device, an optical disk device, a flash memory, or other storage device.

The north bridge 203 is connected to the HMD 10. South bridge 204 is connected to expansion port 207. Expansion port 207 may include various communication ports (eg, USB, Bluetooth®, Ethernet®, wireless Ethernet) and is connected to input / output devices such as keyboards, mice, and network devices. You may.

In the above configuration, the processor 201, the memory 202, and the north bridge 203 correspond to the control unit 21, and the memory 202 and the storage device 205 correspond to the storage unit 22.

As described above, according to the present embodiment, the following effects can be obtained.
(1) Since voice input information as operation information can be input by voice through the microphone 15, it is possible to make it difficult for the player P to lose the immersive feeling when playing a VR game. Further, since the action effect is changed according to the degree of matching between the voice-recognized voice input information and the voice reference information, the entertainment of the game can be enhanced.

(2) The higher the degree of agreement, the higher the action effect. In addition, the action effect can be enhanced when the degree of agreement exceeds the threshold value. As a result, the entertainment of the game can be enhanced.

(3) The higher the degree of matching, the lower the action effect. In addition, the action effect can be lowered when the degree of agreement exceeds the threshold value. As a result, the game can be made difficult for the player P who has a high skill level.

(4) Since the action effect is changed according to the degree of coincidence between the locus object 61 and the drawing reference object 63, it is possible to make it difficult for the player P to lose the immersive feeling when playing the VR game. In addition, the entertainment of the game can be enhanced.

(5) By displaying the drawing reference object 63, it is possible to easily increase the degree of agreement of the locus object 61 with respect to the drawing reference object 63.

(6) By displaying the additional object 73 as the trajectory object in association with the weapon object 72, the game medium such as the weapon object 72 can be arranged according to the preference of the player P.

Each of the above embodiments can be modified and implemented as follows.
The action effect may be increased or decreased by referring to both the degree of matching in the case of voice input and the degree of matching with the locus object. For example, when drawing a magic circle while casting a spell. In such a case, for example, the average value of the degree of coincidence with the locus object in the case of voice input is calculated. Then, the action effect can be changed according to the average value of the degree of matching. Specifically, the higher the average value of the degree of agreement, the higher the action effect can be. In addition, the action effect can be enhanced when the average value of the degree of agreement exceeds the threshold value. On the contrary, the higher the average value of the degree of agreement, the lower the action effect. In addition, the action effect can be lowered when the average value of the degree of agreement exceeds the threshold value.

When both the degree of agreement in the case of voice input and the degree of agreement with the locus object exceed the threshold value, an action effect such as mission clear may be generated. In addition, a judgment such as mission clear may be made based on the degree of matching in the case of voice input, and the acquisition points may be changed according to the degree of matching of the trajectory input (the higher the degree of matching, the higher or lower the acquisition point, etc.). .. In addition, a judgment such as mission clear may be made according to the degree of matching in the case of trajectory input, and the acquisition points may be changed according to the degree of matching of voice input (the higher the degree of matching, the higher or lower the acquisition point, etc.). ..

The object associated with the additional object 73 is not limited to the weapon object 72. For example, the accompanying object associated with the character object 71 may be associated with the additional object 73. The companion object may be in contact with or away from the main character object 71, but is a character that is always accompanied, and in some situations, it can attack, defend, or substitute with the main character object 71. It is a character.

The additional object 73 may be a pattern such as a pattern of an item object such as a character object 71 or a weapon object 72, instead of an object showing the effect of the weapon object 72.

The locus object 61 input during the game is not limited to the spiral figure as shown in FIGS. 14 to 16. As an example, the locus object 61 may be a character indicating a command (for example, a character such as "kick, punch, guard" (katakana, kanji, alphabetic character, number, symbol, etc.)). It may also be a picture or figure of a cake, fish, ship, airplane, or the like.

When drawing the locus object 61, the drawing reference object 63 as a model may or may not be displayed when the locus object 61 is drawn. For example, when the drawing reference object 63 is displayed, it may be set as one of the beginner or intermediate mode settings because drawing becomes easy, and when the drawing reference object 63 is not displayed, the advanced mode setting may be set. It may be one. When the drawing reference object 63 is not displayed, the player P draws the locus object 61 by relying on the memory. For example, when the drawing reference object 63 is a well-known subject such as a sunflower, the player P draws the sunflower locus object 61. Further, when the drawing reference object 63 is displayed in the three-dimensional virtual space V and the locus object 61 is input so as to trace the drawing reference object 63, it can be one of the introductory mode settings.

Further, the presence / absence of display and the display method of the drawing reference object 63 may change depending on the scene. For example, since the player P is also unfamiliar with the operation until the beginning or the middle of the game, the drawing reference object 63 may be displayed, and the drawing reference object 63 may not be displayed at the end of the game.

The process of increasing the action effect when the degree of agreement is high may be set to the beginner or intermediate mode, and the process of lowering the action effect may be set to the advanced mode.
The threshold value set when changing the action effect may be one or a plurality. As the number of thresholds increases, the action effect can be changed in more steps. If no threshold is set, the action effect will be gradually changed according to the degree of agreement.

The types of games, voice input information, and scenes for drawing trajectory objects are not limited to the above examples.
[For voice input]
In the case of a romance simulation game, a specific line may be input as voice input information, and the higher the degree of coincidence with the frequency spectrum as voice reference information, the higher the favorability for the opponent player P may be.

When imitating a celebrity or celebrity in a karaoke game, the frequency spectrum of a specific celebrity or celebrity is used as voice reference information, and the degree of matching with the frequency spectrum of the voice input information of player P is calculated, and the degree of matching is high. It may be determined that the imitations are similar, and a high score may be obtained.

[For trajectory objects]
For example, in the case of a game in which an orchestra conducts, the degree of coincidence between the trajectory object 61 indicating the trajectory of the baton as the operating device 30 and the drawing reference object 63 indicating the trajectory of the correct baton may be calculated. .. In this case, the higher the degree of agreement, the more complete the music or symphony is reproduced, and the lower the degree of agreement, the less complete the music or symphony is reproduced.

In the case of a fishing game, the degree of coincidence between the locus object 61 indicating the locus of the rod tip of the fishing rod as the operating device 30 and the drawing reference object 63 indicating the locus of the correct rod tip may be calculated. In this case, the higher the degree of matching, the higher the probability of catching a fish that is difficult to catch, and the lower the degree of matching, the higher the probability that the line or Harris will break.

The HMD 10 includes, for example, a hat-shaped, non-transparent head-mounted display. Further, a projection device is attached to the upper part or the front part of the spectacles, and the spectacles type may be projected onto the transparent plate portion, or a person who attaches a smart device such as a smartphone to the spectacles type adapter. May be good. Further, the projection device may be attached to the upper part or the front part of the spectacles, and may be a hat type projected on the transparent plate portion. Further, the display may be non-transparent so that the outside state cannot be seen, or the display device may be made of a half mirror and may be optical transmissive so that the outside state can be seen. Further, the projection method may be a virtual image projection method in which a virtual image is formed by using a half mirror or the like, or a retinal projection method in which an image is directly formed on the retina using the crystalline lens of the eye.

The stereoscopic image visually recognized by the HMD 10 may be an anaglyph method in which images taken from different angles on the left and right are superimposed and reproduced with red and blue lights, respectively, and viewed with glasses having red and blue color filters on the left and right. .. Further, a polarized spectacles system may be used in which linearly polarized light orthogonal to the left and right images is applied, superimposed and projected, and the images are separated by spectacles equipped with a polarizing filter. Further, a liquid crystal shutter spectacle system may be used in which images taken from different angles on the left and right are alternately reproduced and viewed with spectacles provided with a liquid crystal shutter in which the left and right fields of view are alternately shielded.

In the above embodiment, the case where the present invention is applied to a VR game system has been mainly described, but it can also be applied to a moving image distribution system. In this case, the motion of the distribution user (actor) can be acquired by using a technique such as motion capture using a studio device and / or facial capture using a smartphone function. At this time, as an example of the moving image to be replaced, in addition to the expression according to the second action data described above, a moving image such as making the facial expression of the character object a special facial expression or the movement of the character object changing clothes can be mentioned. Further, when a plurality of character objects co-star in the moving image, the above-mentioned model behavior data may be prepared for a plurality of persons, and the moving images to be replaced may be different according to the number of users satisfying the conditions.

Further, in order to function as the server device or terminal device according to the above-described embodiment, an information processing device such as a computer or a mobile phone can be preferably used. Such an information processing device stores a program describing processing contents that realize each function of the server device or the terminal device according to the embodiment in the storage unit of the information processing device, and the CPU of the information processing device stores the program. This can be achieved by reading and executing.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

Further, the method described in the embodiment includes, for example, a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), an optical disk (CD-ROM, DVD, MO, etc.), as a program that can be executed by a computer (computer). It can be stored in a recording medium such as a semiconductor memory (ROM, RAM, flash memory, etc.), or transmitted and distributed by a communication medium. The program stored on the medium side also includes a setting program for configuring the software means (including not only the execution program but also the table and the data structure) to be executed by the computer in the computer. A computer that realizes this device reads a program recorded on a recording medium, constructs software means by a setting program in some cases, and executes the above-mentioned processing by controlling the operation by the software means. The recording medium referred to in the present specification is not limited to distribution, and includes a storage medium such as a magnetic disk or a semiconductor memory provided in a device connected inside a computer or via a network. The storage unit may function as, for example, a main storage device, an auxiliary storage device, or a cache memory.

10 ... HMD, 11 ... HMD control unit, 12 ... sensor, 13 ... display, 14 ... speaker, 15 ... microphone, 20 ... game processing device, 21 ... control unit, 21A ... progress management unit, 21B ... scene judgment unit, 21C ... Sound recognition unit, 21D ... 1st determination unit, 21E ... 1st action control unit, 21F ... Drawing unit, 21G ... 2nd determination unit, 21H ... 2nd action control unit, 22 ... Storage unit, 22A ... Image information, 22B ... Object information, 22C ... Game sound information, 30 ... Operation device, 31 ... Information processing unit, 32 ... Tracking information detection unit, 33 ... Operation unit, 36 ... Tracking auxiliary device, 51 ... Reference frequency spectrum, 52 ... Frequency spectrum , 61 ... Trajectory object, 63 ... Drawing reference object, 71 ... Character object, 72 ... Weapon object, 73 ... Additional object, 201 ... Processor, 202 ... Memory, 203 ... North bridge, 203 ... Bridge, 204 ... South bridge, 204 … Bridge, 205… storage, 206… bus, 207… expansion port

Claims (12)

An information processing system that generates moving images including animations of character objects generated based on user behavior, including one or more computer processors.
The one or more computer processors may execute computer-readable instructions.
A reception function that accepts input of behavior data based on the user's behavior,
A processing function for generating video data including a character object that performs an action according to the action data is realized.
When the reception function receives the first action data based on the specific action of the user, the processing function prepares the character object in advance in place of the action according to the first action data. An information processing system that generates video data that is expressed according to the behavior data of. The first aspect of the present invention is characterized in that the processing function displays a state display indicating to the user that moving image data in which the character object is expressed according to the second action data is generated. The information processing system described. When the reception function receives the third action data based on the specified action of the user, the processing function follows the first action data received after the reception function receives the third action data. The information processing system according to claim 1 or 2, wherein the character object generates moving image data in which an expression according to the second action data is performed instead of the operation. The information processing system according to any one of claims 1 to 3, wherein the processing function displays an input guide for the first action data to the user. When the reception function receives the third action data based on the specified action of the user, the processing function displays the input guide of the first action data to the user and displays the input guide. Later, when the reception function receives the first action data, the moving image data in which the character object expresses according to the second action data prepared in advance instead of the action according to the first action data. The information processing system according to claim 1 or 2, wherein the data processing system is generated. The processing function
When the action data started to be accepted by the reception function satisfies the first condition, the action data is prepared in advance as the first action data in the character object in place of the action according to the first action data. Generate video data that is expressed according to the second behavior data
The claim is characterized in that when the action data received by the reception function does not satisfy the first condition, the character object generates moving image data in which the action according to the action according to the first action data is performed. The information processing system according to any one of 1 to 5. The processing function
A claim characterized in that the achievement rate is determined by comparing the behavior data with the model behavior data, and when the achievement rate exceeds a predetermined value, it is determined that the behavior data satisfies the first condition. Item 6. The information processing system according to item 6. The processing function
The second action depends on the case where the first action data received by the reception function satisfies the second condition and the case where the first action data received by the reception function does not satisfy the second condition. The information processing system according to any one of claims 1 to 7, wherein the effects of expressions according to data are different. The processing function
The accuracy rate within a predetermined time is determined by comparing the behavior data with the model behavior data, and when the accuracy rate exceeds a predetermined value, the first behavior data satisfies the second condition. The information processing system according to claim 8, wherein a determination is made. The information processing system according to claim 9, wherein the processing function displays a state display indicating the accuracy rate to the user. An information processing method in an information processing system that generates a moving image including an animation of a character object generated based on a user's behavior, and is applied to one or more computer processors.
A reception step that accepts input of behavior data based on the user's behavior,
A processing step of generating moving image data including a character object that performs an action according to the action data is executed.
In the processing step, when the first action data based on the specific action of the user is received in the reception step, the character object is provided with a second action prepared in advance in place of the action according to the first action data. An information processing method that generates video data that is expressed according to the behavior data of. A computer program for generating moving images containing animations of character objects generated based on user behavior, which can be applied to one or more computer processors.
A reception function that accepts input of behavior data based on the user's behavior,
A processing function for generating video data including a character object that performs an action according to the action data is realized.
When the reception function receives the first action data based on the specific action of the user, the processing function prepares the character object in advance in place of the action according to the first action data. A computer program that generates video data that is expressed according to the behavior data of.

Images