TWI825111B - Game system, game system control method and game program - Google Patents

Abstract

The subject of the present invention is to apply the vocalization assessment of language learning to games to enhance entertainment, and to enrich the training of vocalization in the natural atmosphere and timing of real life. The solution includes an input interface (12) for inputting a voice signal, a voice recognition processing unit (144) that recognizes the player's voice and generates game-related control signals based on the voice recognition result, and event processing that advances the story according to the description. The scenario control unit (142) allows the game to proceed sequentially according to the scenario. The game execution processing unit (141) performs the scenario according to the input signal and control signal caused by the operation input. The execution evaluates the correctness of the pronunciation caused by the player. The game mode execution part (143a) of the specific game mode, and the story mode execution part (143b) of the story mode that executes the story mode related to the actions of the characters, respond to the player's problem with the voice recognition target of the question. The correctness of pronunciation is evaluated by points or level improvements.

Description

Game system, game system control method and game program

The present invention relates to a game system that recognizes input sounds, a control method of the game system, and a game program.

With the advancement of computer technology in recent years, electronic game consoles using computer graphics technology have gradually been widely used. In recent years, various game softwares have been developed, so such home electronic game devices are very popular in entertainment. In this type of electronic game console, by operating the buttons of the controller, the character is moved or the game screen is switched. In this type of game, a system has also been developed that recognizes vocalization, which is the most natural means of communication for humans, and attempts to control the game based on the recognition result (for example, prior art document 1).

On the other hand, systems that apply computer-based video games to language learning are also being developed. As such a system for linguistics, for example, a so-called automatic pronunciation assessment method has been developed that automatically evaluates the pronunciation of a language. This automatic assessment of pronunciation is a structure that automatically estimates and evaluates the degree of difference between a non-native speaker's phonemes, word pronunciations, or article pronunciation compared to a native speaker's pronunciation (for example, Patent Document 2).

[Prior technical literature] [Patent Document]

[Patent Document 1] International Publication WO98/02223

[Patent Document 2] Japanese Patent Application Publication No. 2006-84966

However, Guzhuo, which is purely an automatic assessment of the pronunciation of video game combinations, can easily become monotonous and difficult to continue learning, and it can also become a pronunciation that is used as a command operation in the game, so there is a risk in the natural atmosphere and timing of real life. Possibility of insufficient vocal training.

The present invention was devised to solve a related problem, and its purpose is to provide a game system and game that utilizes voice assessment for language learning in games to enhance entertainment and at the same time enrich the training of voice production in the natural atmosphere and timing of real life. System control methods and game programs.

In order to solve the above-mentioned problems, the present invention is a game system that advances the game in response to operation input and voice input by the player. It is characterized by having: a voice input unit for inputting a voice signal; and a voice recognition unit. It recognizes the player's voice by comparing the input voice signal with predetermined voice recognition dictionary data, and generates game-related control signals based on the voice recognition results. The script control unit makes the story progress according to the description. The event processing script enables the game to proceed in sequence; the game control part makes the script proceed based on the input signal caused by the operation input and the control signal generated by the sound recognition part; the game mode execution part serves as an event handler 1. Execute the game mode that follows the control caused by the script control unit and evaluates the correctness of the pronunciation caused by the player based on the control signal generated by the voice recognition unit; and the story mode execution unit serves as one of the event processes , executes the story mode in which the story related to the actions of the characters proceeds according to the control caused by the script control unit; the game mode execution unit detects the accuracy of the pronunciation caused by the player in relation to the voice recognition target problem that is presented. , the evaluation is carried out based on points or level increases corresponding to the detected accuracy; the story mode execution unit has the function of executing actions related to the character in response to the situation where the sound signal evaluated by the game mode execution unit is input by the player. The function of response event processing.

Furthermore, the present invention is a game system that advances the game in response to operation input and voice input by the player, and is characterized by having: a voice input unit for inputting a voice signal; and a voice recognition unit for Compares the input sound signal with the predetermined dictionary data for voice recognition to identify the player's voice, and generates game-related control signals based on the voice recognition results; the script control unit processes events according to the description to progress the story. The script causes the game to proceed in sequence; the game control unit causes the aforementioned script to proceed based on the input signal caused by the aforementioned operation input and the aforementioned control signal generated by the aforementioned voice recognition unit; and the game mode execution unit acts as the aforementioned event One of the processes is to execute a game mode in which the control by the aforementioned script control unit is followed and the correctness of the pronunciation by the aforementioned player is evaluated based on the control signal generated by the voice recognition unit; the story mode execution unit is the aforementioned One of the event processing is to execute the story mode in which the story related to the actions of the characters is carried out in accordance with the control caused by the aforementioned script control department; and the post-recording data generation department is to obtain the information obtained from the story progressed by the aforementioned story mode execution department. The regenerated story mode animation is used to obtain the sound input synchronously with the animation for post-recording animation to generate post-recording data; the aforementioned game mode execution unit is configured to detect the sound input relative to or synchronized with the aforementioned post-recording animation. , or the function of performing the aforementioned evaluation based on the points or level increases corresponding to the detected accuracy of the player's pronunciation of the sound recognition target questions posed using the aforementioned post-recording data.

The present invention is a control method for a game system that advances the game in response to operational input and sound input from players. It is characterized by including: (1) In the voice recognition step, the voice input unit accepts the input of a voice signal. The voice recognition unit recognizes the player's voice by comparing the input voice signal with predetermined voice recognition dictionary data, and based on the voice recognition result, Generate game-related control signals; (2) The game control step is based on the input signal caused by the operation input and the control signal generated by the voice recognition unit. The script control unit follows the script describing the event processing that makes the story progress, so that the game proceeds in sequence; (3) In the game mode execution step, the game mode execution unit, as one of the event processes, executes control according to the script control unit, and evaluates the accuracy of the pronunciation caused by the player based on the control signal generated by the voice recognition unit. sexual play patterns; and (4) The story mode execution step is a story mode in which the story mode execution department, as one of the event processes, executes the story in accordance with the control caused by the script control department to progress the story related to the character's actions; (5) In the game mode execution step, the correctness of the player's pronunciation relative to the sound recognition target problem is detected, and the aforementioned evaluation is performed based on points or level increases corresponding to the detected correctness. ; (6) In the story mode, response event processing related to the character's actions is executed in response to the situation where the sound signal evaluated by the game mode execution unit is input by the player.

Furthermore, the present invention is a game system control method that advances the game in response to the player's operation input and voice input, and is characterized by including: (1) In the voice recognition step, the voice input unit accepts the input of a voice signal, and the voice recognition unit recognizes the player's voice by comparing the input voice signal with predetermined dictionary data for voice recognition, and based on the voice recognition result , generate game-related control signals; (2) The game control step is based on the input signal caused by the operation input and the control signal generated by the voice recognition unit. The script control unit follows the script describing the event processing that makes the story progress, so that the game proceeds in sequence; (3) In the game mode execution step, the game mode execution unit, as one of the event processes, executes control according to the script control unit, and evaluates the accuracy of the pronunciation caused by the player based on the control signal generated by the voice recognition unit. sexual play patterns; (4) The story mode execution step is a story mode in which the story mode execution department, as one of the event processes, executes the control caused by the script control department to progress the story related to the character's actions; and (5) In the post-recording data generation step, the post-recording data generation unit obtains the sound input in synchronization with the animation reproduction of the story mode animation reproduced for the story performed by the story mode execution unit, and generates the post-recording data generation step. Later recording materials; (6) In the game mode execution step, the correctness of the pronunciation caused by the player relative to the animation used for post-recording, or the sound input synchronously with it, or the sound recognition target problem using the post-recording data is detected, Evaluation is based on points or level increases corresponding to the correctness detected.

In the aforementioned invention, it is preferable that two or more languages are used in the story mode, and the correctness of the sound recognition target problem related to the two or more languages is evaluated in the game mode. Furthermore, in the aforementioned invention, in the story mode, in the event processing that occurs in response to the input of a voice signal by the player, it is preferable that the character performs actions in accordance with the points or level improvements obtained in the game mode. Furthermore, in the invention, it is preferable to use two or more languages in the story mode, and to pose sound recognition target questions related to two or more languages in the game mode.

Furthermore, in the aforementioned invention, in the story mode, in the event processing that occurs in response to the input of a voice signal by the player, it is preferable that the character performs actions in accordance with the points or level improvements obtained in the game mode.

Furthermore, the above-mentioned system and method of the present invention can be realized by executing the program of the present invention written in a predetermined language on a computer. That is to say, by installing the program of the present invention on the IC chips and memories of mobile terminal devices and other information processing terminals such as smartphones, wearable terminals, and mobile PCs, personal computers, and server computers, By using a physical device and executing it on a CPU, a system with the above functions can be constructed to implement the method of the present invention.

Furthermore, the program of the present invention can be distributed through a communication line, for example, or can be recorded on a computer-readable recording medium and transferred as a packaged application program that operates on a stand-alone computer. Specifically, as the recording medium, in addition to magnetic recording media such as flexible disks and cassettes, and optical disks such as CD-ROM and DVD-ROM, various recording media such as RAM cards can also be recorded. Then, depending on the computer-readable recording medium on which the program is recorded, a general-purpose computer or a special-purpose computer can be used to easily implement the above-mentioned system and method, and the program can be easily saved, transported, and installed.

As described above, according to these inventions, it is possible to enhance the entertainment by applying the vocalization assessment of language learning to games, and at the same time, enrich the training of vocalization in the natural atmosphere and timing of real life.

Hereinafter, embodiments of the game system and game program of the present invention will be described in detail with reference to the attached drawings. In addition, the present invention is not entirely limited to each of the above-described embodiments. In the implementation stage, the constituent elements can be changed and embodied without departing from the scope of the gist. In addition, various inventions can be formed by appropriate combinations of the plurality of constituent elements disclosed in the aforementioned embodiments. For example, some components may be deleted from the overall components shown in the embodiment.

(The overall structure of the game system) FIG. 1 is a conceptual diagram showing the overall structure of the game system according to this embodiment. As shown in FIG. 1, the game system of this embodiment is composed of a smartphone 1 (1a, 1b) used by players, that is, a plurality of players U (Ua, Ub), and a game server installed on the Internet 2. The general structure of device 3. In addition, in this embodiment, the smartphone 1 is demonstrated as an example of a mobile terminal device.

In this embodiment, the game server 3 is a server that processes games. It can be implemented by a single server device or a plurality of server device groups. Multiple function modules are virtually built on the CPU, and each function Modules work together to perform the process. In addition, the game server 3 can send and receive data through the Internet 2 through the communication function, and can prompt Web pages through the browser software through the Web server function.

The smartphone 1 is a portable information processing terminal device that utilizes wireless communication. Relay points such as the wireless base station 22 communicate wirelessly with the mobile phone, and can receive communication services such as calls and data communication while moving. As a communication method of the mobile phone, for example, in addition to the 3D (3rd Generation) method, the LTE (Long Term Evolution) method, the 4G method, the FDMA method, the TDMA method, the CDMA method, and the W-CDMA method, PHS (Personal Handyphone System) ) method, etc. Furthermore, in addition to the communication function, the Smartphone 1 series is equipped with various functions such as a digital camera function, an application software execution function, a position information acquisition function such as GPS (Global Positioning System), etc., and also includes a tablet PC. Waiting for mobile computers.

The location information acquisition function is a function that acquires and records location information indicating the location of one's own device. The location information acquisition function includes, for example, a method of detecting the location of one's own device based on signals from satellites like GPS, as shown in Figure 1 As shown, the method of detecting the location is based on the wireless base station 22 of the mobile phone and the intensity of radio waves from the Wi-Fi communication access point.

The smartphone 1 is equipped with a liquid crystal display as a display unit for displaying information, and is equipped with an operating device such as operation buttons for the user to perform input operations. The operating device includes an acquisition device that is superimposed on the liquid crystal display. A touch panel that is an input unit for operating signals caused by touch operations that specify coordinate positions on the liquid crystal display. Specifically, this touch panel is an input device that inputs operation signals by using pressure and static electricity detection caused by touch operations of the user's fingertips and stylus pens, etc., and is a liquid crystal display that displays graphics in a superimposed manner. , and is composed of a touch sensor that receives an operation signal corresponding to the coordinate position of a graphic displayed on the liquid crystal display.

(Internal structure of each device) Next, the internal structure of each device constituting the above-mentioned game system will be described. FIG. 2 is a block diagram showing the internal structure of the game server 3 of this embodiment, and FIG. 3 is a block diagram showing the internal structure of the smartphone 1 of this embodiment. Furthermore, the "module" used in the description refers to a functional unit that is composed of hardware such as a device or machine, or software having the function, or a combination thereof, and is used to achieve a predetermined operation.

(1)Game server First, the internal structure of the game server 3 will be described. The game server 3 is a server device configured on the Internet 2 and can send and receive data with each smart phone 1 through the Internet 2 .

The game server 3 is provided with a communication interface 31 for data communication through the Internet 2, an authentication unit 33 for authenticating the authority of users and user information, an access management unit 32 for managing access from each user terminal, and execution The game progress processing unit 36 processes the overall game and each user's game, the voicing learning management unit 37 manages the voicing learning of each user, and the game data delivery unit delivers game data to each user. 34, and various database groups 35 (35a~35c).

The database group 35 includes a voice dictionary database 35a that contains a learning game executed in a memory game and a voice dictionary file used for voice recognition executed in a story movie, a user database 35b that accumulates user-related information, and The game database 35c stores information related to the game processing as a whole and the game processing of each user, and accumulates scenario data and other data. Each of these databases may be a single database, or may be a relational database in which data are divided into plural databases and each data is associated with each other by setting a mutual relationship.

The sound dictionary database 35a is a memory device for sound dictionary files used in the learning game performed in the memory game and the sound recognition performed in the story film. In addition to the foreign language of the learning target (English in this case), the mother tongue (English in this case) is memorized. This is a dictionary for Japanese). Furthermore, in this embodiment, the two-way English and Japanese translations between English and Japanese are also asked as questions targeted at sound recognition, so the sound dictionary files are set into two categories: foreign language and native language. However, any data is omitted depending on the format of the questions. It is possible to install one audio dictionary file, and to support multiple languages, it is also possible to install three or more audio dictionary files.

The information accumulated in the user database 35b includes authentication information that associates an identifier (user ID, terminal ID) of a specific user or a mobile terminal device used by the user with a password, etc. It also includes information related to the user. The personal information of the user associated with the ID, the model of the terminal device, etc. In addition, the user database 35b also accumulates authentication histories (access histories) corresponding to each user or each user terminal, and learning levels related to the game progress of each user based on the association with the game database 35c. , game status, scores, usage history, etc., game-related settlement information, etc.

The information accumulated in the game database 35c includes, as game data, scenarios used for game progress, characteristics of characters and objects, information related to event processing, graphics information, and the like.

The authentication unit 33 is a module that establishes communication sessions with each smartphone 1 through the communication interface 31 and performs authentication processing corresponding to each communication session established. As this authentication process, authentication information is obtained from the smartphone 1 of the user who is the accessor, the user database 35b is referenced, the user, etc. are specified, and the authority is authenticated. The authentication results (user ID, authentication time, session ID, etc.) by the authentication unit 33 are sent to the game progress processing unit 36, and are accumulated as authentication history in the user database 35b through the access management unit 32.

The game processing unit 36 follows the script data and should launch the game mode and the story mode to enable various event processing to occur in order to proceed with the game module, execute the game program containing certain rules, logic, and algorithms, and respond to the positional relationships of characters and objects. (Proximity, contact, etc.) to cause event processing such as duel battles, mini-games, and video regeneration to occur. Furthermore, in this embodiment, the game progress processing unit 36 on the game server 3 side operates in cooperation with the game progress processing unit 141 on the smartphone 1 side, and performs part of the game progress processing on the game server 3 side, so as to The game progress processing unit 141 on the smartphone 1 side executes part of graphics processing and event processing. For example, on the game server 3 side, in addition to the progress of the scenario and the user's learning level, event processing that may occur is predicted based on the position of the user's character, the position of the position, etc., and the game server 3 side makes it possible The occurrence condition occurs and its condition is sent to the smartphone 1 side. The occurrence of actual event processing and the graphics processing used therefor are executed on the smartphone 1 side based on the occurrence condition received from the game server 3 .

The voicing learning management unit 37 is a module that poses voicing questions to the user in accordance with the progress of the game caused by the game progress processing unit 36, and updates the evaluation of the user in accordance with the accuracy of the utterance. The utterance question is a situation generated on the game server 3 side, a situation generated on the smartphone 1 side, or a situation generated by the cooperative operation of both the game server 3 and the smartphone 1, and is managed by the utterance learning management unit 37 The dictionary file for voice learning generated or accumulated on the game server side 3 is compared with the dictionary file generated or accumulated on the smartphone side 1, and all or part of the dictionary file is delivered as necessary. Go to the smartphone side and seek synchronization between the two.

The game data delivery unit 34 follows the control of the vocalization learning management unit 37 based on the user's learning level, and synchronizes the dictionary files for generating questions and voice recognition generated by the vocalization learning management unit 37. Modules delivered to each user through the communication interface 31.

(2)Smartphone 1 Next, the internal structure of the smartphone 1 will be described. As shown in FIG. 3 , the smart phone 1 has a communication interface 11 , an input interface 12 , an output interface 13 , an application execution unit 14 , and a memory 15 .

The communication interface 11 is a communication interface used for data communication, and has the function of non-contact communication by wireless, etc., and contact (wired) communication by means of cables, adapters, etc. The input interface 12 is a mouse, a keyboard, an operation button, a touch panel 12a, etc., and is used to input devices operated by the user. In addition, the output interface 13 is a device such as a display or a speaker that outputs images and sounds. In particular, the output interface 13 includes a display portion 13a such as a liquid crystal display, and this display portion overlaps the touch panel 12a that is the input interface.

The memory 15 is a storage device that stores OS (Operating System) and firmware, programs for various applications, other data, etc. In the memory 15, in addition to the user ID for identifying the user, information from the game server is stored. The game application data downloaded by the server 3 is stored, and the game data processed by the application execution unit 14 is accumulated. In particular, in this embodiment, the memory 15 stores scenario data obtained from the game server 3, video data used in the story mode, voice learning questions, dictionary files for recognition, and the like.

The application execution unit 14 is a module that executes applications such as general OS, game applications, browser software, etc., and is usually implemented by a CPU or the like. In the application execution unit 14, the game program of the present invention is used to virtually construct a game progress processing unit 141, a scenario control unit 142, an event control unit 143, a voice recognition processing unit 144, a display control unit 145, and a display data generation unit. 146, and the post-recording data control department 147.

The game progress processing unit 141 executes control according to the scenario control unit 142 as one of the event processes, and evaluates the accuracy of the pronunciation caused by the player U based on the control signal generated by the voice recognition processing unit 144. The game mode execution part of the mode. In this embodiment, the game progress processing unit 141 uses the same rules, logic, and algorithms as the game program executed in the game server 3 to sequentially develop the story mode and the game mode, so that various event processing occurs. The function of playing the game is synchronized with the game progress processing unit 36 on the game server 3 side through the scenario control unit 142, and the battle game is made in accordance with the user's learning level, the positional relationship of characters and objects (proximity, contact, etc.) and event processing such as video playback occurs.

In this embodiment, the game progress processing unit 141 operates in cooperation with the game progress processing unit 141 on the game server 3 side, and performs a part of the game progress processing on the game server 3 side, and performs game processing on the smartphone 1a side. The unit 141 performs part of graphics processing and event processing. For example, the game server 3 generates conditions for event occurrence, etc., and sends the conditions to the smartphone 1 side. The actual occurrence of event processing and the graphics processing used therefor are executed on the smartphone 1 side.

The game progress processing unit 141 includes a scenario control unit 142 and an event control unit 143. The event control unit 143 includes a game mode execution unit 143a, a story mode execution unit 143b, and a post-recording mode execution unit 143c.

The story mode execution unit 143b follows the control of the scenario control unit 142 and executes a module of the story mode related to the character's actions as one of the event processes. The game mode execution unit 143a detects a pronunciation problem (sound) related to the question. A module that evaluates the correctness of the pronunciation of the target question) based on the number of points or level improvements detected. In the story mode execution unit 143b, the reaction action of the character corresponding to the event processing that occurs in response to the input of the voice signal by the player U, and the display data of the event processing corresponding to the reaction action are generated by the reaction action processing unit 146a .

Furthermore, the post-recording mode execution unit 143c executes the acquisition of the post-recording animation from the story mode animation reproduced for the story performed by the story mode execution unit 143b, and inputs the sound in synchronization with the animation reproduction of the acquired post-recording animation, A module for post-recording mode that generates post-recording data. Specifically, when the post-recording data generation unit 144b executes the post-recording mode through the post-recording mode execution unit 143c, it records the input sound data D3 and analyzes its sound waveform, and processes it through the voice changing function as needed. By Lip synchronization processing is used to adjust the movements of the characters in the animation to match the sound waveform of the input sound data D3. Furthermore, the post-recording data generation unit 144b synthesizes the input sound data D3 to which the voice processing and lip-syncing processing has been performed in this manner, and the post-recording data D2 to generate the post-recording animation D5.

The scenario control unit 142 is a module that causes the scenario to proceed based on the input signal caused by the operation input and the control signal generated by the voice recognition processing unit 144. In this embodiment, it has a development in accordance with the description of the game mode and the story mode. The script data is used to synchronize the game processing on the smartphone side 1 with the game processing on the game server 3 side. Specifically, on the game server 3 side, based on the game progress level of the scenario data, the positions of other users' characters, the positions of objects, etc., the event processing that may occur is predicted, and the conditions for its occurrence are set on the game server 3 side occurs and sends its condition to the smartphone 1a side. The game progress processing unit 141 uses the scenario control unit 142 to receive the occurrence conditions sent from the server side, and based on the occurrence conditions received from the game server 3, the game progress processing unit 141 on the smartphone 1 side executes the actual event. The occurrence of processing and the graphics processing used for it.

The results of the event processing executed by the game progress processing unit 141 of the smartphone 1 (the progress of the story, the victory or defeat of battles and mini-games, scores, etc.) are notified to the game progress processing unit on the game server 3 side through the scenario control unit 142 141, reflected to subsequent games for processing.

The voice recognition processing unit 144 compares the input The module uses the sound signal input in page 12 and the predetermined dictionary data for voice recognition to identify the voice of the player U, and generates a game-related control signal based on the voice recognition result. In this embodiment, it has an evaluation unit 144a and the post-recording data generation unit 144b. The evaluation unit 144a is a module that evaluates the accuracy of the pronunciation caused by the player U based on the control signal generated by the voice recognition processing unit 144 in compliance with the control of the scenario control unit 142. The evaluation results by the evaluation unit are sent to the game progress processing unit 141. Furthermore, the post-recording data generation unit 144b is a module that generates post-recording data by acquiring the sound input in synchronization with the animation reproduction of the post-recording animation for the story reproduced by the story mode execution unit 143b.

The display data generating unit 146 is a module that generates display data to be displayed on the display unit 13a. Display data is data generated by combining image data, text data, animation data, sound and other data in addition to graphic data. The display data generating unit 146 is provided with a reaction action processing unit 146a, which is a module that generates an animation representing the movement of the user's character in accordance with the game progress caused by the game progress processing unit 141. The display control unit 145 of this embodiment includes a display mode switching unit 145a. The display mode switching unit 145a displays graphics according to the display mode.

The post-recording data control unit 147 is a module that manages and controls various data used in the post-recording mode. Specifically, the post-recorded data control unit 147 manages and controls the post-recorded animation D5. In the post-recording mode, in addition to reproducing the post-recorded animation D5 as a result of the execution of the post-recording mode, for example, the recall scene of the story, etc. Yu story animation The post-recorded animation D5 is also reproduced in the regeneration process, or the post-recorded animation D5 is converted into the game mode question-producing animation D41 and the question-proposing sound, and is reproduced in the game mode question-producing process.

(The game uses image processing actions)

The control method of the game system of the present invention can be implemented by operating the system described above. Figure 4 is a sequence diagram showing the actions of the system. In addition, the processing steps described below are only an example, and each processing may be changed as much as possible. In addition, regarding the processing steps described below, the steps can be omitted, replaced, and added appropriately according to the embodiment.

As shown in Figure 4, first, start the game application on the smartphone 1 and start learning the game (S201). At this time, the scenario data is referenced on the application, and the user's current learning level and game progress are read out, starting from the time after the last ending time. When the game is started, scenario data is referenced as the game progresses, and event occurrence conditions are monitored (S202), and the process is repeated until the event occurrence conditions are met (S203-N).

When the event occurrence conditions are met ("Y" in S203), refer to the script data to select whether the event processing that should occur is game mode, story mode, or post-recording mode (S204). These modes are judged based on the scenario data. In the story mode situation, the battle starts by contacting objects such as enemies and monsters on the map, or when reaching the story branching point specified in the scenario data, etc.

When selecting the game mode ("Game Mode" in S204), Switch to the battle scene screen as shown in Figure 6 (a) and (b). In this battle scene screen, in order to defeat the enemy character Ob1, use the prop I1 or prop I2 displayed on the graphical user interface G1 and execute attack. The attack is as shown in Figure (b). The user selects the props used for the attack. The choice can be made in accordance with the type and intensity of the user's level (physical strength, magic power, language ability, etc.) at that time. implement. At this time, according to the control by the scenario control unit 142, based on the control signal generated by the voice recognition processing unit 144, the accuracy of the pronunciation caused by the player U is evaluated, and the attack power corresponding to the evaluation level is executed. To explain in detail, the game mode execution unit 143a displays the window G3 of the words and idioms that should appear in the question as shown in Figure (c), and detects the correctness of the pronunciation of the player with respect to the pronunciation problem of the question. An attack of the type and strength corresponding to the correctness of the detection is performed, and the evaluation is made based on the points or level increase of the damage inflicted on the enemy by the attack.

On the other hand, in step S204, when the story mode is selected ("Story Mode" of S204), the story mode execution unit 143b executes the story mode (S205) in response to the input of the voice signal by the player. Event processing makes the character act (S206). To explain in detail, in the story mode of this embodiment, a so-called "play within a play" format is adopted. As shown in Figures 5(a) and (b), the theater St1 becomes the virtual space of the story mode. In the theater St1 On the stage St2, the story unfolds in the form of a drama with virtual characters Ca1~3. Furthermore, as the video broadcast in the theater St1, so-called VR (Virtual Reality: Virtual Reality) and AR (Augmented Reality: Augmented Reality) technologies can be used for three-dimensional display.

Players of the game can participate in the game as one of the audience members sitting in the auditorium St10 in the theater St1. The players use the microphone of the smartphone 1 to interact with the characters Ca1 to 3 displayed on the stage St2 of the display unit 13a. Wait for a chat.

For example, as shown in FIGS. 7(a) and (b) , the character Ca1 who becomes a girl on the stage St2 performs for the development of the story, and the player participates in the game as one of the audience members sitting in the auditorium St10. In this setting, regardless of the story of the game, when the player talks to the character Ca1 on the stage St2 through the microphone of the smartphone 1, an event will occur in a manner that responds to his voice.

For example, when a player speaks to a microphone, cheering, taunting, paying attention, etc., the character will take irregular actions in response to his or her voice. As this irregular movement, for example, in addition to the "camera line of sight" in the direction of the character toward the virtual camera that illuminates the character, the direction from the level in the virtual space toward the auditorium where the character corresponding to the player is sitting (sound from the character's perspective) In addition to the direction of occurrence), etc., you can cite when danger is approaching a character on the level, call the character's name, or issue a warning of "Behind! Behind!" etc.

Furthermore, as an opportunity for the player to talk, for example, a graphic mark representing the microphone is displayed on the screen, or as shown in Figure 7 (a), from the auditorium (corresponding to the side where the character of the player is sitting) , it is also possible to display a text string like "Watch Out!" which is an example of a conversation along with the occurrence. Furthermore, as another method of creating the timing, the chatting sounds caused by the characters of other spectators can be automatically generated so that the players can easily imitate the chatting. Furthermore, the microphone may be turned ON and OFF in synchronization with the timing of the conversation, or the event processing of the timing of the conversation may not occur while other characters are speaking.

At this time, in the play within the play in the ongoing game, for example, two or more languages are used, such as English and Japanese. In the above-mentioned game mode, the sound recognition objects related to the two or more languages, that is, English and Japanese pronunciation problem. For example, if a Japanese idiom is displayed as a character string and an English word corresponding to the character string is displayed, and conversely, an English character string is displayed and a Japanese word corresponding to the English word is produced, etc., a bidirectional translation problem is raised.

Then, the script is composed so that the words or idioms containing the pronunciation questions asked in the game mode are used in the conversations caused by the characters Ca1 to Ca3, etc. even in the drama within the story mode. When the player talks to the character on stage St2, he or she may speak with the pronunciation problem that has been tested and is subject to assessment, or with vocabulary and idioms that have not been tested. The evaluation department 144a performs conversation-related evaluations based on the fact that the vocabulary and idioms spoken by the player are the subject of the question, or that one of the two or more languages mentioned above is a foreign language or native language for the player. The character Ca1 has the ability to respond to the question. Evaluation results, change response type functions.

Furthermore, in the story mode, the events that occur when the characters Ca1~3 on the stage St2 are accosted in response to the input of the player's voice signal are processed according to the development of the story, the level, and the position on the map. , the points obtained in the game mode and the level increase, the characters Ca1~3 react in different ways. In addition, the game effect caused by the conversation in the story mode may be, in addition to the character's reaction, value information related to game progress such as points, props, level improvement, etc. The content of the value information (for example, Points earned, etc.) can be synchronized with the level, level, and score of the game mode linked to the story mode. In addition, when idioms trained in the game mode are used for conversations in the story mode, value information such as points and props, changes in game effects, and character reactions may also be changed.

On the other hand, in step S204, when the post-recording mode is selected ("post-recording mode" of S204), post-recording is executed by the post-recording execution unit 143c (S212), and animation reproduction synchronization with the post-recording animation is acquired and evaluated. Input sound (S213), generate and accumulate post-recording data (S214).

To explain the post-recording mode in detail, first, the post-recording data generation unit 144b reads the story mode animation D1 from the memory 15 and the like (S301), and obtains the post-recording data D2 (S302). The post-recording animation D2 may be obtained by cutting out a predetermined scene in the story mode animation D1, or may be an animation corresponding to a predetermined scene of the post-recording data D2 in advance. As the post-recording data D2, in this embodiment, in addition to animation data showing the background and character movements, sound data such as graphic data of the mouth part for lip-syncing, sound effects or background sounds, and sample sounds, To generate text data for navigation or subtitles, etc., set the project file format to decompose each data into different layers or files. This post-recording data D2 is sent to the post-recording data generation unit 144b of the voice recognition processing unit 144, and is accumulated in the memory 15 so as to be usable in the game executed by the game mode execution unit 143a.

The post-recording data D2 generated by the post-recording data generation unit 144b is reproduced when the post-recording mode execution unit 143c executes the post-recording mode (S303), and is displayed on the display unit 13a and the like (S304). The post-recording data D2 is as shown in Figure 9. The text data Txt1 for the sound guide or the text data Txt2 for the subtitles are drawn on different layers in the animation data representing the background and the movement of the character Ca4, and each layer is overlapped. display. In addition, the post-recording data D2 also includes data for GUI (Graphical User Interface) Ob21 for operating the start or stop of synchronized recording, and data for displaying a graphic Ob22 indicating that recording is in progress.

Then, in synchronization with the reproduction process of the post-recording data D2, the player U inputs a voice, and the input voice is recorded (S402). The sound input in step S401 is controlled by the scenario control unit 142, and the correctness of the pronunciation is evaluated by the evaluation unit 144 based on the control signal generated by the sound recognition processing unit 144. The accuracy is evaluated based on numerical values indicating the degree of agreement with the sample sound waveform of the post-recording data D2. When sufficient accuracy is ensured (exceeding a predetermined threshold, etc.), synchronous recording is terminated. When the accuracy is insufficient ( is lower than a predetermined threshold, etc.), processing such as urging re-recording will be performed.

The input sound data D3 of the synchronous recording is analyzed by the post-recording data generation unit 144b (S403), and is processed by a voice changing function as needed (S404). The processed input sound data D3 is passed to the lip-syncing process S405, and is accumulated in the memory 15 as the question sound D42. In the lip-syncing process of step S405, in order to further improve the synchronization accuracy with the input sound data D3 that matches the post-recording data D2, the animation is adjusted in a manner that matches the sound waveform of the input sound data D3. the actions of the characters. The input sound data D3 subjected to the sound processing and lip synchronization processing in this way is synthesized with the post-recording data D2 to form the post-recorded animation D5.

The post-recorded animation D5 is managed and controlled by the post-recorded data control unit 147. In the post-recording mode, in addition to being reproduced in the post-recorded animation reproducing process (S407), it is also used for, for example, story recall scenes, etc. Animation reproduction processing (S406) is in progress. In addition, the post-recorded animation D5 is converted into a question-producing animation D41 of the game mode, and is also used in the question-producing process (S408) of the game mode.

Then, in response to the sound evaluation during game mode execution (S208) and the event processing during story mode execution (S206), the learning level is updated (S209), and the scenario is performed (S210). The above-mentioned processes S202 to S210 are repeated ("N" in S211) until the game ends ("Y" in S211).

(Effect) As explained above, according to this embodiment, by adopting a structure for evaluating the occurrence of foreign languages by players in a game application that can be executed using a terminal device such as a smartphone 1, it is possible to enhance entertainment while It is easy to learn language, and by adopting settings such as talking to the characters on stage St2, you can enrich the training of natural atmosphere and timing in real life.

G1‧‧‧Graphical User Interface G3‧‧‧Window I1, I2‧‧‧Props U(Ua,Ub)‧‧‧Entertainer 1(1a, 1b)‧‧‧Smartphone 2‧‧‧Internet 3‧‧‧Game Server 11‧‧‧Communication interface 12‧‧‧Input interface 12a‧‧‧Touch panel 13‧‧‧Output interface 13a‧‧‧Monitor 14‧‧‧Application Execution Department 15‧‧‧Memory 22‧‧‧Wireless base station 31‧‧‧Communication interface 32‧‧‧Access Management Department 33‧‧‧Certification Department 34‧‧‧Game Data Delivery Department 35‧‧‧Database Group 35a‧‧‧Sound Dictionary Database 35b‧‧‧User database 35c‧‧‧Game database 36‧‧‧Game Processing Department 37‧‧‧Voice Learning Management Department 141‧‧‧Game Processing Department 142‧‧‧Script Control Department 143‧‧‧Incident Control Department 143a‧‧‧Game Mode Execution Department 143b‧‧‧Story Mode Execution Department 143c‧‧‧Post recording mode execution department 144‧‧‧Voice Recognition Processing Department 144a‧‧‧Assessment Department 144b‧‧‧Post-recording data generation department 145‧‧‧Display Control Department 145a‧‧‧Display mode switching part 146‧‧‧Display data generation department 146a‧‧‧Reaction Processing Department 147‧‧‧Post-recording material control department

[Fig. 1] A functional block diagram showing the overall structure of the game system according to the embodiment. [Fig. 2] A block diagram showing the internal structure of the game server according to the embodiment. [Fig. 3] A block diagram showing the internal structure of the user terminal according to the embodiment. [Fig. 4] A sequence diagram showing the operation of the game system according to the embodiment. [Fig. 5] An explanatory diagram showing the screen structure of the theater space of the game system according to the embodiment. [Fig. 6] An explanatory diagram showing the screen structure of the game mode of the game system according to the embodiment. [Fig. 7] An explanatory diagram showing the screen structure of the story mode of the game system according to the embodiment. [Fig. 8] A flowchart showing the operation in the post-recording mode according to the embodiment. [Fig. 9] An explanatory diagram showing the screen structure of the post-recording mode of the game system according to the embodiment.

1‧‧‧Smartphone

11‧‧‧Communication interface

12‧‧‧Input interface

12a‧‧‧Touch panel

13‧‧‧Output interface

13a‧‧‧Monitor

14‧‧‧Application Execution Department

15‧‧‧Memory

141‧‧‧Game Processing Department

142‧‧‧Script Control Department

143‧‧‧Incident Control Department

143a‧‧‧Game Mode Execution Department

143b‧‧‧Story Mode Execution Department

143c‧‧‧Post recording mode execution department

144‧‧‧Voice Recognition Processing Department

144a‧‧‧Assessment Department

144b‧‧‧Post-recording data generation department

145‧‧‧Display Control Department

145a‧‧‧Display mode switching part

146‧‧‧Display data generation department

146a‧‧‧Reaction Processing Department

147‧‧‧Post-recording material control department

Claims (12)

A game system that advances a game in response to operational input and voice input from players, and is characterized by having a voice input unit that inputs voice signals; and a voice recognition unit that compares the input voice signals. The predetermined voice recognition dictionary data is used to identify the player's voice, and based on the recognition results of the voice, it generates game-related control signals; the script control unit follows the script that describes the event processing that makes the story progress, so that the game can be made according to the script. The program proceeds; the game control unit makes the aforementioned script proceed based on the input signal caused by the aforementioned operation input and the aforementioned control signal generated by the aforementioned voice recognition unit; the game mode execution unit executes the aforementioned event processing as one of the aforementioned event processes A game mode in which the correctness of the pronunciation caused by the player is evaluated based on the control signal generated by the voice recognition unit in accordance with the control by the aforementioned script control unit; and the story mode execution unit processes the aforementioned event 1. Execute a story mode in which the story related to the actions of the characters proceeds according to the control caused by the aforementioned script control unit; the aforementioned game mode execution unit detects the voice recognition containing idioms relative to the questions given in the training of the aforementioned game mode The accuracy of the player's pronunciation of the target question is evaluated based on points or level increases corresponding to the detected accuracy; The aforementioned story mode execution unit has a function of executing reaction event processing related to the movement of the aforementioned character in response to the situation in which the sound signal including the aforementioned idiom is input by the player in the aforementioned story mode as evaluated by the aforementioned game mode execution unit. A game system that advances a game in response to operational input and voice input from players, and is characterized by having a voice input unit that inputs voice signals; and a voice recognition unit that compares the input voice signals. The predetermined voice recognition dictionary data is used to identify the player's voice, and based on the recognition results of the voice, it generates game-related control signals; the script control unit follows the script that describes the event processing that makes the story progress, so that the game can be made according to the script. The program proceeds; the game control unit makes the aforementioned script proceed based on the input signal caused by the aforementioned operation input and the aforementioned control signal generated by the aforementioned voice recognition unit; the game mode execution unit executes the aforementioned event processing as one of the aforementioned event processes The game mode is a game mode in which the correctness of the pronunciation caused by the player is evaluated based on the control signal generated by the voice recognition unit according to the control caused by the aforementioned script control unit; the story mode execution unit is one of the aforementioned event processes. , executes a story pattern that advances the story related to the actions of the characters in accordance with the control caused by the aforementioned script control department; and the post-recording data generation department obtains and targets the aforementioned story model The story mode execution unit performs the story reproduction to obtain the animation of the story mode and then reproduce the animation of the post-recording animation to synchronize the input sound to generate the post-recording data; A function to evaluate the correctness of pronunciation caused by the player's pronunciation of the sound recognition target problem posed by the aforementioned post-recording data of the sound input simultaneously with the animation, based on the points or level increases corresponding to the detected correctness. . A game system as described in Item 1 or 2 of the patent application, wherein two or more languages are used in the aforementioned story mode, and sound recognition related to the aforementioned two or more languages is evaluated in the aforementioned game mode. Correctness of object issues. For example, the game system described in Item 1 or 2 of the patent application, wherein in the aforementioned story mode, in the event processing that occurs in response to the input of the sound signal caused by the aforementioned player, the process obtained in the aforementioned game mode is followed. The aforementioned points or levels are increased to allow the aforementioned character to perform actions. A method for controlling a game system, which is a method for controlling a game system to progress a game in response to operational inputs and voice inputs from players, and is characterized by including: In the voice recognition step, the voice input unit accepts the input of a voice signal. The voice recognition unit recognizes the player's voice by comparing the input voice signal with predetermined voice recognition dictionary data, and generates a game based on the voice recognition result. Relevant control signals; game control steps are based on the input signals caused by the aforementioned operation inputs and the aforementioned control signals generated by the aforementioned voice recognition unit. The script control unit follows the script describing the event processing that makes the story progress, so that the game can be made according to the script. The program proceeds; the game mode execution step is when the game mode execution unit, as one of the aforementioned event processes, executes control in accordance with the aforementioned script control unit, and evaluates the aforementioned player based on the control signal generated by the aforementioned voice recognition unit. The correctness of the pronunciation of the game mode; and the story mode execution steps, the story mode execution department, as one of the aforementioned event processing, executes the control caused by the aforementioned script control department, so that the story related to the actions of the characters proceeds; In the aforementioned game mode execution step, the correctness of the player's pronunciation of the sound recognition target question including the idiom given in the training of the aforementioned game mode is detected, and the points corresponding to the detected correctness are or level up to perform the aforementioned evaluation; in the aforementioned story mode, in response to the situation where the sound signal containing the aforementioned idiom evaluated by the aforementioned game mode execution unit is input by the player in the aforementioned story mode, execution of actions related to the aforementioned character Reactive event handling. A control method for a game system that advances a game in response to operational input and voice input from a player. The method is characterized by including: a voice recognition step in which a voice input unit accepts the input of a voice signal. The recognition unit identifies the player's voice by comparing the input voice signal with predetermined voice recognition dictionary data, and generates game-related control signals based on the voice recognition results; the game control step is based on the aforementioned operation input. Based on the input signal and the aforementioned control signal generated by the aforementioned voice recognition unit, the script control unit follows the script describing the event processing for progressing the story, so that the game proceeds in sequence; the game mode execution step is the game mode execution unit as mentioned above. One of the event processes is to execute a game mode in which the control by the aforementioned script control unit is followed and the correctness of the pronunciation caused by the aforementioned player is evaluated based on the control signal generated by the aforementioned voice recognition unit; the story mode execution step is As one of the aforementioned event processes, the story mode execution department executes the story mode in which the story related to the actions of the characters proceeds according to the control caused by the aforementioned script control department; and the post-recording data generation step is obtained and obtained from the post-recording data generation department. For the story mode animation reproduced by the story mode execution unit, the animation of the post-recording animation is synchronized with the input sound to generate the post-recording data; in the aforementioned game mode execution step, detecting The aforementioned post-recording data including sounds input simultaneously with the aforementioned animation for post-recording The accuracy of the pronunciation caused by the player's voice recognition target question is evaluated based on the points or level increases corresponding to the detected accuracy. The control method of a game system as described in Item 5 or 6 of the patent application, wherein two or more languages are used in the aforementioned story mode, and the correlation between the two or more languages is evaluated in the aforementioned game mode. The correctness of the sound recognition object problem. The control method of the game system as described in Item 5 or 6 of the patent application, wherein in the aforementioned story mode, in the event processing that occurs in response to the input of the sound signal caused by the aforementioned player, the aforementioned game mode is followed. The aforementioned points or level improvements obtained in the game allow the aforementioned characters to perform actions. A game program that advances the game in response to operation input and voice input by the player, and is characterized by causing a computer to function as the following components: a voice input unit for inputting voice signals; and a voice recognition unit for The player's voice is recognized by comparing the input voice signal with predetermined voice recognition dictionary data, and based on the voice recognition result, game-related control signals are generated; the script control unit follows the events described to progress the story. drama to deal with This enables the game to proceed in sequence; the game control unit causes the aforementioned script to proceed based on the input signal caused by the aforementioned operation input and the aforementioned control signal generated by the aforementioned voice recognition unit; and the game mode execution unit acts as the aforementioned event One of the processes is to execute a game mode in which the accuracy of the pronunciation caused by the player is evaluated based on the control signal generated by the aforementioned voice recognition unit in accordance with the control caused by the aforementioned scenario control unit; and the story mode execution unit is As one of the aforementioned event processes, a story mode is executed in which the story related to the actions of the characters proceeds according to the control by the aforementioned script control unit; the aforementioned game mode execution unit detects the question posed in the training with respect to the aforementioned game mode. The accuracy of the pronunciation caused by the player's voice recognition target problem including the idiom is evaluated based on the points or level increases corresponding to the detected accuracy; the aforementioned story mode execution unit has the ability to respond to the aforementioned game The mode execution unit evaluates the situation in which the sound signal containing the idiom is input by the player in the story mode, and executes the function of processing reaction events related to the action of the character. A game program that advances a game in response to operation input and voice input by a player, and is characterized by causing a computer to function as the following components: a voice input unit for inputting voice signals; The voice recognition unit recognizes the player's voice by comparing the input voice signal with predetermined voice recognition dictionary data, and generates game-related control signals based on the voice recognition results; the script control unit follows the description The script of the event processing that makes the story progress makes the game proceed in sequence; the game control unit makes the aforementioned script proceed based on the input signal caused by the aforementioned operation input and the aforementioned control signal generated by the aforementioned voice recognition unit; the game mode The execution unit, as one of the aforementioned event processes, executes the control performed by the aforementioned script control unit, and evaluates the correctness of the pronunciation caused by the aforementioned player based on the control signal generated by the aforementioned voice recognition unit; The story mode execution department, as one of the aforementioned event processing, executes the story mode in which the story related to the actions of the characters proceeds according to the control caused by the aforementioned script control department; and the post-recording data generation department obtains and targets the aforementioned The story mode execution unit performs the reproduction of the story mode animation to obtain the animation of the post-recording animation and reproduce the synchronously input sound to generate the post-recording data; the aforementioned game mode execution unit has the function of detecting the use of the animation including the aforementioned The correctness of pronunciation caused by the player's pronunciation of the sound recognition target problem posed by the aforementioned post-recording data of the sound input simultaneously with the animation is evaluated based on the points or level increases corresponding to the detected accuracy. Function. For example, the game program described in Item 9 or 10 of the patent application, wherein two or more languages are used in the aforementioned story mode, and sound recognition related to the aforementioned two or more languages is evaluated in the aforementioned game mode. Correctness of object issues. For example, the game program described in Item 9 or 10 of the patent application, wherein in the aforementioned story mode, in the event processing that occurs in response to the input of a sound signal caused by the aforementioned player, the process obtained in the aforementioned game mode is followed. The aforementioned points or levels are increased to allow the aforementioned character to perform actions.