JP7090216B2 - Game programs, methods, information processing equipment, systems - Google Patents

Description

The present disclosure relates to game programs, methods, information processing devices, and systems.

An application for encouraging tooth brushing is provided for a child who does not want to brush teeth (Non-Patent Document 1). In Non-Patent Document 1, the user executes an application installed on a mobile terminal and starts a game. In the game, the user takes a picture of himself brushing his teeth with the camera of the mobile terminal. The captured image and the game screen are displayed on the display of the mobile terminal. The user brushes his teeth by moving the toothbrush while checking his own image displayed on the display.

LITALICO, "Brushing Hero", [online], [Search on May 1, 2nd year of Reiwa], Internet <URL: https://app.litalico.com/brushinghero/jp.html>

In the application according to Non-Patent Document 1, a time limit for brushing one tooth is set, and evaluation is given according to the movement of the toothbrush within the time limit. Therefore, the user will continue to move the toothbrush within the time limit in order to improve the evaluation.

However, in Non-Patent Document 1, the time limit is timed even while the user is not brushing his teeth. For this reason, even if there are temporary circumstances such as when an adult brushes a child's teeth for finishing brushing after the child brushes his / her own teeth, or when the user who moves the toothbrush changes, the tooth brushing cannot be continued. As the game progresses, there is a risk that the user's willingness to brush their teeth will be discouraged.

Therefore, an object of the present disclosure is to provide a game program, a method, and an information processing device that do not interfere with the user's brushing of teeth while maintaining the interest.

According to one embodiment, a game program for running a computer having a processor and memory is provided. The game program causes the processor to detect the user's face area from the step of starting the time counting that defines one play of the toothbrushing game, the step of shooting the user, and the shot image according to the start of the toothbrushing game. , A step to set the area around the mouth by detecting facial organs from the detected facial area, a step to detect the movement of an object including a toothbrush and a hand in the area around the mouth, and a detected object. A step of controlling the toothbrushing game based on the movement of the tooth and a step of stopping the time counting when the area around the mouth is not set without stopping the time counting when the movement of the object is not detected.

According to the present disclosure, it is possible to prevent the user from hindering tooth brushing while maintaining the interest.

It is a figure which shows the whole structure of a game system 1. It is a block diagram of the terminal apparatus 10 which constitutes the game system 1 of 1st Embodiment. It is a figure which shows the functional configuration of a server 20. It is a figure which shows the example of the data structure of the user information 181 and the comment information 182 stored in the terminal apparatus 10. It is a figure which shows the data structure of the user information database 281 and the log information database 282 stored in a server 20. It is a figure for demonstrating the operation of the terminal apparatus 10 and the server 20 at the time of playing a tooth brushing game. 6 is a flowchart showing an example of the operation of the control unit 190 during the tooth brushing process shown in FIG. It is a schematic diagram which shows the display example of the display 132. It is a schematic diagram which shows the example of the object detected by the image analysis of the data processing unit 196. It is a schematic diagram which shows the example of the effect generated by the game control unit 193. It is a schematic diagram which shows the example of the effect generated by the game control unit 193. It is a schematic diagram which shows an example of the capture screen of a boss character. It is a schematic diagram which shows the display example of the display 132 when the face area of a user cannot be detected. It is a schematic diagram which shows the display example of the display 132 when the description prompting tooth brushing is displayed. It is a schematic diagram which shows the display example of the display 132 when the description praising tooth brushing is displayed. It is a schematic diagram which shows the display example of the display 132 when the description prompting the replacement of a toothbrush is displayed. It is a flowchart which shows the example of the operation of the data processing unit 196 when updating the item "exchange date". It is a schematic diagram which shows an example of the display screen of the user who covered the headgear.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are the same. Therefore, the detailed description of them will not be repeated.

<First Embodiment>
<1 Configuration diagram of the entire game system>
FIG. 1 is a diagram showing the overall configuration of the game system 1. In the game system 1, the terminal device (in FIG. 1, the terminal device 10A and the terminal device 10B are shown; hereinafter collectively referred to as "terminal device 10") provides a tooth brushing game to the user, and the server 20 brushes the teeth. Manage game user information.

As shown in FIG. 1, the game system 1 includes a plurality of terminal devices 10 and a server 20. The terminal device 10 and the server 20 communicate with each other via the network 80.

In the present embodiment, each device (terminal device, server, etc.) may be used as an information processing device. Further, the aggregate of each device may be used as one information processing device. That is, the game system 1 may be formed as an aggregate of a plurality of devices. The method of allocating the plurality of functions required to realize the game system 1 according to the present embodiment to one or a plurality of hardware includes the processing capacity of each hardware and / or the specifications required for the game system 1. It can be decided as appropriate in view of the above.

The terminal device 10 is a device operated by the user. The terminal device 10 is realized by a mobile terminal such as a smartphone or a tablet compatible with a mobile communication system. In addition, the terminal device 10 may be, for example, a stationary PC (Personal Computer), a laptop PC, or a dedicated game machine. Further, the terminal device 10 may function as a head-mounted display, and may function as, for example, a transmissive type, a non-transparent type, or a see-through type head-mounted display. For example, the terminal device 10 may function as a portable terminal when it does not function as a head-mounted display, and may function as a head-mounted display when attached to a goggle-type device. In this case, the terminal device 10 can switch between a mode that functions as a mobile terminal and a mode that functions as a head-mounted display. In the mode that functions as a head-mounted display, the terminal device 10 detects the movement of the user's head by the motion sensor built in the terminal device 10, and the display image of the display is linked to the movement of the user's head. To update.

The terminal device 10 is compatible with wireless base stations 81 compatible with communication standards such as 5G and LTE (Long Term Evolution), and wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11. It is connected to the network 80 by communicating with a communication device such as a wireless LAN router 82.

As shown as the terminal device 10B in FIG. 1, the terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19.

The communication IF 12 is an interface for inputting / outputting signals because the terminal device 10 communicates with an external device.

The input device 13 is an input device (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.) for receiving an input operation from a user.

The output device 14 is an output device (display, speaker, etc.) for presenting information to the user.

The memory 15 is for temporarily storing a program, data processed by the program, or the like, and is, for example, a volatile memory such as a DRAM (Dynamic Random Access Memory).

The storage unit 16 is a storage device for storing data, and is, for example, a flash memory or an HDD (Hard Disc Drive).

The processor 19 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

The server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29.

The server 20 manages information of each user who uses the tooth brushing game. The server 20 manages, for example, user-identifying information, a user's name, a game character captured by the user, a game item owned by the user, and various information set by the user as user information. The server 20 may manage the amount of virtual currency held (including the one given to the user free of charge and the one given to the user for a fee).

In addition, the server 20 manages log information in which the user has played the tooth brushing game. The server 20 manages, for example, user identification information, a game execution date, a game start time, information related to tooth brushing evaluation, and the like as log information.

The communication IF 22 is an interface for inputting / outputting signals because the server 20 communicates with an external device.

The input / output IF 23 functions as an interface with an input device for receiving an input operation from the user and an output device for presenting information to the user.

The memory 25 is for temporarily storing a program, data processed by the program, or the like, and is, for example, a volatile memory such as a DRAM (Dynamic Random Access Memory).

The storage 26 is a storage device for storing data, and is, for example, a flash memory or an HDD (Hard Disc Drive).

The processor 29 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

<1.1 Configuration of terminal device 10>
FIG. 2 is a block diagram of a terminal device 10 constituting the game system 1 of the first embodiment. As shown in FIG. 2, the terminal device 10 operates with a plurality of antennas (antenna 111, antenna 112) and a wireless communication unit (first wireless communication unit 121, second wireless communication unit 122) corresponding to each antenna. Reception unit 130 (including touch sensitive device 131 and display 132), voice processing unit 140, microphone 141, speaker 142, position information sensor 150, camera 160, motion sensor 170, and storage unit 180. And the control unit 190.

The terminal device 10 also has functions and configurations (for example, a battery for holding electric power, a power supply circuit for controlling the supply of electric power from the battery to each circuit, and the like) which are not particularly shown in FIG. As shown in FIG. 2, each block included in the terminal device 10 is electrically connected by a bus or the like.

The antenna 111 radiates a signal emitted by the terminal device 10 as a radio wave. Further, the antenna 111 receives radio waves from the space and gives a received signal to the first wireless communication unit 121.

The antenna 112 radiates a signal emitted by the terminal device 10 as a radio wave. Further, the antenna 112 receives radio waves from the space and gives a received signal to the second radio communication unit 122.

Since the terminal device 10 communicates with other wireless devices, the first wireless communication unit 121 performs modulation / demodulation processing for transmitting / receiving signals via the antenna 111. Since the terminal device 10 communicates with another wireless device, the second wireless communication unit 122 performs modulation / demodulation processing for transmitting / receiving signals via the antenna 112. The first wireless communication unit 121 and the second wireless communication unit 122 are communication modules including a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high frequency circuit, and the like. The first wireless communication unit 121 and the second wireless communication unit 122 perform modulation / demodulation and frequency conversion of the wireless signal transmitted / received by the terminal device 10 and supply the received signal to the control unit 190.

The operation receiving unit 130 has a mechanism for receiving a user's input operation. Specifically, the operation receiving unit 130 is configured as a touch screen and includes a touch sensitive device 131 and a display 132. The touch-sensitive device 131 accepts an input operation by the user of the terminal device 10. The touch-sensitive device 131 detects the user's contact position with respect to the touch panel, for example, by using a capacitive touch panel. The touch-sensitive device 131 outputs a signal indicating the contact position of the user detected by the touch panel to the control unit 190 as an input operation.

The display 132 displays data such as an image, a moving image, and text according to the control of the control unit 190. The display 132 is realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

The voice processing unit 140 performs modulation / demodulation of the voice signal. The voice processing unit 140 modulates the signal given from the microphone 141 and gives the modulated signal to the control unit 190. Further, the voice processing unit 140 gives a voice signal to the speaker 142. The voice processing unit 140 is realized by, for example, a processor for voice processing. The microphone 141 receives the voice input and gives the voice signal corresponding to the voice input to the voice processing unit 140. The speaker 142 converts the voice signal given from the voice processing unit 140 into voice and outputs the voice to the outside of the terminal device 10.

The position information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. The satellite positioning system receives signals from at least three or four satellites and detects the current position of the terminal device 10 on which the GPS module is mounted based on the received signals. For example, in the game system 1, when the position of the user registered as a friend can be referred to, the terminal device 10 can display a list of friends near the user on the display 132.

The camera 160 is a device for receiving light by a light receiving element and outputting it as a shooting signal. The camera 160 is, for example, a depth camera that can detect the distance from the camera 160 to the shooting target. In the camera 160, for example, a lens that receives light is installed side by side with the display 132. As a result, the camera 160 can take a picture of the user who visually recognizes the display 132.

The motion sensor 170 includes an acceleration sensor, an angular velocity sensor, and the like, detects the movement of the terminal device 10, and outputs a sensing result.

The storage unit 180 is composed of, for example, a flash memory or the like. The storage unit 180 stores a program related to the tooth brushing game. The storage unit 180 stores data used in the tooth brushing game, for example, user information 181 and comment information 182.

User information 181 is user information in a tooth brushing game based on a game program. The user information includes information for identifying the user, a user's name, a game character captured by the user, a game item owned by the user, various information set by the user, and the like.

The comment information 182 is information about a comment displayed to the user while the tooth brushing game is in progress. The comments include advice on brushing or useful information on brushing. In the comment information 182, for example, the comment is associated with the evaluation of the toothpaste given during the toothpaste game. The comment information 182 is downloaded from the server 20, for example, and stored in the storage unit 180.

The control unit 190 is realized by the processor 19 reading a program stored in the storage unit 180 and executing an instruction included in the program. The control unit 190 controls the operation of the terminal device 10. By operating according to the program, the control unit 190 functions as an input operation reception unit 191, a transmission / reception unit 192, a game control unit 193, a shooting control unit 194, a notification control unit 195, and a data processing unit 196. Demonstrate.

The input operation receiving unit 191 performs a process of accepting a user's input operation to an input device such as a touch sensitive device 131. The input operation receiving unit 191 is based on the information of the coordinates when the user touches the touch-sensitive device 131 with a finger or the like, and the user's operation is a flick operation, a tap operation, or a drag (swipe). Determine the type of operation, such as whether it is an operation.

The transmission / reception unit 192 performs a process for the terminal device 10 to transmit / receive data to / from an external device such as a server 20 according to a predetermined communication protocol.

The game control unit 193 performs a process of advancing the tooth brushing game. For example, the game control unit 193 starts the tooth brushing game based on the data input from the user. Further, the game control unit 193 advances the tooth brushing game based on the movement of a predetermined object detected by the data processing unit 196. Further, the game control unit 193 ends one play of the tooth brushing game when the play time reaches a predetermined time.

The shooting control unit 194 performs a process of controlling the camera 160.

The notification control unit 195 performs a process of presenting information to the user. The notification control unit 195 performs a process of displaying the display image and the game image on the display 132, a process of outputting the sound to the speaker 142, and the like.

The data processing unit 196 performs a calculation on the data received by the terminal device 10 according to a program, and outputs the calculation result to the memory 15 or the like. Specifically, the data processing unit 196 performs image analysis on the captured image, for example, and detects a predetermined region in the captured image. The data processing unit 196 detects the regular motion of a predetermined object in the detected region.

<1.2 Functional configuration of server 20>
FIG. 3 is a diagram showing a functional configuration of the server 20. As shown in FIG. 3, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

The communication unit 201 performs a process for the server 20 to communicate with an external device.

The storage unit 202 stores a program related to the tooth brushing game. The storage unit 202 is provided with a database in which data related to the tooth brushing game is stored, for example, a user information database 281 and a log information database 282.

The user information database 281 is a database for holding information of each user in the tooth brushing game. Details will be described later.

The log information database 282 is a database for holding a play log of a tooth brushing game. Details will be described later.

The control unit 203 is realized by the processor 29 reading the program stored in the storage unit 202 and executing the instruction included in the program. By operating according to the program, the control unit 203 exerts the functions shown as the operation content acquisition module 2041, the reception control module 2042, the transmission control module 2043, the first management module 2044, and the second management module 2045.

The operation content acquisition module 2041 acquires the operation content of the user. The operation content acquisition module 2041 acquires, for example, which game unit the user has designated as the target of game play, as the operation content of the user.

The reception control module 2042 controls the process by which the server 20 receives a signal from an external device according to a communication protocol.

The transmission control module 2043 controls a process in which the server 20 transmits a signal to an external device according to a communication protocol.

The first management module 2044 manages the user information database 281. For example, when the first management module 2044 receives information about the tooth brushing game from the terminal device 10, the first management module 2044 updates the user information database 281 based on the received information.

The second management module 2045 manages the log information database 282. For example, when the second management module 2045 receives information about the tooth brushing game from the terminal device 10, the second management module 2045 updates the log information database 282 based on the received information.

<2 data structure>
FIG. 4 is a diagram showing an example of the data structure of the user information 181 and the comment information 182 stored in the terminal device 10.

As shown in FIG. 4, the record of the user information 181 includes the item "user ID", the item "user name", the item "age", the item "language", the item "date of birth", and the item "birth date". Includes "play time", item "notification", item "notification time", item "exchange date", item "owned item", item "captured character", and the like.

The item "user ID" represents information that identifies a user. The user ID is automatically set, for example, when creating an account.

The item "user name" represents a name set by the user. The user name is set based on user input, for example, when creating an account.

The item "age" represents the age of the user. The age is set, for example, based on user input when creating an account.

The item "language" is a language set by the user and used by the user. The language is set, for example, based on user input when creating an account. On the game screen displayed on the display 132, the language set in the item is used.

The item "date of birth" represents the date of birth of the user. The date of birth is set, for example, based on user input when creating an account.

The item "play time" represents the time limit for playing the toothpaste game once. When the toothpaste game starts, the timekeeping of the play time starts. When the play time is reached, one play of the toothpaste game ends. The play time can be set by user operation at any time when the toothpaste game is not played. The play time may be directly input by the user, for example, or may be selected and input by the user at a time such as 1 minute, 1 minute, 30 seconds, or 2 minutes.

The item "notification" indicates the necessity of notification of the start of the tooth brushing game. The necessity of notification is represented by, for example, a "circle" indicating that it is necessary or a "batsu" indicating that it is not necessary.

The item "notification time" represents the time for notifying the start of the tooth brushing game. When the item "notification" is a circle, when the time of the item is reached, the tooth brushing game is automatically executed and the start screen is displayed on the display 132.

The item "replacement date" represents the date when the toothbrush was most recently replaced. The server 20 may accept an operation indicating that the toothbrush has been replaced from the user of the terminal device 10, or may detect that the toothbrush has been replaced regardless of the user's operation. For example, by image analysis in a tooth brushing game described later, the terminal device 10 identifies a toothbrush used by a user in the tooth brushing game, and information for identifying the specified toothbrush (for example, a feature amount of a toothbrush portion in an image by image analysis). The calculated data, etc. calculated from) is stored. The terminal device 10 detects that the toothbrush has been replaced by image analysis. The server 20 writes the date when the toothbrush is replaced in the item by the detection in the terminal device 10.

The item "possessed item" represents information about the game item owned by the user. The game item is given from the server 20 as a reward for playing the tooth brushing game, for example. The game item includes, for example, a game object displayed in association with a part of the user's body in a photographed image of the user in a tooth brushing game described later. For example, the game item includes a headgear displayed in association with a part of the user's body (for example, the head) recognized in the photographed image in the tooth brushing game described later. The headgear is displayed superimposed on the photographed image taken by the user so as to cover the head of the user displayed on the display 132 during the play of the tooth brushing game. There are multiple headgear designs, for example based on the face of the character used in the toothpaste game.

Further, the game item includes a game object displayed as an image superimposed on a photographed image taken by the user in the tooth brushing game, a captured image of the photographed image in the tooth brushing game, or the like. The game object includes a character designed in the motif of a character (including a character that can be captured by the user) appearing in the tooth brushing game, and includes, for example, a sticker-shaped game object that can be superimposed on a still image. The sticker is designed with the character captured by the user as a motif. Although it is described as a sticker here, it may be referred to as a stamp or a mark as long as the shape related to the captured character is arbitrarily given to be usable.

The game item also includes a background image that is composited with the user's background. The background image is based on the landscape associated with the character's characteristics.

The item "captured character" represents information about the game character captured by the user. Game characters are captured, for example, when they can brush their teeth well in a tooth brushing game. Here, "to be able to brush a good tooth in a tooth brushing game" means that in a tooth brushing game described later, an action determined by the user to move a toothbrush is repeatedly detected within a certain period of time, and is repeated a certain number of times. Includes detection and so on. In addition, "you can brush your teeth well in a toothbrushing game" means that in a toothbrushing game described later, when the user detects the movement of the toothbrush and the game progresses according to the detection result, certain game parameters are set. Includes achieving (eg, making a certain number of successful toothbrush movements, doing a certain amount of damage to the boss character, etc.). There are multiple types of game characters, each of which has a different shape or color. Information such as the type of game character is also included in the item.

As shown in FIG. 4, each of the records of the comment information 182 includes an item "evaluation", an item "comment", and the like.

The item "evaluation" indicates a parameter that evaluates the result of the user brushing teeth by the tooth brushing game. For example, on the server 20, the result of evaluating the tooth brushing operation by the user based on the sensing data in which the user brushed the tooth with a toothbrush in the tooth brushing game or the result of advancing the game by detecting the tooth brushing operation. Is shown. In the example shown in FIG. 4, an ABC rating is attached to one play. The evaluation here may be based on any index, for example, an evaluation of unpolished residue, an evaluation of politeness, an evaluation of how to move the toothbrush, and the like. For example, during one play of a toothbrushing game, an evaluation may be given based on the number of times that the way of moving the toothbrush is determined to be suitable. Further, the evaluation index is not limited to ABC, and may be any index.

The item "comment" represents a comment about toothpaste. As for the comment, the comment corresponding to the evaluation of the record is described. Comments include advice on brushing teeth, such as how to use a toothbrush well, how to brush unbrushed teeth well, and so on. The comments also include useful information about brushing, such as effective brushing times, explanations of the causes of tooth decay, and introductions to effective toothbrushes.

FIG. 5 is a diagram showing the data structures of the user information database 281 and the log information database 282 stored in the server 20.

As shown in FIG. 5, in the user information database 281, user information 181 stored in a plurality of terminal devices 10 is managed. Each of the records in the user information database 281 has the item "user ID", the item "user name", the item "age", the item "language", and the item "date of birth", as in the case of the user information 181. , The item "play time", the item "notification", the item "notification time", the item "exchange date", the item "owned item", the item "captured character" and the like.

As shown in FIG. 5, each of the records in the log information database 282 includes an item "user ID", an item "implementation date", an item "start time", an item "evaluation", and the like.

The item "implementation date" represents the date on which the toothpaste game was played.

The item "start time" represents the time when the play of the toothpaste game is started.
The item "evaluation" indicates a parameter that evaluates the result of the user brushing teeth by the tooth brushing game. For example, on the server 20, the result of evaluating the tooth brushing operation by the user based on the sensing data in which the user brushed the tooth with a toothbrush in the tooth brushing game or the result of advancing the game by detecting the tooth brushing operation. Is shown. For example, regardless of the result of the user advancing the tooth brushing game, the user is based on the result of detecting the tooth brushing motion (how many times the toothbrush is moved, whether the tooth brush is brushed evenly at multiple tooth brushing points, etc.). You may evaluate the result of brushing your teeth.

<3 operation>
The operation of the terminal device 10 and the server 20 when playing the tooth brushing game will be described. FIG. 6 is a diagram for explaining the operation of the terminal device 10 and the server 20 when playing a tooth brushing game.

The user of the terminal device 10 selects a tooth brushing game application from the application group displayed on the display 132 of the terminal device 10.

The terminal device 10 receives input from the user by the input operation reception unit 191. When the application of the tooth brushing game is selected by the user, the notification control unit 195 of the terminal device 10 displays the start screen of the tooth brushing game on the display 132.

Further, when the notification control unit 195 detects that the notification time set in the toothpaste game has been reached by the alarm function installed in the OS (Operating System), the start screen of the toothpaste game is displayed on the display 132. Display.

The user presses the start button from the start screen displayed on the display 132.

When the user presses the start button displayed on the display 132, the game control unit 193 of the terminal device 10 starts the tooth brushing game (step S101). When the tooth brushing game is started, the transmission / reception unit 192 of the terminal device 10 transmits a first notification indicating that the tooth brushing game has started to the server 20. The first notification includes, for example, a play ID, a user ID, an implementation date, and a start time.

(A) Toothpaste FIG. 7 is a flowchart showing an example of the operation of the control unit 190 during the toothpaste process shown in FIG. In FIG. 7, a case where the terminal device 10 is a smartphone will be described as an example.

When the tooth brushing game is started by the game control unit 193, the shooting control unit 194 controls the camera 160 and starts shooting a moving image (step S1021).

The notification control unit 195 creates a captured image based on the captured signal acquired by the camera 160. The notification control unit 195 displays the captured image and the game image on the display 132 (step S1022). When the tooth brushing game is started, the game control unit 193 reads the play time from the user information 181 of the storage unit 180. The game control unit 193 starts timing the read play time. The notification control unit 195 displays the timed information on the display 132.

FIG. 8 is a schematic diagram showing a display example of the display 132. In the example shown in FIG. 8, the captured image I1 and the game image I2 are displayed on the display 132. The captured image I1 is displayed in the upper area of the display 132, and the game image I2 is displayed in the lower area of the display 132.

The captured image I1 includes a landscape photographed by the camera 160. In FIG. 8, a user brushing his teeth is displayed. The user refers to the captured image I1 and brushes his / her teeth while confirming that he / she is displayed on the captured image I1. The terminal device 10 may be supported by the user or may be supported by a person other than the user. Further, the terminal device 10 may be placed or fixed in a stable place.

In the game image I2, an image about the tooth brushing game is displayed. In FIG. 8, an image imitating the inside of the mouth is displayed on the terminal device 10. As shown in FIG. 8, in the terminal device 10, an image including a plurality of teeth showing the inside of the mouth from the inside is displayed. The image of FIG. 8 suggests to the user a place to brush teeth. Since FIG. 8 shows an image immediately after the game is started, the displayed teeth are in a dirty state.

Information indicating the play time is displayed on the display 132. The information representing the play time may be a display showing the remaining time or the elapsed time in a graph, or a display showing the remaining time or the elapsed time numerically. In the example shown in FIG. 8, the remaining time 1321 is superimposed and displayed on the captured image I1. Since FIG. 8 is an example of an image immediately after the game is started, “2:00” set as the play time is displayed.

The data processing unit 196 analyzes the captured image I1 displayed on the display 132 and detects the face region included in the captured image I1 (step S1023). In the present embodiment, the facial region includes the region from the chin to the head. In the image analysis method by the data processing unit 196, for example, an existing method is used. For example, the data processing unit 196 inputs the captured image I1 into the trained model trained to detect the face, and detects the face region included in the captured image I1.

When the data processing unit 196 detects the face area, the data processing unit 196 detects the organs constituting the face from the detected face area (step S1024). In this embodiment, the facial organs are, for example, the chin, mouth, nose, eyes, ears, and head. In the image analysis method by the data processing unit 196, for example, an existing method is used. For example, the data processing unit 196 inputs the detected face region into the trained model trained to detect the facial organs, and detects the face region included in the captured image I1. The trained model used in the data processing unit 196 may be trained to output the face region and the facial organs at the same time when the captured image I1 is input.

When the data processing unit 196 detects a facial organ, it detects that a predetermined object region is in regular motion in a predetermined region set based on the detected organ (step S1025). For example, the data processing unit 196 sets a region around the mouth based on the detected organs such as the jaw, mouth, nose, and eyes. The data processing unit 196 takes a difference between frames in the set area, and detects a regular motion in the set area by the object area of a predetermined size. The object area of a predetermined size represents, for example, a mass area representing a toothbrush and a hand holding the toothbrush. The regular movement represents, for example, a repetitive movement at a predetermined speed in a preset direction.

FIG. 9 is a schematic diagram showing an example of an object detected by image analysis of the data processing unit 196. In step S1023, the data processing unit 196 detects the face region R1 indicated by the broken line. In step S1024, the data processing unit 196 detects facial organs (mouth O1, nose O2, eyes O3, head O4) indicated by the alternate long and short dash line. In step S1025, the data processing unit 196 detects the regular motion M1 represented by the alternate long and short dash line in the object region in the region around the mouth.

When the data processing unit 196 detects a regular movement of the object region, the game control unit 193 generates an effect on the game according to the detected movement (step S1026). The game control unit 193 generates a predetermined effect in response to the detected regular movement, for example. The predetermined effect is, for example, an effect of superimposing an animation of a tooth brushing effect such as bubbles and brilliance on a dirty tooth from a toothbrush to remove stains on the tooth located at the destination of the animation and making a white tooth.

FIG. 10 is a schematic diagram showing an example of the effect generated by the game control unit 193. According to the example shown in FIG. 10, bubbles are radiated from the object region to the teeth T1, and the stains on the teeth T1 that have been radiated by the bubbles are washed.

The user is interested in the bubbles radiated from the toothbrush and the whitening teeth and keeps moving the hand holding the toothbrush. In addition, if the user does not brush his teeth, bubbles will not be radiated from the toothbrush, and the user will feel a sense of crisis in the teeth to which dirt adheres, and will continue to move the hand holding the toothbrush. The data processing unit 196 detects the toothbrush and the hand holding the toothbrush, that is, the regular movement of the object region in a plurality of frames. The game control unit 193 radiates bubbles from the toothbrush to the teeth in the frame that detects the movement, and cleans the dirty teeth one by one.

The game control unit 193 may produce an effect for promoting continuous brushing of teeth. For example, the game control unit 193 may make the whitened teeth dirty again when a plurality of frames in which regular motion of the object region is not detected continue.

When the tooth brushing reaches a predetermined condition, the game control unit 193 generates a boss character in the game image I2. The predetermined conditions are, for example, that the game progress has achieved a predetermined situation such that the preset number of teeth of the teeth displayed in the game image I2 are not soiled, and the remaining play time is A predetermined time has been reached, or a combination thereof. The boss character is supposed to be a caries boss, for example.

The appearance of the boss character is based on the appearance of any one of the plurality of game characters. The game control unit 193 randomly selects one of a plurality of game characters as a boss character, for example. When the game control unit 193 generates a boss character in the game image I2, for example, the entire game character cannot be confirmed due to dirt. For example, the game control unit 193 masks the game character with one color and generates a boss character in such a manner that only the silhouette can be visually recognized. Further, for example, the game control unit 193 generates a boss character in such a manner that only a part of the game character can be visually recognized.

The game control unit 193 may select any one of the plurality of game characters as the boss character based on the condition of brushing the teeth. For example, when the game control unit 193 is good at brushing teeth, the game control unit 193 selects a rare (strength) game character according to the skill. The game control unit 193 certifies the skill of brushing teeth based on, for example, the number of consecutive frames in which regular motion of an object region is detected. The game control unit 193 is good at brushing teeth, taking into consideration whether regular movement is detected in a plurality of parts of the area around the mouth or at a position suitable for brushing the area around the mouth. You may certify that.

When the data processing unit 196 detects a regular motion of the object region, the game control unit 193 generates an effect on the game according to the detected motion. For example, the game control unit 193 superimposes an animation of tooth brushing effects such as bubbles and brilliance on the boss character from the toothbrush according to the detected regular movement, and stains the boss character located at the destination of the animation. Drop.

The user is interested in the bubbles emitted from the toothbrush and the boss character from which the dirt has been removed, and keeps moving the hand holding the toothbrush. In addition, if the user does not brush his teeth, bubbles will not be radiated from the toothbrush, and the boss character who is getting dirty will feel a sense of crisis and will continue to move his hand holding the toothbrush. The data processing unit 196 detects the regular movement of the hand holding the toothbrush, that is, the object region, in a plurality of frames. The game control unit 193 radiates bubbles from the toothbrush to the boss character in the frame that detects the movement, and removes dirt from the boss character.

The game control unit 193 may generate a small character in addition to the boss character. The small character is supposed to be a caries bacterium, for example. When the data processing unit 196 detects a regular motion of the object region, the game control unit 193 generates an effect on the game according to the detected motion. For example, the game control unit 193 superimposes an animation of tooth brushing effects such as bubbles and brilliance on a small character from a toothbrush according to the detected regular movement, and eliminates the small character located at the destination of the animation. Let me.

FIG. 11 is a schematic diagram showing an example of the effect generated by the game control unit 193. According to the example shown in FIG. 11, bubbles are radiated from the object area to the boss character B1 and the small character S1. The dirt of the boss character B1 that received the radiation of bubbles is removed. In addition, the small character S1 that received the radiation of bubbles disappears.

The game control unit 193 may produce an effect for promoting continuous brushing of teeth. For example, the game control unit 193 may make the boss character from which the dirt has been removed become dirty again when a plurality of frames in which the motion of the object region is not detected continue. Further, for example, the game control unit 193 may increase the number of small characters when a plurality of frames in which motion of the object region is not detected continue.

When the set play time is reached, the game control unit 193 ends one play of the tooth brushing game. According to the example shown in FIG. 11, when the remaining time 1321 becomes "0:00", the game control unit 193 ends one play of the tooth brushing game. When one play of the tooth brushing game is completed, the game control unit 193 displays a screen for capturing the boss character. On the screen for capturing the boss character, a description is displayed so that the displayed screen can be grasped as the capture screen.

FIG. 12 is a schematic diagram showing an example of a capture screen of a boss character. According to the example shown in FIG. 12, the description of "Get chance!", Which indicates that the screen is a capture screen, is superimposed on the captured image I1. In the game image I2, the boss character B1 and the capture tool 1323 for capturing the boss character B1 are displayed.

The user flies the catcher 1323 to the boss character B1 by, for example, flicking the catcher 1323. The user attempts to capture the boss character B1 by flying the capture tool 1323 to the boss character B1.

When the capture tool 1323 comes into contact with the boss character B1, the game control unit 193 determines whether or not the boss character B1 can be captured. At this time, the game control unit 193 sets the probability that the boss character B1 can be captured based on the skill of brushing teeth. For example, the game control unit 193 may determine whether or not the user has successfully brushed teeth based on whether or not the user has cleared the game task presented to the user regarding the operation of brushing teeth. For example, the game task may be whether a plurality of teeth are brushed evenly, whether the toothbrush can be moved repeatedly within a certain period of time, whether the number of times the toothbrush is moved reaches a certain number of times, and the like. The game control unit 193 improves the capture probability of the boss character B1 when the user can brush his teeth well. On the other hand, for example, the game control unit 193 lowers the capture probability of the boss character B1 when the tooth brushing cannot be done well. More specifically, the game control unit 193 sets the probability that the boss character B1 can be captured based on the degree of dirt removal of the boss character B1. For example, the game control unit 193 improves the capture probability of the boss character B1 when all the dirt on the boss character B1 has been removed. On the other hand, for example, the game control unit 193 lowers the capture probability of the boss character B1 when the dirt of the boss character B1 is not removed. That is, the game control unit 193 sets the probability that the privilege can be acquired according to the detection result of the regular motion of the object region.

When the boss character is captured, the game control unit 193 registers the captured character in the user information 181. The characters registered in the user information 181 can be displayed on the display 132 as a list.

When the boss character is captured, the game control unit 193 gives the user a sticker designed based on the captured character. When the sticker is attached, the game control unit 193 registers the attached sticker in the user information 181 as a possessed item.

The game control unit 193 determines whether or not to attach a headgear when the capture of the boss character is completed regardless of whether or not the capture is successful. The game control unit 193 randomly determines, for example, whether or not to give the user a headgear. The game control unit 193 may decide whether or not to apply the headgear based on the skill of brushing the teeth. When the headgear is given, the game control unit 193 registers the given headgear in the user information 181 as a possessed item. The game control unit 193 may attach the headgear when displaying the start screen.

The game control unit 193 ends the tooth brushing game when it decides to apply the headgear or not to apply the headgear. When the game control unit 193 ends the tooth brushing game, the game control unit 193 reads a comment from the comment information 182. The notification control unit 195 notifies the read comment. For example, the notification control unit 195 displays the read comment on the display 132.

When the game control unit 193 ends the tooth brushing game, the game control unit 193 may calculate the evaluation of the performed tooth brushing. The game control unit 193 evaluates the tooth brushing based on, for example, the detection status of the motion of the object region by the data processing unit 196, the cleaning status of the boss character, the number of times the tooth brushing is praised, and the like. The game control unit 193 may read a comment from the comment information 182 based on the evaluation of the tooth brush.

(A) Play time stop processing Although the operation of the control unit 190 in the tooth brushing game has been described, the operation of the control unit 190 is not limited to the above.

For example, in the tooth brushing game according to the present embodiment, the play time is temporarily stopped. For example, if the data processing unit 196 cannot detect the user's face area in step S1023, the game control unit 193 stops timing the play time. The case where the data processing unit 196 cannot detect the user's face area is, for example, the case where the user is not displayed on the display 132, or the case where the user is not facing the front on the display 132.

If the data processing unit 196 cannot detect the user's face area in step S1023, the notification control unit 195 notifies the user to show the face area.

FIG. 13 is a schematic diagram showing a display example of the display 132 when the user's face area cannot be detected. In the example shown in FIG. 13, the user is facing sideways. Since the user is facing sideways, the data processing unit 196 cannot detect the user's face area. If the data processing unit 196 does not detect the user's face area, the game control unit 193 stops timing the play time. When the timing of the play time is stopped, the notification control unit 195 keeps the remaining time 1321 as it was when it was stopped. According to the example shown in FIG. 13, the notification control unit 195 maintains the remaining time 1321 at “1:45”. Further, the notification control unit 195 causes the display 132 to display a description urging the user to show his / her face. According to the example shown in FIG. 13, the notification control unit 195 causes the captured image I1 to display the description of "Show me the kao!". The notification prompting the user to show the face area may be a voice notification.

(C) Toothpaste improvement display In the toothpaste game according to the present embodiment, for example, a notification for urging improvement of toothpaste is given. For example, if the data processing unit 196 cannot detect the motion of the object region in step S1025, or if the detected motion does not satisfy a predetermined condition, the notification control unit 195 gives a notification prompting improvement of tooth brushing.

Specifically, for example, when the user's hand is stopped, the object area including the toothbrush and the hand does not move. Therefore, in step S1025, the data processing unit 196 does not detect the motion of the object region. When the motion of the object area is not detected by the data processing unit 196, the notification control unit 195 displays a description urging the user to brush the teeth, for example, "Let's brush!" On the display 132.

FIG. 14 is a schematic diagram showing a display example of the display 132 when a description prompting tooth brushing is displayed. In the example shown in FIG. 14, the description of "Let's brush!" Is displayed on the captured image I1. At this time, a predetermined effect indicating tooth brushing does not occur. For example, the effect of radiation such as bubbles and brilliance from the toothbrush, and the effect of whitening teeth do not occur.

Further, for example, when the movement of the user's hand is small, the amount of movement of the object area is also small. At this time, in step S1025, the data processing unit 196 detects the motion of the object region, but determines that the motion amount of this motion has not reached the preset motion amount. When the amount of movement of the motion of the object region detected by the data processing unit 196 does not reach a predetermined value, the notification control unit 195 displays a description that encourages improvement of tooth brushing, for example, "Let's brush more!" Display on 132.

Further, for example, when the movement of the user's hand is slow or the regular repetitive movement is not possible, the data processing unit 196 detects the movement of the object region in step S1025, but the speed of this movement is preset. Judge that the speed has not been reached. When the speed of movement of the object region detected by the data processing unit 196 does not reach a predetermined speed, the notification control unit 195 displays a description prompting improvement of tooth brushing, for example, "Let's move more!" On the display 132. Let me.

Further, for example, when the movement of the user's hand is too fast, the data processing unit 196 detects the movement of the object region in step S1025, but determines that the speed of this movement exceeds the preset speed. .. When the speed of movement of the object region detected by the data processing unit 196 exceeds a predetermined speed, the notification control unit 195 displays a description for promoting improvement of tooth brushing, for example, "Let's brush a little more slowly!" On the display 132. Let me.

The notification prompting the improvement of tooth brushing may be a voice notification.

(D) Toothpaste praise display In the toothpaste game according to the present embodiment, for example, a notification to praise the toothpaste is given. For example, when the motion of the object region detected by the data processing unit 196 in step S1025 satisfies a predetermined condition, the notification control unit 195 gives a notification praising the tooth brushing.

Specifically, for example, in step S1025, the data processing unit 196 continuously detects the regular motion of the object region for a preset number of frames (over a certain period of time). When the data processing unit 196 continuously detects the regular motion of the object area for a preset number of frames, the notification control unit 195 gives a statement praising the tooth brushing, for example, "well brushed!". It is displayed on the display 132.

FIG. 15 is a schematic diagram showing a display example of the display 132 when a description praising tooth brushing is displayed. In the example shown in FIG. 15, the photographed image I1 is displayed with the description "You are well-painted!". The notification praising the toothpaste may be a voice notification.

(E) Toothbrush replacement display In the toothbrushing game according to the present embodiment, for example, a notification prompting the toothbrush replacement is given. For example, the game control unit 193 reads out the date registered in the item “exchange date” from the user information 181 before the play of the tooth brushing game is started. Before the start of playing the toothpaste game, for example, between the time when the user selects the application for the toothpaste game and before the start screen is displayed, or after the start screen is displayed, the start button is pressed. Represents the period before the toothpaste.

The game control unit 193 determines whether or not a predetermined number of days have elapsed from the read date to the current date. If it has passed, the notification control unit 195 gives a notification prompting the replacement of the toothbrush. The predetermined number of days is the number of days for which it is recommended to replace the toothbrush, for example, one month. Specifically, when a predetermined number of days have elapsed from the read date to the current date, the notification control unit 195 displays a description prompting the replacement of the toothbrush on the display 132.

FIG. 16 is a schematic diagram showing a display example of the display 132 when a description prompting the replacement of the toothbrush is displayed. In the example shown in FIG. 16, a description prompting the replacement of the toothbrush is displayed on the upper part of the start button 1324, for example, a description "Let's take a brush!" Is displayed. The notification prompting the replacement of the toothbrush may be a voice notification.

Subsequently, the update of the date registered in the item "exchange date" of the user information 181 will be described. When the start button 1324 is pressed after the user has replaced the toothbrush, the process shown in FIG. 6 is executed. The terminal device 10 executes the tooth brushing process shown in FIG. 7 in step S102.

FIG. 17 is a flowchart showing an example of the operation of the data processing unit 196 when updating the item “exchange date”. FIG. 17 shows a flowchart of an operation in which the data processing unit 196 updates the item “exchange date” in step S1025 shown in FIG. 7. In the description of FIG. 17, it is assumed that the data processing unit 196 stores information about the object region in which regular motion is detected in the storage unit 180 during the previous play of the tooth brushing game. The information about the object area is information that can identify the toothbrush included in the object area, and is, for example, pixel information of the object area.

When the data processing unit 196 detects the motion of the object region in step S1025 of FIG. 7, the data processing unit 196 compares the object region where the motion is detected with the object region stored during the previous play of the tooth brushing game (step S10251). The data processing unit 196 determines whether or not the object area matches the previous time and the current time based on the color, shape, and the like of the bar portion of the object area (step S10252). If they match (Yes in step S10252), the data processing unit 196 ends the processing. If they do not match (No in step S10252), the data processing unit 196 writes the current date in the item "exchange date" of the user information 181 (step S10253).

Here, when the terminal device 10 detects that the toothbrush has been replaced based on the captured image taken by the user without inputting whether or not the user has replaced the toothbrush, the terminal device 10 has a fixed period of time. The user may be rewarded for playing the game by replacing the used toothbrush (that is, after replacing the toothbrush at the time of replacement). For example, when the terminal device 10 identifies whether or not the user's toothbrush is favored in the toothbrushing game by image recognition, the terminal device 10 "replaces the toothbrush" during the toothbrushing game or when the toothbrushing game ends. It may be possible to notify the user of "that" to add a game object or the like, or to make a specific game unit playable. For example, when the toothbrush is used and replaced for a certain period of time, the user may be notified that a special game unit can be played. As a result, the user can feel more enjoyable brushing the toothbrush after replacing the toothbrush at the timing when it should be replaced, so that it is possible to support the user to form a habit of brushing the tooth.

(F) Use of Game Items In the tooth brushing game according to the present embodiment, for example, the user can use the owned game items in the tooth brushing game.

For example, the user can display his / her headgear so as to cover his / her own head included in the captured image while playing the toothpaste game.

Specifically, for example, the notification control unit 195 displays an icon for accepting the use of the item on the start screen of the tooth brushing game. When the icon is pressed by the user, the notification control unit 195 displays a list of the headgear held on the display 132.

The user selects a desired headgear from the headgear displayed on the display 132.

When the headgear is selected by the user, the notification control unit 195 displays the start screen of the tooth brushing game on the display 132. When the start button is pressed by the user, the terminal device 10 executes the process shown in FIG. In step S102 of FIG. 6, the control unit 190 executes the tooth brushing process shown in FIG. 7.

When the head is detected in step S1024 of FIG. 7, the notification control unit 195 superimposes and displays an image of the headgear selected by the user on the detected head. Even if the user displayed on the display 132 moves, the notification control unit 195 follows the user's head and superimposes the headgear on the user's head.

FIG. 18 is a schematic diagram showing an example of a display screen of a user who wears a headgear. According to the example shown in FIG. 18, the user wears a headgear 1325 on which the character's face is designed.

The facial expression of the headgear changes depending on how well you brush your teeth. For example, when the motion of the object region detected by the data processing unit 196 in step S1025 of FIG. 7 satisfies a predetermined condition, the notification control unit 195 changes the facial expression of the headgear.

Specifically, for example, in step S1025, the data processing unit 196 continuously detects the regular motion of the object region for a preset number of frames. When the data processing unit 196 continuously detects the regular motion of the object region for a preset number of frames, the notification control unit 195 makes the facial expression of the headgear smile. Further, if the data processing unit 196 does not continuously detect the regular motion of the object region for a preset number of frames, the notification control unit 195 makes the facial expression of the headgear a sad face.

Further, for example, the user can attach the holding sticker to the still image. The still image includes a still image taken by the user's operation or a still image automatically taken during the play of the tooth brushing game. Here, the operation of attaching a sticker to a still image automatically taken while playing a toothpaste game will be described.

For example, when the game control unit 193 determines that a predetermined event flag is set, the game control unit 193 stores the still image in the storage unit 180. The predetermined event flag is a notification praising tooth brushing when a predetermined condition is satisfied, for example, when the data processing unit 196 first detects a regular movement of the object area, or when a boss character occurs. It stands when the boss character is notified of the improvement of tooth brushing, or when the effect of tooth brushing appears on the boss character. The effect of brushing teeth on the boss character includes, for example, that all the stains on the boss character have been removed.

More specifically, for example, when the event flag is set and a predetermined second elapses, the game control unit 193 stores the frame at that time in the storage unit 180 as a still image. The notification control unit 195 may display the timing of shooting on the display 132 so that the user can recognize the timing of storing the still image. The game control unit 193 stores the game image at that time in the storage unit 180 together with the still image. At this time, the game image may include, for example, a predetermined notification (for example, a notification praising the toothpaste such as "You are brushing well!").

When the game control unit 193 finishes capturing the boss character regardless of whether the capture is successful or not, a plurality of still images stored during play and a plurality of game images corresponding to these still images are used. Create a captured image. In the present embodiment, the captured image represents an image captured (saved) of the screen displayed on the display 132 during the play of the tooth brushing game. The notification control unit 195 displays the created captured image on the display 132.

The user selects at least one captured image to be displayed on the display 132. The user superimposes the held sticker on an arbitrary position of the selected captured image. The user stores the captured image with the sticker attached in the storage unit 180. At this time, the game control unit 193 deletes the still image and the game image not selected by the user from the storage unit 180.

Returning to the description of FIG. When the tooth brushing process of step S102 is completed, the transmission / reception unit 192 of the terminal device 10 transmits a second notification indicating that the tooth brushing game is completed to the server 20. The second notification includes a play ID and a user ID. The terminal device 10 transmits, for example, the updated information among the user information 181 to the server 20 together with the second notification. For example, the terminal device 10 transmits, for example, information about the item "exchange date", the item "possessed item", and the item "captured character" to the server 20 as the information updated in the user information 181.

Further, the terminal device 10 transmits information about tooth brushing to the server 20. The information for toothpaste includes arbitrary information such as the detection status of the motion of the object area by the data processing unit 196, the cleaning status of the boss character, the number of times the toothpaste is praised, and the evaluation of the toothpaste calculated by the game control unit 193. ..

Upon receiving the second notification, the first management module 2044 of the server 20 updates the user information database 281. Specifically, when the first management module 2044 receives the second notification, the first management module 2044 updates the information stored in the user information database 281 with the information updated in the user information 181.

Upon receiving the second notification, the second management module 2045 of the server 20 updates the log information database 282. Specifically, when the second management module 2045 receives the second notification, the second management module 2045 adds a new log for the finished play to the log information database 282.

The information stored in the user information database 281 and the log information database 282 is used at the time of various analyzes. For example, it is possible to analyze the toothbrush replacement time based on the item "replacement date" in the user information database 281. Further, it is possible to analyze the distribution of the play time for each age based on the item "age" and the item "play time" of the user information database 281. In addition, it will be possible to introduce a limited character and analyze the effect of the campaign based on the possession distribution of this limited character.

As described above, in the above embodiment, the game control unit 193 of the terminal device 10 starts timing the time for defining one play of the tooth brushing game in response to the start of the tooth brushing game. The shooting control unit 194 controls the camera 160 and shoots the user. The data processing unit 196 detects the user's face area from the captured image, and detects the facial organs from the detected face area to set the area around the mouth. The data processing unit 196 detects the movement of an object including the toothbrush and the hand in the area around the mouth. The game control unit 193 controls the tooth brushing game based on the detected movement of the object. The game control unit 193 does not stop the timekeeping when the movement of the object is not detected, and stops the timekeeping when the area around the mouth is not set. This makes it possible, for example, to stop timing by removing the user from the screen if there is a reason to temporarily stop brushing. That is, it is possible to stop the timing of the toothbrushing game when giving the toothbrush to the parent to finish the toothbrushing or when the toothbrush is taken out of the mouth for saliva discharge. This makes it possible to prevent the tooth brushing from becoming incomplete due to being overly distracted by the progress of the tooth brushing game.

Therefore, according to the tooth brushing game according to the present embodiment, it is possible to prevent the user from hindering the tooth brushing while maintaining the interest.

Further, in the above embodiment, the data processing unit 196 detects a face region from an image taken by the user, and detects a facial organ different from the user's mouth from the detected face region to obtain a region around the mouth. Set. If the area around the mouth is not set, the face area is not detected. As a result, the user can stop the timekeeping of the play time by turning away from the camera 160.

Further, in the above embodiment, the game control unit 193 restarts the timing when the face region is detected in the state where the face region is not detected. As a result, even if the tooth brushing cannot be continued due to temporary circumstances, the tooth brushing game can be restarted as soon as the circumstances are resolved. This eliminates the need to ask parents to brush their teeth or to put up with the toothbrush out of their mouth to drain saliva.

Further, in the above embodiment, when the face area cannot be detected by the data processing unit 196, the notification control unit 195 notifies the user to show the face. This allows the user to easily recognize that the toothpaste game does not proceed because the face is not photographed correctly.

Further, in the above embodiment, when the movement of the object cannot be detected by the data processing unit 196, or when the detected movement does not satisfy a predetermined condition, the notification control unit 195 notifies the user to improve the tooth brushing. I try to do it. This allows the user to understand how to move their hands to improve brushing.

Further, in the above embodiment, the data processing unit 196 compares the past image of the object with the current image, and if the images do not match, the data processing unit 196 notifies the user to replace the toothbrush. There is. This makes it possible to encourage the replacement of the toothbrush at an appropriate timing. In addition, since it is detected by image analysis, it is possible to encourage replacement of the toothbrush regardless of the type of toothbrush.

Further, in the above embodiment, the game control unit 193 changes the acquisition rate of the privilege based on the detection result of the movement of the object. This makes it possible to enhance the interest.

Further, in the above embodiment, the game control unit 193 increases the acquisition rate when the movement of the object is continuously detected, and decreases the acquisition rate when the movement of the object is not continuously detected. This makes it possible to prevent people from pulling out and brushing their teeth.

Further, in the above embodiment, the game control unit 193 launches the tooth brushing game at a preset time. This makes it possible to include toothpaste in the rhythm of life. It also makes it possible to prevent forgetting to brush your teeth.

Further, in the above embodiment, the data processing unit 196 detects the head region as a facial organ. The notification control unit 195 superimposes a headgear on which the face of the game character is designed on the head region, and changes the facial expression of the headgear based on the detection result of the movement of the object. This makes it possible to know from the facial expression of the headgear that the tooth brushing is working well. Also, the user will do his best to brush his teeth because he does not want to see a good facial expression or a sad facial expression.

Further, in the above embodiment, the game control unit 193 determines that one of the plurality of predetermined conditions predetermined in the tooth brushing game is satisfied. The game control unit 193 displays a notification associated with the satisfied condition on the game screen. The game control unit 193 captures the game screen including the notification. The game control unit 193 displays at least one of the captured images created by capturing the game screen when one play of the tooth brushing game is completed. This makes it possible to collect images taken during the toothpaste game. In other words, the interest will be improved. In addition, since the image during the tooth brushing game is obtained, it is possible to confirm that the user's parents were seriously brushing their teeth.

Here, the terminal device 10 may be able to execute the tooth brushing game based on the game program in either the first mode or the second mode. For example, the first mode is a mode for the child to play a tooth brushing game, and the second mode is a mode for the parent to refer to a record of the child brushing teeth. In the second mode, the terminal device 10 can refer to the log (date, game play result (item "evaluation", etc. of the log information database 282), etc.) in which the user (child) has played the tooth brushing game in chronological order. May be good. Further, it may be comparable to the tooth brushing log of another user, and the terminal device 10 masks the information of the other user (or does not mask the other user who has a friendship with the user). ), The toothpaste log of the other user may be displayed in comparison with the toothpaste log of the user. This can help multiple users to try to form a tooth brushing habit.

Further, it may be possible to register information on the area where the user is brushing teeth to the user. When the user performs an operation to register the area, for example, the user information database 281 of the server 20 holds the information of the user's area.

Further, in the above embodiment, the event flag is set when the movement of the object is first detected, when the boss character is generated, when the notification praising the tooth brushing is given, or when the notification prompting the improvement of the tooth brushing is given. Or, it stands when the effect of brushing teeth appears on the boss character. This makes it possible to acquire images in various cases.

Further, in the above embodiment, the notification control unit 195 notifies the user of a comment about tooth brushing when one play of the tooth brushing game is completed. This allows the user to obtain useful information about toothpaste.

Further, in the above embodiment, the transmission / reception unit 192 transmits information about the tooth brushing game to the server. This makes it possible to perform various analyzes based on the information obtained in the tooth brushing game.

Further, in the above embodiment, the information regarding the tooth brushing game includes the age of the user, the time limit for playing once, the time for notifying the start of the tooth brushing game, the toothbrush replacement date, the tooth brushing game implementation date, and the start of the tooth brushing game. Includes time or evaluation of toothpaste games. This makes it possible to perform various analyzes based on the information obtained in the tooth brushing game.

The present embodiment is not limited to the above.
For example, in the above embodiment, the case where the captured image I1 is displayed in the upper area of the display 132 and the game image I2 is displayed in the lower area of the display 132 has been described as an example, but the arrangement of the images is described. Not limited to this. The captured image may be displayed in the lower area of the display 132 and the game image may be displayed in the upper area of the display 132.

Further, in the above embodiment, the case where a plurality of teeth are displayed in the game image has been described as an example, but what is displayed in the game image is not limited to the teeth.

Further, in the above embodiment, the case where the toothbrush is a general toothbrush having no additional function has been described as an example. However, the toothbrush may be provided with a mechanism for identifying the toothbrush when prompting the replacement of the toothbrush. For example, the toothbrush may be marked with an optical mark. The optical mark is a mark in which identification information of a toothbrush or the like is encoded, for example, a barcode, a one-dimensional pixel code, a two-dimensional pixel code, or the like. For example, when the tooth brushing game is played and the optical mark is photographed by the camera 160, the data processing unit 196 decodes the optical mark and acquires identification information. When the information regarding the identification information is not stored in the storage unit 180, the data processing unit 196 stores the acquired identification information in the storage unit 180.

When the identification information is stored in the storage unit 180, the data processing unit 196 executes the same processing as the processing shown in FIG. That is, the data processing unit 196 compares the acquired identification information with the identification information stored during the previous play of the tooth brushing game. The data processing unit 196 determines whether or not the identification information matches the previous time and the current time. If they match, the data processing unit 196 ends the processing. If they do not match, the data processing unit 196 writes the current date in the item "exchange date" of the user information 181.

Further, the toothbrush may be attached with a small electronic tag (for example, an IC tag, an RFID tag, or a wireless tag) that is driven by receiving radio waves. For example, toothbrush identification information and the like are stored in the electronic tag. At this time, the terminal device 10 has a function of, for example, a tag reader. For example, when the tooth brushing game is played, the data processing unit 196 acquires identification information from the electronic tag by the tag reader function. When the information regarding the identification information is not stored in the storage unit 180, the data processing unit 196 stores the acquired identification information in the storage unit 180. The optical mark or electronic tag is not used to detect the movement of the toothbrush.

Further, in the above embodiment, the case where the information of one user is stored in the user information 181 of the storage unit 180 has been described as an example. However, as the information stored in the user information 181 of the storage unit 180, information of a plurality of users such as, for example, a few people may be stored. When information about a plurality of users is stored in the user information 181. For example, on the start screen of the tooth brushing game, a selection screen for selecting the user information is displayed. After selecting its own information on the selection screen, the user presses the start button from the start screen to start the game. This makes it possible to update new information with respect to the information that oneself has played so far.

When executing the toothpaste game with its own information, it is not always necessary to select the user's information from the selection screen. For example, each piece of information in the user information 181 may store information about a face that can identify the user. The shooting control unit 194 controls the camera 160, for example, on the start screen, and shoots the user's face. For example, on the start screen, the data processing unit 196 collates the face of the user photographed by the camera 160 with the face stored in the user information 181. The game control unit 193 identifies information in which face information matching the user's face is stored, and starts the tooth brushing game using the identified information.

It should be noted that what is stored in each information in the user information 181 is not limited to the information related to the face. Information about the toothbrush may be stored in each information in the user information 181. The information about the toothbrush may be information determined by image analysis, for example, color or shape, identification information read from an optical mark, or identification information read from a wireless tag. You may. The game control unit 193 identifies the stored information about the toothbrush that matches the toothbrush held by the user from the user information 181 and starts the tooth brushing game using the specified information.

Further, in the above embodiment, the case where the server 20 is provided in the game system 1 has been described as an example, but the number of the servers 20 provided in the game system 1 is not limited to one. A plurality of servers 20 including a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29 are provided, and an input operation reception unit 191, a transmission / reception unit 192, a game control unit 193, and a shooting control unit are provided. The 194, the notification control unit 195, and the data processing unit 196 may be realized by any of the servers 20.

Although some embodiments of the present disclosure have been described above, these embodiments can be implemented in various other embodiments, and various omissions, replacements, and modifications are made without departing from the gist of the invention. It can be performed. These embodiments and variations thereof shall be included in the scope of the invention described in the claims and the equivalent scope thereof, as well as in the scope and gist of the invention.
<Additional Notes>
The matters described in each of the above embodiments are added below.

(Appendix 1)
A game program for causing a computer having a processor 19 and a memory 15 to execute the game program.
In response to the start of the toothpaste game, the step (step S1021) of starting the timekeeping for defining one play of the toothpaste game, and
The step of shooting the user (step S1021) and
A step of setting an area around the mouth by detecting a user's face area from the captured image and detecting a facial organ from the detected face area (step S1023, step S1024).
In the area around the mouth, the step of detecting the movement of an object including the toothbrush and the hand (step S1025), and
A step of controlling the tooth brushing game based on the detected movement of the object (step S1026), and
A game program that executes a step to stop the timekeeping when the area around the mouth is not set, without stopping the timekeeping when the movement of the object is not detected.

(Appendix 2)
In the step of setting the area around the mouth, the area around the mouth is set by detecting the face area from the captured image and detecting the facial organs different from the user's mouth from the detected face area.
The game program according to the description (Appendix 1), wherein the area around the mouth is not set in the step of stopping the timekeeping, that is, the face area is not detected.

(Appendix 3)
The game program according to (Appendix 1) or (Appendix 2), wherein when the face area is detected in a state where the face area is not detected, the processor is made to execute a step of restarting the time measurement.

(Appendix 4)
The game program according to any one of (Appendix 1) to (Appendix 3), which notifies the user to show his / her face when the face area cannot be detected in the step of setting the area around the mouth.

(Appendix 5)
In the step of detecting the movement of the object, if the movement of the object cannot be detected, or if the detected movement does not meet the predetermined conditions, the user is notified to urge the improvement of tooth brushing (Appendix 1) to (Appendix 4). ) The game program described in any of.

(Appendix 6)
A step of comparing a past image of the object with a current image,
The game program according to any one of (Appendix 1) to (Appendix 5), which causes the processor to execute a step of notifying the user to replace the toothbrush when the images do not match.

(Appendix 7)
The game program according to any one of (Appendix 1) to (Appendix 6), in which the acquisition rate of benefits is changed based on the detection result of the movement of an object in the step of controlling the tooth brushing game.

(Appendix 8)
The game program according to (Appendix 7), which increases the acquisition rate when the movement of an object is continuously detected and decreases the acquisition rate when the movement of the object is not continuously detected in the step of controlling the tooth brushing game.

(Appendix 9)
The game program according to any one of (Appendix 1) to (Appendix 8), which causes a processor to execute a step of starting a tooth brushing game at a preset time.

(Appendix 10)
When the head area is detected as a facial organ, the step of superimposing a headgear on which the face of the game character is designed is superimposed on the head area.
The game program according to the description (Appendix 2), wherein the processor executes a step of changing the facial expression of the headgear based on the detection result of the movement of the object.

(Appendix 11)
A step for determining the fulfillment of one of a plurality of predetermined conditions in a tooth brushing game, and
A step to display the notification associated with the satisfied condition on the game screen,
Steps to capture the game screen with notifications,
Described in any one of (Appendix 1) to (Appendix 10), which causes the processor to execute a step of displaying at least one of the captured images created by capturing the game screen when one play of the tooth brushing game is completed. Game program.

(Appendix 12)
Steps to determine the end of one play of the toothpaste game,
Steps to evaluate 1 play toothpaste,
The game program according to any one of (Appendix 1) to (Appendix 11), which causes the processor to execute a step of notifying the user of a comment about tooth brushing set based on the evaluation.

(Appendix 13)
The game program according to any one of (Appendix 1) to (Appendix 12), which causes a processor to execute a step of transmitting information about a toothpaste game to a server.

(Appendix 14)
Information about the toothpaste game can be found on the user's age, time limit for playing once, time to notify the start of the toothpaste game, toothbrush replacement date, toothbrushing game execution date, toothpaste game start time, or toothpaste game rating. The game program described in (Appendix 13).

(Appendix 15)
A method executed by a computer equipped with a processor and a memory, wherein the processor reads and executes a game program stored in the memory.
In response to the start of the toothpaste game, the step of starting the timekeeping that defines one play of the toothpaste game, and
Steps to shoot the user and
A step to set the area around the mouth by detecting the user's face area from the captured image and detecting the facial organs from the detected face area.
Steps to detect the movement of objects, including the toothbrush and the hand, in the area around the mouth,
Steps to control the toothpaste game based on the detected movement of the object,
A method of performing a step of stopping the timekeeping when the area around the mouth is not set, without stopping the timekeeping when the movement of the object is not detected.

(Appendix 16)
An information processing device including a control unit and a storage unit.
By operating the control unit based on the game program stored in the storage unit,
In response to the start of the toothpaste game, the step of starting the timekeeping that defines one play of the toothpaste game, and
Steps to shoot the user and
A step to set the area around the mouth by detecting the user's face area from the captured image and detecting the facial organs from the detected face area.
Steps to detect the movement of objects, including the toothbrush and the hand, in the area around the mouth,
Steps to control the toothpaste game based on the detected movement of the object,
An information processing device that executes the step of stopping the timekeeping when the area around the mouth is not set, without stopping the timekeeping when the movement of the object is not detected.

(Appendix 17)
A system with multiple information processing devices
By operating one of the multiple information processing devices based on the game program,
In response to the start of the toothpaste game, the step of starting the timekeeping that defines one play of the toothpaste game, and
Steps to shoot the user and
A step to set the area around the mouth by detecting the user's face area from the captured image and detecting the facial organs from the detected face area.
Steps to detect the movement of objects, including the toothbrush and the hand, in the area around the mouth,
Steps to control the toothpaste game based on the detected movement of the object,
A system that performs steps to stop the timekeeping when the area around the mouth is not set, without stopping the timekeeping when the movement of the object is not detected.

1 ... Game system 10, 10A, 10B ... Terminal devices 111, 112 ... Antenna 121 ... First wireless communication unit 122 ... Second wireless communication unit 13 ... Input device 130 ... Operation reception unit 131 ... Touch sensitive device 132 ... Display 1321 ... Remaining time 1323 ... Capture tool 1324 ... Start button 1325 ... Headgear 14 ... Output device 140 ... Voice processing unit 141 ... Mike 142 ... Speaker 15 ... Memory 150 ... Position information sensor 16 ... Storage unit 160 ... Camera 170 ... Motion sensor 180 ... Storage unit 181 ... User information 182 ... Comment information 19 ... Processor 190 ... Control unit 191 ... Input operation reception unit 192 ... Transmission / reception unit 193 ... Game control unit 194 ... Shooting control unit 195 ... Notification control unit 196 ... Data processing unit 20 ... Server 201 ... Communication unit 202 ... Storage unit 203 ... Control unit 2041 ... Operation content acquisition module 2042 ... Reception control module 2043 ... Transmission control module 2044 ... First management module 2045 ... Second management module 25 ... Memory 26 ... Storage 281 ... User Information database 282 ... Log information database 29 ... Processor 80 ... Network 81 ... Radio base station

Claims (16)

A game program for being executed by a computer including a processor and a memory, wherein the game program is executed by the processor.
Before the start of the toothpaste game, a step of accepting the user to select one of the multiple game items owned by the user,
In response to the user's operation, the step of starting the tooth brushing game and
Steps to receive information taken by the user,
A step of detecting the user's face area and head area from the image based on the information, and
A step of displaying the game item in association with the head area during the tooth brushing game.
During the tooth brushing game, a step of acquiring a plurality of still images including the face area of the user without the user specifying an acquisition timing, and
After the end of one play of the tooth brushing game, a step of storing a captured image selected by the user among a plurality of captured images including the game item created based on the plurality of still images in the memory. A game program to be executed. A game program for being executed by a computer including a processor and a memory, wherein the game program is executed by the processor.
Before the start of the toothpaste game, a step of accepting the user to select one of the multiple game items owned by the user,
In response to the user's operation, the step of starting the tooth brushing game and
Steps to receive information taken by the user,
A step of detecting the user's face area and head area from the image based on the information, and
A step of displaying the game item in association with the head area during the tooth brushing game.
During the tooth brushing game, a step of acquiring a plurality of still images including the face area of the user in a state where the user cannot recognize the acquisition timing, and
After the end of one play of the tooth brushing game, a step of storing a captured image selected by the user among a plurality of captured images including the game item created based on the plurality of still images in the memory.
A game program that runs. The game program according to claim 1 or 2, which changes the display mode of the game item in the step of displaying the game item. From the image based on the information, the processor is made to perform a step of determining whether or not the tooth is brushed.
The game program according to claim 1 or 2 , wherein in the step of displaying the game item, the display mode of the game item is changed based on the determination result of the tooth brushing. The processor is made to execute a step of receiving an instruction from the user to superimpose a game object on the captured image selected by the user.
The game program according to claim 3 or 4 , wherein in the step of storing the captured image, the captured image in which the game object is superimposed is stored in the memory. The game program according to any one of claims 1 to 5 , wherein in the step of storing the captured image, the captured image not selected by the user is deleted from the memory. The game program according to any one of claims 1 to 6, wherein the processor executes a step of launching the tooth brushing game at a preset time. The step of determining the end of one play of the tooth brushing game and
The game program according to any one of claims 1 to 7, wherein the processor executes a step of notifying the user of a comment about the tooth brushing game. The game program according to any one of claims 1 to 8, wherein the processor executes a step of transmitting information about the toothpaste game to a server. The information about the toothpaste game includes the age of the user, the time limit for playing once, the time for notifying the start of the toothpaste game, the toothbrush replacement date, the toothbrushing game execution date, the toothpaste game start time, or the toothpaste game. The game program according to claim 9, which includes evaluation. A method executed by a computer including a processor and a memory, wherein the processor reads and executes a game program stored in the memory.
Before the start of the toothpaste game, a step of accepting the user to select one of the multiple game items owned by the user,
In response to the user's operation, the step of starting the tooth brushing game and
Steps to receive information taken by the user,
A step of detecting the user's face area and head area from the image based on the information, and
A step of displaying the game item in association with the head area during the tooth brushing game.
During the tooth brushing game, a step of acquiring a plurality of still images including the face area of the user without the user specifying an acquisition timing, and
After the end of one play of the tooth brushing game, a step of storing a captured image selected by the user among a plurality of captured images including the game item created based on the plurality of still images in the memory. way to execute. An information processing device including a control unit and a storage unit.
By operating the control unit based on the game program stored in the storage unit, the control unit operates.
Before the start of the toothpaste game, a step of accepting the user to select one of the multiple game items owned by the user,
In response to the user's operation, the step of starting the tooth brushing game and
Steps to receive information taken by the user,
A step of detecting the user's face area and head area from the image based on the information, and
A step of displaying the game item in association with the head area during the tooth brushing game.
During the tooth brushing game, a step of acquiring a plurality of still images including the face area of the user without the user specifying an acquisition timing, and
A step of storing a captured image selected by the user among a plurality of captured images including the game item created based on the plurality of still images after one play of the tooth brushing game is stored in the storage unit. Information processing device that executes. A system with multiple information processing devices
When any of the plurality of information processing devices operates based on the game program,
Before the start of the toothpaste game, a step of accepting the user to select one of the multiple game items owned by the user,
In response to the user's operation, the step of starting the tooth brushing game and
Steps to receive information taken by the user,
A step of detecting the user's face area and head area from the image based on the information, and
A step of displaying the game item in association with the head area during the tooth brushing game.
During the tooth brushing game, a step of acquiring a plurality of still images including the face area of the user without the user specifying an acquisition timing, and
A system that executes a step of storing a captured image selected by the user among a plurality of captured images including the game item created based on the plurality of still images after one play of the tooth brushing game is completed. .. A method executed by a computer including a processor and a memory, wherein the processor reads and executes a game program stored in the memory.
Before the start of the toothpaste game, a step of accepting the user to select one of the multiple game items owned by the user,
In response to the user's operation, the step of starting the tooth brushing game and
Steps to receive information taken by the user,
A step of detecting the user's face area and head area from the image based on the information, and
A step of displaying the game item in association with the head area during the tooth brushing game.
During the tooth brushing game, a step of acquiring a plurality of still images including the face area of the user in a state where the user cannot recognize the acquisition timing, and
After the end of one play of the tooth brushing game, a step of storing a captured image selected by the user among a plurality of captured images including the game item created based on the plurality of still images in the memory.
How to run. An information processing device including a control unit and a storage unit.
By operating the control unit based on the game program stored in the storage unit, the control unit operates.
Before the start of the toothpaste game, a step of accepting the user to select one of the multiple game items owned by the user,
In response to the user's operation, the step of starting the tooth brushing game and
Steps to receive information taken by the user,
A step of detecting the user's face area and head area from the image based on the information, and
A step of displaying the game item in association with the head area during the tooth brushing game.
During the tooth brushing game, a step of acquiring a plurality of still images including the face area of the user in a state where the user cannot recognize the acquisition timing, and
After the end of one play of the tooth brushing game, a step of storing a captured image selected by the user among a plurality of captured images including the game item created based on the plurality of still images in the storage unit.
Information processing device that executes. A system with multiple information processing devices
When any of the plurality of information processing devices operates based on the game program,
Before the start of the toothpaste game, a step of accepting the user to select one of the multiple game items owned by the user,
In response to the user's operation, the step of starting the tooth brushing game and
Steps to receive information taken by the user,
A step of detecting the user's face area and head area from the image based on the information, and
A step of displaying the game item in association with the head area during the tooth brushing game.
During the tooth brushing game, a step of acquiring a plurality of still images including the face area of the user in a state where the user cannot recognize the acquisition timing, and
A step of storing a captured image selected by the user from a plurality of captured images including the game item created based on the plurality of still images after one play of the tooth brushing game is completed.
System to run.

Images