JP7375354B2 - User terminal, server, character exchange system, character exchange method, and program - Google Patents

Description

The present invention relates to user terminals and the like for interacting with characters in a virtual world or the like in the real world.

In the field of subculture content such as anime, manga, and games, traditional users have limited themselves to passively enjoying the content, simply watching anime, reading manga, or manipulating characters in games. Ta. However, at present, there is a strong need for users to have some kind of active interaction with characters in these virtual worlds and the like in the real world.

As an example, many games using AR (Augmented Reality) are currently being enjoyed, and users can interact with game characters that appear in the real world in various ways. Furthermore, Patent Documents 1 and 2 disclose stationary display devices that display characters three-dimensionally and allow users to enjoy dialogue between them and the characters.

Japanese Patent Application Publication No. 2018-146965 Japanese Patent Application Publication No. 2018-14575

However, in the case of games that utilize AR, the interaction between the user and the character is usually within the scope of the game's system. On the other hand, the display devices of Patent Documents 1 and 2 allow everyday interactions such as "good morning" and "welcome home" to be performed with the three-dimensionally displayed characters.

By the way, in order to have interactions that make characters feel closer to the characters in the virtual world, etc., it is also effective to encourage users to take active actions in the interaction scenes and to give the users a feeling that they are doing something for the characters. It is thought that.

In this regard, in the display devices of Patent Documents 1 and 2, the interaction is limited to the character responding when the user speaks to the character, and by encouraging the user to take more active actions, the character becomes more familiar to the user. There is a possibility of realizing an interaction that can be felt.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a user terminal, etc. that enables interaction that allows users to feel closer to characters in a virtual world, etc.

A first invention for solving the above-mentioned problems includes: a notification receiving unit that receives an event start notification from a server when the event start time arrives; a setting unit that accepts setting of the start time; and a notification receiving unit. After receiving an event start notification, an input receiving means receives an input of a user's face image, a playback means plays back voice data of a character instructing the user's actions by inputting the face image, and a playback means takes a picture. The user terminal is characterized in that it has a photographing means for transmitting the image to the server, and an image transmitting means for transmitting the image to the server in order to determine whether the image corresponds to the instruction.

The user terminal of the present invention plays back voice data of a character that instructs the user's actions, and allows the user to take an image corresponding to the instructions, thereby making the user feel as if they have done something for the character. can be obtained. In this way, the present invention can provide a service that allows users to interact with characters in a way that makes them feel more familiar by encouraging users to take active actions such as taking images in response to instructions.

Preferably, the reproduction means reproduces voice data of the character that instructs the user's actions, and then reproduces other voice data of the character when the image corresponds to the instruction.
This makes it possible to create an interaction scene in which the user's actions receive a response such as gratitude from the character. By doing something for the character, the user can feel as if the character has thanked the character for the action, and can feel closer to the character.

Preferably, the audio data is recorded using a stereophonic sound system.
This allows the user to feel as if the character is actually beside him or her .

It is also desirable that the user terminal has audio receiving means for receiving audio data to be reproduced by the reproducing means from the server.
Thereby, services can be provided to users through a character exchange system consisting of user terminals and servers.

It is also desirable that the instruction is to photograph a target object in a predetermined area.
This can encourage users to go to a predetermined area, leading to the revitalization of that area.

A second invention includes a notification means for transmitting an event start notification to a user terminal when the event start time preset by the user arrives; and after transmitting the event start notification to the user terminal, a face image receiving means that receives an input user's face image from the user terminal; and character voice data that instructs the user's actions when the face image matches the person in the face image registered in advance as user information; audio transmitting means for transmitting an image to the user terminal; image receiving means for receiving an image taken by the user terminal from the user terminal; and determining means for determining whether the image corresponds to the instruction. A server characterized by having the following.
The second invention is a server that provides users with a service that allows them to interact with characters in a way that makes them feel closer to the characters.

It is preferable that whether or not the image corresponds to the instruction is determined based on the degree of similarity between the image and a positive image corresponding to the instruction.
It can be suitably determined whether the image taken by the user corresponds to the instruction, for example, using the degree of similarity between the image and a positive image corresponding to the instruction.

It is preferable that the voice transmitting means transmits other voice data of the character to the user terminal when the image corresponds to the instruction.
This makes it possible to create an interaction scene in which the user's actions receive a response such as gratitude from the character .

A third invention is a character exchange system having a user terminal and a server , wherein the user terminal includes a notification receiving means for receiving an event start notification from the server when the event start time arrives, and setting the start time. setting means for receiving an event start notification; input receiving means for receiving an input of a user's face image after receiving the event start notification by the notification receiving means; The server has a reproduction means for reproducing data, a photographing means for photographing an image, and an image transmission means for transmitting the image to the server, and the server is configured to transmit data when the start time of the event is set in advance by the user. , a notification means for transmitting an event start notification to the user terminal, and a face image receiving means for receiving the user's face image input to the user terminal from the user terminal after transmitting the event start notification to the user terminal. and voice transmitting means for transmitting voice data of a character instructing the user's actions to the user terminal when the face image matches the person in the face image registered in advance as user information; A character exchange system is characterized in that it has a determining means for determining whether or not they correspond.
The third invention is a character interaction system that provides users with a service that allows them to interact with characters in a way that makes them feel closer to the characters.

A fourth invention provides a step in which the user terminal receives settings for an event start time, receives an event start notification from a server at the event start time, and displays a user's face after receiving the event start notification. a step of receiving an input of an image; a step of playing back audio data of a character instructing a user's actions by inputting the face image; a step of photographing an image; and a step of determining whether the image corresponds to the instruction. This character interaction method is characterized by executing the step of transmitting the image to the server in order to determine whether or not the image exists.
The fourth invention is a character interaction method in which a user terminal realizes an interaction in which characters feel closer to each other.

A fifth invention provides a user terminal having a photographing means for photographing an image, a notification receiving means for receiving an event start notification from a server when the event start time arrives, a setting means for receiving the setting of the start time, and a setting means for receiving the setting of the start time; an input receiving means for receiving an input of a user's face image after receiving an event start notification by the notification receiving means ; and a reproduction means for reproducing voice data of a character instructing the user's actions by inputting the face image. , is a program that functions as an image transmitting means for transmitting the image to the server in order to determine whether the image corresponds to the instruction.
A fifth invention is a program for causing a user terminal such as a mobile terminal owned by a user to function as the user terminal of the first invention.

A sixth aspect of the present invention provides a computer with a notification means for transmitting an event start notification to a user terminal when the event start time preset by the user arrives; a face image receiving unit that receives a user's face image inputted into the user terminal from the user terminal; and a character that instructs the user's actions when the face image matches the person in the face image registered in advance as user information. audio transmitting means for transmitting audio data of the user to the user terminal; image receiving means for receiving an image taken by the user terminal from the user terminal; and determining whether or not the image corresponds to the instruction. It is a determination means to perform the function, and a program to perform the function.
A sixth invention is a program for causing a computer to function as a server according to the second invention.

According to the present invention, it is possible to provide a user terminal, etc., which allows users to interact with characters in a virtual world or the like more closely.

1 is a diagram showing a schematic configuration of a character exchange system 1. FIG. FIG. 3 is a diagram showing the hardware configuration of the server 3. FIG. A diagram showing user information 300 and event information 400. Example of exchange time zone 305. FIG. 4 is a diagram showing a hardware configuration of a user terminal 4. FIG. FIG. 2 is a diagram illustrating the functional configuration of the character exchange system 1. 12 is a flowchart showing the procedure for a part of a character exchange event in which a character greets a user. An example of a display on the display unit 45 of the user terminal 4. A diagram illustrating an interaction between character A and user B. 2 is a flowchart showing a procedure for a portion of a character exchange event in which a user acts for a character. An example of a display on the display unit 45 of the user terminal 4. A diagram illustrating an interaction between character A and user B. FIG. 5 is a diagram showing categories of images 506. 2 is a flowchart showing a procedure for a portion of a character exchange event in which a user acts for a character. A diagram illustrating an interaction between character A and user B.

Hereinafter, preferred embodiments of the present invention will be described in detail based on the drawings.

[First embodiment]
(1. Character exchange system 1)
FIG. 1 is a diagram showing a schematic configuration of a character exchange system 1 according to an embodiment of the present invention. The character exchange system 1 provides an event (hereinafter referred to as a character exchange event) in which a user interacts with a character from a virtual world such as a manga or anime in the real world as a service. However, the character to be interacted with is not limited to this, and may be a famous person in the real world, such as a celebrity.

Note that the character exchange events of this embodiment include those in which rewards can be received by clearing event achievement conditions (hereinafter referred to as condition achievement type), and those in which such conditions do not exist (hereinafter referred to as non-condition achievement type). The details of these will be described later.

As shown in FIG. 1, the character exchange system 1 includes a server 3, a user terminal 4, an analysis device 5, and the like.

The server 3 executes various processes in the character exchange system 1. FIG. 2 is a diagram showing the hardware configuration of the server 3, and the server 3 can be realized by a computer configured by connecting a control section 31, a storage section 32, a communication section 33, etc. via a bus. However, the configuration of the server 3 is not limited to this, and can be changed as appropriate. Further, the server 3 may be realized by a plurality of computers.

The control unit 31 includes a CPU, ROM, RAM, and the like. The CPU calls a program stored in the storage unit 32, a storage medium such as a ROM, to a work area on the RAM, and executes the program. ROM is a nonvolatile memory that permanently stores programs such as a boot program and BIOS, data, and the like. The RAM is a volatile memory that temporarily stores programs and data loaded from the storage unit 32, ROM, etc., and includes a work area used by the control unit 31 to perform various processes.

The storage unit 32 is a hard disk drive, a solid state drive, or the like, and stores programs to be executed by the server 3 during processing to be described later, data necessary for program execution, an OS, and the like. These programs and data are read and executed by the control unit 31 as necessary.

The communication unit 33 has a communication control device, a communication port, etc., and mediates communication via a network. The server 3 is communicably connected to a user terminal 4, an analysis device 5, etc. via a network, and is capable of transmitting and receiving various data.

The storage unit 32 of the server 3 stores user information 300 and event information 400.

User information 300 is information regarding users who use this service, and is registered in advance. As shown in FIG. 3A, in this embodiment, as user information 300, a user ID 301, registration information 302, destination 303, face image 304, interaction time period 305, etc. are stored in association with each other.

User ID 301 is identification information that identifies a user. The user ID 301 is issued by the server 3, for example, when the user performs pre-registration to use this service.

Registration information 302 is information that is pre-registered by the user, such as the user's name, address, age, etc.

The destination 303 is information that specifies the destination when the server 3 transmits data related to the character exchange event, and is registered at the time of the user's pre-registration. The destination 303 is, for example, the identification information of the user terminal 4, a telephone number, an e-mail address, etc., but is not limited to these.

The face image 304 is a face image of the user, and for example, when performing pre-registration, the user photographs his own face image with the camera 46 (see FIG. 5) of the user terminal 4, transmits it to the server 3, and registers it.

The interaction time period 305 is a time period in which a character interaction event occurs, and is set by the user at the time of pre-registration, and can be changed and reset later.

FIG. 4 is an example of the interaction time period 305, and in this embodiment, a plurality of non-overlapping segments of the day, ie, three segments of morning, noon, and afternoon, are determined as the time periods in which character interaction events occur. . The user sets the time period for each category, and in the example of FIG. to 12:30, and the afternoon exchange time period 305 is set from 15:00 to 22:00.

Event information 400 is information about a character exchange event, and is recorded in advance in the storage unit 32. As shown in FIG. 3(b), in this embodiment, event information 400 includes a date 401, time zone classification 402, audio data (regular) 403, event achievement condition image 404, audio data (when event is achieved) 405, event Images 406 and the like are stored in association with each other.

Date 401 is the date on which the character exchange event occurs. In this embodiment, it is assumed that an event occurs on each day during a certain period from summer to autumn, for example, and includes time zone classification 402, audio data (regular) 403, event achievement condition image 404, and audio data (when event is achieved) 405. , event image 406, etc. are determined for each day.

The time zone division 402 is a time zone division in which the character exchange event occurs. In this embodiment, as described above, three non-overlapping divisions (morning, noon, and afternoon) are defined in one day.

Audio data (regular) 403 is audio data that is played back on the user terminal 4 during the interaction time period 305 as part of the character interaction event. The audio data 403 is data of the voice uttered by the character, and is predetermined according to the scenario of the event. It is also assumed that the audio data 403 was recorded using a stereophonic method such as binaural recording.

The event achievement condition image 404 is an image used to determine whether the user has cleared the event achievement condition. The details will be described later.

The audio data (at the time of event achievement) 405 is defined separately from the audio data 403 as audio data to be played back on the user terminal 4 of the user who has cleared the event achievement condition. The audio data 405 is data of the voice uttered by the character, as described above, and is predetermined according to the scenario of the event. Also, as described above, recording may be performed using a three-dimensional sound system such as binaural recording.

The event image 406 is image data provided as a reward to a user who has cleared the event achievement conditions, and is transmitted from the server 3 to the user terminal 4 of the user.

As described above, in this embodiment, there are two types of character exchange events: condition-fulfilling type and non-condition-fulfilling type, and the event achievement condition image 404, audio data (when event is achieved) 405, and event image 406 are only for the former. Set.

Returning to the explanation of FIG. The user terminal 4 is a terminal owned by a user who uses the service, and is, for example, a mobile terminal such as a smartphone or a tablet terminal.

FIG. 5 is a diagram showing the hardware configuration of the user terminal 4. As shown in FIG. As the user terminal 4, for example, a computer terminal configured by connecting a control section 41, a storage section 42, a communication section 43, an input section 44, a display section 45, a camera 46, an audio input/output section 47, etc. via a bus is used. However, the configuration of the user terminal 4 is not limited to this, and can be changed as appropriate.

The functions of the control unit 41, storage unit 42, and communication unit 43 of the user terminal 4 are substantially the same as those of the control unit 31, storage unit 32, and communication unit 33 described above, and the storage unit 42 includes an application program dedicated to this service. Stored.

The display section 45 is, for example, a liquid crystal display, and is provided with a touch panel as the input section 44. The camera 46 is, for example, a digital camera equipped with an image sensor such as a CCD or CMOS. The audio input/output unit 47 includes a microphone for inputting audio, a speaker for outputting audio, and the like.

The analysis device 5 executes various analysis processes when implementing a character exchange event. As the analysis device 5, a computer having a configuration similar to that shown in FIG. 2 can be used.

Next, the functional configuration of the character exchange system 1 will be explained with reference to FIG. 6. As shown in FIG. 6, in the character exchange system 1, the server 3 has an audio transmitting means 310, a determining means 320, an image receiving means 330, etc., and the user terminal 4 has an audio receiving means 410, a reproducing means 420, a photographing means 430, etc. , image transmitting means 440, and the like.

The audio transmitting means 310 is used by the control unit 31 of the server 3 to transmit audio data 403 and 405 to the user terminal 4 via the communication unit 33. Among these, the audio data 403 may include instructions for the user's actions, and the audio data 405 is played back when the user captures an image corresponding to the instructions, the details of which will be described later. Furthermore, when transmitting the audio data 405, data such as a bonus image of a character or a coupon may also be transmitted as a reward.

The determining means 320 is for the control unit 31 of the server 3 to determine whether the photographed image corresponds to the instruction in the audio data 403 or not.

The image receiving means 330 receives an image photographed by the user terminal 4 from the user terminal 4 via the communication section 33 in order for the control section 31 of the server 3 to perform the above determination.

The audio receiving means 410 is used by the control unit 41 of the user terminal 4 to receive audio data 403 and 405 from the server 3 via the communication unit 43.

The reproduction means 420 is for the control unit 41 of the user terminal 4 to reproduce the audio data 403 and 405 through the speaker of the audio input/output unit 47.

The photographing means 430 corresponds to the camera 46 of the user terminal 4 and is for photographing images.

The image transmitting means 440 is for the control section 41 of the user terminal 4 to transmit an image photographed by the user terminal 4 to the server 3 via the communication section 43 .

(2. Character exchange event)
Next, a character exchange event (character exchange method) carried out in the character exchange system 1 will be explained. In the character exchange event of this embodiment, characters in the virtual world interact with the user during an interaction time slot 305 preset by the user.

In particular, in this embodiment, a scenario is set on the premise that the user and the character are in a romantic relationship, and in the character exchange event, audio data 403 and 405 predetermined according to the scenario is transmitted to the user terminal 4. will be played. Scenarios include one in which a character meets and greets a user, and one in which the user takes further action for the character after the greeting, and each procedure will be described below.

(2-1. Scenario 1; Character's greeting)
FIG. 7 is a flowchart illustrating the procedure of a part of the character exchange event in which a character greets a user. S101, S106 to S109, and S111 in FIG. 7 are processes executed by the control unit 31 of the server 3 while controlling each unit of the server 3, and other processes are executed by the control unit 41 of the user terminal 4 controlling each unit of the user terminal 4. Control and execute.

As shown in FIG. 7, in this embodiment, when the start time of the preset exchange time period 305 arrives, the server 3 transmits an event start notification to the user terminal 4 (S101).

When the user terminal 4 receives the event start notification (S102), the user terminal 4 displays the notification on the display unit 45 of the user terminal 4 (S103). FIG. 8(a) is an example of this event start notification 501, in which “She has arrived” is displayed on the display unit 45 of the user terminal 4. This assumes that character A has come to meet the user in the scenario, as shown in Figure 9(a), and at this point the user (indicated by B in the diagram) has not yet met the character. do not have. Note that although the event start notification 501 of this embodiment does not involve the reproduction of audio, it may be accompanied by the reproduction of audio such as "My girlfriend has arrived."

The user activates the application by, for example, performing a flick operation on the event start notification 501, and the user terminal 4 causes the display unit 45 to display a selection screen as shown in FIG. 8(b). On the selection screen, a "meet now" button 502 and a "wait" button 503 are displayed. When the user selects the "meet now" button 502 by tapping or the like, the user terminal 4 activates the camera 46 and the user In response to the operation, a face image of the user is photographed by the camera 46, and its input is accepted (S104). Note that if the user selects the "wait" button 503, the screen returns to the previous screen shown in FIG. 8(a).

Here, the reason why the user takes a picture of his own face and inputs it into the user terminal 4 is to show the user's face to the character in the scenario, that is, as shown in FIG. The idea is to go and meet face-to-face.

The user terminal 4 transmits the facial image to the server 3 (S105). When the server 3 receives the user's face image (S106), the server 3 determines whether or not the received face image matches the person in the face image 304 (see FIG. 3(a)) of the user information 300 as authentication based on the face image. (S107). In S107, for example, the server 3 sends both face images to the analysis device 5 to request analysis, the analysis device 5 determines whether or not the persons in both face images match, and the determination result is sent from the analysis device 5. Receive. Various known methods can be used for the determination method, and there is no particular limitation.

If the persons in both face images do not match, the server 3 determines that the authentication has failed (S108; NG), and transmits a notification to the user terminal 4 that the authentication has failed (S109). The user terminal 4 receives the notification that the authentication has failed, displays it on the display unit 45 (S110), and ends the process.

If the persons in both face images match, the server 3 determines that the authentication is successful (S108; OK), and transmits the voice data 403 of a predetermined character to the user terminal 4 (S111). When the user terminal 4 receives the audio data 403 (S112), it plays the audio data 403 (S113).

As shown in FIG. 9(c), the audio data 403 assumes a greeting by character A such as "Yaho, how are you?", and the user can feel as if he or she has met character A and been greeted. can. However, the content of the audio data 403 is not particularly limited, and various data can be considered depending on the content of the scenario and character settings. Further, an image of the character may be transmitted to the user terminal 4 together with the audio data 403 and displayed on the display unit 45 when the audio data 403 is played back. Furthermore, the audio data 403 and the above image may be stored in the storage unit 42 of the user terminal 4.

Some character exchange events end with the above processing. When a predetermined period of time has elapsed after the transmission of the audio data 403 or when the end time of the interaction time zone 305 has come, the server 3 transmits a notification to the effect that the event has ended to the user terminal 4, and the user terminal 4 displays the event on the display unit 45. Displays a message indicating termination.

This event is such that the user is greeted by a character in the virtual world when the interaction time period 305 set in advance by the user comes, and corresponds to the above-mentioned non-condition achievement type event. In this embodiment, by having the user actively take the action of photographing and inputting a face image during an event, the user can get the feeling of meeting the character face-to-face, and the character becomes familiar to the user. It is possible to achieve an interaction that feels like this.

On the other hand, in other character exchange events, after the above-mentioned greeting, the user takes further action for the character, and the user aims to clear the event achievement condition by the action. This corresponds to the above-mentioned condition fulfillment type event, and the audio data 403 reproduced in S113 is not only the above-mentioned greeting, but also a sound specifying the above-mentioned action that is emitted immediately after the greeting.

Figure 9(d) is an example of this, where after the above greeting, a voice saying something like "I want to eat sweets, I like strawberry parfait" is played continuously, and the user's behavior is determined by the name of the object (strawberry parfait). It is specified. Note that the method for specifying the user's behavior is not limited to specifying the name of the object itself, but can also be performed using the category or attribute of the object, a keyword that is easily associated with the object, or the like.

(2-2. Scenario 2; Actions for the character)
After hearing the above voice, the user heads to a restaurant in search of a strawberry parfait for the character. FIG. 10 is a flowchart showing the procedure of the portion of the character exchange event in which the user acts for the character. S201 to S202, S207 to S208, and S210 to S211 in FIG. 10 are processes executed by the control unit 41 of the user terminal 4 by controlling each part of the user terminal 4. Control and execute each part of 3.

A user enters a restaurant, orders a strawberry parfait, and starts an application on the user terminal 4. FIG. 11(a) is an example of a screen displayed on the display unit 45 at this time, and a message 504 specifying the user's action "I like strawberry parfait" and a "Make your wish come true" button 505 are displayed. .

When the user performs a flick operation or the like on the "Make your wish come true" button 505, the user terminal 4 activates the camera 46 and photographs the target object, the strawberry parfait, with the camera 46 in accordance with the user's operation (S201).

The user terminal 4 displays the photographed image of the strawberry parfait on the display unit 45 as shown in FIG. 11(b). In addition to the message 504 and the image 506 of the strawberry parfait, a "Show" button 507 and a "Retake" button 508 are displayed on the display section 45.

The “Show” button 507 is a button for transmitting the image 506 of the strawberry parfait to the server 3. When the user selects the "Show" button 507 by tapping or the like, the user terminal 4 transmits the image 506 of the strawberry parfait to the server 3 (S202). Here, photographing the image 506 of the strawberry parfait assumes that the user orders a strawberry parfait for character A and shows it to him in the scenario, as shown in FIG. 12(a). It is something. Note that if the user selects the "Retake" button 508, the user terminal 4 activates the camera 46 again, and the user can retake the picture of the strawberry parfait.

When the server 3 receives the image 506 of the strawberry parfait from the user terminal 4 (S203), the server 3 performs image recognition of the image 506 (S204), and determines whether the image 506 is of the strawberry parfait (object), that is, the image It is determined whether or not 506 corresponds to the instruction based on the audio data 403 (S205).

In S204, for example, a positive image of a strawberry parfait is stored in advance as an event achievement condition image 404 (see FIG. 3(b)), and the positive image and the image 506 received from the user terminal 4 in S203 are analyzed. The image is sent to the device 5 to request analysis, and the analysis device 5 calculates the degree of similarity between both images. Various known methods can be used to calculate the degree of similarity, and there is no particular limitation.

The server 3 receives the degree of similarity from the analysis device 5, and if the degree of similarity is greater than or equal to a predetermined value, the image 506 received from the user terminal 4 is of a strawberry parfait (object) and the event achievement condition is met. Assuming that it has been cleared (S205; YES), the predetermined character's voice data 405 and event image 406 are transmitted to the user terminal 4 (S206). Upon receiving the audio data 405 and the event image 406 (S207), the user terminal 4 plays back the audio data 405 (S208).

This audio data 405 is assumed to be the character A's gratitude to the user, such as "Wow, thank you" as shown in FIG. 12(b), but various data can be considered depending on the content of the scenario and character settings. Similarly to the above, the audio data 405 may be stored in the storage unit 42 of the user terminal 4.

The event image 406 is image data distributed to users as an event achievement reward. The event image 406 is, for example, a bonus image of a character, and is displayed on the display section 45 when the audio data 405 is played back. Note that as a reward for achieving the event, data such as music may be transmitted to the user terminal 4 instead of or in conjunction with the event image 406 and played back.

On the other hand, if the similarity is less than the predetermined value, the server 3 determines that the image received from the user terminal 4 is not of the strawberry parfait (object) and the event achievement condition is not cleared (S205; NO). , transmits voice data 405 of a predetermined character to the user terminal 4 (S209). When the user terminal 4 receives the audio data 405 (S210), it plays the audio data 405 (S211).

The audio data 405 to be transmitted to the user terminal 4 in S209 may also be of various types depending on the content of the scenario and character settings, as described above. For example, if the character is set to have little knowledge of common sense or social systems in the real world, the voice data 405 may say "Wow, thank you" in the same way as the one sent to the user terminal 4 in S206 above. . In other words, the character knows the word strawberry parfait, but does not know what it actually looks like, and therefore assumes that something that is not a strawberry parfait is a strawberry parfait. Users may feel comforted by characters with such settings.

Of course, the audio data 405 transmitted to the user terminal 4 in S209 may be different from that transmitted to the user terminal 4 in S206. For example, if the character is set to know what a strawberry parfait actually looks like, the voice data 405 sent to the user terminal 4 in S209 can be "Isn't it a strawberry parfait?" You can enjoy different interaction scenes by changing the captured images. It is also possible to send the event image 406 and the like together with the audio data 405 to the user terminal 4 in S209, and even if the event image 406 and the like are the same as those sent to the user terminal 4 in S206, They may be different.

Furthermore, as shown in FIG. 12(c), the audio data 405 includes a farewell greeting such as "It's time. See you then" and is played continuously after the above-mentioned "Wow, thank you". Good too.

As described above, the condition fulfillment type event ends. In this case as well, the server 3 sends a notification that the event has ended to the user terminal 4 when a predetermined period of time has passed after the transmission of the audio data 405 or when the end time of the interaction time zone 305 has come, and the user terminal 4 displays the 45 displays a message indicating that the event has ended.

In the condition achievement type event, the user actively takes the action of taking an image 506 of the strawberry parfait, and then plays the voice of the character saying thank you, etc., so that the user can do something for the character. By doing so, the user can feel as if he or she has received gratitude from the character for his or her actions, and an interaction in which the user feels closer to the character can be realized.

Note that after the audio data 405 is reproduced, the end condition of the condition-fulfilling event may be for the user to input audio data such as "Bye-bye" into the user terminal 4 through the microphone of the audio input/output unit 47.

In this case, the user terminal 4 transmits the audio data to the server 3. For example, the user's voice data is registered in advance as user information 300 in the server 3, and the transmitted voice data is identified as the user's voice data using the degree of similarity between the voice data and the voiceprint of the transmitted voice data. Determine whether or not. Similar to the above, calculation of the voiceprint similarity can be performed by the analysis device 5 or the like using a known method. For example, if the similarity between the voiceprints of both pieces of audio data is equal to or greater than a predetermined value, the server 3 determines that the transmitted audio data belongs to the user, and sends a notification to the user terminal 4 that the event has ended.

Further, if the similarity is greater than or equal to a predetermined value and the meaning of the transmitted audio data is a predetermined meaning (for example, a farewell greeting such as "Bye-bye"), a message indicating that the event has ended is sent to the user terminal 4. You can also send it. The semantic content can also be analyzed by the analysis device 5 or the like using a known method.

The above is the same for non-condition fulfillment type events (events that end only with the flow shown in FIG. 7), and after playing the audio data 403 described above, the user inputs audio data such as "Bye-bye" to the audio input/output unit 47. The event termination condition may be input to the user terminal 4 through a microphone or the like.

As explained above, the user terminal 4 of this embodiment plays back the voice data 403 of the character that instructs the user's actions, and allows the user to shoot the image 506 corresponding to the instruction, so that the user can feel as if the character is You can feel like you've done something for them. In this way, in this embodiment, by encouraging the user to take active actions such as photographing the image 506 corresponding to the instruction, it is possible to provide a service that realizes interaction in which the user feels closer to the character. Further, encouraging user behavior is effective as a countermeasure against social withdrawal.

Furthermore, in this embodiment, when the image 506 corresponds to an instruction based on the audio data 403, other audio data 405 such as gratitude to the user is played, and the character receives a reaction such as gratitude based on the user's actions. It is possible to create such an interaction scene. By doing something for the character, the user can feel as if the character has thanked the character for the action, and can feel closer to the character.

Furthermore, the audio data 403 and 405 are recorded using a stereophonic sound system, and by listening to this through earphones or the like, the user can feel as if the characters actually exist beside him or her.

Furthermore, in this embodiment, the user terminal 4 receives audio data 403 and 405 from the server 3, and can provide services to users through the character exchange system consisting of the user terminal 4 and the server 3.

However, the present invention is not limited to this. For example, in this embodiment, authentication using a user's face image is performed in S107 of FIG. 7, but it is also possible to omit authentication using a face image. In this case, when the "Meet Now" button 502 is selected on the selection screen shown in FIG. 8(b), the server 3 transmits the audio data 403 containing the above-mentioned action instructions to the user terminal 4. In addition, it is also possible to reproduce the audio data 403 by reading a code etc. distributed in a predetermined area with the user terminal 4. In this case, the above code is recorded as event information 400 in the storage unit 32 of the server 3, and when the above code read by the user terminal 4 is received from the user terminal 4, the audio data 403 is transmitted to the user terminal 4. and play it.

Furthermore, when the audio data 403 and 405 are played back, it may be possible to display a text version of the audio data 403 and 405 on the display unit 45 in addition to the image of the character. These texts can be transmitted from the server 3 together with the audio data 403 and 405, and may be stored in the user terminal 4.

Furthermore, in this embodiment, the determination as to whether or not the event achievement condition has been cleared is made using the degree of similarity between the image 506 transmitted from the user terminal 4 and the event achievement condition image 404, but this determination method is as follows. Not limited to. For example, if the category of the image 506 transmitted from the user terminal 4 matches a category recorded in advance as the event information 400 (for example, "Strawberry Parfait"), it may be determined that the event achievement condition has been cleared. Similar to the above, category analysis of the image 506 can be performed by the analysis device 5 or the like using a known method.

Further, in this case, the content of the audio data 405 transmitted to the user terminal 4 in S206 or S209 may be changed according to the category obtained by analyzing the image 506. For example, as shown in FIG. 13, if the image 506 is a fruit parfait, it is not the target object "strawberry parfait", but it is classified into the "parfait" category, so it is close to "strawberry parfait", and the image 506 is a pancake. In this case, it is not "Strawberry Parfait" and is not "Parfait", but it is classified in the "Sweets" category, so it is somewhat similar to "Strawberry Parfait", and if image 506 is a rice ball, it is not "Strawberry Parfait" but "Parfait". It's not even a "sweets", so it's far from a "strawberry parfait". In this way, depending on which category the image 506 is classified into or not, it is also possible to determine the audio data 405 from the viewpoint of similarity to the target object.

Furthermore, in this embodiment, the voice data 403 is transmitted from the server 3 to the user terminal 4 when authentication using a face image is successful, but for example, when downloading an application, the voice data 403 is also downloaded in advance, and the authentication When successful, the server 3 may transmit an instruction to reproduce the audio data 403 to the user terminal 4, and the audio data 403 may be reproduced on the user terminal 4. This also applies to the audio data 405 when the event is achieved and the event start notification 501 when the start time of the exchange time period 305 has arrived.

In addition, if the event start notification 501 is sent but the face image authentication is not performed and the event is not held, the event in the interaction time period 305 may be carried over to another day, for example, to the same time period on the next day. is also possible. Similarly, if an event is not held as described above in all time slots in one day, it is also possible to carry over all the events to another day.

Furthermore, the character exchange system 1 of the present invention can also be used for regional revitalization. Therefore, an example in which the character exchange system 1 is used for regional revitalization will be described below as a second embodiment. In the second embodiment, points different from the first embodiment will be explained, and points similar to the first embodiment will be denoted by the same reference numerals in the drawings, etc., and a description thereof will be omitted.

[Second embodiment]
In the second embodiment, regional revitalization is realized using a condition-fulfilling event in which the user acts for the character. FIG. 14 is a flowchart showing the procedure of the part in which the user acts for the character in the character exchange event of the second embodiment. S301, S305 to S308, S313 to S314, and S316 to S317 in FIG. controls and executes each part of the server 3.

In this embodiment, the audio data 403 to be played back on the user terminal 4 in S113 of FIG. It prompts the user to go to a predetermined area (Ikebukuro).

The user who hears the above-mentioned voice heads to the area for the character. The user terminal 4 periodically transmits its location information to the server 3 (S301), and the server 3 receives the location information from the user terminal 4 (S302), and determines whether the user is in a predetermined area based on the location information. It is determined whether or not (S303).

If the server 3 determines that the user is not in the predetermined area (S303; NO), it waits again to receive the location information, but if it determines that the user is in the predetermined area (S303; YES), the server 3 The character's voice data thus obtained is transmitted to the user terminal 4 (S304).

As shown in Figure 15(b), this audio data specifies the user's actions based on the object (game center), such as "What kind of place is a game center?" The user heads to the game center for the character. Note that this audio data is recorded in advance in the event information 400 as audio data to be transmitted to the user terminal 4 when the user is in a predetermined area.

When the user enters the game center, the user starts an application on the user terminal 4. Then, the user operates the user terminal 4 according to the procedure explained in FIG. 11 etc., and the user terminal 4 photographs an image 506 of the game center, which is the target object, with the camera 46 in accordance with the user's operation (S307). The user terminal 4 transmits the photographed image 506 of the game center to the server 3 (S308). Here, shooting the image 506 of the game center assumes that the user takes the character A to the game center and shows him the game center in the scenario, as shown in FIG. 15(c). It is.

The server 3 determines whether the image 506 is of a game center (object) through processes S309 to S311 similar to S203 to S205 in FIG. If the image 506 received from the user terminal 4 is of a game center, the server 3 assumes that the event achievement condition has been cleared (S311; YES), and uses the predetermined character's voice data 405 and the predetermined region's voice data. A coupon code that can be used at the game center is transmitted to the user terminal 4 (S312). When the user terminal 4 receives the audio data 405 and the coupon code (S313), it plays the audio data 405 (S314).

As shown in FIG. 15(d), this audio data 405 is intended to convey character A's impressions such as "Awesome, show us something" and to encourage character A to use the game center afterward. Various options are possible depending on the content and character settings. The coupon code is data distributed to users as a reward for completing an event, and is recorded in advance in the event information 400. The user can use the coupon code to play games at the game center for free, for example.

Note that the processes S315 to S317 when the image received from the user terminal 4 is not from the game center and the event achievement condition is not cleared (SS311; NO) are the same as S209 to S211 in FIG. Various types of audio data 405 can be considered to be transmitted to the user terminal 4 depending on the content of the scenario and character settings.

This second embodiment also provides the same effects as the first embodiment. Further, in the second embodiment, by instructing the user to take a photograph of a target object in a predetermined area as a condition-fulfilling event, it is possible to lead to the use of services in the area and to revitalize the area.

Note that in this embodiment, it was determined that the user is in a predetermined area based on the location information of the user terminal 4, but for example, the server 3 transmits a specific code to the user terminal 4 together with the audio data 403, and the above-mentioned When the code is read by a reading device (not shown) installed in a predetermined area, it may be determined that the user is in that area. In this case, determination based on the location information of the user terminal 4 becomes unnecessary.

Although preferred embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to such examples. It is clear that those skilled in the art can come up with various changes or modifications within the scope of the technical idea disclosed in this application, and these naturally fall within the technical scope of the present invention. Understood.

1: Character exchange system 3: Server 4: User terminal 5: Analysis device 300: User information 301: User ID
302: Registration information 303: Destination 304: Face image 305: Exchange time period 310: Audio transmission means 320: Judgment means 330: Image reception means 400: Event information 401: Date 402: Time zone classification 403, 405: Audio data 404 :Event achievement condition image 406:Event image 410:Audio reception means 420:Reproduction means 430:Photography means 440:Image transmission means 501:Event start notification

Claims (10)

a notification receiving means for receiving an event start notification from the server when the event start time arrives;
a setting means for accepting the setting of the start time;
input receiving means for receiving an input of a user's face image after the notification receiving means receives an event start notification ;
a reproduction means for reproducing voice data of a character that instructs the user's actions by inputting the face image;
a photographing means for photographing an image;
image transmitting means for transmitting the image to the server in order to determine whether the image corresponds to the instruction;
A user terminal comprising: 2. The reproduction means, after reproducing the voice data of the character that instructs the user's actions, reproduces other voice data of the character if the image corresponds to the instruction. 1. The user terminal described in 1. 3. The user terminal according to claim 1 , further comprising audio receiving means for receiving audio data to be reproduced by said reproducing means from said server. a notification means for transmitting an event start notification to a user terminal when the event start time is set in advance by the user;
Facial image receiving means for receiving the user's facial image input to the user terminal from the user terminal after transmitting the event start notification to the user terminal ;
voice transmitting means for transmitting voice data of a character instructing the user's actions to the user terminal when the face image matches the person in the face image registered in advance as user information;
image receiving means for receiving an image taken by the user terminal from the user terminal;
determining means for determining whether the image corresponds to the instruction;
A server characterized by having: 5. The server according to claim 4 , wherein whether or not the image corresponds to the instruction is determined based on the degree of similarity between the image and a positive image corresponding to the instruction. 6. The server according to claim 4 , wherein the voice transmitting means transmits other voice data of the character to the user terminal when the image corresponds to the instruction. A character exchange system having a user terminal and a server,
The user terminal is
notification receiving means for receiving an event start notification from the server when the event start time arrives;
a setting means for accepting the setting of the start time;
input receiving means for receiving an input of a user's face image after the notification receiving means receives an event start notification ;
a reproduction means for reproducing voice data of a character that instructs the user's actions by inputting the face image;
a photographing means for photographing an image;
image transmitting means for transmitting the image to the server;
has
The server is
Notifying means for transmitting an event start notification to the user terminal at the start time of the event preset by the user;
Facial image receiving means for receiving the user's facial image input to the user terminal from the user terminal after transmitting the event start notification to the user terminal ;
voice transmitting means for transmitting voice data of a character instructing the user's actions to the user terminal when the face image matches the person in the face image registered in advance as user information;
determining means for determining whether the image corresponds to the instruction;
A character exchange system characterized by having. The user terminal
a step of accepting settings for the start time of the event;
receiving an event start notification from the server when the event start time arrives;
after receiving the event start notification, receiving input of a user's facial image;
reproducing voice data of a character that instructs the user's actions by inputting the face image;
a step of taking an image;
transmitting the image to the server to determine whether the image corresponds to the instruction;
A character interaction method characterized by performing. A user terminal having a photographing means for photographing an image,
a notification receiving means for receiving an event start notification from the server when the event start time arrives;
a setting means for accepting the setting of the start time;
input receiving means for receiving an input of a user's face image after the notification receiving means receives an event start notification ;
a reproduction means for reproducing voice data of a character that instructs the user's actions by inputting the face image;
image transmitting means for transmitting the image to the server in order to determine whether the image corresponds to the instruction;
A program that makes it work. computer,
a notification means for transmitting an event start notification to a user terminal when the event start time is set in advance by the user;
Facial image receiving means for receiving the user's facial image input to the user terminal from the user terminal after transmitting the event start notification to the user terminal ;
voice transmitting means for transmitting voice data of a character instructing the user's actions to the user terminal when the face image matches the person in the face image registered in advance as user information;
image receiving means for receiving an image taken by the user terminal from the user terminal;
determining means for determining whether the image corresponds to the instruction;
A program that makes it work.

Images