JP2023176404A - Virtual assistant device and program for virtual assistant device - Google Patents

Abstract

To provide a technique capable of providing a topic to a user who utilizes a virtual assistant device and facilitating appropriate continuation of a bidirectional conversation concerning the provided topic.SOLUTION: A virtual assistant device 10 includes: an input unit to which a story from a user is input; a display 15 that displays an image; a control unit 11 that causes the display 15 to display an image of a character 70; and an output unit that outputs a story from the character 70. The control unit 11 displays a topic image different from the image of the character 70 and words concerning the topic image on the display 15. When a story including the words is input to the input unit after the topic image and the words are displayed on the display 15, the control unit 11 causes the output unit to output a story related to the words as a story from the character 70.SELECTED DRAWING: Figure 15

Description

The present invention relates to a virtual assistant device and a program for the virtual assistant device.

Patent Document 1 discloses an audio output system. The audio output system of Patent Document 1 includes audio information acquisition means, output control means, conversation information acquisition means, and the like. The voice information acquisition means acquires voice information regarding the voice uttered by the speaker. The output control means causes an audio output device used by a user of the display medium to output a sentence corresponding to the display content displayed on the display medium in a voice imitating a speaker's voice based on the voice information. The conversation information acquisition means acquires conversation information regarding a conversation between a user and a speaker. When a speaker utters a sentence by voice, an image imitating the speaker's appearance is displayed on the display medium.

JP2020-76885A

In devices where a character is displayed on the display and the character converses with the user, it is desirable to provide the user with a topic, and if a conversation about the provided topic starts, the conversation It is hoped that this will continue in both directions.

When providing topics in this type of device, for example, it is effective to display images that are likely to interest the user on the display unit. However, when a topic is provided by displaying an image, it is difficult to predict exactly what the user will say after seeing the image, so when the device returns a response, it may deviate greatly from what the user said. There is a concern that this may occur. If such a "large discrepancy" occurs, there is a concern that users may feel that the conversation does not mesh with each other or that they are not being listened to properly, and there is a risk that two-way conversation may not continue properly. There is.

One object of the present invention is to provide a technology that can provide a topic to a user using a virtual assistant device and facilitates appropriate continuation of a two-way conversation regarding the provided topic.

A virtual assistant device that is one of the present inventions is
An input section into which a story from a user is input, a display section that displays an image, a control section that causes the display section to display an image of a character, and an output section that outputs the story from the character. A virtual assistant device,
The control unit displays a topic image different from the image of the character and words related to the topic image on the display unit, and after the topic image and the words are displayed on the display unit, a story including the words is displayed. When input to the input section, a story related to the word is outputted to the output section as a story from the character.

When displaying an image that can become a topic (a topic image different from a character image), the virtual assistant device described above can display not only the topic image but also words related to the topic image on the display unit. When such a display is made, the user who sees the topic image is more likely to utter a story that includes the above-mentioned words. Therefore, on the virtual assistant device side, it is possible to prepare a response story regarding the above-mentioned words, assuming that the speech including the above-mentioned words will be given. Furthermore, if the user actually speaks a story that includes the above-mentioned words after the topic image and the above-mentioned words are displayed, the virtual assistant device determines whether the above-mentioned words are included based on the information input to the input section. After confirming that the word is correct, the story related to the above word can be output as a story from the character. Because such an operation is performed, it is easy to prevent the character from replying with an unrelated story in response to the story uttered by the user who viewed the topic image, and it is easy to make the conversation more appropriate.

In the above virtual assistant device, the control section may operate to switch between an effective mode in which analysis of the story input to the input section is enabled and a cancellation mode in which the effective mode is canceled. Further, the control section operates to cause the output section to output a story related to the word as a story from the character when a story including the word is input to the input section during the valid mode. You can. When the control unit causes the output unit to output a story related to the word as a story from the character during the valid mode, the control unit sets a period during which stories from the user are accepted while continuing the valid mode. After setting, the above-mentioned effective mode may be continued until a predetermined termination condition is satisfied.

Since the virtual assistant device described above can switch between the valid mode and the cancel mode, it is possible to analyze the speech during the valid mode, and reduce the processing load during the cancel mode. When the control unit causes the output unit to output a story related to the word as a story from the character, the control unit sets a period for accepting stories from the user while continuing the valid mode, and the end condition is satisfied. This allows the conversation to continue smoothly even after the story related to the above-mentioned words has been provided.

The control unit may operate to display only a single keyword on the display unit as the word related to the topic image.

In this way, when the word displayed in association with the topic image is a single keyword, the user who sees the topic image and the word becomes more likely to utter the keyword. Therefore, there is an increased possibility that the virtual assistant device will recognize a conversation that includes the above-mentioned keyword and continue the conversation appropriately.

When a plurality of types of keywords are displayed in association with a topical image, there is a high possibility that keywords that the user is likely to be interested in are included, and it becomes easier for the user to talk from more viewpoints. This virtual assistant device makes it easier for the user to speak, and when a word that includes any keyword is uttered, it can respond with a story that corresponds to that keyword. It is possible to achieve both ease of use and appropriate conversation.

The control unit displays the topic image together with one or more of the words in a first combination on the display unit, and then displays the topic image together with the words in a combination different from the first combination. It may be displayed in the section.

In this way, if it is possible to change the combination of words that are displayed in association when displaying any topic image, the conversation will become boring for the user even if the same topic image is used continuously or repeatedly. This makes it easier for users to use the service. Even if the combination of the topic image and words is changed, this virtual assistant device can return a story related to the displayed word if a word that includes the displayed word is uttered, so it can improve the conversation. It is possible to balance the difficulty of getting bored and the appropriateness of the conversation.

A program for a virtual assistant device, which is one of the inventions, is
An input section into which a story from a user is input, a display section that displays an image, a control section that causes the display section to display an image of a character, and an output section that outputs the story from the character. A program used for a virtual assistant device,
causing the control unit to perform control to display a topic image different from the image of the character and words related to the topic image on the display unit;
After the topic image and the word are displayed on the display unit, when a story including the word is input to the input unit, a story related to the word is output to the output unit as a story from the character. a step of causing the control unit to perform control to
including.

When displaying an image that can become a topic (a topic image different from a character image), the program for the virtual assistant device described above can display not only the topic image but also words related to the topic image on the display unit. When such a display is made, the user who sees the topic image is more likely to utter a story that includes the above-mentioned words. Therefore, on the virtual assistant device side, it is possible to prepare a response story regarding the above-mentioned words, assuming that the speech including the above-mentioned words will be given. Furthermore, if the user actually utters a story that includes the above words after the topic image and the above words are displayed, this program will detect from the information input to the input section that the above words are included. After checking, it can be operated to output a story related to the word as a story from the character. Because such an operation is performed, it is easy to prevent the character from replying with an unrelated story in response to the story uttered by the user who viewed the topic image, and it is easy to make the conversation more appropriate.

According to the present invention, it is possible to provide a topic to a user using a virtual assistant device, and it is easy to continue a two-way conversation regarding the provided topic appropriately.

FIG. 1 is a block diagram schematically showing the electrical configuration of a virtual assistant system including a virtual assistant device according to a first embodiment. FIG. 2 is an explanatory diagram showing an example 1 of normal display in the virtual assistant device of the first embodiment. FIG. 3 is an explanatory diagram conceptually showing an example of the data structure of user data stored in the management device. FIG. 4 is an explanatory diagram conceptually showing an example of the data structure of content-related data stored in the management device. FIG. 5 is a flowchart illustrating the flow of control in the virtual assistant device of the first embodiment. FIG. 6 is an explanatory diagram showing a second example of normal display in the virtual assistant device of the first embodiment. FIG. 7 is an explanatory diagram showing a third example of normal display in the virtual assistant device of the first embodiment. FIG. 8 is an explanatory diagram showing example 4 of normal display in the virtual assistant device of the first embodiment. FIG. 9 is an explanatory diagram illustrating an example of "predetermined notification" in the virtual assistant device of the first embodiment. FIG. 10 is an explanatory diagram showing a fifth example of normal display in the virtual assistant device of the first embodiment, and is a diagram showing an example in which notification information is displayed. FIG. 11 is an explanatory diagram showing example 1 of content provision by the virtual assistant device of the first embodiment. FIG. 12 is an explanatory diagram showing example 2 of content provision by the virtual assistant device of the first embodiment, and is an example of a display subsequent to the display in FIG. 11. FIG. 13 is an explanatory diagram showing example 3 of content provision by the virtual assistant device of the first embodiment, and is an example of a display subsequent to the display in FIG. 12. FIG. 14 is an explanatory diagram showing example 4 of content provision by the virtual assistant device of the first embodiment, and is an example of a display subsequent to the display in FIG. 13. FIG. 15 is an explanatory diagram illustrating a first specific example of displaying topic images together with words regarding the display performed in the virtual assistant device of the first embodiment. FIG. 16 is an explanatory diagram illustrating a data structure in which a story corresponding to each word is prepared in association with each word. FIG. 17 is an explanatory diagram illustrating a second specific example in which topic images are displayed together with words regarding the display performed in the virtual assistant device of the first embodiment. FIG. 18 is an explanatory diagram illustrating a third specific example in which topic images are displayed together with words regarding the display performed in the virtual assistant device of the first embodiment. FIG. 19 is an explanatory diagram illustrating a fourth specific example in which topic images are displayed together with words regarding the display performed in the virtual assistant device of the first embodiment. FIG. 20 is an explanatory diagram illustrating a specific example 5 in which topic images are displayed together with words regarding the display performed in the virtual assistant device of the first embodiment. FIG. 21 is an explanatory diagram showing a display performed in a virtual assistant device of a comparative example.

<First embodiment>
1. Overview of Virtual Assistant System The virtual assistant system 1 shown in FIG. 1 includes a virtual assistant device 10 and a management device 90. In the following description, the virtual assistant system 1 is also simply referred to as system 1. In a typical example described below, the virtual assistant device 10 functions as a virtual assistant device for elderly people, and can be used by a user at a nursing facility, home, hospital, etc. be.

2. Hardware Configuration of Virtual Assistant Device As shown in FIG. 1, the virtual assistant device 10 installs an application program on a general-purpose information terminal such as a tablet terminal, a smartphone, a personal computer, or a television configured to be able to communicate with an external device. , it may be an information communication terminal that can store and use the information, or it may be a dedicated device that can realize each function described below. The virtual assistant device 10 may be a portable information device having a communication function, or may be a stationary information device having a communication function.

As shown in FIG. 1, the virtual assistant device 10 includes a control section 11, a communication section 12, an interface 13, and a storage section 14. A representative example of the virtual assistant device 10 described below is an example in which the virtual assistant device 10 is implemented by a tablet terminal as shown in FIG.

The control unit 11 shown in FIG. 1 is configured as, for example, a known information processing device. The control unit 11 includes a known arithmetic unit such as a CPU and other peripheral circuits, and can perform various controls and calculations. The control unit 11 has a function of displaying an embodied character on the display unit 15 that constitutes the interface 13.

The communication unit 12 shown in FIG. 1 is a device that can access a wide area communication network directly or indirectly via another device using a known wired communication method or a known wireless communication method. The communication unit 12 may be configured to perform wireless communication with a base station and directly access a wide area communication network (for example, the Internet) via a base station (not shown). The communication unit 12 may be configured to perform wireless communication with an access point (not shown) and indirectly access a wide area communication network via the access point. The communication unit 12 may be configured to perform wired communication with a relay device (such as a router) and access a wide area communication network via the relay device.

The interface 13 shown in FIG. 1 is a device that receives and outputs input from a user. The interface 13 includes a display section 15 , an audio output section 16 , an operation section 17 , and an audio input section 18 .

The display section 15 and the audio output section 16 correspond to an example of an output section and have a function of outputting information. The display unit 15 is configured as a known image display device such as a liquid crystal display or an organic electroluminescent display, and has a function of displaying various images. In the representative example described below, the display unit 15 forms part of a touch panel type display device. The audio output unit 16 is constituted by, for example, a known sounding device such as a speaker. The audio output section 16 has a function of outputting various sounds in cooperation with the control section 11.

The operation unit 17 and the voice input unit 18 function as an input unit for inputting information. The operation unit 17 corresponds to an example of a motion detection unit, and is an input device that allows input operations using a contact method. The operation unit 17 is preferably a touch panel, for example, and may include buttons for inputting information. The audio input unit 18 is configured by, for example, a known audio input device such as a microphone. The audio input section 18 has a function of converting input sound into an electrical signal and providing it to the control section 11 . The audio input unit 18 functions to obtain an audio signal representing the user's voice when the user makes a voice or other sound toward the audio input unit 18. Specifically, when the user utters a voice that can be detected by the voice input unit 18, the voice input unit 18 can acquire a voice signal indicating the content of the voice and convert it into an electrical signal.

In the representative example described below, the display section 15 and the operation section 17 constitute a touch panel display device 20, as shown in FIG. 2 and the like. In the example shown in FIG. 2, the touch panel forming part or all of the operation unit 17 is configured to be able to transmit light from the display unit 15, and is configured to be transparent so that the image from the display unit 15 can be viewed from the outside. It covers the display section 15 as a panel.

The storage unit 14 has a function of storing various information. The storage unit 14 employs a known storage device such as a semiconductor memory, HDD, or SSD. The control unit 11 has a function of writing various information to the storage unit 14 and a function of reading various information stored in the storage unit 14. The storage unit 14 stores various programs such as application programs to be described later. The storage unit 14 stores identification information (for example, URL (Uniform Resource Locator), etc.) and other data for accessing sites, information, programs, etc. managed by the management device 90 via a wide area communication network. is also remembered.

3. Management Device The management device 90 shown in FIG. 1 has various information processing functions and various calculation functions. The management device 90 is an external device provided outside the virtual assistant device 10. The management device 90 has a function of registering various information, a function of distributing various information, and the like. The management device 90 may be any device as long as it has a communication function and an information processing function. The management device 90 is configured as a computer including, for example, a CPU, a storage medium, a communication device, and the like. In the example of FIG. 1, the management device 90 includes a control device 91, a communication section 92, a display section 93, an input section 94, and a storage section 95.

The control device 91 is configured, for example, as a known information processing device. The control unit 11 includes a known arithmetic unit such as a CPU and other peripheral circuits, and can perform various controls and calculations.

The communication unit 92 is a device that can access a wide area communication network directly or indirectly via another device using a known wired communication method or a known wireless communication method. The communication unit 92 may be configured to perform wireless communication with a base station and directly access a wide area communication network via a base station (not shown). The communication unit 92 may be configured to perform wireless communication with an access point (not shown) and indirectly access a wide area communication network via the access point. The communication unit 92 may be configured to perform wired communication with a relay device (such as a router) and access a wide area communication network via the relay device.

The display section 93 is configured as a known image display device. The input unit 94 is configured as a known input device such as a keyboard, a mouse, a touch panel, a voice input unit, etc., and allows information to be input by touch operation, voice input, or the like. The storage unit 95 is a storage device that stores various information. A database may be configured in the storage unit 95.

4. Distribution of Content from Management Device The management device 90 has registered information regarding users to whom information is to be distributed. FIG. 3 shows an example data structure of user data stored in the storage unit 95 of the management device 90 shown in FIG. 1. As shown in FIG. 3, the user data stored in the management device 90 constitutes a database, in which ID, registration type, and user information are stored in association with each user. The ID is information that can identify each user, and is identification information that can identify each user. The registration type is information for registering the type of information desired by the user or the type of information meaningful to the user. User information is various information about the user, and may include personal information such as name, date of birth, email address, hobbies, health information, place of residence, past episodes, etc. It may also include specific delivery destination information. For example, in the user data in Figure 3, if ID1 is specified as the user ID, the type of information desired by the user specified by "ID1" is specified by the information of "registration type 1", and the The user's personal information, etc. is specified by the information of "user information 1".

In the system 1 according to the present embodiment, various types can be registered as "type of information desired by the user" and "information meaningful to the user." Specifically, multiple types of hobbies or purpose in life are available for selection in advance as ``types of information desired by users'' and ``information meaningful to users,'' and any one selected from these multiple types is prepared in advance. One or more types may be included in the registration information. Various types of hobbies or purpose in life can be prepared, such as gardening, sports, recreation, gymnastics, music, art, cooking, nostalgic scenery and toys, travel, animals, vehicles, and shopping information. . Alternatively, as the type of information desired by the user, a plurality of disease-related types may be prepared in advance so as to be selectable, and one or more types selected from the plurality of types may be included in the registered information. Specifically, a plurality of types related to the prevention of geriatric syndromes, rehabilitation, and relaxation may be prepared in advance so as to be selectable, and one or more types selected from the plurality of types may be included in the registered information.

For example, in the user data in FIG. 3, ID1 is information of "00001", registration type 1 is information specifying "gardening, gymnastics", and user information 1 is "Taro Yamada" as the user name. It is assumed that the information specifies "address information 1" as the delivery destination address. In this case, if the control device 91 refers to the user data, the name of the user specified by the ID "00001" is "Taro Yamada", the delivery destination address is "address information 1", and this It is possible to specify that the type desired by the user is "gardening, gymnastics."

The management device 90 is capable of registering and distributing content. FIG. 4 shows an example of the data structure of content-related data stored in the storage unit 95 of the management device 90 shown in FIG. 1. As shown in FIG. As shown in FIG. 4, the content-related data stored in the management device 90 constitutes a database, and for each content, content identification information, content data, delivery date and time, and content type are stored in association with each other. ing. In this specification, content data corresponds to an example of content information, and is also simply referred to as content.

The content identification information may be any information that allows each content to be identified and specified, and may be an identification number or a specific name. Content data is specific data of content, and may include video data, still image data, audio data, etc., and may also include other data (eg, program data, etc.). The distribution date and time is information that specifies the date and time when the associated content (content data) is distributed. Type information is information that specifies the type of content (content data) that is associated. The type of content specified by the type information may be one type or multiple types.

The management device 90 sends each content (content data) specified by each piece of identification information to the virtual assistant device 10 of the user who should distribute each content (the user who has registered the type of each content). Deliver each distribution date and time associated with the content. Note that in FIG. 1, only one virtual assistant device 10 owned by one user is illustrated, but in the system 1, many users can have virtual assistant devices 10, and in FIG. , illustration of the virtual assistant device 10 owned by another user is omitted. When distributing content, if there are multiple users to whom the content is to be distributed, the management device 90 can distribute the content to the virtual assistant devices 10 of any of the users. Note that in this embodiment, the management device 90 identifies the users to whom each content is to be distributed (users who have registered the types of each content), and sends each content ( However, the content distribution is not limited to this. For example, when the user's virtual assistant device 10 requests the management device 90 to distribute content based on information stored in its own storage unit, the user's virtual assistant device 10 requests Each content (content data) may be distributed.

For example, in the content-related data in FIG. 4, "identification information 1" is "0000A" information, "data 1" is "predetermined video data related to gardening", and "date and time information 1" is "2021 January 2021". 10:00 on the first day of the month," and "type information 1" is information specifying "horticulture." In this case, the control device 91 refers to the content-related data in FIG. As data, it is possible to specify that the data should be distributed at "January 1, 2021, 10:00". In this case, the management device 90 transfers the content data (data 1 "predetermined video data related to gardening") specified by identification information 1 (0000A) to a user who registers data of the type "gardening" (see FIG. will be delivered to the virtual assistant device 10 of the user whose registered type is "gardening" in the user data of "January 1, 2021, 10:00".

5. Reception control of virtual assistant device 5-1. Basic Control The virtual assistant device 10 stores an application program. This application program is installed on the virtual assistant device 10. This application program is stored in the storage unit 14, read out and executed by the control unit 11.

As described above, the management device 90 distributes content (content data) to the virtual assistant device 10. The content distributed from the management device 90 to the virtual assistant device 10 may be content that belongs to a type registered in advance by the user who owns the virtual assistant device 10, and may be selected by the management device 90 regardless of the registration by the user. The content may be The virtual assistant device 10 receives content distributed to itself from the management device 90 through the communication unit 12 . When the communication unit 12 receives content (content data) distributed from the outside (management device 90 ), the control unit 11 stores the content data in the storage unit 14 and sends a “predetermined notification” via the interface 13 . ” Executes control that performs. Note that the "predetermined notification" may be performed before the content data is stored in the storage unit 14.

Specifically, the application program is a program that causes the control unit 11 to perform control according to the flow shown in FIG. The control unit 11 controls the control unit 11 when a predetermined start condition is satisfied (for example, when a predetermined start operation is performed on the operation unit 17 (for example, an operation on the touch panel display device 20 to start the application program). case), the above application program is executed, and in step S1, the character 70 embodied in the image is displayed on the display section 15.

FIG. 2 shows a specific example of the display made in step S1. In FIG. 2, a character 70 is a virtual assistant (personification) imitating an ordinary person. The character 70 displayed in FIG. 2 is just an example, and may be a virtual assistant modeled after a person in a specific profession, such as a care worker, nurse, or doctor. Further, the virtual assistant is not limited to a human, but may be a virtual assistant modeled after an animal, a robot, or the like. The above application program may include a program that realizes the function of a chatbot so that the character 70 displayed in FIG. 2 automatically carries out a conversation. The content of the automatic conversation can be changed or selected according to the user's pre-registered attributes (gender, family composition, housing environment, birthday, hobbies, content difficulty level, etc.). The image of the character 70 shown in FIG. 2 can be realized by a still image, a moving image, or the like. 7. As shown in FIG. 8, the facial expressions, postures, movements, actions, etc. of the character 70 may be varied in various ways.

In steps S1, S3, S4, S8, etc., the control unit 11 causes the character 70 to be displayed as shown in FIGS. etc., or may display illustrations, photographs, computer graphics, and other images illustrating morning, noon, evening, and night time zones. For example, FIG. 6 shows an image of a predetermined daytime period, and in this image, the background color outside the character 70 is changed to a bright color (specifically are shown in a predetermined light color). On the other hand, FIG. 8 shows an image of a predetermined night time period, and in this image, the background color outside the character 70 is changed to a dark color (a predetermined color) to indicate the night time period. shown in dark colors).

After step S1, the control unit 11 determines in step S2 whether a conversation start condition is satisfied. The conversation start condition is a condition predetermined as a condition for starting a conversation from the virtual assistant device 10. The conversation start condition may be that a predetermined voice is input to the voice input unit 18 (for example, that a predetermined wake word is input). In the representative example described below, the name of the character 70 is used as a wake word, and one of the conditions for starting a conversation is that this wake word is input by voice. Note that this example is just an example, and a predetermined greeting (for example, the word "hello") may be used as the wake word, and voice input of this wake word may be set as one of the conversation start conditions. The conversation start condition is not limited to these examples, and may be, for example, that a predetermined operation has been performed on the operation unit 17 (for example, that an operation of tapping near the display unit 15 has been performed). Alternatively, the conversation start condition may be that a preset reserved time has arrived.

When the control unit 11 determines in step S2 that the conversation start condition is satisfied, the control unit 11 proceeds to step S3 and starts providing the conversation or content. Conversation and content provision in step S3 can be performed in the same manner as step S8, which will be described later. On the other hand, if it is determined in step S2 that the conversation start condition is not satisfied, the process advances to step S4 and normal display is continued. Note that while the No determination in step S2 and the No determination in step S5 are repeated, the character display as shown in FIGS. Change movements, actions, etc.

After step S3 or step S4, the control unit 11 determines whether there is a new distribution in step S5. If the control unit 11 determines in step S5 that the communication unit 12 has not received the content (content data) distributed from the management device 90, the control unit 11 determines No in step S5 and returns the process to step S2, and returns the process to step S2. Perform the following processing again. When the control unit 11 determines that there is a new distribution in step S5, it determines Yes in step S5, and performs the processing from step S6 onwards.

5-2. When there is new distribution The control unit 11 determines in step S5 whether the communication unit 12 has received the content (content data) distributed from the management device 90, and if it is determined that the communication unit 12 has received the content (content data), the control unit 11 determines in step S6 , causes the interface 13 to perform "predetermined notification". The "predetermined notification" is the display or audio output of "transmission information that indicates that the virtual assistant device 10 has received distribution of new content from the outside." The display and audio output of the above-mentioned communication information may be a message display or audio output, or may be an image display such as a symbol or a picture that indicates that new content has been delivered. It may also be the output of a notification sound (generation of an alarm sound, buzzer sound, etc.) to notify that the distribution of the content has been received. Preferably, the notification sound is, for example, a telephone ring or a charm sound that is familiar to the user in daily life. For example, FIG. 9 shows an example of "predetermined notification". In the example of FIG. 9, an image 72 of a message indicating that the distribution specified by the distribution name has arrived is displayed as a "predetermined notification" on the display unit 15, but instead of displaying such a message, or It may be used in combination with display to notify a message by voice, display images such as pictures, output notification sound, etc. For example, the notification may be made through a conversation between characters.

In this manner, when the communication unit 12 receives content data (content information) distributed from the outside (management device 90), the above application program executes "predetermined notification via the interface 13" in step S6. The control section 11 is caused to execute the "control to be performed."

The control unit 11 causes the interface 13 to perform "predetermined notification" in step S6, and then determines in step S7 whether or not there is a provision instruction. Specifically, in step S7, the control unit 11 determines whether "an input instructing the provision of content has been received via the interface 13."

The "input instructing to provide content" may be, for example, a predetermined operation on the operation unit 17 or a predetermined voice input. For example, in an example where a "predetermined notification" as shown in FIG. 9 is made by the process of step S6, two selection buttons ("not watch" and "watch" button images) are displayed together with the message "predetermined notification". Ru. In this case, the operation of selecting "View" from among these selection buttons (for example, the operation of clicking the "View" image 72) corresponds to an example of "input for instructing provision of content." Therefore, if an operation to select "view" is performed after the display as shown in FIG. 9 is performed in step S6, it is determined in step S7 that an input instructing the provision of content has been received, and the processing in step S8 is performed. Execute.

When the control unit 11 determines in step S7 that the “input instructing the provision of content” has not been received, the control unit 11 stores the newly distributed content data in the storage unit 14 in step S9. For example, if the "input instructing the provision of content" is not made until a predetermined time has elapsed after the "predetermined notification" has been made in step S6, the control unit 11 may generate content data (content information) as shown in FIG. The display unit 15 is caused to display notification information indicating that the information) has been distributed. For example, when two new pieces of content data are distributed from the management device 90 and a "predetermined notification" is performed in step S6 as shown in FIG. If the "view" button is not pressed, the process advances to step S9, where newly distributed content data is accumulated and an image as shown in FIG. 10 is displayed. In the image of FIG. 10, the number of unprovided contents among newly distributed contents is displayed by an image 74 as broadcast information. In addition, when the "predetermined notification" is performed as shown in FIG. 9 in step S6, and the "Do not view" button is selected (that is, when an input instructing to suspend provision of the content is made) In this case, the process advances to step S9 and similar processing is performed.

If the control unit 11 determines in step S7 that an "input instructing the provision of content" has been received, the control unit 11 causes the display unit 15 to display a conversation and content provision in step S8. When determining Yes in step S7 and performing the process in step S8, the control unit 11 outputs content based on content data (content information) newly distributed from the management device 90 (for example, outputs content based on newly distributed content The display section 15 and the audio output section 16 are caused to play back moving images, display still images, output audio, etc. included in the data.

The control performed in step S8 when the answer is Yes in step S7 is to output "conversation regarding content received by the communication unit 12" via the interface 13. FIGS. 11 to 14 show an example of the control that the control unit 11 performs in step S8 when the result in step S7 is Yes. FIGS. 11 to 14 are examples of providing the content in step S8 when content called "lively delivery" related to "gardening" is newly delivered from the management device 90. In this example, the delivered content data contains videos and conversation data related to ``gardening''. In this case, the control unit 11 performs control in step S8 so as to reproduce the motion and emit a voice for conversation. For example, in the example shown in FIG. 13, when a sunflower field is displayed, the user utters the conversation, ``Sunflowers, summer flowers, have a cheerful image.'' When uttering this conversation, by displaying the character 70, it is possible to give the impression that the character 70 is talking to the user. In addition, when uttering a conversation, the control unit 11 may utter the utterance using the nickname of the user registered in advance.

The control unit 11 provides content in step S8, and if a conversation is uttered to the user as shown in FIG. An attempt is made to "recognize the response from the user" based on at least one of the voices. For example, a predetermined response operation on the operation unit 17 (e.g., an operation of tapping the character 70 or other design) or a predetermined voice input on the voice input unit 18 (e.g., inputting words) is considered to be a "response from the user." It is determined. When there is a "response from the user" on the interface 13, the control unit 11 performs control so that further conversation in response to this response is output via the interface 13. "Further conversation in response" may be a topic of the type of content being provided, or a topic different from that type. For "further conversation in response to the response," it is desirable to use both text display and audio output, but either one may be used. Note that when the control unit 11 performs control to issue "further conversation in response to the response," it is desirable to attempt "recognition of the response from the user" again. If there is a "response from the user," it is desirable to output further conversation in response to this response via the interface 13. In this way, the conversation can be continued by repeating "further conversation in response to the response" and "recognition of the response from the user." Further, the status for further conversation may be notified by text, illustrations, audio, visual effects such as flashing, etc. For example, displaying ``continuous conversation'' indicates that continuous conversation is possible, which is helpful to the user.

In this way, when the control unit 11 outputs a "conversation about content" via the interface 13 and then recognizes a "response from the user" via the interface 13, control to output "conversation". Step S8 is a step for causing the control section 11 to perform such control.

Note that when having a conversation in step S<b>8 , the control unit 11 is used in conjunction with voice input/output to input and output the utterance content (user's utterance content) voice input via the voice input unit 18 and the virtual assistant device 10 . The content of the utterance (for example, the content of the utterance of the character 70) may be displayed in text on the display unit 15.

Note that although FIGS. 11 to 14 show examples of providing content of the type registered in association with the virtual assistant device 10 (the type desired by the user), the content and conversation provided in step S8 are It is not limited to this example.

5-3. Details of normal operations during a period in which there is no new distribution The following description relates to details of normal operations performed during a period in which there is no new distribution (period in which it is determined No in step S5). During this period, the control unit 11 mainly switches between a release mode (standby mode) and a valid mode (conversation mode). In the present embodiment, a mode in which the process in step S4 is continued during a period in which there is no new distribution (a period in which it is determined No in step S5) is an example of the cancellation mode. Furthermore, a mode in which the process of step S3 is continued during the above period is an example of an effective mode.

The effective mode (conversation mode) is a mode from when the determination is Yes in step S2 until the end condition of the effective mode is satisfied. The condition for ending the valid mode may be that the content executed in the valid mode has ended, or that no predetermined input has been made to the interface for a certain period of time (e.g., voice input, touch panel operation, etc.). It may also be that the user has made an input to the interface instructing termination (voice input or touch operation instructing termination, etc.). Specifically, the valid mode is a mode in which the control unit 11 detects words other than the wake word. When a voice is input to the interface during the valid mode, the control unit 11 analyzes the voice input to the interface and recognizes the words meant by the voice using a known method.

The release mode (standby mode) is a mode during a period in which the determination is No in step S2, and is a mode when the effective mode is released. The release mode is a mode in which the control unit 11 monitors whether a wake word has been input to the interface. When a voice is input to the interface during the release mode, the control unit 11 determines whether or not the voice is a wake word using a known method. However, the control unit 11 does not recognize any voice other than the wake word from the voice input to the interface during the release mode. In other words, in the release mode, the control unit 11 does not need to recognize speech other than the wake word, and only needs to determine whether it is a wake word or not, so the speech recognition is simpler than in the enable mode. can be converted into

As shown in the flowchart of FIG. 5, the control unit 11 is set to the release mode during a period in which the determination of No in steps S2 and S5 is repeated, and during the release mode, it continuously tries to detect the wake word. When the control unit 11 detects that the wake word is input by voice into the interface 13 during the release mode, it determines Yes in step S2, switches to the valid mode, and advances the process to step S3. Then, the effective mode is continued until the end condition for the effective mode is satisfied. When the termination condition of the effective mode is satisfied, the control unit 11 switches to the release mode and continues the release mode. The control unit 11 can also cause the display unit 15 to display an image in which the character performs a standby action during the release mode.

The content provided by the virtual assistant device in steps S3 and S8 may be content that simply displays a still image or simply plays a moving image, but may also be content that allows a simulated experience. Note that it is desirable that the user be able to select which content is to be implemented in step S3. The content selection method by the user may be, for example, a method in which multiple types of content names are displayed on the touch panel display device 20 and the user selects the desired content. The content type is suggested, such as "Shall we watch a travel video?", and when the user gives the desired instruction (tap operation, voice input of the specified term, etc.), the content is displayed. may be selected. The content is not limited to these methods, and content may be selected using other methods.

5-4. Specific Example of Conversation The following explanation relates to a specific example of the conversation that takes place in steps S3 and S8.
The specific example described below is an example on the premise that the output unit outputs a story from the character 70. In this embodiment, the audio output section 16 and the display section 15 correspond to an example of an output section, and have a function of outputting a story from the character 70. The operation of outputting the story from the character 70 may be an operation of outputting the story from the character 70 in the form of voice, text, etc. while the character 70 is displayed on the display unit 15. Alternatively, the operation of outputting the story from the character 70 may be an operation of outputting the story from the character 70 by voice while the character 70 is displayed on the display unit 15, or an operation of outputting the story from the character 70 with the character 70 not displayed. An operation of outputting audio may also be used. Further, the specific example described below is an example based on the assumption that a message from a user can be input to the input unit. In this embodiment, the voice input section 18 and the operation section 17 correspond to an example of an input section, and function as a device into which a user's speech is input. The control unit 11 has functions to perform various controls, such as a function to display an image of the character 70 on the display unit 15, a function to cause the output unit to perform an output operation, and a function to analyze information input to the input unit. , etc.

In this embodiment, as described above, the control unit 11 operates to switch between the valid mode and the cancel mode. The valid mode is a mode for validating the analysis of the story input to the input section. The cancellation mode is a mode in which the above-mentioned valid mode is canceled. During the release mode, "analysis of the story input to the input section" is not performed.

Although the control unit 11 can adopt various types of conversations as the conversation to be adopted in the process of step S3 in the control of FIG. 5, predetermined provision conditions are satisfied at the time of performing the process of step S3. In this case, the display section 15 is controlled to display the topic image. The above provision condition is a condition for providing a topical image. The above-mentioned provision condition may be that a predetermined time condition is met, that content including a topical image is distributed to the virtual assistant device 10, or that it is another condition. You can. The predetermined time condition may be that a predetermined time, date, or day of the week has arrived, or that a certain amount of time has passed since the last topical image provision ended. Other time conditions may also be used.

When controlling the display unit 15 to display the topic image in the process of step S3, the control unit 11 determines the topic image to be displayed. The topic image is an image different from the image of the character 70, and is an image that becomes a topic of conversation. "Becoming a topic of conversation" means that when at least a word associated with a topic image is uttered by a user, the virtual assistant device 10 provides a story related to the word and it becomes a topic of conversation. including.

In this embodiment, a plurality of types of topic images are prepared in advance, and when providing topic images in the process of step S3, one of the topic images is selected and displayed. In this embodiment, image data of a plurality of topic images are prepared in advance, such as topic image A and topic image B shown in FIG. 16, and although not shown in FIG. A large number of image data such as image D... are also prepared. The types of topic images to be prepared are not particularly limited, but include various images such as landscapes, travel, nature, plants, animals, buildings, food, events, and activities.

Furthermore, in the example of FIG. 16, image data for each of the plurality of topic images A, B, C, D, etc. is prepared in association with data of one or more words related to each topic image. . The word to be associated with the topic image is preferably a keyword consisting of one word or a combination of a small number of words; for example, a noun with a short number of words is a suitable example. However, it is not limited to this example, and may be a short sentence, for example.

Furthermore, in the example of FIG. 16, data for each word is associated with data for one or more stories related to each word. For example, words A1, A2, A3, etc. are prepared in correspondence with topic image A, and stories A11, A12, A13, ... are prepared in correspondence with word A1, and they correspond to word A2. Stories A21, A22, A23, etc. are prepared. Similar correspondence is made for topical image B as well. Examples of words associated with topical images include names of landscapes, plants, animals, buildings, foods, events, activities, etc., and place names of the locations of images. It is desirable that the one or more stories related to a word include the word. For example, stories related to the word "Atami" include stories that praise the target of the word, such as "Atami is nice, isn't it?" and stories that praise the target of the word, such as "I've never been to Atami." It can be a story about an experience related to the target of the word, such as ``Atami seems to be thriving.'' It can also be a story about the situation surrounding the target of the word, such as ``Atami seems to be thriving.'' .

Since correspondence data that associates one or more words and a story related to each word with each topic image is stored in the storage unit 14 in a data structure as shown in FIG. 16, any topic image is selected. In this case, data of the topic image, data of any word associated with the topic image, and data of any story associated with the word can be read out.

When a plurality of types of topic images are prepared as shown in FIG. 16, identification information for identifying to which attribute each topic image belongs may be attached. For example, topic image A belongs to the attribute of "travel," topic image B belongs to the attribute of "sports," topic image C belongs to the attributes of "flowers" and "event," and so on. can be attached. The identification information that specifies the attributes of each topical image may specify only one type of attribute, or may specify two or more types of attributes.

The method by which the control unit 11 selects a topic image in step S3 may be a method of randomly selecting a topic image from a plurality of prepared topic images as shown in FIG. The topic images may be selected in order, or the topic images that match the conditions set by the user may be selected from a plurality of topic images prepared. For example, if the registered information includes one or more types of "information desired by the user" or "information meaningful to the user" mentioned above, topic images belonging to the registered types may be displayed. You may choose. For example, if "travel" is registered as "type of information desired by the user" or "information meaningful to the user," topic images belonging to "travel" may be selected.

When the control unit 11 selects any topic image, the control unit 11 selects one of the words from a plurality of words corresponding to the selected topic image. It may be a method of randomly selecting words from among a plurality of words associated with The topic images that match the conditions set by the user may be selected. When selecting a word corresponding to a selected topic image, only one word may be selected from a plurality of words associated with the selected topic image, or a plurality of words may be selected.

When displaying the topic image in the process of step S3, the control unit 11 displays the topic image and words selected as described above in association with each other. For example, if the topic image 82A showing the scenery of the coast of Atami as shown in FIG. 15 is selected and the word "Atami" is selected from among the plural words associated with the topic image 82A, Control is performed so that the selected topic image 82A and words 84A are displayed at the same time. In the example of FIG. 15, a topic image 82A is displayed as a background image of the character 70 together with the image of the character 70, and the word "Atami" is displayed at the same time as superimposed on the topic image 82A. When displaying the topic image 82A, the period for displaying the word 84A corresponding to the topic image 82A may be such that the word 84A is displayed for the entire period during which the topic image 82A is displayed, or for a portion of the period in which the topic image 82A is displayed. The word 84A may be displayed in the period. Further, the method of displaying the topic image 82A and the word 84A in association with each other does not necessarily have to be a method of displaying both at the same time. For example, after displaying the word "Atami" 84A for a certain period of time, the display may be performed such that the display of the word "Atami" 84A is immediately switched to the display of the topic image 82A.

When displaying the topic image 82A as shown in FIG. 15 in the process of step S3, the control section 11 sends audio to the audio output section 16 so that the character 70 reads out the words 84A displayed on the display section 15 together with the topic image 82A. may also be output.

In the above example, in the program for executing the control shown in FIG. This corresponds to an example of the step of causing the control unit 11 to perform the control to display the display on the unit 15. The control unit 11 operates according to this program to display a topic image different from the image of the character 70 and words related to the topic image on the display unit 15, as shown in FIG.

In the above example, in the program for executing the control shown in FIG. This corresponds to an example of the step of "controlling the control section 11 to output a story related to the word to the output section as a story from the character 70 when a story including the word is input to the input section". According to this program, when a story including the word is input to the input section after the topic image and the word are displayed on the display section 15, the control section 11 displays a story related to the word from the character 70. It operates so that it is output to the output section as .

When the above-described topic image and words are displayed in step S3 or S8, the control unit 11 continues the effective mode after the display until a predetermined termination condition is satisfied. Specifically, for a predetermined period of time after both the topic image and the words are displayed, it is checked whether a story including the words has been input to the input unit (for example, the audio input unit 18). For example, the "predetermined period of time after both the topic image and the words are displayed" means "the display of the topic image has started" and "the display of the words associated with the topic image has started". It may be a certain period of time from the time when both conditions are met, and after both of the above conditions are met, a predetermined condition (for example, that a voice that does not include the above words has been input, that the voice that does not include the above words has been input for a certain period of time) It may also be the period until the conclusion of the agreement (e.g., that the term has been continued). The above-mentioned termination condition (valid mode termination condition) may be that the predetermined period has elapsed, or that another condition is satisfied after the predetermined period has elapsed. When adopting other conditions, the other conditions may be that a specified period longer than the above-mentioned predetermined period has elapsed, or that a set time has arrived, or that a predetermined operation is performed on the operation panel. It may be that there was.

The control unit 11 determines whether a story including the word has been input to the audio input unit 18 during the predetermined period after both the topic image and the word corresponding to the topic image are displayed in the valid mode. If this is confirmed, and a story including the word is input to the input section (for example, the voice input section 18) during the predetermined period in the valid mode, the story related to the word is read as a story from the character 70. It operates to cause the output section (for example, the audio output section 16) to output the signal. In this case, one of the stories is selected and output from among the multiple stories prepared in association with the above words (multiple candidate stories prepared as shown in Figure 16), but the story selection method may be selected at random from among a plurality of stories prepared in association with the above words, or may be selected sequentially from among a plurality of stories in a predetermined order; A story that matches the conditions set by the user may be selected from among the stories.

For example, when the control unit 11 performs control to display a single keyword "Atami" as a word related to the topic image together with the topic image (image of the coast of Atami) as shown in FIG. 15, After the topic image and the single keyword are displayed, the audio input to the audio input unit 18 over the predetermined period is analyzed. Then, the control unit 11 checks whether speech including the single keyword has been input to the voice input unit 18 during the predetermined period. A voice that includes a single keyword may be a voice that utters only the single keyword, such as "Atami," or a voice that utters only the single keyword, such as "It's Atami," or "Atami is nice." It may also be a voice with other words added to the keyword. In the example of FIG. 15, when the control unit 11 determines that the voice including the single keyword has been input to the voice input unit 18 during the predetermined period, the control unit 11 outputs the story corresponding to the single keyword as voice. The unit 16 may be operated to output the information. For example, if a voice such as "It's Atami" is input to the voice input unit 18 during the predetermined period, and the control unit 11 analyzes this voice and determines that the keyword "Atami" is included, the control unit 11 selects one of the stories from a plurality of stories prepared in association with the word "Atami" and displays the character 70 on the display section 15 while outputting the audio of the selected story. 16.

There is also a concern that even if displayed as shown in Figure 15, the word "Atami" may not be uttered. Therefore, if a user says something like "Please tell me where the photo is" (a story that does not include the displayed words) while the display shown in Figure 15 is displayed, the user might say something like "I'm interested." "Could you please read out the textual information written on the photo?" may be output as a speech from the character 70 to encourage the character 70 to read out the textual information displayed on the screen. After that, during the above-mentioned predetermined period, the message ````Atami'' is written.'' '' (a story that includes the displayed words), the story associated with ``Atami'' will be ``You can get there from Tokyo by Shinkansen, and it's a nearby and famous tourist destination.'' You may also output the speech as audio.

In this way, the control unit 11 outputs a story related to words as a story from the character 70. The method of outputting the story as a story from the character 70 may be a method of outputting the story as audio while displaying the character 70, or a method of outputting the story as audio with the voice of the character 70 without displaying the character 70.

Note that when the control unit 11 causes the output unit (for example, the audio output unit 16) to output a story related to the word as a story from the character 70 during the valid mode, the control unit 11 thereafter changes the valid mode to After providing a period during which messages from the user are continuously accepted, the effective mode may be continued until a predetermined termination condition is met. If the control unit 11 continues the valid mode after outputting a story related to the above-mentioned word, and if an additional story is input from the user to the voice input unit 18, the control unit 11 outputs a story corresponding to the word. The output unit 16 may also operate to output the information. For example, in the example of FIG. 15, after selecting a story associated with the word "Atami" and outputting the audio of the selected story while displaying the character 70, the user asks for an additional story about "Atami". When input to the input unit 18, a story other than the story that has already been output is selected from a plurality of prepared stories associated with "Atami", and the selected story is voiced while displaying the character 70. You can also output it. The virtual assistant device 10 allows such a conversation to continue until the effective mode ends.

Note that, in this embodiment, as shown in FIG. 16, a plurality of words are prepared in association with one of the topic images. In such an example, after realizing a conversation as described above by displaying one of the topic images on the display unit 15 in the first combination with "any one or more words out of a plurality of types" (specific Specifically, when a word included in the first combination is input by voice, the conversation is realized so that the story corresponding to the word is output as voice), and then the topic image is changed to the first combination. A different combination of words associated with the topic image may be displayed on the display unit 15 to realize a conversation from a different perspective. In this case, the timing at which the topic image and words are displayed in a combination different from the first combination (the topic image re-display timing) can be set variously. For example, after the first combination is displayed, It may be a time after a predetermined elapsed time (for example, a predetermined time, a predetermined number of days, etc.) has passed, it may be a time when a predetermined date or day of the week has arrived, or it may be another time.

For example, if the combination of the topic image (image of the coast of Atami) and words as shown in FIG. 15 is the first combination, the first combination is displayed on the display unit 15 as shown in FIG. In response to the voice input, when the conversation regarding the first combination is realized and the time for redisplaying the topic image arrives, the conversation corresponding to "Atami" is output as voice, as shown in FIG. 17. As shown in FIG. 15, a topic image (image of the coast of Atami) similar to that shown in FIG. You can. In the example of FIG. 17, the combination of the topic image (image of the coast of Atami) and "sea bathing" is "a combination different from the first combination." Even when the display is performed in this way, it is possible to have a conversation in the same way as the conversation that took place in the first combination, and after displaying as shown in FIG. In response to voice input of a story that includes "Kaisuiyoku" (for example, "It's that time of year for sea bathing."), a story associated with "sea bathing" (for example, "Summer is finally here!") is input by voice. You can control it so that it outputs a voice saying something like, ``I want to split watermelon on the beach in the summer.''

Furthermore, when the above-mentioned topic image re-display time comes after the conversation is performed using the second combination as shown in FIG. 17, the same topic image as in FIGS. 15 and 17 ( Atami beach image) may be displayed on the display unit 15 in a combination (third combination) different from the first and second combinations together with words associated with the topic image. For example, if images such as those shown in FIGS. 15, 17, and 18 are displayed on different days, different topics can be provided depending on the day while using the same topic image.

The example in FIG. 18 is an example in which a plurality of words prepared in association with topic images are combined. For example, if two types of words such as "Atami" and "hot spring" are prepared in advance in association with the topic image 82A (image of the coast of Atami) as shown in FIG. The topic images and words may be provided so that the two types of words (“Atami” and “hot spring”) are displayed at the same time. A method of simultaneously displaying multiple types of words prepared in association with topic images may be a method of displaying multiple types of words in one line as shown in Figure 18, or a method of displaying multiple types of words in one line as shown in Figure 19. It may also be displayed in rows. Alternatively, a method may be used in which multiple types of words are separated and displayed in different locations. The example in FIG. 19 is an example in which three types of words such as "Atami," "hot spring," and "sun beach" are prepared in advance in association with the topic image 82A (image of the coast of Atami). This is an example of displaying three types of words (``Atami'', ``hot spring'', and ``sun beach'') in multiple lines at the same time.

In the examples shown in FIGS. 18 and 19, the control unit 11 causes the display unit 15 to display a plurality of types of keywords as words related to the topic image 82A, and the input unit displays a story that includes only one of these keywords. When the input keyword is input, the output unit (for example, the audio output unit 16) operates to output a story corresponding to the input keyword. For example, after displaying the third combination as shown in FIG. 18, a story containing only "Atami" out of "Atami" and "Onsen" is input as audio to the audio input section 18 during the predetermined period. In this case, the voice output unit 16 is operated to output a voice corresponding to the input keyword (“Atami”).

Alternatively, after displaying the fourth combination as shown in FIG. 19, a story including only "Sun Beach" among "Atami", "Onsen", and "Sun Beach" (for example, "Sun Beach") is displayed during the predetermined period. When a story (such as "I know the beach") is input as voice to the voice input section 18, a story corresponding to the input keyword ("Sun Beach") (for example, "This photo is a photo of Atami Sun Beach. .It looks like a resort in a foreign country and I really like the atmosphere.'') will be outputted by the audio output section 16. This way, you will be able to dig deeper into the topic you are interested in, and the conversation will continue.

Note that in the examples shown in FIGS. 18 and 19, as in the examples shown in FIGS. 15 and 16, there is one or more corresponding stories for each of the plural types of words prepared in association with the topic image. Although it is desirable that the words be prepared, one or more corresponding stories may be prepared for a combination of two or more words selected from a plurality of types of words. For example, in the example shown in FIG. 18, not only stories corresponding to "Atami" and "hot springs" but also one or more stories corresponding to "Atami hot springs" may be prepared. In this case, when the display shown in Figure 18 is displayed, during the predetermined period, a voice that includes both "Atami" and "hot springs" (for example, "Today, let's talk about Atami Onsen") is displayed. (e.g., voice, etc.), a prepared story associated with "Atami Onsen" (for example, "There are many hotels, inns, and famous places around Atami Station. Have you ever been there?") Do you have any?”) can be output as voice. Similarly, in the example shown in Figure 19, there are not only stories corresponding to "Atami", "Onsen", and "Sun Beach", but also stories corresponding to the combination of "Atami" and "Onsen", "Atami" and " There are at least one story corresponding to the combination of "Sun Beach", a story corresponding to the combination of "Onsen" and "Sun Beach", a story corresponding to the combination of "Atami", "Onsen" and "Sun Beach", etc. It may be attached and prepared. In this case, when the display as shown in Fig. 19 is displayed, during the above predetermined period, voices containing both "Atami" and "Sun Beach" (for example, voices such as "It's Sun Beach in Atami") are heard. When input, a prepared story associated with the combination of "Atami" and "Sun Beach" may be output as audio.

In the examples shown in FIGS. 15 and 17 to 19, "topic image 82A" and "words associated with topic image 82A" are displayed, but the topic image can also be changed as shown in FIG. 20. can. After displaying the topic image 82A as shown in FIG. 15 and having a conversation, when displaying a new topic image as shown in FIG. 20, the display timing can be set variously, and the display of the previous topic image It may be that a certain period of time has passed since then, it may be that an arbitrary time, time, day of the week, etc. has arrived, or it may be that other conditions have been met. For example, in the example shown in Fig. 20, if the image shown in Fig. 20 is displayed and a voice such as "The photo has changed" is input, "This is a new topic. The text information written on the photo" is input. By outputting a message such as "Let's read aloud and talk about it" and notifying changes in photos and illustrations, it can lead to new impressions and topics, while also encouraging the reader to read out the textual information displayed in the image. . When an image like that shown in Fig. 20 is displayed and there is audio input of a story that includes the word 84D, such as "I want to go to the cherry blossom festival," the story associated with the word 84D ( For example, by outputting a phrase such as "Where is the best place to see cherry blossoms near here?", it is possible to expand the topic while having an appropriate conversation.

6. Example of Effect When displaying an image that can become a topic (a topic image different from the image of the character 70), the virtual assistant device 10 can display not only the topic image but also words related to the topic image on the display unit 15. . When such a display is made, the user who sees the topic image is more likely to utter a story that includes the above-mentioned words. Therefore, on the virtual assistant device 10 side, it is possible to prepare a response story regarding the above-mentioned words, assuming that the speech including the above-mentioned words will be given. Furthermore, if the user actually speaks a story that includes the above-mentioned words after the topic image and the above-mentioned words are displayed, the virtual assistant device 10 determines the above-mentioned words from the information input to the input section. After confirming that the word is included, a story related to the word can be output as a story from the character 70. Because such an operation is performed, it is easy to prevent the character 70 from responding to a story uttered by a user who has viewed the topic image by saying an unrelated story, and it is easy to make the conversation more appropriate.

For example, the comparative example in FIG. 21 is an example in which a coastal landscape image is displayed in the background of the character. By simply displaying an image as shown in Figure 21, the user's conversation tends to diverge into various content, such as asking various questions such as "Where is that?", "Where are you?" Some people commented, ``That's a beautiful photo.'' In addition to these, countless other topics can be considered. In such a comparative example, in order for the virtual assistant device to continue the conversation, the virtual assistant device must accurately recognize the response from the user who has viewed the image, and based on the recognition results, it must make countless assumptions. The topic (content, gist, etc.) must be accurately identified from the topic candidates presented. However, it is difficult to accurately identify the conversation points from a myriad of potential topics. We have no choice but to adopt such a method. On the other hand, the above-described virtual assistant device 10 can prevent situations where the character returns unrelated information by communicating using the words displayed on the display, so it is possible to carry out the conversation more appropriately. can.

In the example shown in FIG. 16, the correspondence data is composed of a topic image, a word corresponding to the topic image, and a story corresponding to each word. It doesn't have to be attached. In other words, there is data of an image that includes a topic image and words (for example, an image that displays a topic image and words at the same time, such as the background image in FIG. 15), and correspondence data that associates words and stories (for example, an image that displays a topic image and words at the same time, such as the background image in FIG. 16). (corresponding data excluding topic image data from the data structure) may be prepared separately. In this case, if the virtual assistant device 10 is in an environment where the corresponding data can be used, a conversation can be properly established even if the image data (image data that displays the topic image and words at the same time) is independent. can be done. For example, in a system where the image data is distributed independently, even if the virtual assistant device 10 acquires and displays the image data, there is a risk that it will not be able to recognize what kind of image will be displayed. For example, in the example shown in FIG. 21, if the user provides a conversation such as "Please tell me where the photo is," if the virtual assistant device 10 does not know the location of the photo, the character will respond with "I'm sorry. , I don't know,'' and the conversation doesn't continue or the topic doesn't expand. However, if this embodiment is applied, it is easy for the user to say something that includes the above-mentioned words, it is possible to appropriately respond based on the above-mentioned words, and it is easy to spread the topic.

Since the virtual assistant device 10 can switch between a valid mode (conversation mode) and a cancel mode, it is possible to analyze the conversation during the valid mode, and to reduce the processing load during the cancel mode. Then, when the control unit 11 causes the output unit to output a story related to the word as a story from the character 70, the control unit 11 sets a period for accepting stories from the user while continuing the effective mode, and the termination condition is satisfied. This allows the conversation to continue smoothly even after the user has provided the story related to the above-mentioned words.

As shown in FIG. 15, when the word 84A displayed in association with the topic image 82A is a single keyword, the user who sees the topic image 82A and the word 84A is more likely to utter the word 84A (keyword). Therefore, the possibility that the conversation including the word 84A (keyword) will be recognized by the virtual assistant device 10 increases, and the possibility that the conversation will continue appropriately increases.

In the example of FIG. 19, when a word containing only one of the plurality of keywords (words 84E, 84F, 84G) is uttered, the control unit 11 outputs a story corresponding to the uttered keyword. It works so that it outputs. When a plurality of types of keywords are displayed in association with a topical image, there is a high possibility that keywords that the user is likely to be interested in are included, and it becomes easier for the user to talk from more viewpoints. The virtual assistant device 10 improves the user's ease of speaking, and when a word containing any keyword is uttered, it can return a story corresponding to that keyword. It is possible to make the conversation more appropriate.

The control unit 11 displays the topic image 82A together with the word 84A in a first combination on the display unit 15 as shown in FIG. 15, and then displays the topic image 82A with the word 84B in a combination different from the first combination as shown in FIG. It also operates to display on the display unit 15. Alternatively, as shown in FIG. 18, the topic image 82A is displayed on the display unit 15 together with the word 84C in a combination different from the first combination. In this way, if it is possible to change the combination of words displayed in association when displaying the topic image 82A, the conversation will be less boring for the user even when the same type of topic image 82A is used continuously or repeatedly. This makes it easier for users to use the service. Even if the combination of the topic image and the word is changed, the virtual assistant device 10 can return a story related to the displayed word if a word that includes the displayed word is uttered. It is possible to balance the difficulty of getting bored and the appropriateness of the conversation.

<Other embodiments>
The present invention is not limited to the embodiments described above and illustrated in the drawings; for example, the following embodiments are also included within the technical scope of the present invention. Further, various features of the embodiments described above and the embodiments described below may be combined in any combination that does not contradict each other.

In the embodiment described above, the voice input unit 18 to which the user's speech can be input as voice is exemplified as an input unit into which the user's speech is input, but the input unit is not limited to this example. For example, the user's message may be input by inputting characters, and in this example, the operation unit 17 for inputting characters corresponds to an example of the input unit. In this example, the operation unit 17 may employ various known input devices that can input characters.

In the embodiment described above, when the user utters a story that includes the word displayed together with the topic image, the control unit 11 outputs the story corresponding to the word as a story from the character. control, but is not limited to this example. For example, when a user speaks a story that includes a word displayed together with a topic image, the control section 11 causes the display section 15 to output a story corresponding to this word as a story from a character. Alternatively, the display section 15 and the audio output section 16 may be controlled so as to output both text display and audio.

In the embodiment described above, the operation unit 17, which is an example of a motion detection unit, is configured as a touch panel, but the present invention is not limited to this example. The operation unit 17 may be any other known input device (keyboard, mouse, touch pen, etc.). Alternatively, the motion detection section may include another input device (for example, an input device capable of non-contact input operation) instead of or in addition to the operation section 17. Specifically, the motion detection section may include a non-contact sensor such as an imaging section or a motion sensor. In this case, the non-contact sensor and the control unit 11 may cooperate to detect the user's movements, gestures, and the like.

In the embodiment described above, the display unit 15 is configured as a display device that displays still images and moving images in two dimensions, but is not limited to this example. The display unit 15 may be a three-dimensional display capable of three-dimensional display.

In the system 1 described above, the type of information desired by the user is registered in the registration unit (storage unit 95) provided outside the virtual assistant device 10, and the control unit 11 is registered in the registration unit. Conversations regarding different types of content are controlled to be output via the interface 13. However, it is not limited to this example. For example, the virtual assistant device 10 may be provided with a registration section, and the type of information desired by the user may be registered. For example, the virtual assistant device 10 may receive from the management device 90 only content of a type registered in the virtual assistant device 10 (a type desired by the user). Alternatively, when the virtual assistant device 10 receives content from the management device 90, if the content is of the type registered in the virtual assistant device 10 (the type desired by the user), the virtual assistant device 10 performs a new distribution in step S2. It may be determined that there was. Then, in step S7, only the type of content registered in the virtual assistant device 10 (the type desired by the user) may be provided.

In the embodiment described above, an example was shown in which the virtual assistant device 10 was configured as a virtual assistant device mainly for elderly people, but the present invention is not limited to this example. For example, other categories of subjects such as children may be targeted.

In any of the examples in this specification, when having a conversation by providing topic images such as those shown in FIGS. 15, 17, 18, 19, and 20, the character's facial expressions, Display or audio output of one or more of , telops, sound effects, and icons may be generated or changed. For example, the control unit 11 may make the character 70 smile, or cause the character 70 to perform an action such as jumping or skipping. The facial expression of the adopted character is not limited to a smiling face, but may be changed to a depressed expression, an angry expression, a sad expression, etc., and the character 70 may be made to perform a happy motion, a crying motion, etc.

In the embodiment described above, the storage unit 14 is provided in the virtual assistant device 10, and the above-mentioned correspondence information is stored in the storage unit 14, but the present invention is not limited to this example. For example, correspondence information similar to the correspondence information stored in the storage unit 14 of the first embodiment may be stored in a device provided outside the virtual assistant device 10. For example, the storage unit 95 of the management device 90 may function as a storage unit that stores the above-mentioned correspondence information. Alternatively, the above-mentioned correspondence information may be provided in both the storage unit 14 and the storage unit 95. When the correspondence information is stored in an externally provided device (for example, the storage unit 95), the control unit 11 stores the data of the topic image and information associated with the topic image before displaying any topic image. It is sufficient to receive the data of the words from an external device. Further, the control unit 11 may receive the story data associated with the topic image from an external device before or after displaying the topic image, and store it in the storage unit 14.

It should be noted that the embodiments disclosed herein are illustrative in all respects and should not be considered restrictive. The scope of the present invention is not limited to the embodiments disclosed herein, and includes all modifications within the scope indicated by the claims or within the scope equivalent to the claims. is intended.

1...Virtual assistant system 10...Virtual assistant device 11...Control unit 12...Communication unit 13...Interface 14...Storage unit 15...Display unit (output unit)
16...Audio output section (output section)
17...Operation unit (input unit)
18...Audio input section (input section)
70... Character 82A... Topic image 82B... Topic image 84A... Word 84B... Word 84C... Word 84D... Word 84E... Word 84F... Word 84G... Word

Claims (6)

An input section into which a story from a user is input, a display section that displays an image, a control section that causes the display section to display an image of a character, and an output section that outputs the story from the character. A virtual assistant device,
The control unit displays a topic image different from the image of the character and words related to the topic image on the display unit, and after the topic image and the words are displayed on the display unit, a story including the words is displayed. A virtual assistant device that, when input to the input unit, causes the output unit to output a story related to the word as a story from the character. The control unit includes:
switching between an enable mode that enables analysis of the story input into the input section and a cancellation mode that cancels the enable mode;
When a story including the word is input to the input unit during the valid mode, outputting a story related to the word to the output unit as a story from the character;
When the output unit outputs a story related to the word as a story from the character, a predetermined end condition is met after a period is set for accepting stories from the user while continuing the valid mode. The virtual assistant device according to claim 1, wherein the effective mode is continued until the condition is established. The virtual assistant device according to claim 1 or 2, wherein the control unit displays only a single keyword on the display unit as the word related to the topic image. The control unit displays a plurality of types of keywords on the display unit as the words related to the topic image, and when a story containing only one of the plurality of keywords is input to the input unit, The virtual assistant device according to claim 1 or 2, wherein the output unit outputs a story corresponding to the keyword. The control unit displays the topic image together with one or more of the words in a first combination on the display unit, and then displays the topic image together with the word in a combination different from the first combination. The virtual assistant device according to claim 1 or 2, wherein the virtual assistant device is displayed on the screen. An input section into which a story from a user is input, a display section that displays an image, a control section that causes the display section to display an image of a character, and an output section that outputs the story from the character. A program used for a virtual assistant device,
causing the control unit to perform control to display a topic image different from the image of the character and words related to the topic image on the display unit;
After the topic image and the word are displayed on the display unit, when a story including the word is input to the input unit, a story related to the word is output to the output unit as a story from the character. a step of causing the control unit to perform control to
Programs for virtual assistant equipment, including.

Images