JP2021149616A - Display control unit, display control method, and program - Google Patents

Abstract

To provide a display control unit, a display control method, and a program that can provide information to a user in a more appropriate output mode.SOLUTION: A display control unit causes an output device to display an image of a character, and comprises: an image setting unit that receives settings to cause the output device to display an additional image associated with the character together with the image of the character; and an output control unit that, when the image of the character is displayed on the output device together with the additional image, causes the output device to output a content associated with the additional image preferentially over the other contents.SELECTED DRAWING: Figure 2

Description

The present invention relates to a display control device, a display control method, and a program.

Conventionally, there has been disclosed a technique of displaying a character on a vehicle display device and providing services such as information on driving support according to a user's request, vehicle control, and other applications while interacting with the user (for example). , Patent Document 1).

Japanese Unexamined Patent Publication No. 2006-335231

In recent years, due to the expansion of car sharing services, various users may use the vehicle, but it is necessary to display the characters with whom the dialogue is performed in a more appropriate output mode and provide various information according to the user. Was not considered.

The present invention has been made in consideration of such circumstances, and one of the objects of the present invention is to provide a display control device, a display control method, and a program capable of providing information in a more appropriate output mode. do.

The display control device, the display control method, and the program according to the present invention have the following configurations.
(1): The display control device according to one aspect of the present invention is a display control device that causes an output device to display an image of a character, and causes the output device to display an accompanying image associated with the character together with an image of the character. When the image of the character is displayed together with the accompanying image on the output device and the image setting unit that accepts the setting to be displayed, the content associated with the accompanying image is given priority over other contents. It is a display control device including an output control unit for outputting to the output device.

(2): In the aspect of (1) above, an information setting unit that accepts the setting of incidental information to be output from the output device in association with the accompanying image being displayed on the output device together with the image of the character. Further provided, the image setting unit enables each user to set the accompanying information.

(3): In the aspect of (1) or (2) above, the accompanying image includes one or both of an image of an ornament equipped by the character or an image of another character different from the character.

(4): In any one of the above (1) to (3), the content includes recommendation information associated with the accompanying image.

(5): In the aspect of (2) above, the recognition unit that recognizes the user who uses the output device is further provided, and the information setting unit causes the user recognized by the recognition unit to display the accompanying image. When authorized, the output device accepts the setting of the accompanying information when the image of the character is displayed together with the accompanying image.

(6): In the aspect of (5) above, the output control unit outputs the accompanying time information to the output device when the user recognized by the recognition unit has the authority to display the accompanying image. It is something that makes you.

(7): In the aspect of (2) above, the output device is a device that can be mounted on a vehicle, and the information setting unit can set the incidental information in association with the state of the vehicle. Is.

(8): In any one of the above (1) to (5), when the output control unit causes the output device to display an image of another character different from the character as the accompanying image, The content according to the number and types of characters to be displayed is displayed on the output device with priority over other contents.

(9): In the display control method according to another aspect of the present invention, the display control device for displaying the image of the character on the output device displays the accompanying image associated with the character on the output device together with the image of the character. When the image of the character is displayed together with the accompanying image on the output device, the content associated with the accompanying image is output to the output device with priority over other contents. This is a display control method.

(10): The program according to another aspect of the present invention is set so that the display control device for displaying the image of the character on the output device causes the output device to display the accompanying image associated with the character together with the image of the character. Is received, and when the image of the character is displayed together with the accompanying image on the output device, the content associated with the accompanying image is output to the output device with priority over other contents. , The program.

According to the aspects (1) to (10), information can be provided to the user in a more appropriate output mode.

It is a block diagram of the agent system 1 including the agent apparatus 100. It is a figure which shows the structure of the agent apparatus 100 which concerns on 1st Embodiment, and the apparatus mounted on the vehicle M. It is a figure which shows the arrangement example of the display / operation apparatus 20. It is a figure which shows the configuration of the agent server 200, and a part of the configuration of the agent apparatus 100. It is a figure which shows the configuration example of the agent function part 150. It is a figure which shows an example of the content of the user setting information 162. It is a figure which shows an example of the content of the accompanying image setting information 164. It is a figure which shows an example of the content of the incidental information-related DB 255. It is a figure for demonstrating the procedure which displays the agent image in the manner of wearing a predetermined ornament. It is a figure for demonstrating the state of the output mode in the 1st scene. It is a figure for demonstrating the state of the output mode in the 2nd scene. It is a figure for demonstrating the state of the output mode in the 3rd scene. It is a figure for demonstrating the state of the output mode in the 4th scene. It is a figure for demonstrating the state of the output mode in the 5th scene. It is a figure for demonstrating the state of the output mode in the 6th scene. It is a figure for demonstrating the state of the output mode in the 7th scene. It is a figure for demonstrating the state of the output mode in the 8th scene. It is a figure which shows the output example of the agent by the image control unit 152 and the output control unit 158 in a list format. It is a flowchart which shows an example of the processing flow by the agent apparatus 100. It is a flowchart which shows an example of the processing flow by the agent apparatus 100.

Hereinafter, the display control device, the display control method, and the embodiment of the program of the present invention will be described with reference to the drawings. The agent device, which is one aspect of the display control device, is a device that realizes a part or all of the agent system. Hereinafter, as an example of the agent device, an agent device mounted on a vehicle (hereinafter, vehicle M) and having one or more types of agent functions will be described. The agent function is, for example, a function that provides various information based on a request (command) included in a user's utterance or mediates a network service while interacting with a user who uses the vehicle M. be. When a plurality of types of agents are realized, the functions, processing procedures, controls, output modes, and contents performed by the plurality of types of agents may be different. In addition, some of the agent functions may have a function of controlling devices in the vehicle (for example, devices related to driving control and vehicle body control).

Agent functions include, for example, a voice recognition function that recognizes a user's voice (a function that converts voice into text), a natural language processing function (a function that understands the structure and meaning of text), a dialogue management function, and a network. It is realized by using a network search function that searches for another device or a predetermined database owned by the own device in an integrated manner. Some or all of these functions may be realized by AI (Artificial Intelligence) technology. In addition, a part of the configuration for performing these functions (particularly, the voice recognition function and the natural language processing interpretation function) is an agent server capable of communicating with the in-vehicle communication device of the vehicle M or the general-purpose communication device brought into the vehicle M. It may be mounted on (an example of an external device). In the following description, it is assumed that a part of the configuration is installed in the agent server, and the agent device and the agent server cooperate to realize the agent system. Further, a service provider (service entity) in which an agent device and an agent server cooperate to appear virtually is called an agent. The agent device 100 causes an output device (display or the like) to display an image of a character related to the agent.

<Overall configuration>
FIG. 1 is a configuration diagram of an agent system 1 including an agent device 100. The agent system 1 includes, for example, an agent device 100 and an agent server 200. The agent in the present invention is an agent realized by a service provider. Examples of service providers include automobile manufacturers, network service providers, electronic commerce businesses, sellers and manufacturers of mobile terminals, and any entity (corporation, group, individual, etc.) is the agent system 1. Can be a service provider.

The agent device 100 communicates with the agent server 200 via the network NW. The network NW includes, for example, a part or all of the Internet, a cellular network, a Wi-Fi network, a WAN (Wide Area Network), a LAN (Local Area Network), a public line, a telephone line, a wireless base station, and the like. Various web servers 300 are connected to the network NW, and the agent server 200 or the agent device 100 can acquire web pages from the various web servers 300 via the network NW.

The agent device 100 is, for example, a device that can be mounted on the vehicle M. The agent device 100 provides the user with a service in which an agent, which is a virtual subject, provides information by voice guidance using a speaker or image display using a display device. For example, the agent device 100 interacts with the user of the vehicle M, transmits the voice from the user to the agent server 200, and presents the answer obtained from the agent server 200 to the user in the form of voice output or image display. ..

[vehicle]
FIG. 2 is a diagram showing the configuration of the agent device 100 according to the embodiment and the equipment mounted on the vehicle M. The vehicle M includes, for example, one or more microphones 10, a display / operation device 20, a speaker unit 30, a navigation device 40, a vehicle device 50, an in-vehicle communication device 60, an occupant recognition device 80, and an agent device. 100 and are installed. In addition, a general-purpose communication device 70 such as a smartphone may be brought into the vehicle interior and used as a communication device. These devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 2 is merely an example, and a part of the configuration may be omitted or another configuration may be added. The display / operation device 20 is an example of a “display device”. Further, a combination of the display / operation device 20 and the speaker unit 30 is an example of the “output device”.

The microphone 10 is a sound collecting unit that collects sounds emitted in the vehicle interior. The display / operation device 20 is a device (or device group) capable of displaying an image and accepting an input operation. The display / operation device 20 includes, for example, a display device configured as a touch panel. The display / operation device 20 may further include a HUD (Head Up Display) or a mechanical input device. The speaker unit 30 includes, for example, a plurality of speakers (sound output units) arranged at different positions in the vehicle interior. The display / operation device 20 may be shared by the agent device 100 and the navigation device 40. Details of these will be described later.

The navigation device 40 includes a navigation HMI (Human machine Interface), a positioning device such as a GPS (Global Positioning System), a storage device that stores map information, and a control device (navigation controller) that performs route search and the like. .. A part or all of the microphone 10, the display / operation device 20, and the speaker unit 30 may be used as the navigation HMI. The navigation device 40 searches for a route (navigation route) for moving from the position of the vehicle M specified by the positioning device to the destination input by the user, so that the vehicle M can travel along the route. , Navi HMI is used to output guidance information. The route search function may be provided in a navigation server accessible via the network NW. In this case, the navigation device 40 acquires a route from the navigation server and outputs guidance information. The agent device 100 may be constructed based on the navigation controller. In that case, the navigation controller and the agent device 100 are integrally configured on the hardware.

The vehicle equipment 50 includes, for example, a driving force output device such as an engine or a traveling motor, an engine starting motor, a door lock device, a door opening / closing device, a window, a window opening / closing device, a window opening / closing control device, a seat, and a seat position. Control device, room mirror and its angle position control device, lighting device inside and outside the vehicle and its control device, wiper and defogger and their respective control devices, direction indicator and its control device, air conditioner, mileage and tire pressure information And vehicle information devices such as fuel level information.

The in-vehicle communication device 60 is, for example, a wireless communication device that can access the network NW using a cellular network or a Wi-Fi network.

The occupant recognition device 80 includes, for example, a seating sensor, a vehicle interior camera, an image recognition device, and the like. The seating sensor includes a pressure sensor provided at the bottom of the seat, a tension sensor attached to the seat belt, and the like. The vehicle interior camera is a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera installed in the vehicle interior. The image recognition device analyzes the image of the vehicle interior camera, recognizes the presence / absence of the user for each seat, the face of the user, and the like, and recognizes the seating position of the user. Further, the occupant recognition device 80 may identify a user seated in a driver's seat, a passenger seat, or the like included in the image by performing a matching process with a face image registered in advance.

FIG. 3 is a diagram showing an arrangement example of the display / operation device 20. The display / operation device 20 includes, for example, a first display 22, a second display 24, and an operation switch ASSY 26. The display / operation device 20 may further include a HUD 28.

The vehicle M includes, for example, a driver's seat DS provided with a steering wheel SW and a passenger seat AS provided in the vehicle width direction (-Y direction in the drawing) with respect to the driver's seat DS. The first display 22 is a horizontally long display device extending from an intermediate portion between the driver's seat DS and the passenger seat AS on the instrument panel to a position facing the left end portion of the passenger seat AS. The second display 24 is installed at the middle of the driver's seat DS and the passenger seat AS in the vehicle width direction and below the first display. For example, both the first display 22 and the second display 24 are configured as a touch panel, and are provided with an LCD (Liquid Crystal Display), an organic EL (Electroluminescence), a plasma display, and the like as display units. The operation switch ASSY26 is an integrated dial switch, button type switch, and the like. The display / operation device 20 outputs the content of the operation performed by the user to the agent device 100. The content displayed by the first display 22 or the second display 24 may be determined by the agent device 100.

[Agent device]
Returning to FIG. 2, the agent device 100 includes, for example, a management unit 110 and an agent function unit 150. The management unit 110 includes, for example, an sound processing unit 112, a setting unit 116, and a voice control unit 118. The setting unit 116 includes, for example, an image setting unit 116A and an information setting unit 116B. The software arrangement shown in FIG. 2 is simply shown for the sake of explanation, and is actually modified arbitrarily so that, for example, the management unit 110 may intervene between the agent function unit 150 and the in-vehicle communication device 60. can do.

Each component of the agent device 100 is realized, for example, by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). Some or all of these components are hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit) It may be realized by (including circuits), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or a removable storage device such as a DVD or a CD-ROM. It is stored in a medium (non-transient storage medium) and may be installed by mounting the storage medium in a drive device.

The management unit 110 functions by executing a program such as an OS (Operating System) or middleware.

In order to activate the agent, the sound processing unit 112 performs sound processing on the input sound so as to be in a state suitable for recognizing a wakeup word preset for each agent.

The image setting unit 116A accepts a setting for displaying an accompanying image associated with the character on an output device such as the display / operation device 20 together with the image of the character. Specifically, the image setting unit 116A is an image of an anthropomorphic character that communicates with a user in a vehicle interior, for example, by controlling a part of the agent function units 150 in response to an instruction from the agent function unit 150. The setting (hereinafter referred to as an agent image) is accepted, and the accepted agent image is displayed on the first display 22 or the like. The agent image is, for example, an image of a mode of talking to a user. The agent image may include, for example, a face image at least to the extent that the facial expression and face orientation are recognized by the viewer (user). For example, in the agent image, parts imitating eyes and nose are represented in the face area, and the facial expression and face orientation may be recognized based on the positions of the parts in the face area. In addition, the agent image is felt three-dimensionally, and the viewer can recognize the face orientation of the agent by including the head image in the three-dimensional space, or include the image of the main body (body and limbs). The movement (including behavior, posture, facial expression, etc.) of the character may be recognized. Further, the agent image may be an animation image. Further, the agent image may be various objects that can be anthropomorphized, such as animals, plants, insects, anime characters, and game characters.

The agent image may also be displayed with an accompanying image associated with the character. Ancillary images include, for example, one or both of ornaments or other character images. Ornaments include, for example, amulets, bills, amulets, sutras, beads, bells, medai, rosario, bonfin, plaqueruwan, statues and other religious tools, and various ornaments with religious patterns and patterns (necklaces, pendants, bracelets, etc.) Includes rings, broaches, glasses, sunglasses, piercings, earrings, hair ornaments, hats, gloves, ties, scarves, amulets, bags, umbrellas, key holders, etc.). In addition, the ornament may include, for example, clothes worn by the character (for example, suits, dresses, swimwear, ski wear), weapons, armor, and the like. Other character images include, for example, an image of a character associated with a passenger in a vehicle (for example, a occupant other than the driver), an image of a character with a motif of a child, a friend, a lover, a family, or the like. Is done. The image setting unit 116A enables, for example, to set an accompanying image for each user. Further, the accompanying image may be set by the agent server 200 or various web servers 300.

The information setting unit 116B accepts the setting of the accompanying information to be output to the output device such as the display / operation device 20 in association with the display of the accompanying image together with the character image. The accompanying information is, for example, information related to output information (for example, image or sound) or output mode associated with the accompanying image. In addition, the accompanying information may include information regarding the behavior of the character when the accompanying image is displayed. The output information includes, for example, recommendation information that is recommended to the user. For example, the information setting unit 116B is associated with the user's display of the agent image on the first display 22 together with the predetermined accompanying image, and the speaker included in the first display 22, the second display 24, and the speaker unit 30. The microphone 10, the display / operation device 20, and the like accept the setting of the accompanying information to be output from a part or all of them. Further, the information setting unit 116B may accept information regarding whether or not the user has the authority to display the accompanying image or set the output mode such as the operation of the character as the setting of the accompanying information. The information setting unit 116B enables setting of incidental information for each user who uses the agent device 100. Further, the information setting unit 116B may be able to set the accompanying information in association with the state of the vehicle. As a result, when the state of the vehicle is in the state associated with the setting in the information setting unit 116B, the accompanying information can be output from the output device. The state of the vehicle includes, for example, the number of users (occupants) of the vehicle M, the state of the vehicle interior such as who is the user sitting in the driver's seat, the vehicle is stopped, the vehicle is stopped for a predetermined time or longer, and the like. This includes the running state of the vehicle such as running, running at a predetermined speed or higher, the operating mode of the vehicle such as the vehicle operating by manual driving, and the vehicle operating by automatic driving.

The voice control unit 118 causes a part or all of the speakers included in the speaker unit 30 to output voice in response to an instruction from the agent function unit 150. The voice control unit 118 may use a plurality of speaker units 30 to control the localization of the sound image of the agent voice at a position corresponding to the display position of the agent image. The position corresponding to the display position of the agent image is, for example, a position where the user is expected to feel that the agent image is speaking the agent voice. Specifically, the position is near the display position of the agent image (for example, 2). It is within ~ 3 [cm]). Further, localization of the sound image means, for example, determining the spatial position of the sound source felt by the user by adjusting the loudness of the sound transmitted to the left and right ears of the user.

The agent function unit 150 cooperates with the agent server 200 to make an agent appear based on various information acquired by the management unit 110, and a service including a voice response according to a user's utterance existing in the vehicle. I will provide a. For example, the agent function unit 150 activates the agent associated with the wakeup word when the voice processed by the sound processing unit 112 includes the wakeup word. In addition, the agent function unit 150 controls the termination of the agent after the service is provided. Further, the agent function unit 150 transmits a voice stream processed by the sound processing unit 112 to the agent server 200, and provides the user with information obtained from the agent server 200.

Further, the agent function unit 150 may include one to which the authority to control the vehicle device 50 is given. Further, the agent function unit 150 may have a function of cooperating with the general-purpose communication device 70 and communicating with the agent server 200. In this case, the agent function unit 150 pairs with the general-purpose communication device 70 by, for example, Bluetooth (registered trademark), and connects the agent function unit 150 and the general-purpose communication device 70. Further, the agent function unit 150 may be connected to the general-purpose communication device 70 by wired communication using USB (Universal Serial Bus) or the like. The details of the function of the agent function unit 150 will be described later.

[Agent server]
FIG. 4 is a diagram showing a configuration of the agent server 200 and a part of the configuration of the agent device 100. Hereinafter, the operation of the agent function unit 150 and the like together with the configuration of the agent server 200 will be described. Here, the description of the physical communication from the agent device 100 to the network NW will be omitted.

The agent server 200 includes a communication unit 210. The communication unit 210 is a network interface such as a NIC (Network Interface Card). Further, the agent server 200 includes, for example, a voice recognition unit 220, a natural language processing unit 222, a dialogue management unit 224, a network search unit 226, and a response sentence generation unit 228. These components are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or a flash memory (a storage device including a non-transient storage medium), or a removable storage medium such as a DVD or a CD-ROM (non-transient). It is stored in a sex storage medium) and may be installed by attaching the storage medium to a drive device.

The agent server 200 includes a storage unit 250. The storage unit 250 is realized by the above-mentioned various storage devices. The storage unit 250 stores, for example, data and programs such as a personal profile 251, a dictionary DB (database) 252, a knowledge base DB 253, a response rule DB 254, and ancillary information-related DB 255.

In the agent device 100, the agent function unit 150 transmits a voice stream or a voice stream that has undergone processing such as compression or coding to the agent server 200. Further, when the agent function unit 150 recognizes a voice command capable of local processing (processing that does not go through the agent server 200), the agent function unit 150 may perform the processing requested by the voice command. The voice command that can be locally processed is a voice command that can be answered by referring to a storage unit (not shown) included in the agent device 100, or in the case of the agent function unit 150, a voice command that controls the vehicle device 50. (For example, a command to turn on the air conditioner). Therefore, the agent function unit 150 may have a part of the functions provided in the agent server 200.

When the voice stream is acquired, the voice recognition unit 220 performs voice recognition and outputs the textualized character information, and the natural language processing unit 222 interprets the meaning of the character information while referring to the dictionary DB 252. The dictionary DB 252 is associated with abstract semantic information with respect to character information. The dictionary DB 252 may include list information of synonyms and synonyms. The processing of the voice recognition unit 220 and the processing of the natural language processing unit 222 are not clearly separated in stages, and the voice recognition unit 220 corrects the recognition result in response to the processing result of the natural language processing unit 222. It may be done by influencing each other.

For example, when the meanings such as "what is the weather today" and "how is the weather" are recognized as the recognition result, the natural language processing unit 222 generates a command replaced with the standard character information "today's weather". As a result, even if there is a character fluctuation in the voice of the request, it is possible to facilitate the dialogue according to the request. Further, the natural language processing unit 222 may recognize the meaning of character information by using artificial intelligence processing such as machine learning processing using probability, or may generate a command based on the recognition result.

Based on the processing result (command) of the natural language processing unit 222, the dialogue management unit 224 refers to the personal profile 251, the knowledge base DB 253, the response rule DB 254, and the accompanying information-related DB 255, and determines the content of the utterance to the user of the vehicle M. decide. The personal profile 251 includes a user's personal information, hobbies and preferences, a history of past dialogues, and the like, which are stored for each user. The knowledge base DB 253 is information that defines the relationship between things. The response rule DB 254 is information that defines the actions (answers, device control contents, etc.) that the agent should perform in response to the command. The incidental information-related DB 255 stores information regarding an output mode when the agent image is displayed in association with a predetermined incidental image. The information regarding the output mode may include, for example, information regarding the operation of the character of the agent image and the like in addition to the display contents of the character image and the accompanying image. Details of the incidental information-related DB 255 will be described later.

Further, the dialogue management unit 224 may identify the user by collating with the personal profile 251 using the feature information obtained from the voice stream. In this case, personal information is associated with, for example, voice feature information in the personal profile 251. The voice feature information is, for example, information on the characteristics of how to speak such as voice pitch, intonation, and rhythm (sound pitch pattern), and the feature amount based on Mel Frequency Cepstrum Coefficients and the like. .. The voice feature information is, for example, information obtained by having the user utter a predetermined word or sentence at the time of initial registration of the user and recognizing the uttered voice.

The dialogue management unit 224 causes the network search unit 226 to perform a search when the command requests information that can be searched via the network NW. The network search unit 226 accesses various web servers 300 via the network NW and acquires desired information. The "information that can be searched via the network NW" is, for example, an evaluation result by a general user of a restaurant in the vicinity of the vehicle M, a weather forecast according to the position of the vehicle M on that day, or the number of passengers. It may be recommendation information according to.

Further, the dialogue management unit 224 may generate recommendation information according to an output mode such as an agent image or an accompanying image. The recommendation information includes, for example, POI (Point Of Interest) information for recommending a stop-by, and various contents such as images, videos, and sounds. In this case, the dialogue management unit 224 refers to the accompanying information-related DB 255 based on, for example, the type of the agent image to be output from the vehicle M to the user, the type of the accompanying image, and the like, and the output mode associated with the matching information. It controls such as acquiring information (for example, recommendation information) and providing it to a user, or giving a higher priority than other contents. The recommendation information included in the recommendation information DB 256 may be stored in the storage unit 250 in advance, or may be acquired from various web servers 300 via the network NW.

The response sentence generation unit 228 generates a response sentence and transmits it to the agent device 100 so that the content of the utterance determined by the dialogue management unit 224 is transmitted to the user of the vehicle M. The response sentence generation unit 228 generates a response sentence that calls the user's name or makes the speech style similar to the user's speech style when the user is identified as a user registered in the personal profile. May be good.

When the agent function unit 150 acquires the response sentence, the agent function unit 150 instructs the voice control unit 118 to perform voice synthesis and output the voice. Further, the agent function unit 150 generates an image of the agent according to the audio output, and instructs the management unit 110 to display the generated image. Further, the agent function unit 150 generates device control information for adjusting the volume output from the speaker unit 30 and controlling the device related to the vehicle device 50 (for example, an air conditioner) in accordance with the instruction from the agent server 200. Then, the generated device control information is output to the vehicle device 50 or the like. In this way, it is possible to provide the service by the agent function in which the virtually generated agent responds to the user of the vehicle M and the output mode associated with the display content of the accompanying image.

[Agent Function Department]
Next, the details of the agent function unit 150 will be described. FIG. 5 is a diagram showing a configuration example of the agent function unit 150. The agent function unit 150 includes, for example, an image control unit 152, a user recognition unit 154, an agent-related information setting unit 156, an output control unit 158, and a storage unit 160. Data and programs such as user setting information 162 and accompanying image setting information 164 are stored in the storage unit 160, for example. The user recognition unit 154 is an example of the "recognition unit".

The image control unit 152 causes the display / operation device 20 to display an agent image that provides a service including voice guidance. For example, the image control unit 152 causes the display / operation device 20 to display an image in which the agent is equipped with a predetermined accessory or an image in which the agent includes a pet or another agent.

The user recognition unit 154 recognizes a user who uses the output device. For example, the user recognition unit 154 recognizes the user operating the display / operation device 20 or interacts with the agent based on the user's movements recognized by the occupant recognition device 80, the movement of the mouth, and the like. Recognize the user. Further, the user recognition unit 154 determines whether or not the user recognized by the occupant recognition device 80 is a user who is registered in advance as a user who uses the agent device 100, or a user who has an accompanying image set in the agent image. recognize. Further, the user recognition unit 154 may recognize whether or not the user who uses the output device has the authority to display the accompanying image and set the output mode such as the operation of the character.

The agent-related information setting unit 156 is set when the user recognized by the user recognition unit 154 has the authority to set the agent image, the accompanying image, the operation of the agent, and the like. It receives various setting information from the user in cooperation with the unit 116. For example, in the agent-related information setting unit 156, when the user recognized by the user recognition unit 154 has the authority to display the accompanying image, the agent image set by the information setting unit 116B is displayed together with the accompanying image. Accepts the setting of the operation of the agent (character) in the case. The agent-related information setting unit 156 stores information such as a character type, an accompanying image type, and an output mode received from the user in the storage unit 160 as user setting information 162.

FIG. 6 is a diagram showing an example of the contents of the user setting information 162. The user setting information 162 is information in which the user ID that identifies the user is associated with the agent ID that identifies the agent image associated with the agent, the accompanying image information, and the output mode information. The accompanying image information stores, for example, an accompanying image identification ID for identifying the accompanying image and a specific name of the accompanying image. The output mode information includes incidental information. The output mode information stores, for example, the output mode of the agent image identified by the agent ID and the output mode of the accompanying image displayed accompanying the agent image. The output mode may be set, for example, by selecting one from a plurality of output patterns set in advance or registered by the user. The output pattern includes, for example, a pattern related to the facial expression or movement of the character in the agent image (for example, holding hands, waving, dancing, walking). Further, the user setting information 162 may store the condition setting regarding whether or not to display the accompanying image. The condition settings are, for example, "the image of the ornament is not displayed when another person is on board", "the image of the ornament set by the other person is not displayed (not displayed)", and "the other person is not displayed". It is a condition regarding permission or disapproval of display, such as "You may show (display) the image of the set ornament when you are on board."

In addition, the agent-related information setting unit 156 stores information about the accompanying image set by the user in the accompanying image setting information 164. FIG. 7 is a diagram showing an example of the contents of the accompanying image setting information 164. The accompanying image setting information 164 is information in which the image data and the image ID that identifies the image are associated with the accompanying image information. The type of image may include, for example, classification information for identifying whether or not a predetermined ornament may be displayed at the same time when a plurality of users are in the vehicle M. For example, since the ornaments provided by the same organization have a high affinity, they may be displayed at the same time, so they are set to the same type of ornaments. In addition, if each of the religious groups that provide the ornaments has a relationship between the head office (Hongu, Motoyama) and the branch offices (Setsu-sha, Setsusha, Sue-ji), they may be displayed at the same time because they have a high affinity. Set to the same ornament type from. In addition, non-religious ornaments may be displayed at the same time, so they are set to the same type of ornaments.

When a single user is set to display an image in which a plurality of ornaments having different types of ornaments (for example, Shinto ornaments and Buddhist ornaments) are simultaneously equipped on the agent image. The difference in the types of ornaments may be ignored.

When the user acquires the character image or the accompanying image, the agent-related information setting unit 156 may set the information stored in the user setting information 162 or the accompanying image setting information 164 regarding the acquired character image or the accompanying image. , May be set before or after the image is acquired. Further, the agent-related information setting unit 156 may change the already set information according to the user's setting. Further, the agent-related information setting unit 156 may change the user setting information 162 and the accompanying image setting information 164 based on the information acquired from the agent server 200. The user setting information 162 and the accompanying image setting information 164 may be stored in the storage unit 250 of the agent server 200.

The output control unit 158 is based on the user setting information 162 and the accompanying image setting information 164, for example, when the voice generated by the user includes a voice including a wakeup word or when responding to an inquiry from the user. , Generates an agent image and an accompanying image to be displayed on the display / operation device 20. Further, the output control unit 158 generates an agent image and an accompanying image, generates an agent voice, and generates control information about the vehicle device based on the response result and the output mode acquired from the agent server 200.

Here, the dialogue management unit 224 of the agent server 200 refers to the accompanying information-related DB 255 based on the accompanying image displayed together with the agent image displayed to the user, and determines the content to be recommended to the user. Output to the output control unit 158. FIG. 8 is a diagram showing an example of the contents of the incidental information-related DB 255. The accompanying information-related DB 255 is, for example, information in which the output information and the recommended content are associated with the accompanying image information. The output information includes, for example, information on safe driving associated with a predetermined accessory and information on display control contents of an accompanying image. For example, when a predetermined ornament is a "amulet" for traffic safety, the output information includes driving advice, a message from the gods and Buddha, a fortune-telling of traffic transportation, and the like. Also, if the given decoration is a health-related "amulet" or "accessory", the output information includes expert advice on meals, stretch information that can be done in the car, recommended restaurants and supermarkets around the destination. Information etc. are included. In addition, when a predetermined ornament is an ornament having a strong religious meaning, the output information includes information based on the religion from which the ornament is provided (for example, a priest to promote the belief of that religion, etc. Religious leaders such as priests and priests) may be included.

The output information may reflect the contents of the oath (decision, goal, etc.) registered in advance by the user. The oath is, for example, registered when the user registers a predetermined ornament or changes the setting, and like other utterances by the user, the voice recognition unit 220 of the agent server 200 and the natural language. It is analyzed by the language processing unit 222. As a result, the response sentence generation unit 228 can read the information related to the oath from the incidental information-related DB 255 and send it to the agent device 100, and can output the output information reflecting the contents of the oath to the agent. For example, if the specified accessory is a traffic safety amulet and the user has pre-registered "no accidents and no violations" as a vow, the output information should warn the vehicle M before approaching the accident-prone spot. And guidance on legal speeds are included.

In addition, a condition that reflects the content of the oath in the output information and a condition that does not reflect the content of the oath may be set together. For example, when the passenger does not want to know the personal oath, the user sets the output condition of the equipment information as "If the passenger is riding alone, the output information reflects the contents of the oath. However, if another person is on board, the content of the oath will not be reflected in the output information. "

In addition, when the accompanying image is fashion such as clothes or headphones, the output information includes information (equipment information) indicating whether or not the agent is worn or worn, and the movement of the character associated with the accompanying image is the user or the like. May be set by. For example, if the accompanying image is fashion for hiking, the character may be set to walk while wearing hiking clothes, or if the accompanying image is fashion for the sea, the character may be output information. The swimming motion is set when wearing marine equipment. When the accompanying image is another character such as a pet, the output information includes information indicating a display mode and an output pattern (for example, characters are displayed side by side) and information indicating the operation of the other character. It may be set by such as.

The recommendation content is the recommendation information provided to the user in relation to the accompanying image and the output information of the accompanying image. The recommendation information may include incidental information. The recommendation information includes, for example, information on driving (for example, driving route, driving support information), a facility that is a seller or a supervisor of the product (article) shown in the accompanying image, and a related place. For example, if the specified accompanying image is an image of a traffic safety amulet (an example of ornaments), the recommended content is the temple or shrine that provides the amulet and the temple or shrine of the same denomination as the temple or shrine that provides the amulet. include. The accompanying information-related DB 255 may be updated by the user's setting information obtained from the agent device 100.

In the example of FIG. 8, when the traffic safety amulet of the XX shrine is displayed as an accompanying image and "information on safety support (pattern 1)" is selected as the output information, the dialogue management unit 224 causes an accident. The response sentence generation unit 228 is made to generate a response sentence for recommending the temple and shrine that is the provider while giving priority to the route with less information. The route with few accidents is acquired from, for example, a management device in which the network search unit 226 manages traffic information via the network NW. When "information on safety support (pattern 2)" is selected as the output information, the dialogue management unit 224 causes the response sentence generation unit 228 to generate a response sentence for recommending information on driving support. The information on driving support includes, for example, information for adjusting the distance between the vehicle and the vehicle in front, information for suppressing lane deviation, information for instructing the timing of lane change, and the like.

Further, in the example of FIG. 8, when the dialogue management unit 224 attaches a fashion image for the sea to the agent image as an accompanying image, the dialogue management unit 224 recommends music related to the sea or recommends a route along the sea. The response sentence generation unit 228 is made to generate a response sentence for recommending a seafood restaurant.

Further, in the example of FIG. 8, when the output mode in which the headphones are attached to the agent image and displayed is set as the accompanying image, the dialogue management unit 224 recommends the information related to the music. In addition, instead of recommending (or in addition to) information about music, the dialogue management unit 224 reduces the volume of the sound output from the speaker unit 30 by a predetermined amount compared to the case where the agent image does not wear headphones. The response sentence generation unit 228 may generate information regarding the output mode such as increasing the size. Further, when an output mode in which earplugs are attached to the agent image and displayed as an accompanying image is set, the dialogue management unit 224 reduces the sound output from the speaker unit 30 (or does not output sound). Information on the output mode may be generated by the response sentence generation unit 228.

In addition, the dialogue management unit 224 recommends contents according to the case where other characters are not displayed (when only the self-agent image is displayed, or the number and attributes (for example, friends and family) of other characters to be displayed). May be changed.

The output control unit 158, for example, provides the user with content (recommendation information, etc.) related to the predetermined accompanying image with priority over other content. For example, when the output control unit 158 displays an image of another character as an accompanying image on the output device together with the agent image, the content according to the number and types of displayed characters is prioritized over the other content. And let the user provide it. For example, when the family (family) displays the display as an accompanying image, the output control unit 158 acquires restaurants and music for the family from the agent server 200 and displays them on the display unit of the display / operation device 20. Further, when the agent image is wearing an aloha shirt which is an accompanying image, the output control unit 158 displays the recommendation information of the restaurant in the Hawaiian area with priority over other recommendation information and displays the operation device 20. Display on the department. As a result, information can be provided to the user in a more appropriate output mode according to the contents such as the number and types of agent images and accompanying images.

Further, the output control unit 158 may output incidental information such as output mode information to an output device such as the display / operation device 20 when the user is a user who has the authority to display the accompanying image. For example, the output control unit 158 outputs preset information on safe driving (patterns 1 and 2 in the example of FIG. 8) when the user is set to display an image of a traffic safety amulet. It may be presented to the user as a candidate for the mode information or the output information, and the user may select from the presented output pattern candidates. As a result, the user can easily set the incidental information.

[Output mode of accompanying information]
Next, the output mode of the accompanying information in the present embodiment will be specifically described. FIG. 9 is a diagram for explaining a procedure for displaying an agent image in a manner of wearing a predetermined accessory. For example, the user of the vehicle M purchases the actual ornaments at the temples and shrines. The decorative AC purchased by the user is accompanied by a two-dimensional code for image registration, and a dedicated application can be used by using the terminal device T (for example, a smartphone or tablet) used by the user or the in-vehicle camera of the vehicle M. Read the two-dimensional code through. The dedicated application reads the VR (Virtual Reality) information acquired by reading the two-dimensional code, and realizes that the agent image A1 mounted on the vehicle is virtually equipped with the ornament image IMA.

Even if a dedicated application acquires image information that shows the actual appearance of a predetermined ornament using an in-vehicle camera or the like and generates an image of the ornament image IMA that is virtually equipped on the agent image A1. good. In addition, the user accesses the image-dedicated market site provided by the various web servers 300 via the terminal device T, and purchases the right to use the image of the decorative image IMA that can be virtually equipped on the agent image A1. You may. In the following description, the user who has the authority to display the predetermined ornament is the user who is permitted to display the ornament image IMA by purchase or transfer as described above, and the usage setting of the agent device 100 is registered. A possible (or pre-registered) user.

When the user of the agent device 100 recognized by the user recognition unit 154 has the authority to display a predetermined accessory, the image control unit 152 displays the agent image equipped with the predetermined accessory on the display / operation device 20. Display it. The output control unit 158 refers to the accompanying information-related DB 255 and provides the output information to the user when the image control unit 152 displays the agent image equipped with the predetermined ornament on the display / operation device 20. In the case of "displaying the agent image equipped with the predetermined ornament on the display / operation device 20", in addition to the state in which the agent image equipped with the predetermined accessory is already displayed on the display / operation device 20. This includes providing output information to the user that the display is about to be started (at the same time as the image display is started on the display / operation device 20 or immediately before the image display).

For example, when the user has the authority to display a predetermined accessory, the output control unit 158 can accept the setting of the accompanying information for each user, and the setting content is stored in the user setting information 162. .. Further, the output control unit 158 can set output information in association with the state of the vehicle M (which user is on board, whether there is one or more users, etc.). Hereinafter, the state of the output mode for each scene will be specifically described.

[1st scene]
FIG. 10 is a diagram for explaining the state of the output mode in the first scene. The first scene shows a scene in which the user UA is riding alone in the driver's seat DS of the vehicle M, and the accompanying image of the amulet is provided to the user UA together with the agent image A1. In the first scene, the image control unit 152 generates an image IM1 in a state where the agent image A1 is equipped with the accompanying image IMA as set in advance by the user UA, and displays the generated image IM1 of the operation device 20. Display on the display. Further, the output control unit 158 causes the speaker unit 30 to output a voice such as "This is a message from the god of the XX shrine" as the output information (information at the time of accompanying) OP1 in the first scene.

[Second scene]
FIG. 11 is a diagram for explaining the state of the output mode in the second scene. The second scene is a talisman when the user recognition unit 154 recognizes that the user UA seated in the driver's seat DS of the vehicle M and the user UB seated in the passenger seat AS are on board. It shows a scene where output information including an accompanying image is provided to users UA and UB. In the second scene, it is assumed that the users UA and UB are set to output the same agent image A1 and the same type of accompanying image IMA with the same output pattern. In this case, the image control unit 152 generates an image IM1 in which a single agent image A1 is equipped with an accompanying image IMA, and displays the generated image IM1 as the users UA and UB preset the display. -Display on the display unit of the operating device 20. Further, the output control unit 158 causes the speaker unit 30 to output a voice such as "This is a message from the god of the XX shrine" as the output information OP1 in the second scene, as in the first scene.

In the second scene, "the accompanying images are of the same type" means, for example, that the accompanying image identification IDs are the same. In addition, "the accompanying images are of the same type" means that when the accompanying images are images of ornaments such as amulets, the religious groups that sell the ornaments are the same, and the distributors of the ornaments. The religion and denomination of the ornament may be the same, the purpose of the ornament (eg, "safety", "health", etc.) may be the same, and the appearance of the ornament may be the same. Further, when the users UA and UB are set to display different ornaments, and none of the predetermined ornaments set by the users UA and UB is religious, the image control unit 152 sets the image control unit 152. , Display control may be performed assuming that the types of ornaments are the same.

Even if the users UA and UB are set to output different agent images and the same predetermined ornaments, the image control unit 152 determines each of the agent images as in the second scene. An image with ornaments may be displayed. Further, when the users UA and UB are set to display the same agent image and the same type of predetermined ornaments, and the users UA and UB have registered different oaths, the output control unit The 158 may output output information based on the oaths of the user UA and UB as incidental information, or output information based on the oath of the user UA seated in the driver's seat DS among the user UA and UB. It may be output, or the output information based on the oath may not be output. For example, when the user UA has registered an oath regarding traffic safety and the user UB has registered an oath regarding health, the output control unit 158 is related to traffic safety-related output information and health as incidental information. Output both output information.

[Third scene]
FIG. 12 is a diagram for explaining the state of the output mode in the third scene. The third scene is a scene in which the user recognition unit 154 recognizes that there is a user UA seated in the driver's seat DS of the vehicle M and a user UC seated in the passenger seat AS. In the third scene, it is assumed that the users UA and UC are set to output the same agent image and different amulet images. In this case, the image control unit 152 generates an image IM2 in which the agent image A1 is not equipped with the amulet image, and displays the generated image on the display unit of the display / operation device 20. This is because the amulet image set by the user UA and the amulet image set by the user UC belong to different types of ornaments in the accompanying image setting information 164. In this way, the agent device 100 controls so that the respective ornaments are not displayed in the case of different ornaments, so that the users UA and UC can image the ornaments of other religions that they do not anticipate. It is possible to reduce the possibility of feeling uncomfortable by seeing.

Further, in the third scene, the output control unit 158 generates the output information OP2 which is not related to each accessory and causes the speaker unit 30 to output the output information OP2. In the example of FIG. 12, the voice "Let's keep in mind safe driving" is output. As a result, the agent device 100 may feel uncomfortable when the user UA and UC hear other religious remarks that they did not anticipate, or the user UA in the vehicle M after the output information OP2 is provided. And the possibility that the atmosphere of each other of UB becomes bad can be reduced.

[Scene 4]
FIG. 13 is a diagram for explaining the state of the output mode in the fourth scene. The fourth scene is a scene in which the user recognition unit 154 recognizes that there is a user UA seated in the driver's seat DS of the vehicle M and a user UD seated in the passenger seat AS. In the fourth scene, the user UA and the UD are set to output different agent images A1 and A2, respectively, and the user UA is set to display a predetermined decoration, but the user UD is set to display a predetermined decoration. When the decoration is not set to be displayed, the image control unit 152 generates and generates the image IM3 in the state where the agent image A1 and the agent image A2 are not equipped with the predetermined decoration in the fourth scene. The image IM3 is displayed on the display unit of the display / operation device 20. Further, the output control unit 158 causes the speaker unit 30 to output the output information OP2 that is not related to the predetermined ornament.

Thereby, the agent device 100 can reduce the possibility that the user UA and the user UD feel uncomfortable when they see the ornaments of other religions that they did not anticipate. Further, the agent device 100 feels uncomfortable when the user UA and the user UD hear other religious remarks that they do not anticipate, or the atmosphere in the vehicle M after the output information OP2 is provided. It is possible to reduce the possibility of making it worse.

If the user UA and the user UD allow other users to display ornaments of other religions, the image control unit 152 determines the agent image A1 and the agent image A2, respectively. You may display an image equipped with the ornaments of.

[Scene 5]
FIG. 14 is a diagram for explaining the state of the output mode in the fifth scene. The fifth scene is a diagram showing a scene in which a predetermined operation is performed on the agent image when the agent image is displayed together with the accompanying image. For example, when the user recognition unit 154 recognizes that there is a user UA seated in the driver's seat DS of the vehicle M, or when the user UA is alone in the driver's seat DS of the vehicle M, it is a talisman. The scene where the accompanying image is provided to the user UA together with the agent image A1 is shown. In the fifth scene, the image control unit 152 generates an image IM4 in which the agent image A1 is equipped with the accompanying image IMA as set in advance by the user UA, and displays / operates the generated accompanying image IM4. It is displayed on the display unit of 20. Here, when the user UA has set the action of the characters of the agent image A1 to join hands as incidental information, the image control unit 152 generates an image in which the characters of the agent image A1 perform the actions of joining hands. .. The accompanying image IM4 may be an animation image. In addition, the output control unit 158 outputs a voice such as "This is a message from the god of the XX shrine" as the output information OP1 in the first scene. As a result, the agent image can be made to perform a predetermined operation according to the accompanying image displayed together with the agent image, so that the user can be made familiar with the agent.

[Scene 6]
FIG. 15 is a diagram for explaining the state of the output mode in the sixth scene. The sixth scene is different from the fifth scene described above in that the user seated in the driver's seat DS is not the user UA but the user UE. In this case, when the user UE recognized by the user recognition unit 154 is recognized, the image control unit 152 is in a state where the agent image A1 is equipped with the accompanying image IMA as set in advance by the user UE. The image IM5 is generated, and the generated image IM5 is displayed on the display unit of the display / operation device 20. Here, when the user UE has set an operation of bowing the character of the agent image A1 as incidental information, the image control unit 152 generates an image in which the character of the agent image A1 bows. The image IM5 may be an animation image in which the character of the agent image A1 bows.

As shown in the fifth and sixth scenes, by setting the operation as incidental information set for each user, it is possible to make the agent execute an appropriate operation for each user.

Further, the output control unit 158 may control the character so as not to perform an operation when a plurality of users are recognized by the user recognition unit 154 and each user sets a different operation. good. Further, the output control unit 158 may display the movements of the respective characters in synchronization or cooperation when the movements of the respective characters are set to be common or highly related. For example, the output control unit 158 may cause a plurality of characters to wave their hands together, walk, shake hands, or jump alternately. In this way, by operating the characters in synchronization or cooperation with each other, it is possible to support the deepening of friendship between the users.

[Scene 7]
FIG. 16 is a diagram for explaining the state of the output mode in the seventh scene. The seventh scene shows a scene of recommending to the user according to the incidental information associated with the agent. In the seventh scene, it is assumed that the user UA is seated in the driver's seat of the vehicle M. When the user UA recognized by the user recognition unit 154 is recognized, the image control unit 152 displays the image IM6 in which the agent image A1 wears a fashion for the sea as preset by the user UA. Let me. Here, the output control unit 158 acquires the recommendation information about the sea from the agent server 200, and outputs the acquired recommendation information to the user with priority over other recommendation information.

In the example of FIG. 16, when the user UA inquires the agent about the traveling route to the destination, among the routes obtained as the inquiry result, the agent image is fashioned for the sea, so that the user travels along the sea. Priority is given to the route to be displayed. Further, the output control unit 158 outputs a sound such as "Traveling along the route along the sea" as the output information OP3 in the seventh scene. According to the scene of FIG. 7, the recommendation information associated with the accompanying image set by the user can be provided to the user with priority over other recommendation information.

[Eighth scene]
FIG. 17 is a diagram for explaining the state of the output mode in the eighth scene. The eighth scene shows a scene where the recommendation information is output when there are a plurality of users recognized by the user recognition unit 154. In the example of FIG. 17, it is assumed that the user UA is seated in the driver's seat DS and the user UD is seated in the passenger seat AS as in FIG. In this case, the image control unit 152 displays the agent images A1 and A2 corresponding to the respective users UA and UD, and also displays tourist spots and the like that can be enjoyed by two people as recommendation information when two agents are displayed. Make a recommendation. In addition, the output control unit 158 outputs a sound such as "Introducing a sightseeing spot that can be enjoyed by two people" as the output information OP4 in the eighth scene.

As a result, more appropriate recommendation information can be preferentially provided to the user according to the user on which the vehicle M is on board and the displayed character image. As a result, more appropriate information can be provided to the user.

FIG. 18 is a diagram showing an output example of an agent image by the image control unit 152 and the output control unit 158 in a list format. The information shown in FIG. 18 is used, for example, when controlled by the image control unit 152 and the output control unit 158. In the example of FIG. 18, the output image is associated with information indicating whether or not the agents are the same, information indicating a group of ornaments, and information indicating whether or not to display the ornaments of another person. .. For example, when there are a plurality of users recognized by the user recognition unit 154, the agent images to be displayed are the same, and the type of the predetermined ornament is the same (in the case of No. 1 shown in FIG. 18). The image control unit 152 causes the display / operation device 20 to display an image in which the agent image A1 is equipped with a predetermined accessory image IMA. Similarly, when there is only one user recognized by the user recognition unit 154 and it is set to display a predetermined ornament, the agent image A1 generates an image equipped with the predetermined ornament image IMA. , The generated image is displayed on the display / operation device 20. At this time, the output control unit 158 causes the agent to provide incidental information regarding the equipment.

Further, there are a plurality of users recognized by the user recognition unit 154, the agent images to be displayed are the same, and the types of predetermined ornaments are not the same, and at least one of the users is the decoration of another person. When the display of the product image is not permitted (in the case of No. 2 shown in FIG. 18), the image control unit 152 displays an image in which the agent image A1 is not equipped with the predetermined ornament image IMA. Display / Display on the operation device 20. At this time, the output control unit 158 does not allow the agent to provide the incidental information as a service.

Further, there are a plurality of users recognized by the user recognition unit 154, the agent images to be displayed are the same, and the predetermined types of ornaments are not the same, and all the users are the ornaments of others. When the image display is set to be permitted (No. 3 shown in FIG. 18), the image control unit 152 superimposes the agent image A1 on the predetermined decorative image IMA set by each user. The image is displayed on the display / operation device 20. At this time, the output control unit 158 may cause the agent to provide output information related to each of the predetermined ornaments as a service, or may cause the agent to provide only the equipment-time information set by any user as a service. It is okay, and it is not necessary to provide information at the time of equipment.

On the other hand, when there are a plurality of users recognized by the user recognition unit 154, the agent images to be displayed are different, and the type of the predetermined ornament is the same (in the case of No. 4 shown in FIG. 18), the image. The control unit 152 causes the display / operation device 20 to display an image in which each of the agent image A1 and the agent image A2 is equipped with a predetermined ornament image IMA. At this time, the output control unit 158 may cause the agent to provide output information related to each of the predetermined ornaments as a service, or may cause the agent to provide only the equipment-time information set by any user as a service. It is okay, and it is not necessary to provide information at the time of equipment.

Further, when there are a plurality of users recognized by the user recognition unit 154, the agent images to be displayed are different, and the predetermined types of ornaments are not the same, at least one of the users is an ornament image of another person. When the display of is not permitted (in the case of No. 5 shown in FIG. 18), the image control unit 152 does not equip the agent image A1 and the agent image A2 with the predetermined ornament image IMA. The image is displayed on the display / operation device 20. At this time, the output control unit 158 does not allow the agent to provide the equipped information as a service.

Further, there are a plurality of users recognized by the user recognition unit 154, the agent images to be displayed are different, and all the users are set to allow the display of the ornament image of another person, and one of the agents. Is a setting to equip a predetermined ornament, and when the other agent is set not to equip the ornament (in the case of No. 6 shown in FIG. 18), the image control unit 152 sets the agent image A1 by the user. Equipped with the predetermined decorative image IMA, the agent image A2 causes the display / operation device 20 to display an image without the ornament. At this time, the output control unit 158 may or may not have the agent provide the equipped time information as a service, or may not provide the equipped time information.

Further, there are a plurality of users recognized by the user recognition unit 154, the agent images to be displayed are different, and all the users are set to allow the display of the ornament image of another person, and each agent. When the image is set to be equipped with different predetermined decorations (in the case of No. 7 shown in FIG. 18), the image control unit 152 has different predetermined decorations in which the agent image A1 and the agent image A2 are set by the user. Product image An image equipped with IMA is displayed on the display / operation device 20. At this time, the output control unit 158 may cause the agent to provide output information related to each of the predetermined ornaments as a service, or may cause the agent to provide only the equipment-time information set by any user as a service. It is okay, and it is not necessary to provide information at the time of equipment.

In addition to the information shown in FIG. 18, the list format of the output example may include an output image associated with a group of attached images other than ornaments and information on whether or not to allow display. .. In addition, the output image shown in FIG. 18 may include information regarding the operation of the agent (for example, holding hands, bowing, waving, dancing, walking, etc.).

[Processing flow]
Next, the processing flow in the agent device 100 will be described. In the following, the setting process for accepting the setting of the agent and the accompanying image by the user and the display control process for the agent based on the set information will be described separately.

FIG. 19 is a flowchart showing an example of the flow of the setting process by the agent device 100. The processing flow shown in FIG. 19 is performed, for example, before the user gets on the vehicle M, or at the timing when the user purchases a predetermined accessory.

First, the agent device 100 accepts the user's setting of the display agent (step S100). Next, the agent-related information setting unit 156 accepts registration of incidental information to be displayed together with the image of the agent to be displayed on the display / operation device 20 (step S102). The accompanying information includes, for example, registration of an article such as a purchased accessory or clothes, and an accompanying image relating to the registered article. In addition, information on the user's oath may be registered together with the accompanying information. Next, the output control unit 158 presents candidates for incidental information such as output information and output mode with reference to the incidental information-related DB 255 (step S104), accepts the user's selection of incidental information, and user setting information. It is reflected in 162 and the accompanying information-related DB 255 (step S106). As a result, the processing of this flowchart ends.

FIG. 20 is a flowchart showing an example of the flow of display control processing by the agent device 100. The processing flow shown in FIG. 20 is, for example, recognized by a new user during the process of displaying the agent image by the agent function unit 150 and providing the service, the timing of starting the display of the agent image, and the occupant recognition device 80. It is done at the timing that was done. In the following, an image of an ornament will be used as an example of the accompanying image.

First, the user recognition unit 154 calls the user setting information 162 of the user recognized by the occupant recognition device 80 (step S200). Next, the user recognition unit 154 determines whether or not there is only one recognized user (step S202). When it is determined that there is only one user, the image control unit 152 starts displaying an image in which the agent image is equipped with a predetermined accessory according to the user's preset stored in the user setting information 162. The output control unit 158 provides information based on the setting of the output information (step S204). In the process of step S204, the output control unit 158 controls the agent to perform a predetermined operation according to the accessory image displayed together with the agent image, and preferentially outputs the recommendation information according to the accessory image. May be done.

Further, in the process of step S202, when it is determined that the recognized user is not one person (there are a plurality of people), the image control unit 152 determines the type of decoration set by each of the recognized users. It is determined whether or not they are the same (step S206). In the process of step S206, "the ornaments are the same" may include that the ornaments are completely the same and that the types of the ornaments are the same (or similar). Further, in the process of step S206, if at least one of the ornaments is not religious, it may be determined that the ornaments are the same.

If it is determined in the process of step S206 that the ornaments are the same, the image control unit 152 determines to display the agent image equipped with the ornament (step S208). When it is determined in the process of step S206 that the ornaments are not of the same type, the image control unit 152 refers to the user setting information 162 and the like, and determines whether or not the user allows the display of the ornament image. (Step S210). “Allow display” indicates a case where the output mode information of the user setting information 162 is set to display an accompanying image with some output pattern. If the display is allowed, the image control unit 152 proceeds to step S208. If the display is not allowed, the image control unit 152 determines to display the agent image without the ornament (step S212).

After the processing of step S208 or step S212, the image control unit 152 determines whether or not to display the same agent (step S214). When it is determined that the same agent image is to be displayed, the image control unit 152 determines to display a single agent image on the display / operation device 20 (step S216). When it is determined in step S214 that different agent images are to be displayed, the image control unit 152 determines to display a plurality of agent images on the display / operation device 20 (step S218). After the process of step S216 or step S218, the image control unit 152 proceeds to step S204. This is the end of the description of the processing of this flowchart.

According to the agent device 100 of the embodiment described above, information can be provided to the user in a more appropriate output mode. Further, according to the present embodiment, the user can reflect his / her mood, preference, etc. in the vehicle recommendation system by selecting an agent or an accessory. Further, according to the embodiment, by attaching a decorative item to the agent image, the user can set the operation content of the agent image for each decorative item. In addition, the agent image can display an image in which the ornament is equipped. As a result, the user can virtually carry an ornament such as an amulet. In addition, this frees the user from the hassle of forgetting to carry the ornament or leaving the ornament in the vehicle.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

10 Microphone 20 Display / operation device 30 Speaker unit 40 Navigation device 50 Vehicle equipment 60 In-vehicle communication device 70 General-purpose communication device 80 Crew recognition device 100 Agent device 110 Management unit 112 Sound processing unit 114 Display control unit 116 Setting unit 116A Image setting unit 116B Information setting unit 118 Voice control unit 150 Agent function unit 152 Image control unit 154 User recognition unit 156 Agent-related information setting unit 158 Output control unit 160, 250 Storage unit 200 Agent server 220 Voice recognition unit 222 Natural language processing unit 224 Dialogue management unit 226 Network search unit 228 Response statement generation unit

Claims (10)

A display control device that displays a character image on an output device.
An image setting unit that accepts a setting for displaying an accompanying image associated with the character on the output device together with the image of the character.
When the image of the character is displayed together with the accompanying image on the output device, the output control unit that outputs the content associated with the accompanying image to the output device with priority over other contents. ,
A display control device. The output device further includes an information setting unit that accepts the setting of the accompanying information to be output from the output device in association with the display of the accompanying image together with the image of the character.
The image setting unit enables the setting of the accompanying information for each user.
The display control device according to claim 1. The accompanying image includes one or both of an image of an ornament equipped by the character or an image of another character different from the character.
The display control device according to claim 1 or 2. The content includes recommendation information associated with the accompanying image.
The display control device according to any one of claims 1 to 3. A recognition unit that recognizes a user who uses the output device is further provided.
The information setting unit is used when the user recognized by the recognition unit has the authority to display the accompanying image, and when the image of the character is displayed together with the accompanying image on the output device. Accept information settings,
The display control device according to claim 2. The output control unit causes the output device to output the incidental information when the user recognized by the recognition unit has the authority to display the accompanying image.
The display control device according to claim 5. The output device is a device that can be mounted on a vehicle.
The information setting unit makes it possible to set the incidental information in association with the state of the vehicle.
The display control device according to claim 2. When the output control unit displays an image of another character different from the character as the accompanying image on the output device, the output control unit gives priority to the content according to the number and types of the displayed characters over the other content. Displayed on the output device,
The display control device according to any one of claims 1 to 7. The display control device that displays the character image on the output device
The output device accepts a setting to display an accompanying image associated with the character together with the image of the character.
When the image of the character is displayed together with the accompanying image on the output device, the content associated with the accompanying image is output to the output device with priority over other contents.
Display control method. For display control devices that display character images on output devices,
The output device is made to accept the setting to display the accompanying image associated with the character together with the image of the character.
When the image of the character is displayed together with the accompanying image on the output device, the content associated with the accompanying image is output to the output device with priority over other contents.
program.

Images