JP7252029B2 - SERVER DEVICE, INFORMATION PROVISION METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to a server device, an agent device, an information providing method, and a program.

It is known in the prior art to provide production-related data in which a camera is used to record an image of a defect in a motor vehicle and to transmit this recorded image to at least one central device during the manufacturing process (e.g. US Pat. reference).

Japanese Patent Publication No. 2004-505337

However, the conventional technology described above is intended to provide information to vehicle manufacturers, and does not consider providing information to users of vehicle delivery destinations.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a server device, an agent device, an information providing method, and a program capable of providing useful information to users. one.

A server device, an agent device, an information providing method, and a program according to the present invention employ the following configurations.

(1): A server device according to an aspect of the present invention includes an acquisition unit that acquires a manufacturing process of a vehicle associated with a user who is a delivery destination of the vehicle, and acquires the manufacturing process acquired by the acquisition unit. and a transmission unit configured to transmit the information to the terminal device of the user.

(2): In the aspect of (1) above, the manufacturing process includes an assigning process of assigning a name associated with the vehicle, and setting the name to the vehicle in response to a request from the user. It further comprises a part.

(3): In the aspect of (2) above, the manufacturing process includes the step of giving the name to an agent device that is mounted on the vehicle and provides services including voice responses in response to voices. and the setting unit sets the name to the agent device in response to a request from the user.

(4): In the above aspect (3), the agent device is activated when the name is input by voice.
It is.

(5): In any one of the above aspects (1) to (4), the acquisition unit acquires the user's post, and the transmission unit responds to the post acquired by the acquisition unit, The manufacturing process is transmitted to the terminal device of the user in association with an agent device that provides a service including a response by voice in response to the voice or the vehicle.

(6): In any one of the above aspects (1) to (5), the manufacturing process includes starting assembly of the vehicle, finishing assembly of the vehicle, starting painting the vehicle, or painting the vehicle. and the transmission unit transmits an icon, an image, or a deformed image corresponding to the manufacturing process to the terminal device of the user.

(7): In any one of the aspects (1) to (6) above, the transmitting unit transmits the information indicating the manufacturing process to the terminal device of the user according to the real-time manufacturing process of the vehicle. It is something to do.

(8): A server device according to an aspect of the present invention includes an acquisition unit that acquires a request for a wake-up word for an agent device from a user, and a setting that sets the wake-up word acquired by the acquisition unit in the agent device. and the agent device is activated when the user's utterance matches a preset wake-up word, and provides a service including a voice response in response to the user's utterance. The setting unit performs processing for manufacturing the agent device, until the agent device is delivered to the user, during the manufacturing process of the target on which the agent device is installed, and during the period until the agent device is delivered to the user. The server device sets the wake-up word in response to a request from the user until the target is delivered to the user.

(9): An agent device according to an aspect of the present invention includes an acquisition unit that acquires an utterance of a vehicle user, and the user's utterance acquired by the acquisition unit matches a preset wake-up word. and an agent function unit that is activated when the user speaks and performs processing for providing a service including a voice response in response to the user's utterance acquired by the acquisition unit, The wake-up word includes a manufacturing process of the agent device, a period until the agent device is delivered to the user, a manufacturing process of the target on which the agent device is mounted, and a target on which the agent device is mounted. The agent device is set by the user who uses the agent device until it is delivered to the user.

(10): An information providing method according to an aspect of the present invention is such that a computer acquires a manufacturing process of a vehicle associated with a user to whom the vehicle is delivered, and transmits information indicating the acquired manufacturing process. It is an information providing method for transmitting to the terminal device of the user.

(11): A program according to an aspect of the present invention causes a computer to acquire a manufacturing process of a vehicle associated with a user who is a delivery destination of the vehicle, and uses the acquired information indicating the manufacturing process. It is a program for transmitting to the terminal device of the person.

According to (1), (5) to (7), (10), and (11), the server device provides useful information to the user by providing the user with information indicating the manufacturing process. can be done.

According to (2) to (4), by setting the name to the vehicle by the setting unit, the user's convenience is improved, and the user's attachment to the vehicle is improved.

According to (8), the setting unit sets the wake-up word in response to the user's request, thereby improving the user's convenience and increasing the user's attachment to the vehicle.

According to (9), the agent device is activated in response to a wake-up word set in advance, so that the user's convenience is improved and the user's attachment to the vehicle is improved.

1 is a configuration diagram of an agent system 1 including an agent device 100; FIG. 3 is a diagram showing an example of a functional configuration of a general-purpose communication device 70; FIG. 1 is a diagram showing the configuration of an agent device 100 and devices mounted on a vehicle M according to the first embodiment; FIG. FIG. 2 is a diagram showing an arrangement example of a display/operation device 20; 3 is a diagram showing an example of arrangement of speaker units 30. FIG. 2 is a diagram showing the configuration of an agent server 200 and part of the configuration of an agent device 100; FIG. It is a figure which shows an example of a functional structure of the information provision apparatus 300 (server apparatus). 4 is a sequence diagram showing an example of the flow of processing executed by the agent system 1; FIG. 7 is a diagram showing an example of a reception image IM1 displayed on a display unit 71; FIG. 3 is a diagram showing an example of contents of vehicle information 324. FIG. 4 is a sequence diagram showing an example of the flow of processing executed by the agent system 1; FIG. FIG. 10 is a diagram showing an example of an image displayed on the display unit 71 by the processing of steps S200 to S208; FIG. FIG. 10 is a diagram showing an example of an image displayed on the display unit 71 by the processing of steps S210 to S214; FIG. FIG. 10 is a diagram showing an example of an image displayed on the display unit 71 by the processing of steps S216 to S224; 3 is a diagram showing an example of contents of vehicle information 324 generated by an information providing device 300. FIG. 4 is a flow chart showing an example of the flow of processing executed by the agent device 100; FIG. 4 is a diagram showing an example of how the agent device 100 is activated by a wakeup word set in the manufacturing process; FIG. 13 is a diagram showing an example of the functional configuration of an agent server 200A according to the second embodiment; FIG. FIG. 11 is a sequence diagram showing an example of the flow of processing executed by the agent system 1 of the second embodiment;

Hereinafter, embodiments of a server device, an agent device, an information providing method, and a program according to the present invention will be described with reference to the drawings.

<First Embodiment>

An agent device is a device that implements part or all of the agent system. As an example of the agent device, an agent device installed in a vehicle (hereinafter referred to as vehicle M) and having multiple types of agent functions will be described below. The agent function is, for example, a function of providing various types of information based on requests (commands) contained in the user's utterances while interacting with the user of the vehicle M, and mediating network services. be. A plurality of types of agents may have different functions, processing procedures, controls, and output modes/contents. In addition, the agent function may include a function of controlling devices in the vehicle (for example, devices related to operation control and vehicle body control).

Agent functions include, for example, a voice recognition function that recognizes the user's voice (a function that converts voice into text), a natural language processing function (a function that understands the structure and meaning of text), a dialogue management function, and a network It is realized by comprehensively using a network search function or the like for searching for other devices using the device or for searching a predetermined database held by the device itself. Some or all of these functions may be realized by AI (Artificial Intelligence) technology. Also, part of the configuration for performing these functions (in particular, the voice recognition function and the natural language processing and interpretation function) is an agent server capable of communicating with an in-vehicle communication device of the vehicle M or a general-purpose communication device brought into the vehicle M. It may be mounted on (an external device). The following description assumes that part of the configuration is installed in the agent server, and that the agent device and the agent server work together to realize the agent system.

<Overall composition>
FIG. 1 is a configuration diagram of an agent system 1 including an agent device 100. As shown in FIG. The agent system 1 includes, for example, a general-purpose communication device 70, agent devices 100-1 and 100-2, a plurality of agent servers 200-1, 200-2, 200-3, . A terminal device 400 and a store terminal device 450 are provided. When the agent devices 100-1 and 100-2 are not distinguished, they may simply be referred to as the agent device 100. FIG. It is assumed that the number following the hyphen at the end of the code of the agent server 200 is an identifier for distinguishing agents. It may simply be referred to as agent server 200 when it is not distinguished which agent server it is. Although three agent servers 200 are shown in FIG. 1, the number of agent servers 200 may be two, or four or more. Each agent server 200 is operated by a different agent system provider. Therefore, agents in the present invention are agents implemented by different providers. Providers include, for example, automobile manufacturers, network service providers, e-commerce companies, and mobile terminal sellers, and any entity (corporation, organization, individual, etc.) can serve as the provider of the agent system.

Agent device 100 communicates with agent server 200 via network NW. The network NW includes, for example, some or all of the Internet, cellular network, Wi-Fi network, WAN (Wide Area Network), LAN (Local Area Network), public line, telephone line, wireless base station, and the like. Various web servers 500 are connected to the network NW, and the agent server 200 or the agent device 100 can acquire web pages from the various web servers 500 via the network NW.

The agent device 100 communicates with the user of the vehicle M, transmits the voice of the user to the agent server 200, and presents the answer obtained from the agent server 200 to the user in the form of voice output or image display. do.

[General-purpose communication device]
FIG. 2 is a diagram showing an example of the functional configuration of the general-purpose communication device 70. As shown in FIG. The general-purpose communication device 70 is a portable or portable device such as a smart phone or a tablet terminal. The general-purpose communication device 70 includes, for example, a display unit 71, a speaker 72, a microphone 73, a communication unit 74, a pairing execution unit 75, a sound processing unit 76, a control unit 77, and a storage unit 78. . The storage unit 78 stores a manufacturing monitoring application program (manufacturing monitoring application 79). The production monitoring application 79 is provided by, for example, an application providing server (not shown).

The manufacturing monitoring application 79 transmits information acquired by the general-purpose communication device 70 to the agent device 100 or uses information transmitted by the agent device 100 based on the operation performed by the user on the general-purpose communication device 70 . provide it to a person.

The display unit 71 includes a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display. The display section 71 displays an image based on the control of the control section 77 . The speaker 72 outputs sound under the control of the control section 77 . A microphone 73 collects the voice input by the user.

The communication unit 74 is a communication interface for communicating with the agent device 100 .
The pairing executing unit 75 executes pairing with the agent device 100 using wireless communication such as Bluetooth (registered trademark), for example. The acoustic processing unit 76 performs acoustic processing on the input sound.

The control unit 77 is implemented by a processor such as a CPU (Central Processing Unit) executing a manufacturing monitoring application 79 (software). The control unit 77 controls each unit of the general-purpose communication device 70 (for example, the display unit 71, the speaker 72, etc.). The control unit 77 manages information input to its own device and information provided by the agent device 100 . Further, the control unit 77 performs processing for participating in a chat service provided by the information providing device 300 in accordance with the user's operation.

[vehicle]
FIG. 3 is a diagram showing the configuration of the agent device 100 and equipment mounted on the vehicle M according to the first embodiment. The vehicle M includes, for example, one or more microphones 10, a display/operation device 20, a speaker unit 30, a navigation device 40, a vehicle device 50, an in-vehicle communication device 60, an occupant recognition device 80, and an agent device. 100 and a storage unit 130 are mounted. Also, the general-purpose communication device 70 may be brought into the vehicle and used as a communication device. These devices are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, and the like. Note that the configuration shown in FIG. 2 is merely an example, and a part of the configuration may be omitted, or another configuration may be added.

The microphone 10 is a sound pickup unit that collects sounds emitted inside the vehicle. The display/operation device 20 is a device (or device group) that displays images and can accept input operations. The display/operation device 20 includes, for example, a display device configured as a touch panel. The display/operation device 20 may further include a HUD (Head Up Display) or a mechanical input device. The speaker unit 30 includes, for example, a plurality of speakers (sound output units) arranged at different positions in the vehicle interior. The display/operation device 20 may be shared by the agent device 100 and the navigation device 40 . Details of these will be described later.

The navigation device 40 includes a navigation HMI (Human Machine Interface), a positioning device such as a GPS (Global Positioning System), a storage device that stores map information, and a control device (navigation controller) that performs route search and the like. . A part or all of the microphone 10, the display/operation device 20, and the speaker unit 30 may be used as the navigation HMI. The navigation device 40 searches for a route (navigation route) for moving from the position of the vehicle M specified by the positioning device to the destination input by the user, and makes the vehicle M travel along the route. Then, guidance information is output using the navigation HMI. The route finding function may reside in a navigation server accessible via the network NW. In this case, the navigation device 40 acquires a route from the navigation server and outputs guidance information. The agent device 100 may be constructed on the basis of the navigation controller, in which case the navigation controller and the agent device 100 are integrated in terms of hardware.

The vehicle equipment 50 includes, for example, a driving force output device such as an engine and a running motor, an engine starting motor, a door lock device, a door opening/closing device, windows, a window opening/closing device and a window opening/closing control device, a seat, and a seat position control device. Control devices, rearview mirrors and their angular position control devices, lighting devices inside and outside the vehicle and their control devices, wipers and defoggers and their respective control devices, direction indicator lights and their control devices, air conditioners, mileage and tire pressure information and information on the remaining amount of fuel, etc.

The vehicle-mounted communication device 60 is, for example, a wireless communication device that can access the network NW using a cellular network or a Wi-Fi network.

The occupant recognition device 80 includes, for example, a seat sensor, an in-vehicle camera, an image recognition device, and the like. Seating sensors include a pressure sensor provided under the seat, a tension sensor attached to the seat belt, and the like. The vehicle interior camera is a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera provided in the vehicle interior. The image recognition device analyzes the image captured by the camera inside the vehicle and recognizes the presence or absence of a user for each seat and the orientation of the face. In this embodiment, the occupant recognition device 80 is an example of a seating position recognition section.

FIG. 4 is a diagram showing an arrangement example of the display/operation device 20. As shown in FIG. The display/operation device 20 includes, for example, a first display 22, a second display 24, and an operation switch ASSY26. The display/operation device 20 may further include a HUD 28 .

The vehicle M has, for example, a driver's seat DS provided with a steering wheel SW and a passenger's seat AS provided in the vehicle width direction (Y direction in the figure) with respect to the driver's seat DS. The first display 22 is a horizontally long display device that extends from the middle of the instrument panel between the driver's seat DS and the passenger's seat AS to a position facing the left end of the passenger's seat AS. The second display 24 is installed below the first display 22 and midway between the driver's seat DS and the front passenger's seat AS in the vehicle width direction. For example, both the first display 22 and the second display 24 are configured as touch panels, and have LCDs (Liquid Crystal Displays), organic ELs (Electroluminescence), plasma displays, etc. as display units. The operation switch ASSY 26 is a combination of dial switches, button switches, and the like. The display/operation device 20 outputs the content of the operation performed by the user to the agent device 100 . The content displayed by the first display 22 or the second display 24 may be determined by the agent device 100 .

FIG. 5 is a diagram showing an arrangement example of the speaker units 30. As shown in FIG. The speaker unit 30 includes, for example, speakers 30A-30H. The speaker 30A is installed on a window pillar (so-called A pillar) on the driver's seat DS side. The speaker 30B is installed under the door near the driver's seat DS. The speaker 30C is installed on the window pillar on the side of the passenger seat AS. The speaker 30D is installed under the door near the passenger seat AS. The speaker 30E is installed under the door near the right rear seat BS1 side. The speaker 30F is installed under the door near the left rear seat BS2. The speaker 30G is installed near the second display 24 . The speaker 30H is installed on the ceiling (roof) of the passenger compartment.

In such an arrangement, for example, if the speakers 30A and 30B exclusively output sound, the sound image is localized near the driver's seat DS. Further, when the sound is output exclusively from the speakers 30C and 30D, the sound image is localized near the front passenger seat AS. Further, when the sound is exclusively output from the speaker 30E, the sound image is localized near the right rear seat BS1. Further, when the sound is exclusively output from the speaker 30F, the sound image is localized near the left rear seat BS2. Further, when the sound is exclusively output from the speaker 30G, the sound image is localized near the front of the vehicle compartment, and when the sound is exclusively output from the speaker 30H, the sound image is localized near the upper part of the vehicle compartment. Become. Not limited to this, the speaker unit 30 can localize a sound image at an arbitrary position in the vehicle compartment by adjusting distribution of sound output from each speaker using a mixer or an amplifier.

[Agent device]
Returning to FIG. 3, agent device 100 includes management unit 110, agent function units 150-1, 150-2, and 150-3, and pairing application execution unit 152. FIG. The management unit 110 includes, for example, a sound processing unit 112 , a WU (Wake Up) determination unit 114 for each agent, a display control unit 116 , and a voice control unit 118 . It is simply referred to as the agent function unit 150 when it is not distinguished which agent function unit it is. The illustration of three agent function units 150 is merely an example corresponding to the number of agent servers 200 in FIG. 1, and the number of agent function units 150 may be two, or four or more. may be The software arrangement shown in FIG. 3 is simply shown for explanation. In practice, for example, the management unit 110 may intervene between the agent function unit 150 and the vehicle-mounted communication device 60, or may be modified arbitrarily. be able to.

Each component of the agent device 100 is implemented by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components are hardware (circuit part; circuitry) or by cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as a HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage such as a DVD or CD-ROM. It may be stored in a medium (non-transitory storage medium) and installed by loading the storage medium into a drive device.

The management unit 110 functions by executing programs such as an OS (Operating System) and middleware.

The sound processing unit 112 of the management unit 110 performs sound processing on the input sound so that the state is suitable for recognizing the wake-up word preset for each agent.

WU determination unit 114 for each agent exists corresponding to each of agent function units 150-1, 150-2, and 150-3, and recognizes a wakeup word predetermined for each agent. The WU determination unit 114 for each agent recognizes the meaning of the voice from the voice (audio stream) that has undergone acoustic processing. First, the WU determination unit 114 for each agent detects a voice section based on the amplitude and zero crossing of the voice waveform in the voice stream. The WU determination unit for each agent 114 may perform segment detection based on speech identification and non-speech identification for each frame based on a Gaussian mixture model (GMM).

Next, the WU determination unit 114 for each agent converts the voice in the detected voice section into text and uses it as character information. Then, the WU determination unit 114 for each agent determines whether or not the textual information corresponds to the wakeup word. If determined to be a wakeup word, the WU determination unit 114 for each agent activates the corresponding agent function unit 150 . Note that the agent server 200 may have a function corresponding to the WU determination unit 114 for each agent. In this case, the management unit 110 transmits to the agent server 200 the audio stream that has been acoustically processed by the acoustic processing unit 112, and if the agent server 200 determines that it is a wake-up word, it follows the instruction from the agent server 200. Agent function unit 150 is activated. It should be noted that each agent function unit 150 may be always activated and determine the wakeup word by itself. In this case, the management unit 110 does not need to have the WU determination unit 114 for each agent.

The storage unit 130 stores, for example, a preset wakeup word 132 . The wake-up word 132 may be a default wake-up word set in advance, or a wake-up word set by the user during the manufacturing process of the vehicle M by a process described later (for example, a name such as "Andy"). ). A user is, for example, a user at a delivery destination of a manufactured car.

The agent function unit 150 cooperates with the corresponding agent server 200 to cause an agent to appear, and provides services including voice responses in response to the user of the vehicle M's speech. The agent function unit 150 may include those authorized to control the vehicle device 50 . Further, the agent function unit 150 may cooperate with the general-purpose communication device 70 via the pairing application execution unit 152 and communicate with the agent server 200 . For example, the agent function unit 150-1 is authorized to control the vehicle device 50. FIG. Agent function unit 150-1 communicates with agent server 200-1 via in-vehicle communication device 60. FIG. Agent function unit 150-2 communicates with agent server 200-2 via in-vehicle communication device 60. FIG. Agent function unit 150-3 cooperates with general-purpose communication device 70 via pairing application execution unit 152 and communicates with agent server 200-3.

The pairing application execution unit 152, for example, performs pairing with the general-purpose communication device 70 to connect the agent function unit 150-3 and the general-purpose communication device 70. FIG. The agent function unit 150-3 may be connected to the general-purpose communication device 70 by wired communication using USB (Universal Serial Bus) or the like.

The display control unit 116 causes the first display 22 or the second display 24 to display an image according to an instruction from the agent function unit 150 . In the following, it is assumed that the first display 22 is used. The display control unit 116, under the control of a part of the agent function unit 150, for example, generates an image of an anthropomorphic agent (hereinafter referred to as an agent image) that communicates with the user inside the vehicle. The agent image is displayed on the first display 22. - 特許庁The agent image is, for example, an image of a mode of speaking to the user. The agent image may include, for example, a facial image that allows at least the viewer (user) to recognize the facial expression and facial orientation. For example, the agent image may include parts simulating eyes and nose in the face area, and the facial expression and facial orientation may be recognized based on the positions of the parts in the face area. The agent image is an image that the viewer feels stereoscopically and recognizes the face orientation of the agent by including the head image in the three-dimensional space. The agent image may include an image of the agent's body (torso, limbs, etc.) in which the agent's actions, behavior, posture, etc. are recognized. Also, the agent image may be an animation image.

The audio control unit 118 causes some or all of the speakers included in the speaker unit 30 to output audio according to instructions from the agent function unit 150 . The voice control unit 118 may use a plurality of speaker units 30 to perform control to localize the sound image of the agent's voice at a position corresponding to the display position of the agent's image. The position corresponding to the display position of the agent image is, for example, the position where the user is expected to feel that the agent image is speaking the agent voice. within 2 to 3 [cm]). Further, the localization of the sound image means that the spatial position of the sound source perceived by the user is determined by, for example, adjusting the volume of the sound transmitted to the left and right ears of the user.

[Agent server]
FIG. 6 is a diagram showing the configuration of the agent server 200 and part of the configuration of the agent device 100. As shown in FIG. The configuration of the agent server 200 and the operation of the agent function unit 150 and the like will be described below. A description of physical communication from the agent device 100 to the network NW is omitted here.

The agent server 200 has a communication unit 210 . The communication unit 210 is a network interface such as a NIC (Network Interface Card). Further, the agent server 200 includes, for example, a speech recognition unit 220, a natural language processing unit 222, a dialogue management unit 224, a network search unit 226, and a response sentence generation unit 228. These components are implemented by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as LSI, ASIC, FPGA, GPU, etc., or by cooperation of software and hardware. good too. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium), or may be stored in a removable storage medium such as a DVD or CD-ROM (non-transitory storage medium). physical storage medium), and may be installed by mounting the storage medium in a drive device.

The agent server 200 has a first storage unit 250 . The first storage unit 250 is implemented by the various storage devices described above. The first storage unit 250 stores data and programs such as a personal profile 252, a dictionary DB (database) 254, a knowledge base DB 256, a response rule DB 258, and the like.

In the agent device 100 , the agent function unit 150 transmits to the agent server 200 an audio stream or an audio stream that has undergone processing such as compression or encoding. When the agent function unit 150 recognizes a voice command capable of local processing (processing not via the agent server 200), the agent function unit 150 may perform processing requested by the voice command. A voice command capable of local processing is a voice command that can be answered by referring to a storage unit (not shown) provided in the agent device 100, or, in the case of the agent function unit 150-1, controls the vehicle device 50. It may be a voice command (eg, a command to turn on an air conditioner). Therefore, the agent function unit 150 may have some of the functions that the agent server 200 has.

When the agent device 100 acquires the voice stream, the voice recognition unit 220 performs voice recognition and outputs character information converted into text, and the natural language processing unit 222 interprets the meaning of the character information while referring to the dictionary DB 254. I do. The dictionary DB 254 associates abstracted semantic information with character information. The dictionary DB 254 may include synonyms and synonym list information. The processing of the speech recognition unit 220 and the processing of the natural language processing unit 222 are not clearly divided into stages, and the speech recognition unit 220 receives the processing result of the natural language processing unit 222 and corrects the recognition result. It may be done by influencing each other.

For example, when a meaning such as "today's weather" or "how is the weather?" As a result, even when the voice of the request has character fluctuations, it is possible to facilitate dialogue that meets the request. Also, the natural language processing unit 222 may recognize the meaning of character information using artificial intelligence processing such as machine learning processing using probability, or generate a command based on the recognition result.

Based on the processing result (command) of the natural language processing unit 222, the dialogue management unit 224 determines the content of the utterance to the user of the vehicle M while referring to the personal profile 252, the knowledge base DB 256, and the response rule DB 258. The personal profile 252 includes user's personal information, hobbies and tastes, history of past interactions, and the like, which are saved for each user. The knowledge base DB 256 is information that defines relationships between things. The response rule DB 258 is information that defines actions (responses, device control contents, etc.) that agents should perform in response to commands.

Further, the dialogue management unit 224 may identify the user by matching with the personal profile 252 using feature information obtained from the audio stream. In this case, in the personal profile 252, for example, the feature information of the voice is associated with the personal information. Voice feature information is, for example, information related to speaking style features such as pitch, intonation, and rhythm (pitch pattern of sound), and feature quantities such as Mel Frequency Cepstrum Coefficients. . The voice feature information is information obtained, for example, by having the user utter predetermined words, sentences, or the like at the time of initial registration of the user, and recognizing the uttered voice.

If the command requests information that can be searched via the network NW, the interaction manager 224 causes the network searcher 226 to search. The network search unit 226 accesses various web servers 500 via the network NW and acquires desired information. "Information that can be searched via the network NW" is, for example, the results of evaluations by general users of restaurants around the vehicle M, or the weather forecast according to the location of the vehicle M on that day.

The response sentence generation unit 228 generates a response sentence so that the content of the utterance determined by the dialogue management unit 224 is conveyed to the user of the vehicle M, and transmits the generated response sentence to the agent device 100 . When the user is identified as the user registered in the personal profile, the response sentence generation unit 228 calls the user's name or creates a response sentence in a style of speaking similar to that of the user. may be generated.

Upon acquiring the response sentence, the agent function unit 150 instructs the voice control unit 118 to perform voice synthesis and output voice. Also, the agent function unit 150 instructs the display control unit 116 to display the image of the agent in accordance with the voice output. In this way, an agent function is realized in which a virtually appearing agent responds to the user of the vehicle M. FIG.

FIG. 7 is a diagram showing an example of the functional configuration of the information providing device 300 (server device). The information providing device 300 includes, for example, a communication section 310, an information management section 312, an information analysis section 314, an information provision section 316, a setting section 318, and a storage section 320.

The information management unit 312, the information analysis unit 314, the information provision unit 316, and the setting unit 318 are implemented by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as LSI, ASIC, FPGA, GPU, etc., or by cooperation of software and hardware. good too. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium), or may be stored in a removable storage medium such as a DVD or CD-ROM (non-transitory storage medium). physical storage medium), and may be installed by mounting the storage medium in a drive device.

The communication unit 310 is a communication interface that communicates with the general-purpose communication device 70, the factory terminal device 400 installed in the automobile factory, the dealer terminal device 450, and the like.

The information management unit 312 (acquisition unit) manages information transmitted by other devices. For example, the information management unit 312 generates vehicle information 324, for example. The information management unit 312 acquires the manufacturing process of the vehicle associated with the user who is the delivery destination of the vehicle.

The information analysis unit 314 analyzes the text transmitted by the general-purpose communication device 70 to analyze the meaning of the text. The information analysis unit 314 may refer to the database 322 stored in the storage unit 320 to analyze the meaning of the text.

The information providing unit 316 (transmitting unit) transmits information indicating the manufacturing process to the general-purpose communication device 70 of the user. The information providing unit 316 transmits information indicating the manufacturing process to the user's general-purpose communication device 70 based on the analysis result of the information analyzing unit 314, for example. That is, the information providing unit 316 transmits to the user information that the user intends to obtain or information that is useful for the user.

The setting unit 318 sets a name associated with the vehicle or the agent device 100 to the vehicle or the agent device 100 in response to a user's request. For example, the setting unit 318 sets the name to the vehicle or the agent device 100 by transmitting instruction information to the factory terminal device 400 to set the name based on the user's request to the vehicle or the agent device 100. .

The storage unit 320 stores, for example, a database 322 and vehicle information 324 . The database 322 contains information in which abstracted semantic information is associated with character information, information defining relationships between things, information on responses to information sent to users, and the like. Details of the vehicle information 324 will be described later.

[Sequence (Part 1)]
FIG. 8 is a sequence diagram showing an example of the flow of processing executed by the agent system 1. As shown in FIG. For example, it is assumed that a user visits a dealership and concludes a contract to purchase a vehicle. In this case, first, the dealer terminal device 450 transmits the identification information of the general-purpose communication device 70 provided by the user, the order number of the vehicle, the details of the order of the vehicle, etc. to the information providing device 300 (step S100). The information providing device 300 acquires the information transmitted in step S100, and stores the acquired information in the storage unit 320 (step S102).

Next, the dealer terminal device 450 transmits the order number of the vehicle, the details of the order of the vehicle, etc. to the factory terminal device 400 (step S104). The factory terminal device 400 acquires the information transmitted in step S104 and stores the acquired information in the storage device (step S106). Then, the factory terminal device 400 generates a vehicle manufacturing schedule according to the order content, and based on the generated manufacturing schedule, provides information to various terminal devices so that the ordered vehicle is generated, and performs work. Generate work schedules for employees.

Next, the factory terminal device 400 transmits reception completion information to the information providing device 300 after the processing of step S106 is completed (for example, after receiving the order) (step S108). When acquiring the reception completion information, the information providing apparatus 300 stores the acquired reception completion information in the storage unit 320 (step S110). The information stored in the storage unit 320 in steps S102 and S110 is information included in the vehicle information 324, for example.

Next, the information providing device 300 generates information according to the reception completion information, and transmits the generated generated information to the general-purpose communication device 70 (step S112). The generated information is information for the manufacturing monitoring application 79 of the general-purpose communication device 70 to display the reception image on the display unit 71 . The acceptance image is an image showing that the order has been accepted at the factory.

Next, after obtaining the generation information, the general-purpose communication device 70 causes the display unit 71 to display the received image based on the desired generation information (step S114). FIG. 9 is a diagram showing an example of the reception image IM1 displayed on the display unit 71. As shown in FIG. In the illustrated example, the acceptance image IM is an image (for example, an image showing an egg) indicating that the manufacture of a car has been accepted and is being prepared for manufacture.

Next, the factory terminal device 400 transmits information indicating the manufacturing process to the information providing device 300 (step S116). The information indicating the manufacturing process is information indicating events that occur from the time an order for the vehicle is received until the vehicle is delivered to the orderer. The information indicating the manufacturing process includes, for example, information indicating the start of a predetermined manufacturing process, information indicating the end of the predetermined manufacturing process, and the like. The factory terminal device 400 may acquire information indicating the start of a predetermined manufacturing process or information indicating the end of a predetermined manufacturing process through an operator's input operation or from another terminal device.

The information providing device 300 acquires information indicating the manufacturing process, and generates vehicle information 324 based on the acquired information (step S118). FIG. 10 is a diagram showing an example of the contents of the vehicle information 324. As shown in FIG. The vehicle information 324 is information in which, for example, identification information of the vehicle, identification information of the orderer, identification information of the general-purpose communication device 70 of the orderer, details of the order, information indicating the manufacturing process, etc. are associated and stored. be. In the illustrated example, the information indicating the start of a predetermined manufacturing process is associated with the start date and time of the manufacturing process and information indicating whether or not the information on the predetermined manufacturing process has been provided to the user. After step S118, the information providing device 300 acquires information indicating the manufacturing process from the factory terminal device 400 at the timing when a predetermined manufacturing process starts or ends at the factory.

The information providing device 300 may request the factory terminal device 400 to transmit information indicating the manufacturing process. In this case, the factory terminal device 400 transmits information indicating the manufacturing process to the information providing device 300 in response to the request.

Thus, the information providing device 300 can manage information indicating the manufacturing process.

[Sequence (Part 2)]
FIG. 11 is a sequence diagram showing an example of the flow of processing executed by the agent system 1. As shown in FIG. First, after the user activates the manufacturing monitoring application 79 (step S200), when the user inputs text, the manufacturing monitoring application 79 transmits the input text to the information providing device 300 (step S202). Next, when the information providing device 300 acquires the transmitted text, it analyzes the meaning of the transmitted text (step S204), and transmits information according to the analysis result to the general-purpose communication device 70 (step S206). When acquiring the information transmitted in step S206, the general-purpose communication device 70 displays the acquired information on the display unit 71 (step S208).

FIG. 12 is a diagram showing an example of an image displayed on the display section 71 by the processing of steps S200 to S208. The information providing unit 316 associates the agent device 100 (UG(M) in the figure), which provides a service including a voice response in response to the acquired post, with the vehicle M or the vehicle M, and informs the user of the manufacturing process. to the terminal device of For example, when the user inputs the text "Tell me about your current situation" into the operation unit of the general-purpose communication device 70, the information analysis unit 314 of the information providing device 300 analyzes the meaning of the input text. Then, based on the analyzed meaning, the information providing unit 316 transmits information indicating the manufacturing process from the vehicle information 324 to the general-purpose communication device 70 . Further, the information providing unit 316 transmits an image IM2 showing the ordered vehicle (or the agent device 100) to the general-purpose communication device 70. FIG. The image representing the vehicle may be an image of an actually manufactured vehicle, or an image obtained by deforming an icon corresponding to the manufacturing process or an image of the vehicle being manufactured.

Returning to the description of FIG. When the information providing device 300 receives information indicating a predetermined manufacturing process from the factory terminal device 400, the information providing device 300 stores the received information in the storage unit 320 (step S210), and transmits the received information to general-purpose communication. It is transmitted to the device 70 (step S212). When receiving the information transmitted in step S212, the general-purpose communication device 70 causes the display unit 71 to display the received information (step S214).

FIG. 13 is a diagram showing an example of an image displayed on the display section 71 by the processing of steps S210 to S214. For example, when the information providing device 300 receives information indicating that the assembly of the vehicle will be completed after a predetermined time, this information is transmitted to the general-purpose communication device 70 . That is, information indicating the manufacturing process is transmitted to the user's general-purpose communication device 70 according to the real-time manufacturing process of the vehicle. Then, the general-purpose communication device 70 causes the display unit 71 to display the text "Assembly will be completed soon."

Further, when the information providing device 300 receives information indicating that the assembly of the vehicle has been completed, this information is transmitted to the general-purpose communication device 70 . The general-purpose communication device 70 causes the display unit 71 to display the text "Assembly completed" and an image IM3 showing an icon representing a vehicle. Furthermore, when the information providing device 300 receives information indicating that the vehicle is being painted, it transmits this information to the general-purpose communication device 70 . The general-purpose communication device 70 causes the display unit 71 to display the text "Painting" and an image IM4 showing an icon showing the vehicle being painted.

In addition to the start and end of assembly of the vehicle and the start and end of painting, information indicating other manufacturing processes may be provided to the user. Further, for example, the image IM3 or IM4 may be an image of an actual vehicle, an image obtained by deforming this image, or an icon representing a virtual vehicle. Information indicating the manufacturing process is displayed on the display unit 71 in chronological order.

Returning to the description of FIG. When the information providing device 300 receives from the factory terminal device 400 the information indicating the manufacturing process for naming the agent device 100 (or vehicle), the information providing device 300 stores the received information in the storage unit 320 ( Step S216), the received information and information indicating that the user is to be asked about the assignment of the name are transmitted to the general-purpose communication device 70 (step S218). When receiving the information transmitted in step S218, the general-purpose communication device 70 causes the display unit 71 to display the received information (step S220). When the user operates the general-purpose communication device 70 to input a name, the general-purpose communication device 70 transmits the input name to the information providing device 300 (step S222). The information providing device 300 associates the name transmitted in step S222 with the vehicle ordered by the user and stores it in the storage unit 320 (step S224).

Next, the setting unit 318 of the information providing device 300 transmits the name received in step S222 to the factory terminal device 400 (step S226). The factory terminal device 400 stores the name received in step S266 in the storage device. Then, the factory terminal device 400 performs setting processing for setting the received name to the name of the agent device 100 of the vehicle ordered by the user (step S228). For example, the set name corresponds to a wakeup word.

FIG. 14 is a diagram showing an example of an image displayed on the display section 71 by the processing of steps S216 to S224. For example, when the information providing device 300 receives information indicating a manufacturing process that assigns a name associated with the agent device 100 , this information is transmitted to the general-purpose communication device 70 . The general-purpose communication device 70 causes the display unit 71 to display the text “What is your name?”. The general-purpose communication device 70 causes the display unit 71 to display the information of the name input by the user, and transmits the input name to the information providing device 300 . The information providing device 300 then transmits information indicating that the name has been accepted to the general-purpose communication device 70 . The general-purpose communication device 70 causes the display section 71 to display information indicating that the name has been accepted.

As in steps S216 to S224 described above, the information providing device 300 manages information indicating the manufacturing process and vehicle names. FIG. 15 is a diagram showing an example of the content of vehicle information 324 generated by the information providing device 300. As shown in FIG. In the vehicle information 324, the name of the agent device 100 set by the user (for example, "Andy" corresponding to the wakeup word) and information on each manufacturing process are stored as a result of the above-described processing.

The information providing device 300 then refers to the vehicle information 324 to provide the user with information on each manufacturing process and to provide the user with an opportunity to set the name of the agent device 100 . As a result, the information providing device 300 can provide useful information to the user.

[flowchart]
FIG. 16 is a flow chart showing an example of the flow of processing executed by the agent device 100. As shown in FIG. This process is, for example, a process performed after the vehicle manufactured by the above process is delivered to the user and the user gets into the vehicle.

First, the agent device 100 determines whether or not the wakeup word set in the manufacturing process has been obtained (step S300). Upon acquiring the wakeup word set in the manufacturing process, the agent device 100 activates the agent function unit 150 (step S302). This completes the processing of this flowchart.

FIG. 17 is a diagram showing an example of how the agent device 100 is activated by a wakeup word set in the manufacturing process. For example, when the user utters the wake-up word "Andy" set during the manufacturing process, the agent device 100 is activated in response to the utterance.

In this way, the user can activate the agent device 100 under the name given by the user immediately after delivery of the vehicle, thereby improving the user's convenience. In addition, since the user can watch information on the manufacturing process with the general-purpose communication device 70 and name the agent device 100 during the manufacturing process, it is expected that the user will become more attached to the vehicle and the agent. User satisfaction with purchase is improved. Furthermore, since the information providing device 300 provides the user with information indicating the manufacturing process, the entertainment for the user during the manufacturing process of the vehicle is improved.

In the above example, information is provided or input using text or images, but instead (in addition), information is provided by voice, or information is input by voice. Information may be entered. In this case, the information providing device 300 includes a speech recognition unit that analyzes the speech acquired from the user and converts it into text.

According to the first embodiment described above, the information providing device 300 acquires the manufacturing process of the vehicle associated with the user who is the delivery destination of the vehicle, and provides the information indicating the acquired manufacturing process to the general purpose information of the user. By transmitting to the communication device 70, useful information can be provided to the user.

<Second embodiment>
The agent system 1A of the second embodiment will be described below. In the second embodiment, instead of the agent device 100, the agent server 200A has a WU determination unit 230 for each agent. The following description will focus on differences from the first embodiment.

FIG. 18 is a diagram showing an example of the functional configuration of the agent server 200A of the second embodiment. The agent server 200A of the second embodiment further includes a WU determination unit 230 for each agent in addition to the functional configuration of the agent server 200 of the first embodiment. In the agent device 100, the WU determination unit 230 for each agent is omitted. The agent server 200A includes a storage unit 250A instead of the storage unit 250. FIG. In addition to the information stored in the storage unit 250, the storage unit 250A further stores wakeup word information 260. FIG. Wakeup word information 260 is information in which vehicle identification information and wakeup word information are associated with each other.

The WU determination unit 230 for each agent recognizes the wakeup word set for the corresponding agent. The per-agent WU determination unit 230 determines whether or not the information based on the audio stream acquired from the agent device 100 is the set wakeup word. When the per-agent WU determination unit 230 determines that the information based on the voice stream is a wake-up word, it transmits to the agent device 100 instruction information instructing activation of the corresponding agent function unit 150 . When the agent device 100 acquires the instruction information, the agent device 100 activates the agent function unit 150 .

[sequence]
FIG. 19 is a sequence diagram showing an example of the flow of processing executed by the agent system 1 of the second embodiment. In FIG. 19, the processing of steps S200 to S214 is the same as the processing with the same number in FIG. 11, so illustration is omitted. Further, the processing of steps S216 to S224 in FIG. 19 is the same as the processing with the same number in FIG. 11, so the description is omitted.

In step S224, the information providing device 300 associates the name transmitted by the general-purpose communication device 70 with the vehicle ordered by the user and stores it in the storage unit 320, and then stores the information in which the identification information and the name of the vehicle are associated with each other. It is transmitted to the agent server 200A (step S227). Agent server 200A causes storage unit 250A to store information in which the vehicle identification information and the name are associated as wake-up word information 260 (step S229).

Then, after the vehicle is completed and delivered to the user, the user gets into the vehicle and inputs the wake-up word stored in the storage unit 250A in step S229 into the microphone 10 (step S231). Agent device 100 transmits a voice stream corresponding to the voice input to microphone 10 to agent server 200A (step S233).

The agent server 200A determines whether or not the information based on the audio stream transmitted in step S233 is the set wakeup word (step S235). The agent server 200A transmits the determination result of step S235 to the agent device 100 (step S237). Then, if the determination result obtained in step S237 is the input wakeup word, the agent device 100 activates the agent function unit 150 (step S239).

According to the second embodiment described above, the agent server 200A determines whether or not the wake-up word set in the manufacturing process has been input. The processing load on the agent device 100 is reduced.

In the above example, the agent device 100 is mounted on the vehicle M, but instead (or in addition), the agent device 100 may be mounted on another object. In this case, information indicating the manufacturing process of the object is transmitted to the general-purpose communication device 70 of the user. Also, a name associated with a target (or agent device 100) is set for that target (or agent device 100) in response to a request from the user. Then, the user can activate the target agent device 100 by inputting the name immediately after the target is delivered.

In the above example, the name is set during the manufacturing process of the vehicle M (or object). (for example, the period from ordering to delivery), the manufacturing process of the target on which the agent device 100 is installed, and the time until the target on which the agent device 100 is installed is delivered to the user. A name (wakeup word) may be set according to the request of

Also, instead of (or in addition to) setting the name, the agent's icon displayed on the vehicle's display may be set or generated by the user before the vehicle is delivered.

As described above, the mode for carrying out the present invention has been described using the embodiments, but the present invention is not limited to such embodiments at all, and various modifications and replacements can be made without departing from the scope of the present invention. can be added.

1... Agent system 20... Display/operation device 30... Speaker unit 70... General-purpose communication device 71... Display unit 79... Production monitoring application 100, 100A, 100B... Agent device 110... Management unit 114... UW determination unit for each agent 116 Display control unit 118 Audio control unit 130 Storage unit 132 Wake-up word 150 Agent function unit 200 Agent server 300 Information providing device 312 Information management section 314 information analysis section 316 information provision section 318 setting section 320 storage section 324 vehicle information

Claims (6)

an acquisition unit that acquires the manufacturing process of the vehicle associated with the user who is the delivery destination of the vehicle;
a transmission unit configured to transmit information indicating the manufacturing process acquired by the acquisition unit to the terminal device of the user;
The acquisition unit acquires the post of the user,
The transmission unit transmits the manufacturing process to the terminal device of the user in association with the vehicle or an agent device that provides a service including a voice response according to the voice in response to the post acquired by the acquisition unit. do,
Server device. The manufacturing process includes a step in which assembly of the vehicle begins, assembly of the vehicle ends, painting of the vehicle begins, or painting of the vehicle finishes;
The transmission unit transmits an icon, image, or deformed image according to the manufacturing process to the terminal device of the user.
The server device according to claim 1 . The transmission unit transmits information indicating the manufacturing process to the terminal device of the user according to the real-time manufacturing process of the vehicle.
3. The server device according to claim 1 or 2 . The agent device is activated when the user's utterance matches a preset keyword, and performs processing for providing a service including a voice response in response to the user's utterance,
A manufacturing process of the agent device, until the agent device is delivered to the user, a manufacturing process of an object on which the agent device is installed, and an object on which the agent device is installed is delivered to the user A setting unit that sets the keyword according to the request of the user until
The server device according to any one of claims 1 to 3 . the computer
Acquire the manufacturing process of the vehicle associated with the user who is the delivery destination of the vehicle,
transmitting the acquired information indicating the manufacturing process to the terminal device of the user;
Obtaining posts by said user;
Sending the manufacturing process to the terminal device of the user in association with an agent device that provides a service including a response by voice according to the voice in response to the acquired post or the vehicle;
How to provide information. to the computer,
Get the manufacturing process of the vehicle associated with the user who is the delivery destination of the vehicle,
causing the terminal device of the user to transmit the acquired information indicating the manufacturing process;
Acquire the user's post,
Sending the manufacturing process to the terminal device of the user in association with an agent device that provides a service including a voice response according to the voice in response to the acquired post or the vehicle;
program.

Images