JP2023170239A - Output control method and output control device - Google Patents

Abstract

To appropriately transmit a voice message from a user on the outer side of a vehicle in a situation where a user on the inner side of the vehicle can hear the voice message.SOLUTION: An output control method allows an on-vehicle speaker 204 to output a voice message based on a voice file to be transmitted from a second OPS 300 by using wireless communication. The output control method includes: acquisition processing (step S501) for acquiring the voice file; estimation processing for estimating a message reception degree which indicates an index on whether a state of a user on the inner side of a vehicle C1 is the state where the user can hear the voice message (step S503); and determination processing (step S504 or S510) for determining timing to output the voice message from the on-vehicle speaker 204 based on a level of the message reception degree.SELECTED DRAWING: Figure 7

Description

The present invention relates to an output control method and an output control device for outputting a voice message from an output device in a vehicle.

BACKGROUND ART Conventionally, there is a technology in which an in-vehicle user who rides in a vehicle exchanges voice messages with a user outside the vehicle. For example, a technology has been proposed that, if the user inside the vehicle does not speak for a predetermined period of time after the end of the user's utterance, a voice message is sent to the user outside the vehicle to notify the user that the response from the user inside the vehicle will be delayed (e.g. , see Patent Document 1).

Japanese Patent Application Publication No. 2011-147052

In the above-mentioned conventional technology, the user outside the vehicle can understand that the response of the user inside the vehicle is delayed, but cannot grasp the situation of the user inside the vehicle. Therefore, it is assumed that the user outside the vehicle tries to convey necessary information even in a situation where the user inside the vehicle cannot hear the voice message. In this case, there is a risk that the communication time will increase due to the re-listening from the in-vehicle user, and the amount of calculation for each device will increase. Further, in a situation where the user inside the vehicle cannot hear the voice message, the voice message from the user outside the vehicle cannot be appropriately conveyed, and the quality of response to the voice message may deteriorate.

An object of the present invention is to appropriately convey a voice message from a user outside the vehicle in a situation where the user inside the vehicle can hear the voice message.

One aspect of the present invention is an output control method that causes an output device inside a vehicle to output a voice message based on voice information transmitted from a device outside the vehicle using wireless communication. This output control method includes an acquisition process for acquiring voice information, an estimation process for estimating a degree of acceptance that indicates whether or not the state of the user inside the vehicle is in a state where it is possible to listen to the voice message, and and determining the timing at which the output device outputs the voice message based on the voice information, based on the magnitude of the frequency.

According to the present invention, it is possible to appropriately convey a voice message from a user outside the vehicle in a situation where the user inside the vehicle can hear the voice message.

FIG. 1 is a diagram showing an example of the system configuration of an operator system. FIG. 2 is a block diagram showing an example of the configuration of the first OPS. FIG. 3 is a block diagram showing an example of the configuration of the second OPS. FIG. 4 is a diagram schematically showing an example of communication of an audio file and an example of outputting an audio message. FIG. 5 is a diagram schematically showing an example of communication of an audio file and an example of outputting an audio message. FIG. 6 is a diagram schematically showing a division example in which a sentence included in a voice message is divided at division positions based on a division criterion, and an output example in which a voice message is output from the division position. FIG. 7 is a flowchart illustrating an example of voice message output control processing in the output control device.

Embodiments of the present invention will be described below with reference to the accompanying drawings.

[Example of operator system configuration]
FIG. 1 is a diagram showing an example of the system configuration of the operator system 1. As shown in FIG. The operator system 1 is a communication system that supports the in-vehicle users of the vehicle C1. For example, the operator system 1 is a voice dialogue system used for operator services for automobiles. This operator service for automobiles is a service that allows an in-vehicle user to talk to an external operator inside the vehicle, and is a service that solves various problems of in-vehicle users (drivers or other passengers). Note that the present embodiment is also applicable to a voice dialogue system that allows communication with occupants of other vehicles in addition to communication with an external operator.

Note that the vehicle C1 is a vehicle such as an internal combustion engine vehicle, a hybrid vehicle, or an electric vehicle. Further, in FIG. 1, only one vehicle C1 is shown for ease of explanation, but the present embodiment is also applicable to a case where a plurality of vehicles exist.

The operator system 1 is configured such that each device can communicate with each other via a network 10 . For example, a first OPS (operator system) 200, a second OPS 300, etc. are configured to be able to communicate via the network 10. For example, the first OPS 200 is connected to the network 10 via the base station 20 using a communication method using wireless communication. Further, for example, the second OPS 300 is connected to the network 10 using either a communication method using wireless communication or a communication method using wired communication, or both methods.

Furthermore, in this embodiment, the description will be made assuming that voice chat is the communication performed between the first OPS 200 and the second OPS 300. This voice chat refers to a system in which multiple people exchange voice messages in real time using a predetermined network such as the Internet. This voice chat is realized by converting sounds emitted by a person into audio files at predetermined intervals, and exchanging these audio files between devices via a predetermined network. Note that in this embodiment, an example of voice chat is shown as the communication performed between the first OPS 200 and the second OPS 300, but the present invention is similarly applicable to voice communication other than voice chat. In this way, the first OPS 200 and the second OPS 300 function as a voice communication system.

Furthermore, in the present embodiment, an example will be shown in which a delay process is executed to delay audio output from the first OPS 200 in a voice chat performed between the first OPS 200 and the second OPS 300. This delay processing will be explained in detail with reference to FIGS. 5 to 7 and the like.

The network 10 is, for example, a communication line network that can connect callers, and various communication networks such as a mobile phone network, a public network such as the Internet, and a private network can be used.

The first OPS 200 is an on-vehicle operator system configured by one or more devices mounted on the vehicle C1, and executes various communications with the second OPS 300 to realize the voice dialogue system. Further, the in-vehicle user of the vehicle C1 is the driver or a passenger other than the driver. As described above, this embodiment is particularly useful when the in-vehicle user is a driver, but the in-vehicle user may also be a passenger.

The second OPS 300 is a remote operator system composed of one or more devices installed in an inquiry center or the like, and executes each communication with the first OPS 200 to realize a voice dialogue system. Further, the user outside the vehicle (remote user) shown in this embodiment is an operator who supports the driver of the vehicle at an inquiry center or the like. Note that the user outside the vehicle (remote user) may be someone other than a person. For example, the synthesized voice of the information processing system may be a sound emitted by an external operator.

Various information processing devices such as a mobile information terminal, an in-vehicle system, and a personal computer can be used as the first OPS 200 and the second OPS 300.

In this manner, the present embodiment is applicable to a communication system in which users interact with each other via the network 10.

[Configuration example of operator system (vehicle side)]
FIG. 2 is a block diagram showing a configuration example of the first OPS 200. The first OPS 200 is an in-vehicle system that has main functions for realizing operator services for automobiles.

The first OPS 200 includes an on-vehicle microphone 201 , on-vehicle sensors 202 , an operation reception section 203 , an on-vehicle speaker 204 , and an output control device 210 .

The in-vehicle microphone 201 acquires sound inside the vehicle interior of the vehicle C1, and outputs sound information regarding the acquired sound to the transmission processing unit 214. As the in-vehicle microphone 201, for example, one or more microphones or a sound acquisition sensor can be used.

The on-vehicle sensors 202 are various sensors installed in the vehicle C1, and output their detected values to the acceptance degree estimation section 215 and the user permission recognition section 216. In-vehicle sensors 202 include various sensors such as a steering angle sensor, an accelerator opening sensor, a vehicle speed sensor, an interior camera, an exterior camera, an interior sound acquisition sensor, an exterior sound acquisition sensor, a position information acquisition sensor, a seat type sensor, and a sweat sensor. Can be used. Note that each of these sensors is an example of a sensor that can be installed in the vehicle C1, and other sensors may be used. Further, other information that can be obtained from an external device may be used.

The steering angle sensor is a sensor that measures the rotation angle of the steering wheel. Further, the accelerator opening sensor is a sensor that measures the amount of depression of the accelerator pedal. Further, the vehicle speed sensor is a sensor that measures the speed of the vehicle C1.

Further, the external camera is a camera that captures an image of a subject outside the vehicle C1 and generates an image (image data). Further, the in-vehicle camera is a camera that captures an image of a subject inside the vehicle interior of the vehicle C1 and generates an image (image data). Note that the exterior camera and the interior camera are configured by, for example, one or more camera devices and image sensors that can image a subject. In addition, although this example shows an example provided with at least two cameras outside the vehicle and cameras inside the vehicle, it is also possible to provide one or three or more imaging devices and use images from some of these imaging devices.

The in-vehicle sound acquisition sensor is a sensor that acquires the sound in the cabin of the vehicle C1. Note that the in-vehicle microphone 201 can be used as the sound acquisition sensor. Further, the vehicle external sound acquisition sensor is a sensor that acquires external sound of the vehicle C1.

The position information acquisition sensor acquires position information regarding the position where the vehicle C1 is present. For example, it can be realized by a GNSS receiver that acquires position information using GNSS (Global Navigation Satellite System). Further, the position information includes various data related to the position such as latitude, longitude, altitude, etc. at the time of receiving the GNSS signal. Alternatively, the location information may be acquired using other location information acquisition methods. For example, location information may be derived using information from nearby access points and base stations. Alternatively, location information may be acquired using a beacon. Further, for example, position information may be derived using position estimation technology using a navigation device.

The seat-type sensor is a sensor that measures the driver's heart rate and breathing rate. Further, the sweat sensor is a sensor that measures the amount of sweat of the driver.

The operation reception unit 203 accepts an operation from a user (driver or other occupant) inside the vehicle C1, and outputs the content of the accepted operation to the user permission recognition unit 216. The operation reception unit 203 may be configured as a user interface integrated with a display unit capable of displaying various information, or may be configured as a separate user interface. As this display section, a display panel such as an organic EL (Electro Luminescence) panel or an LCD (Liquid Crystal Display) panel can be used. Further, as the integrated user interface, a touch panel can be used that allows the user to perform operation input by touching or approaching the display surface with his or her finger. Various operating members, such as buttons, can be used as a separate user interface.

The in-vehicle speaker 204 outputs sound into the cabin of the vehicle C1. For example, one or more speakers can be used as the in-vehicle speaker 204. Note that the in-vehicle microphone 201, the operation reception unit 203, and the in-vehicle speaker 204 are examples of a user interface, and some of these may be integrated, or another user interface may be used.

The output control device 210 includes a communication unit 211, a reception processing section 212, a division processing section 213, a transmission processing section 214, an acceptance estimation section 215, a user permission recognition section 216, a delay processing determination section 217, It includes a storage section 218, an audio output processing section 219, and a delay information output processing section 220.

The communication unit 211 is connected to the base station 20 using wireless communication, and exchanges various information with the second OPS 300 via the network 10.

The reception processing unit 212 executes reception processing on each piece of information received from the second OPS 300 via the communication unit 211. For example, the reception processing unit 212 executes a reception process of receiving the audio file transmitted from the second OPS 300 via the communication unit 211. Note that the reception processing unit 212 temporarily stores the received audio file in the storage unit 218 as necessary. Further, the reception processing section 212 outputs the received audio file to the division processing section 213 and the audio output processing section 219. Furthermore, when the delay processing determining section 217 determines to execute the delay processing, the reception processing section 212 causes the storage section 218 to store the received audio file.

The division processing unit 213 executes a division process of dividing the audio information based on the audio file output from the reception processing unit 212, that is, the text included in the audio message, based on preset division criteria. Further, the division processing unit 213 outputs division information indicating the division position of each sentence determined by the division processing to the delay processing determining unit 217. This division process will be explained in detail with reference to FIG. Note that the division processing by the division processing unit 213 can also be referred to as utterance break determination processing.

The transmission processing unit 214 executes a transmission process of transmitting the sound information output from the in-vehicle microphone 201 to the second OPS 300 via the communication unit 211. For example, the transmission processing unit 214 executes a transmission process of generating an audio file based on the sound information output from the in-vehicle microphone 201 and transmitting this audio file to the second OPS 300 via the communication unit 211. Note that the transmission processing unit 214 temporarily stores the audio file to be transmitted in the storage unit 218 as necessary.

The receptivity estimating unit 215 executes an estimation process to estimate the message receptivity of the in-vehicle user U1 (driver or passenger) of the vehicle C1 based on each piece of information from the on-vehicle sensors 202. Further, the acceptability estimating unit 215 outputs the message acceptability estimated by the estimation process to the communication unit 211 and the delay process determining unit 217. Note that the communication unit 211 transmits the message acceptance degree to the second OPS 300.

[Example of estimating message acceptance level]
Here, a method for estimating the degree of message acceptance will be explained. The message acceptance level means an index of whether or not the user inside the vehicle C1 is in a state where he or she can listen to the voice message. In this embodiment, the message acceptance level is set to a high value when the user inside the vehicle C1 can hear the voice message, and the message acceptance level is set to a low value when the inside user of the vehicle C1 cannot listen to the voice message. An example is shown below.

In other words, the message acceptance level is an index related to at least one of the driving load of the driver of the vehicle C1 and the conversation status of the in-vehicle user (driver or other occupant) of the vehicle C1. Furthermore, in the present embodiment, the message acceptance level is determined based on condition data of at least one of the behavior of the vehicle C1, the state of the user (driver or other occupant) inside the vehicle C1, and surrounding information of the vehicle C1. An example of estimating is shown below.

Here, the behavior of the vehicle C1 can be grasped not only as the behavior of the vehicle C1 itself but also as the behavior of the driver of the vehicle C1. The behavior of the vehicle C1 can be acquired based on the amount of operation of the vehicle C1, for example. The behavior of the driver of the vehicle C1 can be acquired based on the driving state of the vehicle C1, for example.

[Example of estimating message acceptance level based on vehicle operation amount]
The operation amount of the vehicle C1 is, for example, a steering angle, an accelerator opening, a vehicle speed, etc. Each of these values can be obtained from the on-vehicle sensors 202.

For example, if the amount of steering wheel operation is greater than or equal to a threshold value, or if the frequency of steering wheel operations is greater than or equal to a threshold value, the direction of travel of vehicle C1 is changing significantly or the direction of travel of vehicle C1 is frequently changing. It is likely that it has changed. In this case, it is assumed that the driving load on the driver of the vehicle C1 is high. Furthermore, when the frequency of steering wheel operations is equal to or higher than the threshold value, it is assumed that the driver of the vehicle C1 is turning the steering wheel in small steps. Therefore, it is possible to set the message acceptance degree to a low value in accordance with an increase in the amount of operation of the steering wheel or an increase in the frequency of operation of the steering wheel. Note that the steering wheel operation frequency can be obtained, for example, by holding steering wheel operations as time series data.

Further, for example, if the accelerator opening is equal to or greater than a threshold value, or if the accelerator pedal operation frequency is equal to or greater than a threshold value, the speed of the vehicle C1 is high, or acceleration and deceleration of the vehicle C1 occurs frequently. Probability is high. In this case, it is assumed that the driving load on the driver of the vehicle C1 is high. Therefore, it is possible to set the message acceptance degree to a low value in accordance with an increase in the degree of opening of the accelerator or an increase in the frequency of operation of the accelerator pedal.

Further, for example, when the vehicle speed is equal to or higher than the threshold value, it is assumed that the driving load on the driver of the vehicle C1 is high. Therefore, it is possible to lower the message acceptance level as the vehicle speed increases. Note that each of these threshold values can be set as appropriate based on experimental data and the like.

In this way, for example, the driving load of the driver of the vehicle C1 is increased in response to an increase in the amount of steering wheel operation, an increase in the frequency of steering wheel operation, an increase in the degree of accelerator opening, an increase in the frequency of accelerator pedal operation, and an increase in vehicle speed. , the message acceptance level is set to a low value.

[Example of estimating message receptivity based on driver status]
The driving state of the vehicle C1 includes, for example, the driver's heart rate, the amount of perspiration of the driver, the driver's posture, the driver's line of sight, the driver's in-vehicle conversation status, and the like. Each of these values can be obtained from the on-vehicle sensors 202.

For example, if the driver's heart rate is above the threshold, or if the driver's sweat rate is above the threshold, there is a high possibility that the driver is concentrating on driving or is nervous about driving. . In this case, it is assumed that the driving load on the driver of the vehicle C1 is high. Therefore, it is possible to set the message receptivity to a low value in accordance with an increase in the driver's heart rate or an increase in the amount of sweat of the driver. Note that the driver's heart rate can be acquired by, for example, a seat-type sensor. Further, the amount of sweat of the driver can be acquired by, for example, a sweat sensor. Further, the driver's heart rate and the driver's sweat rate may be acquired from an electronic device worn by the driver, such as a wearable device. Further, the heart rate of the driver and the amount of sweat of the driver may be estimated using a predetermined calculation formula using each value acquired by another sensor.

For example, if the amount of change in the driver's posture is greater than or equal to the threshold, or if the frequency of changes in the driver's posture is greater than or equal to the threshold, the amount of operation of the steering wheel is high, the frequency of operation of the steering wheel is high, etc. The driver's body is often swayed. In such a case, there is a high possibility that the direction of travel of vehicle C1 is changing significantly or that the direction of travel of vehicle C1 is changing frequently, so it is assumed that the driving load on the driver of vehicle C1 is high. Ru. Therefore, it is possible to set the message acceptance degree to a low value in accordance with an increase in the amount of change in the driver's posture or an increase in the frequency of changes in the driver's posture. Note that the amount of change in the driver's posture or the frequency of change in the driver's posture can be acquired based on an image of the driver acquired by an in-vehicle camera.

For example, if the amount of change in the driver's line of sight is greater than or equal to the threshold, or if the frequency of change in the driver's line of sight is greater than or equal to the threshold, the driver may be driving on an unknown road or looking for something. It is assumed that the driving load on the driver of the vehicle C1 is high due to such reasons. Therefore, it is possible to set the message acceptance level to a low value in accordance with an increase in the amount of change in the driver's line of sight or an increase in the frequency of changes in the driver's line of sight. Note that the amount of change in the driver's line of sight or the frequency of change in the driver's line of sight can be acquired based on an image of the driver's face acquired by an in-vehicle camera.

For example, when a driver is having a conversation inside a car, it is important to prevent the driver from interrupting the conversation. Therefore, when it is determined that the driver is having a conversation inside the car, the message acceptance degree can be set to a low value. In this case, it is possible to set the message acceptance level to a low value depending on the state of conversation in the car. For example, it is possible to set the message acceptance level to a low value depending on the volume of the conversation in the car, the content of the conversation in the car, and the like. For example, the message acceptance level is set to a lower value as the volume of conversation in the car becomes louder. Further, for example, the message acceptance level is set to a lower value as the frequency of laughter in the car increases.

Furthermore, although this embodiment mainly assumes a case where the user inside the vehicle C1 is a driver, the present embodiment can also be applied to a case where the inside user of the vehicle C1 is a passenger. In this way, when the user in the vehicle C1 is a passenger, it is preferable to estimate the message acceptance level using the conversation situation in the vehicle, etc., instead of the driving load of the vehicle C1. In this case, it is assumed that the conversation situation in the car may be such that the occupant is on another phone call and cannot talk. In such a case, the message acceptance level is also set to a low value.

[Example of estimating message acceptance level based on vehicle surrounding information]
The surrounding information of the vehicle C1 is information that can be obtained from the surroundings of the vehicle C1. For example, it can be acquired based on an image of the surroundings of the vehicle C1 acquired by an external camera, sounds around the vehicle C1 acquired by an external sound acquisition sensor, and position information of the vehicle C1.

For example, when the vehicle C1 is traveling on a narrow road such as a mountain road or a residential area, it is assumed that the driving load on the driver of the vehicle C1 is high. In this case, it is possible to set the message acceptance degree to a low value depending on whether the vehicle C1 is traveling on a road where the driver's driving load is high. For example, it is possible to set the message acceptance degree to a low value in accordance with the fact that the vehicle C1 is traveling on a road where the driver's driving load is high. Roads where the driver of the vehicle C1 has a high driving load can be determined based on the position information acquisition sensor and map information.

Further, for example, when the vehicle C1 is traveling on a dark road, such as a road in the evening or at night, a road in the rain, or a snowy road, it is assumed that the driving load on the driver of the vehicle C1 is high. In this case, it is possible to set the message acceptance level to a low value depending on whether the vehicle C1 is traveling with a high driving load on the driver. For example, it is possible to set the message acceptance degree to a low value in accordance with the fact that the vehicle C1 is traveling with a high driving load on the driver. Roads where the driver of the vehicle C1 has a high driving load can be determined based on time information and weather information.

Note that the method for estimating the message receptivity described above is just an example, and the message receptivity may be estimated by other estimation methods. For example, driver recognition information (gender, age, behavior, line of sight, etc.) obtained from other sensor data installed in the vehicle C1 (e.g., Driver Monitoring System (DMS)), advanced driver assistance system (ADAS), etc. (Advanced Driving Assistant System)), surrounding conditions of the car (latitude and longitude, vehicle distance, pedestrian detection information, etc.), vehicle behavior (vehicle speed, steering angle, blinkers, accelerator opening, etc.), and the situation of conversations inside the car. may be used as condition data to estimate message acceptance. Furthermore, the message acceptability may be estimated by combining at least two of the above-described message acceptability estimation methods. For example, the degree of message acceptance may be quantified and determined by a prediction method such as multiple regression analysis using at least one of the above-mentioned condition data.

Furthermore, the degree of message acceptance may be estimated using artificial intelligence (AI). For example, newly acquired condition data is input as input data to a trained learning model that has been trained to estimate message acceptance from each condition data described above, and a new output corresponding to the input data is generated. The degree of message acceptance may be estimated by outputting the degree of message acceptance as data.

For example, when using artificial intelligence, a neural network that includes an input layer and an output layer, inputs condition data as input data to the input layer, and outputs message acceptance level as output data from the output layer, and and a learning unit that acquires a learned model by causing a neural network to learn using actual values of output data as teacher data. In this case, the receptivity estimating unit 215 estimates the message receptivity by inputting the newly acquired condition data as input data to the trained model acquired by the learning unit.

Note that the degree of message acceptance varies from user to user. Therefore, we log the permission information for voice messages by the in-vehicle user and the various condition data when the permission was granted, and based on that information, we provide the user with a reference value to be used when determining whether or not delay processing is necessary. It may be changed dynamically as required. Note that the determination of whether or not delay processing is necessary is performed by the delay processing determining unit 217. Furthermore, based on each piece of information, masking may be performed so that the delay processing is not executed under certain conditions. Note that the permission information for the voice message is output from the user permission recognition unit 216.

The user permission recognition unit 216 recognizes a user instruction to permit output of a voice message based on information from the in-vehicle sensors 202 or the operation reception unit 203, and recognizes the recognition result, that is, voice message permission information. is output to the delay processing determining unit 217. For example, the delay processing being executed in the first OPS 200 can be forcibly canceled by an instruction method such as device operation, voice input, gesture, etc. by the user inside the vehicle C1. For example, if a user in vehicle C1 presses a predetermined button, or if a user in vehicle C1 utters a predetermined voice, this is recognized as a user instruction to permit the output of a voice message. . Thereby, even if the message acceptability estimated by the acceptability estimator 215 is low, output of the voice message from the operator of the second OPS 300 can be permitted.

The delay process determining unit 217 determines whether or not to perform a delay process to delay the timing of outputting the voice message, based on the message acceptability estimated by the acceptability estimation unit 215. Then, the delay processing determining unit 217 outputs delay information (including division information) that is the determination result to the audio output processing unit 219 and the delay information output processing unit 220. Further, the delay processing determining section 217 outputs the delay information to the communication unit 211 and the reception processing section 212. Note that the communication unit 211 transmits the delay information to the second OPS 300. Furthermore, the reception processing unit 212 determines whether or not to store the voice message in the storage unit 218 based on the delay information.

For example, the delay processing determining unit 217 determines whether or not to perform delay processing based on whether the message acceptance degree is equal to or higher than a reference value. Specifically, the delay processing determining unit 217 determines to execute normal processing when the message acceptance degree is equal to or higher than the reference value, and executes the delay processing when the message acceptance degree is less than the reference value. Then it is decided. However, even if the message acceptance level is less than the reference value, when the user permission recognition unit 216 recognizes that the output of the voice message is permitted, the delay processing determining unit 217 performs normal processing without executing the delay processing. Decide to execute the process. Here, the reference value is a message acceptance level that is a preset reference. This reference value can be set as appropriate based on experimental data and the like. Alternatively, the reference value may be set based on user operation.

Furthermore, based on the division information output from the division processing unit 213, the delay processing determining unit 217 determines the timing for delaying the voice message in units of sentences based on the division positions determined by the division processing.

The storage unit 218 is a storage medium that stores various information. For example, the storage unit 218 stores various information (eg, control programs, audio files) necessary for the output control device 210 to perform various processes. Note that as the storage unit 218, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a combination thereof can be used.

The audio output processing unit 219 executes audio output processing to cause the in-vehicle speaker 204 to output an audio message based on the audio file received by the reception processing unit 212. In this case, the audio output processing unit 219 executes a delay process to delay the timing at which the audio message is output from the in-vehicle speaker 204 based on the delay information output from the delay process determination unit 217.

The delay information output processing section 220 executes an output process of outputting the delay information output from the delay processing determining section 217. For example, the delay information output processing unit 220 executes output processing to cause the in-vehicle speaker 204 to output an audio message based on the delay information. Further, for example, the delay information output processing unit 220 executes output processing to display an image based on the delay information on the display unit. In this way, it is possible to output a voice message based on the delay information or to display an image based on the delay information. In this way, the delay information output processing unit 220 outputs delay information and notifies the driver of the vehicle C1 that the voice message from the operator of the second OPS 300 is delayed.

[Example of configuration of operator system (remote side)]
FIG. 3 is a block diagram showing a configuration example of the second OPS 300. The second OPS 300 is a system that has main functions for realizing operator services for automobiles.

The second OPS 300 includes a communication unit 301 , a microphone 302 , a transmission processing section 303 , a reception processing section 304 , a speaker 305 , an output processing section 306 , and an output section 307 .

The communication unit 301 is connected to the network 10 using wireless communication or wired communication, and exchanges various information with the first OPS 200 via the network 10.

The microphone 302 is for acquiring sound at the installation location of the second OPS 300, and outputs sound information regarding the acquired sound to the transmission processing unit 303. As the microphone 302, for example, one or more microphones or a sound acquisition sensor can be used.

The transmission processing unit 303 executes a transmission process of transmitting the sound information output from the microphone 302 to the first OPS 200 via the communication unit 301. Note that the transmission processing section 303 corresponds to the transmission processing section 214 shown in FIG. 2, and detailed description thereof will be omitted here.

The reception processing unit 304 executes reception processing and output processing for each piece of information received from the first OPS 200 via the communication unit 301, for example, an audio file. For example, the reception processing unit 304 executes a reception process of receiving an audio file transmitted from the first OPS 200 via the communication unit 301. Note that the reception processing unit 304 temporarily stores the received audio file in a storage unit (not shown) as necessary. The reception processing unit 304 also executes audio output processing to cause the speaker 305 to output an audio message based on the received audio file.

The speaker 305 outputs sound at the location where the second OPS 300 is installed. As the speaker 305, for example, one or more speakers can be used. Note that the microphone 302 and the speaker 305 are an example of a user interface, and a part of these may be integrated, or another user interface may be used.

The output processing unit 306 executes reception processing and output processing for each piece of information received from the first OPS 200 via the communication unit 301, such as message acceptance degree and delay information. For example, the reception processing unit 304 executes a reception process of receiving delay information transmitted from the first OPS 200 via the communication unit 301. Further, the reception processing unit 304 executes output processing to cause the output unit 307 to output a voice message or an image based on the received delay information. For example, it is possible to output a voice message or display an image.

The output unit 307 outputs a voice message or an image based on the message acceptance degree and delay information received from the first OPS 200. The output unit 307 is realized by, for example, a user interface such as a speaker or a display panel. Note that instead of the output unit 307, the speaker 305 may be used to output the voice message. In this way, the output unit 307 functions as an output unit for message acceptability and delay information.

In this way, the operator O1 of the second OPS 300 can know the situation of the in-vehicle user U1 (driver or passenger) of the vehicle C1 through the output unit 307.

[Example of audio file communication, audio message output example]
4 and 5 are diagrams schematically showing an example of communication of an audio file and an example of outputting an audio message.

[Example of transition when message acceptance level is higher than the standard value]
FIG. 4 shows an example in which an in-vehicle user U1 who wants to find a delicious restaurant around the vehicle C1 asks a question to the operator O1 of the second OPS 300. Further, FIG. 4 shows an example in which the message acceptability estimated by the acceptability estimator 215 is equal to or higher than the reference value.

As shown in FIG. 4(A), it is assumed that the in-vehicle user U1 utters the voice S1, "Is there any delicious restaurant you recommend near here?" In this case, the in-vehicle microphone 201 acquires the voice S1 and outputs voice information regarding the voice S1 to the transmission processing unit 214. The transmission processing unit 214 executes a transmission process of generating an audio file 401 based on the audio information regarding the audio S1 and transmitting the audio file 401 to the second OPS 300 via the communication unit 211.

Upon receiving the audio file 402 from the first OPS 200, the communication unit 301 of the second OPS 300 outputs the audio file 402 to the reception processing unit 304. Note that the audio file 402 corresponds to the audio file 401.

Upon acquiring the audio file 402, the reception processing unit 304 causes the speaker 305 to output audio information S2 based on the audio file 402, "Is there a delicious restaurant you recommend near here?"

FIG. 4B shows an example in which the second OPS 300 responds to a question from the in-vehicle user U1. As shown in FIG. 4(B), it is assumed that the operator O1 of the second OPS 300 utters the voice S3 "We recommend the Yakiniku restaurant XYZ in ABC Town." In this case, the microphone 302 acquires the audio S3 and outputs audio information regarding the audio S3 to the transmission processing unit 303. The transmission processing unit 303 executes a transmission process of generating an audio file 411 based on the audio information regarding the audio S3 and transmitting it to the first OPS 200 via the communication unit 301.

When the communication unit 211 of the first OPS 200 receives the audio file 412 from the second OPS 300, it outputs the audio file 412 to the reception processing unit 212. Further, the reception processing unit 212 outputs the audio file 412 to the audio output processing unit 219 in order to perform normal processing. Note that the audio file 412 corresponds to the audio file 411.

When the audio output processing unit 219 acquires the audio file 412, it causes the in-vehicle speaker 204 to output audio information S4 based on the audio file 412, "We recommend the Yakiniku restaurant XYZ in ABC Town." Note that in normal processing, the audio file 412 is stored in the storage unit 218 as necessary during audio output processing.

[Example of transition when message acceptance level is less than the standard value]
FIG. 5 shows a transition example when the message acceptance level becomes equal to or higher than the reference value after the message acceptance level becomes less than the reference value, or when the output of the voice message is permitted. Further, FIGS. 5A and 5B show an example in which the message acceptability estimated by the acceptability estimation unit 215 is less than the reference value and the output of the voice message is not permitted by the in-vehicle user U1. show. Further, FIG. 5C shows an example in which the message acceptance degree estimated by the acceptance degree estimating unit 215 is equal to or higher than the reference value, or the output of the voice message is permitted by the in-vehicle user U1.

Note that the example shown in FIG. 5 is a partially modified example of FIG. 4, and in FIG. 5(B), each process up to the second OPS 300 transmitting the audio file 411 to the first OPS 200 is common. Therefore, description of each of these processes will be omitted.

If the message acceptability estimated by the acceptability estimator 215 is less than the reference value, the delay process determining unit 217 determines to execute a delay process that delays the timing at which the audio message is output from the in-vehicle speaker 204. do. In this delay process, as shown in FIG. 5(B), the audio file 412 received from the second OPS 300 is stored in the storage unit 218 until the timing for outputting the audio message from the in-vehicle speaker 204 is reached. That is, upon receiving the audio file 412 from the second OPS 300, the communication unit 211 of the first OPS 200 outputs the audio file 412 to the reception processing unit 212. Further, the reception processing section 212 stores the audio file 412 in the storage section 218 based on the delay information from the delay processing determining section 217, and executes the delay processing. This delay processing is continuously executed until the message acceptability estimated by the acceptability estimator 215 becomes equal to or higher than the reference value or until output of the voice message is permitted by the in-vehicle user U1. That is, the in-vehicle user U1 cannot listen to the audio information S4 based on the audio file 412 from the second OPS 300 until the user U1 can hear the audio message.

As shown in FIG. 5(C), when the message acceptance level is equal to or higher than the reference value, or when the output of the voice message is permitted by the in-vehicle user U1, the delay processing determining unit 217 It is decided to perform normal processing to return the timing for outputting voice messages to normal. However, in this normal processing, if there is an audio file stored in the storage unit 218 for delay processing before returning to the normal processing, that audio file is output preferentially.

That is, as shown in FIG. 5C, the audio output processing unit 219 acquires the audio file 412 stored in the storage unit 218 based on the delay information from the delay processing determining unit 217. Then, the audio output processing unit 219 causes the in-vehicle speaker 204 to output audio information S4 based on the audio file 412, "We recommend the Yakiniku restaurant XYZ in ABC town." In this way, the in-vehicle user U1 can listen to the audio information S4 based on the audio file 412 from the second OPS 300 after being able to listen to the audio message.

In this manner, in the operator service for automobiles, when the degree of message acceptance is low, for example, when the driving load of the in-vehicle user U1 is high, the output of the voice message from the operator O1 of the second OPS 300 is stopped. Furthermore, when the output of the voice message is stopped, both the parties to the conversation, that is, the operator O1 and the in-vehicle user U1, are notified that the output of the voice message has been stopped. Furthermore, if the in-vehicle user U1's acceptance of the message becomes high after the voice message is stopped, the output of the voice message from the operator O1 is resumed.

In addition, if it is automatically determined that the message acceptance degree has become high, or if the in-vehicle user U1 confirms that the message acceptance degree has become high and is permitted by the in-vehicle user U1, the operator O1 You can resume outputting voice messages from. Thereby, the output of the voice message can be resumed smoothly.

[Example of audio message output from split position in audio file]
FIG. 6 is a diagram schematically showing a division example in which a sentence included in a voice message is divided at division positions based on a division criterion, and an output example in which a voice message is output from the division position. Note that the processes from when the in-vehicle user U1 asks the second OPS 300 a question until the audio file 421 is sent to the first OPS 200 in response to the question are similar to those in FIGS. 4(A) and 5(A). However, the content of the audio file transmitted from the second OPS 300 is different. Therefore, descriptions of points common to FIGS. 4 and 5 will be omitted.

FIGS. 6A to 6C show examples in which the message acceptability estimated by the acceptability estimator 215 is equal to or higher than the reference value.

FIG. 6(A) shows an audio file 421 transmitted as a response from the second OPS 300 in response to a question from the in-vehicle user U1. The voice message corresponding to voice file 421 is "We recommend the Yakiniku restaurant XYZ in ABC Town. The special ribs are especially popular."

When the communication unit 211 of the first OPS 200 receives the audio file 421 from the second OPS 300, it outputs the audio file 421 to the reception processing unit 212. Further, the reception processing section 221 outputs the audio file 412 to the division processing section 213 and the audio output processing section 219 in order to perform normal processing.

As shown in FIG. 6(B), the division processing unit 213 executes a division process of dividing the text included in the voice message based on the voice file 421 based on division criteria. In this division process, the division positions at which the sentences included in the voice message are divided into sentences are determined. This division position can be a position where one sentence is completed in the sentences included in the voice message, or a position where a group of sentences with strong contextual relationships are completed in the sentences included in the voice message. That is, the division position in the sentence included in the voice message means a predetermined meaningful break.

For example, the positions of sentences separated by punctuation marks can be used as division positions. These punctuation marks are full periods (.) and commas (,) in Japanese, and full stops (.) and commas (,) in English. Furthermore, the division position may be determined based on descriptive symbols such as a question mark (?), an exclamation mark (!), an ellipsis, and various parentheses. FIG. 6B shows an example in which the position of a sentence ending with "." is the division position. In other words, the division processing unit 213 sets the position of the sentence ending with "." to the division position SP1 for the voice message "We recommend the Yakiniku restaurant XYZ in ABC town. Especially the special ribs are the most popular." Determine as. In this way, by determining the division position SP1 in the sentence included in the voice message, it is possible to execute delay processing so that the operator O1's speech is not stopped at an awkward position, such as in the middle of a topic. It is. Further, the division processing unit 213 outputs division information indicating the division position of each sentence determined by the division processing to the delay processing determining unit 217. Note that a known voice recognition technique can be used to divide these text messages.

As shown in FIG. 6(C), upon acquiring the audio file 421, the audio output processing unit 219 outputs audio information S5 based on the audio file 421, "We recommend Yakiniku restaurant XYZ in ABC town." to the in-vehicle speaker. output from 204. However, in this example, it is assumed that the message acceptance degree changes to less than the reference value immediately after the audio information S5 is output from the in-vehicle speaker 204. In this case, the audio output processing unit 219, based on the delay information from the delay processing determining unit 217, outputs the audio message being output from the in-vehicle speaker 204, “We recommend Yakiniku restaurant XYZ in ABC town. ” is output up to the division position SP1 of the sentence currently being output. The sentences after the division position SP1 are output after the message acceptance level becomes equal to or higher than the reference value, or after the output of the voice message is permitted.

Specifically, when audio information S5 "We recommend Yakiniku restaurant XYZ in ABC Town" is being output from the in-vehicle speaker 204, the audio information up to the division position SP1 is output. Then, as shown in FIG. 6(D), the audio file 421 is stored in the storage unit 218. After that, after the message acceptance level becomes equal to or higher than the reference value, or after output of the voice message is permitted, the sentences starting from the division position SP1 are output as shown in FIG. 6(E). That is, the audio information S6 based on the audio file 421, ``Specially, special ribbi is the most popular.'' is output from the in-vehicle speaker 204.

In the above example, the second OPS 300 is asked about stores around the vehicle C1, but the second OPS 300 may be asked other questions. For example, similar implementation is possible when asking the second OPS 300 about the functions of the vehicle C1.

[Operation example of output control device]
FIG. 7 is a flowchart illustrating an example of voice message output control processing in the output control device 210. Further, this output control processing is executed based on a program stored in the storage unit 218. Moreover, this output control process is always executed every control period. Further, this output control process will be explained with reference to FIGS. 1 to 6 as appropriate.

In step S501, the reception processing unit 212 executes a reception process of receiving the audio file transmitted from the second OPS 300 via the communication unit 211. This audio file is temporarily stored in the storage unit 218 as needed.

In step S502, the division processing unit 213 executes a division process of dividing the audio information based on the audio file received in step S501, that is, the text included in the audio message, at division positions based on preset division criteria. .

In step S503, the receptivity estimating unit 215 executes an estimation process to estimate the message receptivity of the in-vehicle user U1 (driver or passenger) of the vehicle C1, based on each piece of information from the on-vehicle sensors 202.

In step S504, the delay processing determining unit 217 determines whether the message acceptability estimated in step S503 is greater than or equal to a reference value. If the message acceptance degree is equal to or higher than the reference value, the process advances to step S505. On the other hand, if the message acceptance degree is less than the reference value, the process advances to step S507.

In step S505, the audio output processing unit 219 determines whether there is any audio message that has not been output from the in-vehicle speaker 204 among the audio messages transmitted from the second OPS 300. Specifically, it is determined whether there is a voice message based on the voice file received by the reception processing unit 212 in step S501 or a voice message based on the voice file stored in the storage unit 218 in step S510. Ru. If a voice message exists, the process advances to step S506. On the other hand, if there is no voice message, the output control process ends.

In step S506, the audio output processing unit 219 executes output processing to output the audio message from the in-vehicle speaker 204. Note that among the voice messages based on the voice files stored in the storage unit 218 in step S510, if there is a voice message that has not been output from the in-vehicle speaker 204, this voice message is output preferentially.

In step S507, the user permission recognition unit 216 determines whether the output of the voice message is permitted by the user (driver or passenger) of the vehicle C1, based on information from the on-board sensors 202 or the operation reception unit 203. Determine. If the output of the voice message is permitted, the process advances to step S505. On the other hand, if the output of the voice message is not permitted, the process advances to step S508.

In step S508, the audio output processing unit 219 determines whether a voice message is being output from the in-vehicle speaker 204. If a voice message is being output, the process advances to step S509. On the other hand, if the voice message is not being output, the process advances to step S510.

In step S509, the audio output processing unit 219 determines whether or not the sentence up to the division position determined by the division processing unit 213 in step S502 has ended in the audio message being output from the in-vehicle speaker 204. If the sentence up to the division position is completed, the process advances to step S510. On the other hand, if the sentence up to the division position is not completed, monitoring continues until the sentence up to the division position is completed.

In this way, when the operator's voice message is being output, it is determined whether the related topic has ended based on the content of the voice message. If it is determined that there is a break in the topic being output, the process proceeds to a process of accumulating the operator's voice message (step S510).

In step S510, the reception processing unit 212 stores the audio file received in step S501 in the storage unit 218.

In step S511, the delay information output processing unit 220 outputs delay information to the in-vehicle user U1 (driver or passenger) of the vehicle C1 while the audio file received in step S501 is stored in the storage unit 218. Execute processing. Furthermore, while the audio file received in step S501 is being stored in the storage unit 218, the communication unit 211 executes a transmission process of transmitting delay information to the operator O1 of the second OPS 300. In this way, delay information indicating that the output of the voice message is delayed is presented to both the in-vehicle user U1 of the vehicle C1 and the operator O1 of the second OPS 300. Furthermore, the message acceptance degree is also transmitted to the operator O1 of the second OPS 300.

In the above, after the message acceptance level becomes less than the reference value, the delay process continues to be executed until the message acceptance level becomes equal to or higher than the reference value or the output of the voice message is permitted by the in-vehicle user. An example was given. However, after the message acceptance level becomes less than the reference value, the delay process may be executed for a predetermined period. This predetermined period can be set based on the level of message acceptance. For example, the predetermined period can be set longer as the message acceptance level decreases. Furthermore, after the delay processing has been executed for a predetermined period of time, the determination as to whether the message acceptance level has become equal to or higher than the reference value may be performed again, and based on the result of this determination, the delay processing for the predetermined period of time may be sequentially set. .

[Effects of this embodiment]
In this embodiment, the voice message from the operator O1 of the second OPS 300 is output only when the message acceptance level of the user in the vehicle C1 is high or when output of the voice message is permitted. Therefore, it is possible to reduce the chance of missing the content of the utterance, and it becomes possible to concentrate on the dialogue with the operator. In addition, from the operator's perspective of the second OPS 300, even if the user speaks without considering the situation of the user in the car, delay processing is automatically executed, reducing the burden of having to grasp the situation that the operator cannot see. can be reduced. Further, it is also possible to suppress an increase in operation time due to feedback from users in the vehicle.

For example, even in a situation where the in-vehicle user U1 has a low degree of message acceptance, the operator O1 of the second OPS 300 is unable to grasp the situation, and therefore often continues the conversation. Here, when the in-vehicle user U1 performs a voice dialogue while driving, the cognitive load is large for the in-vehicle user U1, and it is important to consider safety. For example, it is important to consider the driver's reaction delay associated with voice interaction while driving. Therefore, in this embodiment, in consideration of the reaction delay in a situation where the operating load is high, the conversation with the operator O1 of the second OPS 300 is stopped in a situation where the operating load is high. In this way, it is possible to have a conversation with the operator O1 depending on the situation such as the driving load of the in-vehicle user U1. Therefore, the safety associated with the use of operator services can be improved in situations where the operating load is high.

In addition, although an example has been shown above in which the output control device 210 executes the message receptivity estimation process and the determination process for determining the timing to output the audio message from the in-vehicle speaker 204, all or part of these processes may be executed. The portion may also be executed on other equipment. In this case, each device that executes a part of each of these processes constitutes an information processing system. For example, it is possible to connect to the second OPS 300, an electronic device used by the in-vehicle user U1 (for example, a smartphone, a tablet terminal, a personal computer, a car navigation device, an IVI (In-Vehicle Infotainment)), and a predetermined network such as the Internet. At least a part of each process can be executed using various information processing devices such as servers and various electronic devices.

Furthermore, a part (or all) of the information processing system that can execute the functions of the output control device 210 may be provided by an application that can be provided via a predetermined network such as the Internet. This application is, for example, SaaS (Software as a Service).

[Configuration example of this embodiment and its effects]
The output control method according to the present embodiment transmits an audio message based on an audio file (an example of audio information) transmitted from the second OPS 300 (an example of a device outside the vehicle) to an in-vehicle speaker 204 (an example of an output device inside the vehicle) using wireless communication. This is an output control method for outputting from (an example). This output control method includes an acquisition process (step S501) that acquires an audio file, and a message acceptance level (receptivity) that indicates whether or not the user in the vehicle C1 is in a state where he or she can listen to an audio message. (step S503), and a determination process (steps S504 to S510) that determines the timing for outputting the audio message from the in-vehicle speaker 204 based on the magnitude of the message acceptance level.

According to this configuration, by changing the timing at which the voice message is output according to the degree of message acceptance, the in-vehicle user can receive the voice message in a situation where the degree of message acceptance is high. Thereby, it is possible to suppress an increase in communication time due to re-listening from the in-vehicle user, and it is possible to suppress an increase in the amount of calculation of each device. Furthermore, a voice message from a user outside the vehicle can be appropriately conveyed in a situation where the user inside the vehicle can hear the voice message.

In the output control method according to the present embodiment, in the determination process (steps S504, S507 to S510), if the message acceptability is lower than a reference value (an example of a preset standard acceptability), the in-vehicle speaker 204 It is decided to execute delay processing to delay the timing of outputting the voice message.

According to this configuration, by delaying the output timing of the voice message when the message acceptance level is lower than the reference value, the in-vehicle user can receive the voice message in a situation where the message acceptance level is high.

In the output control method according to the present embodiment, in the determination process (steps S504 and S505), if the message acceptance level is higher than a reference value (an example of a preset standard acceptance level), the acquisition process (step S501) At the timing when the audio file is acquired, it is decided to execute the normal process of causing the in-vehicle speaker 204 to output an audio message based on the audio file.

According to this configuration, when the message acceptance level is higher than the reference value, the output timing of the voice message is not delayed, so that if the user in the car is in a situation where the message acceptance level is high, the voice message is sent in real time. can be received.

In the output control method according to the present embodiment, in the determination process (steps S504 to S510), if the delay process is being executed, the message acceptability is higher than a reference value (an example of a preset standard acceptability). If this happens, the delay process is finished, the voice message that was delayed by the delay process is output from the in-vehicle speaker 204, and then the voice message based on the voice file acquired in the acquisition process (step S501) is output from the in-vehicle speaker 204. A decision is made to sequentially output the output from the speakers 204.

According to this configuration, if the message receptivity recovers during execution of the delay process, the delay process is ended, thereby eliminating unnecessary delay of the voice message.

In the output control method according to the present embodiment, the message acceptance degree is an index related to at least one of the driving load of the in-vehicle user of the vehicle C1 and the utterance status of the in-vehicle user, and the message acceptance level is an index related to at least one of the driving load of the in-vehicle user and the utterance status of the in-vehicle user, and the message acceptance degree is an index related to at least one of the driving load of the in-vehicle user and the utterance situation of the in-vehicle user, Then, the degree of message acceptance is estimated based on condition data of at least one of the behavior of the vehicle C1, the state of the user inside the vehicle, and information about the surroundings of the vehicle C1.

According to this configuration, the degree of message acceptance can be estimated using condition data of at least one of the behavior of the vehicle C1, the state of the user inside the vehicle, and the surrounding information of the vehicle C1. Changes in message acceptance can be estimated in real time.

In the output control method according to the present embodiment, in the estimation process (step S503), newly acquired condition data is added to the trained learning model that has been trained to estimate the message acceptance degree from the condition data. The message acceptance level is estimated by inputting .

According to this configuration, it is possible to estimate the message acceptance level with higher accuracy by executing the message acceptance level estimation process using the learning model.

In the output control method according to the present embodiment, the estimation process (step S503) includes an input layer and an output layer, and condition data is input as input data to the input layer, and message acceptance degree is input as output data from the output layer. Using a neural network to output and a learning unit that acquires a trained model by causing the neural network to learn using actual values of input data and output data as teaching data, the trained model acquired by the learning unit is , the message acceptance degree is estimated by inputting the newly acquired condition data as input data.

According to this configuration, it is possible to estimate the message acceptance level with higher accuracy by executing the message acceptance level estimation process using the neural network and the learning section.

The output control method according to the present embodiment is an output control method in a communication system that performs a conversation between the operator of the second OPS 300 (an example of a user outside the vehicle) and the user inside the vehicle C1, and is estimated by the estimation process (step S503). The process further includes a presentation process (step S<b>511 ) of transmitting the received message acceptance degree to the second OPS 300 and presenting the message acceptance degree to the operator of the second OPS 300 .

According to this configuration, since the degree of message receptivity of the in-vehicle user can be presented to the operator of the second OPS 300, the operator (a user in a remote location) can easily confirm the degree of message receptivity of the in-vehicle user. .

In the output control method according to the present embodiment, when delay processing of a voice message is being executed, delay information indicating that the delay processing is being executed is transmitted to the second OPS 300 (an example of a device outside the vehicle); It further includes a presentation process (step S511) of presenting the delay information to the operator of the second OPS 300 (an example of a user outside the vehicle).

According to this configuration, the fact that the output of the voice message is delayed can be presented to the operator of the second OPS 300, so the operator can easily understand the reason for the slow response from the in-vehicle user.

In the output control method according to the present embodiment, when a voice message delay process is being executed, a presentation process (step S511) of presenting delay information indicating that the delay process is being executed to the in-vehicle user is further performed. include.

According to this configuration, the fact that the voice message from the second OPS 300 is delayed can be presented to the in-vehicle user, so the in-vehicle user can easily understand the reason why the response from the second OPS 300 is delayed.

The output control method according to the present embodiment includes a recognition process (step S507) for recognizing that the in-vehicle user has given a delay end instruction to end the delay process when the delay process is being executed; In the determination process (steps S504 to S510), when a delay process is being executed and an instruction to end the delay is issued, the delay process is ended, and the audio message that was delayed due to the delay process is transmitted to the in-vehicle speaker. After the audio messages are output from the in-vehicle speaker 204, a decision is made to sequentially output audio messages based on the audio files acquired in the acquisition process (step S501) from the in-vehicle speaker 204.

According to this configuration, when the in-vehicle user instructs to end the delay process, the delay process is terminated, so that stress caused by voice delay caused by individual differences in message acceptance among the in-vehicle users can be alleviated. .

The output control method according to the present embodiment further includes a division process (step S502) of dividing the text included in the voice message at division positions determined based on preset division criteria, and a determination process (step S508). In steps S510 to S510), the timing for outputting the voice message from the in-vehicle speaker 204 is determined in units of sentences divided by the division process, based on the message acceptance level.

According to this configuration, if the in-vehicle user cannot hear the voice message while the voice message is being output from the in-vehicle speaker 204, the delay process can be executed for each sentence divided by the division process. becomes possible. For example, it is possible to start delay processing immediately after a group of sentences with strong contextual relationships are finished. As a result, even if delayed processing occurs, users in the car can exchange voice messages like normal dialogue.

In the output control method according to the present embodiment, the division position is a position where one sentence in a sentence is completed, or a position where a group of sentences with strong contextual relationships are completed in a sentence.

According to this configuration, division positions can be determined based on more appropriate division criteria.

The output control device 210 uses wireless communication to send a voice message based on an audio file (an example of audio information) transmitted from the second OPS 300 (an example of a device outside the vehicle) from an in-vehicle speaker 204 (an example of an output device inside the vehicle). This is an output control device for outputting. The output control device 210 includes a communication unit 211 (an example of an acquisition unit) that acquires an audio file, and a message reception unit that indicates whether or not the user in the vehicle C1 is in a state where it is possible to listen to an audio message. a receptivity estimating unit 215 (an example of an estimating unit) that estimates the degree of acceptance (an example of the receptivity); and a delay processing determining unit that determines the timing for outputting the voice message from the in-vehicle speaker 204 based on the magnitude of the message receptivity. 217 (an example of a determining unit).

According to this configuration, by changing the timing at which the voice message is output according to the degree of message acceptance, the in-vehicle user can receive the voice message in a situation where the degree of message acceptance is high. Thereby, it is possible to suppress an increase in communication time due to re-listening from the in-vehicle user, and it is possible to suppress an increase in the amount of calculation of each device. Furthermore, a voice message from a user outside the vehicle can be appropriately conveyed in a situation where the user inside the vehicle can hear the voice message.

Note that each processing procedure shown in this embodiment is an example for realizing this embodiment, and the order of a part of each processing procedure may be changed to the extent that this embodiment can be realized. Often, a part of each processing procedure may be omitted or other processing steps may be added.

Note that each process shown in this embodiment is executed based on a program for causing a computer to execute each process procedure. Therefore, this embodiment can also be understood as an embodiment of a program that implements the function of executing each of these processes, and a recording medium that stores the program. For example, by performing an update process to add a new function to the output control device, the program can be stored in the storage device of the output control device. This makes it possible to cause the updated output control device to perform each process shown in this embodiment.

Although the embodiments of the present invention have been described above, the above embodiments merely show a part of the application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments. do not have.

1 operator system, 10 network, 20 base station, 200 1st OPS, 201 on-board microphone, 202 on-board sensors, 203 operation reception section, 204 on-board speaker, 210 output control device, 211 communication unit, 212 reception processing section, 213 division processing unit, 214 transmission processing unit, 215 acceptance estimation unit, 216 user permission recognition unit, 217 delay processing determination unit, 218 storage unit, 219 audio output processing unit, 220 delay information output processing unit, 300 second OPS, 301 communication unit, 302 microphone, 303 transmission processing unit, 304 reception processing unit, 305 speaker, 306 output processing unit, 307 output unit 307

Claims (14)

An output control method for outputting a voice message from an output device inside a vehicle based on voice information transmitted from a device outside the vehicle using wireless communication,
an acquisition process for acquiring the audio information;
an estimation process for estimating a degree of acceptance that indicates whether or not the state of the user in the vehicle is such that he or she can hear the voice message;
determining a timing for outputting the audio message based on the audio information from the output device based on the magnitude of the acceptance level;
Output control method. The output control method according to claim 1,
In the determination process, if the acceptability is lower than a preset standard acceptability, it is determined to execute a delay process that delays the timing at which the audio message is output from the output device.
Output control method. The output control method according to claim 2,
In the determination process, if the acceptance level is higher than the reference acceptance level, normal processing causes the output device to output the voice message based on the voice information at the timing when the voice information is acquired in the acquisition process. decide to carry out
Output control method. The output control method according to claim 2 or 3,
In the determination processing, when the delay processing is being executed and the acceptance level becomes higher than the reference acceptance level, the delay processing is terminated, and the voice message that is subject to delay due to the delay processing is terminated. after outputting from the output device, determining to sequentially output the voice message based on the voice information acquired in the acquisition process from the output device;
Output control method. The output control method according to any one of claims 1 to 3,
The degree of acceptance is an index related to at least one of the driving load of the in-vehicle user and the utterance status of the in-vehicle user,
In the estimation process, the degree of acceptance is estimated based on condition data of at least one of the behavior of the vehicle, the state of the user in the vehicle, and surrounding information of the vehicle.
Output control method. The output control method according to claim 5,
In the estimation process, the acceptance level is estimated by inputting the newly acquired condition data as input data to a trained learning model that has been trained to estimate the acceptance level from the condition data. conduct,
Output control method. The output control method according to claim 6,
In the estimation process,
a neural network including an input layer and an output layer, inputting the condition data as the input data to the input layer, and outputting the acceptability as the output data from the output layer;
a learning unit that acquires the trained model by causing the neural network to learn using actual values of the input data and the output data as teaching data,
Estimating the acceptability by inputting the newly acquired condition data as the input data to the trained model acquired by the learning unit;
Output control method. The output control method according to any one of claims 1 to 3,
The output control method is an output control method in a communication system that performs a conversation between a user outside the vehicle of the device outside the vehicle and a user inside the vehicle,
further comprising a presentation process of transmitting the acceptability estimated in the estimation process to a device outside the vehicle and presenting the acceptability to the user outside the vehicle;
Output control method. The output control method according to claim 2 or 3,
The output control method is an output control method in a communication system that performs a conversation between a user outside the vehicle of the device outside the vehicle and a user inside the vehicle,
If the delay processing is being executed, a presentation process of transmitting delay information indicating that the delay processing is being executed to the device outside the vehicle, and presenting the delay information to a user outside the vehicle of the device outside the vehicle. In addition, including
Output control method. The output control method according to claim 2 or 3,
further comprising, when the delay processing is being executed, presenting processing for presenting delay information indicating that the delay processing is being executed to the in-vehicle user;
Output control method. The output control method according to claim 2 or 3,
comprising a recognition process that recognizes that the in-vehicle user has given a delay end instruction to end the delay process when the delay process is being executed;
In the determination process, when the delay process is being executed and the instruction to end the delay is given, the delay process is ended, and the audio message that has been delayed due to the delay process is sent from the output device. After outputting, determining to sequentially output the voice message from the output device based on the voice information acquired in the acquisition process;
Output control method. The output control method according to any one of claims 1 to 3,
further comprising a division process of dividing the text included in the voice message at division positions determined based on preset division criteria,
In the determination process, a timing for outputting the voice message from the output device is determined in units of sentences divided in the division process, based on the acceptance level.
Output control method. The output control method according to claim 12,
The division position is a position where one sentence is completed in the sentence, or a position where a group of sentences with strong contextual relationships are completed in the sentence.
Output control method. An output control device that causes an output device inside the vehicle to output a voice message based on voice information transmitted from a device outside the vehicle using wireless communication,
an acquisition unit that acquires the audio information;
an estimating unit that estimates a degree of acceptance that indicates whether or not a state of a user in the vehicle is such that the voice message can be heard;
a determining unit that determines a timing for outputting the audio message based on the audio information from the output device based on the magnitude of the acceptance level;
Output control device.