JP7178630B2 - VOICE OPERATION METHOD, PROGRAM, VOICE OPERATION SYSTEM, AND MOVING OBJECT - Google Patents

Description

TECHNICAL FIELD The present disclosure relates to a voice operation method, a program, a voice operation system, and a mobile object. The present disclosure particularly relates to a voice operation method, a program, a voice operation system, and a mobile object for operating a mobile object by voice.

Patent Literature 1 discloses a vehicle device. The vehicular device of Patent Document 1 performs wireless communication with a mobile device, controls a vehicle movement device, and adjusts the parking position of the vehicle according to an instruction signal from the mobile device.

JP 2016-7959 A

Patent Literature 1 discloses that a mobile device may be equipped with a microphone, and a user's voice may instruct adjustment of a parking position. In this case, it is disclosed that the user may voice-input "a little while ago" into the portable device, and the vehicle device may move the vehicle forward by 10 [cm].

However, in Patent Literature 1, the adverb phrase "a little while ago" is simply associated with the control "to move the vehicle forward by 10 [cm]". Therefore, it is not possible to operate the moving object by natural speech.

An object of the present invention is to provide a voice operation method, a program, a voice operation system, and a mobile object that can operate the mobile object with natural speech.

A voice manipulation method according to one aspect of the present disclosure is a voice manipulation method executed by a computer. The voice manipulation method includes a transforming step, an extracting step, an interpreting step, and a generating step. The conversion step is a step of converting voice data for operating a mobile object into character data. The extracting step is a step of extracting an ambiguous portion from the character string represented by the character data. The interpretation step is a step of interpreting the ambiguous portion using mobile information including at least one of an operation history of the mobile and a state of the mobile. The generating step is a step of generating an operation command for the moving body from the character data based on the interpretation result of the ambiguous part. The interpreting step interprets the ambiguous portion extracted in the extracting step using a trained model that has learned the relationship between the moving body information and the interpretation result of the ambiguous portion.

A program according to one aspect of the present disclosure is a program for causing one or more processors to execute the voice operation method described above.

A voice manipulation system according to one aspect of the present disclosure includes a converter, an extractor, an interpreter, and a generator. The conversion unit converts voice data for operating a mobile object into character data. The extraction unit extracts an ambiguous part from the character string represented by the character data. The interpretation unit interprets the ambiguous portion using mobile information including at least one of an operation history of the mobile and a state of the mobile. The generation unit generates an operation command for the moving body from the character data based on the interpretation result of the ambiguous part. The interpreting unit interprets the ambiguous part extracted by the extracting unit by using a learned model that has learned the relationship between the moving body information and the interpretation result of the ambiguous part.

A moving object according to one aspect of the present disclosure includes the voice operation system described above and a main body on which the voice operation system is mounted.

According to the aspect of the present disclosure, it is possible to operate a moving object with natural speech.

FIG. 1 is an explanatory diagram of a moving object equipped with a voice operation system according to one embodiment. FIG. 2 is a block diagram of the voice operation system of the above embodiment. FIG. 3 is a flow chart of the operation of the voice operation system of the above embodiment.

1. Embodiment 1.1 Overview FIG. 1 shows a mobile object 100 . A mobile object 100 includes a voice operation system 10 that executes the voice operation method of one embodiment. The voice manipulation method of this embodiment includes a conversion step, an extraction step, an interpretation step, and a generation step. The conversion step is a step of converting voice data for operating the mobile object 100 into character data. The extraction step is a step of extracting an ambiguous portion from a character string represented by character data. The interpretation step is a step of interpreting the ambiguous part using the mobile body information 141 including at least one of the operation history of the mobile body 100 and the state of the mobile body 100 . The generating step is a step of generating an operation command for the moving body 100 from the character data based on the interpretation result of the ambiguous part.

In the voice operation method, the ambiguous part of the character string is interpreted based on the moving body information including at least one of the operation history of the moving body 100 and the state of the moving body 100, and based on the interpretation result of the ambiguous part, character data is converted to the moving body. Generate 100 operation commands. Therefore, even if the character string has an ambiguous part, an operation command reflecting the meaning of the ambiguous part can be given to the moving body 100 . Therefore, when operating the mobile object 100, the user does not have to intentionally speak without ambiguous parts. Therefore, according to the voice operation method, the moving object 100 can be operated with natural speech.

1.2 Details The voice operation method of this embodiment will be described in more detail below. The voice operation method of this embodiment is executed by the voice operation system 10 shown in FIG. The voice operation system 10 enables the user to operate the moving body 100 by voice. The voice operating system 10 is also referred to as a so-called voice user interface (VUI) or voice interaction system. Further, in this embodiment, the mobile object 100 is an automobile.

In this embodiment, the operation of the mobile body 100 includes not only the operation of the mobile body 100 itself but also the operation of devices or systems associated with the mobile body 100 . As the operation of the moving body 100 itself, driving operation of the moving body 100 is assumed. Devices or systems associated with the mobile object 100 include various devices or systems mounted on the main body 101 of the mobile object 100 . For example, devices or systems associated with the moving body 100 include an air conditioning system, a car navigation system, an audio system, a seat adjustment system (reclining system), a power window system, and the like. A device or system associated with the mobile object 100 does not necessarily have to be mounted on the main body 101 of the mobile object 100 , and may be a device or system capable of communicating with the mobile object 100 . Examples of devices or systems that can communicate with the mobile object 100 include terminal devices such as smartphones, tablet terminals, and wearable terminals.

The voice operation system 10 is mounted on a main body 101 of a moving body 100, as shown in FIG. The main body 101 further includes a voice input system 20 , an information acquisition system 30 , a travel control device 40 , an air conditioner 50 , a car navigation system 60 and an audio system 70 .

The voice operation system 10 includes an input/output unit 11, a processing unit 12, a communication unit 13, and a storage unit 14, as shown in FIG. Voice operation system 10 may be implemented by a computer system. A computer system may include one or more processors, one or more connectors, one or more communication devices, one or more memories, and the like.

The input/output unit 11 inputs and outputs information to and from a system mounted on the main body 101 of the mobile object 100 . In this embodiment, the input/output unit 11 is communicably connected to the voice input system 20, the information acquisition system 30, the travel control device 40, the air conditioner 50, the car navigation system 60, and the audio system . The input/output unit 11 includes one or more input/output devices and uses one or more input/output interfaces.

The voice input system 20 is a system for inputting voice data for operating the mobile object 100 . Voice input system 20 includes one or more microphones. The voice input system 20 collects voices with one or more microphones, generates and outputs voice data representing the collected voices. Thereby, the input/output unit 11 acquires voice data from the voice input system 20 .

The information acquisition system 30 is a system for acquiring the state of the mobile object 100 . In particular, the information acquisition system 30 is a system for acquiring the internal state of the mobile object 100 . The state acquired by the information acquisition system 30 particularly includes the dynamic internal state of the mobile object 100 . The dynamic internal state of the mobile object 100 may include at least one of a state related to movement of the mobile object 100 and a state related to a person riding the mobile object 100 . Examples of states related to movement of the mobile object 100 include the speed (current speed) of the mobile object 100, the steering angle of the mobile object 100, and the remaining amount of energy (gasoline, electricity) of the mobile object 100. FIG. Further, examples of states related to the movement of the mobile object 100 include the temperature (current temperature) around the mobile object 100, the noise level around the mobile object 100, and the situation around the mobile object 100 (presence or absence of objects, etc.). mentioned. An example of the state of the person riding the mobile object 100 is the state (current state) of the person riding the mobile object 100 . The information acquisition system 30 may include various sensors such as acceleration sensors, cameras, ultrasonic sensors, microphones, temperature sensors, pressure sensors, millimeter wave (terahertz wave) sensors, and breath sensors. The information acquisition system 30 also acquires information on the mobile object 100 (for example, the steering angle of the mobile object 100 and the remaining amount of energy (gasoline, electricity) of the mobile object 100) from other devices such as the travel control device 40. may include an input/output interface for

A travel control device (travel control system) 40 is a system that controls travel of the moving body 100 . The travel control device 40 can have functions of adjusting the speed of the mobile body 100 , adjusting the steering angle of the mobile body 100 , and switching between forward and backward movement of the mobile body 100 . Therefore, according to the travel control device 40 , it is possible to reduce the speed of the mobile body 100 or stop the mobile body 100 without the driver of the mobile body 100 operating.

An air conditioner (air conditioning system) 50 is a system for adjusting the temperature inside the moving object 100 . The car navigation system 60 is a system for providing guidance to a destination. The car navigation system 60 may include an image display device such as a liquid crystal display, an organic EL display, a 3D head-up display (HUD), and an audio output device. The audio system 70 is a system for reproducing voice, music, and the like. Audio system 70 may include one or more speakers. Since the functions of the air conditioner 50, the car navigation system 60, and the audio system 70 are well known, detailed description thereof will be omitted.

The communication unit 13 inputs and outputs information to and from a system not mounted on the main body 101 of the mobile object 100 . In this embodiment, the communication unit 13 is communicably connected to the external system 80 . The communication unit 13 includes one or more communication devices and implements communication via a communication network. The communication network may consist of not only a network conforming to a single communication protocol, but also a plurality of networks conforming to different communication protocols. Communication protocols may be selected from a variety of well-known wired and wireless communication standards. However, communication between the communication unit 13 and the external system 80 is preferably performed by mobile communication. Note that the communication network may include data communication equipment such as repeater hubs, switching hubs, bridges, gateways, routers, etc., as required.

The external system 80 is a system for acquiring the state of the mobile object 100 . In particular, the external system 80 is a system for acquiring the external state of the mobile object 100 . The external state may include at least one of a state related to the position of the mobile object 100 , a state around the mobile object 100 , and a state of a system that cooperates with the mobile object 100 . An example of the state related to the position of the mobile object 100 is positional information of the mobile object 100 . Examples of the state of the surroundings of the mobile object 100 include traffic information (traffic information, etc.), noise level, weather information, and time information. Examples of the state of the system that cooperates with the mobile unit 100 include information from a mobile unit other than the mobile unit 100, information from a service provider regarding the mobile unit 100, and the like. For example, if another moving body is a moving body (vehicle) in front of the moving body 100, examples of information from the other moving body include detection information of a collision-probable object and information on occurrence of sudden braking. be. Information about the mobile object 100 from the service provider includes the average speed of the road on which the mobile object 100 is traveling. External system 80 may include one or more information providing systems. Examples of information providing systems include the Global Positioning System (GPS) and road traffic information and communication systems.

The storage unit 14 is used to store information used by the processing unit 12 . Storage unit 14 includes one or more storage devices. The storage device is, for example, RAM (Random Access Memory) or EEPROM (Electrically Erasable Programmable Read Only Memory).

The storage unit 14 stores mobile unit information 141 . The mobile body information 141 may include at least one of the operation history of the mobile body 100 and the state of the mobile body 100 . In this embodiment, the mobile body information 141 includes both the operation history of the mobile body 100 and the state of the mobile body 100 .

The operation history is the history of operations of the moving object 100 . The operation history may include the operation history of the mobile object 100 by the voice operation system 10 and the manual operation history of the mobile object 100 by the user. As described above, the operation of the mobile object 100 includes not only the operation of the mobile object 100 itself, but also the operation of devices or systems associated with the mobile object 100 . In the present embodiment, the operation of the mobile object 100 includes operation of the air conditioner 50, operation of the car navigation system 60, and operation of the audio system 70 in addition to operation of the travel control device 40 (operation of the mobile object 100 itself). include.

The state of the mobile body 100 may include at least one of an internal state of the mobile body 100 and an external state of the mobile body 100 . The internal state may include at least one of a state related to movement of the mobile body 100 , a state related to performance of the mobile body 100 , and a state related to a person riding on the mobile body 100 . Here, the state of movement of the mobile object 100 and the state of the person riding the mobile object 100 can be acquired from the information acquisition system 30 . The state related to the performance of the mobile object 100 includes specifications of the mobile object 100 (dimensions of the main body 101 of the mobile object 100, fuel efficiency, etc.). The external state may include at least one of a state related to the position of the mobile body, a surrounding state of the mobile body, and a state of an external system that cooperates with the mobile body. The external state can be obtained from external system 80 . Among the states of the moving body 100, static states may be stored in advance in the storage unit 14, or may be acquired from the external system 80 as necessary.

The processing unit 12 can be implemented by, for example, one or more processors (microprocessors). That is, one or more processors function as the processing unit 12 by executing one or more programs (computer programs) stored in one or more memories. One or more programs may be pre-recorded in one or more memories, or may be provided by being recorded in a non-temporary recording medium such as a memory card or through a telecommunication line such as the Internet.

The processing unit 12 has an acquisition unit 121, a conversion unit 122, an extraction unit 123, an interpretation unit 124, and a generation unit 125, as shown in FIG. The acquisition unit 121, the conversion unit 122, the extraction unit 123, the interpretation unit 124, and the generation unit 125 do not represent actual configurations, but represent functions realized by the processing unit 12. .

Acquisition unit 121 acquires voice data for operating mobile object 100 . That is, the processing unit 12 executes an acquisition step of acquiring voice data for operating the mobile object. The acquisition unit 121 acquires voice data from the voice input system 20 through the input/output unit 11 .

In this embodiment, the acquisition unit 121 acquires voice data from the voice of a specific person. The acquiring unit 121 acquires the voice data from the voice input system 20 when the speaker of the voice represented by the voice data from the voice input system 20 is a specific person. The specific person may be a person registered in the voice operation system 10 in advance. As an example, the specific person is the driver of mobile object 100 . Based on the voice data acquired from the voice input system 20, the acquisition unit 121 determines whether or not the speaker of the voice represented by the voice data is a specific person using speaker recognition technology. The acquisition unit 121 acquires the voice data when the speaker of the voice represented by the voice data is a specific person, but otherwise discards the voice data. That is, the acquisition unit 121 provides the conversion unit 122 with voice data from a specific person. This reduces the possibility that the mobile object 100 is operated by the voice operation system 10 by an unintended person. Note that a conventionally well-known technique can be adopted for the speaker recognition technique, so a detailed description thereof will be omitted.

The conversion unit 122 converts voice data for operating the mobile object 100 into character data. That is, the processing unit 12 executes a conversion step of converting voice data for operating the mobile body 100 into character data. Character data is data representing character strings (text) in a specific language. In this embodiment, Japanese is assumed as the specific language.

The conversion unit 122 converts the voice data from the acquisition unit 121 into character data using voice recognition technology. Note that a conventionally well-known technique can be adopted for the speech recognition technique, so a detailed description thereof will be omitted. In this embodiment, the conversion unit 122 starts outputting the character data to the extraction unit 123 when the character data includes a specific character string. A specific character string is a so-called wake-up phrase. The specific character string can be freely determined, but should be determined so as not to cause confusion with everyday words and phrases and words and phrases for operating the mobile body 100 . Note that the conversion unit 122 may output the character data to the extraction unit 123 until the termination condition is satisfied, even if the character data does not contain a specific character string. As a result, the user does not need to utter a voice containing a specific character string each time. The end condition may be that no voice data is input from the acquisition unit 121 for a predetermined period. The predetermined period may be determined such that the user is allowed to make a sound corresponding to the content of the operation of the mobile object 100 after making a sound including a specific character string.

Extraction unit 123 extracts an ambiguous part from a character string represented by character data. That is, the processing unit 12 executes an extraction step of extracting ambiguous parts from the character string represented by the character data. An ambiguous part is a part or all of a character string whose meaning cannot be uniquely determined by itself. An ambiguous part is an adverbial part that is an adverb or an adverbial phrase. Types of adverbs include state, degree, and the like. In particular, in this embodiment, the adverb part that expresses degree is assumed as the adverb part. Adverbs (or adverbial phrases) of degree include adverbs corresponding to "large", "middle", and "small" degrees. Adverbs with a degree of ``great'' include ``more'', ``much'', and ``much''. Adverbs with a degree of ``middle'' include ``somewhat'' and ``moderate''. Adverbs with a degree of ``small'' include ``slightly'' and ``slightly''. For example, in the case of the character string "Raise the speed a little more", the ambiguous part is "a little more". In the case of the character string "lower the temperature a little", the ambiguous part is "a little". In the case of the character string "show me a further map", the ambiguous part is "more". In the case of the string "turn up the volume significantly", the ambiguous part is "significantly". Also, the ambiguous part is not limited to a part of the character string. The entire string may be an ambiguous part, which is when the string does not form a sentence. An example is a sentence lacking a predicate (verb), a sentence missing a required object, and the like. For example, in the case of character strings such as "more", "not yet", "not yet", "much", "very", "a lot", "a little more", "a little more", and "not yet", all of these are ambiguous parts. be. As another example, for the strings "stop", "no", "stop", all of these are ambiguous parts.

In this embodiment, the extraction unit 123 determines whether the character string includes an ambiguous part. If the character string contains an ambiguous part, the extraction unit 123 extracts the ambiguous part from the character string and provides it to the interpretation unit 124 . On the other hand, the extraction unit 123 provides the character string to the generation unit 125 when the character string does not contain an ambiguous part.

The interpretation unit 124 interprets the ambiguous part using mobile information including at least one of the operation history of the mobile 100 and the state of the mobile 100 . In other words, the processing unit 12 executes an interpretation step of interpreting the ambiguous portion using mobile information including at least one of the operation history of the mobile 100 and the state of the mobile 100 .

The interpretation unit 124 interprets the ambiguous part extracted by the extraction unit 123 . In particular, in interpreting the adverb part, the interpreting unit 124 determines the variable amount or the target value of the parameter related to the operation of the moving body 100 according to the degree of expression of the adverb part. In particular, the interpreting unit 124 determines the degree of expression of the adverb part (for example, three categories of large, medium, and small), and reflects this in the parameter variable or target value.

In this case, first, the interpreting unit 124 uses at least one of the part of the character string excluding the ambiguous part and the moving body information 141 to determine the parameter to be changed according to the voice data. In this embodiment, the interpreter 124 determines whether or not the part of the character string excluding the ambiguous part contains the object.

If there is an object, the interpreter 124 determines parameters based on the object. For example, if the string is "speed up a little more", the ambiguous part is "a little more" and the disambiguating part contains the predicate (verb) "speed up" and the object "speed". there is In this case, the interpreting unit 124 determines the parameters by using the part of the character string that is not ambiguous (the object "velocity"). That is, the interpreting unit 124 selects the speed in the cruise control device 40 as a parameter to be changed according to the voice data. Note that the interpretation unit 124 may determine whether to increase or decrease the parameter based on the verb. Since the verb here is "raise", the interpreter 124 can decide to increase the parameter. For example, if the string is "Turn the temperature down a little", the disambiguated part has the object "temperature". In this case, the interpretation unit 124 selects the temperature (set temperature) of the air conditioner 50 as a parameter to be changed according to the voice data. In addition, when the character string is "display a further map", the part excluding the ambiguous part has the object "map". In this case, the interpretation unit 124 selects the distance from the current location on the map displayed by the car navigation system 60 as a parameter to be changed according to the voice data. Also, if the string is "very loud", then the disambiguated part has the object "volume". In this case, the interpreter 124 selects the volume of the audio system 70 as a parameter to be changed according to the audio data.

If there is no object, the interpreter 124 uses the mobile information 141 to determine parameters to change according to the voice data. For example, the interpretation unit 124 refers to the operation history of the mobile object 100 included in the mobile object information 141 . For example, if the string is "a little more", the string is an adverb only and does not contain an object. In this case, the interpretation unit 124 uses the operation history included in the moving body information 141 to determine the parameters. In other words, the interpreting unit 124 determines the word/phrase related to the ambiguous part based on the operation history of the mobile object 100 . For example, if the operation history indicates that the driver increased the speed immediately before inputting the voice data, the interpretation unit 124 selects the speed in the cruise control device 40 as a parameter to be changed according to the voice data. Furthermore, the interpretation unit 124 may determine whether to increase or decrease the parameters from the operation history. Since the operation history mentions the speed, the interpretation unit 124 can decide to increase the parameter. Further, according to the operation history, if the driver has raised the temperature (set temperature) of the air conditioner 50 immediately before inputting the voice data, the interpretation unit 124 changes the set temperature of the air conditioner 50 according to the voice data. parameter. Further, according to the operation history, if the driver has increased the screen size of the car navigation system 60 immediately before inputting the voice data, the interpreting unit 124 adjusts the screen size of the car navigation system 60 according to the voice data. to select as the parameter to change. Further, according to the operation history, if the driver has increased the volume of the audio system 70 immediately before inputting the audio data, the interpreting unit 124 sets the volume of the audio system 70 according to the audio data as the parameter select.

In this way, the interpreting unit 124 uses at least one of the portion of the character string excluding the ambiguous portion and the moving body information 141 to determine the parameter to be changed according to the voice data. When the interpretation unit 124 determines the parameter to be changed, it is unclear from the character string how much the parameter should be changed. Therefore, the interpreting unit 124 determines the variable amount or target value of the parameter related to the operation of the moving body 100 according to the degree of expression of the adverb part.

In this embodiment, the interpretation unit 124 interprets the ambiguous part according to one or more preset rules. Table 1 below shows an example of a rule for the parameter velocity. The rules in Table 1 are rules for determining the velocity variable from the degree of the adverb part and the current value of the velocity. The rules in Table 1 determine the variables from the current values of the parameters to be changed.

According to this rule, the interpreting unit 124 acquires the current value of speed from the moving body information, and determines the variation of speed according to the degree of the adverb part and the current value of speed. For example, if the ambiguous part is "a little more", the degree of degree of the adverb part is "small". Then, if the current speed value is 90 km/h, the interpreting unit 124 interprets that the vague part (adverb part) of "just a little more" is "5 km/h" for the parameter of speed. Also, if the current speed value is 15 km/h, the interpretation unit 124 interprets that the vague part (adverb part) of "just a little more" is "1 km/h" for the parameter of speed.

Table 2 below shows an example of rules relating to the parameter of the distance from the current location of the map displayed on the screen of the car navigation system 60 . The rules in Table 2 determine the target value from the degree of the adverb part and the current value of speed. Here, the target value is the distance from the current location on the map displayed on the screen of car navigation system 60 . In this example, the current value of velocity is used to determine the target value of the parameter distance from the current location. That is, the interpretation of the ambiguity may refer to another parameter in determining the variable or target value of the parameter.

According to this rule, the interpreting unit 124 acquires the current speed value from the moving object information, and determines the target value of the distance from the current location of the map to be displayed according to the degree of the adverb part and the current speed value. For example, when the ambiguous part is "more", the degree of degree of the adverb part is "small". Then, if the current value of the speed is 90 km/h, the interpreting unit 124 determines that the degree of the ambiguous part (adverb part) of "more" is "2 km" for the parameter of the distance from the current value of the map to be displayed. , the interpretation. Also, if the current value of the speed is 15 km/h, the interpretation unit 124 determines that the degree of the ambiguous part (adverb part) of "more" is "300 m" for the parameter of the distance from the current value of the map to be displayed. , the interpretation.

On the other hand, if the ambiguous part is not an adverb of degree, the interpretation unit 124 determines whether the ambiguous part has a negative connotation. For example, the interpreting unit 124 determines whether the ambiguous part extracted by the extracting unit 123 matches the ambiguous part having a negative connotation. Examples of ambiguities that have a negative connotation are the strings "stop", "no", and "stop". If the ambiguous part has a negative connotation, the interpreting part 124 gives the generating part 125 an interpretation result indicating that the ambiguous part has a negative connotation.

As described above, the interpretation unit 124 interprets the ambiguous part extracted by the extraction unit 123 . Then, the interpreting unit 124 gives the interpretation result of the ambiguous part to the generating unit 125 .

The generation unit 125 generates an operation command for the moving body 100 from character data. That is, the generation unit 125 executes a generation step of generating an operation command for the moving body 100 from character data.

The generation unit 125 generates an operation command for the moving body 100 from the character data when there is no ambiguous part in the character data. For example, if the character string is "Increase speed by 5 kilometers", the generation unit 125 determines parameters based on the object of the character string. Also, the generation unit 125 determines whether to increase or decrease the parameter based on the predicate (verb) of the character string. Since the verb here is "raise", the generator 125 determines to increase the parameter. Furthermore, the generation unit 125 is: Determines the variability of a parameter from a string. Here, based on the part "5 km", the variation of the speed parameter is determined to be 5 km/h. As a result, the generator 125 generates an operation command instructing to increase the speed by 5 km/h. Generation unit 125 outputs the generated operation command to cruise control device 40 via input/output unit 11 . Such processing when there is no ambiguous part in the character data can be realized by a conventionally known technique, so detailed description thereof will be omitted.

If the character data has an ambiguous part, the generation unit 125 generates an operation command for the moving object 100 from the character data based on the result of interpretation by the interpretation unit 124 . Here, if the ambiguous part is an adverb part that expresses a degree, the generation unit 125 acquires the parameter to be changed according to the speech data and the target value or variable of the parameter from the interpretation result of the interpretation unit 124. , to generate operation commands from character data. For example, if the character string is "speed up a little more", the interpreting unit 124 interprets that the vague part (adverb part) of "a little more" is "5 km/h" for the parameter speed. Suppose In this case, the generating unit 125 generates an operation command instructing to increase the speed by 5 km/h from the character data representing the character string “Increase the speed a little more” based on the interpretation result of the interpreting unit 124. do. Generation unit 125 outputs the generated operation command to cruise control device 40 via input/output unit 11 . Therefore, even if there is an ambiguous part in the character string represented by the character data, the voice operation system 10 enables voice operation of the moving object 100 .

On the other hand, when the interpretation unit 124 interprets the ambiguous part to have a negative connotation, the generation unit 125 generates an operation command for canceling the operation of the moving body 100 . In particular, the generation unit 125 generates an operation command for canceling the previous operation of the moving body 100 . The previous operation of the moving body 100 is, for example, the operation of the moving body 100 by the operation command generated immediately before. As an example, when the driver of the moving object 100 utters the sound "stop", the extraction unit 123 extracts the part "stop" from the character data representing the character string "stop" as the ambiguous part. Then, the interpretation unit 124 interprets the ambiguous part "stop" as having a negative connotation. Then, if the generation unit 125 generated an operation command instructing to increase the speed by 5 km/h immediately before, it generates an operation command for canceling this operation. Generation unit 125 then outputs the generated operation command to cruise control device 40 via input/output unit 11 . As a result, the voice operation system 10 can cope with the case where the driver performs an erroneous operation and wants to cancel it immediately.

1.3 Operation The operation of the voice operation system 10 will be briefly described below with reference to the flowchart of FIG. First, when a voice is input to the voice input system 20 (S11: Yes), in the voice operation system 10, the acquisition unit 121 acquires voice data from the voice input system 20 through the input/output unit 11 (S12). Note that in the present embodiment, the acquisition unit 121 acquires voice data only when the voice of the voice data is the voice of a specific person. In other words, voice data of voices other than those of a specific person are discarded. The voice data acquired by the acquisition unit 121 is converted into character data by the conversion unit 122 (S13). Then, it is determined by the extraction unit 123 whether or not the character string represented by the character data has an ambiguous part (S14). If there is no ambiguous part (S14: No), the generator 125 generates an operation command from the character data (S15). The generator 125 then outputs the generated operation command to the corresponding system (S16). On the other hand, if there is an ambiguous part in the character string represented by the character data (S14: No), the interpretation unit 124 interprets the ambiguous part (S17). Then, the generation unit 125 generates an operation command from the character data based on the result of interpretation by the interpretation unit 124 (S18). Then, the generation unit 125 outputs the operation command generated using the result of interpretation by the interpretation unit 124 to the corresponding system (S16).

1.4 Summary The voice operation system 10 described above comprises a converter 122 , an extractor 123 , an interpreter 124 and a generator 125 . The conversion unit 122 converts voice data for operating the mobile object 100 into character data. Extraction unit 123 extracts an ambiguous part from a character string represented by character data. The interpreting unit 124 interprets the ambiguous part using the moving body information 141 including at least one of the operation history of the moving body 100 and the state of the moving body 100 . The generation unit 125 generates an operation command for the moving body 100 from the character data based on the interpretation result of the ambiguous part. According to the voice operation system 10, the moving object 100 can be operated by natural speech.

In other words, it can be said that the voice operation system 10 executes the following method (voice operation method). The voice manipulation method includes a conversion step, an extraction step, an interpretation step, and a generation step. The conversion step is a step of converting voice data for operating the mobile object 100 into character data. The extraction step is a step of extracting an ambiguous portion from a character string represented by character data. The interpretation step is a step of interpreting the ambiguous part using the mobile body information 141 including at least one of the operation history of the mobile body 100 and the state of the mobile body 100 . The generating step is a step of generating an operation command for the moving body 100 from the character data based on the interpretation result of the ambiguous part. According to this voice operation method, like the voice operation system 10, the moving object 100 can be operated by natural speech.

The voice operation method is implemented by one or more processors executing a program (computer program). This program is a program for causing one or more processors to execute the voice operation method. According to such a program, the mobile object 100 can be operated with natural speech, similar to the voice operation method.

2. Modifications Embodiments of the present disclosure are not limited to the embodiments described above. The above-described embodiment can be modified in various ways according to design and the like, as long as the object of the present disclosure can be achieved. Modifications of the above embodiment are listed below.

In one modification, the input/output unit 11, the communication unit 13, and the storage unit 14 are not essential. The voice operation system 10 is configured such that the processing unit 12 exchanges information with the voice input system 20, the information acquisition system 30, the travel control device 40, the air conditioner 50, the car navigation system 60, the audio system 70, and the external system 80. It can be.

In a modified example, the acquisition unit 121 does not necessarily have to acquire voice data from a specific person's voice. The conversion unit 122 may output the character data to the extraction unit 123 even if a specific character string is not confirmed in the character string represented by the character data. Alternatively, the acquisition unit 121 may acquire voice data and provide it to the conversion unit 122 when a specific operation is performed, so that the conversion unit 122 outputs character data to the extraction unit 123 . A specific operation includes pressing a button for notifying the start of speech. This makes it possible to input voice like push-to-talk (PTT).

In one variation, the rules used by interpreter 124 may be functions. That is, it is possible to use a function that gives a parameter target value or variable to the degree of the adverb part and the numerical value from the moving body information 141 . This improves the degree of freedom in setting parameter target values or variables.

In one variation, the interpreter 124 can utilize a trained model. For example, the interpreting unit 124 may interpret the ambiguous portion extracted by the extracting unit 123 using a learned model that has learned the relationship between the moving object information 141 and the interpretation result of the ambiguous portion. In other words, the interpretation step may be a step of interpreting the ambiguous portion extracted in the extraction step using a learned model that has learned the relationship between the moving body information 141 and the interpretation result of the ambiguous portion.

In this case, the storage unit 14 stores interpretation models used for interpretation by the interpretation unit 124 . The interpretation model provides an artificial intelligence program (algorithm) with the relationship between the moving object information 141 and the interpretation result of the ambiguous part using a learning data set showing the relationship between the moving object information 141 and the interpretation result of the ambiguous part. Generated by learning. An artificial intelligence program is a model of machine learning, and for example, a neural network, which is a kind of hierarchical model, is used. The interpreted model is generated by having the neural network perform machine learning (eg, deep learning) on a training dataset. In other words, the interpretation model is a trained model that has learned the relationship between the moving object information 141 and the interpretation result of the ambiguous part. The interpretation model may be generated by the processing portion 12 of the voice manipulation system 10 or by an external system 80 .

The interpretation unit 124 uses the interpretation model stored in the storage unit 14 to interpret the ambiguous part based on the character data in which the character string includes the ambiguous part. Upon receiving the character data, the interpretation unit 124 inputs the character data and the moving body information 141 to the interpretation model, and causes the interpretation model to output the interpretation result of the ambiguous part. The interpreting unit 124 provides the generation unit 125 with the interpretation result of the ambiguous part when it is obtained.

In the voice operation system 10, the interpreter 124 may collect and accumulate learning data for generating an interpretation model. The learning data newly collected by the interpretation unit 124 in this way can be used for re-learning the interpretation model, thereby improving the performance of the interpretation model (learned model). In particular, it is considered that the adverbial portion of degree was misinterpreted if the ambiguous portion received after the adverbial portion of degree was interpreted had a negative connotation. Therefore, the performance of the interpretation model can be improved by using the erroneous interpretation of the adverb part that expresses the degree as training data.

In the above embodiment, when the parameters to be changed by the voice data are the speed and the distance from the current location on the map to be displayed, the current value of the speed is used as the moving body information. However, the moving object information 141 is not limited to the current speed value, and any of the above-described operation history of the moving object 100 and information (internal information and external information) of the moving object 100 may be used. For example, in the interpretation of the ambiguous portion regarding the travel control device 40, the speed (current speed) of the mobile object 100, the steering angle of the mobile object 100, the performance of the mobile object 100, and the average speed of the road on which the mobile object 100 is traveling are used. can. For example, in the interpretation of the ambiguous part regarding the air conditioner 50, the ambient temperature (current temperature) of the mobile object 100, the state of the person riding the mobile object 100 (current state), the position information of the mobile object 100, the weather information , time information is available. For example, in the interpretation of the ambiguous part regarding the car navigation system 60, the speed (current speed) of the mobile object 100, the steering angle of the mobile object 100, the remaining amount of energy (gasoline, electricity) of the mobile object 100, the position of the mobile object 100 Information, weather information, time information is available. For example, in interpreting the ambiguous portion regarding the audio system 70, the noise level around the vehicle 100, the state (current state) of the person riding the vehicle 100, weather information, and time information can be used.

Although the voice operation system 10 is mounted on the main body 101 of the moving object 100 in the above embodiment, the voice operation system 10 may be mounted on a device capable of communicating with the moving object 100 . In other words, the voice operation system 10 can also be used for remote operation of the moving object 100 . Further, the mobile object 100 to be operated by the voice operation system 10 is not limited to automobiles, and can be applied to mobile objects other than automobiles, such as motorcycles, trains, drones, aircraft, construction machinery, and ships. be.

In one variation, the voice operation system 10 may be composed of multiple computers. For example, the functions of the voice manipulation system 10 (particularly the acquisition unit 121, the conversion unit 122, the extraction unit 123, the interpretation unit 124, and the generation unit 125) may be distributed among multiple devices. Furthermore, at least part of the functions of the voice operation system 10 may be realized by cloud (cloud computing), for example.

The execution subject of the voice operation system 10 described above includes a computer system. A computer system has a processor and memory as hardware. The processor executes a program recorded in the memory of the computer system, thereby realizing the function of the voice operation system 10 of the present disclosure as an execution entity. The program may be prerecorded in the memory of the computer system, or may be provided through an electric communication line. Also, the program may be provided by being recorded on a non-temporary recording medium such as a computer system-readable memory card, optical disk, hard disk drive, or the like. A processor in a computer system consists of one or more electronic circuits including semiconductor integrated circuits (ICs) or large scale integrated circuits (LSIs). A field programmable gate array (FGPA), an ASIC (application specific integrated circuit), or a reconfigurable field programmable gate array (FGPA) or ASIC (application specific integrated circuit), which can be programmed after the LSI is manufactured, or which can reconfigure junction relationships inside the LSI or set up circuit partitions inside the LSI. Logical devices can also be used for the same purpose. A plurality of electronic circuits may be integrated into one chip, or may be distributed over a plurality of chips. A plurality of chips may be integrated in one device, or may be distributed in a plurality of devices.

3. Aspects As is clear from the above embodiments and modifications, the present disclosure includes the following aspects. In the following, reference numerals are attached with parentheses only for the purpose of clarifying correspondence with the embodiments.

A first aspect is a voice manipulation method comprising a conversion step, an extraction step, an interpretation step, and a generation step. The "conversion step" converts voice data for operating the mobile body (100) into character data. The extraction step extracts an ambiguous part from the character string represented by the character data. , the ambiguous part is interpreted using information of the mobile body (100) including at least one of an operation history of the mobile body (100) and a state of the mobile body (100). generates an operation command for the moving body 100 from the character data based on the interpretation result of 1. According to this mode, the moving body 100 can be operated by natural speech.

The second aspect is based on the voice manipulation method of the first aspect. In a second aspect, the ambiguous part is an adverbial part that is an adverb or an adverbial phrase. According to this aspect, the moving object (100) can be operated with more natural speech.

A third aspect is based on the voice manipulation method of the second aspect. In a third aspect, the adverbial part expresses degree. According to this aspect, the moving object (100) can be operated with more natural speech.

A fourth aspect is based on the voice operation method of the third aspect. In the fourth aspect, in the interpretation of the adverb part, a variable amount or a target value of a parameter related to the operation of the moving body (100) is determined according to the degree expressed by the adverb part. According to this aspect, the moving object (100) can be operated with more natural speech.

A fifth aspect is based on the voice operation method of the fourth aspect. In a fifth aspect, the interpreting step uses at least one of the portion of the character string excluding the ambiguous portion and the information of the moving body (100) to determine the parameter to be changed according to the voice data. . According to this aspect, the moving object (100) can be operated with more natural speech.

A sixth aspect is based on the voice operation method of any one of the first to fifth aspects. In the sixth aspect, the interpreting step interprets the ambiguous portion extracted in the extracting step using a trained model that has learned the relationship between the moving body (100) information and the interpretation result of the ambiguous portion. . According to this aspect, the accuracy of interpretation of the ambiguous part is improved.

A seventh aspect is based on the voice operation method of any one of the first to fifth aspects. In a seventh aspect, the interpretation step interprets the ambiguous part according to one or more preset rules. According to this aspect, the accuracy of interpretation of the ambiguous part is improved.

An eighth aspect is based on the voice operation method of any one of the first to seventh aspects. In the eighth aspect, when the ambiguous portion is interpreted to have a negative connotation in the interpretation step, the generation step generates an operation command for stopping the operation of the moving body (100). According to this aspect, it becomes easier to perform an operation to interrupt the movement of the moving object (100).

A ninth aspect is based on the voice operation method of any one of the first to eighth aspects. In the ninth aspect, the interpreting step determines a phrase related to the ambiguous part based on the operation history of the moving object (100). According to this aspect, the accuracy of interpretation of the ambiguous part is improved.

A tenth aspect is based on the voice operation method of any one of the first to ninth aspects. A tenth aspect includes an obtaining step of obtaining the voice data from a specific person's voice before the converting step. According to this aspect, it is possible to reduce the possibility that the moving body (100) is operated by an unintended person.

The eleventh aspect is based on the voice operation method of any one of the first to tenth aspects. In the eleventh aspect, the state of the moving body (100) includes at least one of an internal state of the moving body (100) and an external state of the moving body (100). According to this aspect, the accuracy of interpretation of the ambiguous part is improved.

The twelfth aspect is based on the voice operation method of the eleventh aspect. In the twelfth aspect, the internal state is at least one of a state related to movement of the moving object (100), a state related to performance of the moving object (100), and a state related to a person riding the moving object (100). including one. The external state includes at least one of a state related to the position of the moving body (100), a state of the surroundings of the moving body (100), and a state of a system that cooperates with the moving body (100). According to this aspect, the accuracy of interpretation of the ambiguous part is improved.

A thirteenth aspect is a program (computer program) for causing one or more processors to execute the voice operation method of any one of the first to twelfth aspects. According to this aspect, the mobile object (100) can be operated by natural speech.

A fourteenth aspect is a voice manipulation system (10) comprising a transforming unit (122), an extracting unit (123), an interpreting unit (124), and a generating unit (125). The conversion unit (122) converts voice data for operating the mobile body (100) into character data. The extraction unit (123) extracts an ambiguous part from the character string represented by the character data. The interpreting unit (124) interprets the ambiguous part using mobile (100) information including at least one of an operation history of the mobile (100) and a state of the mobile (100). The generation unit (125) generates an operation command for the moving body (100) from the character data based on the interpretation result of the ambiguous part. According to this aspect, the mobile object (100) can be operated by natural speech.

A fifteenth aspect is a moving object (100) comprising the voice operation system (10) of the fourteenth aspect and a main body (101) on which the voice operation system (10) is mounted. According to this aspect, the mobile object (100) can be operated by natural speech.

10 voice operation system 122 conversion unit 123 extraction unit 124 interpretation unit 125 generation unit 100 moving body 101 main body

Claims (14)

A computer implemented voice manipulation method comprising:
a conversion step of converting voice data for operating a mobile object into character data;
an extraction step of extracting an ambiguous part from a character string represented by the character data;
an interpretation step of interpreting the ambiguous portion using mobile information including at least one of an operation history of the mobile and a state of the mobile;
a generation step of generating an operation command for the moving body from the character data based on the interpretation result of the ambiguous portion;
including
The interpreting step interprets the ambiguous portion extracted in the extracting step using a trained model that has learned the relationship between the moving body information and the interpretation result of the ambiguous portion.
Voice operation method. the ambiguous part is an adverbial part that is an adverb or an adverbial phrase;
2. The voice operation method according to claim 1. The adverbial part expresses degree,
3. The voice operation method according to claim 2. In the interpretation of the adverb part, determining a variable amount or target value of a parameter related to the operation of the moving object according to the degree expressed by the adverb part;
4. The voice operation method according to claim 3. The interpreting step uses at least one of a part of the character string excluding the ambiguous part and the moving body information to determine a parameter to be changed according to the voice data.
5. The voice operation method according to claim 4. The interpretation step interprets the ambiguous part according to one or more preset rules.
The voice operation method according to any one of claims 1-5. If the interpretation step interprets the ambiguous part to have a negative connotation, the generation step generates an operation command for stopping the operation of the moving body.
The voice operation method according to any one of claims 1 to 6. The interpreting step determines a phrase related to the ambiguous part based on the operation history of the moving object.
The voice operation method according to any one of claims 1-7. An obtaining step of obtaining the audio data from a specific person's speech before the converting step.
The voice operation method according to any one of claims 1-8. The state of the moving body includes at least one of an internal state of the moving body and an external state of the moving body,
The voice operation method according to any one of claims 1-9. The internal state includes at least one of a state related to movement of the mobile body, a state related to performance of the mobile body, and a state related to a person riding the mobile body,
The external state includes at least one of a state related to the position of the moving body, a state surrounding the moving body, and a state of a system that cooperates with the moving body.
11. The voice operation method of claim 10. for causing one or more processors to execute the voice operation method according to any one of claims 1 to 11,
program. a conversion unit that converts voice data for operating a mobile object into character data;
an extraction unit that extracts an ambiguous part from a character string represented by the character data;
an interpretation unit that interprets the ambiguous part using mobile information including at least one of an operation history of the mobile and a state of the mobile;
a generation unit that generates an operation command for the moving body from the character data based on the interpretation result of the ambiguous part;
with
The interpreting unit interprets the ambiguous portion extracted by the extracting unit using a trained model that has learned the relationship between the moving object information and the result of interpretation of the ambiguous portion.
voice control system. a voice operation system of claim 13;
a main body on which the voice operation system is mounted;
comprising
Mobile.

Images