CN112261234B

CN112261234B - Method for voice assistant to execute local task and voice assistant

Info

Publication number: CN112261234B
Application number: CN202011147839.XA
Authority: CN
Inventors: 张伟萌; 李国华
Original assignee: Beijing Moran Cognitive Technology Co Ltd
Current assignee: Huizhou Jiakang Technology Co ltd
Priority date: 2020-10-23
Filing date: 2020-10-23
Publication date: 2021-11-16
Anticipated expiration: 2040-10-23
Also published as: CN112261234A

Abstract

The invention discloses a method for a voice assistant to execute local tasks and the voice assistant, wherein the method comprises the following steps: when the voice assistant is in a user call mode, the voice assistant executes the local task instruction and informs a user of a first execution result, and the voice assistant continuously receives a voice instruction of a first user in the next round, determines that the voice instruction is a call content instruction and receives a second call voice of a second user in the next round; and judging whether the received call content instruction and/or the second call voice include the update information of the executed local task instruction, if so, replacing all or part of information in the executed local task instruction according to the update information to generate a second local task instruction, executing the second local task instruction, and informing a user of a second execution result. By the method and the voice assistant, user experience is improved.

Description

Method for voice assistant to execute local task and voice assistant

Technical Field

The embodiment of the invention relates to the technical field of information processing, in particular to a method for a voice assistant to execute a local task and the voice assistant.

Background

In recent years, with the development of artificial intelligence, voice assistants can provide users with more and more abundant services, for example, the voice assistants can take the place of the users to receive calls and provide call contents to the users in a voice or text manner for the users to view or listen to. However, the voice assistant in the prior art has the following disadvantages when replacing the user to answer the call:

1. in the process of the conversation between the voice assistant and the calling party, the user can check or listen to the conversation content in real time, and can intervene the conversation at any time according to the self desire, at the moment, the participants of the conversation are changed into the called user and the calling party by the voice assistant and the calling party, and the voice assistant is in a suspension state and does not participate in the conversation any more. Then, if the user needs the voice assistant to replace the user to communicate with the calling user, at this time, because the voice assistant cannot acquire the previous communication content between the user and the called user, the voice assistant cannot have a smooth conversation with the calling user, and the communication effect is not good.

2. In the prior art, a user can control a response given by a voice assistant to a certain extent, when the privacy of a called user is involved in a call between the voice assistant and the calling user, the voice assistant can prompt the user and respond to the calling user according to a response of the user, for example, when the calling user inquires about an address of the called user, the voice assistant inquires about whether the calling user provides the address to the opposite side, and if the calling user does not select to provide the address, the voice assistant responds that the called user's address is' wrong ', and the address is inconvenient to provide'. However, in the above method, the user needs to be prompted under certain conditions, and the user needs to be preset or configured, so that the requirement of the user for diversified control of the voice assistant in the call cannot be met.

3. The prior art does not consider the scenario where a call and a user's conversation with a voice assistant are ongoing simultaneously. A typical scenario is that before an incoming call is received, the user has a voice conversation with the voice assistant, for example, the user is inquiring an air ticket through the voice assistant, and when the incoming call is received and answered by the voice assistant, if the user makes a voice input, the voice assistant cannot judge whether the voice instruction is used for continuing the voice conversation between the user and the voice assistant or controlling the voice assistant to answer the calling user during the call. Another typical scenario is when a user has a need to talk to a voice assistant during a call, for example, when the calling user asks the called user whether the called user is available or not, the called user wants to determine whether the called user is available or not by asking the voice assistant, and then issues a voice command "i have available or not" which should be sent to the voice assistant, but at this time, the mobile terminal cannot determine whether the user's voice input is sent to the calling user or the voice assistant. In the scenario where a call and a user's conversation with a voice assistant are simultaneously performed, how to handle the user's voice input is not known in the prior art, nor is a corresponding solution given.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a method for a voice assistant to participate in a call and the voice assistant.

The invention provides a method for a voice assistant to execute local tasks, which comprises the following steps:

step 1, when the voice assistant is in a user communication mode, the voice assistant receives a voice instruction of a first user and determines that the voice instruction is a communication content instruction, and the voice assistant sends the communication content instruction to a second user as a third communication voice; the voice assistant receives a second call voice of a second user and provides the second call voice to the first user;

step 2, the voice assistant receives the voice command of the first user again and judges that the voice command is a local task command;

step 3, the voice assistant executes the local task instruction and informs a user of a first execution result;

step 4, the voice assistant continuously receives the voice command of the first user of the next round, determines that the voice command is a call content command, and receives the second call voice of the second user of the next round;

step 5, judging whether the call content instruction and/or the second call voice received in the step 4 comprise the update information of the executed local task instruction, if so, executing the step 6;

and 6, replacing all or part of information in the executed local task instruction according to the updated information to generate a second local task instruction, executing the second local task instruction, and informing a user of a second execution result.

A voice assistant for performing local tasks, wherein the voice assistant is in a user talk mode, the voice assistant comprising a transceiver unit, a processing unit;

the receiving and sending unit is also used for receiving a voice instruction of the first user, receiving second call voice of the second user, providing the second call voice for the user, responding to the trigger of the processing unit, and sending the third call voice to the second user;

the processing unit is further configured to, when it is determined that the voice instruction is a call content instruction, take the call content instruction as a third call voice, and trigger the transceiver unit to send the third call voice to a second user;

the receiving and sending unit is also used for receiving the voice instruction of the first user again;

the processing unit is further configured to determine whether the re-received voice instruction is a local task instruction, and if so, execute the local task instruction, and trigger the transceiver unit to notify the user of the first execution result;

the receiving and sending unit is also used for responding to the trigger of the processing unit and informing a user of the first execution result;

the receiving and sending unit is also used for receiving the voice instruction of the first user in the next round and receiving the second call voice of the second user in the next round;

the processing unit is further configured to determine that the voice instruction of the first user in the next round is a call content instruction, determine whether the call content instruction and/or the second call voice of the second user in the next round includes update information of the executed local task instruction, if so, replace all or part of information in the executed local task instruction according to the update information to generate a second local task instruction, execute the second local task instruction, and trigger the transceiver unit to notify the user of a second execution result;

the transceiver unit is further configured to notify the user of the second execution result in response to the trigger of the processing unit.

The invention also provides a computer device characterized in that it comprises a processor and a memory, in which a computer program is stored that is executable on the processor, which computer program, when executed by the processor, implements the method as described above.

The invention also provides a computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program is executable on a processor, and when executed implements the method as described above.

The invention also provides a terminal which is characterized by comprising the voice assistant.

By the method, the local task instruction is automatically updated according to the voice call content of the user and the calling party, so that the time delay of the user for answering the calling party can be reduced, and the user experience is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a voice assistant system in one embodiment of the invention.

FIG. 2 is a method for switching operating modes of a voice assistant in one embodiment of the invention.

FIG. 3 is a method for a voice assistant to engage in a call in one embodiment of the invention.

FIG. 4 is a voice assistant in one embodiment of the invention.

FIG. 5 is a method for a voice assistant to engage in a call in one embodiment of the invention.

FIG. 6 is a method for a voice assistant to recognize and execute user instructions in one embodiment of the invention.

FIG. 7 is a method for a voice assistant to perform local tasks in one embodiment of the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments and specific features of the embodiments of the present invention are detailed descriptions of technical solutions of the embodiments of the present invention, and are not limited to technical solutions of the present invention, and the technical features of the embodiments and the embodiments of the present invention may be combined with each other without conflict.

1. Voice assistant system

FIG. 1 illustrates a block diagram of a voice assistant system that may be implemented on a stand-alone device or across multiple devices. In some embodiments, some of the modules, units or functions of the voice assistant system belong to a server, and the rest of the modules, units or functions belong to a terminal, and the terminal can communicate with the server through one or more networks. In some embodiments, some of the modules of the voice assistant system, such as the processing unit 102 and the interaction engine 112, may belong to both the server and the terminal, and the remaining modules belong to one of the server or the terminal.

The voice assistant system mainly comprises: a transceiver unit 101, a processing unit 102, a database 103, etc. The processing unit includes n interaction engines 112, where n is a positive integer greater than or equal to 1, and each interaction engine 112 may include a semantic understanding module 201, a dialog management and control module 202, a dialog generation module 203, and a command execution module 204. The processing unit 102 is connected to the transceiver 101, and can receive data input by a user through the transceiver 101, and output interactive data, such as session data, task execution process and result, to the user through the transceiver 101. When the voice assistant replaces the user to make a call, such as to answer a call, the transceiving unit is further configured to receive a call voice of the opposite-end user from, for example, a communication unit of the mobile terminal, receive an instruction of the voice assistant user from a user instruction collecting unit, such as an input method or a microphone, and transmit the call voice generated by the voice assistant or received from the voice assistant user to the opposite-end user through the communication unit of the mobile terminal and the like. That is, the transceiving unit 101 is used for the voice assistant to receive and/or send messages in the present invention, and the rest of the functions of the voice assistant are performed by the processing unit. The transceiving unit 101 may comprise a human-machine interface or the like.

In some embodiments, the interaction engine 112 may include an interaction main engine and/or at least one interaction sub-engine. Wherein the interaction main engine is a default engine of the voice assistant system. The interaction sub-engine on the server can be owned by the voice assistant system, can be generated by local training of the terminal and uploaded through a proprietary interface, and can be owned by the voice assistant system, can be generated by local training of the terminal and loaded in the voice assistant system, and can be downloaded by the terminal from the server and loaded in the voice assistant system.

In some embodiments, each interaction engine (interaction main engine, interaction sub engine) is capable of performing at least one task, i.e. each interaction engine may be associated with at least one task, and the tasks capable of being performed by different interaction sub engines may be the same or different. The interaction engine defines at least one slot for each task associated therewith. The terminal can download one or more interaction sub-engines from the server according to the needs of the user and load the interaction sub-engines in the voice assistant system of the terminal, and typical interaction sub-engines are exemplified as follows:

the task executable by the weather interaction sub-engine is weather query, the slot defined by the weather query task is a city and a date, and a user can download the weather interaction sub-engine and interact with the weather interaction sub-engine to query the weather forecast of the city;

the task executable by the music interaction sub-engine is music playing, and the slot defined by the music playing task comprises a singer name and a song name, so that a user can download the music interaction sub-engine and interact with the music interaction sub-engine to play music according to the requirements of the user;

the tasks executable by the incoming call answering sub-engine are incoming call answering tasks, including answering incoming calls in various different call modes and providing records of call contents for users. The slot defined for the task includes caller identity, social relationship, etc., and the incoming call answering sub-engine may further define at least one incoming call intention slot group for the task, where the incoming call intentions of different incoming call intention slot groups are different, and the included sub-slots are also different, for example, for a greeting incoming call intention slot group, which includes an incoming call intention sub-slot (the sub-slot is filled with a greeting by default at this time), a greeting sub-slot, a greeted person sub-slot, etc., and for an express delivery incoming call intention slot group, which includes an incoming call intention sub-slot (the slot is filled with an express delivery by default at this time), a time sub-slot, a location sub-slot, an express company sub-slot, a parcel size sub-slot, etc. The user can download the incoming call answering sub-engine and answer the incoming call by using the incoming call answering sub-engine, and the specific working mode of the incoming call answering sub-engine is described in detail below. When an incoming call is detected, an interactive main engine of the voice assistant or the voice assistant automatically triggers or calls an incoming call answering sub-engine, the incoming call answering sub-engine or the interactive main engine determines that the incoming call should be answered by a user or answered by the voice assistant according to an incoming call answering rule, and the incoming call answering sub-engine establishes a call session process for the call as long as the incoming call is connected and the call is established with a calling user no matter the incoming call is answered by the user or the voice assistant, so that the incoming call answering sub-engine is distinguished from a local session process of the called user and the voice assistant. In some call modes, after receiving the second call voice of the calling user, the incoming call answering sub-engine performs voice recognition and semantic recognition on the second call voice, fills a slot position corresponding to the call task based on a recognition result, generates alternative call voice or third call voice according to the slot position filling result, and performs multiple rounds of conversations with the calling user.

The task executable by the calling sub-engine is a calling task, and specifically, the calling sub-engine replaces a calling user to actively call a called user and wholly or partially replaces the calling user to communicate with the called user. The slot defined for the task includes the called subscriber number, the called subscriber category, the primary call intention, and the like. The user can download the calling sub-engine and use the calling sub-engine to make automatic calling. When the called user answers the call, the main calling sub-engine establishes a call with the called user, and establishes a call session process for the call, which is distinguished from the local conversation process of the user and the voice assistant.

The incoming call answering task and the calling task both belong to a conversation task. In the invention, other tasks which can be executed by the voice assistant and are except for the conversation task are called as local tasks, a user can trigger the local tasks through local task instructions, the local task instructions do not belong to the conversation session process, and the local task instructions have the function of enabling the voice assistant to execute specific other tasks, namely the local tasks, but the local task instructions are not unrelated to the conversation contents of a calling user and a called user, and only the local task instructions are not required to be loaded to the conversation session process so as not to be sent to a second user, and the execution result of the local tasks is not required to be loaded to the conversation session process. For example, the incoming call answering sub-engine of the voice assistant is in a user call mode, and the calling user inquires whether the called user is available in the open air, the called user wants to determine whether the called user is available in the open air by inquiring the voice assistant, and then sends out a voice instruction "mute", i.e., whether i are available in the open air ", which is a local task instruction, and the voice instruction triggers the main engine or the schedule sub-engine to execute a local task, i.e., a schedule inquiry task, so as to determine the schedule of the user in the open air, and reply an inquiry result, such as" available in the afternoon in the open air ", to the user. It should be noted that the above local task instruction and the meaning of "local" in the local task are used only to illustrate that the task is not a conversational task, and not that the voice assistant does not need to access the network and utilize information in the network when performing the task.

Aiming at the local task, a user can interact with the voice assistant system in a voice or text mode, the voice assistant system determines the user intention (namely determines the task) according to the instruction of the user, determines key knowledge data corresponding to each slot position associated with the user intention, and fills the key knowledge data into the corresponding slot position. The voice assistant system then performs the task based on the populated slot or slots.

In some embodiments, the interaction main engine determines, based on a user instruction only, an interaction sub-engine capable of processing the user instruction, which itself does not perform a specific task. Therefore, the determination of the user intention during the local task may be performed by an interaction main engine in the voice assistant system, the interaction main engine being further configured to select one or more interaction sub-engines to process the user instruction based on the determined user intention, the slot filling and the local task execution being performed by the one or more interaction sub-engines determined by the interaction main engine.

In some embodiments, the above-mentioned function of the incoming call answering sub-engine is performed by the interactive main engine, i.e. a separate incoming call answering sub-engine is not set up.

2. Call mode

The voice assistant can provide three call modes, namely a voice assistant call mode, an auxiliary call mode and a user call mode.

In the present invention, no matter which kind of call mode is adopted, the voice assistant is the session establishing process of the call, which is called as the session process, that is, no matter which kind of call mode is adopted, the voice assistant participates in the call, but in different call modes, the voice assistant participates in the call in different ways, which will be described in detail below.

Hereinafter, the calling user and the second user are the same, the calling user and the second user can be replaced by each other in any scene, the called user, the user of the voice assistant and the first user are the same, and the calling user, the second user and the first user can be replaced by each other in any scene. In the following, the mentioned users refer to the first user unless explicitly indicated as calling user or second user.

In the present invention, the voice of the calling party during the call is referred to as a second call voice, and for the second call voice, the voice assistant sends a third call voice to the calling party as a response to the second call voice, wherein, in different call modes, the process of the voice assistant generating or acquiring the third call voice is different, which will be described in detail below.

Regardless of the call mode, the voice assistant may receive user instructions from the first user while the call is in progress. The user instruction may be one or more of a mode switch instruction, a local task instruction, and a call content instruction. The user instruction collecting unit of the mobile terminal, such as an input method or a microphone, collects the user instruction and then sends the user instruction to the voice assistant, the voice assistant judges the type of the user instruction is one of the one or more instructions, corresponding operation is executed according to the type of the instruction, and the user instruction can be a voice instruction or a text instruction. In addition, the mode switching instruction can be generated by a virtual key or a physical key for the user.

In the prior art, all user voices collected by a mobile terminal in a call process are sent to a second user. In the invention, no matter which communication mode is in, in the communication process, the user instruction collected by the mobile terminal is firstly sent to the voice assistant (such as a receiving and sending unit of the voice assistant), when the user instruction is judged to be the communication content instruction by the voice assistant, the third communication voice is generated based on the communication content instruction according to the corresponding mode of the communication mode and sent to the calling user, and when the user instruction is judged to be the local task instruction by the voice assistant, the local task is executed, thereby ensuring that the user can also control the voice assistant to execute the local task simultaneously in the communication process. And when the voice assistant judges that the user instruction is a mode switching instruction, switching the communication mode of the voice assistant.

Preferably, the mode switching instruction includes a call mode to which switching is desired, or the mode switching instruction corresponds to the call mode to which switching is desired. For example, the voice assistant receives a mode switching instruction of the user in the voice assistant call mode, where the auxiliary call mode is carried, or the mode switching instruction corresponds to the auxiliary call mode, for example, in the case that the mode switching instruction is generated by a key corresponding to the auxiliary call mode, the voice assistant switches to the auxiliary call mode, and the other two modes are similar. Fig. 2 shows a procedure for switching between three call modes by a mode switching instruction. In another embodiment, the three call modes are ordered to form a directed loop, and the mode switching instruction only represents that the call mode needs to be switched, and does not carry an instruction of which call mode to switch to specifically, for example, the directed loop is: the voice assistant communication mode- > auxiliary communication mode- > user communication mode- > voice assistant communication mode, namely, if the voice assistant is currently in the voice assistant communication mode, after receiving a mode switching instruction from a user, the voice assistant switches to the auxiliary communication mode, in the auxiliary communication mode, after receiving the mode switching instruction, the voice assistant switches to the user communication mode, in the user communication mode, after receiving the mode switching instruction, the voice assistant communication mode is switched to, and so on. As mentioned above, the mode switching instruction may be voice or text, and the user may also generate the mode switching instruction by touching or pressing a virtual key or an entity key located on the mobile terminal. Besides the mode switching instruction, the invention can also adopt other mechanisms to realize the automatic switching between different conversation modes.

Preferably, the voice assistant may receive local task instructions from the first user during the ongoing call, the local task instructions being user instructions related to the local task. After receiving a local task instruction, the voice assistant loads the task instruction into a local session process for processing when the local session process associated with the task instruction exists between a user and the voice assistant, and when the local session process associated with the task instruction does not exist between the user and the voice assistant, a local session process is newly established and the task instruction is loaded. Specifically, after receiving a user instruction, the voice assistant determines whether the user instruction includes a local task instruction identification word, and if so, determines that the user instruction does not belong to the conversation session process, and loads the task instruction into the local session process of the voice assistant.

Preferably, when the call is not in progress, the user sends the local task instruction without carrying the local task instruction identification word.

Preferably, the call content instruction is an instruction related to responding to the second communication voice, and when the voice assistant is in a different call mode, the operation is also different after receiving the call content instruction, which will be described in further detail below.

(1) Voice assistant talk mode

When it is determined that the incoming call should be answered by the voice assistant according to the incoming call answering rules, the voice assistant answers the incoming call and automatically operates in a voice assistant call mode. In the voice assistant talk mode, the voice assistant replaces the user to talk with the calling user, i.e., the voice assistant responds to the received second communication voice. Specifically, the voice assistant connects the incoming call to establish a call with the calling party, and establishes a new call session process for the call. Thereafter, the voice assistant receives the second call voice of the calling user and generates a third call voice to be sent to the calling user so as to respond to the second call voice of the calling user. Meanwhile, the voice assistant provides the call content comprising the second call voice and the third call voice to the user in real time for the user to view or listen. The call content may be voice or text.

And in the voice assistant communication mode, after receiving the second communication voice of the calling user, the incoming call answering sub-engine performs voice recognition and semantic recognition on the second communication voice, fills the slot corresponding to the communication task based on the recognition result, and generates third communication voice according to the slot filling result. Meanwhile, in the voice assistant call mode, the voice assistant reserves a mechanism for the first user to indirectly participate in the call, and the user can change the logic (also referred to as "decision diagram state") of the voice assistant for generating the third call voice through the call content instruction, so that the call process can proceed according to the direction desired by the user. Specifically, after receiving a call content instruction of a user, performing voice recognition or voice recognition and semantic recognition on the call content instruction, determining an intention of the user to insert the instruction, changing logic for generating third call voice, generating the third call voice according to the changed logic, and giving a different answer or response to the call content instruction which is not received.

The timing of the user inserting the call content command may be any timing during the call, such as receiving a second call voice but the voice assistant has not responded to the second call voice, or the voice assistant has responded to the previous second call voice but has not received the next second call voice. For example, if the calling user is an intermediary, the received second call voice is "mr. Zhang, you recommend two sets of houses today", the third call voice generated by the voice assistant is "good, you say", then, the second call voice received by the voice assistant is "first set of houses is in central garden cell, 89 th, price is 450 ten thousand", and the third call voice generated by the voice assistant is "several floors of the set of houses? Is there an elevator? "the voice assistant in the prior art will ask continuously until the required information is recorded and the call is ended or the user takes over the call. In the application, the user checks the call content provided by the voice assistant in real time, and gives a call content instruction "i do not have an interest in a cell" after the house is located in a central garden cell, the voice assistant determines that the user intends to inquire about a second set of houses, and therefore does the generated third call voice "is the second set of houses? "; after that, before receiving the next second speech sound, the user gives the speech content instruction again to "end the call as soon as possible", then after receiving the second speech sound "the second set of houses is in a happy cell, 110 is flat, the price is 550 ten thousand, the second top level", because the logic for generating the third speech sound is changed based on the speech content instruction, the voice assistant does not generate the third speech sound "does the set of houses have an elevator", but generates "good" and i consider the third speech sound again, thanks, and see again ".

And in the voice assistant call mode, calling other interaction engines in the process of generating the third call voice by the voice assistant, and generating the third call voice based on task execution results given by the other interaction engines.

In the communication process and during the duration that the voice assistant is in the voice assistant communication mode, when the number of times that the user gives the communication content instruction accords with a first preset rule, the communication mode in which the voice assistant is located is automatically switched from the voice assistant communication mode to the auxiliary communication mode, at this time, the voice assistant adopts a first prompting mode to prompt the user of the automatic switching of the communication mode, and the first prompting mode can be one or more of sound, vibration, voice or characters. The first preset rule may be: the number of times that a user gives a call content instruction in a first preset time exceeds a first preset number; or; and the user continuously gives corresponding call content instructions and the like for a second preset number of second call voices.

In the voice assistant working mode, after receiving a mode switching instruction of a user, the voice assistant switches the working mode to be an auxiliary call mode or a user call mode.

It should be noted that, in the present application, the call mode of the voice assistant refers to the operating mode thereof, which may be one of three call modes, and the call mode of the voice assistant refers to a call mode in which the voice assistant replaces the user to make a call with the calling user and reserves an indirect participation mechanism for the user.

(2) Auxiliary talk mode

In the auxiliary call mode, the voice assistant reserves a mechanism for the first user to directly participate in the call, and the first user can participate in the call process between the voice assistant and the calling user through the call content instruction. In the auxiliary call mode, the voice assistant generates a third call voice and transmits the third call voice to the calling party when receiving the second call voice from the calling party. The generation process specifically comprises the following steps: and starting a timer when receiving a second call voice of the calling user, generating an alternative call voice, sending the call content instruction as a third call voice to the calling user if the call content instruction of the first user is received before the timer expires, and sending the alternative call voice as the third call voice to the calling user if the call content instruction of the first user is not received before the timer expires. I.e., in the secondary talk mode, the received second communication voice is responded to by the voice assistant and/or the user.

In the scenario of the intermediary phone, if the voice assistant is in the auxiliary call mode, the execution process when the user does not give the call content instruction is the same as the call mode of the voice assistant, and when the user gives the call content instruction "i do not interest in this cell", the voice assistant sends the call content instruction as a third speech sound to the calling user.

In the auxiliary call mode, if a call content instruction of a first user is received before a timer expires, and the call content instruction is sent to a calling user as a third speech sound, the voice assistant also compares the call content instruction with the alternative call voice, and modifies or updates slot filling content corresponding to a call task, so that the subsequently generated alternative call voice is more suitable for the user.

In the auxiliary call mode, in the process of generating the alternative call voice by the voice assistant, calling other interaction engines, and generating the alternative call voice also based on task execution results given by the other interaction engines.

And sending the call content instruction as a third speech sound to a calling party, specifically: when the user gives a call content instruction in a text form, the voice assistant generates a third speech sound in a voice synthesis mode according to the call content instruction, or the voice assistant generates third call content with the same or similar semantics based on the call content instruction and generates third call voice in a voice synthesis mode according to the third call content. For example, when the call content command is "null", the third call content is "meaningless, i do not null now".

And sending the call content instruction as a third speech sound to a calling party, specifically: when a user gives a call content instruction in a voice form, the voice assistant generates third call voice in a voice synthesis mode, specifically, the voice assistant performs voice recognition on the call content instruction and generates third speech sound in the voice synthesis mode according to a voice recognition result, and by the mode, a calling user can be shielded to know environmental sound in the environment where a called user is located, so that privacy of the called user is protected, and meanwhile definition of the third call voice can be ensured; or, when the user gives the call content command in the form of voice, the voice assistant uses the voice synthesis mode to transfer the call content command to generate a third call voice, specifically, performing voice recognition on the call content instruction, generating third call content with the same or similar semantics based on the voice recognition result, generating third call voice by adopting a voice synthesis mode according to the third call content, for example, the speech recognition result is "null", the third communication content may be "meaningless, i are now null", by the method, the environment sound of the calling user in the environment of the called user can be shielded, therefore, the privacy of the called user is protected, the definition of the third call voice can be ensured, in addition, the user can participate in the call with short voice without making the calling user feel offensive or discomfortable, and the method and the device are particularly suitable for scenes that the user is busy or inconvenient to communicate with the third call voice for a long time. Or, when the user gives the call content instruction in the form of voice, the voice assistant takes the call content instruction as a third call voice.

In the auxiliary call mode, the manner in which the voice assistant generates the third call voice may be set by the user in advance.

In the auxiliary call mode, during the call and during the duration that the voice assistant is in the auxiliary call mode, when the number of times that the user gives a call content instruction conforms to a second preset rule, the call mode in which the voice assistant is located is automatically switched from the auxiliary call mode to the user call mode, at this time, the voice assistant adopts a second prompt mode to prompt the user of automatic switching of the call mode, and the second prompt mode may be one or more of sound, vibration, voice or characters. The second prompting mode is different from the first prompting mode, such as different sound or different vibration mode. The second preset rule may be: the number of times that the user gives the call content instruction in the second preset time exceeds a third preset number; or; and the user continuously gives corresponding call content instructions and the like for a fourth preset number of second call voices. The second preset rule may be the same as or different from the first preset rule, that is, all or part of the second preset time and the first preset time, the third preset number and the first preset number, and the fourth preset number and the second preset number may be the same as or different from each other.

In the auxiliary call mode, when a third preset rule is met, the call mode in which the voice assistant is located is automatically switched from the auxiliary call mode to the voice assistant call mode, wherein the third preset rule is as follows: the user gives no call content instruction within a third preset time; or the user does not give the call content instruction continuously aiming at the fifth preset number of second call voices. The third preset time may be the same as or different from the second preset time and the first preset time, and the fifth preset number may be the same as or different from the fourth preset number and the second preset number.

In another embodiment, the operating mode of the voice assistant is not automatically switched according to the second preset criteria when the mobile terminal is in a mute or vibrate mode.

In the auxiliary call mode, after receiving a mode switching instruction of a user, the voice assistant switches the working mode of the voice assistant into a voice assistant working mode or a user call mode.

(3) User talk mode

When the incoming call is determined to be answered by the first user according to the incoming call answering rules, the voice assistant automatically works in the user call mode, or when the incoming call is connected by the first user, the voice assistant automatically works in the user call mode.

In the user call mode, after the incoming call is connected, the voice assistant establishes a call session process for the call. In the user communication mode, the voice assistant sends the instruction of the first user as a third communication voice to the calling user only after receiving the instruction of the first user and judging the instruction as a communication content instruction, namely in the user communication mode, the voice assistant passively takes the communication content instruction of the user as the third communication voice to respond to the calling user according to a preset mode only after receiving the communication content instruction of the user, and does not actively respond like in other communication modes, and in the user communication mode, the active response is carried out by the user. In the user call mode, the voice assistant also receives a second call voice of a second user and provides it to the user. The voice assistant repeatedly executes the above process as the calling party and the called party communicate with each other.

In addition, in the user call mode, when the communication unit of the mobile terminal receives a second call voice of a second user, the second call voice is sent to the voice assistant and provided to the user by the voice assistant. The voice assistant understands the call contents and the call progress of the first user and the second user based on the call content instruction of the first user and the second call voice of the second user, so that when the voice assistant is switched to other call modes, a third call voice is generated based on the comprehension of the call contents and the call progress.

In the user communication mode, after receiving a mode switching instruction of a user, the voice assistant switches the working mode of the voice assistant into a voice assistant working mode or an auxiliary communication mode.

And in the user communication mode, after receiving a second communication voice, the voice assistant judges whether a communication content instruction is not received within a fourth preset time, if so, the voice assistant switches the communication mode from the user communication mode to the voice assistant communication mode or an auxiliary communication mode, generates a third communication voice and sends the third communication voice to the second user. When the auxiliary call mode is switched, the timer is not needed to be started for the second call voice, and the generated alternative call voice is directly sent to the second user as the third call voice, so that the second user is prevented from waiting for too long.

The duration of the fourth preset time is the same as or different from the timer in the auxiliary call mode.

In the user call mode, when the received user instruction is a call content instruction, the voice assistant sends the call content instruction as a third speech sound to the calling user in the specific process and the auxiliary call mode. And will not be described in detail herein.

In the user call mode, the call content instruction is specifically a call content text instruction or a call content voice instruction.

The invention provides a method for a voice assistant to participate in a call, and referring to fig. 3, the method comprises the following steps:

the method comprises the steps that the voice assistant is in a user call mode, in the call process of a first user and a second user, the voice assistant receives an instruction of the first user and judges the type of the instruction, and when the instruction is a call content instruction, the voice assistant sends the call content instruction to the second user as a third speech sound; the voice assistant receives a second call voice of a second user and provides the second call voice to the first user; the voice assistant understands the call content and the call progress based on the call content instruction of the first user and the second call voice of the second user;

the voice assistant judges whether the call mode of the voice assistant needs to be switched, if so, the voice assistant switches the call mode of the voice assistant, and the voice assistant generates a third speech sound based on the understanding of the call content and the call progress and sends the third speech sound to the second user.

Preferably, understanding the call content and the call progress based on the call content instruction of the first user and the second call voice of the second user specifically includes: performing voice recognition on the second communication voice of the second user, performing semantic recognition based on the voice recognition result and the communication content instruction of the first user, and understanding the communication content and the communication progress of the first user and the second user; or performing voice recognition on the call content instruction of the first user and the second call voice of the second user, performing semantic recognition based on the voice recognition result, and understanding the call content and the call progress of the first user and the second user.

Preferably, the voice assistant is in the user call mode specifically: the voice assistant detects whether the incoming call is connected by the first user, and if so, the voice assistant is in a user communication mode; or, when the voice assistant is in other communication modes, judging whether the received instruction of the first user is a mode switching instruction for indicating switching to the user communication mode, and if so, switching to the user communication mode based on the mode switching instruction.

Preferably, the voice assistant receives an instruction of the first user, and specifically, the voice assistant receives an instruction from a user instruction collecting unit of the mobile terminal.

Preferably, the voice assistant receives a second call voice of the second user, and specifically, the voice assistant receives the second call voice of the second user from a communication unit of the mobile terminal. The communication unit of the mobile terminal receives a second call voice from a second user through the network.

Preferably, the call content instruction is sent to the second user as a third speech sound, specifically, the third speech sound is sent to a communication unit of the mobile terminal, and the communication unit sends the third speech sound to the second user.

Preferably, when the received instruction of the first user is a local task instruction, the local task instruction is not sent to the second user by the voice assistant, and the voice assistant loads the local task instruction into the local session process.

Preferably, the local task instruction is not sent to the second user by the voice assistant, and specifically, the local task instruction is not sent to the communication unit of the mobile terminal by the voice assistant.

Preferably, the voice assistant determines whether to switch the call mode, specifically: the voice assistant judges whether the received instruction of the first user is a mode switching instruction; or after receiving a second call voice, the voice assistant determines whether a call voice instruction is not received within a fourth preset time.

Preferably, the voice assistant determines whether the received instruction of the first user is a mode switching instruction, specifically, determines whether the mode switching instruction is generated by a user through a key on the mobile terminal, and/or determines whether the type of the text instruction or the voice instruction is a mode switching instruction when the instruction is a text instruction or a voice instruction.

Preferably, the mode switching instruction is not sent by the voice assistant to the second user, and in particular, the mode switching instruction is not sent by the voice assistant to the communication unit.

Preferably, the voice assistant switches the call mode, specifically, the voice assistant switches the call mode from the user call mode to the voice assistant call mode or the auxiliary call mode.

Preferably, the understanding of the call content and the call progress of the first user and the second user specifically includes: and filling the slot position corresponding to the call task based on the call voice instruction and the second call voice.

Preferably, the understanding of the call content and the call progress of the first user and the second user specifically includes: and filling the slot position corresponding to the call task based on the semantic recognition result.

Preferably, the voice assistant generates a third speech sound based on the understanding of the call content and the call progress and sends the third speech sound to the second user, specifically, generates the third call sound based on the filled slot.

The above method is described below in a specific scenario.

Assuming that the user a sets other calls except for the phones of the family, the friends and the colleagues to be answered by the voice assistant, when receiving the call of the friend B, the mobile terminal determines that the call should be answered by the user a. When the user is detected to answer the incoming call and establish a call, the voice assistant establishes a call session process for the call. And the user A and the friend B start the following call, and in the call process, the voice assistant continuously receives the call voice instruction and the second voice call from the user, performs voice recognition and voice recognition on the call instruction and the second voice call, and understands the call content and the call progress. The conversation process is as follows:

and (4) friend B: university classmate C, Beijing;

the user A: when?

And (4) friend B: yesterday;

the user A: is a business trip?

And (4) friend B: kah-kah, about five days in Beijing;

the user A: then we find a time to gather a bar; (thereafter, user A goes to an emergency meeting)

And (4) friend B: is you available on friday evening?

The voice assistant determines that "you are available on friday night" on receiving the second call voice? If the call voice instruction of the user is not received within five seconds later, the call mode of the voice assistant is automatically switched to the call mode of the voice assistant, and the call is processed according to the understanding of the call content and the call progress:

the voice assistant: searching a user schedule, determining that the user A is available at friday night, generating a third pronunciation of ' I ' available ' and sending the third pronunciation to the friend B;

and (4) friend B: that is friday night bar, we see bar in the new street, you can get a few points roughly?

The voice assistant: searching a user schedule to determine that the user A goes off duty at six points, searching for the time of 45 minutes for six points from the location of the user company to a new street through a map interaction sub-engine, generating a third speech sound of 'I can arrive at about 6 points 45', and sending the third speech sound to a friend B;

and (4) friend B: that time is up.

The voice assistant: arrival time, arrival time.

After the call is finished, the voice assistant adds a project of ' 6 points for departure ' in the user schedule, goes to a new street to get together with friends B and college classmates C ', and provides the call process to the user A in a voice or text mode.

By the method, in the call process, the voice assistant in the user call mode can acquire the call content of the first user and the second user, understand the call content and the call progress, and can more intelligently respond to the second user after the voice assistant is switched to the voice assistant call mode or the auxiliary call mode, so that the user experience is improved.

The invention provides a voice assistant for participating in a call, which comprises a receiving and sending unit and a processing unit, and is shown in fig. 4; the voice assistant is in a user talk mode;

the receiving and sending unit is used for receiving an instruction of a first user, receiving second call voice of a second user and providing the second call voice to the first user in the call process of the first user and the second user; further for transmitting the third spoken voice to a second user in response to a trigger by the processing unit;

the processing unit is configured to: judging the type of the instruction of the first user, and triggering the receiving and sending unit to send the call content instruction to a second user as a third speech sound when the instruction is the call content instruction; understanding call content and call progress based on the call content instruction of the first user and the second call voice of the second user;

the processing unit is further configured to determine whether a call mode in which the voice assistant is located needs to be switched, if so, switch the call mode in which the voice assistant is located, generate a third call voice based on understanding of call content and call progress, and trigger the transceiver unit to send the third call voice to the second user.

Preferably, the voice assistant is for the mobile terminal.

Preferably, the processing unit understands the call content and the call progress based on the call content instruction of the first user and the second call voice of the second user, specifically: performing voice recognition on the call voice of the second user, performing semantic recognition based on the voice recognition result and the call content instruction of the first user, and understanding the call content and the call progress of the first user and the second user; or performing voice recognition on the call content instruction of the first user and the second call voice of the second user, performing semantic recognition based on the voice recognition result, and understanding the call content and the call progress of the first user and the second user.

Preferably, the voice assistant is in the user call mode specifically: the processing unit detects whether an incoming call is connected by a first user, and if so, controls the voice assistant to be in a user call mode; or, the processing unit judges whether the received instruction of the first user is a mode switching instruction for instructing switching to the user call mode when the voice assistant is in other call modes, and if so, switches to the user call mode based on the mode switching instruction.

Preferably, the instruction of the first user is received, specifically, the transceiver unit receives the instruction from the user instruction acquisition unit of the mobile terminal.

Preferably, the second call voice of the second user is received, specifically, the second call voice of the second user is received from the communication unit of the mobile terminal. The communication unit of the mobile terminal receives a second call voice from a second user through the network.

Preferably, the processing unit triggers the transceiver unit to transmit the call content instruction as a third speech sound to the second user, specifically, the processing unit triggers the transceiver unit to transmit the third speech sound to a communication unit of the mobile terminal, and the communication unit transmits the third speech sound to the second user.

Preferably, when the processing unit determines that the instruction of the first user is a local task instruction, the processing unit loads the local task instruction into a local session process without triggering the transceiving unit.

Preferably, the local task instruction is not sent to the communication unit of the mobile terminal by the transceiving unit.

Preferably, the processing unit determines whether the call mode needs to be switched, specifically: judging whether the instruction of the first user received by the transceiving unit is a mode switching instruction; or after the transceiver unit receives a second call voice, judging whether the transceiver unit does not receive a call content instruction within a fourth preset time.

Preferably, the determining whether the received instruction of the first user is a mode switching instruction is specifically performed by determining whether the mode switching instruction is generated by a user through a key on the mobile terminal, and/or determining whether the type of the text instruction or the voice instruction is a mode switching instruction when the instruction is a text instruction or a voice instruction.

Preferably, when the processing unit determines that the instruction of the first user is the mode switching instruction, the processing unit does not trigger the transceiver unit, that is, the mode switching instruction is not sent to the second user by the transceiver unit via the communication unit of the mobile terminal.

Preferably, the switching of the call mode of the voice assistant is performed, specifically, the call mode of the voice assistant is switched from the user call mode to the voice assistant call mode or the auxiliary call mode.

Preferably, a third speech sound is generated based on the understanding of the call content and the call progress and sent to the second user, specifically, a third call sound is generated based on the filled slot.

The present invention also provides a method for a voice assistant to participate in a call, referring to fig. 5, where the voice assistant is in a voice assistant call mode, and the method includes the following steps:

step 1, in the process of communication, a voice assistant receives a second communication voice of a calling party, generates a third communication voice and sends the third communication voice to the calling party so as to respond to the second communication voice of the calling party;

step 2, when the voice assistant receives the instruction of the first user and judges that the instruction is a call content instruction, the voice assistant determines the intention of the user to insert the instruction and changes the logic for generating third call voice; the voice assistant receives a next round of second call voice of the calling user and generates a third call voice based on the changed logic;

step 3, the voice assistant judges whether the times of giving the call content instructions by the user meet a first preset rule, if so, the call mode of the voice assistant is switched from the call mode of the voice assistant to an auxiliary call mode;

and 4, in the auxiliary call mode, starting a timer when receiving the next round of second call voice of the calling user, generating alternative call voice, if receiving the instruction of the first user before the timer expires and judging the instruction as a call content instruction, sending the call content instruction as a third call voice to the calling user, and if not, sending the alternative call voice as the third call voice to the calling user.

Preferably, the voice assistant is in a voice assistant call mode, specifically: when it is determined that the incoming call should be answered by the voice assistant according to the incoming call answering rules, the voice assistant answers the incoming call and automatically operates in a voice assistant call mode.

Preferably, in step 2, after receiving the instruction of the first user, the voice assistant determines whether the instruction includes a local task instruction identification word, and if not, determines that the user instruction is a call content instruction.

Preferably, in step 2, after receiving the instruction of the first user, the voice assistant determines whether the instruction includes a local task instruction identification word or a mode switching identification word, and if none of the instructions includes the local task instruction identification word or the mode switching identification word, determines that the user instruction is a call content instruction.

Preferably, the first preset rule is as follows: the number of times that a user gives a call content instruction in a first preset time exceeds a first preset number; or; and the user continuously gives corresponding call content instructions aiming at a second preset number of second call voices.

Preferably, the user gives the call content instructions when a second call voice is received but the voice assistant has not responded to the second call voice, or when the voice assistant has responded to the previous second call voice but has not received the next second call voice.

Preferably, when the voice assistant determines that the number of times that the user gives the call content instruction does not satisfy the first preset rule, the voice assistant does not change the working mode thereof, and continues to execute step 1 and/or step 2.

Preferably, in the auxiliary call mode, when the number of times that the user gives the call content instruction conforms to the second preset rule, the call mode in which the voice assistant is located is automatically switched from the auxiliary call mode to the user call mode.

Preferably, in the auxiliary call mode, when a third preset rule is met, the call mode in which the voice assistant is located is automatically switched from the auxiliary call mode to the voice assistant call mode, where the third preset rule is: the user gives no call content instruction within a third preset time; or the user does not give the call content instruction continuously aiming at the fifth preset number of second call voices.

By the method, the voice assistant can automatically switch between different call modes to participate in the call in different call modes, and the call experience of the user is improved.

The present invention also provides a voice assistant for participating in a call, as described above and shown in fig. 4, where the voice assistant includes a transceiver unit and a processing unit, and the above two units, in addition to the above-described functions, have the following functions when the voice assistant is in a voice assistant call mode:

the transceiver unit is also used for receiving a second call voice of the calling party and sending the third call voice to the calling party in response to the triggering of the processing unit;

the processing unit is further configured to generate a third call voice, and trigger the transceiver unit to send the third call voice to the calling party so as to respond to the second call voice of the calling party;

the receiving and sending unit is also used for receiving an instruction of a first user;

the processing unit is further configured to determine an intention of a user to insert the instruction and change a logic for generating a third call voice when the instruction is determined to be a call content instruction;

the receiving and sending unit is also used for receiving the next round of second call voice of the calling user;

the processing unit is further used for generating a third speech sound based on the changed logic and triggering the transceiving unit to send the third speech sound to the calling user so as to respond to the next round of second call voice; the voice assistant is also used for judging whether the times of giving the call content instructions by the user meet a first preset rule or not, and if so, switching the call mode of the voice assistant from the call mode of the voice assistant to an auxiliary call mode;

when the voice assistant is in the auxiliary call mode, the transceiver unit is further configured to receive a next round of second call voice of the calling user; the processing unit is further used for starting a timer and generating an alternative call voice, if an instruction of the first user is received from the transceiver unit before the timer expires and the instruction is judged to be a call content instruction, the call content instruction is used as a third call voice to trigger the transceiver unit to send the third call voice to the calling user, otherwise, the alternative call voice is used as the third call voice to trigger the transceiver unit to send the third call voice to the calling user.

Preferably, the voice assistant is in a voice assistant call mode, specifically: the processing unit determines that the incoming call should be answered by the voice assistant according to the incoming call answering rule, connects the incoming call and controls the voice assistant to work in a voice assistant conversation mode.

Preferably, whether the received instruction of the first user includes the local task instruction identification word is judged, and if not, the user instruction is determined to be the call content instruction.

Preferably, whether the received instruction of the first user includes a local task instruction identification word or a mode switching identification word is judged, and if not, the user instruction is determined to be a call content instruction.

The first preset rule is as follows: the number of times that a user gives a call content instruction in a first preset time exceeds a first preset number; or; and the user continuously gives corresponding call content instructions aiming at a second preset number of second call voices.

Preferably, the transceiver unit receives the instruction of the first user at a timing when a second communication voice is received but the processing unit has not responded to the second communication voice, or when the processing unit has responded to a previous second communication voice but has not received the next second communication voice.

When the voice assistant judges that the times of giving the call content instructions by the user do not meet the first preset rule, the voice assistant does not change the working mode of the voice assistant, and the transceiving unit and the processing unit continue to execute the functions of the voice assistant in the call mode.

Preferably, in the auxiliary call mode, when the number of times that the user gives the call content instruction conforms to a second preset rule, the processing unit switches the call mode in which the voice assistant is located from the auxiliary call mode to the user call mode.

Preferably, in the auxiliary call mode, when a third preset rule is met, the processing unit switches the call mode in which the voice assistant is located from the auxiliary call mode to the voice assistant call mode, where the third preset rule is: the user gives no call content instruction within a third preset time; or the user does not give the call content instruction continuously aiming at the fifth preset number of second call voices.

The invention also provides a method for the voice assistant to recognize and execute the instruction of the called user, referring to fig. 6, the method comprises the following steps:

step 1, when an incoming call is connected and a call is established with a calling party, the voice assistant establishes a new call session process for the call;

step 2, in the process of the first user and/or the voice assistant communicating with the calling user, the voice assistant receives a user instruction from the first user;

step 3, the voice assistant judges whether the user instruction comprises a local task instruction identification word, if so, the user instruction is determined to be a local task instruction which does not belong to the conversation session process, and the task instruction is loaded into the local session process of the voice assistant; otherwise, determining that the user instruction is a call content instruction, and the voice assistant generates a third call voice based on the call content instruction and sends the third call voice to the calling party, or sends the call content instruction as a third call voice to the calling party.

Preferably, the user instruction is a voice instruction or a text instruction.

Preferably, the loading the task instruction into a local session process of the voice assistant specifically includes: and when a local session process associated with the task instruction exists between the user and the voice assistant, loading the task instruction into the local session process for processing, and when the local session process associated with the task instruction does not exist between the user and the voice assistant, newly establishing a local session process and loading the task instruction.

Preferably, when the voice assistant is in the voice assistant call mode, a third call voice is generated based on the call content instruction.

Preferably, the generating of the third call voice based on the call content instruction specifically includes: performing semantic recognition on the call content instruction, determining the intention of a user to insert the call content instruction, changing the logic for generating third call voice based on the intention, and generating a third voice call based on the changed logic.

Preferably, when the voice assistant is in the auxiliary call mode or the user call mode, the call content instruction is sent to the calling user as a third speech sound. In the process of sending the call content command as the third speech sound to the calling party, reference is made to the above description when the auxiliary call mode or the user call mode is introduced, and details are not repeated herein.

Preferably, the local task instruction identification word is a wake-up word of the voice assistant.

Preferably, the local task instruction identification word is an arbitrary word defined by a user.

Preferably, in step 3, the voice assistant determines whether the user instruction includes a local task instruction identification word, if so, determines that the user instruction is a local task instruction, otherwise, further determines whether the user instruction includes a mode switching identification word, if so, determines that the user instruction is a mode switching instruction, and completes switching of the voice assistant mode based on the mode switching instruction, otherwise, determines that the user instruction is a call content instruction.

Preferably, the method further comprises the step of judging whether the call mode of the voice assistant needs to be automatically switched, and if so, switching the call mode of the voice assistant. The specific automatic switching conditions are as described above. And will not be described in detail herein.

By the method, the user can talk with the voice assistant at any time in the process of the call between the user and the calling user or the call between the voice assistant and the calling user, so that the voice assistant can complete other tasks different from the call task, and the user experience is improved.

The present invention also provides a voice assistant for identifying and executing the instruction of the called user, as described above and shown in fig. 4, the voice assistant comprises a transceiver unit and a processing unit, and the two units have the following functions in addition to the functions described above:

the processing unit is further configured to: when an incoming call is connected and a call is established with a calling party, establishing a call session process for the call;

the transceiver unit is further configured to: receiving a user instruction from a first user during a call between the first user and a calling user and/or a voice assistant; further for transmitting the third spoken voice to a second user in response to a trigger by the processing unit;

the processing unit is further configured to: judging whether the user instruction comprises a local task instruction identification word or not, if so, determining that the user instruction is a local task instruction which does not belong to the conversation session process, and loading the task instruction into the local session process of the voice assistant; otherwise, determining that the user instruction is a call content instruction, generating a third call voice based on the call content instruction, and triggering the transceiver unit to send the third call voice to the calling party, or triggering the transceiver unit to send the third call voice to the calling party by taking the call content instruction as the third call voice.

Preferably, the user instruction is a voice instruction or a text instruction.

Preferably, when the voice assistant is in the voice assistant call mode, the processing unit generates a third call voice based on the call content instruction.

Preferably, when the voice assistant is in the auxiliary call mode or the user call mode, the call content instruction is sent to the calling user as a third speech sound.

Preferably, the processing unit is further configured to determine whether the user instruction includes a local task instruction identification word, if so, determine that the user instruction is a local task instruction, otherwise, further determine whether the user instruction includes a mode switching identification word, if so, determine that the user instruction is a mode switching instruction, complete switching of a voice assistant mode based on the mode switching instruction, and otherwise, determine that the user instruction is a call content instruction.

Preferably, the processing unit is further configured to: and judging whether the call mode of the voice assistant needs to be automatically switched, and if so, switching the call mode of the voice assistant. The specific automatic switching conditions are as described above. And will not be described in detail herein.

The present invention also provides a method for a voice assistant to perform local tasks, referring to FIG. 7, the method comprising the steps of:

Preferably, the voice assistant provides the call content instruction and the second communication voice in step 4 to the second user and the second user respectively.

Preferably, after receiving the call voice instruction and the second call voice, the voice assistant understands the call content and the call progress based on the call content instruction and the second call voice of the second user.

Preferably, step 5 specifically comprises: the voice assistant judges whether the local task instruction executed in the step 3 is related to the call content instruction and/or the second communication voice, if so, judges whether the call content instruction and/or the second communication voice received in the step 4 comprise the updated information of the executed local task instruction, and if so, executes the step 6;

preferably, it is determined whether the type of the re-received voice instruction is a local task instruction, specifically, it is determined whether the voice instruction includes a local task instruction identification word, and if so, it is determined that the user instruction is a local task instruction.

Preferably, the voice instruction of the first user of the next round is generated by the user based on the first execution result.

Preferably, the method further comprises: and judging whether the call mode of the voice assistant needs to be switched, if so, switching the call mode into the voice assistant call mode or the auxiliary call mode by the voice assistant, and generating a third speech sound by the voice assistant based on the understanding of the call content and the call progress and sending the third speech sound to the second user.

Preferably, the method further comprises the step of judging whether the call mode of the voice assistant needs to be switched in the call mode of the voice assistant or the auxiliary call mode, and if so, switching the call mode of the voice assistant. The specific handover procedure is as described above, and is not described herein again.

The above-described method is described below in conjunction with specific scenarios,

assuming that the user a sets other calls except for the phones of the family, the friends and the colleagues to be answered by the voice assistant, when receiving the call of the friend B, the mobile terminal determines that the call should be answered by the user a. When the user is detected to answer the incoming call and establish a call, the voice assistant establishes a call session process for the call. And the user A and the friend B start a call, and the voice assistant continuously receives a call voice command and second communication voice from the user in the call process, performs voice recognition and semantic recognition on the call voice command and the second communication voice, and understands the call content and the call progress. The conversation process is as follows:

and (4) friend B: university classmate C, Beijing;

the user A: when?

And (4) friend B: yesterday;

the user A: is a business trip?

And (4) friend B: kah-kah, about five days in Beijing;

the user A: then we find a time to gather a bar;

and (4) friend B: is you available on friday evening?

The user A: is there a silence, do i have a schedule on friday evening?

The voice assistant judges that the instruction is a local task instruction through 'silence' in the user instruction, so that the user schedule is searched, and a task execution result of 'schedule on friday night' is provided for the user A.

The user A: not so, i am not empty at five nights every week;

and (4) friend B: that saturday noon?

At this time, the voice assistant judges that the second communication voice of the friend B includes the update information of the first local task instruction, at this time, the voice assistant replaces the "friday evening" in the first local task instruction according to the update information to generate a second local task instruction "mute", does i have a schedule at friday evening? ", the voice assistant executes the second local task instruction, and notifies the user of a second execution result, which in this scenario is" no schedule on saturday noon ".

The user a replies "no question in saturday noon" to the friend B based on the second execution result.

The present invention also provides a voice assistant for performing local tasks, the voice assistant being in a user talk mode, as described above and shown in fig. 4, the voice assistant comprising a transceiver unit and a processing unit, both of which, in addition to the functions described above, have the following functions:

Preferably, the transceiver unit sends the received voice instruction of the first user of the next round to the second user, and the transceiver unit provides the received second communication voice of the second user of the next round to the first user.

Preferably, the processing unit further understands the call content and the call progress based on the call content instruction and the second call voice.

Preferably, the processing unit further determines whether the executed local task instruction is related to the call content instruction and/or the second communication voice, and if so, the processing unit determines whether the call content instruction and/or the second communication voice of the second user of the next round include the updated information of the executed local task instruction.

Preferably, the local task instruction identification word is an arbitrary word customized by a user, such as "mute".

The processing unit is further used for judging whether the call mode of the voice assistant needs to be switched, if so, the call mode of the voice assistant is switched to the call mode of the voice assistant or an auxiliary call mode, and a third speech sound is generated and sent to the second user based on the understanding of the call content and the call progress.

The processing unit is further configured to determine whether the call mode of the voice assistant needs to be switched in the voice assistant call mode or the auxiliary call mode, and if so, switch the call mode of the voice assistant.

All methods/apparatus of the present invention relate to the operational modes of the voice assistant, which may be combined with each other in any desired manner.

Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. The computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), a flash memory, an erasable programmable read-only memory (EPROM), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Computer program code for carrying out operations of the present invention may be written in one or more programming languages, or a combination thereof.

The above description is only an example for the convenience of understanding the present invention, and is not intended to limit the scope of the present invention. In the specific implementation, a person skilled in the art may change, add, or reduce the components of the apparatus according to the actual situation, and may change, add, reduce, or change the order of the steps of the method according to the actual situation without affecting the functions implemented by the method.

While embodiments of the invention have been shown and described, it will be understood by those skilled in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents, and all changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

1. A method for a voice assistant to perform a local task, the method comprising:

step 3, the voice assistant executes the local task instruction and informs a first user of a first execution result;

step 4, the voice assistant continuously receives the voice instruction of the first user of the next round, the voice instruction of the first user of the next round is generated by the first user based on the first execution result, the voice assistant determines that the voice instruction is a call content instruction, and sends the call content instruction to the second user as a third speech sound; the voice assistant receives a second call voice of a second user of a next round and provides the second call voice to the first user;

and 6, replacing all or part of information in the executed local task instruction according to the updated information to generate a second local task instruction, executing the second local task instruction, and notifying a second execution result to the first user.

2. The method of claim 1, wherein after receiving the call content instruction and the second call voice, the voice assistant understands call content and call progress based on the call content instruction and the second call voice of the second user.

3. The method according to claim 1, wherein step 5 is specifically: the voice assistant judges whether the local task instruction executed in the step 3 is related to the call content instruction and/or the second communication voice, if so, judges whether the call content instruction and/or the second communication voice received in the step 4 comprise the updated information of the executed local task instruction, and if so, executes the step 6.

4. The method according to claim 1, wherein it is determined whether the type of the re-received voice command is a local task command, specifically, whether the voice command includes a local task command identification word, and if so, it is determined that the voice command is a local task command.

5. A voice assistant for performing local tasks, wherein the voice assistant is in a user talk mode, the voice assistant comprising a transceiver unit, a processing unit;

the receiving and sending unit is also used for receiving a voice instruction of the first user, receiving second call voice of the second user and providing the second call voice to the first user;

the transceiver unit is also used for responding to the trigger of the processing unit and sending the third communication voice to a second user;

the processing unit is further configured to determine whether the re-received voice instruction is a local task instruction, and if so, execute the local task instruction, and trigger the transceiver unit to notify the first user of a first execution result;

the receiving and sending unit is further used for responding to the trigger of the processing unit and informing a first user of a first execution result;

the receiving and sending unit is further used for receiving a voice instruction of a first user in a next round, wherein the voice instruction of the first user in the next round is generated by the first user based on the first execution result, receiving a second call voice of a second user in the next round and providing the second call voice to the first user;

the processing unit is further configured to determine that the voice instruction of the first user in the next round is a call content instruction, and trigger the transceiver unit to send the call content instruction to the second user as a third speech sound; judging whether the call content instruction and/or the second call voice of the second user in the next round comprise the update information of the executed local task instruction, if so, replacing all or part of information in the executed local task instruction according to the update information to generate a second local task instruction, executing the second local task instruction, and triggering the transceiver unit to notify a second execution result to the first user;

the transceiver unit is further configured to notify the first user of the second execution result in response to the trigger of the processing unit.

6. The voice assistant of claim 5, wherein the processing unit further understands call content and call progress based on the call content instructions and the second communication voice.

7. The voice assistant of claim 5, wherein the processing unit further determines whether the executed local task instruction is associated with the call content instruction and/or a second communication voice, and if so, the processing unit determines whether the call content instruction and/or the second communication voice of the second user of the next round includes updated information of the executed local task instruction.

8. The voice assistant of claim 5, wherein the type of the re-received voice command is determined whether the voice command is a local task command, and specifically, whether the voice command includes a local task command identification word, and if so, the voice command is determined to be a local task command.

9. A computer arrangement, characterized in that the computer arrangement comprises a processor and a memory, in which a computer program is stored which is executable on the processor, which computer program, when being executed by the processor, carries out the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, in which a computer program that is executable on a processor is stored, which computer program, when being executed, carries out the method according to any one of claims 1 to 4.