CN112261233B

CN112261233B - Method for calling by voice assistant and voice assistant

Info

Publication number: CN112261233B
Application number: CN202011147783.8A
Authority: CN
Inventors: 张伟萌; 李俊; 吴克石
Original assignee: Beijing Moran Cognitive Technology Co Ltd
Current assignee: Beijing Moran Cognitive Technology Co Ltd
Priority date: 2020-10-23
Filing date: 2020-10-23
Publication date: 2021-09-17
Anticipated expiration: 2040-10-23
Also published as: CN112261233A

Abstract

The invention discloses a method for calling by a voice assistant and the voice assistant. The method comprises the following steps: step 201, the voice assistant determines that a calling user is required to control a first call; step 202, the voice assistant determines whether the calling user is controlling the second call; if yes, go to step 203; step 203, the voice assistant outputs a first prompt message to the calling user, receives a call content instruction given by the calling user in response to the first prompt message while controlling the second call, and the voice assistant continues the first call with the first called user according to the call content instruction. By the method and the voice assistant, the calling user can control the communication of a plurality of terminals or the voice assistant simultaneously, and the user experience is improved.

Description

Method for calling by voice assistant and voice assistant

Technical Field

The embodiment of the invention relates to the technical field of information processing, in particular to a method for calling by a voice assistant and the voice assistant.

Background

In recent years, with the development of artificial intelligence, voice assistants can also provide more and more abundant services for users, for example, the voice assistant can replace a calling user to automatically make a call and perform call interaction with a called user, and in the call interaction process, the calling user can actively or by triggering of a terminal to participate in the call interaction process to control the call. However, when the calling party has multiple terminals and the multiple terminals are in call interaction at the same time, there is a case that calls of the multiple terminals all need to be controlled by the calling party, and for this case, how to control which terminal the calling party controls calls of which terminal and how to cooperate among the multiple terminals, how to distinguish which terminal the voice command given by the calling party is directed to when the calling party controls calls of the multiple terminals at the same time, how to improve the efficiency of controlling the calls by the user, and how to reduce the number of times that the voice assistant triggers the user to control the calls, and the prior art does not provide a solution.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a method for calling by a voice assistant and the voice assistant.

The invention provides a method for calling by a voice assistant, which comprises the following steps:

step 201, the voice assistant determines that a calling user is required to control a first call, wherein the first call is a call between the voice assistant and a first called user, the call being initiated by the calling user and replaced by the voice assistant;

step 202, the voice assistant judges whether the calling user is controlling a second call, wherein the second call is a call between the second voice assistant and a second called user, which is initiated by the calling user; if yes, go to step S203;

step 203, the voice assistant outputs a first prompt message to the calling user, receives a call content instruction given by the calling user in response to the first prompt message while controlling the second call, and the voice assistant continues the first call with the first called user according to the call content instruction.

The invention provides a voice assistant, which comprises a transceiving unit and a processing unit;

the processing unit is configured to determine that a calling user is required to control a first call, where the first call is a call initiated by the voice assistant and made to a first called user; judging whether the calling user is controlling a second call, wherein the second call is a call between the calling user and a second called user initiated by a second voice assistant; if yes, triggering the transceiving unit to output a first prompt message to a calling user, and receiving a call content instruction given by the calling user in response to the first prompt message while controlling a second call; the processing unit is further configured to continue the first call with the first called user according to the call content instruction.

And the receiving and sending unit is used for responding to the trigger of the processing unit, outputting a first prompt message to a calling user, and receiving a call content instruction given by the calling user in response to the first prompt message while controlling a second call.

The invention also provides a computer device characterized in that it comprises a processor and a memory, in which a computer program is stored that is executable on the processor, which computer program, when executed by the processor, implements the method as described above.

The invention also provides a computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program is executable on a processor, and when executed implements the method as described above.

The invention also provides a terminal which is characterized by comprising the voice assistant.

By the method and the voice assistant, the invention provides a plurality of calling user call control modes for the user, and provides a reasonable calling call control strategy, so that the calling user can control the calls of a plurality of terminals or the voice assistant simultaneously, and the user experience of the voice assistant in calling is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a voice assistant system in one embodiment of the invention.

Fig. 2 is a method of controlling a call in one embodiment of the invention.

FIG. 3 is a voice assistant in one embodiment of the invention.

FIG. 4 is a method of voice assistant calling in one embodiment of the invention.

FIG. 5 is a method for a voice assistant to process a caller voice instruction in one embodiment of the present invention.

Fig. 6 is a scheduling method for a calling subscriber in one embodiment of the invention.

Fig. 7 is a scheduling apparatus for a calling subscriber in one embodiment of the present invention.

Fig. 8 is a method of controlling call synchronization in one embodiment of the invention.

Fig. 9 is a method of clustering and controlling calls in one embodiment of the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments and specific features of the embodiments of the present invention are detailed descriptions of technical solutions of the embodiments of the present invention, and are not limited to technical solutions of the present invention, and the technical features of the embodiments and the embodiments of the present invention may be combined with each other without conflict.

1. Voice assistant system

FIG. 1 shows a block diagram of a voice assistant system. In the present invention, the voice assistant system may also be referred to as a voice assistant. The voice assistant system may be implemented on a stand-alone device or across multiple devices. In some embodiments, some of the modules, units or functions of the voice assistant system belong to a server, and the rest of the modules, units or functions belong to a terminal, and the terminal can communicate with the server through one or more networks. In some embodiments, some of the modules of the voice assistant system, such as the processing unit 102 and the interaction engine 112, may belong to both the server and the terminal, and the remaining modules belong to one of the server or the terminal.

The voice assistant system mainly comprises: a transceiver unit 101, a processing unit 102, a database 103, etc. The processing unit includes n interaction engines 112, where n is a positive integer greater than or equal to 1, and each interaction engine 112 may include a semantic understanding module 201, a dialog management and control module 202, a dialog generation module 203, and a command execution module 204. The processing unit 102 is connected to the transceiver 101, and can receive data input by a user through the transceiver 101, and output interactive data, such as session data, task execution process and result, to the user through the transceiver 101. When the voice assistant takes a call instead of the user, such as receiving a call or initiating a call, the transceiving unit is further configured to receive a call voice of the opposite user from a component, such as a communication unit of the mobile terminal, receive a message, such as a priority control response, a user status message, from a scheduler from a component, such as a communication unit of the mobile terminal, receive an instruction of the voice assistant user from a user instruction collecting unit, such as an input method or a microphone, and send the call voice generated by the voice assistant or received from the voice assistant user to the opposite user through a component, such as a communication unit of the mobile terminal, and send a message generated by the voice assistant, such as a first update message, a second update message, a priority control request, mentioned below, to the scheduler through a component, such as a communication unit of the mobile terminal. That is, the transceiver unit of the present invention is used for the voice assistant to receive and/or send messages, and the rest functions of the voice assistant are completed by the processing unit. The transceiving unit 101 may comprise a human-machine interface or the like.

In some embodiments, the interaction engine 112 may include an interaction main engine and/or at least one interaction sub-engine. Wherein the interaction main engine is a default engine of the voice assistant system. The interaction sub-engine on the server can be owned by the voice assistant system, can be generated by local training of the terminal and uploaded through a proprietary interface, and can be owned by the voice assistant system, can be generated by local training of the terminal and loaded in the voice assistant system, and can be downloaded by the terminal from the server and loaded in the voice assistant system.

In some embodiments, each interaction engine (interaction main engine, interaction sub engine) is capable of performing at least one task, i.e. each interaction engine may be associated with at least one task, and the tasks capable of being performed by different interaction sub engines may be the same or different. The interaction engine defines at least one slot for each task associated therewith. The terminal may download one or more interaction sub-engines from the server and load them in the terminal's voice assistant system, a typical example of which is as follows:

the task executable by the weather interaction sub-engine is weather query, the slot defined by the weather query task is a city and a date, and a user can download the weather interaction sub-engine and interact with the weather interaction sub-engine to query the weather forecast of the city;

the task executable by the music interaction sub-engine is music playing, and the slot defined by the music playing task comprises a singer name and a song name, so that a user can download the music interaction sub-engine and interact with the music interaction sub-engine to play music according to the requirements of the user;

the tasks executable by the incoming call answering sub-engine are incoming call answering tasks, including answering incoming calls in various different call modes and providing records of call contents for users. The slot defined for the task includes caller identity, social relationship, and the like. The user can download the incoming call answering sub-engine and answer the incoming call by using the incoming call answering sub-engine. When an incoming call is detected, the voice assistant or an interactive main engine of the voice assistant automatically triggers or calls an incoming call answering sub-engine, the incoming call answering sub-engine or the interactive main engine determines that the incoming call should be answered by a user or answered by the voice assistant according to an incoming call answering rule, and the incoming call answering sub-engine establishes a call session process for the call so as to be distinguished from a local session process of the user and the voice assistant as long as the incoming call is connected and the call is established with a user of the calling voice assistant no matter the user answers the incoming call or the voice assistant answers the incoming call.

The task executable by the calling sub-engine is a calling task, and specifically, the calling sub-engine replaces a calling user to actively call a called user and wholly or partially replaces the calling user to communicate with the called user. The slot defined for the task includes the called subscriber number, the called subscriber category, the primary call intention, and the like. The user may download the caller sub-engine and use the caller sub-engine to make an automatic caller call, and the specific call mode of the caller sub-engine is described in detail below. When the called user answers the call, the main calling sub-engine establishes a call with the called user, and establishes a call session process for the call, which is distinguished from the local conversation process of the user and the voice assistant.

The incoming call answering task and the calling task both belong to a conversation task. In the invention, other tasks which can be executed by the voice assistant and are except for the conversation task are called as local tasks, a user can trigger the local tasks through local task instructions, the local task instructions do not belong to the conversation session process, and the local task instructions have the function of enabling the voice assistant to execute specific other tasks, namely the local tasks, but the local task instructions are not unrelated to the conversation contents of a calling user and a called user, and only the local task instructions are not required to be loaded to the conversation session process so as not to be sent to a second user, and the execution result of the local tasks is not required to be loaded to the conversation session process. For example, the incoming call answering sub-engine of the voice assistant is in a user call mode, and the calling user inquires whether the called user is available in the open air, the called user wants to determine whether the called user is available in the open air by inquiring the voice assistant, and then sends out a voice instruction "mute, i.e., whether i are available in the open air", which is a local task instruction, but is related to the call content, and the local task instruction triggers the main engine or the schedule sub-engine to execute a local task, i.e., a schedule query task, so as to determine the schedule of the user in the open air, and reply the query result to the user, e.g., "available in the afternoon in the open air". It should be noted that the above local task instruction and the meaning of "local" in the local task are used only to illustrate that the task is not a conversational task, and not that the voice assistant does not need to access the network and utilize information in the network when performing the task.

Aiming at the local task, a user can interact with the voice assistant system in a voice or text mode, the voice assistant system determines the user intention (namely determines the task) according to the instruction of the user, determines key knowledge data corresponding to each slot position associated with the user intention, and fills the key knowledge data into the corresponding slot position. The voice assistant system then performs the task based on the populated slot or slots.

In some embodiments, the interaction main engine determines, based on a user instruction only, an interaction sub-engine capable of processing the user instruction, which itself does not perform a specific task. Therefore, the determination of the user intention during the local task may be performed by an interaction main engine in the voice assistant system, the interaction main engine being further configured to select one or more interaction sub-engines to process the user instruction based on the determined user intention, the slot filling and the local task execution being performed by the one or more interaction sub-engines determined by the interaction main engine.

In some embodiments, the functions of the incoming call answering sub-engine and/or the calling sub-engine are performed by the interactive main engine, i.e. a separate incoming call answering sub-engine and/or calling sub-engine is not set up.

The present invention is primarily directed to improving the calling function of a voice assistant, and therefore, in the present invention, a voice assistant may also be referred to as a call assistant, but this should not be construed as limiting the present invention, and in particular does not mean that the voice assistant of the present invention is incapable of performing other functions such as a call listening function.

2. Calling mode

Hereinafter, the calling user, the user of the voice assistant and the first user are the same, the calling user, the user of the voice assistant and the first user can be replaced mutually in any scene, the called user and the second user are the same, and the calling user, the user of the voice assistant and the second user can be replaced mutually in any scene. Hereinafter, unless explicitly indicated, all users referred to refer to the first user.

In the invention, the voice assistant replaces a calling user to actively call a called user and can completely or partially replace the calling user to communicate with the called user. The calling user may actively control the call at will, or may control the call in response to a trigger of the voice assistant, for example, the voice assistant determines that the calling user needs to control the current call according to the call content (e.g., intervene in the current call). In the present invention, the latter is mainly improved.

In the present invention, the voice of the called user in the call is called as a second call voice, and for the second call voice, the voice assistant sends a third call voice to the called user as a response to the second call voice, wherein, in different calling call modes, the process of the voice assistant generating or acquiring the third call voice is different, which will be described in detail below.

According to whether the user controls the call and the difference of the control modes, the voice assistant can provide three calling call modes, namely a voice assistant call mode, an auxiliary call mode and a user call mode. It should be noted that, in the present application, the calling call mode of the voice assistant refers to the working mode thereof, which may be one of the three calling call modes, and the voice assistant call mode refers to a call mode in which the voice assistant replaces the calling party to make a call with the called party.

(1) Voice assistant talk mode

The voice assistant may automatically place the call to the called party instead of the user (e.g., according to call rules preset by the user), and when the call is connected, the voice assistant defaults to operating in the voice assistant talk mode. In the voice assistant communication mode, the voice assistant replaces the calling user to communicate with the called user. Specifically, after the called user answers the incoming call, the voice assistant establishes a call with the called user, and the voice assistant establishes a call session process for the call. Thereafter, the voice assistant autonomously generates a third speech sound to be transmitted to the called user and receives a second speech sound from the called user. Meanwhile, the voice assistant provides the call content comprising the third call voice and the second call voice to the user in real time for the user to view or listen. The call content may be voice or text.

(2) Auxiliary talk mode

In the auxiliary call mode, a calling user performs auxiliary control of a first preset round of conversation on a call in a first control mode, specifically, the voice assistant receives a call content instruction from the calling user, generates a third speech sound based on the call content instruction, and sends the third speech sound to a called user. In the auxiliary-call mode, the voice assistant also receives a second call voice of the called user and provides it to the calling user. The voice assistant repeats the above process as the call progresses. The call content instruction may be voice or text.

After the preset wheel is finished, the auxiliary control of the calling user on the voice assistant is finished, and the voice assistant automatically switches the calling call mode into the voice assistant call mode.

Generating a third speech sound based on the call content instruction and sending the third speech sound to a called user, wherein the third speech sound is specifically as follows: the voice assistant determines the intention of the calling user based on the call content instruction, updates the state of a decision diagram for generating third call voice based on the intention, and generates the third call voice according to the changed state of the decision diagram.

Generating a third speech sound based on the call content instruction and sending the third speech sound to a called user, wherein the third speech sound is specifically as follows: and sending the call content instruction as a third voice to the called user.

And sending the call content instruction as a third speech sound to a called user, specifically: when the user gives a call content instruction in a text form, the voice assistant generates a third speech sound in a voice synthesis mode according to the call content instruction, or the voice assistant generates third call content with the same or similar semantics based on the call content instruction and generates third call voice in a voice synthesis mode according to the third call content. For example, when the call content instruction is "reserve a specific time", the third call content is "what time you watch and call you conveniently? ".

And sending the call content instruction as a third speech sound to a called user, specifically: when the user gives the call content instruction in the form of voice, the voice assistant takes the call content instruction as a third call voice.

And sending the call content instruction as a third speech sound to a called user, specifically: when the user gives the call content instruction in a voice form, the voice assistant generates third call voice in a voice synthesis mode, specifically, the voice assistant performs voice recognition on the call content instruction and generates third speech sound in the voice synthesis mode according to a voice recognition result, and by the mode, the called user can be shielded to know the environmental sound in the environment where the calling user is located, so that the privacy of the calling user is protected, and meanwhile the definition of the third call voice can be ensured.

And sending the call content instruction as a third speech sound to a called user, specifically: when a user gives a call content instruction in a voice form, the voice assistant transfers the call content instruction to generate a third call voice in a voice synthesis mode, specifically, performs voice recognition on the call content instruction, generates third call content with the same or similar semantics based on a voice recognition result, and generates the third call voice in the voice synthesis mode according to the third call content. Through this mode, can shield calling party and know the environmental sound in the called party's environment, thereby protect calling party's privacy, can also ensure the definition of third conversation pronunciation, in addition, make the user participate in the conversation with more brief pronunciation, feel time nervous and too chaotic when having avoided calling party to handle the calling party's of the voice assistant of a plurality of terminals simultaneously, can not let the called party feel offence or not polite simultaneously.

When the call content instruction is sent to the called user as a third call voice, the voice assistant understands the call content and the call progress of the first user and the second user based on the call content instruction of the calling user and the second call voice of the called user, so that when the voice assistant switches to other call modes such as a voice assistant call mode, the third call voice can be generated based on the comprehension of the call content and the call progress.

In the invention, the calling user can simultaneously control a plurality of calls of a plurality of voice assistants only in the first control mode, or can simultaneously control a plurality of calls of a plurality of voice assistants in the first control mode, and simultaneously control one call except the plurality of calls in the second control mode.

(3) User talk mode

In the user call mode, the calling user fully controls the call in the second control mode. When the calling user controls in the second control mode or when the voice assistant is in the user call mode, the voice assistant receives a call content instruction from the calling user, generates a third call voice based on the call content instruction, sends the third call voice to the called user, receives the second call voice of the called user, and provides the second call voice to the calling user. As the call progresses, the voice assistant repeatedly performs the above-described process until the call is ended or a call control end instruction is received from the calling subscriber/a control mode switching instruction is received from the scheduler. The call content instruction may be voice or text, and preferably, is voice.

And in the user call mode, generating a third speech sound based on the call content instruction and sending the third speech sound to the called user, specifically, sending the call content instruction serving as the third speech sound to the called user. The specific implementation manner of sending the call content instruction as the third voice to the called user is the same as the auxiliary call mode, and is not described herein again.

The user call mode may continue until the call is finished, or the calling user sends a call control end instruction when determining that the call control by the calling user is finished according to the call content, and the voice assistant responds to the instruction to switch the call mode from the user call mode to the voice assistant call mode, or the voice assistant receives a control mode switching instruction from the scheduler and switches the call mode from the user call mode to the auxiliary call mode.

In the process of the call between the calling user and the called user, the voice assistant understands the call content and the call progress of the first user and the second user based on the call content instruction of the calling user and the second call voice of the called user, so that when the voice assistant is switched to other call modes such as a voice assistant call mode, a third call voice can be generated based on the understanding of the call content and the call progress.

In the invention, the calling user can only control one call in the second control mode within a period of time, but can control other calls in the first control mode while controlling one call in the second control mode, namely the calling user cannot control more than two calls in the second control mode.

3. Voice assistant calling communication process

In the invention, a calling user A uses or has at least P terminals, including a terminal 1, a terminal 2, … and a terminal P, each terminal includes a voice assistant, which is respectively a voice assistant 1, a voice assistant 2, … and a voice assistant P, and each voice assistant has a corresponding identifier; wherein, P is greater than or equal to 2, M of the P terminals are all in the calling call process, that is, M calls exist simultaneously, the M calls can be initiated by the M terminals or the M voice assistants at the same time or at different times, and the called users corresponding to the M terminals are called users 1, called users 2, … and called users M, wherein, M is less than or equal to P. Hereinafter, the first terminal (first voice assistant) may be any one of the M terminals (voice assistants), and the second terminal (second voice assistant) may be any one of the M terminals different from the first terminal (first voice assistant).

In one embodiment, the scheduler records and maintains the status information of the calling subscriber, for example, the scheduler records whether the current calling subscriber is in an idle state or is controlling one or several calls; preferably, the scheduler also records which of the first control mode or the second control mode the calling party is currently controlling the call.

Preferably, the scheduler further maintains a waiting control queue for the calling subscriber, in which the voice assistant identifier currently waiting for the calling subscriber to control the call between the calling subscriber and the corresponding called subscriber is stored. Preferably, the waiting control queue may further store the requested calling party control mode corresponding to each voice assistant identifier in the queue. Table 1 shows one possible queue for wait control.

Table 1 wait for control queue

Serial number	Voice assistant identificationSymbol	Requested control mode
			1	Voice assistant 2	First control mode
2	Voice assistant 5	Second control mode
			3	Voice assistant 3	Second control mode

The scheduler may be a device separate from the P voice assistants or terminals or may be located in one of the P voice assistants or terminals, such as a master voice assistant or master terminal.

The invention provides a method for controlling a call, which comprises the following steps:

step 101, the voice assistant determines that a calling user is required to control a first call, wherein the first call is a call between the voice assistant and a first called user, and the call is initiated by the voice assistant instead of the calling user;

step 102, the voice assistant judges whether the calling user is controlling a second call, wherein the second call is a call between the second voice assistant and a second called user, which is initiated by the calling user; if yes, go to step S103;

step 103, the voice assistant acquires a first time length, wherein the first time length is the time length required by the calling user to control the first call from the current time;

and 104, adjusting a conversation strategy according to the first duration, so that the conversation between the voice assistant and a first called user lasts for the first duration, and the calling user controls the first conversation after the first duration.

The voice assistant is a voice assistant of a first terminal, the second voice assistant is a voice assistant of a second terminal, and the first terminal and the second terminal both belong to the calling party.

Preferably, the second call is any one of the calls (such as the aforementioned M calls) that are ongoing by the voice assistants of the calling user, except for the first call. The called users corresponding to the calls which are carried out by the voice assistants are different.

Preferably, before the voice assistant determines that the calling user needs to control the first call in step 101, the voice assistant is in a voice assistant call mode, that is, the voice assistant autonomously generates a third speech sound to be sent to the called user.

Preferably, in step 101, the calling user is required to control the first call, specifically, the calling user is required to give a call content instruction, and the voice assistant generates a third speech sound based on the call content instruction and sends the third speech sound to the first called user, so as to perform call interaction with the first called user.

Preferably, in step 101, the voice assistant determines that the calling user is required to control the first call based on the content of the call between the voice assistant and the first called user.

Preferably, in step 101, the voice assistant determines that the calling user is required to control the first call in the second control mode.

Preferably, in the step 102, if the determination result is no, that is, if the calling party does not control the second call, step 105 is executed, and the voice assistant outputs a control request message to the calling party and acquires a control response message from the calling party. After the control response message is acquired, the voice assistant switches the call mode of the voice assistant from the call mode of the voice assistant to the call mode of the user, so that the calling user controls the first call in a second control mode.

Preferably, in step 102, if the calling user is controlling the second call, it is further determined whether the calling user is controlling the second call in the second control mode, and if yes, step 103 is executed.

Preferably, in step 102, if the calling party is not controlling the second call in the second control mode, step 106 is executed, the voice assistant outputs a control request message to the calling party and acquires a control response message of the calling party, and after the voice assistant acquires the control response message of the calling party, the voice assistant switches the call mode from the voice assistant call mode to the user call mode, so that the calling party controls the first call in the second control mode, and simultaneously controls the second call in another control mode other than the second control mode, for example, the first control mode.

Preferably, in step 105 and/or step 106, after the voice assistant switches its call mode from the voice assistant call mode to the user call mode, the voice assistant also sends first update information to the scheduler, so that the scheduler updates the user status information recorded and maintained by the scheduler. The first updating information carries the identifier of the voice assistant and the control mode of the calling user for the first call.

Preferably, the voice assistant determines whether the calling user is controlling the second call and whether the calling user is controlling the second call in the second control mode in step 102, and specifically, the voice assistant receives a user status message from the scheduler and determines whether the calling user is controlling the second call and whether the calling user is controlling the second call in the second control mode based on the user status message.

Preferably, after step 103, the method further includes step 103a, where the voice assistant determines whether the first duration is less than or equal to a first waiting threshold, and if so, performs step 104.

Preferably, if the first duration is greater than the first waiting threshold, step 108 is executed to send a preset third call voice to the first called user, and end the first call. For example, the preset third spoken voice may be "sorry, current system busy, we will contact you again ten minutes later, see again".

Preferably, in step 103, the voice assistant acquires the first duration, specifically, the voice assistant acquires the first duration from the scheduler.

Preferably, the scheduler determines the first duration based on a call progress between the calling user and the second called user.

Preferably, the scheduler determines a second time length required for the calling user to finish controlling the second call from the current time to the calling user, determines a third time length required for the calling user to control the call corresponding to the voice assistant requesting the second control mode in the waiting control queue, and takes the sum of the second time length and the third time length as the first time length.

Preferably, after the first period of time, the voice assistant outputs a control request message to the calling subscriber and acquires a control response message from the calling subscriber in step 104. After the control response message is acquired, the voice assistant switches the call mode of the voice assistant from the call mode of the voice assistant to the call mode of the user, so that the calling user controls the first call in a second control mode.

Preferably, after step 103, the method includes step 103b, where the voice assistant determines whether the first time length is less than or equal to a second waiting threshold, determines whether the length of the waiting control queue is less than or equal to a queue threshold, and if both of the two determinations are yes, then step 104 is executed. And if at least one of the two judgment results is negative, sending a preset third call voice to the first called user, and ending the first call. The second wait threshold is greater than the first wait threshold.

Preferably, in step 104, the voice assistant sends second update information to the scheduler, so that the scheduler maintains a wait control queue according to the second update information. For example, the second update information carries the identifier of the voice assistant and the requested control mode of the calling party, and the scheduler adds the identifier of the voice assistant to the waiting control queue and correspondingly stores the requested control mode of the calling party.

Preferably, after step 102 and before step 103, step 102a is further included, where the voice assistant acquires the user rating of the second called user (for example, from a scheduler), and determines whether the user rating of the second called user is lower than the user rating of the first called user, if so, step 107 is executed, and otherwise, step 103 is executed.

The step 107 is: the voice assistant sends a priority control request to the scheduler, outputs a control request message to the calling party after receiving a priority control response of the scheduler, acquires the control response message of the calling party, switches the call mode of the voice assistant from the voice assistant call mode to the user call mode after the voice assistant acquires the control response message of the calling party, and controls the first call in the second control mode and controls the second call in the first control mode. Specifically, the voice assistant sends a priority control request to a scheduler, so that the scheduler sends a control mode switching instruction to the second voice assistant after receiving the priority control request, the second voice assistant switches its call mode from a user call mode to an auxiliary call mode after receiving the control mode switching instruction, outputs a first prompt message to the calling user, and sends a control mode switching response to the scheduler, the scheduler sends a priority control response to the voice assistant after receiving the control mode switching response, the voice assistant outputs a control request message to the calling user and acquires a control response message of the calling user after receiving the priority control response, and the voice assistant switches its call mode from the voice call mode to the user call mode after acquiring the control response message of the calling user, and controlling the first call by the calling user in a second control mode, and simultaneously controlling the first call in the second control mode.

Preferably, after step 104 or step 105 or step 106 or step 107, the method further comprises step 108, the voice assistant switches its call mode from the user call mode to the voice assistant call mode.

Preferably, in step 108, the voice assistant switches its call mode according to the call control end instruction obtained from the calling user, and the voice assistant sends first update information to the scheduler, so that the scheduler updates its recorded and maintained user status information. The first updated information indicates that the control of the first call by the calling user has ended.

Preferably, in step 107, the scheduler further stores the voice assistant and the second voice assistant in an associated manner, in step 108, after receiving the first update information sent by the voice assistant, if it is determined that the control of the calling user over the first control mode of the second call of the second voice assistant stored in an associated manner with the voice assistant has not been completed, the scheduler sends a control mode switching instruction to the second voice assistant, after receiving the control mode switching instruction, the second voice assistant outputs a control request message to the calling user and obtains a control response message of the calling user, and after obtaining the control response message of the calling user, the second voice assistant switches its call mode from the auxiliary call mode to the user call mode, so that the calling user controls the second call in the second control mode, the second voice assistant also sends a control mode switching response to the scheduler, and the scheduler updates the recorded and maintained user state information after receiving the control mode switching response.

Preferably, when the user controls the first call in the second control mode, the voice assistant receives a control mode switching instruction sent by the scheduler, and the following operation is similar to the operation of the second voice assistant described in the above steps 107 and 108, and is not described herein again. In the two steps, the operations of receiving and sending information are completed by the receiving and sending unit of the voice assistant, and the rest operations are completed by the processing unit.

Preferably, in step 107, the scheduler updates the recorded and maintained user status information after sending the priority control response.

The method of the invention provides a plurality of calling user call control modes and gives a reasonable calling call control strategy, so that the calling user can flexibly control the calls of a plurality of terminals or voice assistants, and the user experience of the voice assistant active call is improved.

The present invention also provides a voice assistant, referring to fig. 3, comprising a transceiver unit and a processing unit;

the processing unit is used for determining that a calling party is required to control a first call, wherein the first call is a call between the voice assistant and a first called party, and the call is initiated by the voice assistant instead of the calling party; judging whether the calling user is controlling a second call, wherein the second call is a call between a second voice assistant and a second called user, which is initiated by the calling user; if so, triggering the transceiver unit to obtain a first time length, wherein the first time length is the time length required by the calling user to control the first call from the current moment, and adjusting a conversation strategy according to the first time length to enable the conversation between the voice assistant and a first called user to be continued for the first time length, so that the calling user controls the first call after the first time length.

And the transceiver unit is used for responding to the trigger of the processing unit to acquire a first time length.

Preferably, before it is determined that the calling user is required to control the first call, the voice assistant is in the voice assistant call mode, that is, the processing unit of the voice assistant autonomously generates a third call voice sent to the called user by the transceiver unit.

Preferably, the first call needs to be controlled by the calling user, specifically, the calling user needs to give a call content instruction, the processing unit generates a third call voice based on the call content instruction, and the transceiver unit sends the third call voice to the called user.

Preferably, the processing unit determines that the calling user is required to control the first call based on the call content between the processing unit and the first called user.

Preferably, the processing unit determines that the calling user is required to control the first call in the second control mode.

Preferably, if the determination result is negative, that is, if the calling user does not control the second call, the processing unit triggers the transceiver unit, and the transceiver unit responds to the triggering of the processing unit, outputs a control request message to the calling user, acquires a control response message from the calling user, and triggers the processing unit after acquiring the control response message. The processing unit responds to the trigger of the receiving and sending unit, and switches the call mode from the voice assistant call mode to the user call mode, so that the calling user controls the first call in a second control mode.

Preferably, if the calling subscriber is controlling the second call, the processing unit further determines whether the calling subscriber is controlling the second call in the second control mode, and if so, triggers the transceiver unit to obtain the first duration.

Preferably, if the calling user is not controlling the second call in the second control mode, the processing unit triggers the transceiver unit, the transceiver unit responds to the triggering of the processing unit, outputs a control request message to the calling user, acquires a control response message of the calling user, and triggers the processing unit after acquiring the control response message. The processing unit responds to the trigger of the receiving and sending unit, and switches the call mode from the voice assistant call mode to the user call mode, so that the calling user controls the first call in a second control mode, and simultaneously controls the second call in other control modes except the second control mode, such as the first control mode.

Preferably, after the processing unit switches the call mode from the voice assistant call mode to the user call mode, the processing unit further triggers the transceiver unit, and the transceiver unit sends first update information to the scheduler in response to the triggering of the processing unit, so that the scheduler updates the user state information recorded and maintained by the scheduler. The first updating information carries the identifier of the voice assistant and the control mode of the calling user for the first call.

Preferably, the determining whether the calling party is controlling the second call and whether the calling party is controlling the second call in the second control mode is that the transceiver unit receives a user status message from the scheduler, and the processing unit determines whether the calling party is controlling the second call and whether the calling party is controlling the second call in the second control mode based on the user status message.

Preferably, after the transceiver unit obtains the first duration, the processing unit determines whether the first duration is less than or equal to a first waiting threshold, and if so, performs the dialog strategy adjustment according to the first duration and the subsequent operations thereof.

Preferably, if the first time length is greater than a first waiting threshold, the processing unit triggers the transceiver unit to send a preset third call voice to the first called user, and ends the first call. For example, the preset third spoken voice may be "sorry, current system busy, we will contact you again ten minutes later, see again". And the transceiver unit responds to the trigger of the processing unit and sends a preset third call voice to the first called user.

Preferably, the first duration is obtained, and specifically, the transceiver unit obtains the first duration from the scheduler.

Preferably, after the first duration, the processing unit triggers the transceiver unit, and the transceiver unit responds to the triggering of the processing unit, outputs a control request message to the calling subscriber, acquires a control response message of the calling subscriber, and triggers the processing unit after acquiring the control response message. The processing unit responds to the trigger of the receiving and sending unit, and switches the call mode from the voice assistant call mode to the user call mode, so that the calling user controls the first call in a second control mode.

Preferably, the processing unit determines whether the first time length is less than or equal to a second waiting threshold, determines whether the length of the waiting control queue is less than or equal to a queue threshold, and if both of the two determinations are yes, then executes the dialogue adjustment strategy according to the first time length and the subsequent operations thereof. If at least one of the two judgment results is negative, the processing unit triggers the transceiver unit to send a preset third call voice to the first called user, and ends the first call. And the transceiver unit responds to the trigger of the processing unit and sends a preset third call voice to the first called user. The second wait threshold is greater than the first wait threshold.

Preferably, the processing unit triggers the transceiver unit to send second update information to the scheduler before or after adjusting the session policy according to the first duration, so that the scheduler maintains a wait control queue according to the second update information. For example, the second update information carries the identifier of the voice assistant and the requested control mode of the calling party, and the scheduler adds the identifier of the voice assistant to the waiting control queue and correspondingly stores the requested control mode of the calling party.

Preferably, before triggering the transceiving unit to obtain the first duration, the processing unit triggers the transceiving unit to obtain a user class of a second called user (for example, obtained from a scheduler), the processing unit determines whether the user class of the second called user is lower than the user class of the first called user, if so, the processing unit triggers the transceiving unit to send a priority control request to the scheduler and receive a priority control response sent by the scheduler, outputs a control request message to the calling user, and obtains a control response message of the calling user, the processing unit switches the call mode of the processing unit from the voice assistant call mode to the user call mode, and the calling user controls the first call in the second control mode and controls the second call in the first control mode. And the transceiver unit responds to the trigger of the processing unit, sends a priority control request to the scheduler, receives a priority control response sent by the scheduler, outputs a control request message to the calling subscriber and acquires a control response message of the calling subscriber. Specifically, the transceiver unit sends a priority control request to a scheduler, the scheduler sends a control mode switching instruction to the second voice assistant after receiving the priority control request, the second voice assistant outputs a first prompt message to the calling user after receiving the control mode switching instruction, switches the call mode of the calling user from a user call mode to an auxiliary call mode, and sends a control mode switching response to the scheduler, the scheduler sends a priority control response to the transceiver unit of the voice assistant after receiving the control mode switching response, the transceiver unit receives the priority control response from the scheduler, outputs a control request message to the calling user, acquires a control response message of the calling user, and triggers the processing unit after acquiring the control response message. The processing unit responds to the trigger of the receiving and sending unit and switches the call mode from the voice assistant call mode to the user call mode, so that a calling user controls the first call in a second control mode and controls the first call in the second control mode; otherwise, the processing unit triggers the transceiver unit to acquire the first duration and execute the subsequent operation.

Preferably, the processing unit further switches the call mode from the user call mode to the voice assistant call mode after the user finishes controlling the call.

Preferably, the processing unit switches the call mode of the calling party according to a call control ending instruction acquired by the transceiver unit from the calling party, and triggers the transceiver unit to send first update information to the scheduler, so that the scheduler updates the recorded and maintained user state information, and selects a next call controlled by the user (see specifically the following description). The first updated information indicates that the control of the first call by the calling user has ended.

The receiving and sending unit is further configured to receive a control mode switching instruction sent by the scheduler when the user controls the first call in a second control mode, trigger the processing unit to switch the call mode of the processing unit from a user call mode to an auxiliary call mode, and trigger the receiving and sending unit to output a first prompt message to a calling user.

Preferably, the scheduler updates the recorded and maintained user status information after sending the priority control response.

By the method and the voice assistant, the invention provides a plurality of calling user call control modes for the user, and provides a reasonable calling call control strategy, so that the calling user can flexibly control the calls of a plurality of terminals or the voice assistant, and the user experience of the user when the voice assistant actively calls is improved.

The invention provides a method for calling by voice assistant, referring to fig. 4, the method comprises the following steps:

Preferably, before the voice assistant determines that the calling user is required to control the first call in step 201, the voice assistant is in a voice assistant call mode, that is, the voice assistant autonomously generates a third speech sound to be sent to the called user.

Preferably, in step 201, the calling user is required to control the first call, specifically, the calling user is required to give a call content instruction, and the voice assistant generates a third speech sound based on the call content instruction and sends the third speech sound to the called user.

Preferably, in step 201, the voice assistant determines that the calling user is required to control the first call based on the content of the call between the voice assistant and the first called user.

Preferably, in step 201, the voice assistant determines that the calling user is required to control the first call in the second control mode.

Preferably, in the step 202, if the determination result is no, that is, if the calling party does not control the second call, step 205 is executed, and the voice assistant outputs a control request message to the calling party and acquires a control response message from the calling party. After the control response message is acquired, the voice assistant switches the call mode of the voice assistant from the call mode of the voice assistant to the call mode of the user, so that the calling user controls the first call in a second control mode.

Preferably, in step 202, if the calling user is controlling the second call, it is further determined whether the calling user is controlling the second call in the second control mode, and if so, step 203 is executed.

Preferably, in step 202, if the calling party is not controlling the second call in the second control mode, step 206 is executed, the voice assistant outputs a control request message to the calling party and acquires a control response message of the calling party, and after the voice assistant acquires the control response message of the calling party, the voice assistant switches the call mode from the voice assistant call mode to the user call mode, so that the calling party controls the first call in the second control mode, and simultaneously controls the second call in another control mode other than the second control mode, for example, the first control mode.

Preferably, in step 205 and/or step 206, after the voice assistant switches its call mode from the voice assistant call mode to the user call mode, the voice assistant also sends first update information to the scheduler, so that the scheduler updates the user status information recorded and maintained by the scheduler. The first updating information carries the identifier of the voice assistant and the control mode of the calling user for the first call.

Preferably, the voice assistant determines whether the calling user is controlling the second call and whether the calling user is controlling the second call in the second control mode in step 202, and specifically, the voice assistant receives a user status message from the scheduler and determines whether the calling user is controlling the second call and whether the calling user is controlling the second call in the second control mode based on the user status message.

Step 203 essentially downgrades control of the first call from the second control mode to the first control mode. Step 203 further comprises the voice assistant switching its talk mode from the voice assistant talk mode to the auxiliary talk mode, the voice assistant also sending first update information to the scheduler to cause the scheduler to update its recorded and maintained user status information. The first updating information carries the identifier of the voice assistant and the control mode of the calling user for the first call.

Preferably, after step 202 and before step 203, further comprising step 202a, the voice assistant acquires the user rating of the second called user (for example, from a scheduler), and determines whether the user rating of the second called user is lower than the user rating of the first called user, if so, step 207 is executed, otherwise, step 203 is executed.

The step 207 specifically includes: the voice assistant sends a priority control request to the scheduler, outputs a control request message to the calling party after receiving a priority control response of the scheduler, acquires the control response message of the calling party, switches the call mode of the voice assistant from the voice assistant call mode to the user call mode after the voice assistant acquires the control response message of the calling party, and controls the first call in the second control mode and controls the second call in the first control mode. Specifically, the voice assistant sends a priority control request to a scheduler, so that the scheduler sends a control mode switching instruction to the second voice assistant after receiving the priority control request, the second voice assistant switches its call mode from a user call mode to an auxiliary call mode after receiving the control mode switching instruction, outputs a first prompt message to the calling user, and sends a control mode switching response to the scheduler, the scheduler sends a priority control response to the voice assistant after receiving the control mode switching response, the voice assistant outputs a control request message to the calling user and acquires a control response message of the calling user after receiving the priority control response, and the voice assistant switches its call mode from the voice call mode to the user call mode after acquiring the control response message of the calling user, and controlling the first call by the calling user in a second control mode, and simultaneously controlling the first call in the second control mode.

Preferably, in step 207, the scheduler updates the recorded and maintained user status information after sending the priority control response.

Preferably, after step 205 or step 206 or step 207, the method further comprises step 208a, the voice assistant switches its call mode from the user call mode to the voice assistant call mode.

Preferably, in step 208a, the voice assistant switches its call mode according to the call control end instruction obtained from the calling user, and the voice assistant sends first update information to the scheduler, so that the scheduler updates its recorded and maintained user status information. The first updated information indicates that the control of the first call by the calling user has ended. In addition, the related descriptions of steps 107 and 108 in the embodiment of fig. 2 also apply to steps 207 and 208a in this embodiment, and are not repeated here.

Preferably, after the first call with the first called user is continued according to the call content instruction in step 203, step 208b is executed: the voice assistant switches the call mode from the auxiliary call mode to the voice assistant call mode, for example, after the preset call is ended.

Preferably, in step 203, before the voice assistant outputs the first prompt message to the calling user, the voice assistant determines whether the required control can be degraded from the second control mode to the first control mode, if so, the voice assistant outputs the first prompt message to the calling user, otherwise, the aforementioned step 103 is executed, that is, the voice assistant acquires the first duration and the subsequent operations thereof.

The first prompt message is used for prompting a calling user that the call corresponding to the voice assistant needs to be controlled in a first control mode.

Preferably, the voice assistant determines whether the required control can be degraded from the second control mode to the first control mode, specifically, the voice assistant determines whether the required control is urgent, and if so, determines that the required control can be degraded from the second control mode to the first control mode.

Preferably, in step 101 or step 201, if the voice assistant determines that the calling user needs to control the first call in the first control mode, step 203 is executed, that is, the voice assistant outputs a first prompt message to the calling user, receives a call content instruction given by the calling user in response to the first prompt message, and continues the first call with the first called user according to the call content instruction.

Preferably, in step 101 or step 201, if the voice assistant determines that the calling user needs to control the first call in the first control mode, step 109 or step 209 is executed, it is determined whether the number of calls currently controlled by the calling user is smaller than a preset number, if so, step 203 is executed, that is, the voice assistant outputs a first prompt message to the calling user, receives a call content instruction given by the calling user in response to the first prompt message, and continues the first call with the first called user according to the call content instruction; otherwise, step 110 or step 210 is executed, the voice assistant acquires a fourth time length, where the fourth time length is a time length required from the current time to the time length when the calling user can control the first control mode for the first call, and according to the fourth time length, the conversation policy is adjusted, so that the conversation between the voice assistant and the first called user is continued for the fourth time length, and after the fourth time length, the calling user controls the first call in the first control mode.

Preferably, in step 110 or step 210, the voice assistant further sends second update information to the scheduler, so that the scheduler maintains a wait control queue according to the second update information. For example, the second update information carries the identifier of the voice assistant and the requested control mode of the calling party, and the scheduler adds the identifier of the voice assistant to the waiting control queue and correspondingly stores the requested control mode of the calling party.

And when the voice assistant judges that the number of calls currently controlled by the calling user is less than the preset number, the voice assistant is also used for sending first updating information to a scheduler so that the scheduler updates the recorded and maintained user state information. The first updating information carries the identifier of the voice assistant and the control mode of the calling user.

And when the voice assistant judges that the number of calls currently controlled by the calling user is not less than the preset number, the voice assistant is also used for sending second updating information to a scheduler so that the scheduler maintains a waiting control queue according to the second updating information. For example, the second update information carries the identifier of the voice assistant and the requested control mode of the calling party, and the scheduler adds the identifier of the voice assistant to the waiting control queue and correspondingly stores the requested control mode of the calling party.

The voice assistant judges whether the number of calls currently controlled by the calling party is smaller than a preset number, specifically, obtains a user state message from a scheduler, determines the number of calls currently controlled by the calling party based on the user state message, and further judges whether the number of calls currently controlled by the calling party is smaller than the preset number.

The voice assistant obtains the fourth duration from a scheduler.

The present invention also provides a voice assistant, as described above and shown in fig. 3, comprising a transceiver unit and a processing unit; the two units have the following functions in addition to the functions described above:

Preferably, the second call is any one of the calls that the voice assistants of the calling user are in progress, except the first call. The called users corresponding to the calls which are carried out by the voice assistants are different.

Preferably, if the calling user is controlling the second call, the processing unit further determines whether the calling user is controlling the second call in the second control mode, if so, triggers the transceiver unit to output a first prompt message to the calling user, and receives a call content instruction given by the calling user in response to the first prompt message while controlling the second call; the processing unit is further configured to continue the first call with the first called user according to the call content instruction.

The processing unit triggers the transceiver unit to output a first prompt message to the calling user, receives a call content instruction given by the calling user in response to the first prompt message while controlling the second call, and substantially degrades the control of the first call from the second control mode to the first control mode. The processing unit also switches the call mode from the voice assistant call mode to the auxiliary call mode, and the processing unit also triggers the transceiver unit to send first updating information to the scheduler so that the scheduler updates the recorded and maintained user state information. The first updating information carries the identifier of the voice assistant and the control mode of the calling user.

Preferably, before the processing unit triggers the transceiver unit to output the first prompting message to the calling subscriber, the processing unit triggers the transceiver unit to acquire the subscriber class of the second called subscriber (for example, from the scheduler), the processing unit judges whether the user grade of the second called user is lower than the user grade of the first called user, if yes, the processing unit triggers the transceiver unit to send a priority control request to the scheduler and receive a priority control response sent by the scheduler, outputs a control request message to the calling subscriber, and acquiring a control response message of the calling user, switching the call mode of the processing unit from a voice assistant call mode to a user call mode by the processing unit, controlling the first call by the calling user in a second control mode, and controlling the second call in the first control mode. And the transceiver unit responds to the trigger of the processing unit, sends a priority control request to the scheduler, receives a priority control response sent by the scheduler, outputs a control request message to the calling subscriber and acquires a control response message of the calling subscriber. Specifically, the transceiver unit sends a priority control request to a scheduler, the scheduler sends a control mode switching instruction to the second voice assistant after receiving the priority control request, the second voice assistant outputs a first prompt message to the calling user after receiving the control mode switching instruction, switches the call mode of the calling user from a user call mode to an auxiliary call mode, and sends a control mode switching response to the scheduler, the scheduler sends a priority control response to the transceiver unit of the voice assistant after receiving the control mode switching response, the transceiver unit receives the priority control response from the scheduler, outputs a control request message to the calling user, acquires a control response message of the calling user, and triggers the processing unit after acquiring the control response message. The processing unit responds to the trigger of the receiving and sending unit and switches the call mode from the voice assistant call mode to the user call mode, so that a calling user controls the first call in a second control mode and controls the first call in the second control mode; otherwise, the processing unit triggers the receiving and sending unit to output a first prompt message to the calling user.

Preferably, after the user's control of the call is finished, the processing unit switches the call mode from the user call mode to the voice assistant call mode.

Preferably, when the user controls the first call in the second control mode, the processing unit switches the call mode according to the call control end instruction acquired by the transceiver unit from the calling user, and triggers the transceiver unit to send first update information to the scheduler, so that the scheduler updates the recorded and maintained user state information, and selects the next call controlled by the user (see the following description specifically). The first updated information indicates that the control of the first call by the calling user has ended.

Preferably, after continuing the first call with the first called user according to the call content instruction, the processing unit is further configured to switch the call mode from the auxiliary call mode to the voice assistant call mode, for example, after the preset call is ended. The processing unit is further configured to trigger the transceiving unit to send first update information to a scheduler to cause the scheduler to update its recorded and maintained user status information, e.g., to delete the voice assistant from a list of voice assistants that the user is controlling.

Preferably, before the processing unit triggers the transceiver unit to output the first prompt message to the calling party, the processing unit determines whether the required control can be degraded from the second control mode to the first control mode, if so, the transceiver unit is triggered to output the first prompt message to the calling party, otherwise, the processing unit triggers the transceiver unit to acquire a first duration, where the first duration is a duration required by the calling party to be able to control the first call from the current time, and adjusts a conversation policy according to the first duration, so that a conversation between the voice assistant and a first called party is continued for the first duration, and the calling party controls the first call after the first duration.

Preferably, the processing unit determines whether the required control can be degraded from the second control mode to the first control mode, specifically, the processing unit determines whether the required control is urgent, and if so, determines that the required control can be degraded from the second control mode to the first control mode.

Preferably, if the processing unit determines that the calling party needs to control the first call in the first control mode, the processing unit triggers the transceiver unit to output a first prompt message to the calling party, and receives a call content instruction given by the calling party in response to the first prompt message; the processing unit is further configured to continue the first call with the first called user according to the call content instruction.

Preferably, if the processing unit determines that the calling party needs to control the first call in the first control mode, the processing unit determines whether the number of calls currently controlled by the calling party is smaller than a preset number, if so, the processing unit triggers the transceiver unit to output a first prompt message to the calling party, and receives a call content instruction given by the calling party in response to the first prompt message while controlling the second call; the processing unit is further used for continuing a first call with a first called user according to the call content instruction; otherwise, triggering the transceiver unit to obtain a fourth time length, where the fourth time length is a time length required by the calling user to control the first call from the current time, and adjusting a conversation strategy according to the fourth time length so that the conversation between the voice assistant and the first called user can be continued for the fourth time length, so that the calling user controls the first call after the fourth time length.

And when the processing unit judges that the number of calls currently controlled by the calling subscriber is less than the preset number, the processing unit is further used for triggering the transceiver unit to send first updating information to the scheduler so that the scheduler updates the recorded and maintained user state information. The first updating information carries the identifier of the voice assistant and the control mode of the calling user.

And when the processing unit judges that the number of calls currently controlled by the calling user is not less than the preset number, the processor is further used for triggering the transceiver unit to send second updating information to the scheduler, so that the scheduling unit maintains a waiting control queue according to the second updating information. For example, the second update information carries the identifier of the voice assistant and the requested control mode of the calling party, and the scheduler adds the identifier of the voice assistant to the waiting control queue and correspondingly stores the requested control mode of the calling party.

The processing unit judges whether the number of calls currently controlled by the calling subscriber is smaller than a preset number, specifically, the processing unit triggers the transceiver unit to acquire a subscriber status message from a scheduler, and the processing unit determines the number of calls currently controlled by the calling subscriber based on the subscriber status message and further judges whether the number of calls currently controlled by the calling subscriber is smaller than the preset number.

The transceiving unit obtains the fourth duration from a scheduler.

By the method and the voice assistant, the invention provides a plurality of calling user call control modes for the user, and provides a reasonable calling call control strategy, so that the calling user can simultaneously control the calls of a plurality of terminals or the voice assistant, and the user experience of the user when the voice assistant actively calls is improved.

The invention also provides a method for processing the voice instruction of the calling party by the voice assistant, and referring to fig. 5, the method comprises the following steps:

step 301, the voice assistant receives a voice instruction of a calling user;

step 302, the voice assistant determines whether the calling user is controlling the first call, the first call is a call between the voice assistant and a first called user, the call is initiated by the calling user instead of the calling user, if yes, step 303 is executed;

step 303, the voice assistant determines whether the calling user is simultaneously controlling at least one second call, where the at least one second call is a call between the calling user and at least one second called user, and the at least one second voice assistant and the voice assistant are voice assistants of different terminals, and the different terminals all belong to the calling user; if so, go to step 304;

step 304, the voice assistant determines whether to process the content of the voice instruction based on the controlled voice assistant identifier, wherein the voice instruction comprises the controlled voice assistant identifier and the content of the voice instruction.

Preferably, in step 304, if the controlled voice assistant identifier included in the voice command is the identifier of the voice assistant, the voice command is processed, otherwise, the voice command is not processed.

Preferably, after step 301 and before step 302, step 301a is further included, the voice assistant determines whether the voice instruction includes a local task instruction identification word, if so, the voice instruction is determined to be a local task instruction, step 304 is executed, otherwise, the voice instruction is determined to be a call voice instruction, and step 302 is executed.

Preferably, the local task instruction identification word is a wake-up word of the voice assistant.

Preferably, the local task instruction identification word is an arbitrary word defined by a user.

Preferably, when the voice instruction is a local task instruction, the voice instruction is processed in step 304, specifically, the local task instruction is loaded into a local session process of the voice assistant.

Preferably, the loading the task instruction into a local session process of the voice assistant specifically includes: and when a local session process associated with the local task instruction exists between the calling user and the voice assistant, loading the task instruction into the local session process for processing, and when the local session process associated with the task instruction does not exist between the user and the voice assistant, newly establishing a local session process and loading the task instruction.

Preferably, in step 302, if the voice assistant determines that the calling party is not controlling the first call, the voice instruction is not processed.

Preferably, in step 302, if the voice assistant determines that the calling party is controlling the first call, before performing step 303, the voice assistant further determines whether the calling party is controlling the first call in a first control mode or a second control mode, if the calling party is controlling the first call in the second control mode, step 303 is performed, and if the calling party is controlling the first call in the first control mode, step 304 is performed.

Preferably, if the calling user controls the first call in the second control mode, before executing step 303, it is determined whether the voice instruction is a call control end instruction, and if not, the voice instruction content in the voice instruction is a call content instruction, and step 303 is executed. If yes, the voice assistant switches the call mode from the user call mode to the voice assistant call mode, and sends first updating information to the scheduler.

Preferably, when the voice instruction is not a call control end instruction, the voice instruction is processed in step 304, specifically, the content of the voice instruction is loaded into a call session process corresponding to the first call.

Preferably, if the calling user controls the first call in the first control mode, after the voice command is processed in step 304, the voice assistant updates the number of control turns of the calling user, and determines whether the updated number of control turns reaches a first preset turn, if so, the voice assistant switches the call mode from the auxiliary control mode to the voice assistant control mode, and sends first update information to the scheduler, so that the scheduler selects a next call controlled by the calling user and updates the recorded and maintained user status information, for example, the voice assistant is deleted from a voice assistant list being controlled by the user, and the next call is added to a voice assistant list being controlled by the user.

Preferably, in step 303, if the voice assistant determines that the calling user is not simultaneously controlling at least one second call, the voice instruction is processed.

Preferably, when the voice instruction is a call voice instruction and the calling party controls the first call in the first control mode, the voice instruction is processed in step 304, specifically, the content of the voice instruction is loaded into a call session process corresponding to the first call, and the content of the voice instruction is a call content instruction.

Preferably, the voice assistant determines whether the calling party is controlling the first call, and/or the voice assistant further determines whether the calling party is controlling the first call in a first control mode or a second control mode, specifically, the voice assistant completes the determination by determining a call mode thereof, if the voice assistant is in the voice assistant call mode, the calling party is not controlling the first call, if the voice assistant is in the auxiliary call mode, the calling party is controlling the first call in the first control mode, and if the voice assistant is in the user call mode, the calling party is controlling the first call in the second control mode.

Preferably, the voice assistant determines whether the calling subscriber is simultaneously controlling at least one second call, and specifically, the voice assistant receives a subscriber status message from a scheduler, and determines whether the calling subscriber is simultaneously controlling at least one second call based on the subscriber status message.

the receiving and sending unit is used for receiving the voice instruction of the calling user;

the processing unit is configured to determine whether the calling user is controlling the first call, where the first call is a call between the calling user and a first called user initiated by the voice assistant, and if the calling user is controlling the first call, determine whether the calling user is simultaneously controlling at least one second call, where the at least one second call is a call between the calling user and at least one second called user initiated by at least one second voice assistant, the at least one second voice assistant and the voice assistant are voice assistants of different terminals, and the different terminals all belong to the calling user; and if the calling user is controlling at least one second call at the same time, determining whether to process the voice instruction content based on the controlled voice assistant identifier, wherein the voice instruction comprises the controlled voice assistant identifier and the voice instruction content.

And if the controlled voice assistant identifier included in the voice instruction is the identifier of the voice assistant, processing the voice instruction by a processing unit, otherwise, not processing the voice instruction.

Preferably, before determining whether the calling user is controlling the first call, the processing unit is further configured to determine whether the voice instruction includes a local task instruction identification word, if so, determine that the voice instruction is a local task instruction, determine whether to process the content of the voice instruction based on a controlled voice assistant identifier, otherwise, determine that the voice instruction is a call voice instruction, and determine whether the calling user is controlling the first call.

Preferably, when the voice instruction is a local task instruction, the voice instruction is processed, specifically, the local task instruction is loaded into a local session process of the voice assistant.

Preferably, the loading the task instruction into a local session process of the voice assistant specifically includes: and when a local session process associated with the task instruction exists between the user and the voice assistant, loading the task instruction into the local session process for processing, and when the local session process associated with the task instruction does not exist between the user and the voice assistant, newly establishing a local session process and loading the task instruction.

Preferably, if the processing unit determines that the calling party is not controlling the first call, the processing unit does not process the voice command.

Preferably, if the processing unit determines that the calling party is controlling the first call, before determining whether the calling party is controlling at least one second call at the same time, the voice assistant further determines whether the calling party is controlling the first call in a first control mode or a second control mode, if the calling party is controlling the first call in the second control mode, the determining whether the calling party is controlling at least one second call at the same time and subsequent operations are performed, and if the calling party is controlling the first call in the first control mode, whether to process the voice instruction content is determined based on a controlled voice assistant identifier.

Preferably, if the calling user controls the first call in the second control mode, before determining whether the calling user is simultaneously controlling at least one second call, the processing unit determines whether the voice instruction is a call control end instruction, if not, the voice instruction content in the voice instruction is a call content instruction, and at this time, determines whether the calling user is simultaneously controlling at least one second call. If the current conversation mode is the voice-assisted conversation mode, the processing unit switches the conversation mode from the user conversation mode to the voice-assisted conversation mode and triggers the transceiver unit to send first updating information to the scheduler. The transceiving unit sends first update information to a scheduler in response to a trigger of the processing unit.

Preferably, when the voice instruction is not a call control end instruction, the voice instruction is processed, specifically, the content of the voice instruction is loaded into a call session process corresponding to the first call.

Preferably, if the calling party controls the first call in the first control mode, after the voice command is processed, the processing unit updates the number of control rounds of the calling party, and determines whether the updated number of control rounds reaches a first preset round, if so, the processing unit switches the call mode from the auxiliary control mode to the voice assistant control mode, and triggers the transceiver unit to send first update information to the scheduler, so that the scheduler updates the recorded and maintained user state information, for example, the voice assistant is deleted from a voice assistant list being controlled by the user. The transceiving unit sends first update information to a scheduler in response to a trigger of the processing unit.

Preferably, if the processing unit determines that the calling subscriber is not simultaneously controlling at least one second call, the processing unit processes the voice command.

Preferably, when the voice instruction is a call voice instruction and the calling party controls the first call in the first control mode, the voice instruction is processed, specifically, the content of the voice instruction is loaded into a call session process corresponding to the first call, and the content of the voice instruction is a call content instruction.

Preferably, the processing unit determines whether the calling party is controlling the first call, and/or the processing unit further determines whether the calling party is controlling the first call in a first control mode or a second control mode, specifically, the processing unit determines the call mode of a voice assistant to complete the determination, if the voice assistant is in a voice assistant call mode, the calling party is not controlling the first call, if the voice assistant is in an auxiliary call mode, the calling party is controlling the first call in the first control mode, and if the voice assistant is in a user call mode, the calling party is controlling the first call in the second control mode.

Preferably, the processing unit determines whether the calling subscriber is simultaneously controlling at least one second call, specifically, the processing unit triggers the transceiver unit to receive a user status message from the scheduler, and determines whether the calling subscriber is simultaneously controlling at least one second call based on the user status message. The transceiving unit receives a user status message from a scheduler in response to a trigger of the processing unit.

By the method and the voice assistant, when the user controls the calls of the plurality of voice assistants simultaneously, the voice assistant can accurately process the user instruction, and meanwhile, local tasks and the calls can be smoothly carried out, so that the user experience of the user when the voice assistant actively calls is improved.

The present invention also provides a method for scheduling a calling party, referring to fig. 6, the method includes the following steps:

step 401, determining whether a first update message is received from a first voice assistant of a plurality of voice assistants, if yes, executing step 402;

wherein the first voice assistant sends the first update information when determining that the calling user has completed controlling the first call corresponding thereto;

step 402, inquiring the control mode of the calling user for the first call in the calling user state information based on the first updating information;

the state information of the calling party records a plurality of voice assistants and corresponding control modes corresponding to a plurality of calls which are simultaneously controlled by the calling party, wherein the calls are the communications established between the plurality of voice assistants and a plurality of different called users respectively; the first voice assistant belongs to the plurality of voice assistants, and the first call belongs to the plurality of calls;

step 403, based on the control mode obtained by the query, selecting a second voice assistant from the waiting control queue, sending a control mode switching instruction to the second voice assistant, and after receiving the control mode switching instruction, triggering the calling user to control the call of the second voice assistant in the control mode by the second voice assistant.

Preferably, after step 403, the caller status information is updated, specifically, the first voice assistant and the corresponding control mode are deleted, and a second voice assistant and the corresponding control mode are added.

Preferably, after the step 402 and before the step 403 are performed, the step 402a is performed to determine whether a third voice assistant stored in association with the first voice assistant exists, if so, determine whether the third call of the third voice assistant is finished by the calling party based on the calling party status information, and if not, send a control mode switching instruction to the third voice assistant, so that the calling party controls the third call in the second control mode; if there is no third call or if the control of the third call has ended, step 403 is executed.

Preferably, after receiving the control mode switching instruction, the third voice assistant outputs a control request message to the calling user, obtains a control response message of the calling user, and switches the call mode of the calling user from the auxiliary call mode to the user call mode.

Preferably, when the first voice assistant determines that the calling user needs to control the first call in the second control mode, the calling user is controlling the third call in the second control mode, but the called user level of the first call is higher than the called user level of the third call, and at this time, the calling user is triggered to control the first call in the second control mode, and the third call is controlled in the first control mode.

Preferably, when receiving the priority control request sent by the first voice assistant, the first voice assistant sends a control mode switching instruction to the third voice assistant, and receives a control mode switching response from the third voice assistant, and then stores the first voice assistant and the third voice assistant in an associated manner.

Preferably, a control mode switching response is further received from the third voice assistant, and the state information of the calling party is updated, specifically, the control mode corresponding to the third voice assistant is modified to the second control mode.

Preferably, if the control mode obtained by querying the status information of the calling party is the second control mode, step 402a is executed, that is, "determine whether there is a third voice assistant stored in association with the first voice assistant" and its subsequent operations. If the control mode found by the query in the status information of the calling party is the first control mode, step 403 is executed, that is, "select the second voice assistant from the waiting control queue based on the control mode" and its subsequent operations.

Preferably, after receiving the control mode switching instruction, the second voice assistant further switches its call mode from the voice assistant call mode to a call mode corresponding to the control mode.

Preferably, if the control mode obtained by the query is the first control mode, the control mode switching instruction instructs the second voice assistant to switch its call mode from the voice assistant call mode to the auxiliary call mode.

Preferably, if the control mode obtained by the query is the first control mode, the second voice assistant triggers the calling party to control the call of the second voice assistant in the control mode, specifically, the second voice assistant outputs a first prompt message to the calling party, receives a call content instruction given by the calling party in response to the first prompt message, and continues the first call with the first called party according to the call content instruction.

Preferably, if the control mode obtained by the query is a second control mode, the second voice assistant triggers the calling party to control the call of the second voice assistant in the control mode, specifically, the second voice assistant outputs a control request message to the calling party and obtains a control response message of the calling party.

Preferably, if the control mode obtained by the query is the second control mode, the control mode switching instruction instructs the second voice assistant to switch the call mode of the second voice assistant from the voice assistant call mode to the user call mode.

Preferably, the method is performed by the scheduler described above.

Preferably, the plurality is the preset number.

Preferably, the second voice assistant is the voice assistant which is added to the waiting control queue at the earliest and the corresponding control mode is the control mode obtained by the query.

Preferably, the process of forming the wait control queue refers to the method embodiment corresponding to fig. 2 and fig. 4. And will not be described in detail herein.

The present invention also provides a scheduling apparatus for calling users, referring to fig. 7, the apparatus includes a transceiver unit, a processing unit and a storage unit;

the receiving and sending unit judges whether first updating information is received from a first voice assistant in the plurality of voice assistants, and if so, the processing unit is triggered;

the processing unit is used for responding to the trigger of the transceiver unit and inquiring the control mode of the calling party on the first call in the state information of the calling party based on the first updating information;

the processing unit is further configured to select a second voice assistant from a waiting control queue based on the control mode obtained through query, trigger the transceiver unit to send a control mode switching instruction to the second voice assistant, and trigger the calling user to control a call of the second voice assistant in the control mode after the second voice assistant receives the control mode switching instruction;

the receiving and sending unit is used for responding to the trigger of the processing unit and sending a control mode switching instruction to the second voice assistant;

the storage unit is used for storing the calling user state information and the waiting control queue.

Preferably, the processing unit is further configured to update the status information of the calling party, specifically, delete the first voice assistant and the corresponding control mode, and add a second voice assistant and the corresponding control mode.

Preferably, the processing unit is further configured to: after the control mode is inquired and obtained and before a second voice assistant is selected from a waiting control queue, judging whether a third voice assistant which is stored in association with the first voice assistant exists or not, if so, judging whether the control of the calling user on a third call of the third voice assistant is finished or not based on the calling user state information, and if not, triggering the transceiver unit to send a control mode switching instruction to the third voice assistant, so that the calling user controls the third call in a second control mode; the processing unit is further to: said selecting of the second voice assistant from the waiting control queue is performed if no third voice assistant is present or if the control of the third call by the calling user has ended.

The receiving and sending unit is used for responding to the trigger of the processing unit and sending a control mode switching instruction to the third voice assistant.

Preferably, the transceiver unit sends a control mode switching instruction to the third voice assistant when receiving the priority control request sent by the first voice assistant, receives a control mode switching response from the third voice assistant, and triggers the processing unit, and the processing unit stores the first voice assistant and the third voice assistant in a storage unit in association with each other in response to the triggering of the transceiver unit.

Preferably, the storage unit is further configured to store the first voice assistant and a third voice assistant in association.

Preferably, the transceiver unit is further configured to receive a control mode switching response from the third voice assistant and trigger the processing unit; and the processing unit responds to the trigger of the transceiver unit, updates the state information of the calling party based on the control mode switching response, and specifically modifies a control mode corresponding to a third voice assistant in the state information of the calling party into a second control mode.

Preferably, the processing unit is further configured to: and if the control mode inquired in the state information of the calling party is the second control mode, executing the step of judging whether a third voice assistant stored in association with the first voice assistant exists or not and subsequent operations of the third voice assistant, and if the control mode inquired in the state information of the calling party is the first control mode, executing the step of selecting the second voice assistant from the waiting control queue based on the control mode and subsequent operations of the second voice assistant.

Preferably, if the control mode obtained by querying is the first control mode, the second voice assistant switches the call mode of the second voice assistant from the voice assistant call mode to the auxiliary call mode.

Preferably, if the control mode obtained by querying is the second control mode, the second voice assistant switches the call mode of the second voice assistant from the voice assistant call mode to the user call mode.

Preferably, the scheduler comprises the scheduling means.

Preferably, the plurality is the preset number.

Preferably, the process of forming the wait control queue and the process of updating and maintaining the wait control queue refer to the method embodiments corresponding to fig. 2 and fig. 4. And will not be described in detail herein.

By the method and the device, when calling calls of a plurality of voice assistants need to be controlled by a user, the calls can be efficiently scheduled, and user experience of calling calls of the voice assistants is improved.

The invention also provides a method for synchronously controlling the call, and referring to fig. 8, the method comprises the following steps:

step 501, obtaining a call progress of M calls being performed by M voice assistants of a calling user, where one of the M calls is a call initiated by one of the M voice assistants to one of M called users instead of the calling user, and the one of the M voice assistants replaces the calling user to perform a conversation with the one of the M called users;

step 502, clustering the M calls based on the call progress of the M calls to form N call classes; each of the M calls belongs to only one of the N call classes;

wherein M and N are positive integers, M is greater than 1, and N is less than M;

step 503, performing call control synchronization among a plurality of calls included in one call class, so that when the calling party controls a first call in the plurality of calls, a second call in the plurality of calls is synchronously controlled by the calling party.

Preferably, the execution subject of the method is the aforementioned scheduler.

Preferably, the clustering is performed on the M calls, specifically: and clustering the calls with the same call progress or the call progress difference smaller than the threshold value into a call class.

The call progress difference may be characterized by a number of conversation rounds.

Preferably, the call intentions and/or called user attributes of the M calls are further obtained, the M calls are clustered, and based on the call intentions and the called user attributes, specifically, calls with the same call intentions and/or the same or similar user attributes and the same or similar call progress or with a call progress difference smaller than a threshold value are clustered into a call class.

Preferably, the call class further records a plurality of voice assistant identifiers for initiating a plurality of calls included therein, i.e. a plurality of voice assistant identifiers corresponding to the plurality of calls.

Preferably, when a calling party controls a first call among a plurality of calls included in one call class, a second call among the plurality of calls is synchronously controlled by the calling party, specifically, a first update message sent by a first voice assistant is received, it is determined that the calling party starts to control the first call based on the first update message, a call control synchronization message is sent to a second voice assistant, so that the second voice assistant switches its call mode, the first voice assistant is a voice assistant that initiates the first call, and the second voice assistant is a voice assistant that initiates the second call.

Preferably, based on the user control mode carried in the first update message, it is determined that the calling user controls the first call in the first control mode, and a call control synchronization message is sent to the second voice assistant, so that the second voice assistant switches its call mode to the auxiliary control mode.

Preferably, based on the user control mode carried in the first update message, it is determined that the calling user controls the first call in the second control mode, and a call control synchronization message is sent to the second voice assistant, so that the second voice assistant switches its call mode to the user control mode.

Preferably, the call control synchronization is performed among a plurality of calls included in one call class, so that when a calling user controls a first call in the plurality of calls, a second call in the plurality of calls is synchronously controlled by the calling user, and the method further includes receiving a third call voice of the calling user from the first voice assistant, and sending the third speech voice to the second voice assistant, so that the second voice assistant sends the third speech voice to a second called user according to the call progress of the second call, and the second called user is a called user corresponding to the second call.

In this embodiment, how the first voice assistant sends the first update message under what condition and how to generate the third call voice under the auxiliary control mode or the user control mode are the same as those in the foregoing embodiment, and details are not described here again.

In the invention, only the first voice assistant and the corresponding user control mode are recorded in the user state information of the scheduler.

Preferably, the method further includes step 504, receiving a priority control request from a third voice assistant (i.e. the voice assistant in step 107 of the foregoing embodiment), sending a control mode switching instruction to a fourth voice assistant (i.e. the second voice assistant in step 107 of the foregoing embodiment), and sending a call control synchronization message to a fifth voice assistant, where a fifth call corresponding to the fifth voice assistant and a fourth call corresponding to the fourth voice assistant belong to the same call class; after receiving the instruction or the message, the fourth voice assistant and the fifth voice assistant switch their call modes to an auxiliary call mode and send a control mode switch response to a scheduler, where the scheduler sends a priority control response to the third voice assistant (i.e., the voice assistant in step 107) and sends a call control synchronization message to a sixth voice assistant so that the sixth voice assistant switches its call mode to a user control mode, and a sixth call corresponding to the sixth voice assistant and a third call corresponding to the third voice assistant (i.e., the first call in step 107) belong to the same call class. The other matters involved in step 107 are equally applicable to step 504.

Preferably, the method further comprises the steps of: step 505, receiving a first update message from a first voice assistant (i.e. the first voice assistant in step 401), determining that the calling user has completed controlling the first call (i.e. the first call in step 401) based on the first update message, the first voice assistant being the voice assistant initiating the first call; and sending a call control synchronization ending message to a second voice assistant so that the second voice assistant switches the call mode thereof into a voice assistant call mode.

Preferably, the method further comprises the steps of: step 506, inquiring the control mode of the calling party for the first call in the calling party state information based on the first updating information; step 507, based on the control mode obtained by the query, selecting a seventh voice assistant from the waiting control queue, and performing call control synchronization between the seventh call and an eighth call, where the seventh call corresponding to the seventh voice assistant and the eighth call corresponding to the eighth voice assistant belong to the same call class. Specifically, a control mode switching instruction is sent to the seventh voice assistant, a call control synchronization message is sent to the eighth voice assistant, and the seventh voice assistant and the eighth voice assistant switch their call modes to the call mode corresponding to the control mode.

Preferably, the step 505 may further be that first update information is received from a third voice assistant, it is determined that the calling user has completed controlling the third call based on the first update information, and a call control synchronization end message is sent to a sixth voice assistant; after receiving the call control synchronization end message, the sixth voice assistant switches the call mode of the sixth voice assistant into a voice assistant call mode; the method further includes step 505a, determining whether the control of the calling user on the fourth call of the fourth voice assistant is finished, if not, sending a control mode switching instruction to the fourth voice assistant, and sending a call control synchronization message to the fifth voice assistant, so that the calling user controls the fourth call in the second control mode and synchronously controls the fifth call. The fourth voice assistant and the fifth voice assistant switch the communication mode thereof from the auxiliary communication mode to the user communication mode after receiving the instruction or the message; if so, the above steps 506 and 507 are performed.

Preferably, when the current call of a certain voice assistant is ended and the next call is started, whether the next call can be aggregated into a certain call class is judged.

The invention also provides a device for synchronously controlling the call, which is the dispatching device in fig. 7 and comprises a receiving and sending unit, a processing unit and a storage unit; the three units have the following functions in addition to the functions described above:

the receiving and sending unit is configured to acquire a call progress of M calls being performed by M voice assistants of a calling user, where one of the M calls is a call initiated by one of the M voice assistants to one of M called users instead of the calling user, and the one of the M voice assistants replaces the calling user to have a conversation with the one of the M called users;

the processing unit is used for clustering the M calls based on the call progress of the M calls to form N call classes; each of the M calls belongs to only one of the N call classes; wherein M and N are positive integers, M is greater than 1, and N is less than M;

the storage unit is used for storing the formed N conversation classes;

the processing unit is further to: the method comprises the steps of carrying out call control synchronization among a plurality of calls included in one call class, so that when a calling party controls a first call in the plurality of calls, a second call in the plurality of calls is synchronously controlled by the calling party.

Preferably, the device is the aforementioned scheduler.

Preferably, the transceiver unit is further configured to: acquiring the call intentions and/or called user attributes of the M calls; the processing unit clusters the M calls, and specifically clusters the calls with the same call intention and/or the same or similar user attributes and the same call progress or the call progress difference smaller than a threshold value into a call class based on the call intention and the called user attributes.

Preferably, the transceiver unit is further configured to receive a first update message sent by a first voice assistant, the processing unit determines that a calling party controls the first call based on the first update message, and triggers the transceiver unit to send a call control synchronization message to a second voice assistant, so that the second voice assistant switches its call mode, where the first voice assistant is a voice assistant that initiates the first call, and the second voice assistant is a voice assistant that initiates the second call. The transceiving unit sends a call control synchronization message to a second voice assistant in response to the trigger of the processing unit.

Preferably, the processing unit is further configured to: and determining that a calling user controls the first call in a first control mode based on a user control mode carried in the first updating message, and triggering the transceiver unit to send a call control synchronization message to the second voice assistant so that the second voice assistant switches the call mode to an auxiliary control mode.

Preferably, the processing unit is further configured to: and determining that a calling user controls the first call in a second control mode based on the user control mode carried in the first updating message, and triggering the transceiver unit to send a call control synchronization message to the second voice assistant so that the second voice assistant switches the call mode to the user control mode.

Preferably, the transceiver unit is further configured to receive a third call voice of the calling user from the first voice assistant, and send the third call voice to the second voice assistant under the trigger of the processing unit, so that the second voice assistant sends the third call voice to a second called user according to the call progress of the second call, where the second called user is a called user corresponding to the second call.

Preferably, the transceiver unit is further configured to: receiving a priority control request from a third voice assistant (i.e., the voice assistant in step 107 of the foregoing embodiment), sending a control mode switching instruction to a fourth voice assistant (i.e., the second voice assistant in step 107 of the foregoing embodiment) under the trigger of the processing unit, and sending a call control synchronization message to a fifth voice assistant, where a fifth call corresponding to the fifth voice assistant and a fourth call corresponding to the fourth voice assistant belong to the same call class; after receiving the instruction or the message, the fourth voice assistant and the fifth voice assistant switch their call modes to an auxiliary call mode, and send a control mode switch response to the transceiver unit, where the transceiver unit sends a priority control response to the third voice assistant (i.e., the voice assistant in step 107) and sends a call control synchronization message to the sixth voice assistant under the trigger of the processing unit after receiving the control mode switch response, so that the sixth voice assistant switches its call mode to the user control mode, and a sixth call corresponding to the sixth voice assistant and a third call corresponding to the third voice assistant (i.e., the first call in step 107) belong to the same call class. Similarly, the transceiver unit is further configured to receive a third call voice of the calling user from the third voice assistant, and send the third call voice to the sixth voice assistant under the triggering of the processing unit. The transceiving unit is also used for receiving a third call voice of a calling user from the fourth voice assistant and sending the third call voice to the fifth voice assistant under the triggering of the processing unit. In the present invention, the above steps are performed for a plurality of calls belonging to the same call class, and are not described in detail below.

Preferably, the transceiver unit is further configured to: receiving first update information from a first voice assistant (i.e., the first voice assistant in step 401), the processing unit being further configured to determine that a calling user has completed control of a first call (i.e., the first call in step 401) based on the first update information, the first voice assistant being a voice assistant that initiated the first call; and triggering the transceiver unit to send a call control synchronization ending message to a second voice assistant so that the second voice assistant switches the call mode thereof into a voice assistant call mode. And the transceiver unit responds to the trigger of the processing unit and sends a call control synchronization ending message to the second voice assistant.

Preferably, the processing unit is further configured to: and inquiring the control mode of the calling party to the first call in the state information of the calling party based on the first updating information, selecting a seventh voice assistant from a waiting control queue based on the control mode obtained by inquiry, and carrying out call control synchronization between the seventh call and an eighth call, wherein the seventh call corresponding to the seventh voice assistant and the eighth call corresponding to the eighth voice assistant belong to the same call class. Specifically, the transceiver unit is triggered to send a control mode switching instruction to the seventh voice assistant, and send a call control synchronization message to the eighth voice assistant, and the seventh voice assistant and the eighth voice assistant switch their call modes to the call mode corresponding to the control mode.

Preferably, the transceiver unit is further configured to receive first update information from a third voice assistant, and the processing unit is further configured to determine that the calling party has completed controlling the third call based on the first update information, and trigger the transceiver unit to send a call control synchronization end message to a sixth voice assistant; the processing unit is further configured to determine whether the control of the calling user on the fourth call of the fourth voice assistant is finished, and if not, trigger the transceiver unit to send a control mode switching instruction to the fourth voice assistant and send a call control synchronization message to the fifth voice assistant, so that the calling user controls the fourth call in the second control mode and synchronously controls the fifth call. The fourth voice assistant and the fifth voice assistant switch the communication mode thereof from the auxiliary communication mode to the user communication mode after receiving the instruction or the message; if the call control synchronization is finished, the processing unit inquires the control mode of the calling party on the first call in the calling party state information based on the first updating information, selects a seventh voice assistant from a waiting control queue based on the control mode obtained by inquiry, and performs call control synchronization between the seventh call and the eighth call.

By the method and the device, the call control synchronization is carried out among the multiple calls included in one call class, so that when a calling party controls a first call in the multiple calls, a second call in the multiple calls is synchronously controlled by the calling party, the efficiency of controlling the call mainly called by the voice assistant by the user is improved, and the user experience of the active call of the voice assistant is improved.

The invention also provides a method for clustering and controlling calls, and referring to fig. 9, the method comprises the following steps:

step 601, clustering M calls ongoing by M voice assistants of a calling user to form L call classes; one of the M calls is a call initiated by one of the M voice assistants to one of M called users in place of the calling user, and the one of the M voice assistants is in conversation with the one of the M called users in place of the calling user, each of the M calls belonging to only one of the L call classes, wherein M and L are positive integers, and M >1, L < M;

step 602, receiving user control call content of the calling user in controlling a first call from a first voice assistant, and sending the user control call content to a second voice assistant, so that the second voice assistant generates a third call voice based on the user control call content; wherein the first voice assistant is a voice assistant initiating the first call; the second voice assistant is a voice assistant which initiates a second call, and the first call and the second call belong to the first call class.

Preferably, the method is performed by the scheduler described above.

Preferably, in step 601, the call intentions and/or called user attributes of the M calls are obtained, the M calls are clustered based on the call intentions and/or called user attributes, and the calls with the same call intentions and/or the same user attributes or similar calls are clustered into a call class.

Preferably, the user control call content includes a call content instruction sent by the calling user, a third call voice generated by the voice assistant, and a second call voice received from the called user.

Preferably, after the control of the calling user on the first call is finished, the first voice assistant sends the user control call content, and at this time, the user control call content is received from the first voice assistant.

Preferably, each time the first voice assistant receives a call content instruction from a calling user and generates a third voice and sends the third voice to the first called user, the first voice assistant immediately carries the call content instruction, the third voice and the associated second voice received from the called user to the user-controlled call content to send, for example, to the scheduler.

Preferably, when the calling user controls the first call in the first control mode, after the control of the calling user on the first call is finished, the first voice assistant sends the call control content of the user.

Preferably, when the calling user controls the first call in the second control mode, the first voice assistant immediately carries the call content instruction, the third call sound and the associated second call sound received from the called user on the user-controlled call content for transmission after receiving the call content instruction from the calling user and generating the third call sound each time and transmitting the third call sound to the first called user.

Preferably, the second voice assistant generates a third call voice based on the user-controlled call content, specifically, the second voice assistant determines whether the user-controlled call content matches the current call progress based on the call progress between the second voice assistant and a second called user, and if so, generates the third call voice based on the user-controlled call content.

Preferably, in the case of the above matching, the second voice assistant further determines whether the calling user needs to control the second call in a second preset round of conversation thereafter, if so, the second voice assistant sends a message that the user control call content is invalid, and performs step 102 and subsequent steps, and if not, the second voice assistant sends a message that the user control call content is valid.

Preferably, the method further comprises step 603, receiving a user control call content invalidation message or a user control call content validation message from the second voice assistant.

Preferably, the method further comprises: step 604, receiving a third call ending message and a fourth call starting message from a third voice assistant, determining whether the fourth call can be aggregated to the first call class, and if so, sending the user control call content to the third voice assistant.

The third call and the fourth call are two calls initiated by the third voice assistant in sequence and correspond to different called users.

Preferably, the third voice assistant may be the first voice assistant, the second voice assistant, or any other one of the M voice assistants.

Preferably, the method further comprises: and when a preset condition is met, calculating the ratio of the number of the received effective messages of the control call content to the number of the received ineffective messages of the control call content, if the ratio is greater than an updating threshold value, updating the main call sub-engine, and sending updating messages to the M voice assistants so that the M voice assistants download the updated main call sub-engine.

The clustering method in fig. 8 is referred to as a first clustering method, the clustering method in fig. 9 is referred to as a second clustering method, the calling subscriber can preset the clustering method as the first clustering method or the second clustering method according to the requirement, and the scheduler performs clustering and subsequent operations according to the preset clustering method.

The invention also provides a device for clustering and controlling calls, which comprises a transceiver unit, a processing unit and a storage unit, and the units have the following functions besides the functions shown in the figure 7:

the processing unit is used for clustering M calls which are currently carried out by M voice assistants of a calling user to form L call classes; one of the M calls is a call initiated by one of the M voice assistants to one of M called users in place of the calling user, and the one of the M voice assistants is in conversation with the one of the M called users in place of the calling user, each of the M calls belonging to only one of the L call classes, wherein M and L are positive integers, and M >1, L < M;

the storage unit is used for storing the formed L call classes;

the receiving and sending unit is used for receiving user control call content of the calling user when controlling the first call from the first voice assistant;

the processing unit is also used for triggering the receiving and sending unit to send the received user control call content to a second voice assistant;

the transceiving unit is further used for responding to the trigger of the processing unit and sending the user control call content to a second voice assistant so that the second voice assistant generates a third call voice based on the user control call content; wherein the first voice assistant is a voice assistant initiating the first call; the second voice assistant is a voice assistant which initiates a second call, and the first call and the second call belong to the first call class.

Preferably, the device is the aforementioned scheduler.

Preferably, the transceiver unit is further configured to obtain call intentions and/or called user attributes of the M calls, and the processing unit clusters the M calls based on the call intentions and/or the called user attributes, and clusters calls with the same call intentions and/or the same or similar user attributes into one call class.

Preferably, after the control of the calling user on the first call is finished, the first voice assistant sends the call content controlled by the user.

Preferably, the transceiving unit is further configured to receive a user control call content invalidation message or a user control call content validation message from the second voice assistant.

Preferably, the transceiver unit is further configured to receive a third call end message and a fourth call start message from a third voice assistant, and the processing unit is further configured to determine whether the fourth call can be aggregated to the first call class, and if so, trigger the transceiver unit to send the user control call content to the third voice assistant. The transceiving unit is further used for responding to the trigger of the processing unit and sending the user control call content to the third voice assistant

Preferably, the processing unit is further configured to calculate a ratio of the number of received control call content valid messages to the number of received control call content invalid messages when a preset condition is met, update the main call sub-engine if the ratio is greater than an update threshold, and trigger the transceiver unit to send an update message to the M voice assistants, so that the M voice assistants download the updated main call sub-engine.

Preferably, the processing unit performs clustering and performs subsequent operations according to a clustering mode preset by the calling subscriber.

By the method and the device, the second voice assistant corresponding to the second call belonging to the same call class as the first call can acquire the control condition of the calling user on other calls and adjust the conversation logic between the calling user and the second called user without switching the call mode of the calling user, so that the frequency of triggering the user by the second voice assistant to control is reduced, and the user experience of the active call of the voice assistant is improved.

All methods/devices in the present invention relate to the calling mode of the voice assistant and the dispatching of the calling user, which can be combined and combined with each other at will.

The processing unit triggers the receiving and sending unit to receive or send a certain message or instruction, and the receiving and sending unit responds to the triggering of the processing unit to receive or send the certain message or instruction.

Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. The computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), a flash memory, an erasable programmable read-only memory (EPROM), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Computer program code for carrying out operations of the present invention may be written in one or more programming languages, or a combination thereof.

The above description is only an example for the convenience of understanding the present invention, and is not intended to limit the scope of the present invention. In the specific implementation, a person skilled in the art may change, add, or reduce the components of the apparatus according to the actual situation, and may change, add, reduce, or change the order of the steps of the method according to the actual situation without affecting the functions implemented by the method.

While embodiments of the invention have been shown and described, it will be understood by those skilled in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents, and all changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

1. A method of voice assisted main call, wherein the calling party has a first terminal and a second terminal, the method comprising:

step 201, the voice assistant of the first terminal determines that the calling user is required to control a first call between the voice assistant of the first terminal and a first called user, where the first call is a call between the voice assistant of the first terminal and the first called user, which is initiated by the calling user and is replaced by the voice assistant of the first terminal; before step 201, the voice assistant of the first terminal is in a voice assistant communication mode, and in the voice assistant communication mode, the voice assistant of the first terminal replaces a calling user to communicate with a called user, that is, the voice assistant of the first terminal autonomously generates a third speech sound and sends the third speech sound to the first called user;

step 202, the voice assistant of the first terminal receives a user state message from a scheduler, and judges whether a calling user is controlling a second call based on the user state message, wherein the second call is a call between the second voice assistant of the second terminal and a second called user, which is initiated by the calling user and replaced by the second voice assistant of the second terminal; if yes, go to step 202 a;

step 202a, the voice assistant of the first terminal judges whether the user level of the second called user is lower than the user level of the first called user, and if the judgment result is not lower than the user level, step 203 is executed;

step 203, the voice assistant of the first terminal outputs a first prompt message to the calling user, receives a call content instruction given by the calling user in response to the first prompt message while controlling the second call, determines the intention of the calling user according to the call content instruction, updates the decision diagram state for generating a third call voice based on the intention, generates a third call voice according to the changed decision diagram state, and sends the third call voice to the first called user, and then the voice assistant of the first terminal continuously replaces the calling user to carry out the first call with the first called user.

2. The method of claim 1, wherein in step 201, the voice assistant determines that the calling user is required to control the first call based on the content of the call between the voice assistant and the first called user.

3. The method according to claim 1, wherein in step 201, the voice assistant of the first terminal determines that the calling user is required to control the first call in a second control mode, wherein the first call is controlled in the second control mode, and specifically, the voice assistant of the first terminal receives a call content command from the calling user, generates a third call voice based on the call content command, sends the third call voice to the first called user, and receives the second call voice of the first called user, and provides the second call voice to the calling user.

4. The method of claim 3, wherein in step 202, if the calling party is controlling the second call, then before performing step 202a, it is further determined whether the calling party is controlling the second call in the second control mode, and if so, step 202a is performed.

5. A voice assistant is characterized in that the voice assistant belongs to a first terminal, the first terminal is a first terminal of a calling party, the calling party is also provided with a second terminal, and the voice assistant comprises a transceiving unit and a processing unit;

the processing unit is configured to determine that a calling user is required to control a first call between the voice assistant of the first terminal and a first called user, where the first call is a call between the voice assistant of the first terminal and the first called user; before it is determined that a calling user is required to control a first call, a voice assistant of the first terminal is in a voice assistant call mode, and in the voice assistant call mode, the voice assistant of the first terminal replaces the calling user to make a call with a called user, namely a processing unit of the voice assistant of the first terminal autonomously generates a third speech sound to trigger the transceiving unit to send the third speech sound to the first called user;

the receiving and sending unit is used for receiving the user state information from the dispatcher;

the processing unit is further configured to determine, based on the user status message, whether the calling user is controlling a second call, where the second call is a call between a second voice assistant of a second terminal of the calling user and a second called user; if yes, judging whether the user level of the second called user is lower than the user level of the first called user, if not, triggering the transceiver unit to output a first prompt message to the calling user, and receiving a call content instruction given by the calling user in response to the first prompt message while controlling the second call; the processing unit is further used for determining the intention of a calling user according to the call content instruction, updating a decision diagram state used for generating a third call voice based on the intention, generating the third call voice according to the changed decision diagram state, and triggering the transceiving unit to send the third call voice to a first called user; the processing unit is further configured to continue to perform the first call with the first called user instead of the calling user after the transceiving unit sends the third voice communication to the first called user.

The receiving and sending unit is used for responding to the trigger of the processing unit, outputting a first prompt message to a calling user, receiving a call content instruction given by the calling user in response to the first prompt message while controlling a second call, and sending the third call voice to a first called user.

6. The voice assistant of claim 5, wherein the processing unit determines that a calling user is required to control the first call based on the content of the call between the processing unit and the first called user.

7. The voice assistant of claim 5, wherein the processing unit determines that a calling user is required to control a first call in a second control mode, and specifically, the processing unit receives a call content command from the calling user, generates a third call voice based on the call content command, and triggers the transceiving unit to send the third call voice to a first called user, and the transceiving unit receives a second call voice of the first called user and provides the second call voice to the calling user.

8. The voice assistant of claim 7, wherein if the calling subscriber is controlling the second call, the processing unit first determines whether the calling subscriber is controlling the second call in the second control mode, and if so, then determines whether the user rating of the second called subscriber is lower than the user rating of the first called subscriber.

9. A computer arrangement, characterized in that the computer arrangement comprises a processor and a memory, in which a computer program is stored which is executable on the processor, which computer program, when being executed by the processor, carries out the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, in which a computer program that is executable on a processor is stored, which computer program, when being executed, carries out the method according to any one of claims 1 to 4.