WO2021103449A1

WO2021103449A1 - Interaction method, mobile terminal and readable storage medium

Info

Publication number: WO2021103449A1
Application number: PCT/CN2020/092385
Authority: WO
Inventors: 任权
Original assignee: 深圳传音控股股份有限公司
Priority date: 2019-11-29
Filing date: 2020-05-26
Publication date: 2021-06-03
Also published as: CN110944056A

Abstract

An interaction method, a mobile terminal and a readable storage medium. The method comprises: obtaining voice control information, and outputting prompt information when the voice control information satisfies a preset condition. By limiting the voice control information received by the mobile terminal, and establishing a control relationship between the user intention and the mobile terminal under the premise that the voice control information satisfies the preset condition, the voice control information which cannot represent the user intention can be excluded, so that the probability of voice misoperation can be reduced when the mobile terminal is subjected to voice control.

Description

Interactive method, mobile terminal and readable storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 201911216873.5, and the invention title is "Interactive Method, Mobile Terminal, and Readable Storage Medium" on November 29, 2019, the entire content of which is incorporated by reference Applying.

Technical field

This application relates to the field of communication technology, and in particular to an interaction method, a mobile terminal and a readable storage medium.

Background technique

With the development of voice control technology, voice control technology is increasingly adopted in mobile terminals that are now widely used to make the operation of the mobile terminal more convenient. For example, voice control technology involves various fields such as home appliances, communications, automotive electronics, medical treatment, and consumer electronics. That is, using voice to control the operation of equipment is faster and more convenient than manual control.

However, when voice control is performed on a mobile terminal, it may be caused by receiving multiple external devices connected to the mobile terminal to perform voice control on it, or the mobile terminal may be disturbed by noise in the surrounding environment, which may increase the problem of voice misoperation.

Summary of the invention

technical problem

The solution to the problem

Technical solutions

The main purpose of this application is to provide an interaction method, a mobile terminal, and a readable storage medium, which aims to solve the technical problem of increasing voice misoperation when voice control is performed on the mobile terminal.

In order to achieve the above objective, this application provides an interaction method, which includes:

S10. Acquire voice control information;

S20: When the voice control information meets a preset condition, output prompt information.

Further, the step that the voice control information satisfies a preset condition includes:

Acquiring the operation information that triggers the reception of the voice control information, and/or the device information that receives the voice control information;

Determine whether the device information and/or the operation information meets preset rules;

If yes, the voice control information meets the preset condition.

Further, the judging whether the device information and/or the operation information meets a preset rule includes at least one of the following:

Whether the device information is the mobile terminal and/or an external device connected to the mobile terminal;

Whether the operation information is based on the mobile terminal and/or the operation information of an external device connected to the mobile terminal;

Whether the operation information complies with preset operation information.

Obtain environmental information;

Judging whether the environmental information conforms to preset environmental information;

If not, the voice control information meets the preset condition.

Further, the environmental information includes: at least one of environmental noise, location information, and movement information;

The step of judging whether the environmental information conforms to preset environmental information includes:

Whether the environmental noise reaches the preset environmental noise; and/or,

Whether the location information is located in a public place; and/or,

Whether the movement information reaches a preset movement speed.

Acquiring state information of the mobile terminal;

Judging whether the state information meets the preset state information;

If yes, the voice control information meets the preset condition.

Further, the status information includes: operating mode and/or communication status;

The judging whether the state information meets preset state information includes:

Judging whether the operating mode is a preset operating mode; and/or,

It is determined whether the communication state is a preset communication state.

Further, the step of outputting prompt information includes:

Output the prompt information through an interface display and/or voice prompt;

According to the received feedback operation information, perform corresponding voice control.

The present application also provides a mobile terminal. The mobile terminal includes a memory, a processor, and a computer program stored on the memory and running on the processor. When the computer program is executed by the processor, Implement the steps of the interactive method as described above.

The present application also provides a readable storage medium, characterized in that a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the steps of the interaction method as described above are realized.

In the interaction method proposed in the embodiment of the present application, by acquiring voice control information, when the voice control information meets a preset condition, prompt information is output. In this way, the voice control information of the mobile terminal is restricted, so that the voice control information controls the setting of the mobile terminal to meet the preset conditions, and establishes the control relationship between the user's intention and the mobile terminal, so that the mobile terminal can judge by the set preset conditions User intention can exclude voice control information that does not meet preset conditions, and eliminate voice control information that does not represent the user's intention, so that when voice control is performed on the mobile terminal, the probability of voice misoperation can be reduced.

The beneficial effects of the invention

Brief description of the drawings

Description of the drawings

FIG. 1 is a schematic structural diagram of a mobile terminal with hardware operation involved in a solution of an embodiment of the present application;

FIG. 2 is a schematic flowchart of a first embodiment of an interaction method according to the present application;

Fig. 3 is a schematic diagram of the application based on the interaction between the wireless headset and the mobile terminal;

Figure 4 is a schematic diagram of the interaction between the user of this application and the mobile terminal;

FIG. 5 is a schematic diagram of the connection of multiple earphones with a mobile terminal according to the present application;

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

As shown in Fig. 1, Fig. 1 is a schematic structural diagram of a mobile terminal in a hardware operating environment involved in a solution of an embodiment of the present application.

The mobile terminal in the embodiment of this application can be a PC, a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III, dynamic image expert compression standard audio layer 3) player, and a MP4 (Moving Picture Experts) player. Experts Group Audio Layer IV, the dynamic image expert compresses the standard audio layer 3) Players, portable computers and other portable terminal devices with display functions.

As shown in FIG. 1, the mobile terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Optionally, the mobile terminal may also include a camera, an RF (Radio Frequency, radio frequency) circuit, a sensor, an audio circuit, a WiFi module, and so on. Among them, sensors such as light sensors, motion sensors and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor can turn off the display screen and/or when the mobile terminal is moved to the ear. Backlight. As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in various directions (usually three-axis), and can detect the magnitude and direction of gravity when it is stationary. It can be used to identify the application of the mobile terminal's posture (such as horizontal and vertical screen switching, Related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, percussion), etc.; of course, the mobile terminal can also be equipped with other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc. No longer.

Those skilled in the art can understand that the structure of the mobile terminal shown in FIG. 1 does not constitute a limitation on the mobile terminal, and may include more or less components than shown in the figure, or combine certain components, or different component arrangements.

As shown in FIG. 1, a memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a computer program.

In the mobile terminal shown in FIG. 1, the network interface 1004 is mainly used to connect to the back-end server and communicate with the back-end server; the user interface 1003 is mainly used to connect to the client (user side) and communicate with the client; The device 1001 can be used to call a computer program stored in the memory 1005 and perform the following operations:

S10. Acquire voice control information;

Further, obtaining the operation information that triggers the reception of the voice control information, and/or the device information that receives the voice control information;

If yes, the voice control information meets the preset condition.

Further, whether the device information is the mobile terminal and/or an external device connected to the mobile terminal;

Whether the operation information complies with preset operation information.

Further, obtain environmental information;

If not, the voice control information meets the preset condition.

Further, whether the environmental noise reaches a preset environmental noise; and/or,

Whether the location information is located in a public place; and/or,

Whether the movement information reaches a preset movement speed.

Further, acquiring state information of the mobile terminal;

Judging whether the state information meets the preset state information;

If yes, the voice control information meets the preset condition.

Further, the judging whether the state information satisfies preset state information includes:

Judging whether the operating mode is a preset operating mode; and/or,

Further, output the prompt information through interface display and/or voice prompt;

2, this application proposes various embodiments of the method of this application based on the above-mentioned mobile terminal hardware structure.

This application provides an interaction method, which is applied to a mobile terminal. In the first embodiment of the interaction method, referring to FIG. 2, the method includes:

Step S10, acquiring voice control information;

The mobile terminal can obtain the user's voice control information in real time or regularly. Among them, mobile terminals include devices such as PCs, smart phones, and tablet computers. The voice control information may be a control command of the user to the mobile terminal. In this embodiment, the mobile terminal can directly obtain the user's voice control information, parse the voice control information to obtain the voice control text, and perform corresponding operations or output corresponding prompt information according to the control text. For example, if the mobile terminal obtains that the user's voice control instruction is "please open Alipay", the mobile terminal may perform a corresponding operation or output prompt information according to the voice control instruction.

In some embodiments, the mobile terminal may also be connected with other external devices, and receive a user's voice control instruction based on the other external device, and perform corresponding operations or output prompt information according to the voice control instruction. Among them, the external devices may include wireless earphones, smart phones, smart watches, smart glasses, and other mobile devices. For example, when the user is driving, the wireless headset is connected to the mobile terminal, and the mobile terminal receives the voice control command "please navigate to Shenzhen North Station" based on the wireless headset, and the mobile terminal will output and display the prompt information. The route is output and displayed.

To assist in understanding the technical solution of this embodiment, refer to FIG. 3, which is a schematic diagram based on the interaction between the wireless headset and the mobile terminal.

Step S20: When the voice control information meets the preset condition, output prompt information.

The preset condition here may be the internal setting of the mobile terminal or the user's own setting. Only the voice control information that meets the preset condition can trigger the mobile terminal to respond, output prompt information or perform corresponding operations. In addition, the preset conditions include but are not limited to the following situations: whether the device information and/or operation information meets the preset rules, whether the environmental information meets the preset environmental information, or whether the state information meets the preset state information . The user’s voice control information controls the mobile terminal to perform corresponding operations or output prompt information to meet the aforementioned preset conditions. For example, the preset condition is that the environmental noise in the environmental information is higher than the preset environmental noise, and the mobile terminal obtains it from the current environmental information The voice control information may be inaccurate, that is, because the acquired voice control information is inaccurate due to high environmental noise, the mobile terminal does not perform corresponding operations or output prompt information.

In this embodiment, the mobile terminal obtains the voice control information, and then parses the voice control information to obtain text control instructions, extract keywords from the text control instructions, and determine whether the keywords match the preset keywords. If the keyword matches the preset keyword, the application associated with the preset keyword is executed, or prompt information related to the preset keyword is output, if the keyword does not match the preset keyword, then Do nothing. Among them, an association relationship between keywords and applications is established in advance. For example, if the text control instruction is "open WeChat Moments", the mobile terminal extracts the keywords "WeChat" and "Moments" and matches the preset keywords, then opens WeChat Moments.

Optionally, the step of outputting prompt information includes:

Step 21: Output prompt information through interface display and/or voice prompt;

Step 22: Perform corresponding voice control according to the received feedback operation information.

When the mobile terminal determines that the voice control information meets the preset condition, it may output prompt information through interface display and/or voice prompts, and perform corresponding voice control according to the received feedback operation information. For example, when the user opens the phone operation interface of the mobile terminal, and at this time, the user wants to switch to the memo function, the user’s voice control information is "Please open the memo", and the mobile terminal receives the voice control information and gives feedback. And turn on the memo function and display it.

To assist in understanding the technical solution of this embodiment, refer to FIG. 4, which is a schematic diagram of the interaction between the user and the mobile terminal.

In this embodiment, by acquiring the voice control information, when the voice control information meets a preset condition, prompt information is output. In this way, the voice control information of the mobile terminal is restricted, so that the voice control information controls the setting of the mobile terminal to meet the preset conditions, and establishes the control relationship between the user's intention and the mobile terminal, so that the mobile terminal can judge by the set preset conditions User intention can exclude voice control information that does not meet preset conditions, and eliminate voice control information that does not represent the user's intention, so that when voice control is performed on the mobile terminal, the probability of voice misoperation can be reduced.

Further, in step S20 of the above-mentioned first embodiment, the step that the voice control information satisfies a preset condition includes:

Step S201: Obtain the operation information that triggers the reception of voice control information, and/or the information of the device that receives the voice control information;

Step S202: Determine whether the device information and/or operation information meets preset rules;

Step S203, if yes, the voice control information meets the preset condition.

The mobile terminal obtains the operation information that triggers the reception of voice control information and/or the device information that receives the voice control information, and determines whether the device information and/or the operation information meets the preset rules; if so, the voice control information meets the preset conditions. Wherein, the device information includes the identification of the device that receives the voice control information. The preset rule here may be an internal setting of the mobile terminal or a preset rule set by the user. Only device information and operation information that meet the preset rule can trigger a response from the mobile terminal and output prompt information. In addition, the preset rules include, but are not limited to, the following situations: it can be judging whether the device information meets the preset device information, or it can be judging whether the operation information meets the preset operation information. If the preset condition is to press the volume key twice to trigger the mobile terminal to receive voice control information, the user presses the volume key twice to trigger the mobile terminal to receive the operation information of the voice control information to output prompt information.

By restricting the operation information that triggers the reception of voice control information and/or the device information that receives the voice control information, so that the device information and the operation information meet the preset rules, the mobile terminal can be triggered to receive the voice control information to output prompt information. In this way, voice control information that cannot represent the user's intention can be eliminated, and the probability of false triggering of the mobile terminal by other external devices can also be reduced.

Optionally, step S202 may specifically include:

Step S2021, whether the device information is a mobile terminal and/or an external device connected to the mobile terminal;

Step S2022, whether the operation information is based on the mobile terminal and/or the operation information of the external device connected to the mobile terminal;

Step S2023, whether the operation information conforms to the preset operation information.

Whether the device information is a mobile terminal and/or an external device connected to the mobile terminal, whether the operation information is based on the mobile terminal, and/or the operation information of an external device connected to the mobile terminal; whether the operation information complies with preset operation information. Among them, the external devices may include wireless earphones, smart watches, smart bracelets, smart glasses, and other mobile devices. The operation information may be voice command information. It can also be gesture operation information, or track operation information and operation information of pressing the volume key and/or power key. It should be noted that there is no specific limitation here, and the device information can be the identification of the external device connected to the mobile terminal information. Specifically, the mobile terminal obtains the identification of the wireless headset, and determines whether the identification of the wireless headset exists in the preset authority list, and if the identification of the wireless headset exists in the preset authority list, it establishes a communication connection with the search wireless headset. In this embodiment, the mobile terminal can send wireless detection signals through the wireless communication module, search for wireless, and have a one-to-one correspondence with the mobile terminal. The mobile terminal has a radio module that can send and receive radio signals, which can be matched with the radio signals sent and received by the wireless headset. The detection signal can detect the radio signal sent by the wireless headset, and the wireless communication module of the mobile terminal determines the identity of the wireless headset according to the detected radio signal. After the wireless communication module of the mobile terminal establishes a communication connection with the wireless headset, it can receive the information sent by the wireless headset, and at this time, obtain the identification of the wireless headset.

The preset operation information here restricts device information and operation information, and only when the preset operation information is satisfied can the mobile terminal be triggered to receive voice control information. If the preset operation information is set to be based on the wireless headset 1 connected to the mobile terminal, if the mobile terminal is connected to the wireless headset 1 and the wireless headset 2, only the wireless headset 1 can trigger the mobile terminal to receive voice control information to output prompts information. In this way, voice control of the mobile terminal can be avoided when multiple devices are connected, and the problem of misoperation of the mobile terminal can be avoided.

To assist in understanding the technical solution of this embodiment, refer to FIG. 5, which is a schematic diagram of multiple headsets connected to a mobile terminal.

Step A, obtain environmental information;

Step B, judging whether the environmental information meets the preset environmental information;

Step C, if not, the voice control information meets the preset condition.

Wherein, the environmental information includes: at least one of environmental noise, location information, and movement information. In this embodiment, the mobile terminal can receive environmental information based on the wireless headset, and determine whether at least one of the environmental noise, location information, and movement information in the environmental information meets the preset environmental information. If not, the voice control information meets the preset environmental information. Set the condition, if yes, the voice control information meets the preset condition. The preset environment information here may be set by the internal setting program of the mobile terminal, or set by the user. Only when the environment information does not meet the preset environment information, the mobile terminal can output prompt information according to voice control information.

For example, the mobile terminal receives voice control information based on wireless earphones, and obtains environmental information. If the environmental noise in the environmental information does not meet the preset environmental noise, the audio data of the voice control information will be denoised, and the audio data of the voice control information will be denoised. Voiceprint extraction is performed on the audio data after noise processing, and the text control instructions of the voice control information are obtained through voice recognition to determine whether the voiceprint information of the voice control information matches the preset voiceprint information. If the voiceprint information of the voice control information When it matches the preset voiceprint information, a corresponding response operation is made or prompt information is output according to the text control information.

Optionally, step B may specifically include the following:

Step B1, whether the environmental noise reaches the preset environmental noise; and/or,

Step B2, whether the location information is located in a public place; and/or,

Step B3, whether the movement information reaches the preset movement speed.

The mobile terminal can determine whether the environmental noise reaches the preset environmental noise; and/or, determine whether the location information is located in the public place information; and/or, determine whether the mobile information reaches the preset moving speed. The preset ambient noise and preset moving speed here can be set for the internal program of the mobile terminal, and can also be set by the user. For example, the current location information obtained by the mobile terminal is a shopping mall, that is, it is a public place, and there are too many people around, the sound is noisy, the mobile terminal does not receive other voices for operation, to avoid the problem of misoperation of the mobile terminal receiving voice.

Specifically, obtain multiple sample environmental information containing noise, use the sample environmental information as a training set, and construct a noise recognition model based on deep neural network training; input the environmental information into the noise recognition model to determine whether the environmental information contains the environment Noise, if it is determined that the environmental information includes environmental noise, if it is determined that the environmental information includes environmental noise, it is determined whether the environmental noise reaches the preset environmental noise. In this way, the mobile terminal can avoid the problem of real-time judgment of environmental noise when the mute is acquired, which causes the mobile terminal to waste electricity, thereby prolonging the life of the mobile terminal.

Step D: Obtain the state information of the mobile terminal, and determine whether the state information meets the preset state information;

Step E, judging whether the state information meets the preset state information;

Step F, if yes, the voice control information meets the preset condition.

In this embodiment, the state information of the mobile terminal is acquired, and it is determined whether the state information meets the preset state information. If so, the voice control information meets the preset condition, and if not, the voice control information does not meet the preset condition. Among them, the status information includes: operating mode and/or communication status. The preset state information here may be the internal program setting of the mobile terminal or the user sets the preset state information according to his own needs, and only when the preset state information is satisfied can the received voice control information output prompts. The preset operating models include voice control mode, driving mode, and child mode. For example, when the user turns on the driving mode on the mobile terminal, and receives the user saying "Xiaoyou, please turn me on the camera", the mobile terminal performs the corresponding operation. The preset communication state is not a call state, which includes incoming and outgoing calls without making calls, video calls, and voice calls. For example, when the user is in a voice call state, and the user says "you and Xiaoming are calling", the mobile terminal will not perform corresponding operations. In this way, it can be avoided that the user erroneously triggers the mobile terminal to perform the corresponding operation during the conversation with others.

In addition, the embodiment of the present application also proposes a readable storage medium (ie, a computer readable memory), and the readable storage medium may be a non-volatile readable storage medium.

A computer program is stored on the readable storage medium of the present application, and when the computer program is executed by a processor, the following operations are realized:

S10. Acquire voice control information;

Further, receiving environmental voice data collected based on the wireless headset, and determining whether the environmental voice data includes human voice data;

If the environmental voice data includes the human voice data, the environmental voice data is used as the target voice data.

Whether the operation information complies with preset operation information.

Further, obtain environmental information;

If not, the voice control information meets the preset condition.

Whether the location information is located in a public place; and/or,

Whether the movement information reaches a preset movement speed.

Further, acquiring state information of the mobile terminal;

Judging whether the state information meets the preset state information;

If yes, the voice control information meets the preset condition.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, method, article, or system. Without more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or system that includes the element.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes a number of instructions to enable a mobile terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

An interaction method applied to a mobile terminal, wherein the method includes:

S10. Acquire voice control information;

S20: When the voice control information meets a preset condition, output prompt information.
The method according to claim 1, wherein the step of the voice control information satisfying a preset condition comprises:

Acquiring the operation information that triggers the reception of the voice control information, and/or the device information that receives the voice control information;

Determine whether the device information and/or the operation information meets preset rules;

If yes, the voice control information meets the preset condition.
The method according to claim 2, wherein said determining whether said device information and/or said operation information meets a preset rule comprises at least one of the following:

Whether the device information is the mobile terminal and/or an external device connected to the mobile terminal;

Whether the operation information is based on the mobile terminal and/or the operation information of an external device connected to the mobile terminal;

Whether the operation information complies with preset operation information.
The method according to claim 1, wherein the step of the voice control information satisfying a preset condition comprises:

Obtain environmental information;

Judging whether the environmental information conforms to preset environmental information;

If not, the voice control information meets the preset condition.
The method according to claim 4, wherein the environmental information includes at least one of environmental noise, location information, and movement information;

The step of judging whether the environmental information conforms to preset environmental information includes:

Whether the environmental noise reaches the preset environmental noise; and/or,

Whether the location information is located in a public place; and/or,

Whether the movement information reaches a preset movement speed.
The method according to claim 1, wherein the step of the voice control information satisfying a preset condition comprises:

Acquiring state information of the mobile terminal;

Judging whether the state information meets the preset state information;

If yes, the voice control information meets the preset condition.
8. The method of claim 6, wherein the status information includes: operating mode and/or communication status;

The judging whether the state information meets preset state information includes:

Judging whether the operating mode is a preset operating mode; and/or,

It is determined whether the communication state is a preset communication state.
The method according to claim 1, wherein the step of outputting prompt information comprises:

Output the prompt information through an interface display and/or voice prompt;

According to the received feedback operation information, perform corresponding voice control.
A mobile terminal, wherein the mobile terminal includes: a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the The following steps:

S10. Acquire voice control information;

S20: When the voice control information meets a preset condition, output prompt information.
A readable storage medium, wherein a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the following steps are implemented:

S10. Acquire voice control information;

S20: When the voice control information meets a preset condition, output prompt information.