CN109901698B - Intelligent interaction method, wearable device, terminal and system - Google Patents
Intelligent interaction method, wearable device, terminal and system Download PDFInfo
- Publication number
- CN109901698B CN109901698B CN201711294122.6A CN201711294122A CN109901698B CN 109901698 B CN109901698 B CN 109901698B CN 201711294122 A CN201711294122 A CN 201711294122A CN 109901698 B CN109901698 B CN 109901698B
- Authority
- CN
- China
- Prior art keywords
- voice
- user
- intelligent hardware
- wearable device
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The embodiment of the invention discloses an intelligent interaction method, wearable equipment, a terminal and a system, which are used for solving the problem that the range of intelligent interaction is limited and do not limit the moving range of a user. The embodiment of the invention provides an intelligent interaction method, which comprises the following steps: the wearable device acquires an input signal of a user; the wearable device determines that voice wakeup is required according to the input signal; the wearable device collects voice signals when voice wakeup is determined; the wearable device transmits the voice signal to the intelligent hardware body, and the intelligent hardware body performs voice instruction recognition on the voice signal.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an intelligent interaction method, a wearable device, a terminal, and a system.
Background
At present, intelligent terminals are increasingly popular, and users can realize intelligent interaction based on the intelligent terminals. Such as the user interacting directly with the intelligent terminal.
In the prior art, a user can directly speak, then the intelligent terminal carries out voice recognition to corresponding instructions, and finally responds to the instructions, so that intelligent interaction can be completed. The prior art has at least the following defects: due to the limitations of sound propagation, speech recognition places high demands on the distance of the recognition body from the sounding body, which must be very close and not obstructed by obstacles. Thus, the prior art defines the coverage and flexibility of intelligent interactions.
Disclosure of Invention
The embodiment of the invention provides an intelligent interaction method, wearable equipment, a terminal and a system, which are used for solving the problem that the range of intelligent interaction is limited and do not limit the moving range of a user.
In order to solve the technical problems, the embodiment of the invention provides the following technical scheme:
in a first aspect, an embodiment of the present invention provides an intelligent interaction method, including:
the wearable device acquires an input signal of a user;
the wearable device determines that voice wakeup is required according to the input signal;
the wearable device collects voice signals when voice wakeup is determined;
the wearable device transmits the voice signal to the intelligent hardware body, and the intelligent hardware body performs voice instruction recognition on the voice signal.
In a second aspect, an embodiment of the present invention further provides a wearable device, including:
the signal input module is used for acquiring an input signal of a user;
the voice wake-up module is used for determining that voice wake-up is required according to the input signal;
the signal acquisition module is used for acquiring voice signals when voice wakeup is determined;
and the voice transmission module is used for transmitting the voice signal to the intelligent hardware body.
In a third aspect, embodiments of the present application provide a computer-readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform the method of the above aspects.
In a fourth aspect, an embodiment of the present application provides a terminal, where the terminal may include an entity such as a mobile terminal, an intelligent terminal, a wearable device, or a chip, and the terminal includes: a processor, a memory; the memory is used for storing instructions; the processor is configured to execute the instructions in the memory to cause the terminal to perform the method according to any one of the preceding first aspects.
In a fifth aspect, embodiments of the present application provide an intelligent interaction system, the system comprising: the wearable device, the smart hardware body of any of the second aspect of the preceding claims, wherein,
the intelligent hardware body is used for carrying out voice instruction recognition on voice signals sent by the wearable equipment.
From the above technical solutions, the embodiment of the present invention has the following advantages:
in the embodiment of the invention, the wearable device firstly acquires the input signal of the user, then the wearable device determines that voice wakeup is required according to the input signal, the wearable device acquires the voice signal when determining to perform voice wakeup, and finally the wearable device transmits the voice signal to the intelligent hardware body, and the intelligent hardware body performs voice instruction recognition on the voice signal. Because the wearable equipment keeps very close distance with the user at any time, real-time voice awakening can be performed according to the voice of the user in time through the wearable equipment, the wearable equipment can also transmit collected voice signals to the intelligent hardware body in real time after voice awakening to complete the whole intelligent interaction, and therefore the interaction between the user and the intelligent hardware body is not limited by the distance range, the problem that the intelligent interaction range is limited is solved, and the moving range of the user is not limited.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings for those skilled in the art.
FIG. 1 is a schematic diagram of a system architecture of an intelligent interaction method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of another system application scenario of the intelligent interaction method according to the embodiment of the present invention;
FIG. 3 is a schematic block diagram of a process of an intelligent interaction method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an interaction flow between a wearable device and an intelligent hardware entity according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a transmission flow of a voice signal according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of an application scenario of intelligent interaction according to an embodiment of the present invention;
fig. 7-a is a schematic diagram of a composition structure of a wearable device according to an embodiment of the present invention;
fig. 7-b is a schematic diagram of a composition structure of a voice transmission module according to an embodiment of the present invention;
fig. 7-c is a schematic diagram of a composition structure of another voice transmission module according to an embodiment of the present invention;
Fig. 7-d is a schematic diagram of a composition structure of a voice wake-up module according to an embodiment of the present invention;
fig. 7-e is a schematic diagram of a composition structure of another wearable device according to an embodiment of the present invention;
fig. 7-f is a schematic structural diagram of another wearable device according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a composition structure of an intelligent interaction method applied to a terminal according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a composition structure of an intelligent interaction system according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides an intelligent interaction method, wearable equipment, a terminal and a system, which are used for solving the problem that the range of intelligent interaction is limited and do not limit the moving range of a user.
In order to make the objects, features and advantages of the present invention more comprehensible, the technical solutions in the embodiments of the present invention will be clearly described in conjunction with the accompanying drawings, and it is apparent that the embodiments described below are only some embodiments of the present invention, but not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention fall within the scope of protection of the present invention.
The terms "comprises" and "comprising," along with any variations thereof, in the description and claims of the invention and in the foregoing drawings, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The following will describe in detail.
As shown in fig. 1 and fig. 2, a schematic diagram of a system architecture of the intelligent interaction method is shown. The embodiment of the intelligent interaction method can be particularly applied to wearable equipment, wherein the wearable equipment can be particularly equipment such as intelligent watches, intelligent glasses, intelligent bracelets, intelligent shoes and the like, and the wearable equipment is not limited herein. The wearable device may communicate with the smart hardware entity over a network, such as near field transmission by the wearable device and the smart hardware entity. Or the wearable device may communicate over a network with a remote server that may communicate with the smart hardware entity, such as a wide area transmission of the wearable device and the smart hardware entity. The intelligent hardware body can be an intelligent terminal, such as an intelligent sound box or intelligent household equipment. In the embodiment of the invention, the wearable device generally follows the user, so that the wearable device can acquire the input signals from the user in real time, and the user can freely move without being limited by the distance range between the intelligent body and the user. According to the method, the coverage range and flexibility of intelligent interaction can be greatly expanded, the wearable device can keep very close distance with the user at any time, the voice of the user can be conveniently and timely awakened in real time, and the voice is transmitted to the intelligent hardware in real time to complete the whole intelligent interaction.
Referring to fig. 3, the intelligent interaction method provided by the embodiment of the present invention is introduced from the wearable device side, and may include the following steps:
101. the wearable device obtains an input signal of a user.
In the embodiment of the invention, the wearable device is configured on the body of the user, and the wearable device can move along with the user in real time, so that the wearable device can acquire the input signal of the user, thereby capturing the intention of the user in real time. The input signal collected by the wearable device from the user may be a sensor signal, such as an action performed by the user, or somatosensory information generated by the user, and the like. The input signal collected by the wearable device may also be a signal input into the wearable device by the user.
For example, as follows, the wearable device generally has a monitoring function, and for example, motion information and health information of a user can be detected through a sensor of the smart watch, and then the motion information and the health information can be used as input signals.
It should be noted that, in the embodiment of the present invention, the input signal of the user may not be limited to the signal directly input to the wearable device by the user, and the input signal may also be a user signal actively collected by the wearable device, such as a limb action of the user.
102. The wearable device determines that voice wakeup is required according to the input signal.
In the embodiment of the invention, after the wearable device acquires the input signal of the user, the wearable user can identify the current intention of the user based on the input signal, and in the embodiment of the invention, the wearable device judges whether the user has the intention of generating the voice command or not through the input signal of the user, and determines whether to wake up the voice of the wearable device or not based on whether the user wakes up the voice. If the wearable device recognizes that the user needs to send out a voice command through the input signal, the wearable device can wake up by voice, for example, a microphone of the wearable device is started.
In some embodiments of the present invention, step 102 the wearable device determines that voice wakeup is required according to an input signal, including:
the wearable device determines a user state according to the input signal;
the wearable equipment acquires processing logic corresponding to the user state according to the preset corresponding relation between the state and the processing logic;
the wearable device determines whether the user wakes up by voice according to the acquired processing logic.
Wherein the wearable device recognizes an input signal of the user, e.g. the input signal may be a sensor signal. The wearable device determines the current state of the user according to the identified input signal, acquires processing logic corresponding to the state of the user according to the corresponding relation between the state and the processing logic, and stores the corresponding relation between the state and the processing logic in the wearable device in advance in a processing logic list mode, and the wearable device can look up a table to determine whether the acquired processing logic determines that the user performs voice awakening, for example, the state of the user is a wrist lifting action, and the processing logic should perform voice awakening, for example, a microphone is turned on and then waits for receiving a voice instruction sent by the user.
Further, in some embodiments of the present invention, in addition to executing the foregoing steps, the intelligent interaction method provided by the embodiment of the present invention may further include the following steps:
the wearable device dynamically learns and revises the state and processing logic correspondence using a reinforcement learning (Reinforcement learning) model.
Wherein the wearable device can determine from the sensor signal that the user is speaking the voice command intent. According to the preset corresponding relation between the state and the processing logic, the wearable device can determine the identification result matching the current state of the user, and make corresponding processing logic according to the processing logic corresponding table. It should be noted that, the training and the sample of the reinforcement learning model can be completed based on a time difference learning (Q learning) algorithm, Q in the Q learning is an action utility function, and meanwhile, the corresponding relationship between the state and the processing logic can be dynamically expanded and corrected according to the use habit of the user by using the strategy of the Reinforcement learning model, so that the model can better respond to the user requirement. Whether the user has an intention to issue a voice instruction is detected.
It should be noted that in order to avoid misoperation in the prior art, a wake-up word is often required to be designated during voice wake-up, that is, voice recognition can be started only after a user speaks the designated wake-up word, and the solution is not simple, convenient and intuitive, and increases the learning cost of the user. The embodiment of the invention can adopt the corresponding relation between the state and the processing logic, only the input signal of the user is needed to be identified for voice awakening, the voice awakening is not perceived by the user, and the voice of the user can be identified as a user instruction without deliberately speaking a certain awakening word. In the embodiment of the invention, the wearable device can obtain the intention of the user to send the voice instruction by detecting the sensor signal triggered by the user. The embodiment of the invention can detect the intention of using the main body to send the voice instruction, and the detection can be based on no voice, thereby realizing intelligent hardware awakening without awakening words.
103. The wearable device collects voice signals when voice wakeup is determined.
In the embodiment of the present invention, after determining that the voice wake-up is performed in the foregoing step 102, the wearable device may use the microphone to collect a voice signal from the surrounding environment of the wearable device, where the voice signal is a sound signal collected by the wearable device.
104. The wearable device transmits voice signals to the intelligent hardware body, and the intelligent hardware body performs voice instruction recognition on the voice signals.
In the embodiment of the invention, after the wearable device obtains the voice signal, the wearable device transmits the voice signal to the intelligent hardware body, and the intelligent hardware body performs voice instruction recognition on the voice signal, wherein after recognizing the voice instruction of the user, the intelligent hardware body can execute the voice instruction or perform question-answer interaction according to the voice instruction.
It should be noted that, in the embodiment of the present invention, the wearable device is limited by the device performance or the battery power, and the wearable device does not further analyze the collected voice signal, but sends the voice signal to the intelligent hardware body, and the intelligent hardware body performs voice recognition on the voice signal. The wearable device is thus implemented in a miniaturized manner, which can follow the user himself for real-time movements.
In some embodiments of the present invention, in addition to executing the foregoing steps, the intelligent interaction method provided by the embodiment of the present invention may further include the following steps:
the wearable device sends the signal transmission result to the user.
After the wearable device performs the step 104, the wearable device may further inform the user of the signal transmission result in a screen prompt, voice prompt or vibration manner, so that the user may know whether the voice command is issued successfully.
In some embodiments of the present invention, in addition to executing the foregoing steps, the intelligent interaction method provided by the embodiment of the present invention may further include the following steps: the wearable device ends the voice acquisition function, e.g. the wearable device may turn off its own sensor. From the energy saving point of view, after the interaction is completed, the sensor (for example, the microphone) with higher energy consumption is turned off, so that the electric energy of the wearable device is saved.
In some embodiments of the present invention, step 104 the wearable device transmits a voice signal to the smart hardware entity, comprising:
the wearable equipment is matched with the intelligent hardware body in a short distance;
if the wearable device and the intelligent hardware are successfully matched in a short distance, the wearable device sends a voice signal to the intelligent hardware body.
The wearable device may first search whether there is an already paired smart hardware body in a nearby location through a short-range wireless communication protocol, and if there is an already paired device, directly enter a near-field transmission step. If not, an attempt is made to search for the presence of a smart hardware entity in the vicinity that can be paired and send personal credentials of the wearable device for automatic pairing. After the pairing is successful through a short distance, the wearable equipment directly transmits the received voice signals to the intelligent hardware body through the corresponding coordination. In other embodiments of the present invention, the wearable device may further perform encryption processing and compression processing on the voice signal before the voice signal is transmitted, so as to ensure reliable transmission of data and improve data transmission efficiency.
In other embodiments of the present invention, step 104 the wearable device transmits a voice signal to the smart hardware entity, comprising:
the wearable device obtains user credentials of a user;
the wearable device sends the user credentials and the voice signals to a remote server, which forwards the user credentials and the voice signals to the intelligent hardware entity.
Wherein the wearable device may transmit the voice signal over a network, for example the wearable device may transmit the user credentials and corresponding voice signal to a remote server using a wide area network. In other embodiments of the present invention, the wearable device may also asymmetrically encrypt the user credentials and voice signals, then transmit to the remote server, then decrypt and identify the voice signals by the remote server using the user public key, and then the remote server may forward the user credentials and voice signals to the intelligent hardware entity that the user has pre-registered.
As can be seen from the description of the embodiments of the present invention by the above embodiments, the wearable device first obtains an input signal of a user, then determines that voice wakeup is required according to the input signal, the wearable device collects a voice signal when determining to perform voice wakeup, and finally the wearable device transmits the voice signal to the intelligent hardware body, and the intelligent hardware body performs voice instruction recognition on the voice signal. Because the wearable equipment keeps very close distance with the user at any time, real-time voice awakening can be performed according to the voice of the user in time through the wearable equipment, the wearable equipment can also transmit collected voice signals to the intelligent hardware body in real time after voice awakening to complete the whole intelligent interaction, and therefore the interaction between the user and the intelligent hardware body is not limited by the distance range, the problem that the intelligent interaction range is limited is solved, and the moving range of the user is not limited.
In order to better understand and implement the above-mentioned schemes of the embodiments of the present invention, the following specific description will exemplify the corresponding application scenario.
The embodiment of the invention provides an intelligent interaction method based on wearable equipment, which can be applied to an intelligent robot system to realize man-machine natural interaction chat. The technology can enable the wearable equipment to finish voice awakening and transmit specific voice signals to the intelligent hardware body on the basis of collected sensor information and user voice signals, and the intelligent hardware body performs voice instruction recognition and instruction control to finally finish the whole intelligent interaction.
As shown in fig. 4, the interaction process between the wearable device and the smart hardware body is first described in detail, and taking the smart hardware body as a smart speaker, voice wake-up and voice signal transmission of the wearable device are first described.
The wearable device may accept input signals from the wearable device to determine whether user voice input is received.
Wherein the input signal is a source of parameters controlling the intelligent hardware entity. In general, it is possible to determine whether or not the sensor signal is wake-up voice according to the type of the input signal. Other ways of input signals, such as gestures, acceleration, body temperature, heart rate, etc. of the user are not excluded in the embodiments of the present invention, as long as the input signals can correctly express the control intention of the user, i.e. the input scope of the present scheme is met.
Since wearable devices generally have a monitoring function, for example, motion information and health information of a user can be detected through a sensor of a smart watch. In the scheme, the wearable device first obtains the intention of a user to send a voice instruction by detecting a sensor signal triggered by the user. For example, according to a preset user state and a processing logic corresponding table, the wearable device may determine an identification result matching the current state of the user, and make corresponding processing logic according to the processing logic corresponding table. For example, when the wearable device detects an action of the user lifting the wrist, it may be determined that the user has a hope of giving a voice instruction. Meanwhile, the corresponding table of the user state and the processing logic can dynamically expand and correct according to the use habit of the user by utilizing the strategy of the reinforced learning model, so that the user can respond to the user requirement better. After ascertaining the intent of the voice instruction issued by the user subject, the wearable device records the voice signal issued by the user. Ready for further transmission.
The voice signal transmission of the wearable device is described next. As shown in fig. 5, after receiving a voice command from a user, a voice signal may be transmitted to the smart speaker. Take the example of dynamic protocol switching of a wearable device. First, by short-range wireless communication protocol including but not limited to near field communication (Near Field Communication, NFC), zigBee, bluetooth, etc., search for whether there is already paired smart speakers in nearby locations, and if there is already paired device, enter near field transmission directly. If the pairing is not carried out, the intelligent loudspeaker box capable of being paired is tried to search whether the nearby positions exist or not, the personal certificate is sent to carry out automatic pairing, if the pairing is successful, near-field transmission is carried out, and if the pairing is failed, the wide-area transmission mode is carried out. After the voice signals are successfully paired in a short distance, the wearable equipment directly transmits the received voice signals to the intelligent sound box through a corresponding protocol, and before transmission, the wearable equipment can compress and encrypt the voice signals.
When the short-range matching does not search for the corresponding smart speaker, the wearable device may attempt to transmit the user credential and the corresponding voice signal to the remote server through a relay means, such as a smart phone, using a wide area network after asymmetrically encrypting, then decrypt and recognize the voice signal through the remote server using the user public key, and then send the voice signal to the smart speaker registered in advance by the user.
After the above process is completed, the wearable device may also inform the user of the transmission result by means of a screen prompt, a voice prompt or vibration.
The instruction recognition of the intelligent hardware body is mainly implemented in an end-to-end training (end-to-end training) mode, an input end (voice waveform or feature sequence) is directly connected with an output end (word or character sequence) through a neural network, a Deep Learning (Deep Learning) technology is utilized, instructions are taken as an output through voice signals, a cyclic neural network (Recurrent Neural Networks, RNN) model is directly utilized for training, the trained model can be used for recognizing the instructions, the model is divided into two versions, a lightweight level and an accurate level, the lightweight level is deployed on the intelligent hardware body for voice recognition, the accurate level is deployed on a remote server, the accurate level is used for recognition when network connection is good, and if the network connection is not used for recognition, the trained model can be divided into two versions.
As shown in FIG. 5, the intelligent hardware body in the embodiment of the invention can also perform question-answer interaction, and is mainly divided into two interaction modes, wherein one interaction mode is to control other intelligent hardware, such as an intelligent television, an intelligent refrigerator and an intelligent water boiling kettle in an intelligent home. The other is a question-answering robot, the two are distinguished at the level of the instruction recognition module, for controlling other intelligent hardware, the intelligent hardware body and the other intelligent hardware are firstly subjected to interactive control, the result information is returned, and in the aspect of the question-answering robot, the deep learning technology is also utilized, the user intention is recognized through the specific content of the instruction, then the specific content is searched and sorted according to the specific intention, and finally the interactive content of the user is returned. In the embodiment of the invention, the wearable device can be directly connected with the server, so as to support more complex and wider application scenes. The wearable device can be used as the extension of the intelligent sound box, the inconvenience of the intelligent sound box is perfectly solved, and the user can enjoy intelligent interaction of the intelligent sound box in any scene without time through voice awakening and transmission of the wearable device.
As shown in fig. 6, an application scenario of the intelligent interaction method provided by the embodiment of the invention is an intelligent home scenario, which can support a user to perform any time and any place on the intelligent hardware in the home or outside the home to complete intelligent interaction. The intelligent household intelligent control system can transfer through the wearable equipment both inside and outside the household, can control any equipment in the household through intelligent hardware, and can support safety authentication and a complex control mechanism. The intelligent hardware bodies can be interconnected and networked, so that coverage is enhanced, and portability is improved.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present invention. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present invention.
In order to facilitate better implementation of the above-described aspects of embodiments of the present invention, the following provides related devices for implementing the above-described aspects.
Referring to fig. 7-a, a wearable device 700 provided in an embodiment of the present invention may include: a signal input module 701, a voice wake-up module 702, a signal acquisition module 703, a voice transmission module 704, wherein,
a signal input module 701, configured to obtain an input signal of a user;
a voice wake-up module 702, configured to determine that voice wake-up is required according to the input signal;
a signal acquisition module 703, configured to acquire a voice signal when voice wake-up is determined;
and the voice transmission module 704 is used for transmitting the voice signal to the intelligent hardware body, and the intelligent hardware body carries out voice instruction recognition on the voice signal.
In some embodiments of the present invention, referring to fig. 7-b, the voice transmission module 704 includes:
a matching module 7041, configured to perform short-distance matching with the intelligent hardware entity;
the first voice sending module 7042 is configured to send the voice signal to the intelligent hardware body if the wearable device and the intelligent hardware are successfully matched in a short distance.
In some embodiments of the present invention, referring to fig. 7-c, the voice transmission module 704 includes:
a credential acquisition module 7043, configured to acquire a user credential of the user;
A second voice sending module 7044, configured to send the user credential and the voice signal to a remote server, where the remote server forwards the user credential and the voice signal to the intelligent hardware entity.
In some embodiments of the present invention, referring to fig. 7-d, the voice wake module 701 includes:
a state determining module 7011, configured to determine a user state according to the input signal;
the logic obtaining module 7012 is configured to obtain a processing logic corresponding to the user state according to a preset corresponding relationship between the state and the processing logic;
the instruction determining module 7013 is configured to determine, according to the acquired processing logic, whether the user wakes up by voice.
In some embodiments of the present invention, referring to fig. 7-e, the wearable device 700 further includes: and the updating module 705 is used for dynamically learning and correcting the corresponding relation between the state and the processing logic by using the reinforcement learning model.
In some embodiments of the present invention, referring to fig. 7-f, the wearable device 700 further includes: and the sending module 706 is configured to send a signal transmission result to the user.
As can be seen from the above description of the embodiments of the present invention, the wearable device first obtains an input signal of a user, then determines that voice wakeup is required according to the input signal, the wearable device collects a voice signal when determining to perform voice wakeup, and finally the wearable device transmits the voice signal to the intelligent hardware body, and the intelligent hardware body performs voice instruction recognition on the voice signal. Because the wearable equipment keeps very close distance with the user at any time, real-time voice awakening can be performed according to the voice of the user in time through the wearable equipment, the wearable equipment can also transmit collected voice signals to the intelligent hardware body in real time after voice awakening to complete the whole intelligent interaction, and therefore the interaction between the user and the intelligent hardware body is not limited by the distance range, the problem that the intelligent interaction range is limited is solved, and the moving range of the user is not limited.
The embodiment of the present invention further provides another wearable device, as shown in fig. 8, for convenience of explanation, only the portion relevant to the embodiment of the present invention is shown, and specific technical details are not disclosed, please refer to the method portion of the embodiment of the present invention. The wearable device may include: wrist-supported watches (including products such as watches and wristbands), foot-supported footwear (including shoes, socks, or other future leg wear products), head-supported glasses (including glasses, helmets, headbands, etc.), and various non-mainstream product forms such as smart clothing, schoolbags, crutches, accessories, etc.
Fig. 8 is a block diagram showing a part of the structure of a wearable device related to a terminal provided by an embodiment of the present invention. Next, taking the terminal as a specific wearable device as an example, referring to fig. 8, the wearable device includes: memory 1020, input unit 1030, display unit 1040, sensor 1050, audio circuit 1060, wireless fidelity (wireless fidelity, wiFi) module 1070, processor 1080, and power source 1090. Those skilled in the art will appreciate that the wearable device structure shown in fig. 8 is not limiting of the wearable device and may include more or fewer components than shown, or certain components in combination, or a different arrangement of components.
The following describes the various constituent components of the wearable device in detail with reference to fig. 8:
memory 1020 may be used to store software programs and modules that processor 1080 performs the various functional applications and data processing of the wearable device by executing the software programs and modules stored in memory 1020. The memory 1020 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created from the use of the wearable device (such as audio data, phonebooks, etc.), and so on. In addition, memory 1020 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state memory device.
The input unit 1030 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function controls of the wearable device. In particular, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 1031 or thereabout using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a predetermined program. Alternatively, the touch panel 1031 may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 1080 and can receive commands from the processor 1080 and execute them. Further, the touch panel 1031 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The input unit 1030 may include other input devices 1032 in addition to the touch panel 1031. In particular, other input devices 1032 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a track ball, a mouse, a joystick, etc.
The display unit 1040 may be used to display information input by a user or information provided to the user, as well as various menus of the wearable device. The display unit 1040 may include a display panel 1041, and alternatively, the display panel 1041 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel 1031 may overlay the display panel 1041, and when the touch panel 1031 detects a touch operation thereon or thereabout, the touch panel is transferred to the processor 1080 to determine a type of touch event, and then the processor 1080 provides a corresponding visual output on the display panel 1041 according to the type of touch event. Although in fig. 8, the touch panel 1031 and the display panel 1041 are two separate components to implement the input and input functions of the wearable device, in some embodiments, the touch panel 1031 may be integrated with the display panel 1041 to implement the input and output functions of the wearable device.
The wearable device may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel 1041 according to the brightness of ambient light, and a proximity sensor that may turn off the display panel 1041 and/or the backlight when the wearable device moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (typically three axes), and can detect the gravity and direction when stationary, and can be used for identifying the application of the gesture of the wearable device (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration identification related functions (such as pedometer, knocking) and the like; other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured with the wearable device are not described in detail herein.
Audio circuitry 1060, a speaker 1061, and a microphone 1062 may provide an audio interface between a user and the wearable device. Audio circuit 1060 may transmit the received electrical signal after audio data conversion to speaker 1061 for conversion by speaker 1061 into an audio signal output; on the other hand, microphone 1062 converts the collected sound signals into electrical signals, which are received by audio circuit 1060 and converted into audio data, which are processed by audio data output processor 1080 and sent to, for example, another wearable device, or which are output to memory 1020 for further processing.
WiFi belongs to a short-distance wireless transmission technology, and the wearable device can help a user to send and receive emails, browse webpages, access streaming media and the like through the WiFi module 1070, so that wireless broadband Internet access is provided for the user. Although fig. 8 shows a WiFi module 1070, it is understood that it does not belong to the necessary constitution of the wearable device, and can be omitted entirely as required within a range that does not change the essence of the invention.
Processor 1080 is a control center of the wearable device, connects the various parts of the entire wearable device with various interfaces and lines, performs various functions of the wearable device and processes the data by running or executing software programs and/or modules stored in memory 1020, and invoking data stored in memory 1020, thereby performing overall monitoring of the wearable device. Optionally, processor 1080 may include one or more processing units; preferably, processor 1080 may integrate an application processor primarily handling operating systems, user interfaces, applications, etc., with a modem processor primarily handling wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 1080.
The wearable device also includes a power source 1090 (e.g., a battery) that powers the various components, preferably logically coupled to the processor 1080 via a power management system, to manage charging, discharging, and power consumption.
In an embodiment of the present invention, the processor 1080 included in the wearable device also has a control to perform the above intelligent interaction method flow executed by the wearable device.
As shown in fig. 9, an embodiment of the present application provides an intelligent interaction system 900, where the intelligent interaction system 900 includes: the wearable device 901, the smart hardware body 902 as described in the previous embodiments, wherein,
the intelligent hardware 902 is configured to perform voice command recognition on a voice signal sent by the wearable device 901.
For the functional description of the wearable device 901 and the intelligent hardware 902, reference may be made to the foregoing embodiments, and the description is omitted here.
It should be further noted that the above-described apparatus embodiments are merely illustrative, and that the units described as separate units may or may not be physically separate, and that units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the embodiment of the device provided by the invention, the connection relation between the modules represents that the modules have communication connection, and can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the present invention may be implemented by means of software plus necessary general purpose hardware, or of course by means of special purpose hardware including application specific integrated circuits, special purpose CPUs, special purpose memories, special purpose components, etc. Generally, functions performed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same functions can be varied, such as analog circuits, digital circuits, or dedicated circuits. However, a software program implementation is a preferred embodiment for many more of the cases of the present invention. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a readable storage medium, such as a floppy disk, a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random-access Memory (RAM, random Access Memory), a magnetic disk or an optical disk of a computer, etc., including several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to execute the method according to the embodiments of the present invention.
In summary, the above embodiments are only for illustrating the technical solution of the present invention, and are not limited thereto; although the present invention has been described in detail with reference to the above embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the above embodiments can be modified or some technical features thereof can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (7)
1. An intelligent interaction method is characterized by comprising the following steps:
the wearable device acquires an input signal of a user, wherein the input signal is used for determining whether to wake up voice of the wearable device;
the wearable device determines that voice wakeup is required according to the input signal;
the wearable device collects voice signals when voice wakeup is determined;
the wearable equipment is matched with the intelligent hardware body in a short distance; the wearable device performs short-distance matching with the intelligent hardware body, including: the wearable device searches whether paired intelligent hardware bodies exist in nearby positions through a short-distance wireless communication protocol; if the paired intelligent hardware bodies exist, the wearable equipment is matched with the paired intelligent hardware bodies in a short distance; if the matched intelligent hardware bodies do not exist, searching whether the matched intelligent hardware bodies exist in the nearby positions; if the paired intelligent hardware bodies exist, sending personal credentials of the wearable equipment to the paired intelligent hardware bodies for short-distance matching;
If the wearable equipment is successfully matched with the intelligent hardware body in a short distance, the wearable equipment sends a voice signal subjected to compression encryption processing to the intelligent hardware body, and the intelligent hardware body carries out voice instruction recognition on the voice signal; the intelligent hardware body performs voice instruction recognition on the voice signal, and the intelligent hardware body comprises: the intelligent hardware body carries out voice instruction recognition on the voice signal based on a trained cyclic neural network, the trained cyclic neural network is trained in an end-to-end training mode, the trained cyclic neural network comprises a lightweight cyclic neural network and an accurate cyclic neural network, the lightweight cyclic neural network is deployed on the intelligent hardware body, and the accurate cyclic neural network is deployed on a remote server;
if the short-distance matching between the wearable equipment and the intelligent hardware body is unsuccessful, the wearable equipment acquires a user credential of the user;
the wearable equipment carries out asymmetric encryption on the user credentials and the voice signals and then sends the encrypted user credentials and the voice signals to a remote server, the remote server uses a user public key to decrypt and identify the voice signals and then forwards the user credentials and the voice signals to the intelligent hardware body registered in advance by the user, and the intelligent hardware body registered in advance by the user carries out voice instruction identification on the voice signals;
The wearable device ends the voice acquisition function;
the wearable device determines that voice wakeup is required according to the input signal, and the method comprises the following steps:
the wearable device determines a user state according to the input signal;
the wearable equipment acquires processing logic corresponding to the user state according to a preset corresponding relation between the state and the processing logic; the corresponding relation between the state and the processing logic is dynamically learned and corrected by the wearable device by using an enhanced learning model, and training and sample of the enhanced learning model are completed based on a time difference learning algorithm;
and the wearable device determines whether the user wakes up by voice according to the acquired processing logic.
2. The method according to claim 1, wherein the method further comprises:
the wearable device sends a signal transmission result to the user.
3. A wearable device, comprising:
the signal input module is used for acquiring an input signal of a user, and the input signal is used for determining whether voice wakeup of the wearable equipment is performed or not;
the voice wake-up module is used for determining that voice wake-up is required according to the input signal;
The signal acquisition module is used for acquiring voice signals when voice wakeup is determined;
the voice transmission module is used for transmitting the voice signal to the intelligent hardware body so that the intelligent hardware body can recognize the voice instruction on the voice signal; the intelligent hardware body performs voice instruction recognition on the voice signal, and the intelligent hardware body comprises: the intelligent hardware body carries out voice instruction recognition on the voice signal based on a trained cyclic neural network, the trained cyclic neural network is trained in an end-to-end training mode, the trained cyclic neural network comprises a lightweight cyclic neural network and an accurate cyclic neural network, the lightweight cyclic neural network is deployed on the intelligent hardware body, and the accurate cyclic neural network is deployed on a remote server;
the signal acquisition module is also used for ending the voice acquisition function;
the voice wake-up module comprises:
the state determining module is used for determining the state of the user according to the input signal;
the logic acquisition module is used for acquiring the processing logic corresponding to the user state according to the preset corresponding relation between the state and the processing logic;
the instruction determining module is used for determining whether the user wakes up by voice according to the acquired processing logic;
The voice transmission module comprises:
the matching module is used for carrying out short-distance matching with the intelligent hardware body;
the first voice sending module is used for sending a voice signal subjected to compression encryption processing to the intelligent hardware body if the wearable equipment is successfully matched with the intelligent hardware body in a short distance, and the intelligent hardware body carries out voice instruction recognition on the voice signal;
the credential acquisition module is used for acquiring the user credential of the user if the short-distance matching between the wearable equipment and the intelligent hardware body is unsuccessful;
the second voice sending module is used for carrying out asymmetric encryption on the user certificate and the voice signal and then sending the user certificate and the voice signal to a remote server, and the remote server uses a user public key to decrypt and identify the voice signal and then forwards the user certificate and the voice signal to an intelligent hardware body which is registered in advance by the user;
the matching module is specifically configured to:
searching whether the paired intelligent hardware bodies exist in the nearby positions through a short-distance wireless communication protocol;
if the paired intelligent hardware bodies exist, carrying out short-distance matching with the paired intelligent hardware bodies;
If the matched intelligent hardware bodies do not exist, searching whether the matched intelligent hardware bodies exist in the nearby positions;
if the paired intelligent hardware bodies exist, sending personal credentials of the wearable equipment to the paired intelligent hardware bodies for short-distance matching;
the wearable device further comprises: and the updating module is used for dynamically learning and correcting the corresponding relation between the state and the processing logic by using an enhanced learning model, and training and sample of the enhanced learning model are completed based on a time difference learning algorithm.
4. The wearable device of claim 3, wherein the wearable device further comprises: and the sending module is used for sending the signal transmission result to the user.
5. A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 2.
6. A terminal, the terminal comprising: a processor, a memory; the processor and the memory are communicated with each other;
the memory is used for storing instructions;
the processor is configured to execute the instructions in the memory and perform the method of any one of claims 1 to 2.
7. An intelligent interactive system, the system comprising: the wearable device, the smart hardware body of any one of claim 3-4, wherein,
the intelligent hardware body is used for carrying out voice instruction recognition on voice signals sent by the wearable equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711294122.6A CN109901698B (en) | 2017-12-08 | 2017-12-08 | Intelligent interaction method, wearable device, terminal and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711294122.6A CN109901698B (en) | 2017-12-08 | 2017-12-08 | Intelligent interaction method, wearable device, terminal and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109901698A CN109901698A (en) | 2019-06-18 |
| CN109901698B true CN109901698B (en) | 2023-08-08 |
Family
ID=66940314
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711294122.6A Active CN109901698B (en) | 2017-12-08 | 2017-12-08 | Intelligent interaction method, wearable device, terminal and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109901698B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112820283A (en) * | 2019-11-18 | 2021-05-18 | 阿里巴巴集团控股有限公司 | Voice processing method, device and system |
| CN113497744B (en) * | 2020-03-18 | 2022-10-04 | 深圳市云动创想科技有限公司 | Method and system for controlling smart home, wearable device and readable storage medium |
| CN113542483B (en) * | 2020-03-30 | 2022-03-18 | Oppo广东移动通信有限公司 | Data transmission method and device, wearable device and storage medium |
| CN111812588B (en) * | 2020-07-20 | 2023-08-18 | 百度在线网络技术（北京）有限公司 | Multi-device voice wake-up implementation method and device, electronic device and medium |
| CN112433372A (en) * | 2020-11-18 | 2021-03-02 | 江西台德智慧科技有限公司 | Intelligent glasses, intelligent glasses system and intelligent glasses interaction method |
| CN113534715B (en) * | 2021-07-21 | 2023-01-24 | 歌尔科技有限公司 | Intelligent wearable device, and control method and system of target device |
| CN114415837A (en) * | 2022-01-25 | 2022-04-29 | 中国农业银行股份有限公司 | Operation auxiliary system and method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103558916A (en) * | 2013-11-07 | 2014-02-05 | 百度在线网络技术（北京）有限公司 | Man-machine interaction system, method and device |
| CN204129661U (en) * | 2014-10-31 | 2015-01-28 | 柏建华 | Wearable device and there is the speech control system of this wearable device |
| CN105096951A (en) * | 2015-05-28 | 2015-11-25 | 惠州Tcl移动通信有限公司 | Voice control realizing method and system based on intelligent wearable equipment |
| CN106249614A (en) * | 2016-10-12 | 2016-12-21 | 杭州创辉农业科技有限公司 | Intelligent terminal, intelligent domestic system, audio recognition method and ordering method |
| CN107134279A (en) * | 2017-06-30 | 2017-09-05 | 百度在线网络技术（北京）有限公司 | A kind of voice awakening method, device, terminal and storage medium |
| CN107193810A (en) * | 2017-05-19 | 2017-09-22 | 北京小蓦机器人技术有限公司 | A kind of method, apparatus and system for natural language contents title disambiguation |
| CN107290970A (en) * | 2016-04-05 | 2017-10-24 | 深圳市优思曼科技有限公司 | A kind of intelligent home control system based on intelligent sound |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130133629A (en) * | 2012-05-29 | 2013-12-09 | 삼성전자주식회사 | Method and apparatus for executing voice command in electronic device |
| US9620116B2 (en) * | 2013-12-24 | 2017-04-11 | Intel Corporation | Performing automated voice operations based on sensor data reflecting sound vibration conditions and motion conditions |
| KR101663113B1 (en) * | 2015-01-13 | 2016-10-14 | 현대자동차주식회사 | Apparatus for communicating with wearable device in car and method for controlling the same |
-
2017
- 2017-12-08 CN CN201711294122.6A patent/CN109901698B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103558916A (en) * | 2013-11-07 | 2014-02-05 | 百度在线网络技术（北京）有限公司 | Man-machine interaction system, method and device |
| CN204129661U (en) * | 2014-10-31 | 2015-01-28 | 柏建华 | Wearable device and there is the speech control system of this wearable device |
| CN105096951A (en) * | 2015-05-28 | 2015-11-25 | 惠州Tcl移动通信有限公司 | Voice control realizing method and system based on intelligent wearable equipment |
| CN107290970A (en) * | 2016-04-05 | 2017-10-24 | 深圳市优思曼科技有限公司 | A kind of intelligent home control system based on intelligent sound |
| CN106249614A (en) * | 2016-10-12 | 2016-12-21 | 杭州创辉农业科技有限公司 | Intelligent terminal, intelligent domestic system, audio recognition method and ordering method |
| CN107193810A (en) * | 2017-05-19 | 2017-09-22 | 北京小蓦机器人技术有限公司 | A kind of method, apparatus and system for natural language contents title disambiguation |
| CN107134279A (en) * | 2017-06-30 | 2017-09-05 | 百度在线网络技术（北京）有限公司 | A kind of voice awakening method, device, terminal and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109901698A (en) | 2019-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109901698B (en) | Intelligent interaction method, wearable device, terminal and system | |
| CN108735209B (en) | Wake-up word binding method, intelligent device and storage medium | |
| US10547609B2 (en) | Method, apparatus, and system for controlling intelligent device, and storage medium | |
| KR102573383B1 (en) | Electronic apparatus and controlling method thereof | |
| CN103578474B (en) | A kind of sound control method, device and equipment | |
| CN108712566B (en) | Voice assistant awakening method and mobile terminal | |
| WO2020020063A1 (en) | Object identification method and mobile terminal | |
| CN108462697B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN105976813A (en) | Speech recognition system and speech recognition method thereof | |
| CN107870674B (en) | Program starting method and mobile terminal | |
| WO2019114522A1 (en) | Screen control method, screen control apparatus, and mobile terminal | |
| CN106878390B (en) | Electronic pet interaction control method and device and wearable equipment | |
| CN107734170B (en) | Notification message processing method, mobile terminal and wearable device | |
| CN111387978B (en) | Method, device, equipment and medium for detecting action segment of surface electromyographic signal | |
| CN108108111B (en) | Method and device for checking lock screen picture information and mobile terminal | |
| CN109521684B (en) | Household equipment control method and terminal equipment | |
| WO2019154360A1 (en) | Interface switching method and mobile terminal | |
| CN107728920B (en) | Copying method and mobile terminal | |
| CN110796918A (en) | Training method and device and mobile terminal | |
| CN111417929A (en) | Interface display method and control terminal | |
| WO2017215615A1 (en) | Sound effect processing method and mobile terminal | |
| JP2021524203A (en) | Object recognition method and mobile terminal | |
| CN110069184B (en) | Mobile terminal control method, wearable device and computer readable storage medium | |
| CN112286122A (en) | Intelligent household control method, device, terminal and storage medium | |
| CN108170360B (en) | Control method of gesture function and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |