WO2024244895A1 - 用于助听的智能眼镜、助听方法以及辅助系统 - Google Patents

用于助听的智能眼镜、助听方法以及辅助系统 Download PDF

Info

Publication number
WO2024244895A1
WO2024244895A1 PCT/CN2024/091091 CN2024091091W WO2024244895A1 WO 2024244895 A1 WO2024244895 A1 WO 2024244895A1 CN 2024091091 W CN2024091091 W CN 2024091091W WO 2024244895 A1 WO2024244895 A1 WO 2024244895A1
Authority
WO
WIPO (PCT)
Prior art keywords
voice data
wireless communication
communication module
smart glasses
mobile terminal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/091091
Other languages
English (en)
French (fr)
Inventor
罗国华
戴冠林
余智深
范崇智
张惠权
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Solos Technology Shenzhen Ltd
Original Assignee
Solos Technology Shenzhen Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Solos Technology Shenzhen Ltd filed Critical Solos Technology Shenzhen Ltd
Publication of WO2024244895A1 publication Critical patent/WO2024244895A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • G—PHYSICS
    • G02—OPTICS
    • G02C—SPECTACLES; SUNGLASSES OR GOGGLES INSOFAR AS THEY HAVE THE SAME FEATURES AS SPECTACLES; CONTACT LENSES
    • G02C11/00—Non-optical adjuncts; Attachment thereof
    • G02C11/06—Hearing aids
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2225/00—Details of deaf aids covered by H04R25/00, not provided for in any of its subgroups
    • H04R2225/49—Reducing the effects of electromagnetic noise on the functioning of hearing aids, by, e.g. shielding, signal processing adaptation, selective (de)activation of electronic parts in hearing aid

Definitions

  • the embodiments of the present application relate to the technical field of smart glasses, and in particular, to smart glasses for hearing assistance, a hearing assistance method, and an auxiliary system.
  • the embodiments of the present application provide smart glasses for hearing assistance, a hearing assistance method, and an auxiliary system, which are used to realize a real-time hearing assistance function based on smart glasses, thereby increasing product viscosity.
  • an embodiment of the present application provides a pair of smart glasses for hearing aid, comprising: two temples, a sound pickup device, a plurality of speakers, and a wireless communication module;
  • the multiple speakers and the sound pickup device are arranged on at least one of the two temples, the wireless communication module is arranged in the cavity of any one of the two temples and is electrically connected to the sound pickup device and the multiple speakers, and the sound pickup device includes at least one microphone;
  • the at least one microphone is used to obtain first voice data from a first preset direction
  • the wireless communication module is used to amplify the first voice data to obtain amplified voice data, and send the amplified voice data to a target speaker among the multiple speakers for playback, wherein the target speaker and the at least one microphone are respectively arranged on different temples.
  • an embodiment of the present application further provides a hearing aid method based on smart glasses, wherein the smart glasses include: two temples, a wireless communication module, and a sound pickup device and a plurality of speakers electrically connected to the wireless communication module, wherein the sound pickup device includes at least one microphone, and the method includes:
  • the amplified voice data is played through a target speaker among the multiple speakers, wherein the target speaker and the at least one microphone are respectively arranged on different temples.
  • the embodiment of the present application also provides an auxiliary system, including: smart glasses and a smart mobile terminal;
  • the smart glasses include: two temples, a sound pickup device, a plurality of speakers, and a wireless communication module, wherein the plurality of speakers and the sound pickup device are arranged on at least one of the two temples, the wireless communication module is arranged in a cavity of any one of the two temples and is electrically connected to the sound pickup device and the plurality of speakers, and the sound pickup device includes at least one microphone;
  • the wireless communication module is used to control the switching of the working modes of the smart glasses in response to the control operation of the user, and the working modes include: a hearing aid mode and a translation mode;
  • the at least one microphone is used to obtain first voice data from a first preset direction in the hearing aid mode or the translation mode;
  • the wireless communication module is used for, in the hearing aid mode, amplifying the first voice data to obtain amplified voice data, and sending the amplified voice data to a target speaker among the multiple speakers for playback, wherein the target speaker and the at least one microphone are respectively arranged on different temples;
  • the wireless communication module is further used to send the first voice data to the smart mobile terminal in the translation mode
  • the intelligent mobile terminal is used to receive the first voice data and send second voice data to the wireless communication module, wherein the second voice data is obtained by translating the first voice data;
  • the wireless communication module is further used to receive second voice data sent by the smart mobile terminal, and send the second voice data to the target speaker for playing.
  • At least one microphone in the smart glasses is used to obtain first voice data from a first preset direction, and the first voice data is amplified by a wireless communication module in the smart glasses and the amplified first voice data is sent to a target speaker among the multiple speakers for playback, wherein the target speaker and the at least one microphone are respectively arranged on different temples, thereby realizing a real-time hearing aid function based on the smart glasses, thereby expanding the function of the smart glasses and improving product stickiness.
  • the microphone on one side obtains the voice data to be amplified from a specific direction and the speaker on the other side plays the amplified voice data, the echo can be reduced and the quality of the played sound can be improved.
  • FIG1 is a schematic diagram of the structure of smart glasses provided by an embodiment of the present application.
  • FIG2 is a schematic diagram of the internal structure of smart glasses provided by another embodiment of the present application.
  • FIG3 is a schematic diagram of the external structure of the smart glasses in the embodiment shown in FIG2 ;
  • FIG4 is a schematic diagram of an auxiliary system provided by an embodiment of the present application.
  • FIG5 is a schematic diagram of the hardware structure of the intelligent mobile terminal in the auxiliary system shown in FIG4 ;
  • FIG6 is a schematic diagram of an implementation flow of a hearing aid method based on smart glasses provided in an embodiment of the present application.
  • FIG7 is a schematic diagram of the sound beam direction of the microphone in different working modes in the hearing aid method based on smart glasses provided in an embodiment of the present application;
  • FIG. 8 is a schematic diagram of the operating state and working mode of smart glasses in the hearing aid method based on smart glasses provided in an embodiment of the present application;
  • FIG9 is a schematic diagram of a process of processing a voice signal in a call mode in a hearing aid method based on smart glasses provided in an embodiment of the present application;
  • FIG10 is a schematic diagram of a process of processing a speech signal in a hearing aid mode in a hearing aid method based on smart glasses provided in an embodiment of the present application;
  • FIG. 11 is a schematic diagram of the feedback cancellation process in FIG. 10 .
  • FIG. 1 is a schematic diagram of the structure of smart glasses provided in an embodiment of the present application.
  • the smart glasses 10 include: a wireless communication module 101, a sound pickup device 102, a plurality of speakers 103, and two temples 104.
  • the wireless communication module 101 is electrically connected to the sound pickup device 102 and the plurality of speakers 103.
  • the multiple speakers 103 and the sound pickup device 102 are arranged on at least one of the two temples 104 , and the wireless communication module 101 is arranged in the cavity of any one of the two temples 104 and is electrically connected to the sound pickup device 102 and the multiple speakers 103 .
  • the sound pickup device 102 includes at least one microphone.
  • the sound pickup device 102 includes at least one directional microphone.
  • the sound pickup device 102 may also include at least one omnidirectional microphone.
  • the sound pickup device 102 may be a microphone array consisting of at least one directional microphone and/or at least one omnidirectional microphone.
  • the wireless communication module 101 may include, but is not limited to: a controller 1011, a voice data processor 1012, and a wireless signal transceiver 1013.
  • the controller 1011 is mainly used for UI (User Interface), task operation, and real-time operating system operation.
  • the voice data processor 1012 is mainly used to execute the operation of the relatively time-consuming signal processing algorithm and the Bluetooth wireless communication algorithm.
  • the wireless signal transceiver may be, for example, a wireless signal transceiver.
  • the wireless signal transceiver can use, but is not limited to, at least one of the following protocols for data transmission: Bluetooth protocol, WiFi (Wireless Fidelity) protocol, NFC (Near Field Communication) protocol, ZigBee, DLNA (DIGITAL LIVING NETWORK ALLIANCE) protocol, UWB (Ultra Wideband, carrier-free communication), RFID (Radio Frequency Identification) protocol, and cellular mobile communication protocol.
  • At least one microphone is used to obtain first voice data from a first preset direction.
  • the wireless communication module 101 is used to amplify the first voice data to obtain amplified voice data, and send the amplified voice data to a target speaker among the multiple speakers 103 for playback, wherein the target speaker and at least one microphone are respectively arranged on different temples.
  • FIG. 2 is a schematic diagram of the internal structure of smart glasses provided by another embodiment of the present application
  • FIG. 3 is a schematic diagram of the external structure of the smart glasses in the embodiment shown in FIG. 2
  • the smart glasses 20 include: a frame 201, two temples 202, at least one sound pickup device 203 (for ease of understanding, only one is shown in the figure), a first speaker 205, a second speaker 206, and a wireless communication module 204.
  • the temple 202 is connected to the frame 201, the first speaker 205, the second speaker 206 and at least one sound pickup device 203 are arranged on at least one of the two temples 202, and the wireless communication module 204 is arranged in the cavity of any one of the two temples 202 and is electrically connected to the first speaker 205, the second speaker 206 and at least one sound pickup device 203.
  • the sound pickup device 203 is used to obtain first voice data from a first preset direction.
  • the wireless communication module 204 is used to amplify the first voice data to obtain amplified voice data, and send the amplified voice data to a target speaker among the first speaker 205 and the second speaker 206 for playback, wherein the target speaker and the sound pickup device 203 are respectively arranged on different temples.
  • the wireless communication module 204 is also used to control the switching of the working modes of the smart glasses, which include a call mode, a hearing aid mode and a translation mode.
  • the two temples 202 include a first temple 202A and a second temple 202B, the front end of the first temple 202A and the front end of the second temple 202B are respectively connected to two sides of the frame 201, and at least one sound pickup device 203 is installed at the front end of the first temple 202A.
  • the first speaker 205 and the second speaker 206 are used to output voice data or music data.
  • the voice data includes: the voice data obtained by the sound pickup device 203 after being processed by the wireless communication module 204, and the call voice data sent by the smart mobile terminal received by the wireless communication module 204.
  • the music data includes: the music data sent by the smart mobile terminal received by the wireless communication module 204.
  • the amplified voice signal is only output to the speaker configured on the right temple for playback. If the sound pickup device is located on the right temple, the amplified voice will be output to the speaker configured on the left temple for playback.
  • each microphone and each speaker when a microphone and a speaker are provided on each temple, each microphone and each speaker have their own corresponding unique identification information, and the corresponding relationship between the unique identification information of each microphone and each speaker and their positions is stored in the wireless communication module 204.
  • the wireless communication module 204 is further used to determine the installation position of the microphone for acquiring the first voice data according to the corresponding relationship, and determine the target microphone according to the installation position and the corresponding relationship.
  • the smart glasses can independently amplify the voice signal in front of the wearer without connecting to any mobile terminal device.
  • the tail end of the temple is closest to the user's ear. Installing the speaker outlet at the tail end of the temple can make the speaker outlet closest to the user's ear, thereby improving the efficiency of sound output.
  • the first speaker 205 and the second speaker 206 are preferably mono speakers.
  • the mono speakers respectively installed on the two temples can be combined to achieve stereo sound effects.
  • the wireless communication module 204 includes: a controller 2041 , a voice data processor 2042 and a wireless signal transceiver 2043 .
  • the controller 2041 is used to control the switching of the working modes of the smart glasses.
  • the controller 2041 is preferably a MCU (Microcontroller Unit).
  • the voice data processor 2042 is used to process the voice data.
  • the voice data processor 2042 is preferably a DSP (Digital Signal Processing) or a voice data processing integrated circuit.
  • the voice data processing integrated circuit is a common circuit, and the present application does not specifically limit its structure.
  • the sound pickup device 203 is preferably a directional microphone, which is used to directionally pick up the voice signal and then send it to the MCU/DSP for voice processing.
  • the sound pickup device 102 also includes two microphones for picking up voice signals from a second preset direction.
  • the user can pick up voice signals from the second preset direction through the two microphones on the smart glasses to issue voice commands and answer calls.
  • the use of two microphones to pick up sound can accurately determine the direction of the sound pickup, and the processing of the sound pickup data of the two microphones can achieve noise reduction.
  • the voice signal is picked up from the two microphones of the smart glasses, and the noise-reduced voice is played on the speakers of the smart glasses and sounds are emitted.
  • the wireless signal transceiver 2043 is used for data interaction with the smart mobile terminal.
  • the voice data processor 2042 includes a voice equalizer.
  • the wireless signal transceiver 2043 is also used to receive a volume adjustment control instruction sent by the smart mobile terminal and send it to the voice data processor 2042.
  • the voice data processor 2042 is also used to adjust the sound data to be output to the frequency band and volume sound data pointed to by the volume adjustment control instruction by using the voice equalizer, and send the adjusted sound data to the speaker pointed to by the volume adjustment control instruction.
  • the frequency band pointed to by the volume adjustment control instruction can be low frequency, medium frequency or high frequency.
  • the volume adjustment can be to increase the volume or to decrease the volume.
  • the voice data processor 2042 is also used to perform voice equalization processing (using a voice equalizer) and output volume processing on the data on the downlink channel in the call mode, and to perform echo cancellation processing, beamforming processing and noise suppression processing on the data on the uplink channel using a preset acoustic echo cancellation algorithm (Acoustic Echo Cancellation), beamforming algorithm (Beamforming) and noise suppression algorithm (Noise Cancellation).
  • voice equalization processing using a voice equalizer
  • output volume processing on the data on the downlink channel in the call mode
  • echo cancellation processing beamforming processing and noise suppression processing on the data on the uplink channel using a preset acoustic echo cancellation algorithm (Acoustic Echo Cancellation), beamforming algorithm (Beamforming) and noise suppression algorithm (Noise Cancellation).
  • the voice data processor 2042 is also used to perform feedback cancellation processing, voice activity detection processing, noise suppression processing, voice equalization processing and user speech detection processing on the voice data in the hearing aid mode or the translation mode using the preset feedback cancellation algorithm (Feedback Cancellation), voice activity detection processing, noise suppression algorithm, voice equalizer algorithm (Equalizer) and user speech detection algorithm, and send the voice data after the voice equalization processing to the corresponding speaker for output.
  • the voice data after the voice equalization processing is also used as reference data for the above-mentioned feedback cancellation processing during the processing process.
  • the smart glasses further include: at least one sensor 207 electrically connected to the wireless communication module 204 .
  • At least one sensor 207 is mounted on the inner side and/or the outer side of the first temple 102A and/or the second temple 102B.
  • the at least one sensor 207 may include, but is not limited to, at least one of a touch sensor, a proximity sensor, an accelerometer, a gyroscope, a magnetic induction sensor, and an inertial measurement unit.
  • the inertial measurement unit is a 9-axis sensor.
  • the 9-axis sensor is used to collect the user's motion data (3-axis accelerometer data, 3-axis gyroscope data, 3-axis magnetometer data), and send it to the smart mobile terminal through the wireless communication module 204 for monitoring the status of the user's head and body.
  • the at least one sensor 207 includes: a 9-axis sensor, at least one touch sensor and at least one proximity sensor.
  • At least one touch sensor is installed on the outer side of the first temple 102A and/or the second temple 102B.
  • At least one touch sensor is used to detect a first control operation of the user, and send data of the detected first control operation to the controller 2041, where the first control operation is used to adjust the volume.
  • the controller 2041 is also used to control and adjust the volume of the sound output by the smart glasses in response to the first control operation according to the data of the first control operation.
  • the first control operation includes a control operation for increasing the volume and a control operation for decreasing the volume.
  • the control operation for increasing the volume corresponds to an action of the user's finger sweeping on the touch sensor toward the ear
  • the control operation for decreasing the volume corresponds to an action of the user's finger sweeping on the touch sensor toward the frame (i.e., away from the ear).
  • At least one touch sensor is further used as an input device to detect a second control operation of the user, and send data of the detected second control operation to the controller 2041.
  • the controller 2041 is also used to control switching of the working mode of the smart glasses according to the data of the second control operation and in response to the second control operation.
  • At least one proximity sensor is installed on the inner side of the first temple 202A and/or the second temple 202B, and is used to detect whether the user wears or takes off the smart glasses and obtain the time when the user does not wear the smart glasses, and send the detection result to the controller 2041.
  • the controller 2041 is used to control the play or pause of music and control the power on or off of the smart glasses 20 according to the detection result. For example, when the proximity sensor detects that the user wears the smart glasses, the music data is played, when the proximity sensor detects that the user takes off the smart glasses, the music data is stopped from being played, and when the proximity sensor detects that the user does not wear the smart glasses for more than a preset time, the power is turned off.
  • the number of each of the above sensors is preferably one to reduce the overall weight of the smart glasses.
  • the number of various sensors is not limited to one.
  • a proximity sensor can be set on each of the two temples.
  • controller 2041 is also used to send the data acquired by each sensor to the smart mobile terminal via the above-mentioned wireless signal transceiver.
  • the sound pickup device 203 may include a first directional microphone and a second directional microphone.
  • the first directional microphone is used to obtain voice data in front of the user
  • the second directional microphone is used to obtain voice data below the user.
  • the first directional microphone and the second directional microphone may be located at different positions of the temples, respectively.
  • the controller 2041 may also control the directional microphones at corresponding positions to pick up sound according to the working mode of the smart glasses 20, thereby reducing noise in the acquired voice data and improving the speed of signal processing.
  • the smart glasses also include a battery 208, which is installed on the first temple 202A and electrically connected to the wireless communication module 204, so as to provide power for all hardware components on the smart glasses, such as the wireless communication module 204, sensors, speakers, and microphones.
  • a battery 208 which is installed on the first temple 202A and electrically connected to the wireless communication module 204, so as to provide power for all hardware components on the smart glasses, such as the wireless communication module 204, sensors, speakers, and microphones.
  • the smart glasses further include: an input device electrically connected to the wireless communication module 204.
  • the input device is disposed on the frame 201 and at least one of the two temples 202.
  • the input device includes: at least one of a touch sensor and at least one control button (not shown in the figure).
  • the at least one control button is installed on the outside of the first temple 202A and/or the second temple 202B, and is electrically connected to the wireless communication module 204.
  • the at least one control button is used to trigger the wireless communication module 204 to control the switching of the working mode or operating state of the smart glasses.
  • the operating state includes an idle state and a working state, and the working state includes the call mode, the hearing aid mode, and the translation mode.
  • the wireless communication module 204 is also used to switch the working mode of the smart glasses to a translation mode in response to a second switching instruction triggered by the user based on the input device, and in the translation mode, send the first voice data to the smart mobile terminal, receive the second voice data sent by the smart mobile terminal, and send the second voice data to the target speaker for playback, wherein the second voice data is translated from the first voice data.
  • the wireless communication module 204 is also used to switch the working mode of the smart glasses to a call mode in response to a third switching instruction triggered by the user based on the input device.
  • this call mode the third voice data from the second preset direction obtained by the microphone is sent to the target terminal, and the fourth voice data sent by the target terminal is received and output to all speakers for playback.
  • the target terminal is the other terminal.
  • the target terminal is the smart mobile terminal.
  • the first preset direction points to the front of the user, and the second preset direction points to the bottom of the user.
  • the electronic components of the above-mentioned smart glasses can be connected through a bus.
  • the relationship between the components of the above-mentioned smart glasses can be a replacement relationship or a superposition relationship. That is, all the components in the above-mentioned embodiment can be installed on one smart glass, or a part of the above-mentioned components can be selectively installed according to needs.
  • the smart glasses are also provided with a connection interface for peripherals, and the connection interface can be at least one of a PS/2 interface, a serial interface, a parallel interface, an IEEE1394 interface, a USB (Universal Serial Bus) interface, etc.
  • the function of the replaced component can be realized by a peripheral connected to the connection interface, such as: an external speaker, an external sensor, etc.
  • the controller 2041 is also used to determine whether there is a playback device that establishes a data connection with the smart glasses, such as an external speaker or an external hearing aid; if so, the second voice data or the amplified first voice data is sent to the playback device for playback through the playback device; if not, the second voice data or the amplified first voice data is output through a local speaker. In this way, when there is an independent playback device, the power consumption of the smart glasses can be further reduced by using the independent playback device to play the voice data.
  • a playback device that establishes a data connection with the smart glasses, such as an external speaker or an external hearing aid
  • the real-time hearing aid function based on the smart glasses is realized, thereby expanding the function of the smart glasses and improving the product stickiness.
  • the weight of the smart glasses can be reduced, the power consumption can be reduced, and the manufacturing cost of the smart glasses can be reduced.
  • the microphone on one side obtains the voice data to be amplified from a specific direction and the amplified voice data is played by the speaker on the other side, the echo can be reduced and the quality of the played sound can be improved.
  • the real-time translation function based on the smart glasses is also realized, further expanding the function of the smart glasses, and since the voice data to be translated is obtained from a specific direction, the quality of the voice data to be translated can be improved, thereby improving the translation efficiency.
  • Fig. 4 is a schematic diagram of the structure of an auxiliary system provided in an embodiment of the present application.
  • the auxiliary system includes: smart glasses 301 , a smart mobile terminal 302 , and a cloud computing device 303 .
  • the smart glasses include 301: two temples, a sound pickup device, a plurality of speakers and a wireless communication module, wherein the sound pickup device includes at least one microphone.
  • the wireless communication module is used to control the switching of the working modes of the smart glasses in response to the user's control operation, and the working modes include: hearing aid mode, translation mode and call mode.
  • the at least one microphone is used to obtain first voice data from a first preset direction in the hearing aid mode or the translation mode;
  • the wireless communication module is used to amplify the first voice data to obtain amplified voice data in the hearing aid mode, and send the amplified voice data to a target speaker among the multiple speakers for playback, wherein the target speaker and the at least one microphone are respectively arranged on different temples.
  • the wireless communication module is also used to send the first voice data to the smart mobile terminal in the translation mode, receive the second voice data sent by the smart mobile terminal, and send the second voice data to the target speaker for playback, wherein the second voice data is obtained by translating the first voice data.
  • the at least one microphone is further used to obtain third voice data from a second preset direction in the call mode.
  • the wireless communication module is also used to send the third voice data to the target terminal and receive the fourth voice data sent by the target terminal.
  • the multiple speakers are also used to play the fourth voice data.
  • the smart mobile terminal 302 may include, but is not limited to: a cellular phone, a smart phone, other wireless communication devices, a personal digital assistant, an audio player, other media players, a music recorder, a video recorder, a camera, other media recorders, a smart radio, a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a moving picture expert group (MPEG-1 or MPEG-2) audio layer 3 (MP3) player, a digital camera, and a smart wearable device (such as a smart watch, a smart bracelet, etc.).
  • PDA personal digital assistant
  • PMP portable multimedia player
  • MPEG-1 or MPEG-2 moving picture expert group
  • MP3 audio layer 3
  • the smart mobile terminal 302 is installed with an Android or IOS operating system.
  • the smart mobile terminal 302 is used to perform data interaction with the smart glasses 301 and the cloud computer device 303. Specifically, for example, the first voice data sent by the smart glasses 301 is received, and the first voice data is translated into the second voice data of the target language locally at the smart mobile terminal 302, or the first voice data or the first text data corresponding to the first voice data is sent to the cloud computer device 303 for translation; the second text data of the translated target language returned by the cloud computer device 303 is received, and the second text data is converted into the second voice data and sent to the smart glasses 301. For another example, when performing target tasks such as playing music data and making calls, the played music data and the received call voice data are sent to the smart glasses 301.
  • the communication protocol used by the smart mobile terminal 302 when performing data interaction with the smart glasses 301 is consistent with the communication protocol used by the smart glasses 301.
  • the smart mobile terminal 302 determines whether to send the first voice data or the first text data corresponding to the first voice data to the cloud computer device 303 according to a preset conversion mark.
  • the smart mobile terminal 302 is also used to obtain a preset conversion mark. If the preset conversion mark is a first mark, the first voice data is converted into the first text data, and the first text data is sent to the cloud computer device 303. If the preset conversion mark is a second mark, the first voice data is directly sent to the cloud computer device 303.
  • the smart mobile terminal 302 is also used to obtain the network speed and determine whether the network speed is greater than a preset speed; if it is not greater than the preset speed, the conversion mark is set to the first mark; if it is greater than the preset speed, the conversion mark is set to the second mark.
  • the intelligent mobile terminal 302 is further configured to, in response to a setting operation of the user, set the conversion mark to the mark pointed to by the setting operation.
  • the smart mobile terminal 302 may include a control circuit, which may include a storage and processing circuit 300.
  • the storage and processing circuit 300 may include a memory, such as a hard disk drive memory, a non-volatile memory (such as a flash memory or other electronically programmable memory with limited deletion for forming a solid-state drive, etc.), a volatile memory (such as a static or dynamic random access memory, etc.), etc., which is not limited in the embodiment of the present application.
  • the processing circuit in the storage and processing circuit 300 can be used to control the operation of the smart mobile terminal 302.
  • the processing circuit can be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application-specific integrated circuits, display driver integrated circuits, etc.
  • the storage and processing circuit 300 can be used to run software in the smart mobile terminal 302, such as: artificial intelligence housekeeper application, Internet browsing application, Voice over Internet Protocol (VOIP) phone call application, email application, media playback application, operating system function, etc.
  • These software can be used to perform some control operations, such as data processing and analysis of motion data sent by the smart glasses 301 based on a preset analysis algorithm, image acquisition based on a camera, ambient light measurement based on an ambient light sensor, proximity sensor measurement based on a proximity sensor, information display function based on a status indicator such as a status indicator light of a light-emitting diode, touch event detection based on a touch sensor, functions associated with displaying information on multiple (e.g., layered) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals, control operations associated with collecting and processing button press event data, and other functions in the smart mobile terminal 302, etc., which are not limited by the embodiments of the present application.
  • the memory stores executable program code
  • the processor connected to the memory calls the executable program code stored in the memory to execute the relevant steps in the following method embodiments of the present application to implement the various functions of the smart mobile terminal 302 in each embodiment of the present application.
  • the smart mobile terminal 302 may further include an input/output circuit 420.
  • the input/output circuit 420 may be used to enable the smart mobile terminal 302 to input and output data, that is, to allow the smart mobile terminal 302 to receive data from an external device and also to allow the smart mobile terminal 302 to output data from the smart mobile terminal 302 to an external device.
  • the input/output circuit 420 may further include a sensor 320.
  • the sensor 320 may include an ambient light sensor, a proximity sensor based on light and capacitance, a touch sensor (for example, based on a light touch sensor and/or a capacitive touch sensor, wherein the touch sensor may be part of a touch display screen or may be used independently as a touch sensor structure), an accelerometer, and other sensors.
  • the input/output circuit 420 may also include one or more displays, such as the display 140.
  • the display 140 may include a liquid crystal display, an organic light emitting diode display, an electronic ink display, a plasma display, a display using other display technologies, or a combination of several of them.
  • the display 140 may include a touch sensor array (i.e., the display 140 may be a touch display screen).
  • the touch sensor may be a capacitive touch sensor formed by an array of transparent touch sensor electrodes (e.g., indium tin oxide (ITO) electrodes), or may be a touch sensor formed using other touch technologies, such as sonic touch, pressure-sensitive touch, resistive touch, optical touch, etc., which is not limited in the embodiments of the present application.
  • ITO indium tin oxide
  • the smart mobile terminal 302 may also include an audio component 360.
  • the audio component 360 may be used to provide audio input and output functions for the smart mobile terminal 302.
  • the audio component 360 in the smart mobile terminal 302 may include a speaker, a sound pickup device, a buzzer, a tone generator, and other components for generating and detecting sound.
  • the communication circuit 380 can be used to provide the smart mobile terminal 302 with the ability to communicate with external devices.
  • the communication circuit 380 may include analog and digital input/output interface circuits, and wireless communication circuits based on radio frequency signals and/or optical signals.
  • the wireless communication circuit in the communication circuit 380 may include a radio frequency transceiver circuit, a power amplifier circuit, a low noise amplifier, a switch, a filter, and an antenna.
  • the wireless communication circuit in the communication circuit 380 may include a circuit for supporting near field communication (NFC) by transmitting and receiving near field connection electromagnetic signals.
  • the communication circuit 380 may include a near field communication antenna and a near field communication transceiver.
  • the communication circuit 380 may also include a cellular phone transceiver and antenna, a wireless local area network transceiver circuit and antenna such as Bluetooth, WiFi, ZigBee, DLNA, UWB, RFID, etc.
  • the smart mobile terminal 302 may further include a battery, power management circuits and other input/output units 400.
  • the input/output units 400 may include buttons, joysticks, click wheels, scroll wheels, touch pads, keypads, keyboards, cameras, light emitting diodes and other status indicators.
  • the user can input commands through the input/output circuit 420 to control the operation of the smart mobile terminal 302 , and can use the output data of the input/output circuit 420 to receive status information and other outputs from the smart mobile terminal 302 .
  • the smart mobile terminal 302 is also used to pair with the smart glasses 301 based on the Bluetooth protocol, and after the pairing is successful, the played music data is sent to the smart glasses 301 so that the music data can be played through the smart glasses 301.
  • the smart glasses 301 are also used to send the first voice data to be translated to the smart mobile terminal 302 for translation after the pairing is successful.
  • the smart mobile terminal 302 is further used to obtain GPS data through a GPS module configured on the smart mobile terminal, and send the obtained GPS data to the smart glasses 301 for positioning the smart glasses 301.
  • the smart mobile terminal 302 is also used to receive and store the motion data sent by the smart glasses 301 in real time, calculate the motion index according to the motion data and the GPS data, generate real-time voice data for notifying or reminding the user of the motion status according to the calculation result, and send the real-time voice data to the smart glasses 301 for output.
  • the real-time voice data includes the notification or reminder voice of the motion index calculation result.
  • a client program such as an artificial intelligence housekeeper App
  • an artificial intelligence housekeeper App may be installed on the smart mobile terminal 302, through which data interaction operations with the smart glasses 301 are performed, parameters of the smart glasses 301 are configured based on user operations on the App, and motion data sent by the smart glasses 301 are processed and analyzed, such as running index calculation, posture monitoring and reminders, etc.
  • the above-mentioned music data and real-time voice data belong to the data of the downlink channel of the smart glasses 301.
  • the smart glasses 301 perform voice equalization processing and output volume control processing on the received music data and real-time voice data through the voice data processor 1042, and then send them to the first speaker 201 and the second speaker 202 for output.
  • the motion data includes data obtained by the smart glasses 301 through a 9-axis sensor (i.e., Ax, Ay, Az; Gx, Gy, Gz; Mx, My, Mz), and the motion index is a running index, and the running index includes: pace, distance, number of steps, left and right head balance, number of steps, stride length and step frequency.
  • the 9-axis sensor refers to an accelerometer, a gyroscope and a magnetic induction sensor.
  • the data measured by these three types of sensors can be decomposed into forces of the three directional axes of X, Y, and Z in the spatial coordinate system, so they are also called 3-axis accelerometers, 3-axis gyroscopes and 3-axis magnetic induction sensors.
  • the smart mobile terminal 302 is also used to perform index calculation, posture monitoring and exercise reminder operations according to the motion data obtained by the 9-axis sensor and the local GPS data, and display the index calculation results in real time through the display of the smart mobile terminal 302. Furthermore, the smart mobile terminal 302 can also generate voice prompt information according to the index calculation results, and send the voice prompt information to the smart glasses 301 via Bluetooth, so as to notify or remind the user in real time through the smart glasses 301.
  • the smart mobile terminal 302 is also used to respond to voice commands sent by the smart glasses 301 to perform operations of making, answering or hanging up a call, and during a call, send the received call voice data to the smart glasses 301 so that the call voice data can be played through the speakers on the smart glasses 301.
  • the voice command is obtained by the voice data processor of the smart glasses 301 by performing voice command recognition processing on the voice data obtained by the sound pickup device of the smart glasses 301 using a preset voice recognition algorithm.
  • At least two directional and/or omnidirectional microphones are provided on the body of the smart glasses 301 in the present application, and the user can obtain voice data through the microphone on the smart glasses 301 for issuing voice commands and answering calls.
  • the sound emitted by the user is received by the microphone on the smart glasses 301 and played through the speaker of the smart glasses 301 after noise reduction processing. Therefore, during a call, the user can put the smart mobile terminal 302 in a pocket or on a table, and the free hands can be used for other purposes, thereby improving the convenience of answering calls.
  • the cloud computing device 303 may be, for example, a cloud server or a server cluster, which is used to interact with the smart mobile terminal 302, store data sent by the smart mobile terminal 302 and/or the smart glasses 301 (e.g., all sensor data acquired by the smart glasses), and process the data based on a preset processing logic. Specifically, the cloud computing device 303 receives the first voice data sent by the smart mobile terminal 302 or the first text data corresponding to the first voice data, and then translates the first voice data or the first text data into the second text data of the target language, and returns it to the smart mobile terminal 302.
  • a cloud server or a server cluster which is used to interact with the smart mobile terminal 302, store data sent by the smart mobile terminal 302 and/or the smart glasses 301 (e.g., all sensor data acquired by the smart glasses), and process the data based on a preset processing logic. Specifically, the cloud computing device 303 receives the first voice data sent by the smart mobile terminal 302 or the first text data corresponding to the first voice
  • the smart mobile terminal 302 sends Chinese voice data 1, the voice data 1 is first converted into Chinese text data 1, and then the text data 1 is translated into English text data 2 and sent to the smart mobile terminal 302; if the smart mobile terminal 302 sends text data 1 corresponding to the Chinese voice data 1, the text data 1 is translated into English text data 2 and sent to the smart mobile terminal 302.
  • the source language recognition operation may be performed by the smart mobile terminal 302, or may also be performed by the cloud computing device 303. If the source language recognition operation is performed by the smart mobile terminal 302, then when the smart mobile terminal 302 sends the first voice data, it sends the identification information of the source language and the target language to the cloud computing device 303, so that the cloud computing device 303 determines the source language and the target language according to the indication information.
  • the source language and the target language may also be determined by the smart mobile terminal 302 based on the user's selection operation on the smart glasses 301 or the smart mobile terminal 302 .
  • the user may also select the source language and/or the target language by using the control buttons or touch sensors on the smart glasses 301 or the smart mobile terminal 302. If the user performs the selection operation through the smart glasses 301, the smart glasses 301 sends the identification information of the source language and/or the target language selected by the user to the smart mobile terminal 302 while sending the first voice data to the smart mobile terminal 302.
  • the smart glasses 301 can also output corresponding voice prompts in real time according to the user's operation to help the user quickly select the source language and/or the target language from multiple alternative languages.
  • the Bluetooth protocol is used as the communication protocol between the smart mobile terminal 302 and the smart glasses 301
  • the cellular mobile communication protocol (such as 2G, 3G, 4G, 5G protocol, etc.) is used as the communication protocol between the cloud computer device 303 and the smart mobile terminal 302.
  • the real-time hearing aid and real-time translation functions based on smart glasses are realized, thereby expanding the functions of smart glasses and improving product stickiness.
  • the weight of smart glasses can be reduced, power consumption can be reduced, and the manufacturing cost of smart glasses can be reduced.
  • the microphone on one side obtains the voice data to be amplified from a specific direction and the loudspeaker on the other side plays the amplified voice data, the echo can be reduced, the quality of the played voice data and the voice data to be translated can be improved, and the translation efficiency can be improved.
  • FIG. 6 is a schematic diagram of the implementation process of a hearing aid method based on smart glasses provided in an embodiment of the present application.
  • the method can be applied to the smart glasses shown in FIG. 1 to FIG. 3 .
  • the method includes the following steps:
  • S503 Play the amplified voice data through a target speaker in the smart glasses, wherein the target speaker and the at least one microphone are respectively arranged on different temples of the smart glasses.
  • At least one microphone in the smart glasses is used to obtain first voice data from a first preset direction, and the first voice data is amplified by a wireless communication module in the smart glasses and the amplified first voice data is sent to a target speaker among the multiple speakers for playback, wherein the target speaker and the at least one microphone are respectively arranged on different temples, thereby realizing a real-time hearing aid function based on the smart glasses, thereby expanding the function of the smart glasses and improving product stickiness.
  • the microphone on one side obtains the voice data to be amplified from a specific direction and the speaker on the other side plays the amplified voice data, the echo can be reduced and the quality of the played sound can be improved.
  • the smart glasses have an idle state and a working state that can be switched freely, wherein the smart glasses in the working state also include three working modes: a hearing aid mode, a call mode, and a translation mode.
  • a hearing aid mode the user can use the telephone call function of the smart glasses to have a telephone conversation with the person on the other end of the network through a mobile communication terminal and a wireless communication network.
  • the hearing aid mode the user can use the hearing aid function of the smart glasses to have a face-to-face conversation with other people.
  • the translation mode the user can use the translation function of the smart glasses for real-time translation, which can help the user better understand the voice content from speakers of different languages.
  • the method further includes the following steps:
  • the wireless communication module responds to the first switching instruction to switch the working mode of the smart glasses to the hearing aid mode.
  • Step S502 amplifying the first voice data by the wireless communication module in the smart glasses to obtain amplified voice data specifically includes:
  • the first voice data is amplified by the wireless communication module to obtain the amplified voice data.
  • a hearing aid function can be further automatically provided for a specific user, and the output of the second voice data includes: determining whether the currently logged-in user is the target user; if it is the target user, amplifying the second voice data through the wireless communication module, and outputting the amplified second voice data through the target speaker; if it is not the target user, outputting the second voice data through the target speaker.
  • the user before using the functions provided by the smart glasses, the user can register a corresponding user account through the smart mobile terminal, and after logging into the operating system of the smart glasses through the smart mobile terminal using the user account, set whether the user account is used by a specific user with hearing aid needs. If the user selects that the user account is used by a specific user with hearing aid needs, the smart mobile terminal adds a corresponding mark to the user account according to the user's setting operation.
  • the smart glasses Each time after the smart glasses complete Bluetooth pairing with the smart mobile terminal, they send a query request to the smart mobile terminal to confirm whether the currently logged-in user is the target user, and save the query result locally. Each time before the smart glasses output the second voice data, they determine whether an additional hearing aid function is required based on the query result saved locally, that is, whether the translated voice data needs to be amplified. In response to the query request, the smart mobile terminal confirms whether the currently logged-in user is the target user based on the above-mentioned mark, and returns the confirmation result as the query result to the smart glasses.
  • the method further includes the following steps:
  • S602 in the translation mode, obtaining first voice data from a first preset direction through the at least one microphone;
  • the voice signal is picked up by the smart glasses from the microphone, and the smart glasses process the voice signal locally and transmit it to the smart mobile terminal.
  • the smart mobile terminal can use a local translation engine to translate the first voice data into the second voice data in the target language.
  • an application built into the smart mobile terminal uses a cloud-based AI (Artificial Intelligence) system to translate the voice signal from the source language to the target language in real time, and sends the translated voice signal back to the smart glasses and plays it for the user.
  • AI Artificial Intelligence
  • the smart mobile terminal can choose to send the first voice data or the first text data corresponding to the first voice data to the cloud computer device for translation according to the preset conversion mark, receive the second text data in the target language returned by the cloud computer device, convert the second text data into the second voice data and send it to the smart glasses.
  • the smart mobile terminal may also display the second text data through a local display while sending the second voice data to the smart glasses, thereby helping the user to more intuitively understand the content of the other party's conversation in the form of graphics and/or text.
  • the method further includes the following steps:
  • the first preset direction points to the front of the user
  • the second preset direction points to the bottom of the user, that is, the direction of the user's mouth.
  • the audio beam points forward, and the microphone captures the speech signal from the front.
  • the speech signal is captured from the direction of the user's mouth, that is, the microphone captures the speech signal downward.
  • the user can trigger the above switching instruction by using an input device configured on the smart glasses, such as a physical button or a virtual button (touch sensor).
  • an input device configured on the smart glasses, such as a physical button or a virtual button (touch sensor).
  • the smart glasses can be provided with a plurality of state control buttons corresponding to different operating states and working modes, and the smart glasses respond to the switching instruction triggered by the control operation of the user clicking the button through the wireless communication module, and control the smart glasses to enter the operating state or working mode corresponding to the clicked button.
  • any one of the hearing aid mode and the translation mode can be selected as the commonly used listening mode.
  • the above-mentioned physical buttons or virtual buttons (touch sensors) can be used to manually switch between the call mode and the listening mode. For example, if the smart glasses are in a call mode, pressing the physical button will change the smart glasses from the call mode to the listening mode, and pressing the physical button again will change it back to the call mode, thereby reducing the number of buttons, reducing manufacturing costs, and improving the intelligence of the operation, making the operation more in line with the user's usage habits.
  • the wireless communication module responds to the switching instruction triggered by the target control operation to control the switching of the working mode of the smart glasses to a call mode or a listening mode, wherein the specific form of the target control operation can be customized by the user. For example, the action of long pressing the touch sensor or a preset sliding gesture can be set as the target control operation.
  • the DSP in the wireless communication module can execute various corresponding algorithms on the voice data.
  • the smart glasses perform voice equalization processing and output volume processing on the data of the downlink channel through the DSP, and perform echo cancellation processing, first beamforming processing and noise suppression processing on the data of the uplink channel in sequence.
  • the data of the downlink channel is input through the wireless signal transceiver in the wireless communication module.
  • the wireless signal transceiver preferably uses the Bluetooth protocol as the communication protocol.
  • the smart glasses use the DSP to perform voice equalization processing and output volume control processing on the call voice data from the smart mobile terminal input through the wireless signal transceiver (such as the smart glasses wireless Bluetooth input in Figure 9), and send the call voice data after the output volume control processing to the speaker of the smart glasses for output.
  • the DSP performs echo cancellation processing on the voice data obtained by the sound pickup device, using the call voice data after the output volume processing as a reference signal, and then performs beamforming processing and noise suppression processing, and outputs the data after the noise suppression processing to the smart mobile terminal through the wireless signal transceiver.
  • the echo cancellation processing of the voice data of the uplink channel uses the echo cancellation algorithm to compare the output signal of the speaker with the input signal of the microphone, thereby eliminating the echo and interrupting the loop chain between the speaker and the microphone.
  • the noise suppression processing for the voice data of the uplink channel uses a noise suppression algorithm to reduce or eliminate the volume of the noise and amplify the volume of the other party's speech at the same time.
  • a noise suppression algorithm to reduce or eliminate the volume of the noise and amplify the volume of the other party's speech at the same time.
  • the voice equalization processing performed on the voice data of the downlink channel is to use a voice equalizer to perform voice equalization processing on the distant voice signal to strengthen the frequency signal of the user's weak hearing, thereby achieving the purpose of compensating the weak hearing frequency signal.
  • the output volume control processing of the voice data of the downlink channel is to adjust the output volume of the speaker using the output volume control algorithm.
  • the wireless signal transceiver adopts at least one of the Bluetooth protocol, Wireless Fidelity protocol, Near Field Communication protocol, ZigBee, Digital Living Network Alliance protocol, carrier-free communication protocol, radio frequency identification protocol and cellular mobile communication protocol as a communication protocol for data interaction with the smart mobile terminal.
  • the method before acquiring the first voice data or the third voice data through the sound pickup device, the method further includes:
  • the wireless communication module is used to perform Bluetooth pairing with the smart mobile terminal based on the Bluetooth protocol to establish a data transmission channel between the smart glasses and the smart mobile terminal. Subsequent data interaction between the smart glasses and the smart mobile terminal can be carried out through the data transmission channel.
  • the smart glasses use the DSP to perform feedback cancellation processing, voice activity detection processing, noise suppression processing, voice equalization processing, and user speech detection processing on the voice data, and send the voice data after voice equalization processing to the speakers of the smart glasses for output.
  • the voice data after voice equalization processing is used as reference data for the feedback cancellation processing.
  • the voice data after voice equalization processing can also be processed for output gain and output volume control.
  • the specific processing method is the same as the output volume control processing method shown in FIG9 . For details, please refer to the relevant description of FIG9 above, which will not be repeated here.
  • Feedback cancellation processing is to use the feedback cancellation algorithm to eliminate the echo by comparing the output signal of the speaker and the input signal of the microphone array, and interrupt the loop chain between the speaker and the microphone array.
  • the microphone obtains the voice signal in front, but because the general microphone is omnidirectional, the voice signal output by the speaker is also obtained by the microphone at the same time.
  • the voice signal of the speaker is also fed back to the input of the adaptive filter through the internal connection.
  • the adaptive filter adjusts the LMS (least mean square) adaptive filter coefficient according to the previous speaker output data, thereby deleting the signal output by the speaker (i.e., f(n)) from the microphone input.
  • Noise suppression processing uses a noise suppression algorithm to reduce or eliminate the noise volume and amplify the other party's voice volume at the same time.
  • the algorithm specifically divides the voice signal of each frame into speaking and silent. If the frame is detected as a silent frame, the algorithm uses the voice signal of this frame to update the average noise energy value. If the frame is defined as a speaking frame, the voice signal of this frame will be subtracted from the average noise energy value to achieve noise suppression.
  • Voice equalization processing is to use a voice equalizer to strengthen the sound signal of the specific frequency, so as to achieve the purpose of compensating the sound signal of the specific frequency.
  • the microphone array of the smart glasses is very close to the user's mouth, when the user speaks, the microphone array will receive a large signal and play it on the speaker of the smart glasses, so that when the user speaks, he will hear his own voice through the speaker.
  • User voice detection processing uses the user voice detection algorithm to continuously detect the signal received by the microphone array and perform average energy estimation analysis and detection. If the average energy estimation value is lower than the preset average energy threshold, the signal is detected as the user's voice.
  • the voice activity detection process is similar to the user voice detection process, except that the average energy threshold is lower than that of the user voice detection algorithm, thereby detecting the voice of the other party speaking.
  • an independent playback device (such as an external speaker, an external hearing aid) can also be used to play the voice data
  • the above-mentioned step of outputting the amplified voice data or the second voice data may specifically include: determining whether there is a playback device that establishes a data connection with the smart glasses through the wireless communication module; if there is, sending the amplified voice data or the second voice data to the playback device through the wireless communication module, so that the amplified voice data or the second voice data is played through the playback device; if not, outputting the amplified voice data or the second voice data through the target speaker.
  • the wireless communication module of the smart glasses can determine whether there is a playback device that establishes a data connection with the smart glasses based on the local Bluetooth pairing log and the connection status of the peripheral interface.

Landscapes

  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Acoustics & Sound (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • General Physics & Mathematics (AREA)
  • Ophthalmology & Optometry (AREA)
  • Optics & Photonics (AREA)
  • Neurosurgery (AREA)
  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

一种用于助听的智能眼镜、助听方法及系统，其中该智能眼镜包括：两条镜腿、拾音装置、多个扬声器以及无线通信模块，该拾音装置包括用于获取来自第一预设方向的第一语音数据的至少一个麦克风，该无线通信模块用于将第一语音数据进行放大以得到放大后的语音数据，并将放大后的语音数据发送给目标扬声器以进行播放，其中目标扬声器与至少一个麦克风分别设置在不同的镜腿上。本申请实现了基于智能眼镜的助听功能，可增加产品粘度。

Description

用于助听的智能眼镜、助听方法以及辅助系统
本申请要求于2023年5月31日提交至中国国家知识产权局专利局、申请号为CN 202310639291.8、名称为“用于助听的智能眼镜、助听方法以及辅助系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及智能眼镜技术领域，尤其涉及一种用于助听的智能眼镜、助听方法以及辅助系统。
 背景技术
随着计算机技术的发展，智能眼镜越来越普及，然而现有的智能眼镜价格昂贵，且除了其作为智能眼镜的本身的功能之外，通常仅具有听音乐和拨打或接听电话的功能，功能较为单一，产品粘度较低。
 技术问题
本申请实施例提供一种用于助听的智能眼镜、助听方法以及辅助系统，用于实现基于智能眼镜的实时助听功能，从而可增加产品粘度。
 技术解决方案
本申请实施例一方面提供了一种用于助听的智能眼镜，包括：两条镜腿、拾音装置、多个扬声器以及无线通信模块；
所述多个扬声器以及所述拾音装置设置在所述两条镜腿中的至少一条镜腿上，所述无线通信模块设置在所述两条镜腿中的任意一条镜腿的腔体内并与所述拾音装置以及所述多个扬声器电性连接，所述拾音装置包括至少一个麦克风；
所述至少一个麦克风，用于获取来自第一预设方向的第一语音数据；
所述无线通信模块，用于将所述第一语音数据进行放大以得到放大后的语音数据，并将放大后的语音数据发送给所述多个扬声器中的目标扬声器以进行播放，其中所述目标扬声器与所述至少一个麦克风分别设置在不同的镜腿上。
本申请实施例一方面还提供了一种基于智能眼镜的助听方法，所述智能眼镜包括：两条镜腿、无线通信模块以及与所述无线通信模块电性连接的拾音装置和多个扬声器，所述拾音装置包括至少一个麦克风，所述方法包括：
通过所述至少一个麦克风获取来自第一预设方向的第一语音数据；
通过所述无线通信模块将所述第一语音数据进行放大以得到放大后的语音数据；
通过所述多个扬声器中的目标扬声器播放所述放大后的语音数据，其中所述目标扬声器与所述至少一个麦克风分别设置在不同的镜腿上。
本申请实施例一方面还提供了一种辅助系统，包括：智能眼镜以及智能移动终端；
所述智能眼镜包括：两条镜腿、拾音装置、多个扬声器以及无线通信模块，其中，所述多个扬声器以及所述拾音装置设置在所述两条镜腿中的至少一条镜腿上，所述无线通信模块设置在所述两条镜腿中的任意一条镜腿的腔体内并与所述拾音装置以及所述多个扬声器电性连接，所述拾音装置包括至少一个麦克风；
所述无线通信模块，用于响应于用户的控制操作，控制切换所述智能眼镜的工作模式，所述工作模式包括：助听模式以及翻译模式；
所述至少一个麦克风，用于在所述助听模式或所述翻译模式下，获取来自第一预设方向的第一语音数据；
所述无线通信模块，用于在所述助听模式下，将所述第一语音数据进行放大以得到放大后的语音数据，并将放大后的语音数据发送给所述多个扬声器中的目标扬声器以进行播放，其中所述目标扬声器与所述至少一个麦克风分别设置在不同的镜腿上；
所述无线通信模块，还用于在所述翻译模式下，将所述第一语音数据发送给智能移动终端；
所述智能移动终端，用于接收所述第一语音数据，并将第二语音数据发送给所述无线通信模块，其中所述第二语音数据由所述第一语音数据翻译得到；
所述无线通信模块，还用于接收所述智能移动终端发送的第二语音数据，并将所述第二语音数据发送给所述目标扬声器以进行播放。
 有益效果
本申请各实施例，通过利用智能眼镜中的至少一个麦克风获取来自第一预设方向的第一语音数据，利用该智能眼镜中的无线通信模块将该第一语音数据进行放大并将放大后的第一语音数据发送给该多个扬声器中的目标扬声器以进行播放，其中该目标扬声器与该至少一个麦克风分别设置在不同的镜腿上，实现了基于智能眼镜的实时助听功能，从而可扩大智能眼镜的功能，提高产品粘性，同时，由于是由一侧的麦克风从特定方向获取待放大的语音数据，由另一侧的扬声器播放放大后的语音数据，因此可还降低回声，提高播放的声音的质量。
 附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请一实施例提供的智能眼镜的结构示意图；
图2为本申请另一实施例提供的智能眼镜的内部结构示意图；
图3为图2所示实施例中的智能眼镜的外部结构示意图；
图4为本申请一实施例提供的辅助系统的示意图；
图5为图4所示的辅助系统中的智能移动终端的硬件结构示意图；
图6为本申请一实施例提供的基于智能眼镜的助听方法的实现流程示意图；
图7为本申请实施例提供的基于智能眼镜的助听方法中不同工作模式下麦克风的声束指向示意图；
图8为本申请实施例提供的基于智能眼镜的助听方法中智能眼镜的运行状态和工作模式的示意图；
图9为本申请实施例提供的基于智能眼镜的助听方法中通话模式下语音信号的处理过程的示意图；
图10为本申请实施例提供的基于智能眼镜的助听方法中助听模式下语音信号的处理过程的示意图；
图11为图10中的反馈取消处理的示意图。
 本发明的实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
参见图1，图1为本申请一实施例提供的智能眼镜的结构示意图。为了便于说明，图中仅示出了与本申请实施例相关的部分。如图1所示，智能眼镜10包括：无线通信模块101、拾音装置102、多个扬声器103以及两条镜腿104。无线通信模块101与拾音装置102和多个扬声器103电性连接。
多个扬声器103以及拾音装置102设置在两条镜腿104中的至少一条镜腿上，无线通信模块101设置在两条镜腿104中的任意一条镜腿的腔体内并与拾音装置102以及多个扬声器103电性连接。
拾音装置102包括至少一个麦克风。可选的，拾音装置102包括至少一个定向麦克风。进一步的，拾音装置102还可以包括至少一个全向麦克风。例如，拾音装置102可以是由至少一个定向麦克风和/或至少一个全向麦克风构成的麦克风阵列。
进一步的，无线通信模块101可以但不限于包括：控制器1011、语音数据处理器1012以及无线信号收发器1013。其中，控制器1011主要用于UI（User Interface，用户界面）、任务操作和实时操作系统的操作。语音数据处理器1012主要用于执行较为耗时的信号处理算法和蓝牙无线通信算法的运算。无线信号收发器具体例如可以为无线信号收发器。
该无线信号收发器可以但不限于使用蓝牙协议、WiFi（无线保真）协议、NFC（Near Field Communication，近场通信）协议、ZigBee（紫蜂）、DLNA（DIGITAL LIVING NETWORK ALLIANCE，数字生活网络联盟）协议、UWB（Ultra Wideband，无载波通信）、RFID（Radio Frequency Identification，射频识别）协议以及蜂窝移动通信（Cellular Mobile Communication）协议中的至少一种进行数据传输。
具体的，至少一个麦克风，用于获取来自第一预设方向的第一语音数据。
无线通信模块101，用于将该第一语音数据进行放大以得到放大后的语音数据，并将放大后的语音数据发送给多个扬声器103中的目标扬声器以进行播放，其中目标扬声器与至少一个麦克风分别设置在不同的镜腿上。
于本实施例中，通过利用智能眼镜中的至少一个麦克风获取来自第一预设方向的第一语音数据，利用该智能眼镜中的无线通信模块将该第一语音数据进行放大并将放大后的第一语音数据发送给该多个扬声器中的目标扬声器以进行播放，其中该目标扬声器与该至少一个麦克风分别设置在不同的镜腿上，实现了基于智能眼镜的实时助听功能，从而可扩大智能眼镜的功能，提高产品粘性。并且，由于无需在智能眼镜上加装额外的助听设备，还可减轻智能眼镜的重量，减小耗电量，降低智能眼镜的制造成本。同时，由于是由一侧的麦克风从特定方向获取待放大的语音数据，由另一侧的扬声器播放放大后的语音数据，因此可还降低回声，提高播放的声音的质量。
参见图2和图3，图2为本申请另一实施例提供的智能眼镜的内部结构示意图，图3为图2所示实施例中的智能眼镜的外部结构示意图。如图2和图3所示，智能眼镜20包括：镜框201、两条镜腿202、至少一个拾音装置203（为便于理解，图中仅示出了一个）、第一扬声器205、第二扬声器206以及无线通信模块204。
其中，镜腿202连接镜框201，第一扬声器205、第二扬声器206以及至少一个拾音装置203设置在两条镜腿202中的至少一条镜腿202上，无线通信模块204设置在两条镜腿202中的任意一条镜腿202的腔体内并与第一扬声器205、第二扬声器206以及至少一个拾音装置203电性连接。
拾音装置203，用于获取来自第一预设方向的第一语音数据。
无线通信模块204，用于将该第一语音数据进行放大以得到放大后的语音数据，并将放大后的语音数据发送给第一扬声器205和第二扬声器206中的目标扬声器以进行播放，其中该目标扬声器与拾音装置203分别设置在不同的镜腿上。
无线通信模块204，还用于控制切换智能眼镜的工作模式，该工作模式包括通话模式、助听模式以及翻译模式。
拾音装置203包括：至少一个定向麦克风和/或至少一个全向麦克风。当拾音装置203为全向麦克风时，无线通信模块204，还用于在通话模式下对至少一个拾音装置203获取的语音数据进行第一波束成形处理，以使得至少一个拾音装置203的声束收音指向下方，即与用户的嘴对应。此时，该语音数据的音源可以是智能眼镜的佩戴者；在该助听模式和该翻译模式下对至少一个拾音装置203获取的语音数据进行第二波束成形处理，以使得至少一个拾音装置203的声束收音指向前方。此时，该语音数据的音源可以是智能眼镜的佩戴者的交谈对象。无线通信模块204可在切换工作模式后，根据切换后的工作模式，自动控制全向麦克风指向对应的方向。
可选的，两条镜腿202包括第一镜腿202A和第二镜腿202B，第一镜腿202A的前端和第二镜腿202B的前端分别连接镜框201的两侧，至少一个拾音装置203安装在第一镜腿202A的前端。
第一扬声器205和第二扬声器206用于输出语音数据或音乐数据。该语音数据包括：经无线通信模块204处理后的拾音装置203获取的语音数据，以及无线通信模块204接收的智能移动终端发送的通话语音数据。该音乐数据包括：无线通信模块204接收的智能移动终端发送的音乐数据。
优选的，第一扬声器205安装在第一镜腿202A上，且第一扬声器205的输出口位于第一镜腿202A的尾端。
第二扬声器206安装在第二镜腿202B上，且第二扬声器206的输出口位于第二镜腿202B的尾端。
在助听模式和翻译模式下，仅有位于一条镜腿上的开放式扬声器用于语音数据的输出，从而可达到降低回声的效果。
例如，在该助听模式下，如果智能眼镜的拾音装置位于左眼镜腿上，则放大后的语音信号仅输出到配置在右眼镜腿的扬声器进行播放。如果该拾音装置位于右眼镜腿上，则放大后的语音将输出到配置在左眼镜腿的扬声器进行播放。
可选的，于本申请其他一实施例中，每一条镜腿上均设置有麦克风和扬声器时，每一个麦克风和每一个扬声器都具有各自对应的唯一身份标识信息，无线通信模块204中存储有每一个麦克风和每一个扬声器的唯一身份标识信息及其所处位置之间的对应关系。无线通信模块204，还用于根据该对应关系确定获取该第一语音数据的麦克风的安装位置，并根据该安装位置以及该对应关系确定该目标麦克风。
可以理解的，对于助听模式，智能眼镜可以独立放大佩戴者前方的语音信号，而无需连接任何移动终端设备。
由于拾音装置安装在镜腿的前端，各扬声器的输出口安装在镜腿的尾端，从而使得拾音装置和各扬声器的输出口之间可以具有足够的距离，因此可有效减少在拾音装置和扬声器之间生成回路的情况，降低在使用时发生回音和啸叫的机率。
此外，在用户佩戴智能眼镜后，镜腿的尾端最接近用户的耳朵，将扬声器出口安装在镜腿的尾端，可使得扬声器出口最接近用户耳朵，从而提高声音输出的效率。
其中，第一扬声器205和第二扬声器206优选为单声道扬声器。分别安装在两条镜腿上的单声道扬声器结合起来可达到立体声音效。
可选的，无线通信模块204包括：控制器2041、语音数据处理器2042以及无线信号收发器2043。
其中，控制器2041，用于控制切换智能眼镜的工作模式。控制器2041优选为MCU（Microcontroller Unit，微控制单元）。
语音数据处理器2042，用于对语音数据进行处理。语音数据处理器2042优选为DSP（Digital Signal Processing，数字信号处理器）或语音数据处理集成电路。其中，语音数据处理集成电路为常用电路，本申请不对其结构做具体限定。
于本申请其他一实施例中，拾音装置203优选为定向麦克风，用于定向拾取语音信号，然后发送到MCU/DSP进行语音处理。
进一步的，拾音装置102还包括用于拾取来自第二预设方向的语音信号的两个麦克风。用户可通过智能眼镜上的这两个麦克风拾取来自第二预设方向的语音信号，用作发出语音命令和接听电话。采用两个麦克风拾音可以准确确定拾音的方向，并对两个麦克风的拾音数据进行处理可以实现降噪。该语音信号被从智能眼镜的两个麦克风拾取，通过降躁后的语音被在智能眼镜的扬声器播放并发出声音。
无线信号收发器2043，用于与智能移动终端进行数据交互。
可选的，语音数据处理器2042包括语音均衡器（Equalizer）。无线信号收发器2043还用于接收智能移动终端发送的音量调节控制指令并发送给语音数据处理器2042，语音数据处理器2042还用于通过利用该语音均衡器，将待输出的声音数据，调整为该音量调节控制指令指向的频段及音量的声音数据，并将调整后的声音数据发送给该音量调节控制指令指向的扬声器。其中，该音量调节控制指令指向的频段可以是低频、中频或高频。音量调整可以是增大音量，也可以是减小音量。
进一步的，语音数据处理器2042，还用于在通话模式下，对下行通道上的数据进行语音均衡处理（利用语音均衡器（Equalizer））和输出音量处理，以及利用预设的回声消除算法(Acoustic Echo Cancellation)、波束成形算法（Beamforming）以及噪声抑制算法（Noise Cancellation）对上行通道上的数据进行回音消除处理、波束成形处理以及噪声抑制处理。
进一步的，语音数据处理器2042，还用于在助听模式或翻译模式下，利用预设的反馈取消算法（Feedback Cancellation）、语音活动检测处理、噪声抑制算法、语音均衡器算法（Equalizer）以及用户说话检测算法，对该语音数据进行反馈取消处理、语音活动检测处理、噪声抑制处理、语音均衡处理以及用户说话检测处理，并将语音均衡处理后的语音数据发送给对应的扬声器进行输出。同时，该语音均衡处理后的语音数据在处理过程中还用作上述反馈取消处理的参考数据。
可选的，智能眼镜还包括：与无线通信模块204电性连接的至少一个传感器207。
至少一个传感器207安装在第一镜腿102A和/或第二镜腿102B的内侧和/或外侧。
具体的，至少一个传感器207可以但不限于包括：触控传感器、接近传感器、加速度计、陀螺仪、磁感应传感器以及惯性测量单元中的至少一种。
可选的，惯性测量单元为9轴传感器。该9轴传感器用于采集用户的运动数据（3轴加速度计数据，3轴陀螺仪数据，3轴磁力计数据），并通过无线通信模块204发送给智能移动终端，以用于用户的头部和身体的状态监控。
优选的，至少一个传感器207包括：9轴传感器、至少一个触控传感器和至少一个接近传感器。
其中，至少一个触控传感器安装在第一镜腿102A和/或第二镜腿102B的外侧。
可选的，至少一个触控传感器用于检测用户的第一控制操作，并将检测到的该第一控制操作的数据发送给控制器2041，该第一控制操作用于调整音量。控制器2041，还用于根据该第一控制操作的数据，响应于第一控制操作，控制调整智能眼镜输出的声音的音量。
其中，第一控制操作包括用于调大音量的控制操作和用于调小音量的控制操作。该用于调大音量的控制操作与用户手指在触控传感器上向耳方向扫动的动作对应，该用于调小音量的控制操作与用户手指在触控传感器上向镜框方向（即离开耳方向）扫动的动作对应。
可选的，至少一个触控传感器，还用于作为输入装置检测用户的第二控制操作，并将检测到的该第二控制操作的数据发送给控制器2041。控制器2041还用于根据该第二控制操作的数据，响应于该第二控制操作，控制切换智能眼镜的工作模式。
可选的，至少一个接近传感器安装在第一镜腿202A和/或第二镜腿202B的内侧，用于检测用户是否佩戴或摘下智能眼镜以及获取用户未佩戴智能眼镜的时长，并将检测结果发送给控制器2041。控制器2041，用于根据该检测结果，控制音乐的播放或暂停以及控制智能眼镜20电源的开启或关闭。例如，在该接近传感器检测到用户佩戴智能眼镜时播放音乐数据，在该接近传感器检测到用户摘下智能眼镜时停止播放该音乐数据，以及在该接近传感器检测到用户超过预设时长没有佩戴智能眼镜时执行关机操作。
上述每一种传感器的数量优选为一个，以减轻智能眼镜的总体重量。但在具体应用中，根据实际需要，各种传感器的数量也不限为一个，例如，为提高检测结果的准确性，可以分别在两条镜腿各设置一个接近传感器。
可选的，控制器2041还用于将各传感器获取的数据，通过上述无线信号收发器发送给智能移动终端。
可选的，拾音装置203可包括第一定向麦克风和第二定向麦克风。第一定向麦克风用于获取用户前方的语音数据，第二定向麦克风用于获取用户下方的语音数据。第一定向麦克风和第二定向麦克风可分别位于镜腿的不同位置。进一步的，控制器2041还可根据智能眼镜20的工作模式，控制对应位置的定向麦克风进行拾音，从而可减少获取的语音数据中的杂音，提高信号处理的速度。
可选的，智能眼镜还包括电池208，电池208安装在第一镜腿202A上且与无线通信模块204电性连接，以用于为智能眼镜上的无线通信模块204、各传感器、各扬声器以及各麦克风等所有硬件组件提供电能。
可选的，智能眼镜还包括：与无线通信模块204电性连接的输入装置。该输入装置设置在镜框201以及两条镜腿202中的至少任意一个上。
可选的，该输入装置包括：触控传感器以及至少一个控制按键（图中未示出）中的至少一种。该至少一个控制按键安装在第一镜腿202A和/或第二镜腿202B的外侧，并与无线通信模块204电性连接。该至少一个控制按键，用于触发无线通信模块204控制切换智能眼镜的工作模式或运行状态。其中，该运行状态包括空闲状态和工作状态，该工作状态包括该通话模式、该助听模式以及该翻译模式。
具体的，无线通信模块204，还用于响应于用户基于该输入装置触发的第一切换指令将智能眼镜的工作模式切换为助听模式，以及在助听模式下，将来自第一预设方向的第一语音数据进行放大。
无线通信模块204，还用于响应于用户基于该输入装置触发的第二切换指令将智能眼镜的工作模式切换为翻译模式，以及在翻译模式下，将该第一语音数据发送给智能移动终端，接收智能移动终端发送的第二语音数据，并将该第二语音数据发送给目标扬声器以进行播放，其中该第二语音数据由第一语音数据翻译得到。
无线通信模块204，还用于响应于用户基于该输入装置触发的第三切换指令将智能眼镜的工作模式切换为通话模式，在该通话模式下，将麦克风获取的来自第二预设方向的第三语音数据发送给目标终端，接收目标终端发送的第四语音数据，并输出给所有的扬声器以进行播放。
其中，当智能眼镜的用户利用智能眼镜直接与其他终端的用户进行通话时，该目标终端是该其他终端。当智能眼镜的用户利用智能眼镜藉由智能移动终端与其他终端的用户进行通话时，该目标终端是该智能移动终端。
上述第一预设方向指向用户的前方，第二预设方向指向用户的下方。
上述智能眼镜的各电子组成元器件之间可通过总线连接。
需要说明的是，上述智能眼镜的各组件彼此之间的关系可以是替代关系，也可以是叠加关系。即，一个智能眼镜上可以安装上述本实施例中的所有组件，或者，也可以根据需求，选择性地安装上述组件中的一部分。当为替代关系时，智能眼镜还设置有外设的连接接口，该连接接口例如可以是PS/2接口、串行接口、并行接口、IEEE1394接口、USB（Universal Serial Bus，通用串行总线）接口等中的至少一种，被替代的组件的功能可通过连接在该连接接口上的外设实现，如：外部扬声器、外部传感器等。
控制器2041还用于确定是否存在与该智能眼镜建立数据连接的播放设备，如外部扬声器、外部助听器；若存在，则将该第二语音数据或放大后的第一语音数据发送给该播放设备，以通过该播放设备进行播放；若不存在，则通过本地的扬声器输出该第二语音数据或放大后的第一语音数据。像这样，在有独立的播放设备时，利用独立的播放设备播放语音数据，可以进一步降低智能眼镜的功耗。
于本实施例中，通过利用智能眼镜中的至少一个麦克风获取来自第一预设方向的第一语音数据，利用该智能眼镜中的无线通信模块将该第一语音数据进行放大并将放大后的第一语音数据发送给该多个扬声器中的目标扬声器以进行播放，其中该目标扬声器与该至少一个麦克风分别设置在不同的镜腿上，实现了基于智能眼镜的实时助听功能，从而可扩大智能眼镜的功能，提高产品粘性。并且，由于无需在智能眼镜上加装额外的助听设备，还可减轻智能眼镜的重量，减小耗电量，降低智能眼镜的制造成本。同时，由于是由一侧的麦克风从特定方向获取待放大的语音数据，由另一侧的扬声器播放放大后的语音数据，因此可还降低回声，提高播放的声音的质量。此外，通过智能眼镜与智能移动终端的交互，还实现了基于智能眼镜的实时翻译功能，进一步扩大智能眼镜的功能，并且由于是从特定方向获取待翻译的语音数据，因此可提高待翻译的语音数据的质量，进而提高翻译效率。
参见图4，图4为本申请一实施例提供的辅助系统的结构示意图。如图4所示，该辅助系统包括：智能眼镜301、智能移动终端302以及云端计算机设备303。
智能眼镜包括301：两条镜腿、拾音装置、多个扬声器以及无线通信模块，该拾音装置包括至少一个麦克风。
该无线通信模块，用于响应于用户的控制操作，控制切换该智能眼镜的工作模式，该工作模式包括：助听模式、翻译模式以及通话模式。
该至少一个麦克风，用于在该助听模式或该翻译模式下，获取来自第一预设方向的第一语音数据；
该无线通信模块，用于在该助听模式下，将该第一语音数据进行放大以得到放大后的语音数据，并将放大后的语音数据发送给该多个扬声器中的目标扬声器以进行播放，其中该目标扬声器与该至少一个麦克风分别设置在不同的镜腿上。
该无线通信模块，还用于在该翻译模式下，将该第一语音数据发送给智能移动终端，接收该智能移动终端发送的第二语音数据，并将该第二语音数据发送给该目标扬声器以进行播放，其中该第二语音数据由该第一语音数据翻译得到。
该至少一个麦克风，还用于在该通话模式下，获取来自第二预设方向的第三语音数据。
该无线通信模块，还用于将该第三语音数据发送给目标终端，接收该目标终端发送的第四语音数据。
该多个扬声器，还用播放该第四语音数据。
智能眼镜301的结构及其功能具体请参考上述图1至图3所示实施例中的相关描述。
智能移动终端302可以但不限于包括：蜂窝电话、智能手机、其他无线通信设备、个人数字助理、音频播放器、其他媒体播放器、音乐记录器、录像机、照相机、其他媒体记录器、智能收音机、膝上型计算机、个人数字助理（PDA）、便携式多媒体播放器（PMP）、运动图像专家组（MPEG-1或MPEG-2）音频层3（MP3）播放器、数码相机以及智能可穿戴设备（如智能手表、智能手环等）。智能移动终端302上安装有安卓或IOS操作系统。
智能移动终端302用于与智能眼镜301以及云端计算机设备303进行数据交互，具体的，例如：接收智能眼镜301发送的第一语音数据，在智能移动终端302本地将该第一语音数据翻译为目的语言的第二语音数据，或者，将该第一语音数据或该第一语音数据对应的第一文本数据发送给云端计算机设备303以进行翻译；接收云端计算机设备303返回的翻译后的目的语言的第二文本数据，将该第二文本数据转换为第二语音数据并发送给智能眼镜301。又例如，在执行播放音乐数据、通话等目标任务时，将播放的音乐数据、接收的通话语音数据发送给智能眼镜301等。智能移动终端302在与智能眼镜301进行数据交互时使用的通信协议与智能眼镜301使用的通信协议一致。
可选的，智能移动终端302根据预设的转换标记，确定是将该第一语音数据还是将该第一语音数据对应的第一文本数据发送给云端计算机设备303。
具体的，智能移动终端302，还用于获取预设的转换标记，若该预设的转换标记为第一标记，则将该第一语音数据转换为该第一文本数据，并将该第一文本数据发送给云端计算机设备303，若该预设的转换标记为第二标记，则直接将该第一语音数据发送给云端计算机设备303。
进一步的，智能移动终端302，还用于获取网速并确定该网速是否大于预设速度；若不大于该预设速度，则将该转换标记设置为该第一标记；若大于该预设速度，则将该转换标记设置为该第二标记。
进一步的，智能移动终端302，还用于响应于用户的设置操作，将该转换标记设置为该设置操作指向的标记。
如图5所示，智能移动终端302可以包括控制电路，该控制电路可以包括存储和处理电路300。该存储和处理电路300可以包括存储器，例如硬盘驱动存储器，非易失性存储器（例如闪存或用于形成固态驱动器的其它电子可编程限制删除的存储器等），易失性存储器（例如静态或动态随机存取存储器等）等，本申请实施例不作限制。存储和处理电路300中的处理电路可以用于控制智能移动终端302的运行。该处理电路可以基于一个或多个微处理器，微控制器，数字信号处理器，基带处理器，功率管理单元，音频编解码器芯片，专用集成电路，显示驱动器集成电路等来实现。
存储和处理电路300可用于运行智能移动终端302中的软件，例如：人工智能管家应用程序、互联网浏览应用程序，互联网协议语音（Voice over Internet Protocol，VOIP）电话呼叫应用程序，电子邮件应用程序，媒体播放应用程序，操作系统功能等。这些软件可以用于执行一些控制操作，例如，基于预设的分析算法对智能眼镜301发送的运动数据进行的数据处理和分析、基于照相机的图像采集，基于环境光传感器的环境光测量，基于接近传感器的接近传感器测量，基于诸如发光二极管的状态指示灯等状态指示器实现的信息显示功能，基于触摸传感器的触摸事件检测，与在多个（例如分层的）显示器上显示信息相关联的功能，与执行无线通信功能相关联的操作，与收集和产生音频信号相关联的操作，与收集和处理按钮按压事件数据相关联的控制操作，以及智能移动终端302中的其它功能等，本申请实施例不作限制。
进一步的，该存储器存储有可执行程序代码，与该存储器连接的处理器，调用该存储器中存储的该可执行程序代码，执行本申请以下方法实施例中的相关步骤，以实现本申请各实施例中的智能移动终端302的各项功能。
智能移动终端302还可以包括输入/输出电路420。输入/输出电路420可用于使智能移动终端302实现数据的输入和输出，即允许智能移动终端302从外部设备接收数据和也允许智能移动终端302将数据从智能移动终端302输出至外部设备。输入/输出电路420可以进一步包括传感器320。传感器320可以包括环境光传感器，基于光和电容的接近传感器，触摸传感器（例如，基于光触摸传感器和/或电容式触摸传感器，其中，触摸传感器可以是触控显示屏的一部分，也可以作为一个触摸传感器结构独立使用），加速度计，和其它传感器等。
输入/输出电路420还可以包括一个或多个显示器，例如显示器140。显示器140可以包括液晶显示器，有机发光二极管显示器，电子墨水显示器，等离子显示器，使用其它显示技术的显示器中一种或者几种的组合。显示器140可以包括触摸传感器阵列（即，显示器140可以是触控显示屏）。触摸传感器可以是由透明的触摸传感器电极（例如氧化铟锡（ITO）电极）阵列形成的电容式触摸传感器，或者可以是使用其它触摸技术形成的触摸传感器，例如音波触控，压敏触摸，电阻触摸，光学触摸等，本申请实施例不作限制。
智能移动终端302还可以包括音频组件360。音频组件360可以用于为智能移动终端302提供音频输入和输出功能。智能移动终端302中的音频组件360可以包括扬声器，拾音装置，蜂鸣器，音调发生器以及其它用于产生和检测声音的组件。
通信电路380可以用于为智能移动终端302提供与外部设备通信的能力。通信电路380可以包括模拟和数字输入/输出接口电路，和基于射频信号和/或光信号的无线通信电路。通信电路380中的无线通信电路可以包括射频收发器电路、功率放大器电路、低噪声放大器、开关、滤波器和天线。举例来说，通信电路380中的无线通信电路可以包括用于通过发射和接收近场连接电磁信号来支持近场通信（Near Field Communication，NFC）的电路。例如，通信电路380可以包括近场通信天线和近场通信收发器。通信电路380还可以包括蜂窝电话收发器和天线，蓝牙、WiFi、ZigBee、DLNA、UWB、RFID等无线局域网收发器电路和天线等。
智能移动终端302还可以进一步包括电池，电力管理电路和其它输入/输出单元400。输入/输出单元400可以包括按钮，操纵杆，点击轮，滚动轮，触摸板，小键盘，键盘，照相机，发光二极管和其它状态指示器等。
用户可以通过输入/输出电路420输入命令来控制智能移动终端302的操作，并且可以使用输入/输出电路420的输出数据以实现接收来自智能移动终端302的状态信息和其它输出。
进一步的，智能移动终端302，还用于与智能眼镜301基于蓝牙协议进行配对，并在配对成功后，将播放的音乐数据发送给智能眼镜301，以通过智能眼镜301对该音乐数据进行播放。
智能眼镜301，还用于在配对成功后，将待翻译的第一语音数据发送给智能移动终端302以进行翻译。
可选的，智能移动终端302，还用于通过配置在智能移动终端上的GPS模块获取GPS数据，并将获取的GPS数据发送给智能眼镜301，以用于智能眼镜301的定位。
可选的，智能移动终端302还用于实时接收并存储智能眼镜301发送的运动数据，根据该运动数据、该GPS数据，进行运动指标计算，并根据计算结果生成用于通知或提醒用户运动状态的实时语音数据，并将实时语音数据发送给智能眼镜301进行输出。其中，该实时语音数据包括运动指标计算结果的通知或提醒语音。
具体的，智能移动终端302上可安装有客户端程序，如人工智能管家App，通过该APP执行与智能眼镜301的数据交互操作，基于用户在该APP上的操作对智能眼镜301的参数进行配置，以及对智能眼镜301发送的运动数据的处理和分析操作，如：跑步指标计算、姿势监测和提醒等。
上述音乐数据和实时语音数据属于智能眼镜301的下行通道的数据，智能眼镜301通过语音数据处理器1042对接收的音乐数据和实时语音数据进行语音均衡处理和输出音量控制处理后发送给第一扬声器201和第二扬声器202进行输出。
具体的，运动数据包括智能眼镜301通过9轴传感器获取的数据（即，（即Ax，Ay，Az；Gx，Gy，Gz；Mx，My，Mz），运动指标为跑步指标，跑步指标包括：配速、距离、步数、头部左右平衡、步数、步距和步频。其中，9轴传感器即指加速度计、陀螺仪以及磁感应传感器。这三类传感器测量的数据在空间坐标系中都可以被分解为X，Y，Z三个方向轴的力，因此也称为3轴加速度计、3轴陀螺仪和3轴磁感应传感器。
智能移动终端302，还用于根据9轴传感器获取的运动数据和本地的GPS数据，执行指标计算、姿势监测和运动提醒操作，并将指标计算结果通过智能移动终端302的显示器进行实时显示。进一步的，智能移动终端302还可根据该指标计算结果生成语音提示信息，并将该语音提示信息通过蓝牙发送给智能眼镜301，以通过智能眼镜301实时语音通知或提醒用户。
可选的，智能移动终端302，还用于响应于智能眼镜301发送的语音指令，执行拨打、接听或挂断电话的操作，以及在通话过程中，将接收的通话语音数据发送给智能眼镜301，以通过智能眼镜301上的扬声器对该通话语音数据进行播放。
其中，该语音指令由智能眼镜301的语音数据处理器，通过利用预设的语音识别算法对智能眼镜301的拾音装置获取的语音数据进行语音指令识别处理得到。
可以理解的，本申请中的智能眼镜301的本体上设置有至少两个定向和/或全向麦克风，用户可通过智能眼镜301上的麦克风获取语音数据，用作发出语音命令和电话接听。用户发出的声音通过智能眼镜301上的麦克风收音并经过降噪处理后通过智能眼镜301的扬声器予以播放。因此，在通话期间，用户可以将智能移动终端302放在口袋中或桌子上，空出的双手可做其他用途，从而提高了电话接听的便捷性。
云端计算机设备303例如可以是云端服务器或服务器集群，用于与智能移动终端302进行数据交互，存储智能移动终端302和/或智能眼镜301发送的数据（如，智能眼镜获取的所有传感器数据），以及基于预设的处理逻辑对该数据进行处理。具体的，云端计算机设备303接收智能移动终端302发送的第一语音数据或该第一语音数据对应的第一文本数据，然后将第一语音数据或第一文本数据翻译为目的语言的第二文本数据，并返回给智能移动终端302。例如：假设源语言为中文，目的语言为英文，如果智能移动终端302发送的是中文的语音数据1，则将语音数据1先转换为中文的文本数据1，然后再将文本数据1翻译为英文的文本数据2并发送给智能移动终端302；如果智能移动终端302发送的是中文的语音数据1对应的文本数据1，则将文本数据1翻译为英文的文本数据2并发送给智能移动终端302。
源语言的识别操作可由智能移动终端302执行，或者也可由云端计算机设备303。如果源语言的识别操作由智能移动终端302执行，则智能移动终端302在发送第一语音数据时，一并将源语言和目的语言的标识信息发送给云端计算机设备303，以使得云端计算机设备303根据该表示信息确定源语言和目的语言。
可选的，源语言和目的语言也可由智能移动终端302基于用户在智能眼镜301或智能移动终端302的选择操作确定。
具体的，用户也可以通过利用智能眼镜301或智能移动终端302上的控制按键或触控传感器选择源语言和/或目的语言。假如用户是通过智能眼镜301进行的选择操作，则智能眼镜301在向智能移动终端302发送第一语音数据的同时，将用户选定的源语言和/或目的语言的标识信息发送给智能移动终端302。
在一实际应用中，在用户通过智能眼镜301进行选择操作时，智能眼镜301还可根据用户的操作，实时输出对应的语音提示，以帮助用户快速从多个备选语言中选定源语言和/或目的语言。
优选的，智能移动终端302与智能眼镜301之间使用蓝牙协议作为通信协议，云端计算机设备303与智能移动终端302之间使用蜂窝移动通信协议（如：2G、3G、4G、5G协议等）作为通信协议。
需要说明的是，本实施例中的智能眼镜301、智能移动终端302以及云端计算机设备303的功能的具体实现过程，还可以参考其他实施例中的相关描述。
于本实施例中，实现了基于智能眼镜的实时助听和实时翻译功能，从而可扩大智能眼镜的功能，提高产品粘性。并且，由于无需在智能眼镜上加装额外的助听设备，还可减轻智能眼镜的重量，减小耗电量，降低智能眼镜的制造成本。同时，由于是由一侧的麦克风从特定方向获取待放大的语音数据，由另一侧的扬声器播放放大后的语音数据，因此可还降低回声，提高播放的语音数据以及待翻译的语音数据的质量，进而提高翻译效率。
参见图6，图6为本申请一实施例提供的基于智能眼镜的助听方法的实现流程示意图。该方法可应用于如图1至图3所示的智能眼镜。如图6所示，该方法包括以下步骤：
S501、通过智能眼镜中的至少一个麦克风获取来自第一预设方向的第一语音数据；
S502、通过智能眼镜中的无线通信模块将该第一语音数据进行放大以得到放大后的语音数据；
S503、通过智能眼镜中的目标扬声器播放该放大后的语音数据，其中该目标扬声器与该至少一个麦克风分别设置在智能眼镜的不同镜腿上。
于实施例中，通过利用智能眼镜中的至少一个麦克风获取来自第一预设方向的第一语音数据，利用该智能眼镜中的无线通信模块将该第一语音数据进行放大并将放大后的第一语音数据发送给该多个扬声器中的目标扬声器以进行播放，其中该目标扬声器与该至少一个麦克风分别设置在不同的镜腿上，实现了基于智能眼镜的实时助听功能，从而可扩大智能眼镜的功能，提高产品粘性，同时，由于是由一侧的麦克风从特定方向获取待放大的语音数据，由另一侧的扬声器播放放大后的语音数据，因此可还降低回声，提高播放的声音的质量。
可选的，于本申请其他一实施例中，该智能眼镜具有可自由切换的空闲状态和工作状态，其中工作状态下该智能眼镜还包括助听模式、通话模式以及翻译模式三种工作模式。在通话模式下，用户可利用智能眼镜的电话通话功能，通过移动通信终端和无线通信网络，与该网络另一端的人进行电话交谈。在助听模式下，该用户可利用智能眼镜的助听功能与其他人进行面对面的交谈。在翻译模式下，该用户可利用智能眼镜的翻译功能进行实时翻译，从而可以帮助用户更好地理解来自不同语言使用者的语音内容。
进一步的，于本申请其他一实施例中，该方法还包括以下步骤：
通过该无线通信模块响应于第一切换指令，将该智能眼镜的工作模式切换为助听模式。
步骤S502、通过智能眼镜中的无线通信模块将该第一语音数据进行放大以得到放大后的语音数据具体包括：
在该助听模式下，通过该无线通信模块将该第一语音数据进行放大以得到该放大后的语音数据。
可选的，于本申请其他一实施例中，进一步地还可为特定用户自动提供助听功能，该输出该第二语音数据包括：确定当前登录的用户是否为目标用户；若是该目标用户，则通过该无线通信模块将该第二语音数据进行放大，并通过目标扬声器输出放大后的第二语音数据；若不是该目标用户，则通过目标扬声器输出该第二语音数据。
具体的，用户可在使用智能眼镜提供的各项功能之前，通过智能移动终端注册一个对应的使用账户，并在通过智能移动终端利用该使用账户登录智能眼镜的操作系统后，设置该使用账户是否为有助听需求的特定用户所使用。若该用户选择该使用账户为有助听需求的特定用户所使用，则智能移动终端根据该用户的设置操作为该使用账户添加对应的标记。
智能眼镜每次在与智能移动终端完成蓝牙配对后，向智能移动终端发送查询请求，以确认当前登录的用户是否为目标用户，并将查询结果保存在本地。智能眼镜每次在输出该第二语音数据之前，根据本地保存的查询结果确定是否需要附加助听功能，即，是否需要对翻译后的语音数据进行放大处理。智能移动终端响应于该查询请求，根据上述标记，确认当前登录的用户是否为目标用户，并将确认结果作为查询结果返回给智能眼镜。
进一步的，于本申请其他一实施例中，该方法还包括以下步骤：
S601、通过该无线通信模块响应于第二切换指令，将该智能眼镜的工作模式切换为翻译模式；
S602、在该翻译模式下，通过该至少一个麦克风获取来自第一预设方向的第一语音数据；
S603、通过该无线通信模块将该第一语音数据发送给智能移动终端，并接收该智能移动终端发送的第二语音数据，其中该第二语音数据由该第一语音数据翻译得到；
S604、通过该目标扬声器播放该第二语音数据。
具体的，对于实时语言翻译，语音信号由智能眼镜从麦克风拾取，智能眼镜在本地对语音信号进行处理并传输到智能移动终端。智能移动终端在接收该第一语音数据后，可以利用本地的翻译引擎将该第一语音数据翻译为目的语言的该第二语音数据。
或者，智能移动终端内置的应用程序利用基于云端的AI（Artificial Intelligence，人工智能）系统将该语音信号从源语言实时翻译成目的语言，并将翻译后的语音信号发送回智能眼镜并为用户播放。
具体的，智能移动终端可根据预设的转换标记，选择将该第一语音数据或该第一语音数据对应的第一文本数据发送给云端计算机设备以进行翻译，接收该云端计算机设备返回的目的语言的第二文本数据，将该第二文本数据转换为该第二语音数据并发送给该智能眼镜。
可选的，智能移动终端也可在将上述第二语音数据发送给智能眼镜的同时，将第二文本数据通过本地的显示器进行展示，从而以图形和/或文字的方式帮助用户更为直观地了解对方谈话的内容。
进一步的，于本申请其他一实施例中，该方法还包括以下步骤：
S701、通过该无线通信模块响应于第三切换指令，将该智能眼镜的工作模式切换为通话模式；
S702、在该通话模式下，通过该麦克风获取来自第二预设方向的第三语音数据，并通过该无线通信模块将该第三语音数据发送给目标终端；
S703、通过该无线通信模块接收该目标终端发送的第四语音数据，并通过该智能眼镜中的多个扬声器播放该第四语音数据。
其中，上述第一预设方向指向用户的前方，上述第二预设方向指向用户的下方，即用户嘴巴的方向。
换而言之，如图7所示，在翻译模式和助听模式下，音频波束指向前方，麦克风从前方捕获语音信号。在通话模式下，从用户的嘴巴所在的方向捕获语音信号，即麦克风向下捕获语音信号。
此外，在翻译模式和助听模式下，只有一条镜腿上的扬声器可用于输出语音数据，即一侧拾音一侧播音，从而可降低回声。而在通话模式下，所有镜腿上的扬声器均可用于输出语音数据。这样可以兼顾不同工作模式的播放需求，从而达到较好的播放效果。
可选的，用户可以通过利用配置在智能眼镜上的输入装置，如物理按键或虚拟按钮（触摸传感器）触发上述切换指令。例如，智能眼镜上可以设置有多个状态控制按键，分别对应于不同的运行状态和工作模式，智能眼镜通过该无线通信模块响应于用户点击按键的控制操作所触发的切换指令，控制智能眼镜进入被点击的按键对应的运行状态或工作模式。
可选的，可根据用户在智能眼镜或智能移动终端的模式选择操作，选择将助听模式和翻译模式中的任意一个设置为常用的收听模式。如图8所示，如果智能眼镜处于空闲状态，则可利用上述物理按键或虚拟按钮（触摸传感器）在通话模式和收听模式之间进行手动切换。例如，如果智能眼镜处于通话模式，则按下物理按键会将智能眼镜从通话模式更改为收听模式，再次按下该物理按键将变回通话模式，从而可减少按键的数量，降低制造成本，同时提高操作的智能化程度，使得操作更符合用户的使用习惯。
或者，当通过该触控传感器检测到用户的目标控制操作时，通过该无线通信模块响应于该目标控制操作触发的切换指令，控制切换智能眼镜的工作模式为通话模式或收听模式，其中，该目标控制操作的具体形式可由用户自定义，例如，可将长按该触控传感器的动作或预设的滑动手势设置为目标控制操作。
进一步的，上述无线通信模块中的DSP可对语音数据执行对应的各种算法。
如图9所示，在通话模式下，智能眼镜通过该DSP对下行通道的数据进行语音均衡处理和输出音量处理，并对上行通道的数据依次进行回音消除处理、第一波束成形处理以及噪声抑制处理。下行通道的数据通过该无线通信模块中的无线信号收发器输入。该无线信号收发器优先采用蓝牙协议作为通信协议。
进一步的，在通话模式下，智能眼镜通过该DSP对通过该无线信号收发器输入（如图9中的智能眼镜无线蓝牙输入）的来自于智能移动终端的通话语音数据依次进行语音均衡处理和输出音量控制处理，并将输出音量控制处理后的通话语音数据发送给智能眼镜的扬声器进行输出，同时，通过该DSP对拾音装置获取的语音数据，利用输出音量处理后的通话语音数据作为参考信号进行回音消除处理，然后再进行波束成形处理以及噪声抑制处理，并将噪声抑制处理后的数据通过该无线信号收发器输出给智能移动终端。
由于远方说话会通过扬声器播放，因此远方的语音信号会被麦克风接收，从而产生回路。对上行通道的语音数据进行的回声消除处理，是利用回声消除算法通过对比扬声器的输出信号和麦克风的输入信号，从而把回音消除，并中断扬声器与麦克风的回路链。
对上行通道的语音数据进行的噪声抑制处理，是利用噪声抑制算法，将噪音的音量减低或消除，并同时放大对方说话的音量。通过利用噪声抑制算法，即便用户身处于环境噪音很大的地方，远方也听不到喧闹的环境噪音，而智能听到智能眼镜的用户清晰的声音。
对下行通道的语音数据进行的语音均衡处理，是利用语音均衡器，对远方的语音信号进行语音均衡处理，以加强用户弱听的频率信号，从而达到补偿该弱听的频率信号的目的。
对下行通道的语音数据进行的输出音量控制处理，是利用输出音量控制算法调校扬声器的输出音量。
可选的，该无线信号收发器采用蓝牙协议、无线保真协议、近场通信协议、紫蜂、数字生活网络联盟协议、无载波通信协议、射频识别协议以及蜂窝移动通信协议中的至少一种作为与智能移动终端进行数据交互的通信协议。
进一步的，于本申请其他实施例中，在通过拾音装置获取第一语音数据或第三语音数据之前，该方法还包括：
通过该无线通信模块基于蓝牙协议，与智能移动终端进行蓝牙配对，以在智能眼镜和智能移动终端之间建立数据传输通道。智能眼镜和智能移动终端后续的数据交互皆可通过该数据传输通道进行。
如图10所示，在助听模式下，为了实现听力损失辅助，智能眼镜通过该DSP对语音数据进行反馈取消处理、语音活动检测处理、噪声抑制处理、语音均衡处理以及用户说话检测处理，并将语音均衡处理后的语音数据发送给智能眼镜的扬声器进行输出，同时，将语音均衡处理后的语音数据用作该反馈取消处理的参考数据。进一步的，在将语音均衡处理后的语音数据发送给智能眼镜的扬声器进行输出之前，还可以对该语音均衡处理后的语音数据进行输出增益和输出音量控制处理，具体处理方式与图9中所示的输出音量控制处理方式相同，具体可参考上述图9的相关说明，此处不再赘述。
在助听模式下，由于扬声器输出的声波会即时被麦克风阵列接收并形成回路，从而产生刺耳的啸叫。反馈取消处理就是利用反馈取消算法通过对比扬声器的输出信号和麦克风阵列的输入信号把回音消除，并中断扬声器和麦克风阵列的回路链。具体的，如图11所示，麦克风获取前方的语音信号，但因一般的麦克风是全向性的，扬声器输出的语音信号也同时被麦克风获取。扬声器的语音信号也通过内部连线反馈到自适应过滤器的输入。自适应过滤器就根据上一个扬声器输出数据而调整LMS（最小均方）自适应滤波器系数，从而把扬声器输出的信号（即f(n)）从麦克风的输入删除。
此外，在助听模式下，由于需要将对方的声音放大从而使得有听力障碍的用户能够听清对方说话的内容，但在将声音放大的过程中，会将语音和噪音一起放大，从而给用户带来不适。噪声抑制处理是利用噪声抑制算法将噪音音量减低或消除，并同时放大对方说话的音量。该算法具体是把每帧的语音信号分成说活和不说话。假如帧被检测为不说话帧，算法用这帧的语音信号去更新平均噪音能量值。如帧被定义为说话帧，这帧的语音信号就会减去平均噪音能量值，从而达到噪音抑制。
此外，在助听模式下，一般存在听力障碍的用户可能只是听不见或者听不清某一特定频率的声音。语音均衡处理是利用语音均衡器，将该特定频率的声音信号加强，从而达到补偿该特定频率的声音信号的目的。
此外，由于智能眼镜的麦克风阵列距离用户的嘴巴非常近，当用户说话时，麦克风阵列会接收到很大的信号，并在智能眼镜的扬声器上播放，这样当用户自己说话时，就会通过扬声器听到自己的声音。用户语音检测处理，是利用用户语音检测算法不断检测麦克风阵列接收的信号并进行平均能量估算分析和检测，若平均能量估算值低于预设的平均能量阈值，则该信号被检测为用户的声音。
语音活动检测处理与用户语音检测处理类似，只是平均能量阈值比用户语音检测算法低，从而检测对方说话的声音。
可选的，于本申请其他一实施例中，还可使用独立的播放设备（如，外部扬声器、外部助听器）进行语音数据的播放，上述输出该放大后的语音数据或该第二语音数据的步骤具体可包括：通过该无线通信模块确定是否存在与该智能眼镜建立数据连接的播放设备；若存在，则通过该无线通信模块将该放大后的语音数据或该第二语音数据发送给该播放设备，以通过该播放设备播放该放大后的语音数据或该第二语音数据；若不存在，则通过该目标扬声器输出该放大后的语音数据或该第二语音数据。像这样，在有独立的播放设备时，利用独立的播放设备播放语音数据，可以降低智能眼镜的功耗。其中，该智能眼镜的无线通信模块可根据本地蓝牙配对日志和外设接口的连接状态，确定是否存在与该智能眼镜建立数据连接的播放设备。
在本申请所提供的几个实施例中，应该理解到，所揭露的智能眼镜、辅助系统和助听方法，可以通过其它的方式实现。例如多个模块或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的连接或直接连接或通信连接可以是通过一些接口，装置或模块的间接连接或通信连接，可以是电性，机械或其它的形式。
需要说明的是，对于前述的各方法实施例，为了简便描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本申请并不受所描述的动作顺序的限制，因为依据本申请，某些步骤可以采用其它顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定都是本申请所必须的。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其它实施例的相关描述。
以上为对本申请所提供的智能眼镜、助听方法和辅助系统的描述，对于本领域的技术人员，依据本申请实施例的思想，在具体实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本申请的限制。

Claims (17)

  1. 一种用于助听的智能眼镜，其特征在于，包括：两条镜腿、拾音装置、多个扬声器以及无线通信模块；
    所述多个扬声器以及所述拾音装置设置在所述两条镜腿中的至少一条镜腿上，所述无线通信模块设置在所述两条镜腿中的任意一条镜腿的腔体内并与所述拾音装置以及所述多个扬声器电性连接，所述拾音装置包括至少一个麦克风；
    所述至少一个麦克风，用于获取来自第一预设方向的第一语音数据；
    所述无线通信模块，用于将所述第一语音数据进行放大以得到放大后的语音数据，并将放大后的语音数据发送给所述多个扬声器中的目标扬声器以进行播放，其中所述目标扬声器与所述至少一个麦克风分别设置在不同的镜腿上。
  2.  如权利要求1所述的智能眼镜，其特征在于，还包括：镜框、以及电性连接于所述无线通信模块的输入装置，所述输入装置设置在所述镜框以及所述两条镜腿中的至少任意一个上；
    所述无线通信模块，还用于响应于用户基于所述输入装置触发的第一切换指令将所述智能眼镜的工作模式切换为助听模式；
    所述无线通信模块，还用于在所述助听模式下，将所述第一语音数据进行放大。
  3.  如权利要求2所述的智能眼镜，其特征在于，
    所述无线通信模块，还用于响应于所述用户基于所述输入装置触发的第二切换指令将所述智能眼镜的工作模式切换为翻译模式；
    所述无线通信模块，还用于在所述翻译模式下，将所述第一语音数据发送给智能移动终端，接收所述智能移动终端发送的第二语音数据，并将所述第二语音数据发送给所述目标扬声器以进行播放，所述第二语音数据由所述第一语音数据翻译得到。
  4.  如权利要求2所述的智能眼镜，其特征在于，
    所述无线通信模块，还用于响应于所述用户基于所述输入装置触发的第三切换指令将所述智能眼镜的工作模式切换为通话模式；
    所述至少一个麦克风，还用于在所述通话模式下，获取来自第二预设方向的第三语音数据；
    所述无线通信模块，还用于将所述第三语音数据发送给目标终端，接收所述目标终端发送的第四语音数据；
    所述多个扬声器，还用播放所述第四语音数据。
  5.  如权利要求4所述的智能眼镜，其特征在于，所述第一预设方向指向用户的前方，所述第二预设方向指向用户的下方。
  6.  一种基于智能眼镜的助听方法，其特征在于，所述智能眼镜包括：两条镜腿、无线通信模块以及与所述无线通信模块电性连接的拾音装置和多个扬声器，所述拾音装置包括至少一个麦克风，所述方法包括：
    通过所述至少一个麦克风获取来自第一预设方向的第一语音数据；
    通过所述无线通信模块将所述第一语音数据进行放大以得到放大后的语音数据；
    通过所述多个扬声器中的目标扬声器播放所述放大后的语音数据，其中所述目标扬声器与所述至少一个麦克风分别设置在不同的镜腿上。
  7.  如权利要求6所述的方法，其特征在于，所述方法还包括：
    通过所述无线通信模块响应于第一切换指令，将所述智能眼镜的工作模式切换为助听模式；
    所述通过所述无线通信模块将所述第一语音数据进行放大以得到放大后的语音数据，包括：
    在所述助听模式下，通过所述无线通信模块将所述第一语音数据进行放大以得到放大后的语音数据。
  8.  如权利要求6所述的方法，其特征在于，所述方法还包括：
    通过所述无线通信模块响应于第二切换指令，将所述智能眼镜的工作模式切换为翻译模式；
    在所述翻译模式下，通过所述至少一个麦克风获取所述第一语音数据；
    通过所述无线通信模块将所述第一语音数据发送给智能移动终端，并接收所述智能移动终端发送的第二语音数据，其中所述第二语音数据由所述第一语音数据翻译得到；
    通过所述目标扬声器播放所述第二语音数据。
  9.  如权利要求8所述的方法，其特征在于，所述智能移动终端接收所述智能眼镜发送的第一语音数据，根据预设的转换标记，选择将所述第一语音数据或所述第一语音数据对应的第一文本数据发送给云端计算机设备以进行翻译，接收所述云端计算机设备返回的目的语言的第二文本数据，将所述第二文本数据转换为所述第二语音数据并发送给所述智能眼镜。
  10.  如权利要求6所述的方法，其特征在于，所述方法还包括：
    通过所述无线通信模块响应于第三切换指令，将所述智能眼镜的工作模式切换为通话模式；
    在所述通话模式下，通过所述至少一个麦克风获取来自第二预设方向的第三语音数据；
    通过所述无线通信模块将所述第三语音数据发送给目标终端；
    通过所述无线通信模块接收所述目标终端发送的第四语音数据，并通过所述多个扬声器播放所述第四语音数据。
  11.  如权利要求10所述的方法，其特征在于，所述第一预设方向指向用户的前方，所述第二预设方向指向用户的下方。
  12.  一种辅助系统，其特征在于，包括：智能眼镜以及智能移动终端；
    所述智能眼镜包括：两条镜腿、拾音装置、多个扬声器以及无线通信模块，其中，所述多个扬声器以及所述拾音装置设置在所述两条镜腿中的至少一条镜腿上，所述无线通信模块设置在所述两条镜腿中的任意一条镜腿的腔体内并与所述拾音装置以及所述多个扬声器电性连接，所述拾音装置包括至少一个麦克风；
    所述无线通信模块，用于响应于用户的控制操作，控制切换所述智能眼镜的工作模式，所述工作模式包括：助听模式以及翻译模式；
    所述至少一个麦克风，用于在所述助听模式或所述翻译模式下，获取来自第一预设方向的第一语音数据；
    所述无线通信模块，用于在所述助听模式下，将所述第一语音数据进行放大以得到放大后的语音数据，并将放大后的语音数据发送给所述多个扬声器中的目标扬声器以进行播放，其中所述目标扬声器与所述至少一个麦克风分别设置在不同的镜腿上；
    所述无线通信模块，还用于在所述翻译模式下，将所述第一语音数据发送给智能移动终端；
    所述智能移动终端，用于接收所述第一语音数据，并将第二语音数据发送给所述无线通信模块，其中所述第二语音数据由所述第一语音数据翻译得到；
    所述无线通信模块，还用于接收所述智能移动终端发送的第二语音数据，并将所述第二语音数据发送给所述目标扬声器以进行播放。
  13.  如权利要求12所述的系统，其特征在于，所述系统还包括云端计算机设备；
    所述智能移动终端，还用于将所述第一语音数据或所述第一语音数据对应的第一文本数据作为待翻译数据发送给所述云端计算机设备；
    所述云端计算机设备，用于将所述待翻译数据翻译为目的语言的第二文本数据，并将所述第二文本数据发送给所述智能移动终端；
    所述智能移动终端，还用于将所述第二文本数据转换为所述第二语音数据并发送给所述智能眼镜。
  14.  如权利要求13所述的系统，其特征在于，
    所述智能移动终端，还用于获取预设的转换标记，若所述预设的转换标记为第一标记，则将所述第一语音数据转换为所述第一文本数据，并将所述第一文本数据发送给所述云端计算机设备，以及若所述预设的转换标记为第二标记，则将所述第一语音数据发送给所述云端计算机设备。
  15.  如权利要求14所述的系统，其特征在于，
    所述智能移动终端，还用于获取网速并确定所述网速是否大于预设速度，若不大于所述预设速度，则将所述转换标记设置为所述第一标记，若大于所述预设速度，则将所述转换标记设置为所述第二标记。
  16.  如权利要求14所述的系统，其特征在于，
    所述智能移动终端，还用于响应于用户的设置操作，将所述转换标记设置为所述设置操作指向的标记。
  17.  如权利要求12所述的系统，其特征在于，所述工作模式还包括通话模式；
    所述至少一个麦克风，还用于在所述通话模式下，获取来自第二预设方向的第三语音数据；
    所述无线通信模块，还用于将所述第三语音数据发送给目标终端，接收所述目标终端发送的第四语音数据；
    所述多个扬声器，还用播放所述第四语音数据。
PCT/CN2024/091091 2023-05-31 2024-05-04 用于助听的智能眼镜、助听方法以及辅助系统 Pending WO2024244895A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310639291.8A CN119071706A (zh) 2023-05-31 2023-05-31 用于助听的智能眼镜、助听方法以及辅助系统
CN202310639291.8 2023-05-31

Publications (1)

Publication Number Publication Date
WO2024244895A1 true WO2024244895A1 (zh) 2024-12-05

Family

ID=93630540

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/091091 Pending WO2024244895A1 (zh) 2023-05-31 2024-05-04 用于助听的智能眼镜、助听方法以及辅助系统

Country Status (2)

Country Link
CN (1) CN119071706A (zh)
WO (1) WO2024244895A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120602876A (zh) * 2025-05-23 2025-09-05 首都医科大学附属北京同仁医院 一种眼镜式助听器
US12549909B1 (en) 2025-01-25 2026-02-10 Legato Audio, Inc. Open ear system using artificial intelligence (AI) driven audio signal processing

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112001189A (zh) * 2019-05-27 2020-11-27 陈筱涵 实时外语沟通系统
CN113348673A (zh) * 2018-10-31 2021-09-03 美国斯耐普公司 多扬声器和麦克风可穿戴设备中非回声双工会话的交替采样方法
CN113534500A (zh) * 2020-04-14 2021-10-22 所乐思(深圳)科技有限公司 智能眼镜、监控人体姿态的方法、介质、终端及系统
WO2022051097A1 (en) * 2020-09-03 2022-03-10 Spark23 Corp. Eyeglass augmented reality speech to text device and method
CN217656743U (zh) * 2022-05-16 2022-10-25 西安合谱声学科技有限公司 一种眼镜式方向性助听器装置
CN115510875A (zh) * 2022-09-15 2022-12-23 京东方科技集团股份有限公司 翻译方法、翻译系统以及ar眼镜
CN115695620A (zh) * 2021-07-22 2023-02-03 所乐思(深圳)科技有限公司 智能眼镜及其控制方法和系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113348673A (zh) * 2018-10-31 2021-09-03 美国斯耐普公司 多扬声器和麦克风可穿戴设备中非回声双工会话的交替采样方法
CN112001189A (zh) * 2019-05-27 2020-11-27 陈筱涵 实时外语沟通系统
CN113534500A (zh) * 2020-04-14 2021-10-22 所乐思(深圳)科技有限公司 智能眼镜、监控人体姿态的方法、介质、终端及系统
WO2022051097A1 (en) * 2020-09-03 2022-03-10 Spark23 Corp. Eyeglass augmented reality speech to text device and method
CN115695620A (zh) * 2021-07-22 2023-02-03 所乐思(深圳)科技有限公司 智能眼镜及其控制方法和系统
CN217656743U (zh) * 2022-05-16 2022-10-25 西安合谱声学科技有限公司 一种眼镜式方向性助听器装置
CN115510875A (zh) * 2022-09-15 2022-12-23 京东方科技集团股份有限公司 翻译方法、翻译系统以及ar眼镜

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12549909B1 (en) 2025-01-25 2026-02-10 Legato Audio, Inc. Open ear system using artificial intelligence (AI) driven audio signal processing
CN120602876A (zh) * 2025-05-23 2025-09-05 首都医科大学附属北京同仁医院 一种眼镜式助听器

Also Published As

Publication number Publication date
CN119071706A (zh) 2024-12-03

Similar Documents

Publication Publication Date Title
US12526572B2 (en) Smart glasses, system and control method thereof
CN108710615B (zh) 翻译方法及相关设备
WO2020007223A1 (en) Method for controlling earphone switching and earphone
EP3598435B1 (en) Method for processing information and electronic device
CN108966067B (zh) 播放控制方法及相关产品
CN108777827B (zh) 无线耳机、音量调整方法及相关产品
WO2022033176A1 (zh) 音频播放控制方法、装置、电子设备及存储介质
CN108810693A (zh) 设备控制方法及相关产品
WO2024244895A1 (zh) 用于助听的智能眼镜、助听方法以及辅助系统
CN108900231B (zh) 动态天线调整方法及相关产品
CN109067965B (zh) 翻译方法、翻译装置、可穿戴装置及存储介质
CN109150221B (zh) 一种可穿戴设备主从切换方法及相关产品
CN108683980B (zh) 一种音频信号的传输方法及移动终端
CN108769387A (zh) 应用控制方法及相关设备
CN108833683B (zh) 动态天线调整实现方法及相关产品
CN108541080A (zh) 第一电子设备与第二电子设备进行回连的方法及相关产品
CN108897516B (zh) 一种可穿戴设备音量调整方法及相关产品
CN108429956A (zh) 无线耳机、控制操作方法及相关产品
CN108668018B (zh) 移动终端、音量控制方法及相关产品
CN108923810A (zh) 翻译方法及相关设备
CN109121034B (zh) 基于音量的主从切换方法及相关产品
CN108810261B (zh) 通话中天线切换方法及相关产品
CN110058837A (zh) 一种音频输出方法及终端
CN109040425B (zh) 信息处理方法及相关产品
CN112394771A (zh) 通信方法、装置、穿戴设备及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24814100

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE