CN117278898A - Headset, method, device and system for enhancing sound of hearing aid headset, and storage medium - Google Patents

Abstract

The invention provides an auxiliary hearing earphone, a sound enhancement method, a device, a system and a storage medium thereof, relating to the technical field of auxiliary hearing earphones, wherein the method comprises the following steps: determining a first sound source direction of the first target sound signal in the case that the first target sound signal is included in the collected initial sound signal; acquiring gesture information of the auxiliary hearing earphone, and determining face steering of a wearer of the auxiliary hearing earphone based on the gesture information; and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, enhancing the first target sound signal to obtain enhanced sound corresponding to the initial sound signal. The technical scheme provided by the invention can enhance the concerned sound in the collected sound signal.

Description

Headset, method, device and system for enhancing sound of hearing aid headset, and storage medium

Technical Field

The present invention relates to the field of hearing-assisted headphones, and in particular, to a hearing-assisted headphone, and a method, apparatus, system, and storage medium for enhancing sound thereof.

Background

The hearing aid earphone, namely the earphone with the hearing aid function, can amplify external sound and help a wearer, particularly a hearing impaired person, to acquire external sound information more clearly.

At present, when the external sound is amplified by the hearing aid earphone, the same amplification processing is carried out on all the collected sounds, and the sound focused by a wearer cannot be enhanced in a targeted manner.

Disclosure of Invention

The invention provides an auxiliary hearing earphone, a sound enhancement method, a device, a system and a storage medium thereof, which are used for solving the problem that the auxiliary hearing earphone cannot pertinently enhance the sound concerned by a wearer in the prior art, realizing enhancement of the sound concerned in the collected sound signals and enabling the wearer of the auxiliary hearing earphone to hear clearer target sound.

The invention provides a sound enhancement method of an auxiliary hearing earphone, which comprises the following steps:

determining a first sound source direction of a first target sound signal in the case that the first target sound signal is included in the collected initial sound signal;

acquiring the posture information of the auxiliary hearing earphone, and determining the face steering of a wearer of the auxiliary hearing earphone based on the posture information;

and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, enhancing the first target sound signal to obtain enhanced sound corresponding to the initial sound signal.

According to the method for enhancing the sound of the hearing-aid earphone provided by the invention, when the target included angle between the face steering and the first sound source direction is smaller than the preset angle threshold, the first target sound signal is enhanced, and the method comprises the following steps:

and under the condition that the target included angle between the face steering and the first sound source direction is continuously detected to be smaller than the preset angle threshold value in a first preset time period, enhancing the first target sound signal.

According to the sound enhancement method of the hearing-aid earphone provided by the invention, the first target sound signal is enhanced, and the method comprises the following steps:

determining a target enhancement coefficient corresponding to the target included angle based on the corresponding relation between the included angle and the enhancement coefficient;

the first target sound signal is enhanced based on the target enhancement coefficient.

The sound enhancement method of the hearing aid earphone provided by the invention further comprises the following steps:

in the process of enhancing the first target sound signal, under the condition that the change amount of the face turning is detected to be larger than a preset change amount threshold value, determining whether a second target sound signal is detected in the direction indicated by the changed face turning;

and enhancing the second target sound signal under the condition that the second target sound signal is detected in the direction indicated by the changed face steering.

According to the sound enhancement method of the hearing-aid earphone provided by the invention, the second target sound signal is enhanced, and the method comprises the following steps:

under the condition that the included angle between the changed face turning direction and the second sound source direction is smaller than the preset angle threshold value in a second preset time period, the first target sound signal is enhanced based on a first enhancement coefficient, and the second target sound signal is enhanced based on a second enhancement coefficient; the first enhancement coefficient is smaller than the second enhancement coefficient, and the second sound source direction is the sound source direction of the second target sound signal.

The sound enhancement method of the hearing aid earphone provided by the invention further comprises the following steps:

under the condition of receiving a communication connection request sent by terminal equipment, establishing communication connection with the terminal equipment;

and receiving target sound characteristics sent by the terminal equipment, wherein the target sound characteristics are used for determining the first target sound signal in the initial sound signal.

The invention also provides a sound enhancement device of the hearing aid earphone, which comprises:

a sound source direction determining module, configured to determine a first sound source direction of a first target sound signal when it is detected that the collected initial sound signal includes the first target sound signal;

the face steering determining module is used for acquiring the gesture information of the auxiliary hearing earphone and determining the face steering of the wearer of the auxiliary hearing earphone based on the gesture information;

the sound enhancement module is used for enhancing the first target sound signal under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold value, and obtaining enhanced sound corresponding to the initial sound signal.

The invention also provides a hearing aid earphone which comprises a microphone array, a direction sensor, a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the microphone array, the direction sensor and the memory are respectively in communication connection with the processor;

the direction sensor is used for detecting the gesture information of the hearing aid earphone;

the processor, when executing the computer program, implements a method for enhancing sound of the hearing aid earphone as described in any one of the above.

The invention also provides a sound enhancement system, which comprises terminal equipment and the hearing aid earphone, wherein the terminal equipment is in communication connection with the hearing aid earphone;

the terminal equipment is used for determining target sound characteristics according to the detected sound selection operation and sending the target sound characteristics to the hearing aid earphone; the target sound characteristic is used for determining a first target sound signal in initial sound signals collected by the auxiliary hearing earphone.

The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of sound enhancement for a hearing aid headset as described in any of the above.

The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a method of enhancing sound of a hearing aid earpiece as described in any one of the above.

According to the hearing-aid earphone, the sound enhancement method, the device, the system and the storage medium thereof, under the condition that the collected initial sound signals comprise first target sound signals, the first sound source direction of the first target sound signals is determined, the gesture information of the hearing-aid earphone is obtained, and the face steering of a wearer of the hearing-aid earphone is determined based on the gesture information; and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, indicating that the source direction of the current first target sound signal is the direction opposite to the face, and enhancing the first target sound signal at the moment to obtain enhanced sound corresponding to the initial sound signal. In this way, the first target sound signal of the face facing direction of the wearer of the auxiliary hearing earphone in the collected initial sound signal can be enhanced in a targeted manner, and the first target sound signal is the sound focused by the wearer of the auxiliary hearing earphone, namely the enhancement of the focused sound in the collected sound signal is realized, so that the wearer of the auxiliary hearing earphone can hear the focused sound more clearly.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic structural diagram of an auxiliary hearing earphone according to an embodiment of the present invention;

fig. 2 is a flow chart of a method for enhancing sound of a hearing aid earphone according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a sound enhancement device of a hearing aid earphone according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a sound enhancement system according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that, in the present invention, the numbers of the described objects, such as "first", "second", etc., are only used to distinguish the described objects, and do not have any sequence or technical meaning.

An embodiment of the present invention provides an auxiliary hearing earphone, fig. 1 schematically illustrates a structure of the auxiliary hearing earphone provided by the embodiment of the present invention, and referring to fig. 1, the auxiliary hearing earphone may include a microphone array 110, an orientation sensor 120, a memory 130, a processor (processor) 140, and may further include a communication interface (Communication Interface) 150 and a communication bus 160.

Wherein the microphone array 110, the direction sensor 120, and the memory 130 are each communicatively coupled to the processor 140. The microphone array 110 may include at least two microphones for collecting and sound source localization of an initial sound signal. The direction sensor 120 may be used to detect pose information of the hearing aid headset. The gesture information can comprise a pitch angle, an azimuth angle, a rotation angle and the like, and the gesture information can be converted into a direction based on quaternion, euler angle, a rotation matrix and the like, so that the direction information of the hearing aid earphone is obtained. It will be appreciated that the direction information of the hearing aid earphone will change with the rotation of the head of the wearer of the hearing aid earphone, so that the direction information of the hearing aid earphone determined according to the posture information of the direction sensor 120 is the face turning direction of the head of the wearer.

Processor 140, communication interface 150, and memory 130 may communicate with each other via a communication bus 160. The memory 130 can store a computer program that can run on the processor 140, and the processor 140 can call logic instructions in the memory 130 to execute a method for enhancing sound of the hearing aid earphone according to the embodiment of the present invention, where the method may include: determining a first sound source direction of the first target sound signal in the case that the first target sound signal is included in the collected initial sound signal; acquiring gesture information of the auxiliary hearing earphone, and determining face steering of a wearer of the auxiliary hearing earphone based on the gesture information; and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, enhancing the first target sound signal to obtain enhanced sound corresponding to the initial sound signal.

By way of example, the direction sensor 120 may include an acceleration sensor, a gyroscope, a magnetic force sensor, or the like. The acceleration sensor can measure the acceleration of the auxiliary hearing earphone in three directions in the three-dimensional coordinate system, and the direction of the auxiliary hearing earphone relative to the gravity acceleration can be determined according to the acceleration, so that the face steering is obtained. The gyroscope can measure the angular speeds of the hearing aid earphone in three directions in a three-dimensional coordinate system, and the direction of the hearing aid earphone relative to the rotating shaft of the hearing aid earphone can be determined according to the angular speeds, so that the face steering is obtained. The magnetic sensor can measure magnetic field intensity information of the auxiliary hearing earphone in three directions in a three-dimensional coordinate system, and the direction of the auxiliary hearing earphone relative to the earth magnetic field can be determined according to the magnetic field intensity information, so that face steering is obtained.

By way of example, the processor 140 may include a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processing, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.

By way of example, communication bus 160 may be an industry standard architecture (Industry Standard Architecture, ISA) bus, a peripheral component interconnect (Peripheral Component Interconnect, PCI) bus, an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, or the like.

In addition, the logic instructions in the memory 130 may be implemented in the form of software functional units and stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The method for enhancing sound of the hearing aid earphone according to the present invention will be described below with reference to fig. 2 based on the hearing aid earphone according to the corresponding embodiment of fig. 1. The method for enhancing the sound of the hearing aid earphone can be applied to the hearing aid earphone, and can also be applied to a sound enhancing device of the hearing aid earphone arranged in the hearing aid earphone, wherein the sound enhancing device of the hearing aid earphone can be realized by software, hardware or a combination of the two.

Fig. 2 is a schematic flow chart illustrating a method for enhancing sound of a hearing aid earphone according to an embodiment of the present invention, and referring to fig. 2, the method for enhancing sound of a hearing aid earphone may include the following steps 210 to 230.

Step 210: in the case that the first target sound signal is included in the collected initial sound signal, a first sound source direction of the first target sound signal is determined.

The auxiliary hearing earphone can acquire the sound of the surrounding environment through the microphone array to obtain an initial sound signal, the auxiliary hearing earphone can extract the sound characteristic of the initial sound signal, the sound characteristic is compared with the target sound characteristic, and if the sound characteristic comprises the characteristic matched with the target sound characteristic, the sound corresponding to the matched sound characteristic is determined to be the first target sound signal. Meanwhile, based on the first target sound signal collected by the microphone array, the sound source direction of the first target sound signal is determined by utilizing a sound source positioning technology, and the first sound source direction is obtained.

By way of example, the first target sound signal may include at least one of a human sound, a musical instrument sound, a musical sound, a whistling sound, an animal sound, and the like.

The target sound feature may be one or more sound features selected from sound features preconfigured in the auxiliary earphone, for example, the auxiliary earphone may provide a selection key of sound features, and the target sound feature is selected from the provided sound features according to a detected trigger operation for the selection key. Alternatively, the target sound characteristic may be a target sound characteristic selected by a user received from a terminal device communicatively connected to the hearing aid headset.

Step 220: and acquiring the posture information of the auxiliary hearing earphone, and determining the face steering of the wearer of the auxiliary hearing earphone based on the posture information.

The hearing aid earphone can detect attitude information such as a pitch angle, an azimuth angle, a rotation angle and the like of the hearing aid earphone through the direction sensor 120, and then the attitude information can be converted into direction information based on quaternion, euler angles, a rotation matrix and the like, so that the direction information of the hearing aid earphone is obtained. The direction information of the auxiliary hearing earphone will change along with the rotation of the head of the wearer of the auxiliary hearing earphone, so the direction information of the auxiliary hearing earphone determined according to the posture information of the direction sensor 120 is the face turning direction of the rotation of the head of the wearer.

Step 230: and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, enhancing the first target sound signal to obtain enhanced sound corresponding to the initial sound signal.

After the first sound source direction and the face direction of the first target sound signal are obtained, the first sound source direction and the face direction can be compared, if the target included angle between the face direction and the first sound source direction is smaller than a preset angle threshold, the first sound source direction is the direction opposite to the face of the wearer of the hearing aid earphone, the wearer pays attention to the sound in the direction, and the first target sound signal can be enhanced to obtain enhanced sound corresponding to the initial sound signal.

If the target included angle between the face steering and the first sound source direction is greater than or equal to the preset angle threshold, which indicates that the first sound source direction is not the direction opposite to the face of the wearer of the auxiliary hearing earphone, the initial sound signals collected by the auxiliary hearing earphone can be enhanced in a common manner, that is, the same enhancement processing is performed on each sound signal in the initial sound signals.

In an exemplary case, if the target angle between the face steering and the first sound source direction is greater than or equal to the preset angle threshold, it may be determined whether the face of the wearer is steering in the first sound source direction in the third preset time period, that is, whether the face steering is detected to change in the third preset time period (e.g., 3 seconds), and if the target angle between the changed face steering and the first sound source direction is less than the preset angle threshold, it is determined that the face of the wearer is steering in the first sound source direction in the third preset time period, and at this time, the first target sound signal in the first sound source direction may be enhanced, so that the first target sound signal in the sound signal received by the wearer is the clearest.

According to the sound enhancement method of the auxiliary hearing earphone, under the condition that the collected initial sound signals comprise first target sound signals, the first sound source direction of the first target sound signals is determined, gesture information of the auxiliary hearing earphone is obtained, and face steering of a wearer of the auxiliary hearing earphone is determined based on the gesture information; and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, indicating that the source direction of the current first target sound signal is the direction opposite to the face, and enhancing the first target sound signal at the moment to obtain enhanced sound corresponding to the initial sound signal. In this way, the first target sound signal of the face facing direction of the wearer of the auxiliary hearing earphone in the collected initial sound signal can be enhanced in a targeted manner, and the first target sound signal is the sound focused by the wearer of the auxiliary hearing earphone, namely the enhancement of the focused sound in the collected sound signal is realized, so that the wearer of the auxiliary hearing earphone can hear the focused sound more clearly.

Based on the method for enhancing the sound of the hearing aid earphone in the corresponding embodiment of fig. 2, in an example embodiment, in a case that the target included angle between the face steering and the first sound source direction is smaller than the preset angle threshold, enhancing the first target sound signal may include: and under the condition that the target included angle between the face steering and the first sound source direction is continuously detected to be smaller than a preset angle threshold value within a first preset time period, enhancing the first target sound signal.

Specifically, when it is determined that the face steering and the first sound source direction are determined and a target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, it is determined whether the target included angle between the face steering and the first sound source direction is changed to be greater than or equal to the preset angle threshold in a first preset time period (for example, 3 seconds), if the target included angle between the face steering and the first sound source direction is always kept smaller than the preset angle threshold, it means that the face of the wearer of the hearing aid earphone is continuously opposite to the first sound source direction, the sound of the first sound source direction is the sound which needs to be continuously focused on for the wearer, and at this time, the first target sound signal of the first sound source direction can be enhanced.

According to the sound enhancement method of the hearing aid earphone, whether the face of the wearer continuously faces the first sound source direction or not is judged by detecting whether the target included angle between the face steering and the first sound source direction is smaller than the preset angle threshold value or not for a certain time, and further the first target sound signal of the first sound source direction is enhanced under the condition that the face of the wearer continuously faces the first sound source direction, so that accuracy of sound concerned by the wearer, sound enhancement effect and user experience can be improved, and unnecessary sound enhancement in the head rotation process of the wearer is avoided.

The method for enhancing the sound of the hearing aid earphone based on the corresponding embodiment of fig. 2 may, in an example embodiment, include: determining a target enhancement coefficient corresponding to the target included angle based on the corresponding relation between the included angle and the enhancement coefficient; and enhancing the first target sound signal based on the target enhancement coefficient to obtain enhanced sound corresponding to the initial sound signal.

For example, assuming that the preset angle threshold is 10 °, each 1 ° interval may correspond to one enhancement coefficient, assuming that the target included angle between the detected face steering and the first sound source direction is 5 °, which is smaller than the preset angle threshold of 10 °, the face of the hearing aid wearer is considered to face the sound source direction of the first target sound signal, a target enhancement coefficient corresponding to the target included angle of 5 ° may be found from the correspondence between the included angle and the enhancement coefficient, and then the first target sound signal is enhanced by using the target enhancement coefficient.

It can be understood that when the target included angle between the face steering and the first sound source direction is within the preset angle threshold, the smaller the target included angle between the face steering and the first sound source direction is, which indicates that the closer the face is to be against the first sound source direction, the stronger the received first target sound signal is, and the smaller the corresponding enhancement coefficient can be set; conversely, the larger the corresponding enhancement factor can be set. Based on the above, in the correspondence between the established included angle and the enhancement coefficient, the enhancement coefficient decreases as the included angle increases.

According to the sound enhancement method of the hearing aid earphone, under the condition that the target included angle between the face steering and the first sound source direction is smaller than the preset angle threshold, the target enhancement coefficient matched with the target included angle can be determined from the corresponding relation between the included angle and the enhancement coefficient, the target enhancement coefficient is utilized to enhance the first target sound signal, the sound enhancement effect is improved, the intelligent degree is high, and therefore user experience is improved.

In an exemplary embodiment, the head of the wearer of the hearing aid earphone may be turned at any time, that is, the face steering is in a state of changing in real time, so that the target included angle between the face steering and the first sound source direction may be changed from a state of being smaller than the preset angle threshold to a state of being larger than or smaller than the preset angle threshold. Based on the method, the change amount of the face turning can be monitored in real time, when the change amount of the face turning is larger than a preset change amount threshold, whether a new target sound signal is detected in the direction indicated by the face turning after the face turning is further detected, if yes, the new target sound signal can be used as a target sound signal to be enhanced, the enhancement effect of the sound concerned by a user is improved, and the hearing requirement of a hearing-aid earphone wearer on the sound concerned is met.

Specifically, the method for enhancing the sound of the hearing aid earphone can further comprise the following steps: in the process of enhancing the first target sound signal, under the condition that the change amount of the detected face steering is larger than a preset change amount threshold value, determining whether a second target sound signal is detected in the direction indicated by the changed face steering; and enhancing the second target sound signal under the condition that the second target sound signal is detected in the direction indicated by the changed face steering.

The preset change amount threshold may be determined based on a preset angle threshold, for example, the preset angle threshold is greater than the preset angle threshold, and after the face rotates by an angle of the preset change amount threshold, the target included angle between the rotated face steering and the first target sound signal may be changed to be greater than or equal to the preset angle threshold.

For example, in the case that it is detected that the change amount of the face turning is greater than the preset change amount threshold, it may be determined whether the direction indicated by the changed face turning detects the second target sound signal in combination with the sound source localization.

For example, in the case where the second target sound signal is detected in the direction indicated by the changed face direction, only the second target sound signal may be enhanced, or the second target sound signal may be enhanced while the enhancement of the first target sound signal is maintained. The enhancement coefficient for enhancing the second target sound signal may be greater than or equal to the enhancement coefficient for enhancing the first target sound signal.

For example, the wearer of the hearing aid earphone may turn the face in the direction of interest to listen to the target sound in that direction, such as a person turning to the body talking with a person next to the body, a speech of a lecturer being listened to by a lecture table, etc., and during the turning of the face, the face turning may change continuously rapidly until the face turns to the target position and stays, if the stay time exceeds a certain threshold, it means that the face turns to the target position and listens to the sound at that position. Based on the above, a preset time period can be set, if the face steering after the change is continuously detected within the preset time period is consistent with the sound source direction of the new target sound signal, the wearer of the auxiliary hearing earphone is considered to pay attention to the new target sound, the target sound in the direction can be enhanced, the enhancement effect of the sound focused by the user is improved, and the listening effect of the wearer of the auxiliary hearing earphone to the focused sound is met.

Specifically, enhancing the second target sound signal may include: under the condition that the included angle between the changed face steering and the second sound source direction is smaller than a preset angle threshold value in a second preset time period, the first target sound signal is enhanced based on a first enhancement coefficient, and the second target sound signal is enhanced based on a second enhancement coefficient; the first enhancement coefficient is smaller than the second enhancement coefficient, and the second sound source direction is the sound source direction of the second target sound signal.

Based on the method for enhancing the sound of the hearing aid earphone in the foregoing embodiments, in an example embodiment, the hearing aid earphone may be further connected in communication with a terminal device, and the user may control the hearing aid earphone through the terminal device, for example, may set a target sound focused by the user, manage the target sound focused by the user, and so on. Specifically, the method for enhancing the sound of the hearing aid earphone can further comprise the following steps: under the condition of receiving a communication connection request sent by the terminal equipment, establishing communication connection with the terminal equipment; and receiving target sound characteristics sent by the terminal equipment, wherein the target sound characteristics are used for determining a first target sound signal in the initial sound signals.

For example, the terminal device may display a sound management configuration interface, and the sound management configuration interface may include a sound selection menu; the terminal device may determine the target sound feature according to the detected sound selection operation for the sound selection menu. For example, but not limited to, a sound selection menu in the sound management configuration interface may include selectable sound types, such as human, instrument, musical, whistling, animal-whistling, etc., from which a user may select one or more sounds of interest. The terminal equipment determines the sound characteristics corresponding to the selected sound according to the selected sound to obtain target sound characteristics, then sends the target sound characteristics to the auxiliary hearing earphone, and the auxiliary hearing earphone can detect initial voice signals collected by the microphone array based on the target sound characteristics to determine whether the initial voice signals comprise first target sound signals matched with the target sound characteristics, namely the sound concerned selected by the user.

The following describes a sound enhancement device of a hearing aid earphone provided by the present invention, and the sound enhancement device of the hearing aid earphone described below and the sound enhancement method of the hearing aid earphone described above can be referred to correspondingly.

Fig. 3 is a schematic structural diagram illustrating a sound enhancement device of a hearing aid earphone according to an embodiment of the present invention, and referring to fig. 3, the sound enhancement device of a hearing aid earphone may include: a sound source direction determining module 310, configured to determine a first sound source direction of a first target sound signal in a case where it is detected that the collected initial sound signal includes the first target sound signal; the face steering determining module 320 is configured to obtain pose information of the auxiliary hearing earphone, and determine face steering of a wearer of the auxiliary hearing earphone based on the pose information; the sound enhancement module 330 is configured to enhance the first target sound signal to obtain an enhanced sound corresponding to the initial sound signal when the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold.

In an example embodiment, the sound enhancement module 330 may be specifically configured to include: and under the condition that the target included angle between the face steering and the first sound source direction is continuously detected to be smaller than a preset angle threshold value within a first preset time period, enhancing the first target sound signal.

In one example embodiment, the sound enhancement module 330 may include: the determining unit is used for determining a target enhancement coefficient corresponding to the target included angle based on the corresponding relation between the included angle and the enhancement coefficient; and an enhancement unit for enhancing the first target sound signal based on the target enhancement coefficient.

In an example embodiment, the sound enhancement device of the secondary listening earpiece may further include a sound determination module. The sound determining module is used for determining whether the direction indicated by the changed face steering detects a second target sound signal or not under the condition that the change amount of the face steering is detected to be larger than a preset change amount threshold value in the process of enhancing the first target sound signal; the sound enhancement module 330 is further configured to enhance the second target sound signal if the second target sound signal is detected in a direction indicated by the changed face direction.

In an example embodiment, the sound enhancement module 330 may be specifically configured to include: under the condition that the included angle between the changed face steering and the second sound source direction is smaller than a preset angle threshold value in a second preset time period, the first target sound signal is enhanced based on a first enhancement coefficient, and the second target sound signal is enhanced based on a second enhancement coefficient; the first enhancement coefficient is smaller than the second enhancement coefficient, and the second sound source direction is the sound source direction of the second target sound signal.

In an example embodiment, the sound enhancement device of the secondary listening earphone may further include: the communication connection module is used for establishing communication connection with the terminal equipment under the condition of receiving a communication connection request sent by the terminal equipment; and the receiving module is used for receiving the target sound characteristic sent by the terminal equipment, and the target sound characteristic is used for determining a first target sound signal in the initial sound signal.

The embodiment of the present invention further provides a sound enhancement system, fig. 4 schematically illustrates a structure diagram of the sound enhancement system further provided by the embodiment of the present invention, and referring to fig. 4, the sound enhancement system may include a terminal device 410 and an auxiliary earphone 420, where the terminal device 410 is communicatively connected to the auxiliary earphone 420. For example, the terminal device 410 may establish a communication connection with the secondary earpiece 420 through bluetooth, wiFi, mobile hotspot, etc.

The auxiliary earphone 420 may be an auxiliary earphone according to the corresponding embodiment of fig. 1.

The terminal device 410 is configured to determine a target sound feature according to the detected sound selection operation, and send the target sound feature to the hearing aid earphone 420; the target sound characteristic is used to determine a first target sound signal from among the initial sound signals collected by the secondary listening earpiece 420.

By way of example, the terminal device 410 may include, but is not limited to, a cell phone, a computer, a tablet computer, a wearable device, and the like.

For example, the terminal device 410 may display a sound management configuration interface, which may include a sound selection menu therein; the terminal device 410 may determine the target sound feature according to the detected sound selection operation facing the sound selection menu. For example, but not limited to, a sound selection menu in the sound management configuration interface may include selectable sound types, such as human, instrument, musical, whistling, animal-whistling, etc., from which a user may select one or more sounds of interest. The terminal device 410 determines, according to the selected sound, a sound feature corresponding to the selected sound, to obtain a target sound feature, and then sends the target sound feature to the auxiliary hearing device 420, and the auxiliary hearing device 420 may detect an initial voice signal collected by the microphone array based on the target sound feature, to determine whether the initial voice signal includes a first target sound signal matched with the target sound feature, that is, a sound of interest selected by a user.

For example, the terminal device 410 may manage the sound types in the sound selection menu, such as adding, deleting, etc. For each sound type, the terminal device 410 may collect a sound sample of the type in advance, and perform a sound feature analysis on the sound sample to obtain a sound feature corresponding to the sound type.

In another aspect, the present invention further provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a computer readable storage medium, where the computer program when executed by a processor can perform a method for enhancing sound of a hearing aid earphone provided by the foregoing method embodiments, where the method may include: determining a first sound source direction of the first target sound signal in the case that the first target sound signal is included in the collected initial sound signal; acquiring gesture information of the auxiliary hearing earphone, and determining face steering of a wearer of the auxiliary hearing earphone based on the gesture information; and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, enhancing the first target sound signal to obtain enhanced sound corresponding to the initial sound signal.

In still another aspect, the present invention further provides a non-transitory computer readable storage medium, on which a computer program is stored, the computer program being implemented when executed by a processor to perform a method for enhancing sound of a hearing aid earphone provided by the above method embodiments, where the method may include, for example: determining a first sound source direction of the first target sound signal in the case that the first target sound signal is included in the collected initial sound signal; acquiring gesture information of the auxiliary hearing earphone, and determining face steering of a wearer of the auxiliary hearing earphone based on the gesture information; and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, enhancing the first target sound signal to obtain enhanced sound corresponding to the initial sound signal.

By way of example, computer-readable storage media includes non-transitory computer-readable storage media.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method of enhancing sound of a hearing aid earphone, comprising:

determining a first sound source direction of a first target sound signal in the case that the first target sound signal is included in the collected initial sound signal;

acquiring the posture information of the auxiliary hearing earphone, and determining the face steering of a wearer of the auxiliary hearing earphone based on the posture information;

and under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold, enhancing the first target sound signal to obtain enhanced sound corresponding to the initial sound signal.

2. The method for enhancing the sound of the hearing aid earphone according to claim 1, wherein the enhancing the first target sound signal in the case that the target angle between the face steering and the first sound source direction is smaller than a preset angle threshold value includes:

and under the condition that the target included angle between the face steering and the first sound source direction is continuously detected to be smaller than the preset angle threshold value in a first preset time period, enhancing the first target sound signal.

3. The method for enhancing the sound of a hearing aid earphone according to claim 1 or 2, wherein the enhancing the first target sound signal comprises:

determining a target enhancement coefficient corresponding to the target included angle based on the corresponding relation between the included angle and the enhancement coefficient;

the first target sound signal is enhanced based on the target enhancement coefficient.

4. The sound enhancement method of a hearing aid earphone according to claim 1 or 2, further comprising:

in the process of enhancing the first target sound signal, under the condition that the change amount of the face turning is detected to be larger than a preset change amount threshold value, determining whether a second target sound signal is detected in the direction indicated by the changed face turning;

and enhancing the second target sound signal under the condition that the second target sound signal is detected in the direction indicated by the changed face steering.

5. The method of claim 4, wherein the enhancing the second target sound signal comprises:

under the condition that the included angle between the changed face turning direction and the second sound source direction is smaller than the preset angle threshold value in a second preset time period, the first target sound signal is enhanced based on a first enhancement coefficient, and the second target sound signal is enhanced based on a second enhancement coefficient; the first enhancement coefficient is smaller than the second enhancement coefficient, and the second sound source direction is the sound source direction of the second target sound signal.

6. The sound enhancement method of a hearing aid earphone according to claim 1 or 2, further comprising:

under the condition of receiving a communication connection request sent by terminal equipment, establishing communication connection with the terminal equipment;

and receiving target sound characteristics sent by the terminal equipment, wherein the target sound characteristics are used for determining the first target sound signal in the initial sound signal.

7. A sound enhancement device for a hearing aid earphone, comprising:

a sound source direction determining module, configured to determine a first sound source direction of a first target sound signal when it is detected that the collected initial sound signal includes the first target sound signal;

the face steering determining module is used for acquiring the gesture information of the auxiliary hearing earphone and determining the face steering of the wearer of the auxiliary hearing earphone based on the gesture information;

the sound enhancement module is used for enhancing the first target sound signal under the condition that the target included angle between the face steering and the first sound source direction is smaller than a preset angle threshold value, and obtaining enhanced sound corresponding to the initial sound signal.

8. The hearing aid earphone is characterized by comprising a microphone array, a direction sensor, a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the microphone array, the direction sensor and the memory are respectively in communication connection with the processor;

the microphone array comprises at least two microphones, wherein the at least two microphones are used for collecting initial sound signals and performing sound source localization on the initial sound signals;

the direction sensor is used for detecting the gesture information of the hearing aid earphone;

the processor, when executing the computer program, implements a method of sound enhancement for a hearing aid headset according to any one of claims 1 to 6.

9. A sound enhancement system comprising a terminal device and the hearing aid earpiece of claim 8, the terminal device being communicatively coupled to the hearing aid earpiece;

the terminal equipment is used for determining target sound characteristics according to the detected sound selection operation and sending the target sound characteristics to the hearing aid earphone; the target sound characteristic is used for determining a first target sound signal in initial sound signals collected by the auxiliary hearing earphone.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements a sound enhancement method of a hearing aid headset according to any one of claims 1 to 6.