US20140364171A1 - Method and system for improving voice communication experience in mobile communication devices - Google Patents

Method and system for improving voice communication experience in mobile communication devices Download PDF

Info

Publication number
US20140364171A1
US20140364171A1 US14/233,423 US201314233423A US2014364171A1 US 20140364171 A1 US20140364171 A1 US 20140364171A1 US 201314233423 A US201314233423 A US 201314233423A US 2014364171 A1 US2014364171 A1 US 2014364171A1
Authority
US
United States
Prior art keywords
user
mode
communication device
mobile communication
speaker
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/233,423
Other languages
English (en)
Inventor
Arie Heiman
Uri Yehuday
Roei Roeimi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
DSP Group Ltd
Original Assignee
DSP Group Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by DSP Group Ltd filed Critical DSP Group Ltd
Priority to US14/233,423 priority Critical patent/US20140364171A1/en
Assigned to DSP Group reassignment DSP Group ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HEIMAN, ARIE, YEHUDAY, URI, ROEIMI, Roei
Publication of US20140364171A1 publication Critical patent/US20140364171A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/60—Substation equipment, e.g. for use by subscribers including speech amplifiers
    • H04M1/6033—Substation equipment, e.g. for use by subscribers including speech amplifiers for providing handsfree use or a loudspeaker mode in telephone sets
    • H04M1/6041—Portable telephones adapted for handsfree use
    • H04M1/605—Portable telephones adapted for handsfree use involving control of the receiver volume to provide a dual operational mode at close or far distance from the user
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/02—Constructional features of telephone sets
    • H04M1/0202—Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets
    • H04M1/026—Details of the structure or mounting of specific components
    • H04M1/0266—Details of the structure or mounting of specific components for a display module assembly
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2250/00—Details of telephonic subscriber devices
    • H04M2250/12—Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/46—Special adaptations for use as contact microphones, e.g. on musical instrument, on stethoscope
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2460/00—Details of hearing devices, i.e. of ear- or headphones covered by H04R1/10 or H04R5/033 but not provided for in any of their subgroups, or of hearing aids covered by H04R25/00 but not provided for in any of its subgroups
    • H04R2460/03—Aspects of the reduction of energy consumption in hearing devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2460/00—Details of hearing devices, i.e. of ear- or headphones covered by H04R1/10 or H04R5/033 but not provided for in any of their subgroups, or of hearing aids covered by H04R25/00 but not provided for in any of its subgroups
    • H04R2460/13—Hearing devices using bone conduction transducers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/11—Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/02—Circuits for transducers, loudspeakers or microphones for preventing acoustic reaction, i.e. acoustic oscillatory feedback

Definitions

  • the invention relates to the field of communication system. More specifically, the invention relates to voice communication enhancement in mobile devices using bone conduction phenomena.
  • voice communication by mobile devices is an integral part of a person's life.
  • Voice enhancement technology is constantly advancing, and improved devices are developed all the time.
  • voice communication in noisy environment is a challenge with which mobile have difficulty coping.
  • the microphone of a mobile communication device picks up a signal that combines the voice of the mobile phone user and the ambient noise. As a result, a low quality of voice is provided to the party at the other end of the line.
  • speaker mode enables freer movement, most people use the speaker mode only in case that the environment is quite and there is little ambient noise or no ambient noise and, of course, other people are not around so that privacy can be maintained.
  • the handheld mode in which the phone is in contact with one of their ears. This is due to the fact that in speaker mode the party with which the user is conversing hears a lot of ambient noise, and also the phone user cannot hear his party's voice clearly, because of the noise around him.
  • the noise is high, many phone users attach the phone to one of their ears and simultaneously close the other ear with their hand, to reduce the amount of the ambient noise that they hear.
  • noise reduction techniques Sometime also referred to as “noise cancellation techniques”
  • Examples for some mobile phones that use two microphones are: iPhone 4 from Apple Inc. and Samsung's Galaxy S2.
  • the noise reduction technique improves the transmitted voice call to some extent, however that is only a partial solution, since the user that is talking in the noisy environment still suffers from the ambient noise and in many cases the received voice from the other party is not heard well.
  • bone conduction is known in the art, mainly as a solution for people who suffer from hearing impairment; in such mode the sound waves are transformed into vibrations and are transmitted to the inner ear through the skull bone.
  • This technique is also used in the field of earpieces bone conduction, for example, as described in international patent application WO 2010/052720, which discloses an earpiece and a method for playing a stereo and a mono signal using a bone conduction speaker.
  • Bone conduction speakers were also used in Kyocera's smartphone—Urbano Progresso, which employs exclusively bone conduction speakers.
  • the use of bone conduction speakers requires a direct contact between the mobile phone speaker and in any point of the user's skull
  • the invention relates to a mobile communication device having a body that houses electronic components and a front panel comprising a display, said mobile communication device further comprising at least one standard microphone and a vibration sensor located in vibrational connection with the front panel of the device.
  • the mobile communication device further comprises a bone conduction speaker.
  • vibration detector a vibration detector
  • vibration sensor a vibration sensor
  • bone conduction microphone a vibration sensor
  • the vibration sensor is attached to the front panel, but in other embodiments it is located away from the front panel, as long as it is in vibrational connection with it.
  • the invention is directed to a mobile communication device comprising a processor configured to operate differently as a result of a determination that the user of the mobile communication device is operating it in one of the following modes:
  • the mode is i) and the processor activates one or more standard microphones; or the mode is ii) or iii) and the processor activates two or more standard microphones located at two different locations in the communication device, and further operates a vibration detector positioned so as to detect vibration generated by the user's voice on the device's panel; or the mode is iv) and the processor activates two or more standard microphones, a vibration detector and a bone conduction speaker.
  • the processor when the bone conducting speaker is activated the processor is configured to balance the signal transmitted to the ears so as to provide to the user a voice of approximately the same intensity in both ears.
  • the invention provides many advantages so far unavailable in the art. For instance, it allows to provide an indicator suitable to alert the user that the phone is located at a distance from his mouth that is greater than a predetermined value.
  • the indication can be provided, for example, by generating an acoustic signal or a visual signal.
  • the invention allows to provide a noise reduction indicator suitable to provide an indication to the user of the level of noise reduction that a conversation is enjoying.
  • the invention is directed to a method for operating a mobile communication device having a body that houses electronic components and a front panel comprising a display, said mobile communication device further comprising at least one standard microphone and a vibration sensor located in vibrational connection with the front panel of the device, the method comprising providing a processor configured to operate differently as a result of a determination that the user of the mobile communication device is operating it in one of the following modes:
  • FIG. 1 schematically shows a speaker mode, where the mobile device is held at a large distance from the user's mouth;
  • FIG. 2 schematically shows a speaker mode, where the mobile device is held at a relatively close distance from the user's mouth;
  • FIG. 4 schematically shows a handheld mode, where the mobile device is held so that there is direct contact between the device and the user's ear, and simultaneously the other ear is closed by the user;
  • FIG. 5 schematically shows a mobile device panel with a vibration detector according to an embodiment of the invention
  • FIG. 6 shows an example according to an embodiment of the invention, where the mobile device is held so that there is one point of direct contact between the device and the user's skull;
  • FIG. 7 shows an example according to an embodiment of the invention, where the mobile device is held so that there is direct contact between the device and the user's skull;
  • FIG. 8 schematically shows the structure of the mobile device according to an embodiment of the invention.
  • FIG. 9 is a schematic flow chart of the method of the invention according to an embodiment of the invention.
  • FIG. 10 schematically describes the system which estimates the original speech s(n) and the ambient noise d(n);
  • FIG. 11 shows a schematic flow chart which describes an example of a speech detection mechanism.
  • the present invention relates to a noise reduction system and apparatus that improve the transmitted voice quality in noisy environment, and improves the perception of the incoming call by taking into consideration the different modes of use by phone users in noisy environments.
  • different processes are performed for each mode of use, and thus full advantage is taken of the various capabilities offered by different modes of use.
  • multi standard microphones and bone conduction microphones also referred to as “bone vibration detector” are used in the mobile communication device, as well as bone conduction speakers and a processor.
  • phone mobile communication device
  • mobile communication device any other kind of mobile equipment, such as Wi-Fi and radio apparatus, regardless of the medium over which the transmission takes place.
  • the processor is configured according to the invention to automatically analyse the mode of operation of the mobile phone and automatically performs a different process adjusted to each mode, by taking advantage of bone conduction microphones, bone conduction speakers, as appropriate under the specific operating conditions, as will be further elaborated below.
  • the processor is configured to recognize the mode of operation of the mobile phone and automatically switches between the different modes of operation to reduce the effect of the ambient noise.
  • each mode is typically used in a different ambient noise scenario, and is briefly described below with reference to the figures:
  • speaker mode Mobile phone users prefer to use the speaker during voice call (the “speaker mode”).
  • the use of speaker mode has a big advantage as it doesn't force the user to have a contact between the phone and his ear and is normally considered to be healthier because the distance is maintained between the user's head and the phones antenna.
  • voice command or voice search applications such as SIRI from Apple Inc. or Google now, users want to view the feedback to their command or question that the application provides, hence they are holding the phone in front of their face.
  • speaker mode introduces some challenges.
  • One of the problems is echo, namely, the microphone of the mobile devices “hears” also the speaker and retransmitted it to the far end user where he hears himself as an echo.
  • standard adaptive echo cancellation technique can be used, such as that exemplified in http://www.slideshare.net/chintanajoshi/acoustic-echo-cancellation a presentation by Chintan A. Joshi of NIRMA University.
  • Another issue is the fact that the distance between the mouth of the user and the microphone that is located on the mobile phone, varies significantly during the phone conversation.
  • the present invention takes advantage of the fact that the phone is in contact with the skull.
  • the invention uses an array of standard microphones combined with bone conduction microphone and a signal processor.
  • the processor processes all the signals that were picked up by the microphones (including the bone conduction microphones) with a process that can eliminate significantly the ambient noise that the other party hears, and if desired can completely eliminate the background noise that the other party hears.
  • the bone conduction microphone is actually a bone vibration detector (also referred to as “bone conduction detector”).
  • Said bone vibration detector has to be in a vibrational contact with the mobile phone's panel, and therefore it can be embedded in the mobile phone as part of the panel or behind the panel, regardless the exact location of the vibration detector, as long as there is a contact between the bone vibration detector and the panel.
  • FIG. 5 schematically illustrates a mobile phone panel 510 in which the bone vibration detector 520 is embedded as a part of the frame of the panel 530 or behind the panel.
  • the bone vibration detector can be positioned far away from the front panel, e.g., next to the phone battery, and it will still provide the advantages of the invention as long as it is in vibrational contact with said from panel.
  • the bone vibration detector is utilized to detect bone vibration caused when the user is talking and the phone touches the user's skull as schematically shown in FIG. 3 or 4 .
  • FIG. 6A shows an example where the user is talking and there is one point of contact 645 between the phone panel 610 and the user's skull 615 .
  • FIG. 6B is an enlarged view of the phone held by the user of FIG. 6A , rotated by approximately 90° relative to the working position of FIG. 6A , so as to show the front panel.
  • the bone vibration 630 generated by the user's voice at 645 propagates through the panel glass to the vibration detector 620 .
  • the vibration is detected by the bone vibration detector 620 (which is a bone conduction microphone) that is embedded in, or attached to the panel or behind the panel.
  • the picked up vibration represents the speech of the user.
  • vibration detector location does not necessarily have to be behind the location where the user touch his skull, it is sufficient that the vibration detector touches the panel in any location as the vibration that the panel pick propagates over the whole display panel.
  • the vibration detector detects the user's voice when the panel has no contact with the user's skull, if the panel is close enough (e.g., below ⁇ 30 cm) to the user′ mouth, as schematically illustrated in FIG. 2 .
  • the direct pressure caused by the speech on the panel vibrates the panel, and these vibrations are detected by the vibration detector, which is located on the panel or below the panel. If the panel is far from the user's mouth as shown in FIG. 1 , the direct pressure caused by the speech on the panel is too small and no meaningful vibration is detected by said vibration detector.
  • the useful distance between the speakers mouth and the panel of the communication device will vary in different models of phones.
  • the processor that oversees the various modes is configured to activate a process that reduces the noise for the other party by using a signal processor and an array of standard microphones combined with the bone vibration detector. It must be noted that although the panel doesn't touch the bone, the direct pressure caused by the speech on the panel vibrates the panel, since it is close enough to the user's mouth.
  • the processor processes all the signals that were picked up by the microphones (including the bone vibration microphone) with a process that can eliminate significantly the ambient noise that the other party hears, and if desired it can completely eliminate the background noise that the other party hears.
  • the bone vibration detector (bone conduction microphone) can be embedded in the mobile phone as part of the panel or behind the panel attached to it, for example in the battery case. As already noted above, the location of the bone vibration detector does not necessarily have to be in front of the user's mouth, it is sufficient that the bone vibration detector touches the panel in any location, as the vibration that the panel picks propagates over the whole display panel.
  • the invention takes advantage of the fact that the phone is attached to the skull and in addition to the use of bone vibration detector as a bone conduction microphone, the system and apparatus of the invention use a bone conduction speaker which touches the user's skull and which is embedded in the phone.
  • the bone conduction speaker passes the incoming voice on to the user.
  • another advantage of the system of the invention is that the standard speakers and the bone conduction speakers simultaneously deliver the incoming voice to the user, thus increasing the volume of the sound received in the ear.
  • FIG. 7 schematically shows an exemplary case, in which the user attaches the mobile phone 710 to the right ear 701 , and there is more than one point of contact between the mobile phone and the user's skull; hence the standard speaker 705 delivers the voice call to the user through the right side air canal of the ear.
  • the Bone conduction speaker 703 that is also in contact with the user's right skull, works differently; it vibrates the skull and the sound wave 715 propagates through the bone to both ears 701 and 702 .
  • the user hears the voice in both ears 701 and 702 although the phone is in contact only with the right side of the skull.
  • the processor analyses and processes both signals, to achieve balance between the voices that are heard in both ears. The fact that the user hears the voice in both ears improves the received voice intelligibility as both ears hear the incoming call.
  • the penetrated ambient noise decreases and due to the occlusion effect, it boosts the propagated signal to the left ear and significantly improves the SNR in this ear.
  • no balance can be achieved if only one source is used. Assuming the device is attached to the right side, the signal heard in the right side is much stronger than what is heard in the left side (due the attenuation of the sound through the bone), which generates an unbalanced signal; Thus, the user will hear mainly the right-ear signal.
  • balance could be in principle achieved with one source, if the vibrator is located on the top of the head, forehead or on the neck, this is obviously not a practical solution for cellular phone users.
  • the processor automatically analyzes and recognizes the mode of operation of the mobile phone and automatically selects its own appropriate mode of operation.
  • the processor switches between the different processes as the mode of operation is changed and uses a part or all the above microphones and/or speakers (regular and bone conduction) to achieve the best noise reduction effect.
  • FIG. 8 schematically shows an example of a typical structure of a mobile phone 800 according to an embodiment of the invention.
  • the system and apparatus of the invention consist of standard multi microphones 810 that can be located in the phone in different locations.
  • the mobile phone 800 comprises 6 microphones 810 .
  • more microphones can be inserted and used (not shown) or only part of them.
  • only two microphones located in different locations can be used, such as the microphones at the bottom of the phone and another at the top of the phone.
  • the two microphones can be located at the bottom of the phone, or one microphone can be located at the bottom of the phone and another one located at the left or right side of the phone, etc.
  • the microphones 810 are connected to processor 825 via a digital or analog connection 801 .
  • Standard speakers 815 and 816 are fed by processor 825 via the digital or analog connection 801 .
  • Speaker 815 is used in handheld mode as earpiece speaker when the phone is in contact with the user's ear and speaker 816 is used in speaker mode.
  • a bone vibration detector 805 which detects the vibration on the display panel 802 , is embedded in the mobile phone as part of the panel 802 .
  • the vibration detector 805 can also be located behind the panel (not shown) as long as there is a vibrational contact between the panel 802 and the vibration detector 805 .
  • vibration detector 805 When the user talks and brings the phone into contact with his skull, the bone vibration generated by the user's voice passes to the phone panel 802 , which vibrates accordingly, the vibration of the panel 803 , is detected by vibration detector 805 .
  • the picked up vibration represents the speech of the talking person.
  • a typical vibration detector can be made by using piezoelectric elements that are sensitive to voice vibration, or by a standard microphone covered with rubber, where the rubber touches the glass of the panel.
  • the output of the vibration detector 805 is fed to processor 825 via the digital or analog connection 801 .
  • a proximity detector 804 is also embedded in the panel of the phone and it is used to detect if the user hold the phone close to his cheek. The proximity detector can be found in most of the mobile phones that have a touch screen panel.
  • the output of the proximity detector 804 is fed to processor 825 via the digital or analog connection 801 .
  • the vibration detector location does not necessarily have to be behind the location where the user touch his skull, it is sufficient that the skull touches the panel at any location as the vibration that the panel picks, propagates over the whole display panel.
  • a bone conduction speaker 820 is embedded in the phone 800 and is located at the upper part of the phone 800 , so that in a handheld mode the bone conduction speaker 820 is in contact with the user's skull.
  • the speech signal is delivered from processor 825 to the speaker 820 via digital or analog connection 801 .
  • the bone conduction speaker 820 passes incoming voice on to the user.
  • the system of the present invention simultaneously delivers to the user the received voice from speaker 815 and from bone conduction speaker 820 .
  • the standard speaker 815 delivers the voice call to the user through the right side air canal of the ear.
  • the Bone conduction speaker which also is in contact with the user's skull, vibrates the skull and the sound wave propagates through the bone to both ears. In this case the user hears the voice in both ears although the phone is in contact only with one side of the skull.
  • the processor 825 processes both signals to achieve balance between the voices that are heard in both ears. The fact that the user hears the voice in both ears improves the received voice intelligibility as both ears hear the incoming call.
  • the penetrated ambient noise decreases and due to the occlusion effect it boost the propagated signal through the bone to the ear that is not in contact with mobile phone, which significantly improve the SNR in this ear.
  • Processor 825 can be a dedicated processor that includes processing capabilities such as an ARM processor or a DSP processor, which interfaces with the connection 801 .
  • the processor 825 can be implemented in the application processor or in the baseband processor, which are currently found in many mobile phones.
  • the processor 825 detects the mode of operation of the user, namely in which mode out of the four following modes the user acts:
  • processor 825 analyzes two signals: the output of the vibration detector 805 and of the proximity detector 804 .
  • processor 825 analyzes two signals: the output of the vibration detector 805 and of the proximity detector 804 .
  • the output of the vibration detector 805 and of the proximity detector 804 are used, which are fed to processor 825 via connection 801 .
  • the proximity detector 804 is used to detect if the user holds the phone close to his cheek. If the phone is held close to the user's cheek, it means that the phone is used in a handheld mode and so the panel is shut down during this mode of operation in order to save power.
  • the processor uses this information as described in FIG. 9 .
  • FIG. 9 is a schematic flow chart of the method according to one embodiment of the invention.
  • Method 900 is activated when a user operates a voice communication in the mobile phone.
  • the signal Energy-bone(n) which is the energy of the bone signal received from the vibration detector 805 , is filtered by a low pass filter such as
  • Ebone( n ) Alpha*Ebone( n ⁇ 1)+(1 ⁇ alpha)*Energy-bone( n )
  • step 910 the proximity criteria are checked. If the proximity criteria are off, it means that the phone is not in contact with the user's face.
  • step 915 the mode of use detected at this step is speaker mode of “Case II”. If the proximity criteria are on, step 915 is carried out. In step 915 the proximity is on and therefore it means that the mode of use is either “Case III” or “Case IV”. If the environmental noise level “Noise_Level” is below a predefined threshold TH 2 it means that the noise is not too high, and the user doesn't have to close his unused ear. In that case, the detected mode of use is “Case III”.
  • Noise_Level is bigger than the predefined threshold TH 2 , it means that the user is in a high noise environment, and then the system activates the bone speaker, and indicates that it has been activated, e.g., by a vocal prompt or by a short beep, suggesting closing the second ear.
  • the system provides to the user the ability to balance the sound between the two ears by using balance control button 806 .
  • “Case I” is a standard speaker mode, where the mobile phone is far from the user's mouth. In this case it is assumed that the ambient noise is very low, hence the processor 825 activates one microphone 810 , which is located at the bottom of the phone 800 , in addition to some standard echo cancellation process that is performed to eliminate echo generated by the speaker. It should be noted that a different number of microphones 810 can be activated—for example, two microphones or more. Also standard Noise cancellation techniques can be performed.
  • Case II is a standard speaker mode, where the mobile phone is close to the user's mouth.
  • the ambient noise level is in the range of low to medium noise level.
  • the user speaks in speaker mode but holds the phone close to his mouth in order to overcome the environmental noise and to be able to hear the incoming call via the speaker.
  • the processor uses two or more microphones 810 located in different location in the phone, as well as the bone vibration detector 805 that detects the voice vibration on the display panel 802 of the phone. When the user talks and his mouth is close to the display panel of the phone, the vibration generated by the user's voice passes to the phone panel, which vibrates accordingly.
  • vibration detector 805 that is embedded in the phone or attached to the panel.
  • the picked-up vibration represents the speech of the talking person.
  • sensor 805 is sensitive to the user's vibration and much less to the ambient noise. Without attempting to provide a full explanation of this fact, which has been experimentally determined, it is believed that it may be mainly due to the fact that only waves that hit the phone's panel perpendicularly create a substantial vibration, which is picked up by the vibration detector. In most cases significant part of the ambient noise does not perpendicularly impinge on the phone's panel hence the amount of noise the vibration detect is significant smaller compared to the total noise.
  • the user might change the phone position relative to the user's mouth during the phone call.
  • the voice level detected by 805 is strong and its energy Ebone is bigger than the predefined threshold Th 1 .
  • the processor 825 provides a notification to the user how far is the mobile phone from the mouth.
  • the indication can be by voice annotation during the call, or alternatively the processor can forward a command to the panel controller to display a bar that indicates to the user the strength of the energy Ebone. This indication acts as a feedback to the user and indicates to the user if the phone distance from his mouth meets the desired distance.
  • the system is configured to automatically recognize a new mode of use and thus automatically switches and adjusts to handle the user's mode of use as Case I.
  • the signal that is detected in the standard two microphones M 1 (n) and M 2 (n) can be described by:
  • s(n) is the speech produced by the near end user; d(n) is the ambient noise in the near end; n 1 (n) n 2 (n) is noise of the pickup equipment; ⁇ (n) is the filter that the speech undergoes relative to m1; ⁇ (n) is the filter that the noise undergoes relative to m1; and * represent convolution;
  • a typical vibration detector such as 805 is sensitive to the user's vibration up to some voice frequencies, and much less to the ambient noise.
  • the vibration detector 805 i.e. the bone conduction microphone signal M 3 (n)
  • ⁇ (n) is a low pass filter that models the vibration sensor characteristics and n 3 (n) is noise of the vibration sensor.
  • the target is to estimate the original speech s(n) and the ambient noise d(n) which are denoted as ⁇ (n) and ⁇ circumflex over (d) ⁇ (n) respectively.
  • ⁇ (n) is the “clean speech” signal that is transmitted to the far end user.
  • s(n) can be estimated by various known MMSE (Minimum Mean Square Error) technique.
  • e(n) represents the estimation error, namely:
  • E ⁇ ⁇ is the mean operator
  • h1(n), h2(n) and h3(n) can be calculated by LMS adaptation process:
  • h i ( n+ 1) h i ( n )+ ⁇ e *( n ) ⁇ M i ( n )
  • FIG. 10 schematically describes the system, which estimates the original speech s(n) and the ambient noise d(n).
  • System 1000 consists of two main blocks 1001 and 1005 .
  • 1001 estimates the signal s(n) and d(n) denoted as ⁇ (n) and ⁇ circumflex over (d) ⁇ (n).
  • 1005 is the block that updates the values of the filters h1(n), h2(n), h3(n).
  • M1(n) is fed to 1010
  • M2(n) is fed to 1020
  • M3(n) is fed to 1030
  • the sum of the three filters output is ⁇ tilde over (s) ⁇ (n)
  • H k (z) is the Z transform of h k (n)
  • k 1, 3.
  • the decision if a frame is a speech or a silence frame is calculated as described in FIG. 11 .
  • the appropriate error is chosen by mux 1055 .
  • the error is calculated by using filter 1040 and is
  • the error signal is ⁇ tilde over (s) ⁇ (n)
  • the switch of speech/silent frame can also be used to change the adaptation weights (step size) in 1010 , 1020 , 1030 . All the process of 1000 can be implemented in the processor 825 .
  • FIG. 11 shows a schematic flow chart which describes an example of a speech detection mechanism.
  • the vibration detector 805 detects the signal M 3 (n) every speech frame of T ms., M 3 (n) is combined of the low pass version of speech signal and the inherent noise n3(n) of the vibration sensor.
  • the detector 805 is sensitive to the low pass version of speech signal but it is barely sensitive to the ambient noise, so that the ambient noise which is detected by the detector 805 is negligible.
  • TH 1 the energy of M 3 (n) to a predefined threshold TH 1 it can be decided if the user is speaking or not.
  • the talking user who is talking in a noisy environment and whose mobile phone's apparatus operates a noise reduction process, has no indication of the amount of noise that was reduced, or of how much his transmitted voice is clean from ambient noise.
  • the present invention uses the error estimate e(n) as an indication of the amount of noise that was reduced.
  • This information can be used as a visual indication to be displayed on the mobile phone panel, for example by some bars. A full bar indicates a good cancellation and a small bar indicates a poor cancellation. Alternatively an audible indication can also be used.
  • This case is a regular mode where the user attaches the phone to one of his ears and where the ambient noise level is in the range of mid to high noise level, or there is no noise but the user prefers to use this mode.
  • the user attaches the phone to one of his ears in order to increase the incoming sound level as well as to increase the voice level of the talking person relative to the ambient noise level, and in order to reduce the noise effect for the other party.
  • processor 825 uses two microphones 810 located at different location in the phone, as well as the vibration detector 805 .
  • the vibration detector 805 detects the vibration from the phone panel, which vibrates accordingly.
  • This vibration is detected by vibration detector 805 that is embedded in the panel or is in contact with the panel 802 .
  • the picked-up vibration represents the speech of the talking person.
  • detector 805 is sensitive to the user's vibration and much less to the ambient noise.
  • Processor 825 is configured to process those three signals in a manner which is similar to the way used in “Case II”, with different adaptation parameters.
  • the bone conduction speaker passes the incoming voice to the user.
  • the standard speaker 815 and the bone conduction speaker 820 simultaneously deliver the incoming voice to the user.
  • the standard speaker delivers the incoming voice call to the user through the side air canal of said ear.
  • the bone conduction speaker 820 that is also attached to the user's skull works differently—it vibrates the skull and the sound wave propagates through the bone to both ears. In this case the user hears the voice in both ears, although the phone is attached only to the right side.
  • the processor processes both signals to achieve balance between the voices that is heard in both ears.
  • both speakers 815 and 820 are in contact with one side of the skull, and due to the fact that the received voice is injected to the user by the standard speaker 815 and bone conduction speaker 820 , said side which is in contact with the phone hears the signal Sr(n) which is the sum of the standard microphone voice signal and the bone conduction voice signal, namely:
  • Bl(n) and Br(n) are the attenuation filter, that voice signal s(n) undergo while propagating from the contact point on the bone to the left and right sides of the user ears.
  • f denote frequency Ar is adjustable gain of the standard speaker that can be used to balance between the volumes of the two ears.
  • Sl(n) is the incoming voice signal heard in the ear which is not in contact with the mobile phone, due the voice signal s(n) injected by the bone and propagated from the contact point on the bone in the side which is in contact with the mobile phone to the ear which is not in contact with the mobile phone, and so:
  • Bl and Br are parameters that depend on the user's bone conduction of sound, hence to generate the balance it can be measured during a calibration process, or alternatively a balance key 806 is added to the system, which allows the user to change the balance to a point that he hears in both ear the same volume.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Telephone Function (AREA)
  • Telephone Set Structure (AREA)
  • Details Of Audible-Bandwidth Transducers (AREA)
  • Circuit For Audible Band Transducer (AREA)
US14/233,423 2012-03-01 2013-02-25 Method and system for improving voice communication experience in mobile communication devices Abandoned US20140364171A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/233,423 US20140364171A1 (en) 2012-03-01 2013-02-25 Method and system for improving voice communication experience in mobile communication devices

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201261605287P 2012-03-01 2012-03-01
US14/233,423 US20140364171A1 (en) 2012-03-01 2013-02-25 Method and system for improving voice communication experience in mobile communication devices
PCT/IL2013/050164 WO2013128442A1 (en) 2012-03-01 2013-02-25 A method and system for improving voice communication experience in mobile communication devices

Publications (1)

Publication Number Publication Date
US20140364171A1 true US20140364171A1 (en) 2014-12-11

Family

ID=49081735

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/233,423 Abandoned US20140364171A1 (en) 2012-03-01 2013-02-25 Method and system for improving voice communication experience in mobile communication devices

Country Status (6)

Country Link
US (1) US20140364171A1 (de)
EP (1) EP2820861A4 (de)
JP (1) JP2015513854A (de)
KR (1) KR20140145108A (de)
CN (1) CN103718568A (de)
WO (1) WO2013128442A1 (de)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104658536A (zh) * 2015-03-09 2015-05-27 深圳酷派技术有限公司 录音方式切换方法、录音方式切换系统和终端
US9420391B2 (en) * 2014-12-02 2016-08-16 Infineon Technologies Ag Microphone configuration and calibration via supply interface
US20160286299A1 (en) * 2015-03-27 2016-09-29 Intel Corporation Intelligent switching between air conduction speakers and tissue conduction speakers
US20170163778A1 (en) * 2014-06-10 2017-06-08 Sharp Kabushiki Kaisha Audio transmission device with display function
WO2020036534A1 (en) * 2018-08-13 2020-02-20 Audio Zoom Pte Ltd Transducer apparatus embodying non-audio sensors for noise-immunity
WO2020041194A1 (en) * 2018-08-20 2020-02-27 Cirrus Logic International Semiconductor Ltd. Pinna proximity detection
US10631075B1 (en) * 2018-11-12 2020-04-21 Bose Corporation Open ear audio device with bone conduction speaker
WO2021061055A1 (en) * 2019-09-26 2021-04-01 Audio Zoom Pte Ltd Transducer apparatus: positioning and high signal-to-noise-ratio microphones
US11438078B2 (en) * 2018-07-24 2022-09-06 Comcast Cable Communications, Llc Controlling vibration output from a computing device

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9143875B2 (en) * 2013-09-09 2015-09-22 Nokia Technologies Oy Determination of ambient sound processed audio information
CN103778909B (zh) * 2014-01-10 2017-03-01 瑞声科技（南京）有限公司 屏幕发声系统及其控制方法
CN203951519U (zh) * 2014-07-04 2014-11-19 阿里巴巴集团控股有限公司 一种移动通信终端
KR101694592B1 (ko) 2014-11-18 2017-01-09 재단법인 다차원 스마트 아이티 융합시스템 연구단 골전도 스피커를 이용하는 착용형 장치
US11587559B2 (en) * 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
EP3157266B1 (de) 2015-10-16 2019-02-27 Nxp B.V. Steuerung für ein haptisches rückmeldungselement
CN106231088B (zh) * 2016-07-27 2019-12-10 宇龙计算机通信科技(深圳)有限公司 一种语音通话的方法、装置及终端
CN106254599A (zh) * 2016-08-29 2016-12-21 苏州攀特电陶科技股份有限公司 一种移动设备

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5559570A (en) * 1993-05-24 1996-09-24 Nikon Corporation Camera having an anti-vibration device with improved positioning of anti-vibration sensors
US20040209654A1 (en) * 2003-04-15 2004-10-21 Cheung Kwok Wai Directional speaker for portable electronic device
US20050185813A1 (en) * 2004-02-24 2005-08-25 Microsoft Corporation Method and apparatus for multi-sensory speech enhancement on a mobile device
US20090303009A1 (en) * 2008-06-06 2009-12-10 Fujitsu Limited Information processing apparatus for transmitting and receiving RF signals
US20110293105A1 (en) * 2008-11-10 2011-12-01 Heiman Arie Earpiece and a method for playing a stereo and a mono signal
US8255018B2 (en) * 2006-04-04 2012-08-28 Plaster Morgan J Global positioning system (GPS) enabled apparatus for carrying objects and method therefor
US20120306823A1 (en) * 2011-06-06 2012-12-06 Apple Inc. Audio sensors
US8965460B1 (en) * 2004-01-30 2015-02-24 Ip Holdings, Inc. Image and augmented reality based networks using mobile devices and intelligent electronic glasses

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2377581B (en) * 2001-07-10 2005-01-19 New Transducers Ltd Communication device using bone conduction
US7142666B1 (en) * 2002-10-31 2006-11-28 International Business Machines Corporation Method and apparatus for selectively disabling a communication device
CA2535671A1 (en) * 2004-01-16 2005-07-28 Temco Japan Co., Ltd. Portable telephone using bone conduction device
EP1816829A4 (de) * 2004-12-02 2009-09-09 Panasonic Corp Tragbares endgerät
JP2007003702A (ja) * 2005-06-22 2007-01-11 Ntt Docomo Inc 雑音除去装置、通信端末、及び、雑音除去方法
WO2009141828A2 (en) * 2008-05-22 2009-11-26 Bone Tone Communications Ltd. A method and a system for processing signals
KR101068254B1 (ko) * 2011-02-01 2011-09-28 주식회사 보니아코퍼레이션 골전도 기능을 갖는 통신단말기

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5559570A (en) * 1993-05-24 1996-09-24 Nikon Corporation Camera having an anti-vibration device with improved positioning of anti-vibration sensors
US20040209654A1 (en) * 2003-04-15 2004-10-21 Cheung Kwok Wai Directional speaker for portable electronic device
US8965460B1 (en) * 2004-01-30 2015-02-24 Ip Holdings, Inc. Image and augmented reality based networks using mobile devices and intelligent electronic glasses
US20050185813A1 (en) * 2004-02-24 2005-08-25 Microsoft Corporation Method and apparatus for multi-sensory speech enhancement on a mobile device
US8255018B2 (en) * 2006-04-04 2012-08-28 Plaster Morgan J Global positioning system (GPS) enabled apparatus for carrying objects and method therefor
US20090303009A1 (en) * 2008-06-06 2009-12-10 Fujitsu Limited Information processing apparatus for transmitting and receiving RF signals
US20110293105A1 (en) * 2008-11-10 2011-12-01 Heiman Arie Earpiece and a method for playing a stereo and a mono signal
US20120306823A1 (en) * 2011-06-06 2012-12-06 Apple Inc. Audio sensors

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170163778A1 (en) * 2014-06-10 2017-06-08 Sharp Kabushiki Kaisha Audio transmission device with display function
US9832293B2 (en) * 2014-06-10 2017-11-28 Sharp Kabushiki Kaisha Audio transmission device with display function
US9420391B2 (en) * 2014-12-02 2016-08-16 Infineon Technologies Ag Microphone configuration and calibration via supply interface
CN104658536A (zh) * 2015-03-09 2015-05-27 深圳酷派技术有限公司 录音方式切换方法、录音方式切换系统和终端
US20160286299A1 (en) * 2015-03-27 2016-09-29 Intel Corporation Intelligent switching between air conduction speakers and tissue conduction speakers
US10097912B2 (en) * 2015-03-27 2018-10-09 Intel Corporation Intelligent switching between air conduction speakers and tissue conduction speakers
US11757539B2 (en) 2018-07-24 2023-09-12 Comcast Cable Communications, Llc Controlling vibration output from a computing device
US11438078B2 (en) * 2018-07-24 2022-09-06 Comcast Cable Communications, Llc Controlling vibration output from a computing device
CN112673646A (zh) * 2018-08-13 2021-04-16 奥正有限公司 包含非音频传感器的抗干扰换能器设备
US20210314700A1 (en) * 2018-08-13 2021-10-07 Audio Zoom Pte Ltd Transucer apparatus embodying non-audio sensors for noise-immunity
US11647330B2 (en) * 2018-08-13 2023-05-09 Audio Zoom Pte Ltd Transducer apparatus embodying non-audio sensors for noise-immunity
WO2020036534A1 (en) * 2018-08-13 2020-02-20 Audio Zoom Pte Ltd Transducer apparatus embodying non-audio sensors for noise-immunity
US10923097B2 (en) 2018-08-20 2021-02-16 Cirrus Logic, Inc. Pinna proximity detection
GB2591636A (en) * 2018-08-20 2021-08-04 Cirrus Logic Int Semiconductor Ltd Pinna proximity detection
WO2020041194A1 (en) * 2018-08-20 2020-02-27 Cirrus Logic International Semiconductor Ltd. Pinna proximity detection
GB2591636B (en) * 2018-08-20 2022-10-05 Cirrus Logic Int Semiconductor Ltd Pinna proximity detection
WO2020102224A1 (en) * 2018-11-12 2020-05-22 Bose Corporation Open ear audio device with bone conduction speaker
US10631075B1 (en) * 2018-11-12 2020-04-21 Bose Corporation Open ear audio device with bone conduction speaker
WO2021061055A1 (en) * 2019-09-26 2021-04-01 Audio Zoom Pte Ltd Transducer apparatus: positioning and high signal-to-noise-ratio microphones

Also Published As

Publication number Publication date
CN103718568A (zh) 2014-04-09
KR20140145108A (ko) 2014-12-22
EP2820861A4 (de) 2015-08-26
WO2013128442A1 (en) 2013-09-06
JP2015513854A (ja) 2015-05-14
EP2820861A1 (de) 2015-01-07

Similar Documents

Publication Publication Date Title
US20140364171A1 (en) Method and system for improving voice communication experience in mobile communication devices
JP7108071B2 (ja) ノイズ低減のためのオーディオ信号処理
US10791402B2 (en) Hearing aid device for hands free communication
US9066167B2 (en) Method and device for personalized voice operated control
US8543061B2 (en) Cellphone managed hearing eyeglasses
US10269369B2 (en) System and method of noise reduction for a mobile device
US8675884B2 (en) Method and a system for processing signals
US9491542B2 (en) Automatic sound pass-through method and system for earphones
US8320974B2 (en) Decisions on ambient noise suppression in a mobile communications handset device
JP4202640B2 (ja) 短距離無線通信用ヘッドセット、これを用いたコミュニケーションシステム、および短距離無線通信における音響処理方法
US9167333B2 (en) Headset dictation mode
US20110158420A1 (en) Stand-alone ear bud for active noise reduction
CN114466277A (zh) 具有听通模式的头戴式耳机及其操作方法
US20140294182A1 (en) Systems and methods for locating an error microphone to minimize or reduce obstruction of an acoustic transducer wave path
JP2007019898A (ja) 携帯電話機
US20170374476A9 (en) Hearing Eyeglass System and Method
US9654855B2 (en) Self-voice occlusion mitigation in headsets
US20110105034A1 (en) Active voice cancellation system
JP6813169B2 (ja) 音声抑制装置
CN115776637A (zh) 包括用户接口的助听器

Legal Events

Date Code Title Description
AS Assignment

Owner name: DSP GROUP, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HEIMAN, ARIE;YEHUDAY, URI;ROEIMI, ROEI;SIGNING DATES FROM 20140126 TO 20140129;REEL/FRAME:032129/0968

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION