DE102018112883A1 - Systems and methods for error detection in automatic speech recognition in a vehicle - Google Patents

Abstract

Disclosed are methods and apparatus for detecting ASR errors. An exemplary vehicle includes an automatic speech recognition (ASR) microphone, an air conditioner, and a processor. The processor is configured to receive data sensed by the microphone based on the received data to determine that a buzzer level detected by the microphone exceeds a threshold corresponding to buzzing caused by the air conditioner, and Responding to provide a fault message indicating that an air conditioning vent is directed to the microphone.

Description

TECHNICAL AREA

The present disclosure relates generally to automatic speech recognition (ASR) in a vehicle, and more particularly to systems and a method for error detection in vehicle ASR.

GENERAL PRIOR ART

Many modern vehicles may incorporate automatic speech recognition technology for use with hands-free telephony. The ASR technique often includes a microphone located in an interior of the vehicle to detect the driver's voice. Data from the microphone is processed to capture the word and commands spoken by the driver. Then a reasonable action is taken.

While the location of the microphone is helpful in detecting the driver's voice, it may involve sounds from various sources including the speakers, the HVAC system or the vehicle's open windows. These noise sources can cause the ASR to fail, resulting in a poor user experience.

SUMMARY

The appended claims define this application. The present disclosure sets forth aspects of the embodiments in a nutshell and is not to be used to limit the claims. Further implementations according to the techniques described herein are contemplated, as will become apparent to one of ordinary skill in the art after review of the following drawings and detailed description, and it is intended that these implementations fall within the scope of this application.

Embodiments are shown describing systems, apparatus, and methods for detecting the cause of faults in ASR for a vehicle. A disclosed exemplary vehicle includes an automatic speech recognition (ASR) microphone, an air conditioner, and a processor. The processor is configured to receive data sensed by the microphone based on the received data to determine that a buzzer level detected by the microphone exceeds a threshold corresponding to buzzing caused by the air conditioner, and Responding to provide a fault message indicating that an air conditioning vent is directed to the microphone.

One disclosed exemplary method involves receiving data captured by a vehicle microphone used for automatic speech recognition (ASR). The method also includes determining, based on the data, that a buzzer level detected by the microphone exceeds a threshold corresponding to thumps caused by an air conditioning system of the vehicle. The method further includes, in response, providing a fault message indicating that an air conditioning vent is directed to the microphone.

A third example may include means for receiving data detected by a vehicle microphone used for automatic speech recognition (ASR). The third example may also include means for determining on the basis of the data that a buzzer level detected by the microphone exceeds a threshold corresponding to thumps caused by an air conditioner of the vehicle. The third example may further include means for responsively providing a fault message indicating that an air conditioning vent is directed to the microphone.

list of figures

• 1 stellt eine perspektivische Innenansicht eines beispielhaften Fahrzeugs gemäß Ausführungsformen der vorliegenden Offenbarung dar.
• 2 stellt ein beispielhaftes Blockdiagramm elektronischer Komponenten des Fahrzeugs aus 1 dar.
• 3 stellt ein Ablaufdiagramm eines beispielhaften Verfahrens gemäß Ausführungsformen der vorliegenden Offenbarung dar.

For a better understanding of the invention, reference may be made to the embodiments shown in the following drawings. The components in the drawings are not necessarily to scale, and associated elements may be omitted, or in some instances, proportions may be exaggerated to emphasize and clearly illustrate the novel features described herein. In addition, system components may be arranged in different ways as known in the art. Further, in the drawings, like reference characters designate corresponding parts throughout the several views.

• 1 FIG. 12 illustrates an interior perspective view of an exemplary vehicle according to embodiments of the present disclosure. FIG.
• 2 illustrates an exemplary block diagram of electronic components of the vehicle 1 represents.
• 3 FIG. 3 illustrates a flowchart of an example method according to embodiments of the present disclosure. FIG.

DETAILED DESCRIPTION OF EMBODIMENTS

Although the invention may be embodied in various ways, some are exemplary and non-limiting embodiments shown in the drawings and described below, it being understood that the present disclosure is to be regarded as an example of the invention and is not intended to limit the invention to the specific embodiments illustrated.

As noted above, vehicles may include ASR technology for use by a driver such that the driver may drive "hands-free." The use of ASR technology may involve the driver pressing a button to start, the microphone detecting speech and other noise signals, and the processor processing the received signals to detect or determine whether words were being spoken that should be traded , The processing step may often require a signal-to-noise ratio threshold such that words can be extracted. In many cases, however, there are sources of noise that may affect the ability of the ASR system to recognize words spoken by the driver.

If the ASR system is unable to operate effectively due to noise, the driver may want to determine the source of the noise so that it can reduce the noise and allow the ASR system to operate. However, determining the cause of the noise is often difficult or impossible. This can result in the driver being disappointed with the ASR system and having an overall poor user experience.

Examples described herein may allow the ASR system to determine the cause of errors and allow the driver to correct these errors. An exemplary vehicle may include an automatic speech recognition microphone that may be located near a driver's head in a central portion of the vehicle.

The vehicle may also include an air conditioner having vents disposed at various locations in the vehicle. The air conditioner may include one or more fans or blowers that may be set at different levels. The air vents of the air conditioning system may also be configured to rotate, twist, or otherwise direct the exiting air.

In some examples, the air exiting the vents may be directed at the microphone, causing the microphone to be subject to noise that may cause speech recognition errors. Loudness may include vibration or movement of the microphone as air from a vent or vents is directed at the microphone. Microphones operate such that a diaphragm moves or bends in response to changes in air pressure and the movements are converted to an electronic signal. Thus, inadvertent movement by the air from the vents may render the microphone ineffective or unusable for ASR purposes.

The occurrence of noise can be determined based on the signals detected by the microphone. The example vehicle may include a processor configured to receive data collected by the microphone and to determine from the data the occurring buzzer level. The noise level can correlate with the velocity of the air exiting the vents.

In some examples, the buzzer level may be compared to a threshold. ASR technology can be functional even if the microphone is subject to noise noise. Also, buzzer noise may be caused by multiple sources, including air from the ventilation openings of the air conditioner, air from outside the vehicle entering through an open window. Therefore, the threshold value may be set to a value where ASR is no longer possible and such that the threshold value corresponds to a noise level caused by the air conditioner. If the detected buzzer level is above the threshold, the vehicle may indicate to the driver that the air from the vents is causing noise and causing ASR failure. The driver may then be instructed to change the direction of one or more vents to reduce whirring.

1 provides an interior perspective view of an exemplary vehicle 100 According to embodiments of the present disclosure. The vehicle 100 may be a standard gasoline powered vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle or a vehicle with a different type of mobility implementation. The vehicle 100 can be autonomous, semi-autonomous or autonomous. The vehicle 100 includes mobility-related parts, such as a powertrain with a motor, a transmission, a drive shaft and / or wheels, etc. In the example shown, the vehicle may 100 include one or more electronic components (hereinafter referred to as 2 described).

As in 1 shown, the vehicle can 100 a microphone 102 , air conditioning 104 , an ad 110 and a processor 112 include. The vehicle 100 can one or more more include electronic components, with reference to 2 are described in more detail.

The microphone 102 of the vehicle 100 may be a single microphone or include a variety of microphones. If the microphone 102 includes a variety of microphones, the microphone can 102 be an array arranged at a single position or over the vehicle 100 is distributed. Furthermore, the microphone 102 in an overhead portion of the vehicle roof (ie, near the driver's head), or may be in an overhead console, a rearview mirror, a door, a frame, a front panel, or another area of the vehicle 100 be arranged. In some examples, the position of the microphone 102 lying in line with air coming out of one or more vents 106A-D an air conditioner 104 exit.

The microphone 102 can be operated by flexing or moving in response to changes in air pressure. Therefore, bobbing may occur when air is on a diaphragm of the microphone 102 is directed. Tumming may involve low frequency, high amplitude non-periodic noise. In some examples, buzzing can cause noises between 0-2000 Hz. This noise can distort the microphone signal and make ASR difficult or impossible.

In some examples, the position and number of the microphone (s) 102 depend on the type of vehicle. For example, a smaller vehicle may include a single microphone, while larger vehicles may include a plurality.

The vehicle 100 can also have air conditioning 104 include. The air conditioning system may be configured to regulate the temperature of the vehicle, and therefore may include both heating and cooling elements. In some examples, the air conditioner 104 one or more ventilation openings 106A-D involve in different places in the vehicle 100 can be arranged.

The air conditioner 104 may include one or more controllers, buttons, or other user interface components. Furthermore, the air conditioning 104 be configured to operate with a variety of settings, including multiple fan speeds, vent configurations, fan speeds, etc. The fan speeds may range from 1 to 7, with 7 being a high setting and 1 a low setting. The air conditioner 104 may also be able to change the speed, temperature, volume and direction of air coming out of the vents 106A-D exit. Therefore, a buzzing noise level, coming from the air vents 106A-D which corresponds to the speed, temperature, volume and direction of the air exiting the vents.

The vehicle 100 can also have a processor 112 include. The processor may be configured to be from the microphone 102 received data. The data may include background noise and speech sounds (or data). In some examples, the features described herein may be performed in terms of background noise.

In some examples, the processor may 112 be configured to set a buzzer level to which the microphone 102 is exposed. This determination may be made on data acquired over a period of time, such as one second or more from the time a driver presses a PTT button. Therefore, the determination can be made before the driver speaks.

The buzzer level can be determined based on the fluctuation strength and frequency of the signal received by the microphone. The fluctuation strength can be measured in the unit vacil. In some examples, the buzzer level may be measured as a summation over a period of time (eg, 1 second). Further, the determination of the noise level may include an analysis of the data from the microphone with a fast Fourier transform (FFT). Loudness can generally correspond to low frequency, high amplitude noise. In this way, blustering can cause problems for human speech.

In some examples, whirring may be independent of masking sounds (eg, sounds from a vehicle radio, other talking voices in the vehicle, the running engine, etc.). Thus, noises may occur regardless of the noise level from other sources and cause ASR errors.

The processor 112 may be configured to determine that the buzzer level exceeds a threshold corresponding to rumbling caused by the air conditioner. The threshold may be set or dynamic and change over time. In some examples, the threshold level of the buzzer level may be based on the type of vehicle. For example, larger and smaller vehicles may have different thresholds, the vehicle type may include one or more vent locations, different HVAC systems having different power outputs, and configuration of the vehicle Specify ventilation openings, etc. Further, different types of vehicles may have different interior structures that may affect the airflow and thus the noise level.

In some examples, the threshold of a blower stage of the air conditioner 104 correspond. For example, a higher blower level may correspond to a higher threshold. Alternatively, a higher blower level may correspond to a lower threshold. The processor 112 may receive a blower level and responsively adjust the threshold based on the received blower level.

In some examples, the threshold setting process may include receiving initialization data that includes a plurality of voice inputs for ASR. The plurality of voice inputs may be at different air conditioning settings and vent configurations or orientations from the microphone 102 be recorded. The plurality of voice inputs (and any puffing) may then be analyzed to determine if voice can be recognized. The buzzer threshold may be responsively determined and may be tied to the buzzer level at which a threshold number of speech recognition errors occurs (eg, a 90% success rate). In this way, a threshold may correspond to a configuration of the vehicle in practice.

The vehicle 100 can also have an ad 110 include. The processor 112 may be configured over the display 110 to provide a fault message to the driver indicating that ASR will not work due to thumping. The display may include text or images indicating that the vent position causes air to be directed to the microphone. In some examples, the display may alert the driver that the direction of one or more vents may cause ASR to malfunction and may instruct the driver to change the vent configuration for ASR to function.

In some examples, the processor may 112 in response to a determination that the buzzer level to which the microphone is exposed exceeds the threshold, lowering the velocity of the air coming from one or more of the air vents 106A -D flows. This reduction can be automatic and depend on a current fan speed. Blower speed reduction can only take place when data is being received through the microphone (ie when the PTT button is pressed and shortly after). In this way, the air conditioner can continue to operate at a high level when the driver is not trying to speak and be understood by the ASR system.

In some examples, the reduction in fan speed may correspond to a sensed noise level such that the fan speed is reduced until the sensed noise level is below the threshold. For example, if the fan is operating at a high level, it can be reduced by two settings (for example, from 7 to 5). If the detected rumble is still above the threshold, the fan speed can be further reduced (eg, from 5 to 4). The reduction can be temporary, and the blower level can be restored as soon as the microphone detects the data.

In some examples, the processor may 112 cause one or more vents 106A-D Automatically change the direction of the outflowing air to divert air or to avoid putting it on the microphone 102 is directed. In this way, car sound errors can be mitigated without the driver having to intervene further.

2 provides an exemplary block diagram 200 represents the electronic components of the vehicle 100 according to some embodiments. In the example shown, the electronic components include 200 the on-board computer system 210 , an infotainment main unit 220 , Sensors 240 , electronic control unit (s) 250 and a vehicle data bus 260 ,

The onboard computing system 210 may be a microcontroller unit, a controller or a processor 112 and memory 212 include. At the processor 112 It may be any suitable processing device or set of processing devices, such as, without limitation, a microprocessor, a microcontroller-based platform, an integrated circuit, one or more Field Programmable Gate Arrays (FPGAs), and / or one or more application specific integrated circuits (ASICs). At the store 212 it may be volatile memory (eg, RAM including nonvolatile RAM, magnetic RAM, ferroelectric RAM, etc.), nonvolatile memory (eg, disk memory, FLASH memory, EPROMs, EEPROMs, memristor based nonvolatile solid state memory etc.), immutable memory (e.g., EPROMs), read only memory, and / or high capacity memory devices (eg, hard disks, solid state drives, etc.). In some examples, the memory includes 212 several types of memory, in particular volatile memory and non-volatile memory.

The memory 212 may be a computer readable medium upon which one or more sets of instructions may be embedded, such as the software for carrying out the methods of the present disclosure. The instructions may embody one or more of the methods or logic described herein. For example, the instructions are located wholly or at least partially in one or more of the memory 212 , the computer-readable medium, and / or during execution of the instructions in the processor 112 ,

The terms "non-transitory computer readable medium" and "computer readable medium" include a single medium or multiple media, such as a centralized or distributed database and / or associated caches and servers that store one or more sets of instructions. Further, the terms "non-transitory computer-readable medium" and "computer-readable medium" include any tangible medium that is capable of storing, encoding, or containing a set of instructions that are to be executed by a processor or that Have the system perform one or more of the methods or procedures disclosed herein. As used herein, the term computer-readable medium is expressly defined to include any type of computer-readable storage device and / or storage disk and to preclude propagation of signals.

The infotainment main unit 220 can be an interface between the vehicle 100 and provide to a user. The infotainment main unit 220 may include one or more input and / or output devices, such as the display 110 and the air conditioning interface 222 include. The air conditioning interface 222 can be an interface for the air conditioning 104 be. The input devices may include, for example, a controller, an instrument panel, a digital camera for image capture and / or recognition of visual commands, a touch screen, an audio input device (eg, a passenger microphone), buttons, or a touchpad. The output devices may include instrument cluster outputs (eg, scales, light fixtures), actuators, a heads-up display, a center console display (eg, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a flat panel display, a semiconductor display etc.) and / or speakers. In the example shown, the main infotainment unit includes 220 Hardware (eg, a processor or controller, memory, data storage, etc.) and software (eg, an operating system, etc.) for an infotainment system (such as SYNC® and MyFord Touch® from Ford®, Entune® from Toyota ®, IntelliLink® from GMC®, etc.). In some examples, the main infotainment unit 220 a processor with the on-board computing system 210 share. In addition, the main infotainment unit 220 the infotainment system, for example, on a center console display 110 of the vehicle 100 Show.

The sensors 240 can in any suitable way in and around the vehicle 100 be arranged. In the example shown, the sensors include 240 the microphone 102 , The microphone 102 can be electrically connected to the on-board computing system 210 coupled, such that the onboard computing system 210 with the microphone 102 Can receive / send signals. Other sensors may also be included, such as other noise detection sensors, airflow sensors, and more.

The ECUs 250 can subsystems of the vehicle 100 monitor and control. The ECUs 250 can via the vehicle data bus 260 communicate and exchange information. In addition, the ECUs 250 Characteristics (such as the status of the ECU 250 , Sensor readings, control status, fault and diagnostic codes, etc.) to other ECUs 250 transmit and / or receive requests from them. Some vehicles 100 can be seventy or more ECUs 250 at different positions in the vehicle 100 have, via the vehicle data bus 260 are communicatively coupled. The ECUs 250 may be separate electronics sets that include their own circuitry (s) (such as integrated circuits, microprocessors, memory, data storage, etc.) and firmware, sensors, actuators, and / or mounting hardware. In the example shown, the ECUs 250 the telematics control unit 252 , the body control unit 254 and the air conditioning 256 include.

The telematics control unit 252 can the pursuit of the vehicle 100 control, for example, using data provided by a GPS receiver, communication module 230 and / or one or more sensors. The body control unit 254 can be different subsystems of the vehicle 100 Taxes. For example, the body control unit 254 control the power of a trunk bolt, windows, power locks, power sunroof control, immobilizer system and / or powered mirrors, etc. The air conditioner 256 can control the speed, temperature, and volume of air exiting one or more vents. The air conditioner 256 can also detect the blower speed (and other signals) and over the data bus 260 to the onboard computing system 210 transfer. Other ECUs are also possible.

The vehicle data bus 260 may include one or more data buses that the on-board computing system 210 , the main infotainment unit 220 , the sensors 240 , the ECUs 250 and others with the vehicle data bus 260 couple connected devices or systems communicatively. In some examples, the vehicle data bus may 260 according to the CAN (controller area network) bus protocol as specified by the International Standards Organization (ISO) 11898 - 1 be implemented defined. Alternatively, the vehicle data bus 250 in some examples, may be a MOST (Media Oriented Systems Transport) bus or a CAN flexible data (CAN-FD) bus (ISO 11898-7).

3 FIG. 3 illustrates a flowchart of an example method. FIG 300 According to embodiments of the present disclosure. The method 300 For example, an ASR system may enable it to determine that a cause of errors is whirring to which the microphone is exposed and to alert a driver to the cause of the problem. The flowchart off 3 represents machine-readable instructions stored in memory (such as memory 212 ) and may include one or more programs that, when executed by a processor (such as the processor 112 ) the vehicle 100 and / or one or more systems or devices may perform one or more functions described herein. Although the exemplary program with reference to the flowchart of 3 Alternatively, many other methods of performing the functions described herein may be used. For example, the execution order of the blocks may be rearranged or performed in series or in parallel, blocks may be changed, dropped and / or combined to the method 300 perform. Because the procedure 300 further in connection with the components 1-2 is disclosed, some functions of these components will not be described in detail below.

The procedure 300 can to block 302 kick off. To block 304 can the procedure 300 determining a vehicle type. The vehicle type may correspond to the location of one or more microphones or vents, the characteristics of the air conditioner and may be used to determine a threshold level of the noise level.

To block 306 can the procedure 300 determining a blower level. The blower level may correspond to the threshold level of the buzzer level such that the threshold level changes based on a change in blower level.

To block 308 can the procedure 300 determining a threshold of the buzzer level. The threshold may be adjusted based on the particular vehicle type and blower level. In some examples, the threshold may also be determined based on initialization data including voice data processed with various vehicle settings to develop the threshold.

block 310 of the procedure 300 may include receiving data from the microphone. This may include pressing a PTT button by a driver and collecting data in time shortly after the button press. To block 312 can the procedure 300 then processing the data to determine a buzzer level to which the microphone is exposed. In some examples, the data may include both voice data and background data. The background data can be processed to determine the noise level.

The specific noise level can then go to block 314 be compared with the threshold. If the detected noise level is below the threshold, Block 316 of the procedure 00 include performing ASR. But if the rumble is above the threshold, the procedure can 300 to block 318 Include the provision of a fault message. The fault message may indicate to the driver that thumping is occurring and causing a fault in the ASR. The fault message may also indicate that the driver should change a direction of air flowing out of a vent to mitigate the buzzer error.

To block 320 can the procedure 300 include reducing a fan speed of the air conditioner. This can reduce the speed and / or volume of air directed at the microphone, which can partially or completely eliminate the rumble. In some examples, the method may 300 then to block 306 return where the blower level, threshold and buzzer level are redetermined. Thus, the method may continue to reduce the fan speed until the noise level is below the threshold. The procedure 300 can then go to block 322 end up.

In this application, the use of the disjunctive form is intended to include the conjunctive form. The use of definite or indefinite articles should not indicate cardinality. In particular, the reference to "the" object or "an" object should also refer to one of a possible plurality of objects. Further, the conjunction "or" may be used to convey features that coexist, rather than mutually exclusive alternatives. In other words, the conjunction "or" should also include "and / or". The terms "including," "including," and "including" are inclusive, and each have the same scope as "comprising," "comprising," and "comprising."

The embodiments described above, in particular any "preferred embodiments", are possible examples of implementations and are merely for a clear understanding of the principles of the invention. Many modifications and variations can be made to the embodiment or embodiments described above without departing substantially from the spirit and spirit of the techniques described herein. Accordingly, it is intended that all modifications come within the scope of this disclosure and be protected by the following claims.

Claims (15)

Vehicle comprising: a microphone for automatic speech recognition (ASR); an air conditioner; and a processor configured to: Receiving data detected by the microphone; Determining, based on the received data, that a buzzer level detected by the microphone exceeds a threshold corresponding to thumps caused by the vehicle's air conditioner; and, in response, providing a fault message indicating that an air conditioning vent is directed to the microphone. Vehicle after Claim 1 wherein the microphone is disposed in the interior of the vehicle on a path of air flowing from a ventilation opening of the air conditioner. Vehicle after Claim 1 wherein the air conditioner includes a vent, and wherein the noise level detected by the microphone corresponds to a velocity of air flowing out of the vent. Vehicle after Claim 1 wherein the data collected by the microphone comprises speech sounds and background sounds, and wherein the noise level detected by the microphone is based on the background noise. Vehicle after Claim 1 wherein the processor is further configured to: receive initialization data acquired by the microphone, the initialization data including a plurality of voice inputs for ASR; and determining the threshold based on the initialization data, the threshold corresponding to a value at which a number of speech recognition errors are below a threshold number. Vehicle after Claim 1 wherein the processor is further configured to: receive an air conditioning blower stage; and adjusting the threshold based on the air conditioner blower level. Vehicle after Claim 1 wherein the processor is further configured to: reduce a velocity of air from a vent of the air conditioner in response to determining that the noise level detected by the microphone exceeds the threshold corresponding to rumbling caused by the air conditioner. Vehicle after Claim 7 wherein the processor is further configured to reduce a velocity of air flowing out of the ventilation opening of the air conditioner until the noise level detected by the microphone is below the threshold corresponding to rumbling caused by the air conditioner. Method, comprising: Receiving data detected by a vehicle microphone used for automatic speech recognition (ASR); Determining, on the basis of the data, that a buzzer level detected by the microphone exceeds a threshold corresponding to thumps caused by an air conditioner of the vehicle; and, in response, providing a fault message indicating that an air conditioning vent is directed to the microphone. Method according to Claim 9 wherein the air conditioning system includes a vent and wherein the noise level detected by the microphone corresponds to a velocity of air flowing out of the vent. Method according to Claim 9 wherein the data collected by the microphone comprises speech sounds and background sounds, and wherein the noise level detected by the microphone is based on the background noise. Method according to Claim 9 further comprising: receiving initialization data acquired by the microphone, the initialization data including a plurality of voice inputs for ASR; and Determining the threshold based on the initialization data, the threshold corresponding to a value at which a number of speech recognition errors are below a threshold number. Method according to Claim 9 further comprising: receiving an air conditioning blower stage; and adjusting the threshold based on the air conditioner blower level. Method according to Claim 9 , further comprising: reducing a velocity of air from a vent of the air conditioner in response to determining that the noise level sensed by the microphone exceeds the threshold corresponding to rumbling caused by the air conditioner. Method according to Claim 14 , further comprising: reducing a velocity of air flowing out of the air vent of the air conditioner until the noise level detected by the microphone is below the threshold corresponding to rumbling caused by the air conditioner.

Images