CN116959140A

CN116959140A - Automatic analysis method, device, medium and program product for abnormal sound cause of vehicle

Info

Publication number: CN116959140A
Application number: CN202210372882.9A
Authority: CN
Inventors: 朱玉芳
Original assignee: BMW Brilliance Automotive Ltd
Current assignee: BMW Brilliance Automotive Ltd
Priority date: 2022-04-11
Filing date: 2022-04-11
Publication date: 2023-10-27

Abstract

The present disclosure relates to a method, apparatus, medium and program product for automatically analyzing a cause of abnormal sound of a vehicle. The method is used for automatically analyzing the cause of abnormal sound of the vehicle. The method comprises the steps of acquiring unknown abnormal sound data acquired in the vehicle in the running process of the vehicle. The method also includes extracting identifiable features of the unknown abnormal sound data. The method also includes analyzing identifiable features of the unknown abnormal sound data using the trained machine learning model to determine a cause of the abnormal sound. The trained machine learning model is trained using known abnormal sound data that has been labeled for abnormal sound causes.

Description

Automatic analysis method, device, medium and program product for abnormal sound cause of vehicle

Technical Field

The present disclosure relates generally to vehicle noise and vibration detection, and more particularly to detection for vehicle abnormal sound and automatic analysis of the cause of the abnormal sound.

Background

Noise, vibration and harshness (NVH) indicators of a vehicle are directly related to the comfort experience of the user. As users increasingly demand vehicle NVH indicators, vehicle manufacturers also increasingly pay attention to vehicle NVH indicators. On the one hand, the vehicle manufacturer can detect NVH indexes as much as possible before the vehicle goes offline so as to find problems in time, take remedial measures or optimize the vehicle design in advance, and strive to improve the NVH performance by 0 km. On the other hand, as the mileage of the vehicle increases, parts gradually wear and age, and NVH performance also deteriorates. High mileage NVH performance also affects a user's choice of brand of vehicle.

For NVH, since the acoustic roughness reflects transient characteristics of noise and vibration, detection and improvement of NVH is mainly also detection and improvement of vehicle noise and vibration. Among them, the vehicle abnormal sound (BSR) is an important constituent of NVH.

In the past, off-line detection of abnormal sounds in vehicles was mainly performed subjectively by experienced test operators. The test operator needs to pay attention to the sensitivity to abnormal sounds in the running process of the vehicle, has enough experience and tolerance to judge the type of the abnormal sounds, and analyzes the cause of the abnormal sounds. The process is highly subjective, takes a long time, and has no replicability.

Vehicle abnormal sound under high mileage is rarely detected and emphasized by vehicle manufacturers, but is difficult to detect and analyze by only relying on vehicle users. However, if left free, the problem of causing abnormal sound of the vehicle is not solved, and the vehicle is more damaged. In addition, the abnormal sound of the vehicle can cause interference to the driver of the vehicle, so that the driver is tired and dysphoric, and potential safety hazards are brought to the running of the vehicle.

Therefore, a technique capable of rapidly and accurately detecting abnormal sounds of a vehicle and analyzing causes of the abnormal sounds of the vehicle is required for test operators, vehicle users, and vehicle repair factories.

Disclosure of Invention

The following presents a simplified summary of the disclosure in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this summary is not an exhaustive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure or to delineate the scope of the disclosure. Its purpose is to present some concepts related to the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

According to one aspect of the present disclosure, a method for automatically analyzing a cause of abnormal sound of a vehicle is provided. The method comprises the steps of acquiring unknown abnormal sound data acquired in the vehicle in the running process of the vehicle. The method also includes extracting identifiable features of the unknown abnormal sound data. The method also includes analyzing identifiable features of the unknown abnormal sound data using the trained machine learning model to determine a cause of the abnormal sound. The trained machine learning model is trained using known abnormal sound data that has been labeled for abnormal sound causes.

According to another aspect of the present disclosure, an apparatus for automatically analyzing a cause of abnormal sound of a vehicle is provided. The apparatus includes a memory having instructions stored thereon and a processor configured to execute the instructions stored on the memory to perform a method according to the above-described aspects of the present disclosure.

According to yet another aspect of the present disclosure, there is provided a computer-readable storage medium comprising computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to the above aspects of the present disclosure.

According to yet another aspect of the present disclosure, there is provided a computer program product comprising computer executable instructions which, when executed by one or more processors, cause the one or more processors to perform a method according to the above aspects of the present disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.

The disclosure may be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings in which:

FIG. 1 illustrates a flow chart of a method for automatically analyzing a cause of an abnormal sound of a vehicle in accordance with an embodiment of the present disclosure;

FIG. 2 illustrates a flow chart of a method for training a machine learning model in accordance with an embodiment of the present disclosure;

FIG. 3 illustrates a flowchart of a method of acquiring unknown abnormal sound data, according to an embodiment of the present disclosure;

FIG. 4 illustrates a flowchart of a method for automatically analyzing a cause of an abnormal sound of a vehicle according to an embodiment of the present disclosure;

FIG. 5 illustrates a flowchart of a method for training a machine learning model, according to an embodiment of the present disclosure;

FIG. 6 illustrates a flow chart of a method for optimizing an abnormal sound cause according to an embodiment of the present disclosure;

FIG. 7 illustrates a flow chart of a method for optimizing an abnormal sound cause according to an embodiment of the present disclosure;

FIG. 8 illustrates an exemplary configuration of a computing device in which embodiments according to the present disclosure may be implemented.

Detailed Description

The following detailed description is made with reference to the accompanying drawings and is provided to assist in a comprehensive understanding of various example embodiments of the disclosure. The following description includes various details to aid in understanding, but these are to be considered merely examples and are not intended to limit the disclosure, which is defined by the appended claims and their equivalents. The words and phrases used in the following description are only intended to provide a clear and consistent understanding of the present disclosure. In addition, descriptions of well-known structures, functions and configurations may be omitted for clarity and conciseness. Those of ordinary skill in the art will recognize that various changes and modifications of the examples described herein can be made without departing from the spirit and scope of the present disclosure.

Fig. 1 illustrates a flowchart of a method 100 for automatically analyzing a cause of a vehicle abnormal sound according to an embodiment of the present disclosure. The method 100 may be performed by one or more processors of a computing device. An example of a computing device will be described in detail in connection with fig. 8. The computing device may be located inside a vehicle (hereinafter referred to as a "vehicle under analysis" or "vehicle") in which the vehicle is abnormal, for example as part of an electronic device integrated within the vehicle, as part of a vehicle controller, or as an electronic device mounted separately within the vehicle but communicatively coupled to the vehicle electronic device. The computing device may also be located external to the vehicle to be analyzed, communicatively coupled to the vehicle and other electronic devices, by wire or wirelessly, to obtain the various data needed (including abnormal sound data, machine learning model data, vehicle component parameters, vehicle operating conditions, road conditions, etc.).

In some embodiments, a vehicle abnormal sound (BSR) may include a superposition of one or more of three types of sound: vibration sound (buzz), friction sound (squeak), and percussive sound (rattle). The vibration sound is a high frequency "buzzing" sound, resembling the sound of a bee flapping a wing, usually produced under resonance conditions. The frictional noise is frictional noise caused by a stick-slip phenomenon generated by the relative movement of two parts in contact with each other. A clicking sound refers to the presence of relative motion of adjacent but non-contacting parts, resulting in a clicking-like noise. Vehicle abnormal sound is typically caused by inherent design manufacturing defects, later use loss, or poor, improper, or changing assembly or connection between components of the vehicle. It should be appreciated that the method of analyzing the cause of abnormal sound of a vehicle according to the present disclosure may not be limited to the above three types of sounds only, but may be applicable to detection and analysis of vehicle vibrations and noise more widely, including, for example, engine noise, tire noise, road noise, wind noise, or the like.

Analyzing the cause of the abnormal sound of the vehicle may include not only identifying the type of the abnormal sound (e.g., vibration sound, friction sound, or knocking sound), identifying the source of the abnormal sound (i.e., the position where the abnormal sound occurs, such as a front windshield, a glove box, or a backrest seat), but also determining how the abnormal sound is caused (e.g., whether the front windshield rubs against the dashboard, the glove box is not tightly closed, or the backrest seat is loose in snap connection). This is because even with the same two components, the abnormal sound characteristics due to different relative movements (e.g., contact friction or loose impact) are different, and the method according to the present disclosure is able to recognize such differences and make reverse inferences.

In step 110, the processor may obtain unknown abnormal sound data collected inside the vehicle during operation of the vehicle. The vehicle running process means that the vehicle is started, possibly displaced (i.e. running) or immobilized in situ. The term "unknown" of unknown abnormal sound data means that the corresponding cause of abnormal sound is not known, in contrast to known abnormal sound data that has been marked with the cause of abnormal sound. The abnormal sound data refers to vehicle abnormal sound which is collected through a sensor and recorded in an electronic mode, and can be a section of recorded audio. Whether for test operators or vehicle users, vehicle noise is of general concern from a perceptible perspective within the vehicle, and noise outside the vehicle (e.g., other vehicle horns or other environmental noise) is less of a concern, so the unknown noise data discussed herein may be collected within the vehicle.

Unknown abnormal sound data may be collected by a microphone. In some embodiments, the microphone may be a sound collection device that collects sound wave signals over a wide frequency band, wherein the wide frequency band includes a human-ear perceptible frequency band and a human-ear imperceptible frequency band. In other embodiments, the microphone may collect only human ear perceivable sounds.

In some embodiments, the processor may obtain unknown abnormal sound data collected by one or more microphones disposed within the vehicle interior. One or more microphones may be disposed in the vehicle interior. During operation of the vehicle, the microphone may collect unknown abnormal sound data, for example in the form of sound data including the unknown abnormal sound data. This approach is suitable for situations where the external ambient sound is relatively quiet, such as when a test operator is performing an off-line test on the vehicle.

In some embodiments, to reduce interference and extraneous noise in the collected sound data, and improve signal-to-noise ratio, one or more additional microphones may be disposed external to the vehicle for collecting ambient sound data external to the vehicle. Meanwhile, a microphone inside the vehicle still collects sound data including unknown abnormal sound data. Fig. 3 shows a flow chart of the processor employing this method to obtain unknown abnormal sound data. In step 11, the processor acquires sound data collected inside the vehicle from the in-vehicle microphone. In step 12, the processor obtains ambient sound data collected outside the vehicle from a microphone external to the vehicle. Interfering and extraneous noise sounds from outside the vehicle, such as horns from other vehicles, will be picked up by both the external microphone and the internal microphone. Therefore, purer unknown abnormal sound data can be obtained based on the sound data collected by the built-in microphone and the environment sound data collected by the external microphone, as shown in step 13, thereby improving the signal-to-noise ratio. For example, the same sound pulses as in the ambient sound data may be removed from the sound data. This approach is suitable for situations where the external ambient sound is relatively noisy, such as when a driver of a high mileage vehicle is driving the vehicle in a noisy city.

Whether an internal microphone is used alone, or both, the microphone may transmit sound data (including unknown abnormal sound data, sound data including unknown abnormal sound data, or ambient sound data) to a processor located locally or remotely to the vehicle for processing, or may process the sound data before transmitting it to the processor.

Returning to FIG. 1, in step 120, the processor may extract identifiable features of the unknown abnormal sound data. The identifiable characteristic is a parameter capable of characterizing different properties of the unknown abnormal sound data, can reflect the nature of the unknown abnormal sound data, and is useful for classification and analysis of the unknown abnormal sound data. Through feature extraction, the original unknown abnormal sound data can be converted into samples identifiable by a trained machine learning model. The extracted identifiable features should satisfy one or more of the following conditions: (1) The characteristics can better reflect the time-frequency domain characteristics of unknown abnormal sound data so as to provide enough information for analysis of a machine learning model; (2) This feature needs to be robust against environmental and noise disturbances; (3) The features can be extracted quickly because if the features are not easily extracted, the computation is time consuming, the whole system will react slowly and not work effectively.

In some embodiments, the identifiable features may include one or more of psychoacoustic characteristics, audio attribute-based features, and cepstral features. Psychoacoustic features may include loudness, pitch, timbre, etc. Features based on audio properties may include stationary features such as zero-crossing rate, short-time energy, subband energy ratio, short-time spectrum, etc. Such features are computationally inexpensive and are easily fused with other features to provide coarse information about the time and frequency domain content properties of the audio signal. Cepstrum features can well characterize human hearing. The cepstral features may include mel-cepstral coefficients and their differential coefficients, homomorphic cepstral coefficients, barker cepstral coefficients, linear predictive cepstral coefficients, and the like. In some embodiments, the identifiable feature may be selected from one or more of the following: fundamental frequency, harmony degree, spectrum center, zero-crossing rate, short-time energy, subband energy ratio, short-time spectrum, mel cepstrum coefficient, linear prediction cepstrum coefficient, line spectrum pair parameter and the like.

In some embodiments, to extract the identifiable feature, the processor may be based on at least one of three methods: frame-based feature processing method, subframe-based feature processing method, and sequential processing method. In the frame-based feature processing method, unknown abnormal sound data is divided into frames and features are extracted from each frame. In the subframe-based feature processing method, each frame is further divided into smaller subframes having overlaps than the frame-based feature processing method, and features are extracted from each subframe. In the sequential processing method, unknown outlier data is divided into smaller segments, typically 20-30 milliseconds long, overlapping by 50%, and features are extracted for each segment. The preprocessing and feature selection schemes for sound signals under different feature processing methods may be different.

In some embodiments, the processor may pre-process the unknown abnormal sound data prior to extracting the identifiable features. The preprocessing may include at least one of normalization, noise reduction, framing, pre-emphasis, windowing, and fourier transformation.

Normalization refers to the conversion of the specification and the storage format of original unknown abnormal sound data so as to ensure the consistency of the data. Normalization may be achieved, for example, by resampling, adjusting bit width, and amplitude scaling.

Noise reduction is to suppress noise in original unknown abnormal sound data and avoid interference with analysis of a subsequent machine learning model. In some embodiments, a wavelet filtering method may be employed for noise reduction processing. The wavelet filtering method may include spatial correlation filtering, modulo maximum wavelet filtering, and wavelet threshold filtering. In some embodiments, the filtering preprocessing may be performed using a spatial-domain-dependent wavelet filtering method. The spatial correlation wavelet filtering utilizes the correlation of the coefficients of wavelet transformation of signals on each scale, has the advantages of stable filtering performance and good filtering effect, and is particularly suitable for denoising signal-to-noise ratio signals such as abnormal sound data.

The framing is to divide the sound signal into short segments to meet the short-time stationarity, and is suitable for fourier transformation. Pre-emphasis is the emphasis of the high frequency part of the sound signal, increasing the high frequency resolution. The windowing cuts off the long-time sound signal sequence to avoid the Gibbs effect.

Fourier transform is the conversion of sound signals from the time domain to the frequency domain, facilitating the extraction of spectral features.

In other embodiments, preprocessing may also include signal processing means such as endpoint detection and time warping.

In step 130, the processor may analyze the identifiable features of the unknown abnormal sound data using the trained machine learning model to determine an abnormal sound cause.

The trained machine learning model is trained using known abnormal sound data that has been labeled for abnormal sound causes. FIG. 2 illustrates a flowchart of a method 200 for training a machine learning model according to an embodiment of the present disclosure. Method 200 may likewise be performed by a processor of a computing device executing method 100 of fig. 1, or by a processor of one or more other computing devices. In the latter case, the other computing device transmits the trained machine learning model to the processor of the computing device executing the method 100 to execute the method 100.

In step 210, the processor may acquire known abnormal sound data collected inside the vehicle during operation of the vehicle. The known abnormal sound data refers to abnormal sound data to which abnormal sound causes are to be marked for training a machine learning model, as compared with the unknown abnormal sound data. In addition, the known abnormal sound data and the unknown abnormal sound data can be identical in data format and acquisition mode. For example, the known abnormal sound data may be an abnormal sound of the vehicle recorded in an electronic form, which is collected by a sensor, for example, a recording. Similar to the unknown abnormal sound data, the known abnormal sound data can be acquired by one or more microphones in the vehicle, or can be acquired by the vehicle-mounted microphone and the vehicle-mounted microphone respectively and then processed, namely, the unknown abnormal sound data is similar to the description of the unknown abnormal sound data combined with fig. 3. The known abnormal sound data may be collected in the same vehicle or the same or similar model vehicle or the same manufacturer's vehicle as the unknown abnormal sound data. For example, to analyze unknown abnormal sound data of a vehicle of a certain manufacturer, known abnormal sound data for training a machine learning model may be the following samples: selecting N models of the same manufacturer, selecting M vehicles of each model, and selecting K abnormal sound signals for each vehicle for subsequent feature extraction, wherein N, M and K are integers greater than 1, and the total number of samples of known abnormal sound data is N.times.M.times.K.

In step 240, the processor may obtain an annotation of the cause of the abnormal sound of the known abnormal sound data. The known abnormal sound data may be marked for abnormal sound reasons manually (e.g., by a test operator). For example, during the test, the test operator perceives abnormal sound by the human ear and identifies the cause of the abnormal sound according to experience while the abnormal sound is collected by the microphone. Then, the test operator inputs the abnormal sound reason to the processor, and the processor correlates the abnormal sound reason with abnormal sound data acquired by the microphone, so that the marking of the abnormal sound reason of the known abnormal sound data is acquired.

At step 220, the processor may extract identifiable features of the known abnormal sound data. The feature extraction for the known abnormal sound data is similar to the feature extraction for the unknown abnormal sound data described above. In some embodiments, the identifiable feature may be selected from one or more of the following: fundamental frequency, harmony degree, spectrum center, zero-crossing rate, short-time energy, subband energy ratio, short-time spectrum, mel cepstrum coefficient, linear prediction cepstrum coefficient, line spectrum pair parameter and the like. To extract the identifiable feature, the processor may be based on at least one of three feature processing methods: frame-based feature processing method, subframe-based feature processing method, and sequential processing method. The preprocessing of the known alien data prior to extracting the identifiable features may include at least one of normalization, noise reduction, framing, pre-emphasis, windowing, and fourier transformation. In some embodiments, the identifiable characteristics of the known abnormal sound data extracted in step 220 and the feature processing method thereof, and the data preprocessing method for extracting the features may be identical to the identifiable characteristics for the unknown abnormal sound data in the method 100 of fig. 1 and the feature processing method thereof, and the data preprocessing method thereof.

At step 230, the processor may train the machine learning model using known abnormal sound data that has been labeled for abnormal sound reasons. Specifically, the processor may train the machine learning model using the identifiable features extracted at step 220 and the abnormal sound causes noted for the known abnormal sound data obtained at step 240. The trained machine learning model is the trained machine learning model used in step 130 of the method 100 of fig. 1. The machine learning model may be selected from one of a shallow structure machine learning model or a deep learning model. The shallow structure machine learning model may include classifiers such as gaussian mixture models, hidden markov models, decision trees, and support vector machines. The deep learning model may include an artificial neural network, a deep neural network, a convolutional neural network, a recurrent neural network, and the like. In some embodiments, prior to training the machine learning model, the processor may select an appropriate machine learning model based on the identifiable features extracted in step 220.

In some embodiments, known abnormal sound data may be divided into a training set, a validation set, and a test set. In some embodiments, the test set may not be necessary. The amount of data divided into the training set may be much greater than the amount of data divided into the validation set and the test set. The portion of the known abnormal sound data divided into the verification set is subjected to step 240, step 220 and step 230 to complete training of the machine learning model. The portion of the known abnormal sound data that is divided into the verification set is also subjected to step 240 and step 220, but is not used for training of the machine learning model, but is used for verification of the machine learning model, more specifically for evaluation in the model learning process, to assist in adjusting the model parameters. The portion of the known abnormal sound data divided into test sets, after undergoing steps 240 and 220, is then used to evaluate the final performance of the trained machine learning model after it has been trained by the machine learning model of step 230.

Returning to FIG. 1, at step 130, since the machine learning model has learned the inherent regularity between the abnormal sound characterized by the identifiable feature and the associated abnormal sound cause during the training process, it may be used to identify and analyze the unknown abnormal sound cause that is also characterized by the identifiable feature.

The abnormal sound is identified and analyzed by using the trained machine learning model, the method has the advantages of strong objectivity, high accuracy and high efficiency, and the defects of strong subjectivity, poor reliability, unrepeatable property and long time consumption of the traditional manual testing method are overcome. The training process of the machine learning model may be relatively time consuming, but once training is complete, the cause of abnormal sound may be analyzed in real time and efficiently. Moreover, the method is finished by means of a digital machine mode and is independent of individuals. Even users without identification experience can receive prompts or suggestions provided by the processor on abnormal sound reasons and possible repair modes in real time in the running process of the vehicle, so that measures can be taken timely, and larger safety accidents are avoided. And in the scene of offline test of vehicle manufacturers, the test efficiency can be improved, abnormal sound test can be carried out on more or even each vehicle, and the after-sales maintenance cost is reduced.

In some embodiments, after analyzing the abnormal sound cause in step 130, the processor may also generate a repair indication based on the troubleshooting database according to the determined abnormal sound cause. The troubleshooting database may store various abnormal sound causes and their corresponding repair patterns. The troubleshooting database may be entered based on the experience of the vehicle mechanic/serviceman in the form of an electronic database stored in the computing device. For example, if it is determined that the cause of abnormal sound is friction between the front windshield and the instrument panel, repair may be performed by adding a buffer layer to the contact surface between the front windshield and the instrument panel. For another example, if it is determined that the cause of abnormal sound is the glove box vibration, a repair method that improves the rigidity of the glove box plastic may be employed. The processor may obtain the troubleshooting database locally or remotely and retrieve a corresponding repair style in the troubleshooting database based on the determined abnormal sound cause and generate a repair indication including the corresponding repair style.

To improve the accuracy of the abnormal sound cause analysis, in further embodiments, more vehicle information associated with the occurrence of abnormal sound may be used to assist in training the machine learning model. For example, if the machine learning model is trained using only abnormal sound data, there may be a plurality of analyzed abnormal sound causes, but if associated vehicle component parameter data (i.e., structural parameters of each component in a stationary state of the vehicle) is input into the machine learning model to be trained together, the model can learn whether there is a correlation between the occurrence of abnormal sound and the vehicle component parameters, thereby being more confident about some abnormal sound causes at the time of analysis, or excluding some abnormal sound causes. Similarly, vehicle operating condition data, i.e., state parameters of the vehicle and its components during operating conditions, such as vehicle speed, vehicle braking conditions, vehicle window (including side windows, sunroof, etc.) opening conditions, etc., may also be associated with the occurrence of certain abnormal sounds. Thus, vehicle operating condition data may also be used in the training and prediction of machine learning models. Similarly, road condition data, including road type (e.g., distorted road, cobble road, belgium road, etc.), road condition (e.g., icy, muddy, snowy, etc.), etc., may also be associated with the occurrence of certain abnormal sounds. Thus, road condition data may also be used in the training and prediction of machine learning models.

Fig. 4-5 illustrate flow diagrams for analyzing abnormal sound causes (method 300) and for training a machine learning model (method 400) for associated vehicle component parameter data, vehicle operating condition data, road condition data, and the like. Method 300 and method 400 may be performed by the same processor or different processors. When executed by a different processor, the machine learning model trained by the method 400 is transmitted to the processor of the method 300 to perform the analysis.

In the method 300, the steps 310 and 320 are the same as the steps 110 and 120 in fig. 1, and are not described herein. At step 340, the processor obtains at least one of vehicle component parameter data, vehicle operating condition data, and road condition data associated with the unknown abnormal sound data. In some embodiments, "associated" may refer to vehicle component parameter data, vehicle operating condition data, and road condition data being acquired over the same period of time as unknown abnormal sound data. In some embodiments, "associated" may refer to vehicle component parameter data, vehicle operating condition data, and road condition data having an effect on the occurrence of unknown abnormal sound. The vehicle component parameter data, vehicle operating condition data and road condition data may be manually entered into a database for acquisition by the processor, or may be collected and stored by various sensors (including accelerometers, gyroscopes, image sensors, rangefinders, etc.) for acquisition by the processor.

In step 330, the processor may analyze the identifiable characteristics of the unknown abnormal sound data along with the at least one of the associated vehicle component parameter data, vehicle operating condition data, road condition data acquired in step 340 using the trained machine learning model to determine an abnormal sound cause. The trained machine learning model is trained using known abnormal sound data labeled with abnormal sound causes along with at least one of associated vehicle component parameter data, vehicle operating condition data, and road condition data. In some embodiments, the machine learning model may be trained using the method 400 of fig. 5.

In the method 400, the steps 410, 440 and 420 are the same as the steps 210, 240 and 220 of fig. 2, and are not described herein. At step 450, the processor obtains at least one of vehicle component parameter data, vehicle operating condition data, and road condition data associated with the known abnormal sound data. At least one of the associated vehicle component parameter data, vehicle operating condition data, and road condition data acquired for known abnormal sound data in step 450 is of the same type as at least one of the associated vehicle component parameter data, vehicle operating condition data, and road condition data acquired for unknown abnormal sound data in step 340 of fig. 4. That is, the auxiliary vehicle information used in the training process is the same type as the auxiliary vehicle information used in the analysis process. If vehicle operating condition data is used during the training process, the vehicle operating condition data will also be used during the analysis process. In some embodiments, "associated" may refer to vehicle component parameter data, vehicle operating condition data, and road condition data being acquired over the same period of time as known abnormal sound data. In some embodiments, "associated" may refer to vehicle component parameter data, vehicle operating condition data, and road condition data having an effect on the occurrence of known abnormal sounds.

At step 430, the processor trains the machine learning model using the known abnormal sound data labeled with the cause of the abnormal sound along with the at least one of the acquired associated vehicle component parameter data, vehicle operating condition data, and road condition data. Specifically, the processor may train the machine learning model using the identifiable features extracted at step 420, the abnormal sound causes noted for the known abnormal sound data obtained at step 440, along with at least one of the associated vehicle component parameter data, vehicle operating condition data, and road condition data obtained at step 450. The trained machine learning model is the trained machine learning model used in step 330 of the method 300 of fig. 4.

Further features and details regarding method 300 and method 400 may be referred to method 100 of fig. 1 and method 200 of fig. 2, respectively, and are not described in detail herein.

In some cases, even the cause of the abnormal sound determined using method 100 and method 300 may still be non-unique, with multiple. In order to further optimize the abnormal sound cause, the abnormal sound cause may be precisely located, and after the abnormal sound cause is determined using the method 100 or 300, some auxiliary vehicle information may be used to optimize the abnormal sound cause to reduce the investigation range. After optimizing the cause of the abnormal sound, the processor can provide more targeted repair suggestions, thereby improving the efficiency of eliminating the abnormal sound.

Fig. 6 illustrates a flowchart of one method for optimizing an abnormal sound cause according to an embodiment of the present disclosure. In step 140, the processor may determine at least one of a distance and an orientation of the source of the abnormal sound based on the unknown abnormal sound data. As previously described, the unknown abnormal sound data may be a piece of recorded data, i.e., an acoustic signal. The phase, angle of arrival, time of arrival, etc. of the acoustic signal may be used to measure the distance and/or orientation of the location where the abnormal sound occurs (i.e., the source of the abnormal sound). In step 150, after determining the distance and/or the location of the source of the abnormal sound, some abnormal sound causes can be eliminated based on the distance and/or the location, so as to optimize the abnormal sound causes. For example, for one type of rub acoustic, the method 100 or method 300 may determine that the cause of the abnormal sound is the rubbing of the windshield against the operator panel, but may be the front windshield or the rear windshield. In this case, if the direction of analyzing the abnormal sound data is in front of the sensor located in the middle of the vehicle cabin, it can be determined that the cause of the abnormal sound is friction of the front windshield with the operation panel, not friction of the rear windshield with the baffle.

Fig. 7 illustrates a flowchart of another method for optimizing an abnormal sound cause according to an embodiment of the present disclosure. In step 160, the processor may acquire status data of one or more vehicle components collected by one or more sensors associated with the determined cause of abnormal sound. The processor may then optimize the cause of the abnormal sound based on the acquired vehicle component status data in step 170. For example, two possible causes of abnormal sound may be determined using method 100 or method 300, associated with part a and part B, respectively. If the states of the component A and the component B acquired by the sensor are combined again in the occurrence time or the related time period of abnormal sound, whether the abnormal sound is caused by the component A or the component B can be further determined, and therefore the cause of the abnormal sound is optimized.

The processor may perform one of the methods shown in fig. 6 and 7 alone or in combination.

Fig. 8 illustrates an exemplary configuration of a computing device 800 capable of implementing embodiments in accordance with the present disclosure.

Computing device 800 is an example of a hardware device that can employ the above aspects of the present disclosure. Computing device 800 may be any machine configured to perform processing and/or calculations. Computing device 800 may be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a Personal Data Assistant (PDA), a smart phone, an in-vehicle computer, or a combination thereof.

As shown in fig. 8, computing device 800 may include one or more elements that may be connected to or in communication with bus 802 via one or more interfaces. Bus 802 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, among others. Computing device 800 can include, for example, one or more processors 804, one or more input devices 806, and one or more output devices 808. The one or more processors 804 may be any kind of processor and may include, but is not limited to, one or more general purpose processors or special purpose processors (such as special purpose processing chips). The processor 802 may be configured to implement one or more of the methods 100, 200, 300, or 400, for example. Input device 806 may be any type of input device capable of inputting information to a computing device and may include, but is not limited to, a mouse, keyboard, touch screen, microphone, and/or remote controller. Output device 808 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video/audio output terminals, vibrators, and/or printers.

The computing device 800 may also include or be connected to a non-transitory storage device 814, which non-transitory storage device 814 may be any storage device that is non-transitory and may enable data storage, and may include, but is not limited to, disk drives, optical storage devices, solid state memory, floppy disks, flexible disks, hard disksMagnetic tape or any other magnetic medium, a compact disc or any other optical medium, a cache memory and/or any other memory chip or module, and/or any other medium from which a computer may read data, instructions, and/or code. Computing device 800 may also include Random Access Memory (RAM) 810 and Read Only Memory (ROM) 812. The ROM 812 may store programs, utilities or processes to be executed in a non-volatile manner. The RAM 810 may provide volatile data storage and stores instructions related to the operation of the computing device 800. Computing device 800 may also include a network/bus interface 816 that is coupled to data link 818. The network/bus interface 816 may be any kind of device or system capable of enabling communication with external apparatuses and/or networks and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication devices, and/or chipsets (such as bluetooth) ^TM Devices, 802.11 devices, wiFi devices, wiMax devices, cellular communication facilities, etc.).

The present disclosure may be implemented as any combination of apparatuses, systems, integrated circuits, and computer programs on a non-transitory computer readable medium or computer program product. One or more processors may be implemented as an Integrated Circuit (IC), application Specific Integrated Circuit (ASIC), or large scale integrated circuit (LSI), system LSI, super LSI, or ultra LSI assembly that performs some or all of the functions described in this disclosure.

The present disclosure includes the use of software, applications, computer programs, or algorithms. The software, application, computer program or algorithm may be stored on a non-transitory computer readable medium or computer program product to cause a computer, such as one or more processors, to perform the steps described above and depicted in the figures. For example, one or more memories may store software or algorithms in executable instructions and one or more processors may associate a set of instructions to execute the software or algorithms to provide various functions in accordance with the embodiments described in this disclosure.

The software and computer programs (which may also be referred to as programs, software applications, components, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural, object-oriented, functional, logical, or assembly or machine language. The term "computer-readable medium" refers to any computer program product, apparatus or device, such as magnetic disks, optical disks, solid state memory devices, memory, and Programmable Logic Devices (PLDs), for providing machine instructions or data to a programmable data processor, including computer-readable media that receives machine instructions as a computer-readable signal.

By way of example, computer-readable media can comprise Dynamic Random Access Memory (DRAM), random Access Memory (RAM), read Only Memory (ROM), electrically erasable read only memory (EEPROM), compact disk read only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired computer-readable program code in the form of instructions or data structures and that can be accessed by a general purpose or special purpose computer or general purpose or special purpose processor. Disk or disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above are also included within the scope of computer-readable media.

The subject matter of the present disclosure is provided as examples of apparatuses, systems, methods, and programs for performing the features described in the present disclosure. However, other features or variations are contemplated in addition to the features described above. It is contemplated that the implementation of the components and functions of the present disclosure may be accomplished with any emerging technology that may replace any of the above-described implementation technologies.

In addition, the foregoing description provides examples without limiting the scope, applicability, or configuration set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the spirit and scope of the disclosure. Various embodiments may omit, replace, or add various procedures or components as appropriate. For example, features described with respect to certain embodiments may be combined in other embodiments.

Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method for automatically analyzing a cause of abnormal sound of a vehicle, comprising:

acquiring unknown abnormal sound data acquired in the vehicle in the running process of the vehicle;

extracting identifiable characteristics of unknown abnormal sound data; and

using a trained machine learning model to analyze identifiable features of unknown abnormal sound data to determine an abnormal sound cause,

wherein the trained machine learning model is trained using known abnormal sound data labeled with abnormal sound causes.

2. The method of claim 1, further comprising:

the unknown abnormal sound data collected by one or more microphones disposed inside the vehicle is acquired.

3. The method of claim 2, further comprising:

acquiring sound data including unknown abnormal sound data of a vehicle interior collected by one or more microphones arranged in the vehicle interior;

acquiring ambient sound data outside the vehicle collected by one or more additional microphones disposed outside the vehicle; and

the unknown abnormal sound data is determined based on the sound data and the ambient sound data.

4. The method of claim 1, wherein the trained machine learning model is trained using known abnormal sound data annotated with abnormal sound causes along with at least one of associated vehicle component parameter data, vehicle operating condition data, and road condition data, the method further comprising:

acquiring the at least one of vehicle component parameter data, vehicle operating condition data and road condition data associated with the unknown abnormal sound data; and

the identifiable features of the unknown abnormal sound data are analyzed, along with the at least one of vehicle component parameter data, vehicle operating condition data, and road condition data associated with the unknown abnormal sound data, using a trained machine learning model to determine an abnormal sound cause.

5. The method of claim 1, further comprising:

and generating a repair instruction based on the troubleshooting database according to the determined abnormal sound cause.

6. The method of claim 1, wherein the identifiable features comprise one or more of:

a fundamental frequency;

harmony;

a spectral center;

zero crossing rate;

short time energy;

a subband energy ratio;

short-time spectrum;

mel-frequency cepstrum coefficient;

linear prediction coefficients;

linear predictive coefficient cepstral coefficients; and

line spectrum versus parameters.

7. The method of claim 1, wherein the machine learning model comprises a deep learning model.

8. The method of claim 1, wherein the type of abnormal sound comprises at least one of a vibratory sound, a frictional sound, and a clicking sound.

9. The method of claim 1, further comprising:

and before the identifiable characteristics are extracted, filtering the unknown abnormal sound data by using a spatial domain correlation wavelet filtering method.

10. The method of claim 1, further comprising:

determining at least one of a distance and an azimuth of the abnormal sound source based on the unknown abnormal sound data; and

the cause of the abnormal sound is optimized based on at least one of a distance and an azimuth of the source of the abnormal sound.

11. The method of claim 1, further comprising:

acquiring status data of one or more vehicle components associated with an abnormal sound cause acquired by one or more sensors; and

optimizing for abnormal sound causes based on the status data of the one or more vehicle components.

12. An apparatus for automatically analyzing a cause of abnormal sound of a vehicle, comprising:

a memory having instructions stored thereon; and

a processor configured to execute instructions stored on the memory to perform the method according to any one of claims 1 to 11.

13. A computer-readable storage medium comprising computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any of claims 1-11.

14. A computer program product comprising computer-executable instructions which, when executed by one or more processors, cause the one or more processors to perform the method of any of claims 1 to 11.