KR102601171B1 - Method for providing sound detection information, apparatus detecting sound around vehicle, and vehicle including the same - Google Patents

Abstract

Pertains to a method of providing acoustic tracking information that can distinguish between the noise of one's own vehicle or another vehicle, an acoustic tracking device for a vehicle, and a vehicle including the same, which provides acoustic tracking results based on acoustic data generated by detecting sounds occurring around the own vehicle. A step of generating, extracting sound tracking pattern features based on the sound tracking results and analyzing whether the detected sound is the own vehicle sound, and generating a driver notification according to the analysis result, and determining whether the detected sound is the own vehicle sound. The step of analyzing recognition includes a first condition in which the maximum value for the sound source location probability exists within a specific angle range from the sound tracking result, and a predetermined sound source location probability value at any first angle and second angle from the sound tracking result. If the second condition of being greater than the threshold is satisfied, the detected sound can be analyzed as self-vehicle sound.

Description

Method for providing acoustic tracking information, acoustic tracking device for vehicle, and vehicle including same

The present invention relates to a method for providing acoustic tracking information, an acoustic tracking device for a vehicle, and a vehicle including the same, and more specifically, a method for providing acoustic tracking information that can distinguish between the noise of one's own vehicle or another vehicle, an acoustic tracking device for a vehicle, and the same. It relates to vehicles including:

In general, a lot of research has been conducted on sound source tracking technology, but there are not many cases of application to vehicles.

A representative sound source tracking technology is TDOA (Time Difference Of Arrival) technology, which uses multiple microphones to estimate the direction of origin of a sound source using the difference in arrival delay time between microphones.

Examples of applications of sound source tracking technology include sound source tracking systems for CCTV security, noise source measurement systems inside vehicles, and sound source tracking systems for intelligent robots.

This sound source tracking technology recognizes surrounding vehicle sounds by analyzing sound signals measured using multiple microphones, and tracks the direction of origin of the sound source using the magnitude value or arrival delay time difference of the signal input between each microphone. Information can be provided to the driver.

However, existing sound source tracking technology cannot accurately distinguish between sounds generated by other vehicles nearby and sounds generated by the own vehicle, which may result in errors in the notification information provided to the driver.

In other words, the existing sound source tracking technology had problems in that it could not accurately determine sounds generated from other vehicles due to the influence of noise generated from the own vehicle.

Therefore, in the future, there is a need for technology that can clearly distinguish whether the detected sound is the sound of one's own vehicle or the sound of another vehicle and provide accurate notification information to the driver.

The present invention provides a method for providing acoustic tracking information that can provide accurate information about a detected sound by extracting acoustic tracking pattern features based on the acoustic tracking results and analyzing whether the detected sound is the own vehicle sound or another vehicle sound, and a vehicle sound system. The object is to provide a tracking device and a vehicle including the same.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

In order to solve the above technical problem, a method of providing acoustic tracking information according to an embodiment of the present invention includes the steps of generating acoustic tracking results based on acoustic data generated by detecting sounds occurring around the vehicle, A step of analyzing whether the detected sound is the own vehicle sound by extracting acoustic tracking pattern features based on the sound tracking results, generating a driver notification according to the analysis result, and analyzing whether the detected sound is the own vehicle sound. is a first condition that the maximum value for the probability exists within a specific angle range from the acoustic tracking results, and a second condition that the sound source location probability value is greater than a predetermined threshold at any first and second angles from the acoustic tracking results. If is satisfied, the detected sound can be analyzed as self-vehicle sound.

In addition, the acoustic tracking device for a vehicle according to an embodiment of the present invention includes an acoustic tracking unit that generates acoustic tracking results based on acoustic data generated by detecting sounds occurring around the vehicle, and an acoustic tracking pattern based on the acoustic tracking results. It includes an acoustic analysis unit that extracts features and analyzes whether the detected sound is the own vehicle sound, and a notification generation unit that generates a driver notification according to the analysis results. The acoustic analysis unit determines the probability of the sound source location within a certain angular range from the acoustic tracking results. If the first condition that there is a maximum value for and the second condition that the sound source location probability value is greater than a predetermined threshold value at any first and second angles from the sound tracking results are satisfied, the detected sound is analyzed as self-vehicle sound. can do.

In addition, the vehicle according to an embodiment of the present invention includes a plurality of microphones that generate sound data by detecting sounds occurring around the vehicle, and detection by extracting sound tracking pattern characteristics from the sound tracking results generated based on the sound data. It includes a sound tracking device that analyzes whether the generated sound is the sound of the vehicle and generates a driver notification based on the analysis result, and a notification output unit that visually or audibly notifies the driver of the generated driver notification. The sound tracking device includes a sound tracking device, A first condition that the maximum value for the sound source location probability exists within a specific angle range from the tracking results, and a second condition that the sound source location probability value is greater than a predetermined threshold at any first and second angles from the sound tracking results. If satisfied, the detected sound can be analyzed as own vehicle sound.

A method for providing acoustic tracking information, an acoustic tracking device for a vehicle, and a vehicle including the same according to at least one embodiment of the present invention configured as described above, extract acoustic tracking pattern characteristics based on the acoustic tracking results, and detect the detected sound by extracting acoustic tracking pattern characteristics from the vehicle. By analyzing whether it is sound or other sound, it provides the effect of providing accurate information about the detected sound.

In addition, the present invention provides the effect of improving the recognition performance of a rear side warning system (BSD) based on sound source tracking technology.

In addition, the present invention provides the effect of determining the type of failure of the vehicle engine by utilizing the sound tracking pattern of the idling sound, the size of the sound, and RPM information.

In addition, the present invention improves the accuracy of detecting surrounding vehicles by engine sound when the own vehicle accelerates rapidly, and when determining the own vehicle noise, self-diagnosis of vehicle failure is performed through analysis of vehicle noise, and the results are provided to the driver. It provides the effect of

In addition, the present invention provides the effect of distinguishing noise generated inside a vehicle from noise generated outside of another vehicle and visualizing it to the driver by analyzing the pattern of sound source tracking.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a diagram showing a vehicle according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the acoustic tracking device shown in FIG. 1 in more detail.
Figure 3 is a diagram showing an acoustic tracking pattern according to the driving of the own vehicle.
Figures 4 and 5 are diagrams comparing the sound tracking pattern according to the rapid acceleration of the own vehicle and the sound tracking pattern according to the approach of another vehicle.
Figures 6 and 7 are diagrams showing sounds generated from the engine of the vehicle.
Figure 8 is a graph showing an acoustic tracking pattern according to sudden acceleration of the own vehicle.
Figures 9A to 9F are graphs showing various acoustic tracking patterns according to sudden acceleration of the own vehicle.
Figures 10a and 10b are graphs for explaining the process of analyzing an acoustic tracking pattern according to acceleration of the own vehicle.
Figures 11A to 11C are graphs for explaining the process of analyzing sound tracking patterns according to the approach of another vehicle.
12 to 14 are flowcharts illustrating a method of providing acoustic tracking information according to the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to “include” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary. In addition, terms such as "... unit", "... unit", and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. there is.

Throughout the specification, when a part is said to “include” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary. Additionally, parts indicated with the same reference numbers throughout the specification refer to the same components.

Hereinafter, a method for providing acoustic tracking information that can be applied to embodiments of the present invention, an acoustic tracking device for a vehicle, and a vehicle including the same will be described in detail with reference to FIGS. 1 to 14.

1 is a diagram showing a vehicle according to an embodiment of the present invention.

Referring to FIG. 1, the vehicle 10 can generate information about a specific sound, such as what sound is generated around the vehicle and from what direction, during operation according to the driver's operation, and provide a notification to the driver. .

The vehicle 10 includes a multi-channel microphone 50 capable of collecting external sounds of the vehicle 10, and acoustic tracking capable of generating information about a specific sound based on the acoustic information collected by the microphone 50. It may include device 100.

Each microphone of the multi-channel microphone 50 can be understood as one channel.

The number of multi-channel microphones 50 can be implemented as three, and two microphones can be placed at regular intervals on the left and right based on the center of the vehicle 10, and among the two microphones, the left microphone is positioned at a certain distance above the microphone. Another microphone can be placed spaced apart from each other.

If a multi-channel microphone 50 is provided as shown in FIG. 1, the acoustic information collected by the multi-channel microphone 50 is forward (horizontal plane or vertical plane in the range of 0 to 180 degrees) rather than rear (horizontal plane or vertical plane in the range of 0 degrees to 180 degrees). It is assumed that the area (ranging from 180 degrees to 360 degrees in the vertical plane) is used to generate acoustic tracking results for the rear vehicle detection area of the acoustic tracking device 100, under the premise that detection of the other vehicle is unnecessary.

The number of multi-channel microphones 50 (three) and their installation positions on the vehicle 10 are not limited to those shown in FIG. 1 .

The specific operation of the acoustic tracking device 100 will be described later with reference to FIG. 2 .

FIG. 2 is a block diagram showing the acoustic tracking device shown in FIG. 1 in more detail.

Referring to FIG. 2, the sound tracking device 100 includes a signal processing unit 110, a data storage unit 120, a sound recognition unit 130, a sound tracking unit 140, a sound analysis unit 150, and a notification generation unit. It may include unit 160.

The acoustic tracking device 100 is designed for a vehicle and may be implemented as part of the head unit of the vehicle 10, but the scope of the present invention is not limited thereto.

The multi-channel microphone 50 can detect sounds generated around the vehicle 10, generate sound data through analog-to-digital conversion, and transmit it to the signal processor 110.

There are various sounds around the vehicle. For example, there are engine sounds or tire friction sounds of other vehicles located around the vehicle, sounds generated from traffic lights, electronic signs, etc., and general sounds of nature.

The driver wants to know the status (e.g., whether the vehicle is about to overtake) and the relative position of the vehicle running behind or to the rear, which cannot be observed from the front or side while the vehicle 10 is running.

Some of the external sounds do not penetrate the soundproofing system of the vehicle 10 and are not transmitted to the driver. However, when a horn sound is heard from outside the vehicle 10, the driver must determine which direction the horn sound came from and whether it is directed toward his or her vehicle. I want to know if it's not

Depending on the perception of the horn sound, the driver can take various actions such as reducing the vehicle's speed, changing lanes, or activating emergency lights.

Additionally, the driver may have set the volume of the vehicle's audio system so high that the surrounding horn sound cannot be heard by the driver, either visually or through the vehicle's audio system. It is necessary to notify people that honking has occurred.

Drivers may also be interested in other sounds.

For example, when a vehicle suddenly stops, a large friction sound is generated due to friction between the tires and the ground. This friction sound may be related to the occurrence of a traffic accident or the situation immediately before a traffic accident, and therefore requires the driver to be careful.

As another example, when an accident occurs where a vehicle collides with another vehicle, a collision sound is generated. By recognizing collision sounds from the front or side, and providing information to the driver about the direction in which the collision sound occurred, subsequent accidents can be prevented.

If police or ambulance sirens sound around the driver, the driver must take action, such as changing lanes, to allow the vehicle to pass.

In certain cases, there is a need to ensure that drivers are aware of the siren sounds of vehicles belonging to public institutions, as users may be subject to legal punishment for failing to take necessary measures.

The signal processor 110 may perform noise filtering on the acquired acoustic data.

Through this noise filtering, various noises that make it difficult to determine the characteristics or source of the sound can be removed.

In addition, most of the sounds of interest to users, such as horn sounds, sirens, tire rubbing sounds, and crash sounds, have a sufficient decibel level (for example, 70 dB or more).

Accordingly, the signal processing unit 110 may determine whether the decibel (i.e., size) of the noise-removed sound data is greater than or equal to the reference value.

That is, sound data whose size is less than the standard value may be removed by the signal processor 110.

The data storage unit 120 may store sound data from which noise has been removed.

The data storage unit 120 can store sound data in frames and provide it to the sound recognition unit 130 in units of frames.

A frame may refer to sound data collected at the same time, and the interval between frames may have a specific period (eg, 100 ms), but the scope of the present invention is not limited thereto.

The sound recognition unit 130 determines the characteristics of the sound data.

Here, even if the sound data has a decibel level above a predetermined standard, it may not be important to the driver. For example, the sound made when a train passes by or the noise of an airplane near an airport has a fairly high decibel level, but it may not be important to the driver. It may not have a significant impact.

The same applies to noise generated during road restoration or redevelopment work. Rather, continuously informing the driver of such acoustic data actually slows the driver's reaction speed to situations that the driver needs to be aware of. Or it may prevent the driver from reacting.

The sound recognition unit 130 extracts feature values from the sound data received from the data storage unit 120 in the time domain and the frequency domain.

The sound recognition unit 130 can build a database of the average and variance values of feature values.

Here, the feature values may be Mel-Frequency Cepstral Coefficients (MFCC), Total Spectrum Power, Sub-band Spectrum Power, and/or Peach Frequency.

The sound recognition unit 130 may store the average and variance values of frames for sound data over a predetermined time period, for example, 100 ms, in a database.

In the field of voice signal processing, MFC (Mel-Frequency Cepstrum) is one of the methods of expressing the power spectrum of a short-term signal.

This can be obtained by taking a cosine transform on the log power spectrum in the frequency domain of the non-linear Mel scale.

MFCC refers to the coefficient of several MFCs.

MFCC generally applies a pre-emphasis filter to short-term sound data (signal), applies DFT (Discrete Fourier Transform) to this value, and then uses Melscale's Filter Bank (Mel Filter Banks). Obtain the power spectrum using , take the logarithm of the power of each Mel-scale, and then perform DCT (Discrete Cosine Transform) on the obtained value to obtain the MFCC value.

The overall power spectrum refers to the energy distribution of the entire spectrum within a certain frame section, and the subband power is usually divided into four groups such as [0, ⅛f0], [⅛f0, ¼f0], [¼f0, ½f0], and [½f0, f0]. It refers to the energy distribution value of the spectrum in the subband section. The pitch frequency can be obtained by detecting the peak of the normalized autocorrelation function.

The sound recognition unit 130 can determine whether the acquired sound data is a sound of interest to the user by classifying the characteristic values of the sound data obtained in this way through a classifier.

Here, the classifier may be one of a Neural Network (NN) classifier, a Support Vector Machine (SVM) classifier, or a Bayesian classifier.

In this specification, the classifier will be explained using an NN classifier as an example.

The classifier of the sound recognition unit 130 classifies each type of sound into a plurality of classes and uses feature values for the acquired sound data to determine a confidence level (Confidence) based on the similarity of the sound data to the plurality of classes. Level) can be calculated.

That is, the confidence level may mean the probability that sound data corresponds to a specific class of sound, and the total of the trust levels may be 1.

The sound classification result generated by the classifier of the sound recognition unit 130 may include information about each class, the type of sound corresponding to each class, and the confidence level corresponding to each class.

The sound recognition unit 130 may generate a determination result depending on whether the trust level is higher than a reference value (eg, 0.7) and include it in the sound classification result.

That is, when the trust level is higher than the reference value, the sound recognition unit 130 may determine the type of sound of the class corresponding to the trust level as the type of current sound data.

Accordingly, the sound recognition unit 130 may analyze the characteristics of the sound data and generate a sound classification result, which is information about what type of sound the sound data is.

The sound tracking unit 140 can track the direction in which the sound was generated based on the sound data for the sound type (or target sound source) of the class whose confidence level is higher than the standard value. The sound type is determined from the sound recognition unit 130. can be provided.

The sound tracking unit 140 accumulates sound data corresponding to consecutive frames, identifies the identity of the sound input to each microphone through the temporal characteristics (waveform) of the sound, compares the size of the same sound, and The difference in arrival time of the arriving sound may be calculated, and temporal characteristics may be provided by the sound recognition unit 130.

The loudness of the sound is inversely proportional to the square of the distance, so when the distance from the location of the sound increases by two, the loudness of the sound decreases by 1/4 (about 6 dB reduction).

Assuming that the width of a typical vehicle is about 2m and the length is about 3m, the difference in the size of the detected sound can have a sufficiently significant value depending on the location of the point where the sound is generated.

For example, when the multi-channel microphone 50 is arranged as shown in Figure 1, when a sound is generated from the upper right side of the vehicle, the volume of the sound detected by the microphone located at the top is detected by the microphone located on the left and right sides of the bottom. It becomes louder than the average loudness of a sound.

In addition, the volume of sound detected by the microphone located on the bottom right side becomes louder than the volume detected by the microphone located on the left side of the bottom.

Using these characteristics, the approximate direction based on the center of the vehicle 10 can be tracked using the volume of sound collected from each microphone.

Additionally, the angle of the sound generation location can be calculated using the difference in arrival time (signal delay) of the sound reaching each microphone.

At this time, the sound tracking unit 140 stores in advance a table in which the angle of the sound generation location and the signal delay corresponding to each microphone are mapped.

For example, in a table, an angle of 1 degree is mapped to t1 (signal delay to the first microphone), t2 (signal delay to the second microphone), and t3 (signal delay to the third microphone); The probability that there is a tracking sound source can be calculated by applying the signal delays of t1 to t3 to the sound data for each microphone stored in the data storage unit 120 and then summing them.

Of course, when the multi-channel microphone 50 is arranged in pairs up and down and left and right as shown in Figure 1, the table has each microphone at an angle of 0 to 180 degrees on the horizontal or vertical plane to the sound generation position. The signal delay corresponding to can be mapped, and the probability that there is a tracking sound source at an angle of 1 degree can be calculated by applying the signal delay of t1 to t2 to the acoustic data for each microphone stored in the data storage unit 120 and then adding them up. You can.

In other words, the probability that there is a tracking sound source at each angle can be obtained by applying the delay value for all angles to the current signal. Through this, the location of the sound can be estimated.

This is because the combination of the angle of the sound generation location and the signal delay corresponding to each microphone has a one-to-one correspondence.

The sound tracking unit 140 can use this information to generate sound tracking results for each angle of successive frames over time. The sound tracking results show that the sound source corresponding to the target sound source is angular in each successive frame over time. It may be information about the probability of a star existing.

As such, the vehicle 10 of the present invention may include a multi-channel microphone 50 including a plurality of microphones, an acoustic tracking device 100, and a notification output unit 200, as shown in FIGS. 1 and 2. there is.

Here, the multi-channel microphone 50 can generate sound data by detecting sounds occurring around the vehicle.

In addition, the acoustic tracking device 100 may extract acoustic tracking pattern characteristics from the acoustic tracking results generated based on acoustic data, analyze whether the detected sound is the own vehicle sound, and generate a driver notification based on the analysis result.

Here, the acoustic tracking device 100 includes a signal processing unit 110, a data storage unit 120, an acoustic recognition unit 130, an acoustic tracking unit 140, an acoustic analysis unit 150, and a notification generation unit 160. ) may include.

The signal processing unit 110 may signal-process sound data received from the multi-channel microphone 50, and the data storage unit 120 may store the signal-processed sound data.

Then, the sound recognition unit 130 classifies the stored sound data according to characteristics, and the sound tracking unit 140 can generate sound tracking results for each angle of successive frames over time from the classified sound data. .

Next, the acoustic analysis unit 150 determines a first condition in which the maximum value for the sound source location probability exists within a specific angle range from the generated acoustic tracking result, and a sound source at an arbitrary first angle and a second angle from the acoustic tracking result. If the second condition that the location probability value is greater than a predetermined threshold is satisfied, the detected sound can be analyzed as own vehicle sound.

As an example, in the first condition, the specific angle range may be, but is not limited to, about 60 degrees to about 100 degrees.

And, in the second condition, any first angle is 0 degrees, any second angle is 180 degrees, and the predetermined threshold may be, but is not limited to, about 0.5.

Here, if the first condition is not satisfied or the second condition is not satisfied, the sound analysis unit 150 may analyze the detected sound as an external sound.

As an example, when the sound analysis unit 150 analyzes the detected sound as the sound of the own vehicle, it may request the diagnostic system to perform self-diagnosis of a fault in the own vehicle.

In some cases, the acoustic analysis unit 150 may determine a first condition in which the maximum value for the sound source location probability exists within a specific angle range from the acoustic tracking results, and the sound source at any first and second angles from the acoustic tracking results. A second condition that the position probability value is greater than the first threshold, and a third condition that the difference between the maximum value existing within a specific angle range and the minimum value existing at an angle outside the specific angle range from the acoustic tracking result is greater than the second threshold. If is satisfied, the detected sound can be analyzed as self-vehicle sound.

Here, as an example, in the first condition, the specific angle range is about 60 degrees to about 100 degrees, and in the second condition, any first angle is 0 degrees, any second angle is 180 degrees, and the first threshold is is about 0.5, and in the third condition, the second threshold may be, but is not limited to, about 0.4.

At this time, if at least one of the first condition, second condition, and third condition is not satisfied, the sound analysis unit 150 may analyze the detected sound as an external sound from the vehicle.

As an example, when the sound analysis unit 150 analyzes the detected sound as the sound of the own vehicle, it may request the diagnostic system to perform self-diagnosis of a fault in the own vehicle.

Meanwhile, the notification generator 160 may generate a driver notification according to the sound analysis results.

As an example, when the detected sound is analyzed as the sound of the own vehicle, the notification generator 160 may generate a notification to the driver about the result of the self-diagnosis of a fault performed according to the self-diagnosis of the fault of the own vehicle.

In some cases, if the detected sound is not analyzed as the sound of the own vehicle, the notification generator 160 may analyze it as the sound of an external vehicle and generate a driver notification for Blind Spot Detection (BSD).

Additionally, the notification output unit 200 may visually or audibly inform the driver of the generated driver notification.

In this way, the present invention can increase reliability by providing accurate information about the detected sound by extracting sound tracking pattern characteristics based on the sound tracking results and analyzing whether the detected sound is the own vehicle sound or another vehicle sound.

In addition, the present invention provides convenience so that the driver can easily recognize whether there is a problem with the vehicle by notifying the driver of the diagnosis result of the vehicle through self-diagnosis if the analyzed sound is the sound of the own vehicle, and the analyzed sound is If it is the sound of another vehicle, it recognizes the approach and overtaking of another vehicle and provides driver notification of BSD (Blind Spot Detection), allowing the driver to prevent danger.

Figure 3 is a diagram showing an acoustic tracking pattern according to the driving of the own vehicle, and Figures 4 and 5 are diagrams comparing the acoustic tracking pattern according to the rapid acceleration of the own vehicle and the acoustic tracking pattern according to the approach of another vehicle.

As shown in FIG. 3, when the noise generated from the own vehicle is measured when the vehicle is driving, it can be seen that various frequency characteristics appear depending on the driving state of the own vehicle.

For example, when a vehicle accelerates rapidly while driving, it exhibits noise frequency characteristics that are different from the noise frequency characteristics that appear during normal driving.

In other words, the noise frequency characteristics that appear when the own vehicle rapidly accelerates are similar to the noise frequency characteristics that occur when another vehicle from behind approaches the own vehicle.

Therefore, it is difficult for the sound source tracking device to classify whether the noise is due to rapid acceleration of the own vehicle or noise due to the approach of another vehicle based on noise frequency characteristics alone, which may cause errors in driver notification information.

Figure 4 is an acoustic tracking pattern according to the rapid acceleration of the own vehicle, and Figure 5 is an acoustic tracking pattern according to the approach of another vehicle. As shown in Figures 4 and 5, the acoustic tracking pattern according to the rapid acceleration of the own vehicle and the approach of the other vehicle is very different. You can see the similarity.

Therefore, since the present invention cannot accurately classify sounds simply by comparing sound tracking patterns, similar sounds can be classified more precisely by analyzing the sound tracking patterns in more detail.

Figures 6 to 11 are diagrams for explaining a method of classifying sounds between one's own vehicle and another vehicle.

FIGS. 6 and 7 are diagrams showing sounds generated from the engine of the own vehicle, FIG. 8 is a graph showing sound tracking patterns according to rapid acceleration of the own vehicle, and FIGS. 9A to 9F are diagrams showing various sound tracking according to rapid acceleration of the own vehicle. It is a graph showing patterns, and Figures 10a and 10b are graphs to explain the process of analyzing the sound tracking pattern according to the acceleration of the own vehicle, and Figures 11a to 11c are graphs showing the process of analyzing the sound tracking pattern according to the approach of another vehicle. This is a graph for explanation.

As shown in Figures 6 and 7, various noises are generated from the engine room of the own vehicle while driving.

Here, noise may be emitted at various angles and directions.

As an example, noise may be generated in the engine room, mainly in the engine area (A) and exhaust area (B), and may include sounds directed to both sides of the engine, sounds directed to the front of the engine, and exhaust sounds. there is.

Next, as shown in Figure 8, when the noise generated when the vehicle is rapidly accelerating is measured to generate an acoustic tracking pattern, it can be seen that a pattern in the shape of the letter W appears.

Here, between about 60 degrees and about 100 degrees of the acoustic tracking pattern, the sound source location probability appears to be the maximum value, and between 0 degrees and 180 degrees of the acoustic tracking pattern, the sound source location probability appears as a value above a certain threshold. there is.

Next, as shown in FIGS. 9A to 9F, noise generated when the own vehicle is rapidly accelerating may appear in various sound tracking patterns.

In the present invention, common conditions can be found by analyzing various acoustic tracking patterns that appear when a vehicle is rapidly accelerating.

That is, the present invention provides a first condition in which the maximum value for the sound source location probability exists within a specific angle range in the acoustic tracking pattern according to the sudden acceleration of the own vehicle, and an arbitrary condition in the acoustic tracking pattern according to the sudden acceleration of the own vehicle. When the second condition that the sound source location probability value at angle 1 and angle 2 is greater than a predetermined threshold is satisfied, the detected sound can be analyzed as self-vehicle sound.

Here, in the first condition, the specific angle range may be from about 60 degrees to about 100 degrees, but is not limited thereto.

And, in the second condition, any first angle is 0 degrees, any second angle is 180 degrees, and the predetermined threshold may be, but is not limited to, 0.5.

In some cases, the present invention provides a first condition in which the maximum value for the sound source location probability exists within a specific angle range in an acoustic tracking pattern according to rapid acceleration of the own vehicle, and the sound source location at any first and second angles. Detection occurs when the second condition that the probability value is greater than the first threshold and the third condition that the difference between the maximum value within a specific angle range and the minimum value existing at an angle outside the specific angle range are greater than the second threshold are satisfied. The sound can also be analyzed as a vehicle sound.

Here, in the first condition, the specific angle range may be, but is not limited to, about 60 degrees to about 100 degrees.

And, in the second condition, any first angle is 0 degrees and any second angle is 180 degrees and the first threshold may be, but is not limited to, about 0.5.

Then, in the third condition, the second threshold may be, but is not limited to, about 0.4.

FIGS. 10A and 10B are graphs for explaining the process of analyzing sound tracking patterns according to acceleration of the own vehicle. As shown in FIGS. 10A and 10B, the own vehicle tracking device of the present invention detects sounds occurring around the own vehicle. Based on the sound data generated by detecting, sound tracking results are generated, and based on the generated sound tracking results, sound tracking pattern characteristics can be extracted to analyze whether the detected sound is the own sound or the other sound.

As an example, the own vehicle tracking device of the present invention can analyze whether the characteristics of the sound tracking pattern as shown in FIGS. 10A and 10B are the own vehicle sound or the other vehicle sound depending on whether two or three conditions are satisfied.

First, the present invention checks whether the first condition that the maximum value for the sound source location probability exists within a specific angle range of about 60 degrees to about 100 degrees is satisfied, and as a result of the confirmation, the angle range between about 60 degrees and about 100 degrees is satisfied. Since the maximum value a for the sound source location probability exists within the sound source location probability, it can be seen that the acoustic tracking pattern characteristics of FIGS. 10A and 10B satisfy the first condition.

Next, the present invention checks whether the sound source location probability value at the first angle of 0 degrees and the second angle of 180 degrees is greater than the first threshold value of about 0.5, and as a result of the confirmation, the sound source location probability value b at 0 degrees is the first threshold value. Since it is greater than 0.5, and the sound source location probability value c at 180 degrees is greater than the first threshold value of 0.5, it can be seen that the acoustic tracking pattern characteristics of FIGS. 10A and 10B satisfy the second condition.

In this case, if the first and second conditions are satisfied, the present invention can analyze the detected sound as a sound caused by rapid acceleration of the own vehicle.

Additionally, the present invention checks whether the difference between the maximum value a that exists within a specific angle range of about 60 degrees to about 100 degrees and the minimum value d that exists in an angle outside the specific angle range is greater than a second threshold value of about 0.4, As a result of confirmation, the difference between the maximum value a within the angle range between about 60 degrees and about 100 degrees and the minimum value d within the angle range outside of the angle range between about 60 degrees and about 100 degrees is greater than the second threshold value of 0.4. Since it is large, it can be seen that the acoustic tracking pattern characteristics of FIGS. 10A and 10B satisfy the second condition.

In this case, the present invention can analyze the detected sound as a sound caused by rapid acceleration of the own vehicle if the first, second, and third conditions are satisfied.

However, in the present invention, if the first condition is not satisfied or the second condition is not satisfied, the detected sound can be analyzed as the sound of the other car.

In some cases, the present invention may analyze the detected sound as the sound of another vehicle if at least one of the first, second, and third conditions is not satisfied.

FIGS. 11A to 11C are graphs for explaining the process of analyzing an acoustic tracking pattern according to the approach of another vehicle. As shown in FIGS. 11A to 11C, the own vehicle tracking device of the present invention includes the first, second, and If at least one of the third conditions is not satisfied, the detected sound may be analyzed as the sound of the other car.

First, the present invention checks whether the first condition that the maximum value for the sound source location probability exists within a specific angle range of about 60 degrees to about 100 degrees is satisfied, and as a result of the confirmation, the angle range between about 60 degrees and about 100 degrees is satisfied. Since there is no maximum value a for the sound source location probability, it can be seen that the acoustic tracking pattern characteristics of FIGS. 11A to 11C do not satisfy the first condition.

Next, the present invention checks whether the sound source location probability value at the first angle of 0 degrees and the second angle of 180 degrees is greater than the first threshold of about 0.5, and as a result of the confirmation, the sound tracking pattern of FIGS. 11A and 11B is 0 degrees. The sound source location probability value b is greater than the first threshold 0.5, but the sound source location probability value c at 180 degrees is smaller than the first threshold 0.5, and the acoustic tracking pattern in FIG. 11C has the sound source location probability value b at 0 degrees. Although it is smaller than the threshold value of 0.5, the sound source location probability value c at 180 degrees is larger than the first threshold value of 0.5, so it can be seen that the acoustic tracking pattern characteristics of FIGS. 11A to 11C do not satisfy the second condition.

In this case, since the first or second conditions are not satisfied, the present invention can analyze the detected sound as a sound caused by the approach of another vehicle.

Additionally, the present invention checks whether the difference between the maximum value a that exists within a specific angle range of about 60 degrees to about 100 degrees and the minimum value d that exists in an angle outside the specific angle range is greater than a second threshold value of about 0.4, As a result of confirmation, the acoustic tracking pattern characteristics of FIGS. 11A to 11C have a maximum value a that exists within the angle range between about 60 degrees and about 100 degrees, and an angle other than the angle range between about 60 degrees and about 100 degrees. Since the difference value from the minimum value d is smaller than the second threshold value 0.4, it can be seen that the acoustic tracking pattern characteristics of FIGS. 11A to 11C do not satisfy the third condition.

Accordingly, in the present invention, since the sound tracking pattern of FIGS. 11A to 11C does not satisfy at least one of the first, second, and third conditions, the detected sound can be analyzed as the sound of the other vehicle.

Here, the sound tracking pattern in FIGS. 11A and 11B is a pattern based on the sound of another vehicle passing to the left of the own vehicle, and the sound tracking pattern in FIG. 11C is a pattern based on the sound of another vehicle passing to the right of the own vehicle.

Next, the present invention can request the diagnostic system to perform a self-diagnosis of a fault in the vehicle by analyzing the detected sound as the vehicle's sound.

The reason is that the idling sound caused by unnecessary rapid acceleration is generated as a loud noise, which can cause engine failure.

Therefore, the present invention can determine the type of failure of the vehicle's engine by utilizing the sound tracking pattern of the idling sound, the size of the sound, and RPM information.

In addition, the present invention improves the accuracy of detecting surrounding vehicles by engine sound when the own vehicle accelerates rapidly, and when determining the own vehicle noise, self-diagnosis of vehicle failure is performed through analysis of vehicle noise, and the results are provided to the driver. can do.

Additionally, by analyzing sound source tracking patterns, the present invention can distinguish noise generated inside the vehicle from noise generated outside of other vehicles and visualize it to the driver.

12 to 14 are flowcharts illustrating a method of providing acoustic tracking information according to the present invention.

As shown in FIG. 12, the multi-channel microphone of the present invention can detect sound occurring around the vehicle and receive sound data generated through analog-to-digital conversion (S10).

Additionally, the signal processing unit of the present invention may perform noise filtering on the acquired sound data, and the data storage unit of the present invention may store the noise-removed sound data (S20).

Next, the sound recognition unit of the present invention extracts feature values in the time domain and frequency domain for the audio data received from the data storage unit, classifies the feature values through a classifier, and classifies the sound. Results can be generated (S30).

Next, the sound tracking unit of the present invention provides, based on the sound data, the probability that an object corresponding to the sound type exists in each consecutive frame over time for each sound type in the class whose confidence level is higher than the reference value in the sound classification result. Acoustic tracking results, which are information, can be generated (S40).

And, the acoustic analysis unit of the present invention can extract acoustic tracking pattern characteristics based on the acoustic tracking results and analyze whether the detected sound is the own vehicle sound (S50).

Here, as shown in FIG. 13, the acoustic analysis unit determines a first condition in which the maximum value for the sound source location probability exists within a specific angle range from the acoustic tracking results, and the sound source at any first and second angles from the acoustic tracking results. If the second condition that the location probability value is greater than a predetermined threshold is satisfied, the detected sound can be analyzed as own vehicle sound.

In some cases, as shown in FIG. 14, the acoustic analysis unit may provide a first condition in which the maximum value for the sound source location probability exists within a specific angle range from the acoustic tracking results, and an arbitrary first and second angle from the acoustic tracking results. A second condition that the sound source location probability value is greater than the first threshold, and that the difference between the maximum value existing within the specific angle range and the minimum value existing at an angle outside the specific angle range from the sound tracking result is greater than the second threshold. If the third condition is satisfied, the detected sound can be analyzed as self-vehicle sound.

Next, the notification output unit of the present invention generates a driver notification according to the analysis result and provides the generated notification information to the driver (S60).

Here, the notification output unit can provide a screen that the driver can intuitively understand by outputting an image.

Meanwhile, in the present invention, step S50 of analyzing the sound may go through the analysis process as shown in FIG. 13 or the analysis process as shown in FIG. 14.

First, the analysis process of FIG. 13 will be described in detail as follows.

The acoustic analysis unit of the present invention extracts acoustic tracking pattern features based on the acoustic tracking results (S51).

Next, the sound analysis unit checks whether the sound tracking pattern characteristics satisfy the first condition that the maximum value for the sound source location probability exists within a specific angle range of about 60 degrees to about 100 degrees (S52).

Next, the acoustic analysis unit determines that if the maximum value for the sound source location probability exists within the angle range between about 60 degrees and about 100 degrees, the sound source location probability value is about 0.5 at the first angle of 0 degrees and the second angle of 180 degrees. It is checked whether the second condition of being greater than the critical value is satisfied (S53).

And, if the sound source location probability value at the first angle of 0 degrees and the second angle of 180 degrees is greater than the threshold value of about 0.5, the sound analysis unit may analyze the detected sound as the sound of the own vehicle (S54).

Next, when the sound analysis unit analyzes the detected sound as the sound of the own vehicle, it can request the diagnostic system to perform a self-diagnosis of the fault of the own vehicle (S55).

Next, the notification generating unit may generate a driver notification regarding the result of the failure self-diagnosis performed according to the failure self-diagnosis of the own vehicle (S56).

However, when the acoustic analysis unit checks whether the first condition that the maximum value for the sound source location probability exists within a specific angle range of about 60 degrees to about 100 degrees is satisfied, the result of the confirmation is that the angle range is between about 60 degrees and about 100 degrees. If there is no maximum value for the sound source location probability within the sound source location probability, the detected sound can be analyzed as the sound of the other car (S57).

In addition, when checking whether the sound analysis unit satisfies the second condition that the sound source location probability value is greater than the threshold value of about 0.5 at the first angle of 0 degrees and the second angle of 180 degrees, the confirmation result shows that the first angle of 0 degrees and the second angle of 180 degrees are satisfied. If the sound source location probability value at the second angle is not greater than the threshold value of about 0.5, the detected sound can be analyzed as the sound of the other vehicle (S57).

Additionally, the notification generator may generate a driver notification for BSD (Blind Spot Detection) when the detected sound is analyzed as an off-vehicle sound (S58).

Next, the analysis process of FIG. 14 is described in detail as follows.

The acoustic analysis unit of the present invention extracts acoustic tracking pattern features based on the acoustic tracking results (S51).

Next, the sound analysis unit checks whether the sound tracking pattern characteristics satisfy the first condition that the maximum value for the sound source location probability exists within a specific angle range of about 60 degrees to about 100 degrees (S52).

Next, the acoustic analysis unit determines that if the maximum value for the sound source location probability exists within the angle range between about 60 degrees and about 100 degrees, the sound source location probability value is about 0.5 at the first angle of 0 degrees and the second angle of 180 degrees. It is checked whether the second condition of being greater than the first threshold is satisfied (S53).

And, the acoustic analysis unit, if the sound source location probability value at the first angle of 0 degrees and the second angle of 180 degrees is greater than the first threshold of about 0.5, the maximum value existing within a specific angle range of about 60 degrees to about 100 degrees and a specific It is checked whether the third condition that the difference value from the minimum value existing at an angle outside the angle range is greater than the second threshold value of about 0.4 is satisfied (S59).

Next, the acoustic analysis unit determines that, as a result of confirmation, the difference between the maximum value existing within the angle range between about 60 degrees and about 100 degrees and the minimum value existing at an angle outside the angle range between about 60 degrees and about 100 degrees is a second value. If the threshold is greater than 0.4, the detected sound can be analyzed as the sound of the own vehicle (S54).

Next, when the sound analysis unit analyzes the detected sound as the sound of the own vehicle, it can request the diagnostic system to perform a self-diagnosis of the fault of the own vehicle (S55).

Next, the notification generating unit may generate a driver notification regarding the result of the failure self-diagnosis performed according to the failure self-diagnosis of the own vehicle (S56).

However, when the acoustic analysis unit checks whether the first condition that the maximum value for the sound source location probability exists within a specific angle range of about 60 degrees to about 100 degrees is satisfied, the result of the confirmation is that the angle range is between about 60 degrees and about 100 degrees. If there is no maximum value for the sound source location probability within the sound source location probability, the detected sound can be analyzed as the sound of the other car (S57).

In addition, when checking whether the sound analysis unit satisfies the second condition that the sound source location probability value is greater than the first threshold of about 0.5 at the first angle of 0 degrees and the second angle of 180 degrees, as a result of the confirmation, the first angle of 0 degrees and If the sound source location probability value at the second angle of 180 degrees is not greater than the first threshold of about 0.5, the detected sound can be analyzed as the sound of the other car (S57).

In addition, the acoustic analysis unit detects the detected value if the difference between the maximum value existing within a specific angle range of about 60 degrees to about 100 degrees and the minimum value existing at an angle outside the specific angle range is not greater than a second threshold value of about 0.4. The sound can be analyzed as the sound of the vehicle (S57).

Additionally, the notification generator may generate a driver notification for BSD (Blind Spot Detection) when the detected sound is analyzed as an off-vehicle sound (S58).

In this way, the present invention can provide accurate information about the detected sound by extracting sound tracking pattern characteristics based on the sound tracking result and analyzing whether the detected sound is the own vehicle sound or another vehicle sound.

Additionally, the present invention can improve the recognition performance of a rear side warning system (BSD) based on sound source tracking technology.

In addition, the present invention can determine the type of failure of the vehicle's engine by utilizing the sound tracking pattern of the idling sound, the size of the sound, and RPM information.

In addition, the present invention improves the accuracy of detecting surrounding vehicles by engine sound when the own vehicle accelerates rapidly, and when determining the own vehicle noise, self-diagnosis of vehicle failure is performed through analysis of vehicle noise, and the results are provided to the driver. can do.

Additionally, by analyzing the pattern of sound source tracking, the present invention can distinguish noise generated inside the vehicle from noise generated outside of other vehicles and visualize it to the driver.

The above-described present invention can be implemented as computer-readable code on a program-recorded medium. Computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. It also includes those implemented in the form of carrier waves (e.g., transmission via the Internet).

Accordingly, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: acoustic tracking device
110: signal processing unit
120: data storage unit
130: Sound recognition unit
140: Acoustic tracking unit
150: Acoustic analysis unit
160: Notification generation unit

Claims (20)

Generating acoustic tracking results based on acoustic data generated by detecting sounds occurring around the vehicle;
Based on the sound tracking results, extracting sound tracking pattern features and analyzing whether the detected sound is a vehicle sound; and,
According to the analysis results, generating a driver notification,
The step of analyzing whether the detected sound is the vehicle sound is,
A first condition that the maximum value for the sound source location probability exists within a specific angle range from the acoustic tracking result, and that the sound source location probability value is greater than a predetermined threshold at any first angle and second angle from the acoustic tracking result. If the second condition is satisfied, a method of providing acoustic tracking information, characterized in that the detected sound is analyzed as own vehicle sound. According to claim 1,
In the first condition, the specific angle range is 60 to 100 degrees. According to claim 1,
In the second condition, the arbitrary first angle is 0 degrees, the arbitrary second angle is 180 degrees, and the predetermined threshold is 0.5. According to claim 1,
The step of analyzing whether the detected sound is the own vehicle sound is,
A first condition that a maximum value for the sound source location probability exists within a specific angle range from the acoustic tracking result, and that the sound source location probability value is greater than a first threshold at any first angle and a second angle from the acoustic tracking result. If the second condition and the third condition that the difference between the maximum value existing within the specific angle range and the minimum value existing at an angle outside the specific angle range from the sound tracking result are greater than the second threshold are satisfied, the detection A method of providing acoustic tracking information, characterized in that the sound is analyzed as a vehicle sound. According to clause 4,
In the first condition, the specific angle range is 60 degrees to 100 degrees,
In the second condition, the arbitrary first angle is 0 degrees, the arbitrary second angle is 180 degrees and the first threshold is 0.5,
In the third condition, the second threshold is 0.4. According to clause 4,
The step of analyzing whether the detected sound is the vehicle sound is,
A method of providing acoustic tracking information, characterized in that, if at least one of the first condition, second condition, and third condition is not satisfied, the detected sound is analyzed as an external sound. According to claim 1,
The step of analyzing whether the detected sound is the vehicle sound is,
A method of providing acoustic tracking information, characterized in that when the detected sound is analyzed as the sound of the own vehicle, a request is made to perform a self-diagnosis of a malfunction of the own vehicle. According to claim 1,
The step of analyzing whether the detected sound is the vehicle sound is,
A method of providing sound tracking information, characterized in that if the first condition is not satisfied or the second condition is not satisfied, the detected sound is analyzed as an external sound from the vehicle. According to claim 1,
According to the above analysis results, the step of generating a driver notification is:
A method of providing sound tracking information, characterized in that when the detected sound is analyzed as the own vehicle's sound, a driver notification is generated regarding the result of a self-diagnosis of a fault performed according to the fault self-diagnosis of the own vehicle. According to claim 1,
According to the above analysis results, the step of generating a driver notification is:
A method of providing sound tracking information, characterized in that, if the detected sound is not analyzed as the sound of the own vehicle, it is analyzed as the sound of an external vehicle and generates a driver notification for BSD (Blind Spot Detection). An acoustic tracking unit that generates acoustic tracking results based on acoustic data generated by detecting sounds occurring around the vehicle;
An acoustic analysis unit that extracts acoustic tracking pattern characteristics based on the acoustic tracking results and analyzes whether the detected sound is the own vehicle sound; and,
According to the analysis results, it includes a notification generator that generates a driver notification,
The acoustic analysis unit,
A first condition that the maximum value for the sound source location probability exists within a specific angle range from the acoustic tracking result, and that the sound source location probability value is greater than a predetermined threshold at any first angle and second angle from the acoustic tracking result. If the second condition is satisfied, an acoustic tracking device for a vehicle, characterized in that the detected sound is analyzed as own vehicle sound. According to claim 11,
In the first condition, the specific angle range is 60 degrees to 100 degrees,
In the second condition, the arbitrary first angle is 0 degrees, the arbitrary second angle is 180 degrees, and the predetermined threshold is 0.5. According to claim 11,
The acoustic analysis unit,
A first condition that a maximum value for the sound source location probability exists within a specific angle range from the acoustic tracking result, and that the sound source location probability value is greater than a first threshold at any first angle and a second angle from the acoustic tracking result. If the second condition and the third condition that the difference between the maximum value existing within the specific angle range and the minimum value existing at an angle outside the specific angle range from the sound tracking result are greater than the second threshold are satisfied, the detection An acoustic tracking device for vehicles, characterized in that the sound is analyzed as the own vehicle sound. According to claim 13,
In the first condition, the specific angle range is 60 degrees to 100 degrees,
In the second condition, the arbitrary first angle is 0 degrees, the arbitrary second angle is 180 degrees and the first threshold is 0.5,
In the third condition, the second threshold is 0.4. According to claim 13,
The acoustic tracking device for a vehicle, wherein the acoustic analysis unit analyzes the detected sound as an external sound from other vehicles when at least one of the first condition, second condition, and third condition is not satisfied. According to claim 11,
An acoustic tracking device for a vehicle, wherein the sound analysis unit analyzes the detected sound as the sound of the vehicle and requests the vehicle to perform a self-diagnosis of a fault. According to claim 11,
The acoustic tracking device for a vehicle, wherein the acoustic analysis unit analyzes the detected sound as an external sound from other vehicles when the first condition is not satisfied or the second condition is not satisfied. According to claim 11,
When the detected sound is analyzed as the sound of the own vehicle, the notification generator generates a notification to the driver about the result of the self-diagnosis of the fault performed according to the self-diagnosis of the self-vehicle,
If the detected sound is not analyzed as the own vehicle sound, it is analyzed as an external sound from another vehicle and generates a driver notification for BSD (Blind Spot Detection). Multiple microphones that detect sounds occurring around the vehicle and generate acoustic data;
An acoustic tracking device that extracts acoustic tracking pattern characteristics from acoustic tracking results generated based on the acoustic data, analyzes whether the detected sound is the own vehicle sound, and generates a driver notification based on the analysis results; and,
It includes a notification output unit that visually or audibly notifies the driver of the generated driver notification,
The acoustic tracking device,
A first condition that the maximum value for the sound source location probability exists within a specific angle range from the acoustic tracking result, and that the sound source location probability value is greater than a predetermined threshold at any first angle and second angle from the acoustic tracking result. If the second condition is satisfied, the vehicle is characterized in that it analyzes the detected sound as its own vehicle sound. According to clause 19,
In the first condition, the specific angle range is 60 degrees to 100 degrees,
In the second condition, the arbitrary first angle is 0 degrees, the arbitrary second angle is 180 degrees, and the predetermined threshold is 0.5.

Images