KR20220069700A - Apparatus and method for diagnosing vehicle condition - Google Patents

Abstract

According to the present invention, a method for diagnosing a vehicle state comprises: a step of enabling a preprocessor preprocessing noise and vibration data inputted from a noise and vibration sensor and at least one data among driving data inputted from an electronic control unit, object data inputted from an image recognizer, and temperature and humidity data inputted from a temperature and humidity sensor to be in the same data dimension; and a step of enabling an abnormal pattern prediction model to fuse at least one data and the noise and vibration data to predict an abnormal pattern indicating abnormality of a component inside a vehicle from a noise and vibration pattern based on the noise and vibration data.

Description

Vehicle condition diagnosis apparatus and method {Apparatus and method for diagnosing vehicle condition}

The present invention relates to a technology for diagnosing a vehicle condition.

Conventionally, in order to determine the presence or absence of a failure/abnormality of a major component in a vehicle, a sensor mounted on the corresponding component is used, and the result of the determination is alerted through a display device such as a cluster in the driver's seat, so that the driver can determine the failure/abnormality of the main component presence can be checked.

However, the conventional vehicle abnormality determination method cannot detect and warn an abnormality/failure other than a predefined type. Therefore, even when a specific warning occurs, there is a problem in that diagnosis and maintenance must be performed for various causes.

An object of the present invention for solving the above problems is to prevent potential accidents and maximize the lifespan of the vehicle by monitoring noise/vibration inside the vehicle to detect and early warning of vehicle abnormalities regardless of whether or not the vehicle is predefined. An object of the present invention is to provide an apparatus for diagnosing a vehicle condition and a method therefor.

The above and other objects, advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings.

In the vehicle state diagnosis method of the present invention for achieving the above object, the preprocessor includes at least one of driving data inputted from an electronic control unit, object data inputted from an image recognizer, and temperature/humidity data inputted from a temperature/humidity sensor. preprocessing the data of the noise/vibration sensor and the noise/vibration data input from the noise/vibration sensor into the same data dimension; And an abnormal pattern prediction model, fusing the at least one data and the noise/vibration data, predicting an abnormal pattern representing an abnormality in a part in the vehicle from the noise/vibration pattern based on the noise/vibration data; includes; .

A vehicle condition diagnosis apparatus of the present invention includes: a noise/vibration sensor for outputting noise/vibration data obtained by measuring noise/vibration generated inside and outside the vehicle; an electronic control unit outputting driving data related to vehicle driving; a temperature/humidity sensor for outputting temperature/humidity data obtained by measuring the temperature/humidity outside the vehicle; an image recognizer for recognizing an object included in an image around the vehicle captured by the camera and outputting the obtained object data; a preprocessor preprocessing at least one of the driving data, the temperature/humidity data, and the object data and the noise/vibration data in the same data dimension; and an abnormality pattern prediction model that fuses the at least one data and the noise/vibration data, and predicts an abnormality pattern representing an abnormality in a component in a vehicle from the noise/vibration pattern based on the noise/vibration data.

According to the present invention, it is possible to detect and warn of abnormalities/failures not defined in advance, thereby improving the reliability of the safety performance of the vehicle and preventing accidents due to defects not confirmed during the design/mass production process.

In addition, according to the present invention, by inducing preventive maintenance by warning at the point in time when an actual abnormal phenomenon occurs before reaching the warning threshold, it is possible to maximize vehicle lifespan and increase sales in the after-sales service market.

In addition, according to the present invention, it is possible to lead to cost reduction through reduction of diagnostic sensors when diagnostic accuracy is matured.

In addition, according to the present invention, it is possible to minimize the maintenance time and reduce excessive maintenance by estimating and providing the cause point of the vehicle abnormality.

In addition, according to the present invention, it is possible to maximize the life of assets, provide the best vehicle service to customers, furthermore, secure B2B sales competitiveness, and increase the adoption rate of telematics options can be expected.

In addition, according to the present invention, it can be applied to most mobility such as UAM in the future by monitoring generated noise/vibration.

1 is a block diagram illustrating an internal configuration of an apparatus for diagnosing a vehicle state according to an embodiment of the present invention.
2 is a flowchart illustrating a vehicle state diagnosis method according to an embodiment of the present invention.

Examples of the present invention are provided to more completely explain the present invention to those of ordinary skill in the art, and the following examples may be modified in various other forms, and the scope of the present invention is as follows It is not limited to an Example. Rather, these examples are provided so that this disclosure will be more thorough and complete, and will fully convey the spirit of the invention to those skilled in the art. In addition, in the following drawings, each configuration is exaggerated for convenience and clarity of description, and the same reference numerals refer to the same elements in the drawings. As used herein, the term “and/or” includes any one and all combinations of one or more of those listed items.

1 is a block diagram illustrating an internal configuration of an apparatus for diagnosing a vehicle state according to an embodiment of the present invention.

Referring to FIG. 1 , the vehicle condition diagnosis apparatus 100 monitors noise/vibration in a vehicle to detect and early warning of vehicle failure/abnormality. In particular, the vehicle condition diagnosis apparatus 100 detects a failure/abnormality of a vehicle that is not defined in advance from noise/vibration in the vehicle monitored using a pre-trained artificial neural network or a deep learning neural network (hereinafter, an abnormal pattern prediction model). and early warning.

To this end, the vehicle condition diagnosis apparatus 100 includes, for example, a vibration sensor 101 , a noise sensor 103 , an engine control unit 105 , a camera 107 , a temperature/humidity sensor 109 , and a filter ( 111 ), an image recognizer 115 , a preprocessor 117 , an abnormal pattern prediction model 119 , and a display device 121 .

The vibration sensor 101 may be, for example, an acceleration sensor, and generates vibration data representing a vibration pattern that changes with time as a result of measuring vibration inside and outside the vehicle. In this case, the vibration data includes vibration data related to internal vibration of the vehicle (hereinafter referred to as 'internal vibration data of the vehicle') and vibration data related to external vibration of the vehicle (hereinafter, referred to as 'external vibration data of the vehicle').

The vibration sensor 101 may be configured in plurality, and each sensor may be installed at a position where it is possible to easily measure the vibration inside the vehicle and the vibration outside the vehicle.

The vibration inside the vehicle measured by the vibration sensor 101 may be, for example, vibration that is directly related to determining whether there is an abnormality in the vehicle, such as vibration generated in an engine room, vehicle parts, and/or a vehicle body.

Vibration outside the vehicle measured by the vibration sensor 101 is, for example, vibration generated by friction between the tire and the road surface, vibration generated by friction between the vehicle in motion and air, and vibration generated by movement of a passenger. Vibration may be a kind of noise that is not directly related to determining whether there is an abnormality in the vehicle, such as vibration, vibration caused by movement of an object in the vehicle, or vibration caused by the sound of a speaker in the vehicle.

The noise sensor 103 may be, for example, a microphone, and generates noise data representing a noise pattern that changes over time as a result of measuring the noise inside the vehicle and the noise outside the vehicle. In this case, the noise data includes noise data related to internal noise within the vehicle (hereinafter referred to as 'internal noise data of the vehicle') and noise data related to external noise of the vehicle (hereinafter referred to as 'external noise data of the vehicle').

The noise sensor 103 may be configured in plurality, and each noise sensor may be installed at a position where it is possible to easily measure the noise inside the vehicle and the noise outside the vehicle.

The noise inside the vehicle may be, for example, noise that is directly related to determining whether there is an abnormality in the vehicle, such as noise generated by engine room vibration, noise generated from vibration of vehicle parts, or noise generated from vehicle body vibration.

The noise outside the vehicle may include, for example, noise generated by friction between tires and the road surface, noise generated by friction between the vehicle in motion and air (wind noise), voice generated from passengers and/or from multimedia devices in the vehicle. It may be noise corresponding to a kind of noise having low direct relevance to determining whether there is an abnormality in the vehicle, such as generated music sound.

The electronic control unit 105 is, for example, an engine control unit that controls engine functions of the vehicle or a control unit connected to the engine control unit via an internal vehicle communication bus (CAN bus or LIN bus). , to generate driving data related to the driving state of the vehicle.

The driving state may be, for example, a starting on/off (ON/OFF) state, an acceleration state, a deceleration state (non-acceleration or braking), a stop state, or an IDLE STOP state.

The camera 107 outputs image data obtained by photographing the front, rear and/or sides of the vehicle. In order to photograph the front, rear and/or side of the vehicle, the camera 107 may be configured in plurality.

The temperature/humidity sensor 109 measures the external temperature/humidity of the vehicle capable of grasping the driving environment (climate, weather, weather, season, etc.) of the vehicle, and outputs the measurement result as temperature/humidity data.

The filter 111 receives in-vehicle vibration data from the vibration sensor 101 that measures the vibration inside the vehicle, receives the vibration data outside the vehicle from the other vibration sensor 101 that measures the vibration outside the vehicle, and the vibration data inside the vehicle It masks and removes (filters) the vibration data outside the vehicle from

Since the internal vehicle vibration data obtained by the vibration sensor 101 that measures the internal vibration of the vehicle may include an external vehicle vibration component, it is necessary to remove the external vehicle vibration component corresponding to the noise component in order to accurately determine whether there is an abnormality. there is

As a method of removing vibration related to noise by the filter 111, for example, the frequency components of the vehicle internal vibration data and the vehicle external vibration data are analyzed respectively, and the frequency of the vehicle external vibration data is obtained from the frequency component of the internal vehicle vibration data. There may be a way to remove the component.

In addition, the filter 111 receives the vehicle interior noise data from the noise sensor 103 that measures the vehicle interior noise, receives the vehicle exterior noise data from the other noise sensor 103 that measures the vehicle exterior noise, and the vehicle interior noise It masks and removes (filters) the vehicle external noise data from the data.

Similarly, since the vehicle internal noise data measured by the noise sensor 103 for measuring internal vehicle noise may include a vehicle external noise component, the external vehicle noise component corresponding to the noise component is removed to accurately determine whether there is an abnormality. Needs to be.

As a method of removing noise related to noise by the filter 111, for example, frequency components of vehicle internal noise data and external vehicle noise data are analyzed respectively, and the frequency of vehicle external noise data is obtained from the frequency components of internal vehicle noise data. There may be a way to remove the component.

The image recognizer 115 receives image data of a vehicle surrounding situation from the camera 107, analyzes the image data, recognizes an object included in the image data, and generates the recognition result as object data . Here, the object may be a road surface material (asphalt, etc.), a road surface condition (rainy weather, icy), and/or an obstacle (eg, a pothole, a speed bump), and the like.

The preprocessor 117 includes the filtered noise data D1 and vibration data D2 from the filter 111 , driving data D3 from the electronic control unit 105 , and object data from the image recognizer 115 ( D4) and temperature/humidity data D5 from the temperature/humidity sensor 109 are received, and a preprocessing process is performed on these data.

Since the data D1, D2, D3, D4, and D5 are data obtained from different types of measuring devices, their correlation is low. In order to determine whether a vehicle is abnormal in consideration of the relationship between them, it is necessary to transform different dimensions (attributes) of the data D1, D2, D3, D4, and D5 into the same dimension.

The preprocessing process performed by the preprocessor 117 may be to process the data D1 , D2 , D3 , D4 , and D5 having different dimensions as data of the same dimension.

The abnormal pattern prediction model 119 analyzes the abnormal pattern related to the noise / vibration pattern by using the noise data D1 and the vibration data D2 converted to the same data dimension by the preprocessor 117 as inputs, Based on the analysis result, it predicts (estimates or infers) the presence or absence of abnormalities in major parts of the vehicle.

In particular, the abnormal pattern prediction model 119 includes driving data (D3) and object data converted into the same data dimension in addition to noise/vibration data (D1, D2) for accurate analysis and prediction (estimation or inference) of the abnormal pattern. At least one of (D4) and temperature/humidity data (D5) is further used as an input to analyze and predict an abnormal pattern related to a noise/vibration pattern.

That is, the abnormal pattern prediction model 119 further fuses at least one data among the driving data D3, the object data D4, and the temperature/humidity data D5 with the noise/vibration data D1 and D2. / From the vibration pattern, the abnormal pattern can be analyzed more precisely, thereby accurately predicting (estimating or inferring) the presence or absence of abnormality in the main parts of the vehicle.

Here, the abnormal pattern is all noise/vibration patterns except for the noise/vibration pattern that is determined to be normal in the driving state (driving, stopping, acceleration, deceleration, IDLE STOP, ignition on/off, braking, etc.) based on the current driving data (D3). It means a vibration pattern, all noise/vibration patterns except for the noise/vibration pattern that is judged to be normal in the state of the road surface and obstacles based on the object data (D4), or the weather condition based on the temperature/humidity data (D5) It means all noise/vibration patterns except for the noise/vibration patterns judged to be normal.

When the abnormal pattern prediction model 119 continuously determines the above-defined abnormal pattern from the noise/vibration pattern for a predetermined time and a predetermined number of times, the prediction result (estimation) that an abnormality has occurred in major parts of the vehicle or as an inference result). This means that it is not judged as a failure when the abnormal pattern is detected once, but is determined as a failure when the corresponding abnormal pattern is detected more than a certain number of times during a certain period of time.

In addition, the abnormal pattern prediction model 119 may be implemented to predict a failure cause for the abnormal pattern by inquiring the noise, vibration, and harshness (NVH) knowledge base 113 . Here, the NVH knowledge base 113 may be a database in which data indicating characteristic values (eg, peaks, bands, etc.) of abnormal patterns with respect to noise/vibration patterns for each cause of failure are accumulated.

The NVH knowledge base 113 may be built in the vehicle, but is not limited thereto, and may exist on a network connected to the vehicle through wireless communication.

The abnormal pattern prediction model 119 may be embodied as a software module programmed with an artificial neural network or a deep neural network learned in advance through machine learning or the like, or a hardware module having such a software module built-in.

The abnormal pattern prediction model 119 does not stop at predicting the abnormal pattern, but continuously learns and updates itself by using the data D1 to D5 used to generate the prediction result as training data.

The display device 121 displays the prediction results predicted by the abnormality pattern prediction model 119 to warn the driver of vehicle abnormalities (parts in the vehicle) and/or the cause of vehicle abnormalities (causes of abnormalities in parts in the vehicle). , a cluster, an AVN device, or a center display.

Also, when the display device 121 is equipped with an audio function, for a warning effect, the display device 121 may output a prediction result predicted by the abnormal pattern prediction model 119 as an alarm sound.

In addition, when the display device 121 is equipped with a navigation function and a telematics function, the display device 121 recommends a route to the nearest repair shop to the current location of the vehicle, / Alternatively, a reservation function for transmitting data related to a cause of an abnormality in the vehicle (a cause of an abnormality in a part in the vehicle) may be provided.

In FIG. 1 , the NVH knowledge base 113 , the preprocessor 117 , and the abnormal pattern prediction model 119 are separated, but may be integrated into one configuration. For example, the anomaly pattern prediction model 119 may be configured to include the NVH knowledge base 113 and the preprocessor 117 .

2 is a flowchart illustrating a vehicle state diagnosis method according to an embodiment of the present invention.

Referring to FIG. 2 , first, in step 210 , the noise/vibration sensor 103/101 measures noise/vibration generated in the vehicle to generate noise/vibration data.

Next, in step 220 , the image recognizer 115 generates object data by recognizing an object from the image around the vehicle acquired by the camera 107 .

Next, in step 230 , the temperature/humidity sensor 109 measures the temperature/humidity outside the vehicle to generate temperature/humidity data.

Next, in step 240 , the preprocessor 117 receives at least one of the driving data related to vehicle driving input from the electronic control unit 105 , the object data, and the temperature/humidity data and the noise/vibration data. The process of preprocessing to the same data dimension is performed.

Next, in step 250, the abnormal pattern prediction model 119 fuses the at least one data and the noise/vibration data, and an abnormal pattern indicating an abnormality in a component in the vehicle from the noise/vibration pattern based on the noise/vibration data The process of predicting is carried out.

In an embodiment of the present invention, before the pre-processing step, the filter 111 may further perform a step of removing noise/vibration data having low correlation with abnormalities in parts in the vehicle from the noise/vibration data. . In this case, the noise/vibration data having a low correlation with abnormalities in parts in the vehicle includes, for example, noise/vibration generated by friction between the tire and the road surface, noise/vibration generated by friction between the driving vehicle and air, It may include data related to noise/vibration caused by the movement of a passenger, noise/vibration caused by the movement of an object in the vehicle, and noise/vibration generated by the sound of a speaker in the vehicle.

In another embodiment of the present invention, the predicting of the abnormal pattern ( 250 ) includes predicting all noise/vibration patterns except for the noise/vibration pattern determined to be normal in the driving state based on the driving data as the abnormal pattern. may be a step.

In another embodiment of the present invention, in the step of predicting the abnormal pattern (250), all noise/vibration patterns except for the noise/vibration pattern determined to be normal in the road surface condition and obstacles based on the object data as the abnormal pattern may be a predictive step.

In another embodiment of the present invention, in the step of predicting the abnormal pattern (250), all noise/vibration patterns except for the noise/vibration pattern determined to be normal in a weather condition based on the temperature/humidity data as the abnormal pattern This may be a predictive step.

In another embodiment of the present invention, in the step of predicting the abnormal pattern ( 250 ), the knowledge base in which data representing the characteristic value of the abnormal pattern with respect to the noise/vibration pattern for each cause of failure is accumulated, and information about the abnormal pattern is obtained. It may be a step to further predict the cause of the failure.

In another embodiment of the present invention, the method may further include displaying, by the display device 121 equipped with a navigation function and a telematics function, a prediction result generated by predicting the abnormal pattern by the abnormal pattern prediction model.

In another embodiment of the present invention, the display device 121 transmits, by using the telematics function, the abnormal pattern and data indicating the cause of the failure of the abnormal pattern to a repair shop closest to the current location of the vehicle. , it may further include the step of performing a maintenance reservation.

In another embodiment of the present disclosure, the method may further include, by the display device 121, guiding a route to the repair shop using the navigation function.

The embodiments disclosed in this specification should be considered in an illustrative view rather than a limiting point of view. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (16)

The preprocessor performs at least one data and noise from among the driving data related to driving of the vehicle input from the electronic control unit, object data obtained by recognizing the image around the vehicle input from the image recognizer, and temperature/humidity data input from the temperature/humidity sensor. / Pre-processing the noise/vibration data input from the vibration sensor into the same data dimension; and
predicting, by an abnormal pattern prediction model, an abnormal pattern representing an abnormality in a component in a vehicle from the noise/vibration pattern based on the noise/vibration data by fusing the at least one data and the noise/vibration data;
A vehicle condition diagnosis method comprising a. In claim 1,
Before the pre-processing step,
The method further comprising: removing, by a filter, noise/vibration data having a low correlation with abnormalities in parts in the vehicle from the noise/vibration data;
Noise/vibration data with low correlation with abnormalities in parts in the vehicle,
A vehicle condition diagnosis method comprising data related to noise/vibration generated by friction between a tire and a road surface and noise/vibration generated by friction between a vehicle and air in motion. In claim 1,
Before the pre-processing step,
The method further comprising: removing, by a filter, noise/vibration data having a low correlation with abnormalities in parts in the vehicle from the noise/vibration data;
Noise/vibration data with low correlation with parts abnormalities in the vehicle,
A vehicle condition diagnosis method comprising data related to noise/vibration generated by the movement of a passenger and noise/vibration generated by the movement of an object in the vehicle. In claim 1,
Before the pre-processing step,
The method further comprising: removing, by a filter, noise/vibration data having a low correlation with abnormalities in parts in the vehicle from the noise/vibration data;
Noise/vibration data with low correlation with abnormalities in parts in the vehicle,
A vehicle condition diagnosis method comprising data related to noise/vibration generated by the sound of a speaker in the vehicle. In claim 1,
Predicting the abnormal pattern comprises:
Predicting all noise/vibration patterns except for the noise/vibration pattern determined to be normal in the driving state based on the driving data as the abnormal pattern. In claim 1,
Predicting the abnormal pattern comprises:
Predicting all noise/vibration patterns except for noise/vibration patterns determined to be normal in road surface conditions and obstacles based on the object data as the abnormal patterns. In claim 1,
Predicting the abnormal pattern comprises:
Predicting all noise/vibration patterns except for the noise/vibration pattern determined to be normal in a weather condition based on the temperature/humidity data as the abnormal pattern. In claim 1,
Predicting the abnormal pattern comprises:
The method of diagnosing a vehicle condition is the step of further predicting the cause of the failure for the abnormal pattern by inquiring the knowledge base in which data indicating the characteristic value of the abnormal pattern for the noise/vibration pattern for each cause of the failure is accumulated. In claim 8,
displaying, by a display device equipped with a navigation function and a telematics function, a prediction result generated by predicting an abnormal pattern by the abnormal pattern prediction model;
performing, by the display device, a maintenance reservation by using the telematics function to transmit the abnormal pattern and data indicating a failure cause for the abnormal pattern to a repair shop closest to the current location of the vehicle; and
guiding, by the display device, a route to the repair shop using the navigation function
The vehicle condition diagnosis method further comprising a. a noise/vibration sensor that outputs noise/vibration data obtained by measuring noise/vibration generated inside and outside the vehicle;
an electronic control unit outputting driving data related to vehicle driving;
a temperature/humidity sensor for outputting temperature/humidity data obtained by measuring the temperature/humidity outside the vehicle;
an image recognizer for recognizing an object included in an image around the vehicle captured by the camera and outputting the obtained object data;
a preprocessor preprocessing at least one of the driving data, the temperature/humidity data, and the object data and the noise/vibration data in the same data dimension; and
An abnormality pattern prediction model that fuses the at least one data and the noise/vibration data, and predicts abnormal patterns indicating abnormalities in parts in the vehicle from the noise/vibration patterns based on the noise/vibration data
A vehicle diagnostic device comprising a. In claim 10,
Further comprising a display device equipped with a navigation function and a telematics function,
The display device is
The abnormal pattern prediction model predicts the abnormal pattern and displays the generated prediction result, and uses the telematics function to transfer the abnormal pattern and the data indicating the cause of the failure for the abnormal pattern to the nearest repair shop to the current location of the vehicle. A vehicle diagnosis device that transmits, performs a maintenance reservation, and guides a route to the repair shop using the navigation function. In claim 10,
A filter for removing noise/vibration data having low correlation with abnormalities in parts in the vehicle from the noise/vibration data,
Noise/vibration data with low correlation with abnormalities in parts in the vehicle,
Noise/vibration caused by friction between tires and the road surface, Noise/vibration caused by friction between the vehicle and air in motion, Noise/vibration caused by the movement of passengers, Noise caused by the movement of objects in the vehicle /Vibration, a vehicle diagnostic device that includes data related to noise/vibration generated by the sound of a speaker in the vehicle. In claim 10,
The abnormal pattern prediction model is,
A vehicle diagnosis apparatus for further predicting a failure cause for the abnormal pattern by inquiring the knowledge base in which data indicating the characteristic value of the abnormal pattern with respect to the noise/vibration pattern for each cause of the failure is accumulated. In claim 10,
The abnormal pattern prediction model is,
Predicting all noise/vibration patterns except for the noise/vibration pattern determined to be normal in the driving state based on the driving data as the abnormal pattern. In claim 10,
The abnormal pattern prediction model is,
Predicting all noise/vibration patterns except for noise/vibration patterns determined to be normal in road surface conditions and obstacles based on the object data as the abnormal patterns. In claim 10,
The abnormal pattern prediction model is,
Predicting all noise/vibration patterns except for the noise/vibration pattern determined to be normal in a weather condition based on the temperature/humidity data as the abnormal pattern.

Images