KR102656603B1 - Apparatus for estimating of road surface type using velocity estimation of moving object and method thereof - Google Patents

Abstract

The present invention relates to a road surface type estimation device and method using speed estimation of a moving object, and more specifically, to transmit a sound wave signal and receive a reflected signal, and to estimate the speed of a moving object considering the Doppler effect from the received signal. , Provides an apparatus and method for estimating the type of road surface using the speed estimation of a moving object, which learns the estimated speed of the moving object and the received signal using an artificial neural network to estimate the type of the road surface.

Description

Road surface type estimation device and method using speed estimation of moving object {APPARATUS FOR ESTIMATING OF ROAD SURFACE TYPE USING VELOCITY ESTIMATION OF MOVING OBJECT AND METHOD THEREOF}

The present invention relates to a road surface type estimation device and method using speed estimation of a moving object, and more specifically, to transmit a sound wave signal and receive a reflected signal, and to estimate the speed of a moving object considering the Doppler effect from the received signal. , relates to an apparatus and method for estimating the type of road surface using the speed estimation of a moving object, which learns the estimated speed of the moving object and the received signal using an artificial neural network to estimate the type of the road surface.

In the United States, approximately 6 million traffic accidents occur every year. In particular, more than 1.2 million accidents occur on average each year in the United States alone due to slippery road surfaces, and more than 5,000 people die every year from these accidents. Accordingly, the slipperiness of road surfaces is emerging as a social problem that places a great burden on the government and insurance companies.

Korea is no exception. Black ice accidents occur every winter, with 6,500 cases occurring over the past five years and the number of deaths reaching approximately 200.

In Korea, the snow melting process is carried out in four stages, and all of these processes are carried out manually by humans, and it takes several hours to reach the final stage. In the case of the last black ice accident on the Sangju-Yeongcheon Expressway, it was a man-made disaster that occurred because the official in charge checked the forecast but did not request snow melting in advance.

These accidents are due to the absence of sensors that can detect the condition of the road surface in real time and a notification system using them.

Meanwhile, there are two major methods proposed so far to estimate the type of road surface or the friction coefficient of the road surface.

One is a direct method of finding out by causing physical friction, and the other is an indirect method of using radio waves or sound waves.

First, direct methods include wheel-based methods and vehicle dynamics-based methods.

What these two methods have in common is to estimate the friction coefficient of the road surface by applying the brakes regardless of the driver's intention and decelerating the vehicle.

In this method, you can only find out the friction coefficient by stepping on the road surface you want to know, and a vehicle dynamics model is essential. Additionally, since the brakes must be operated, there are limitations that affect ride comfort and fuel efficiency.

Meanwhile, indirect methods include methods using electromagnetic waves and sound waves. Both of these methods do not require physical friction, so they can be used independently of the vehicle dynamics model and have the advantage of knowing the condition of the road surface in advance without contacting the road surface. There is.

The electromagnetic wave-based method uses cameras, lidar, radar, etc., and although it can see quite far away, its accuracy is relatively low, and the system is very expensive, ranging from hundreds to thousands of dollars. Additionally, because the amount of data is large, it is difficult to create a data set.

Additionally, there is a method using an image sensor, which is one of the electromagnetic wave methods, and it uses a camera, an image sensor, to classify the type of road surface. However, image sensors are greatly affected by illumination, and black ice is difficult to see to the human eye, so vision sensors cannot detect it. Due to these limitations, the average performance of methods using image sensors is less than 70%. Additionally, not only an image sensor but also a high-performance image processing device is required, so the cost is relatively high.

In the case of black ice, just as it is invisible to the human eye, it cannot be detected even by an image sensor. In other words, there is a limitation in being unable to distinguish various types of road surfaces using the image sensor method.

Meanwhile, methods using the sound waves include a passive sensing method and an active sensing method. The passive sensing method records noise from tires for a certain period of time with a microphone and then classifies it using an artificial neural network. There is. In this method, the type of road surface can be known only for the surface over which the tire has passed, and there is a limitation in that classification takes a long time. Meanwhile, among active sensing methods, there is research to classify reflected road surfaces using only distance and reflection intensity information, but this method has the disadvantage of being weak to surrounding disturbances.

Meanwhile, many electric and hydrogen cars with light blue license plates are appearing on the roads these days. According to Bloomberg, more than half of vehicles sold will be electric vehicles by 2040, and one-third of vehicles on the road will be electric vehicles.

To increase the mileage, which is the driving range of electric vehicles, vehicle manufacturers are actively using regenerative braking. However, it is not difficult to hear news about accidents caused by a car skidding due to regenerative braking. Symptoms such as loss of traction on the rear wheels or wheels locking occur due to regenerative braking.

Looking at Tesla's current regenerative braking settings, you can change it to standard and low, but there is no option to turn it off. The reason these accidents occur is because the control cycle of the regenerative braking module is 100ms, which is longer than other systems, and the reason why manufacturers do not allow the regenerative braking function to be turned off is because the vehicle's travelable distance drastically decreases.

It may be best to use regenerative braking depending on the condition of the road surface you are driving on, but this problem arises because the type of road surface cannot be known in advance before the vehicle passes.

Meanwhile, when a road classification device is installed on a vehicle (mobile object) to perform road surface classification while driving, as the speed of the vehicle (mobile object) increases, the difference between the original transmission frequency and the frequency of the received signal increases due to the Doppler effect. . As a result, the distortion of the received signal becomes more severe, so it is necessary to consider the speed of the moving object when classifying the road surface.

Korean patent [10-1716270] discloses a front ground state estimation device and method.

Korean registered patent [10-2136576] discloses a road surface condition measuring device and a road surface condition diagnosis system.

Korean registered patent [10-2156295] discloses a method and device for estimating the type of road surface using ultrasonic signals.

Korean registered patent [10-2207816] discloses a method and device for measuring road surface conditions using sensor data.

Korean registered patent [10-1716270] (registration date: 2017. 03. 08) Korean registered patent [10-2136576] (registration date: 2020. 07. 16) Korean registered patent [10-2156295] (registration date: 2020. 09. 09) Korean registered patent [10-2207816] (registration date: 2021. 01. 20)

Therefore, the present invention was created to solve the problems described above. The purpose of the present invention is to transmit a sound wave signal, receive a reflected signal, estimate the speed of a moving object considering the Doppler effect from the received signal, To provide an apparatus and method for estimating the type of road surface using the speed estimation of a moving object, which learns the estimated speed of the moving object and the received signal using an artificial neural network to estimate the type of the road surface.

The purposes of the embodiments of the present invention are not limited to the purposes mentioned above, and other purposes not mentioned will be clearly understood by those skilled in the art from the description below. .

In order to achieve the above-described object, a road surface type estimation device using speed estimation of a moving object according to an embodiment of the present invention transmits a sound wave signal to the road surface of which the type is to be known, and then receives the reflected signal. A sound wave transceiver (sensor) (100) for; An analog-to-digital converter (200) for converting the analog signal of the received signal into a digital signal; a speed estimation unit 300 for estimating the speed of the moving object by processing the converted digital signal; an artificial neural network 400 that uses the converted digital signal and the estimated speed of the moving object as input signals and classifies the input signals based on a learned road surface classification model to estimate the type of the road surface; And a control unit (MCU) 500 that controls the operation of the sound wave transceiver (sensor), the analog-to-digital converter, the speed estimation unit, and the artificial neural network.

In addition, in order to achieve the above-described object, the road surface type estimation method using speed estimation of a moving object according to an embodiment of the present invention transmits a sound wave signal to the road surface whose type is to be known under the control of the control unit. Afterwards, a reflected signal reception step (S20) of receiving the reflected signal; A signal conversion step (S30) of converting an analog signal into a digital signal for a preset area of the received signal under the control of the control unit; A moving object speed estimation step (S40) of estimating the speed of the moving object by processing the digital signal under the control of the control unit; A frequency domain signal acquisition step (S50) of obtaining a frequency domain signal by performing frequency conversion on a preset region of the digital signal under the control of the control unit; And under the control of the control unit, the frequency domain signal and the estimated speed of the moving object are used as input signals of the neural network, and based on the learned road surface classification model, the characteristics of the input signal are extracted and classified to determine the road surface. It is characterized by including a first road surface type determination step (S60) for determining the type.

In addition, in order to achieve the above-described object, the road surface type estimation method using speed estimation of a moving object according to another embodiment of the present invention transmits a sound wave signal to the road surface whose type is to be known under the control of the control unit. Afterwards, a reflected signal reception step (S20) of receiving the reflected signal; A signal conversion step (S30) of converting an analog signal into a digital signal for a preset area of the received signal under the control of the control unit; A moving object speed estimation step (S40) of estimating the speed of the moving object by processing the digital signal under the control of the control unit; A convolution operation step (S70) of receiving the digital signal from an artificial neural network and performing multiple convolution operations under the control of the control unit; and a second unit for extracting, classifying, and determining the type of the road surface based on the road surface classification model learned from the artificial neural network under the control of the controller. It is characterized by including a road surface type determination step (S80).

In addition, according to one embodiment of the present invention, a computer-readable recording medium storing a program for implementing a method for estimating a road surface type using speed estimation of the moving object is provided.

In addition, according to one embodiment of the present invention, a program stored in a computer-readable recording medium is provided to implement a method of estimating the type of road surface using the estimation of the speed of the moving object.

According to an apparatus and method for estimating a road surface type using speed estimation of a moving object according to an embodiment of the present invention, a sound wave signal is transmitted and a reflected signal is received, the speed of the moving object is estimated from the received signal considering the Doppler effect, and Since it is possible to estimate the type of road surface by learning the estimated speed of the moving object and the received signal using an artificial neural network, the type of road surface can be determined with high accuracy through a simple configuration. there is.

In addition, according to the road surface type estimation device and method using the speed estimation of a moving object according to an embodiment of the present invention, the road surface type or road surface condition can be determined with high accuracy, so it is possible to determine whether black ice, in which thin ice is formed, is formed. can be checked quickly, which has the effect of providing advance forecasts to drivers driving in that section.

In addition, according to the road surface type estimation device and method using the speed estimation of a moving object according to an embodiment of the present invention, the road surface type estimation device can be implemented using an inexpensive 40 kHz ultrasonic sensor and a lightweight algorithm, thereby increasing the processing speed. It is fast and economically effective.

In addition, according to the road surface type estimation device and method using the speed estimation of a moving object according to an embodiment of the present invention, since a road surface classification model learned according to the speed of the vehicle is used, the type of road surface can be accurately determined even in a running vehicle. There is an effect that can be estimated quickly.

1 is a configuration diagram of a road surface type estimation device using speed estimation of a moving object according to an embodiment of the present invention.
Figure 2 is a diagram illustrating the Doppler effect in a road surface type estimation device using speed estimation of a moving object according to an embodiment of the present invention.
Figure 3 is a graph for explaining the change in frequency of sound wave transmission and reception signals according to the speed of a moving object.
Figures 4A and 4B are diagrams for explaining learning data about vehicle speed.
Figure 5 is a configuration diagram of an embodiment of an artificial neural network in a road surface type estimation device using speed estimation of a moving object according to the present invention.
FIG. 6 is a diagram illustrating a signal converter in a road surface type estimation device using speed estimation of a moving object according to an embodiment of the present invention.
Figure 7 is a configuration diagram of another embodiment of an artificial neural network in a road surface type estimation device using speed estimation of a moving object according to the present invention.
FIG. 8 is a diagram for explaining the operation of the convolution performance unit of FIG. 7.
Figure 9 is a flowchart of an embodiment of a road surface type estimation method using speed estimation of a moving object according to the present invention.
Figure 10 is a flowchart of another embodiment of a road surface type estimation method using speed estimation of a moving object according to the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be.

On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used herein are merely used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, processes, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, processes, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as having an ideal or excessively formal meaning unless explicitly defined in the present application. No.

Hereinafter, the present invention will be described in more detail with reference to the attached drawings. Prior to this, the terms or words used in this specification and claims should not be construed as limited to their usual or dictionary meanings, and the inventor should appropriately define the concept of terms in order to explain his or her invention in the best way. Based on the principle of definability, it must be interpreted with meaning and concept consistent with the technical idea of the present invention. In addition, if there is no other definition in the technical and scientific terms used, they have meanings commonly understood by those skilled in the art to which this invention pertains, and the gist of the present invention is summarized in the following description and accompanying drawings. Descriptions of known functions and configurations that may be unnecessarily obscure are omitted. The drawings introduced below are provided as examples so that the idea of the present invention can be sufficiently conveyed to those skilled in the art. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other forms. Additionally, like reference numerals refer to like elements throughout the specification. It should be noted that like elements in the drawings are represented by like symbols wherever possible.

1 is a configuration diagram of a road surface type estimation device using speed estimation of a moving object according to an embodiment of the present invention.

As shown in FIG. 1, the road surface type estimation device using speed estimation of a moving object according to an embodiment of the present invention includes a sound wave transceiver (sensor) 100, an analog-digital converter (ADC), It includes a speed estimation unit 300, an artificial neural network 400, and a control unit (MCU) 500.

The sound wave transmitting and receiving unit (sensor) 100 transmits a sound wave signal to the road surface of which the type is to be determined and then receives a reflected signal.

The sound wave transmitting and receiving unit (sensor) 100 includes a sound wave transmitter 101 that outputs a transmission signal under the control of the control unit 500, and a sound wave receiving a reflected signal returned after the transmission signal is reflected on an arbitrary surface. Includes a receiver 102.

The analog-to-digital converter 200 converts the analog signal of the received signal into a digital signal.

The speed estimation unit 300 processes the converted digital signal to estimate the speed of the moving object.

The artificial neural network 400 uses the corrected digital signal and the estimated speed of the moving object as input signals, classifies the input signal based on a learned road surface classification model, and estimates the type of the road surface.

Meanwhile, the artificial neural network 400 uses decision trees, linear discriminant analysis, logistic regression classifiers, Naive Bayes classifier, and support vector machines. machine), nearest neighbor classifiers, and ensemble classifiers are used to classify and learn.

Decision trees include Fine tree, Medium tree, Coarse tree, All tree, and Optimizable tree, and linear discriminant analysis includes Linear discriminant, Quadratic discriminant, All discriminants, Optimizable discriminant, and Naive Bayes classifiers include Gaussian Naive Bayes, Kernel Naive Bayes, All Naive Bayes, and Optimizable Naive Bayes, and support vector machines (SVMs) include Linear SVM, Quadratic SVM, Cubic SVM, and Fine Gaussian. It includes SVM, Medium Gaussian SVM, Coarse Gaussian SVM, All SVM, and Optimizable SVM, and nearest neighbor classifiers include Fine KNN, Medium KNN, Coarse KNN, Cosine KNN, Cubic KNN, Weighted KNN, All KNN, Includes Optimizable KNN, and ensemble classifiers include Boosted trees, Bagged trees, Subspace Discriminant, Subspace KNN, RUSBoosted trees, All Ensembles, and Optimizable Ensemble.

The control unit (MCU) 500 controls the operations of the sound wave transceiver 100, the analog-to-digital converter 200, the speed estimation unit 300, and the artificial neural network 400.

The analog-to-digital converter 200, the speed estimation unit 300, and the artificial neural network 400 are expressed as components of software implemented as a program.

Meanwhile, although not shown in the drawing, the road surface type estimation device using speed estimation of a moving object according to the present invention includes a storage (memory) in which the learned road surface classification model and software implemented by the program are stored. The storage (memory) may be included in the control unit (MCU) 500.

Meanwhile, the road surface type estimation device using speed estimation of a moving object according to the present invention further includes an atmospheric information measuring unit (not shown) capable of measuring temperature, humidity, and atmospheric pressure in the atmosphere.

Under the control of the control unit 500, atmospheric information including the temperature, humidity and atmospheric pressure measured by the atmospheric information measuring unit (not shown) can be used in the speed estimating unit 300, and the artificial neural network ( 400) may also be passed as input.

FIG. 2 is a diagram illustrating the Doppler effect in a road surface type estimation device using speed estimation of a moving object according to an embodiment of the present invention, and FIG. 3 is a graph illustrating the change in frequency of a sound wave transmission/reception signal according to the speed of a moving object. am.

As shown in FIG. 2, a road classification device is mounted on a moving object (vehicle), and a sensor (indicated as T/R in the drawing) is shown as being provided at the front of the vehicle.

When a sensor is installed to perform road surface classification while a moving object is running, as the speed of the moving object (v _c ) increases, the original transmission frequency (f _t ) and the frequencies of the received signal (f _r1 and f _r2 ) increase due to the Doppler effect. ) the frequency difference between them increases.

Referring to the graph shown in FIG. 3, f _t in the middle is the frequency of the transmitted sound wave, f _r1 is the frequency of the reflected wave returning after hitting the road surface behind the center of the sensor based on the traveling direction of the moving object, and f _r2 is the frequency of the reflected wave that hits the road surface behind the center of the sensor and returns based on the moving direction of the moving object. Here, it is a graph when f _t is set to 40 kHz, and f _r1 can be seen as approximately 38 kHz and f _r2 as approximately 42 KHz.

Here, the reason there are two frequencies including f _r1 and f _r2 is because the sound wave has a beam angle.

In other words, as the speed of the moving object increases, the distortion of the received signal becomes more severe, so it is necessary to consider the speed of the moving object when classifying the road surface. However, if the speed of the moving object is not immediately known, the speed of the moving object can be estimated using the Doppler effect.

Hereinafter, with reference to <Equation 1> to <Equation 4> below, the process of calculating (estimating) the speed of a moving object using the sound wave signal received by the speed estimation unit 300 will be described.

First, you can set variables as follows.

c: speed of sound

v _c : speed of moving object

f _t : Frequency of the sound wave signal transmitted from the sensor (sound wave transmitter/receiver)

f _r1 : Since there is a beam angle of the sound wave signal, the frequency of the reflected wave hitting the road surface behind the center of the sensor and returning based on the moving object

f _r2 : Since there is a beam angle of the sound wave signal, the frequency of the reflected wave that hits the road ahead of the center of the sensor and returns based on the moving object

ToF: Time of Flight of the sound wave signal transmitted from the sensor (sound wave transmitter/receiver)

DoF: Distance of flight of the sound wave signal transmitted from the sensor (sound wave transmitter/receiver)

θ: Signal receiving angle of the received sound wave signal (reflected wave)

The frequency (f _r1 ) of the reflected wave hitting the road surface behind the center of the sensor and returning is calculated using Equation 1 below.

[Equation 1]

Here, 'c - v _c ' is the speed of the sound wave reflected and returned from the road surface behind the center of the sensor, which is felt from the moving object, and 'c + v _c ' is the speed of the sound wave transmitted from the sensor, which is felt from the road surface behind the center of the sensor. It is the speed of sound waves.

Meanwhile, the frequency (f _r2 ) of the reflected wave hitting the road surface in front of the center of the sensor and returning is calculated using Equation 2 below.

[Equation 2]

Here, 'c + v _c ' is the speed of the sound wave reflected and returned by the road surface ahead of the center of the sensor, which is felt from the moving object, and 'c - v _c ' is the speed of the sound wave transmitted from the sensor, which is felt from the road surface ahead of the center of the sensor. It is the speed of sound waves.

Then, the speed of the sound wave (v _c ) can be calculated using Equation 3 below.

[Equation 3]

Here, f _t is the frequency of the sound wave signal transmitted from a known sensor (sound wave transmitter and receiver), and f _r1 and f _r2 are obtained by Fourier transforming (FFT: Fast Fourier Transform) the received sound wave signal.

Therefore, the reception angle θ of the reflected wave can be calculated using Equation 4 below.

[Equation 4]

Here, ToF is the Time of Flight of the sound wave signal transmitted from the sensor (sound wave transmitter and receiver), and DoF is the Distance of Flight of the sound wave signal transmitted from the sensor (sound wave transmitter and receiver).

Figures 4A and 4B are diagrams for explaining learning data about vehicle speed.

In the present invention, the speed of the moving object is used as an input to the artificial neural network, so even if the same transmission signal is transmitted, the received signal varies depending on the speed of the moving object (vehicle), so a different type of signal is received. Therefore, for this purpose, the signals received according to the speed of the moving object (vehicle) must be converted into data and the neural network must be trained with the corresponding signals.

For example, FIG. 4A is learning data when the speed of the moving object (vehicle) is 10 km/h or less, and FIG. 4B is learning data when the speed of the moving object (vehicle) is greater than 10 km/h and less than 30 km/h, and FIG. 4c is learning data when the speed of the moving object (vehicle) is greater than 30 km/h and less than 50 km/h, and Figure 4b is learning data when the speed of the moving object (vehicle) is greater than 50 km/h.

In the learning step of the road surface type estimation method using speed estimation of a moving object according to the present invention, a plurality of received signals can be learned for each material and speed.

Figure 5 is a configuration diagram of an embodiment of an artificial neural network in a road surface type estimation device using speed estimation of a moving object according to the present invention.

As shown in FIG. 5, the artificial neural network 400 in the road surface type estimation device using speed estimation of a moving object according to the present invention includes a signal converter 410 and a neural network 420.

The signal converter 410 obtains a frequency domain signal by performing frequency conversion on a preset region in the time domain of the received signal.

The signal converter 410 may be a Short-Time Fourier Transform (STFT) transformer, Fast Fourier Transform (FFT), cepstrum, or wavelet transformer. The frequency domain signal may be 2D or 3D.

The neural network 420 uses the frequency domain signal as an input signal, extracts characteristics of the input signal based on a learned road surface classification model, classifies it, and estimates the type of the road surface.

The neural network 420 is a deep neural network (DNN) of the multi-layer perceptron algorithm including an input layer 401, multiple hidden layers 402, and an output layer 403.

Meanwhile, the structure of the neural network 420 is not limited to the above-mentioned DNN.

The input layer 401 flattens the data of the frequency domain signal and receives it as 1D input.

Data input to the input layer 401 is characterized by extraction and classification through a plurality of hidden layers 402.

The output layer 403 outputs probability values for each type of road surface learned.

The neural network 420 uses the softmax 404 to determine and output the type of road surface with the highest probability among the probability values output from the output layer 403.

Additionally, the neural network 420 may use the speed (v _c ) of the moving object estimated by the speed estimation unit 300 as an input to the input layer 401.

Meanwhile, the neural network 420 can receive atmospheric information (temperature, humidity, and atmospheric pressure information) and use it as input to the input layer 401.

In addition, the neural network 420 can frequency convert the sound wave signal 201 transmitted to the road surface and use it as an input to the input layer 401.

Figure 6 is a diagram for explaining a signal converter in a road surface type estimation device using sound waves according to an embodiment of the present invention.

In FIG. 6 , an STFT converter is used as the signal converter 410 as an example.

As shown in FIG. 6, the STFT converter excludes the crosstalk signal of the sound wave signal transmitted by the sound wave transmitter 101 among the reflected signals received from the sound wave receiver 102, and during the preset time received after the transmission delay. A spectrogram 412 is obtained by performing short-time Fourier transform on the signal 411.

Fourier transform may be performed on a signal for one cycle, or Fourier transform may be performed on a received signal for multiple cycles.

In the present invention, materials are classified using acoustic impedance and surface roughness information. However, acoustic impedance is not a constant, but its value changes depending on the frequency at which the sound wave vibrates. Therefore, analysis in the frequency domain is necessary. You can use the Time Fourier Transform, which is one of several methods to convert a received signal in the time domain into a frequency domain signal. Short-time Fourier Transform is used to check the FFT at each time (sampling time). (Short-Time Fourier Transform) is used.

In addition, frequency analysis is possible using not only Short-Time Fourier Transform but also Wavelet, and the present invention explains that STFT is used to secure sufficient data while reducing the amount of calculation.

Short-time Fourier transform (STFT) is a method designed to consider changes in time that could not be resolved in the existing Fourier transform. STFT divides a long signal that changes with time into short time units and then applies Fourier transform.

However, STFT also has disadvantages. Since the signal is separated according to the window length, the length of the signal used in the Fourier transform is reduced and the frequency resolution is deteriorated accordingly. However, even if the frequency resolution is improved by increasing the window length, the time resolution deteriorates. Wavelet Transform emerged to overcome the limitations in resolution caused by this trade-off relationship between frequency and time.

If the window length is fixed in STFT, WT performs STFT several times while changing the window length. Additionally, if STFT uses a sinusoid that extends infinitely in time as the basic function, wavelets have several types of functions that exist for a finite period of time. Wavelet functions include Morlet, Daubechies, Coiflets, Biorthogonal, Mexican Hat, and Symlets.

Figure 7 is a configuration diagram of another embodiment of an artificial neural network in a road surface type estimation device using speed estimation of a moving object according to the present invention.

As shown in FIG. 7, the artificial neural network 400 is a multi-layer perceptron (Multi-Layer) including a convolution performance unit 501, a transfer layer 502, multiple hidden layers 503, and an output layer 504. It is a deep convolutional neural network (DCNN) of the Perceptron algorithm.

The convolution performing unit 501 performs a 1D convolution operation multiple times on the received digital input signal, and performs batch normalization, Relu function, and MaxPooling function for each convolution operation. , the output of the last convolution operation outputs flattened data to the transfer layer 502.

The transfer layer 502 receives the flattened output data of the convolution performing unit 501 in 1D.

Data input to the transmission layer 502 is characterized by extraction and classification through a plurality of hidden layers 503.

The output layer 504 outputs probability values for each type of road surface learned.

The artificial neural network 400 uses softmax 505 to determine the type of road surface with the highest probability among the probability values output from the output layer 504 and outputs it.

Additionally, the artificial neural network 420 may use the speed (vc) of the moving object estimated by the speed estimation unit 300 as an input to the convolution performing unit 501 or the transfer layer 502.

Meanwhile, the artificial neural network 400 can receive atmospheric information (temperature, humidity, and atmospheric pressure information) and use it as input to the convolution performing unit 501.

Additionally, the artificial neural network 400 may also use the sound wave signal 201 transmitted to the road surface as an input to the convolution performing unit 501.

FIG. 8 is a diagram for explaining the operation of the convolution performance unit of FIG. 7.

As shown in FIG. 8, the convolution performing unit 501 of FIG. 6 performs a 1D convolution operation on the input signal five times, and for each convolution operation, batch normalization, Relu function, and The MaxPooling function is performed, and the output of the final convolution operation is flattened data.

The first convolution result 602 for the input signal 601 is the result of performing 1D conv (64,16), BN, ReLU, and MP(8) on the input signal 601, and the second convolution is performed The result 603 is the result of performing 1D conv (32,32), BN, ReLU, and MP(8) on the first convolution result 602, and the third convolution result 604 is the result of performing the second convolution result 602. It is the result of performing 1D conv (16,64), BN, ReLU, and MP(8) on the convolution result (603), and the fourth convolution result (605) is the 1D convolution result (604). It is the result of performing conv (8,128), BN, and ReLU, and the fifth convolution result 606 is the result of performing 1D conv (4,2568), BN, and ReLU on the fourth convolution result 605. am.

Meanwhile, in the present invention, a method of performing a one-dimensional (1D) convolution operation is described as an example, but convolution operation is possible not only in 1D, but also in 2D and 3D.

Figure 9 is a flowchart of an embodiment of a road surface type estimation method using speed estimation of a moving object according to the present invention.

First, in order to perform the road surface type estimation method using sound waves according to the present invention, the first learning step (S10) must be preceded and a road surface classification model must be created.

In the first learning step (S10), after transmitting a sound wave signal for multiple types of road surfaces and multiple vehicle speeds, the reflected signal is received, the signal is converted into a digital signal, and the converted digital signal is converted into a digital signal. It is converted into a frequency domain signal, and the frequency domain signal is input to the neural network 420 to learn a road surface classification model.

Here, in the first learning step (S10), a Short-Time Fourier Transform (STFT) transformer, Fast Fourier Transform (FFT), cepstrum, or wavelet transformer is used to convert the frequency domain signal. ) can be used. The frequency domain signal may be 2D or 3D.

Thereafter, under the control of the control unit 500, a sound wave signal is transmitted to the road surface of which the type is to be determined, and then the reflected signal is received (S20).

Thereafter, under the control of the control unit 500, the analog signal is converted into a digital signal for the preset area of the received signal (S30).

In the signal conversion step (S30), in the received signal, excluding the crosstalk signal of the transmitted sound wave signal, the point at which the amplitude of the signal received after the transmission delay is greatest is referenced for each period of the sound wave signal. Converts the signal for a preset time into a digital signal.

For example, if t_0 is the point in time when the amplitude of the received signal after the transmission delay is highest, a total of (a+b) ms can be observed from t_0 - a [ms] to t_0 + b [ms], a and b can be adjusted depending on the environment or conditions.

Thereafter, under the control of the controller, the speed of the moving object is estimated using the digital signal (S40).

In the moving object speed estimation step (S40), the speed of the moving object is estimated using Equation 1 to Equation 4 below.

First, you can set variables as follows.

c: speed of sound

v _c : speed of moving object

f _t : Frequency of the sound wave signal transmitted from the sensor (sound wave transmitter/receiver)

f _r1 : Since there is a beam angle of the sound wave signal, the frequency of the reflected wave that hits the road behind the center of the sensor and returns based on the moving object

f _r2 : Since there is a beam angle of the sound wave signal, the frequency of the reflected wave that hits the road ahead of the center of the sensor and returns based on the moving object

ToF: Time of Flight of the sound wave signal transmitted from the sensor (sound wave transmitter/receiver)

DoF: Distance of flight of the sound wave signal transmitted from the sensor (sound wave transmitter/receiver)

θ: Signal receiving angle of the received sound wave signal (reflected wave)

The frequency (f _r1 ) of the reflected wave hitting the road surface behind the center of the sensor and returning is calculated using Equation 1 below.

[Equation 1]

Here, 'c - v _c ' is the speed of the sound wave reflected and returned from the road surface behind the center of the sensor, which is felt from the moving object, and 'c + v _c ' is the speed of the sound wave transmitted from the sensor, which is felt from the road surface behind the center of the sensor. It is the speed of sound waves.

Meanwhile, the frequency (f _r2 ) of the reflected wave hitting the road surface in front of the center of the sensor and returning is calculated using Equation 2 below.

[Equation 2]

Here, 'c + v _c ' is the speed of the sound wave reflected and returned by the road surface ahead of the center of the sensor, which is felt from the moving object, and 'c - v _c ' is the speed of the sound wave transmitted from the sensor, which is felt from the road surface ahead of the center of the sensor. It is the speed of sound waves.

Then, the speed of the sound wave (v _c ) can be calculated using Equation 3 below.

[Equation 3]

Here, f _t is the frequency of the sound wave signal transmitted from a known sensor (sound wave transmitter and receiver), and f _r1 and f _r2 can be obtained by performing Fourier Transform (FFT: Fast Fourier Transform) on the received sound wave signal.

Therefore, the reception angle θ of the reflected wave can be calculated using Equation 4 below.

[Equation 4]

Here, ToF is the Time of Flight of the sound wave signal transmitted from the sensor (sound wave transmitter and receiver), and DoF is the Distance of Flight of the sound wave signal transmitted from the sensor (sound wave transmitter and receiver).

Thereafter, under the control of the control unit 500, signal conversion is performed on the digital signal to obtain a frequency domain signal (S50).

In the frequency domain signal acquisition step (S50), the signal converter 410 performs frequency conversion on the digital signal to obtain the frequency domain signal.

Thereafter, under the control of the control unit 500, the frequency domain signal and the estimated speed (v _c ) of the moving object are used as input signals of the neural network 420, and based on the learned road surface classification model, the input signal The characteristics of are extracted and classified to determine the type of the road surface (S60).

The neural network 420 is a deep neural network (DNN) of the multi-layer perceptron algorithm including an input layer 401, multiple hidden layers 402, and an output layer 403, The output layer 403 outputs probability values for each type of road surface learned, and the neural network uses softmax 404 to determine and output the type of road surface with the highest probability. .

Meanwhile, the structure of the neural network 420 is not limited to the above-mentioned DNN.

The neural network 420 can receive atmospheric information (temperature, humidity, and atmospheric pressure information) and use it as input to the input layer 401.

Additionally, the neural network 420 can frequency-convert the sound wave signal transmitted to the road surface and use it as an input to the input layer 401.

Figure 10 is a flowchart of another embodiment of a road surface type estimation method using speed estimation of a moving object according to the present invention.

First, in order to perform the road surface type estimation method using the speed estimation of the moving object according to the present invention, the second learning step (S90) must first be performed and a road surface classification model must be created.

In the second learning step (S90), after transmitting a sound wave signal for multiple types of road surfaces and multiple vehicle speeds, the reflected signal is received, the signal is converted into a digital signal, and the converted digital signal is converted into a digital signal. The lobster is input into the artificial neural network 400 and multiple convolution operations are performed to learn the road surface classification model.

Thereafter, under the control of the control unit 500, a sound wave signal is transmitted to the road surface of which the type is to be determined, and then the reflected signal is received (S20).

Thereafter, under the control of the control unit 500, the analog signal is converted into a digital signal for the preset area of the received signal (S30).

In the signal conversion step (S30), in the received signal, excluding the crosstalk signal of the transmitted sound wave signal, the point at which the amplitude of the signal received after the transmission delay is greatest is referenced for each period of the sound wave signal. Converts the signal for a preset time into a digital signal.

Thereafter, under the control of the controller, the speed of the moving object is estimated using the digital signal (S40).

In the moving object speed estimation step (S40), the speed of the moving object is estimated using Equation 1 to Equation 4 in the same manner as described in FIG. 9.

Thereafter, under the control of the control unit 500, the digital signal is received from the artificial neural network 400 and multiple convolution operations are performed (S70).

In the convolution operation step (S70), a 1D convolution operation is performed on the digital signal multiple times, and batch normalization, Relu function, and MaxPooling function are performed for each convolution operation, The output of the final convolution operation is flattened data.

Thereafter, under the control of the control unit 500, based on the road surface classification model learned in the artificial neural network, the characteristics of the convolution operation signal (flattened data) and the estimated speed of the moving object are extracted and classified. Thus, the type of the road surface is determined (S80).

The artificial neural network 400 includes a convolution performing unit 501 that receives the digital signal and performs multiple convolution operations, a transfer layer 502, a plurality of hidden layers 503, and an output layer 504. It is a deep convolutional neural network (DCNN) of the multi-layer perceptron algorithm, and the output layer 504 outputs probability values for each type of learned road surface, and the artificial neural network 400 Using softmax 505, the type of road surface with the highest probability is determined and output.

Meanwhile, the structure of the artificial neural network 400 is not limited to the DCNN mentioned above.

The artificial neural network 400 can receive atmospheric information (temperature, humidity, and atmospheric pressure information) and use it as input to the convolution performing unit 501.

Additionally, the artificial neural network 400 can also use the sound wave signal transmitted to the road surface as an input to the convolution performing unit 501.

In the above, a method for estimating a road surface type using speed estimation of a moving object according to an embodiment of the present invention has been described, but a computer-readable recording medium storing a program for implementing the method for estimating a road surface type using estimating the speed of a moving object and a moving object. Of course, a program stored in a computer-readable recording medium to implement a road surface type estimation method using speed estimation can also be implemented.

In other words, those skilled in the art can easily understand that the method for estimating the type of road surface using the speed estimation of the moving object described above can be provided in a recording medium that can be read by a computer by tangibly implementing a program of commands for implementing it. There will be. In other words, it can be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable by those skilled in the computer software art. Examples of the computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and floptical disks. Included are magneto-optical media, and hardware devices specifically configured to store and perform program instructions, such as ROM, RAM, flash memory, USB memory, and the like. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The present invention is not limited to the above-described embodiments, and the scope of application is diverse. Of course, various modifications and implementations are possible without departing from the gist of the present invention as claimed in the claims.

100: Sound wave transceiver
101: sound wave transmitter 102: sound wave receiver
200: Analog-to-digital converter 300: Speed estimation unit
400: Artificial neural network
410: signal converter 420: neural network
501: Convolution execution unit 502: Transfer layer
503: Hidden layer 504: Output layer
500: Control unit (MCU)
S10: First learning stage
S20: Reflected wave reception stage
S30: Signal conversion step
S40: Moving object speed estimation step
S50: Frequency domain signal acquisition step
S60: First road surface type determination step
S70: Convolution operation step
S80: Second road surface type determination step
S90: Second learning stage

Claims (10)

In a road surface type estimation device using speed estimation of a moving object,
A sound wave transmitting and receiving unit (sensor) 100 for transmitting a sound wave signal to the road surface of which the type is to be determined and then receiving the reflected signal;
An analog-to-digital converter (200) for converting the analog signal of the received signal into a digital signal;
a speed estimation unit 300 for estimating the speed of the moving object by processing the converted digital signal;
an artificial neural network 400 that uses the converted digital signal and the estimated speed of the moving object as input signals and classifies the input signals based on a learned road surface classification model to estimate the type of the road surface; and
A control unit (MCU) 500 that controls the operation of the sound wave transmitting and receiving unit (sensor), the analog-to-digital converter, the speed estimation unit, and the artificial neural network.
A road surface type estimation device using speed estimation of a moving object including.
According to paragraph 1,
The road surface classification model is,
A road surface type estimation device using speed estimation of a moving object, which is characterized by learning about multiple types of road surfaces and multiple vehicle speeds.
According to paragraph 1,
In the speed estimation unit 300,
Based on the traveling direction of the moving object, the frequency (f _r1 ) of the reflected wave hitting the road surface behind the center of the sensor and returning is calculated using Equation 1 below,
[Equation 1]

(Here, c is the speed of the sound wave, vc is the speed of the moving object, 'c - v _c ' is the speed of the sound wave reflected on the road surface behind the center of the sensor felt from the moving object, and 'c + v _c ' is the speed of the sensor This is the speed of the sound wave transmitted from the sensor, which is felt from the road surface behind the center of.)
The frequency (f _r2 ) of the reflected wave hitting the road surface in front of the center of the sensor and returning is calculated using Equation 2 below,
[Equation 2]

(Here, 'c + v _c ' is the speed of the sound wave reflected and returned by the road surface ahead of the center of the sensor, which is felt from the moving object, and 'c - v _c ' is the speed of the sound wave transmitted from the sensor, which is felt from the road surface ahead of the center of the sensor. This is the speed of the sound wave.)
The speed of the sound wave (v _c ) is calculated using Equation 3 below,
[Equation 3]

(Here, f _t is the frequency of the sound wave signal transmitted from the sensor.)
The reception angle θ of the reflected wave is calculated using Equation 4 below,
[Equation 4]

(Here, ToF is the Time of Flight of the sound wave signal transmitted from the sensor, and DoF is the Distance of Flight of the sound wave signal transmitted from the sensor (sound wave transmitter and receiver).)
A road surface type estimation device using speed estimation of a moving object, characterized in that the speed of the moving object is estimated using the above [Equation 1] to [Equation 4].
According to paragraph 3,
The frequency of the reflected wave (f _r1 ), which hits the road surface behind the center of the sensor and returns, and the frequency (f _r2 ) of the reflected wave, which hits the road surface ahead of the center of the sensor and returns, is the Fourier transform (FFT: Fast Fourier) of the received sound wave signal. A road surface type estimation device using speed estimation of a moving object, which is obtained by transform.
In the road surface type estimation method using speed estimation of a moving object,
Under the control of the control unit, a sound wave signal is transmitted from the sound wave transmitting and receiving unit (sensor) to the road surface whose type is to be known, and then the reflected signal is received (S20);
A signal conversion step (S30) of converting an analog signal into a digital signal for a preset area of the received signal under the control of the control unit;
A moving object speed estimation step (S40) of estimating the speed of the moving object by processing the digital signal under the control of the control unit;
A frequency domain signal acquisition step (S50) of obtaining a frequency domain signal by performing frequency conversion on a preset region of the digital signal under the control of the control unit; and
Under the control of the control unit, the frequency domain signal and the estimated speed of the moving object are used as input signals of the neural network, and based on the learned road surface classification model, the characteristics of the input signal are extracted and classified to determine the type of the road surface. The first road surface type determination step (S60) to determine
A road surface type estimation method using speed estimation of a moving object including.
According to clause 5,
For multiple types of road surfaces and multiple vehicle speeds, after transmitting a sound wave signal, receiving a reflected signal and converting the signal into a digital signal, converting the converted digital signal into a frequency domain signal, and converting the frequency domain signal. A first learning step (S10) in which signals are input into the neural network to learn the road surface classification model.
A road surface type estimation method using speed estimation of a moving object, further comprising:
In the road surface type estimation method using speed estimation of a moving object,
Under the control of the control unit, a sound wave signal is transmitted from the sound wave transmitting and receiving unit (sensor) to the road surface whose type is to be known, and then the reflected signal is received (S20);
A signal conversion step (S30) of converting an analog signal into a digital signal for a preset area of the received signal under the control of the control unit;
A moving object speed estimation step (S40) of estimating the speed of the moving object by processing the digital signal under the control of the control unit;
A convolution operation step (S70) of receiving the digital signal from an artificial neural network and performing multiple convolution operations under the control of the control unit; and
Under the control of the control unit, based on the road surface classification model learned in the artificial neural network, the characteristics of the convolution calculated signal and the estimated speed of the moving object are extracted and classified to determine the type of the road surface. Type decision step (S80)
A road surface type estimation method using speed estimation of a moving object including.
In clause 7,
For many types of road surfaces and multiple vehicle speeds, after transmitting a sound wave signal, the reflected signal is received, the signal is converted into a digital signal, and the converted digital signal is input into the artificial neural network to perform multiple convolution calculations. The second learning step (S90) of learning the road surface classification model by performing
A road surface type estimation method using speed estimation of a moving object, further comprising:
According to clause 5 or 7,
In the moving object speed estimation step (S40),
Based on the traveling direction of the moving object, the frequency (f _r1 ) of the reflected wave hitting the road surface behind the center of the sensor and returning is calculated using Equation 1 below,
[Equation 1]

(Here, c is the speed of the sound wave, vc is the speed of the moving object, 'c - v _c ' is the speed of the sound wave reflected on the road surface behind the center of the sensor felt from the moving object, and 'c + v _c ' is the speed of the sensor This is the speed of the sound wave transmitted from the sensor, which is felt from the road surface behind the center of.)
The frequency (f _r2 ) of the reflected wave hitting the road surface in front of the center of the sensor and returning is calculated using Equation 2 below,
[Equation 2]

(Here, 'c + v _c ' is the speed of the sound wave reflected and returned by the road surface ahead of the center of the sensor, which is felt from the moving object, and 'c - v _c ' is the speed of the sound wave transmitted from the sensor, which is felt from the road surface ahead of the center of the sensor. This is the speed of the sound wave.)
The speed of the sound wave (v _c ) is calculated using Equation 3 below,
[Equation 3]

(Here, f _t is the frequency of the sound wave signal transmitted from the sensor.)
The reception angle θ of the reflected wave is calculated using Equation 4 below,
[Equation 4]

(Here, ToF is the Time of Flight of the sound wave signal transmitted from the sensor, and DoF is the Distance of Flight of the sound wave signal transmitted from the sensor (sound wave transmitter and receiver).)
A road surface type estimation method using the speed estimation of a moving object, characterized in that the speed of the moving object is estimated using the above [Equation 1] to [Equation 4].
According to clause 9,
In the moving object speed estimation step (S40),
The frequency of the reflected wave (f _r1 ), which hits the road surface behind the center of the sensor and returns, and the frequency (f _r2 ) of the reflected wave, which hits the road surface ahead of the center of the sensor and returns, is the Fourier transform (FFT: Fast Fourier) of the received sound wave signal. A road surface type estimation method using speed estimation of a moving object, which is obtained by transform.