KR102416175B1 - Appratus and method for accident detection - Google Patents

Abstract

An apparatus and method for detecting an accident occurring at a remote location are disclosed. Provided are an accident detection device and method capable of detecting and handling accidents on the road more quickly and accurately by enabling acoustic accident detection through accident sound deep learning.

Description

Accident detection device and method

The present invention relates to an apparatus and method for detecting an accident occurring at a remote location, and more particularly, to an acoustic accident detection apparatus and method through accident sound deep learning.

When many cars are driving on the road, traffic accidents occur due to various causes, resulting in damage to life and property. In particular, in an area where a vehicle is traveling at high speed, such as a highway, since the risk of a secondary accident is very high when an initial accident occurs, an accident detection system capable of notifying the occurrence of an accident to a subsequent driving vehicle is required.

Although video-based accident detection systems have been widely used in recent years, the number of cases of erroneous detection of accidents is increasing depending on environmental factors such as bad weather conditions or the inability to secure a camera view due to smoke in the event of a fire.

An accident detection system based on image data has the advantage of delivering immediate road information, but there is an inevitable limitation in accident detection performance due to the above-described problems.

An object of the present invention is to provide an accident detection device and method for detecting whether an accident has occurred based on a sound collected from a road in order to solve the above problems.

An object of the present invention is to provide an accident detection device and method capable of quickly detecting whether an accident has occurred through deep learning of an accident sound.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

In order to achieve the above object, an embodiment is an apparatus for detecting an accident occurring at a remote location, comprising: a memory buffer for storing a stream of input sound sensed through a sound collector disposed in the vicinity of the remote location; and at least one processor; wherein the processor is configured to: obtain a set of sub sounds from the stream of input sounds, generate a set of input data for running a thought sound learning model based on the set of sub sounds, and generate a set of thought sounds based on the set of input data. An accident detection device is provided, configured to determine a detection result as to whether the input sound includes an accident sound according to the execution result of the learning model.

In order to achieve the above object, an embodiment is a method of detecting an accident occurring at a remote location, comprising: acquiring a stream of input sound sensed through a sound collector disposed in the vicinity of the remote location; Acquiring sub sounds of , generating a series of input data for executing a thought sound learning model based on the series of sub sounds, and the input sound according to the execution result of the thinking sound learning model based on the series of input data It provides an accident detection method comprising the step of determining a detection result as to whether or not an accident sound is included.

The solutions to the technical problems to be achieved in the present invention are not limited to the solutions mentioned above, and other solutions not mentioned are clear to those of ordinary skill in the art to which the present invention belongs from the description below. can be understood clearly.

According to the embodiment, it is possible to more quickly and accurately identify and handle accidents on the road by supplementing the disadvantages of the conventionally operated image-based accident analysis system.

According to the embodiment, the accuracy of accident detection is improved by using a thought sound learning model based on a deep learning algorithm.

Effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a view showing an operating environment of an accident detection system according to an embodiment.
2 is a block diagram showing the configuration of an accident detection system according to an embodiment.
3 is a block diagram of an accident detection device according to an embodiment.
4 is a flowchart of an accident detection method according to an embodiment.
5 is a diagram for explaining a process of acquiring a sub sound according to an exemplary embodiment.
6 is a flowchart of a process of generating input data according to an exemplary embodiment.
7 is a graph exemplarily illustrating classification for generating input data according to an embodiment.
8 is a diagram exemplarily illustrating a thought sound learning model according to an embodiment.
9 is a diagram for explaining a process of determining whether an accident has occurred, according to an exemplary embodiment.
10 is a flowchart of a process of generating input data according to an exemplary embodiment.
11 is a diagram exemplarily illustrating matrix data divided to generate input data according to an embodiment.
12 is a diagram exemplarily illustrating a thought sound learning model according to an embodiment.
13 is a diagram illustrating an operation process of an accident processing unit according to an exemplary embodiment.
14 is a diagram illustrating a data flow of an accident detection system according to an embodiment.
15 is a table exemplarily showing an accident detection result of an accident detection system according to an embodiment.
16 is a table exemplarily showing an accident erroneous detection result of the accident detection system according to an embodiment.

The embodiments disclosed herein will be described in detail with reference to the accompanying drawings, but the same or similar reference numerals are assigned to the same or similar components, and overlapping descriptions thereof will be omitted. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate that a feature, number, step, operation, component, part, or a combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof. Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

1 is a view showing an operating environment of an accident detection system according to an embodiment.

The accident detection system according to the embodiment may determine whether an accident has occurred by analyzing the sound collected from the road, and may provide accident information such as the type and location of the accident.

The accident detection system includes a sound collector 10 and an accident detection device 100 .

The sound collector 10 may be disposed on a road to acquire an input sound generated on the road. In one example, one or more sound collectors 10 may be disposed along the road on which the vehicle travels. In one example, several sound collectors 10 may be disposed inside the tunnel.

The sound collector 10 transmits the collected input sound to the accident detection device 100 . For example, the sound collector 10 may transmit an input sound to the accident detection device 100 in real time or periodically.

For example, if an accident occurs at an arbitrary location S on the road, the sound collector 10 disposed in the vicinity of the accident location S may generate an accident sound (eg, sudden braking, impact sound, An input sound including an explosion sound, a horn sound, etc.) is sensed. The sound collector 10 disposed in the vicinity of the accident location S may transmit an input sound including the accident sound to the accident detection device 100 in real time. In one example, the accident location (S) may be located in the tunnel.

The accident detection device 100 analyzes the input sound received from the sound collector 10 to detect whether an accident has occurred. The configuration of the accident detection device 100 will be described later with reference to FIG. 3 .

The accident detection system may further include an operating device 200 . The accident detection device 100 transmits the accident detection result to the operating device 200 , and the operating device 200 provides necessary information to the operator according to the accident detection result received from the accident detection device 100 .

The sound collector 10 , the accident detection device 100 , and the operating device 200 may be located in other locations that are physically separated from each other. For example, the sound collector 10 may be disposed on a road on which the vehicle travels, such as inside a tunnel. The accident detecting device 100 may be disposed at a location remote from the sound collector 10 to analyze input sounds collected from a plurality of sound collectors 10 . For example, the accident detection device 100 may be disposed outside the tunnel or at a control center. The operating device 200 may be located in the central control center as a central control server. The operation device 200 may determine a response action according to the accident detection result received from the accident detection device 100 , and provide necessary information to the operator.

2 is a block diagram showing the configuration of an accident detection system according to an embodiment.

As described above with reference to FIG. 1 , the accident detection system may include a sound collector 10 and an accident detection device 100 .

The sound collector 100 is disposed on the road and collects sounds generated on the road. The sound collector 10 may include a microphone, an audio codec, a microprocessor, and an Ethernet controller.

Microphones can convert sound from the road into electrical signals. The audio codec can convert the analog signal of the sound collected by the microphone into digital data.

The sound collector transmits the digital data converted by the audio codec to the accident analysis unit 100, and the accident analysis unit 100 analyzes the transmitted digital data.

The microprocessor may control a peripheral chip embedded in the sound collector 10, collect data from the peripheral chip, and perform calculations. The microprocessor may also receive acoustic digital data received from the audio codec and store it in the internal memory. In addition, the microprocessor may control the Ethernet controller to communicate with the accident detection device 100 . In other words, the microprocessor may serve as a CPU of the sound collector 10 .

The Ethernet controller is in charge of communication with the sound collector 10 and the accident detection device 100, for example, the accident analysis unit 100a through TCP/IP. In addition, the Ethernet controller enables communication between the sound collector 10 and an external server, for example, the operating device 200 .

The accident detection device 100 includes an accident analysis unit 100a that determines whether an accident has occurred based on the input sound. The accident analysis unit 100a processes and analyzes the input sound received from the sound collector 10 to determine the accident detection result. Here, the input sound corresponds to the conversion of the analog signal of the sound collected through the microphone of the sound collector 10 into digital information in the audio codec of the sound collector 10 .

The accident analysis unit 100a may determine whether an accident has occurred, that is, an accident detection result by providing input data obtained by processing the input sound to the accident sound learning model. The accident analysis unit 100a may transmit the accident detection result to the accident processing unit 100b and/or the operation device 200 .

The accident detection device 100 may further include an accident processing unit 100b that provides and performs an accident response plan according to the accident detection result of the accident analysis unit 100a. The accident processing unit 100b may determine and perform a response action based on the accident detection result of the accident analysis unit 100a. The operation of the accident processing unit 100b will be described later with reference to FIG. 13 .

Meanwhile, the accident analysis unit 100a may transmit the accident detection result to the operation device 200 through an interface with an external server. Here, the external server includes a web server that provides monitoring and operation services for the accident detection system. The operating device 200 refers to a terminal device that accesses such an external server and receives monitoring and operating services for the accident detection system. Here, the terminal device may include various types of electronic devices capable of running a web application, such as, for example, a desktop computer, a smart phone, a tablet, and a laptop computer.

The accident analysis unit 100a and the accident processing unit 100b schematically describe the accident detection device 100 in terms of the functional aspects of the accident detection device 100 . Hereinafter, the configuration of the accident detection device 100 will be described in more detail with reference to FIG. 3 .

3 is a block diagram of an accident detection device according to an embodiment.

The accident detection device 100 may detect an accident that occurred in the remote location (S). The remote location S may be another location that is far from a location where the accident detection device 100 is located. When an accident occurs at the remote location S, the sound collector 10 disposed in the vicinity of the remote location S acquires an input sound including an accident sound, and transmits the acquired input sound to the accident detection device 100 do.

The accident detection device 100 may include a communication unit 110 , a storage unit 120 , and a processor 130 . The components shown in FIG. 3 are exemplary, and the accident detection device 100 may include some of the components shown in FIG. 3 or further include additional components.

The communication unit 110 performs data communication with an external device including the sound collector 10 and an external server. The accident detection device 100 may transmit data to and receive data from the external device through the communication unit 110 . In an example, the communication unit 110 may include software and hardware modules for transmitting and receiving data using a communication protocol such as TCP/IP.

The storage unit 120 stores input data, intermediate data, and detection results required for accident detection. The storage unit 120 may include a memory buffer 121 , a database 122 , and a model storage unit 123 .

The memory buffer 121 means a memory space allocated to store a stream of input sound received from the sound collector 10 . The input sound collected by the sound collector 10 is transmitted to the accident detection device 100 in real time. For example, the sound collector 10 may transmit the collected input sound to the accident detection device 100 in a real-time streaming manner. The accident detection device 100 may store the stream of the input sound received from the sound collector 10 in the memory buffer 121 .

The accident detection apparatus 100 may continuously read the input sound stream stored in the memory buffer 121 in a predetermined unit and generate input data to be provided as an input value to the accident sound learning model. A subsequent stream may be filled in the memory buffer 121 by the unit in which the accident detection device 100 reads the stream of the input sound from the memory buffer 121 to generate the input data. In an example, the memory buffer 121 may operate in a first in first out (FIFO) manner.

The database 122 indexes and stores the input sound data received from the sound collector 10 . Here, the input sound data may include an audio file. The database 122 provides input sound data stored in the database 120 to the operating device 200 through the external server in response to a request transmitted from the operating device 200 connected to the external server by interworking with the external server. can

The model storage unit 123 may store an accident sound learning model that the accident detection device 100 executes for accident detection. The thinking sound learning model may include a learning model based on an artificial neural network. For example, the thought sound learning model may be implemented using a deep learning algorithm based on a deep neural network (DNN) or a recurrent neural network (RNN).

The processor 130 corresponds to a central processing unit (CPU) of the accident detection device 100 . Processor 130 may include one or more processors.

The processor 130 may control the operation of the accident analysis unit 100a.

The processor 130 obtains a series of sub sounds from a stream of input sounds, generates a series of input data for executing a thought sound learning model based on the obtained series of sub sounds, and adds to the generated series of input data. It may be configured to determine a detection result as to whether the input sound includes an accident sound according to the execution result of the accident sound learning model based on the accident sound.

The processor 130 may be configured to obtain a series of sub sounds from a stream of input sounds stored in the memory buffer 121 . To this end, the processor 130 may obtain a series of sub sounds by dividing the stream of the input sound according to a predetermined length of time. The sub sound acquisition process will be described later in detail with reference to FIG. 5 .

In one example the series of sub sounds includes a first sub sound and a second sub sound subsequent to the first sub sound, and the processor 130 is configured to generate a second sub sound overlapping at least a portion of the first sub sound from the stream of input sound. You can get a sub sound.

In one example, the series of sub sounds includes a first sub sound and a second sub sound subsequent to the first sub sound, and the processor 130 is configured to: can be obtained.

The processor 130 may be configured to generate a series of input data for executing the thought sound learning model based on the obtained series of sub sounds. To this end, the processor 130 converts the current sub sound among a series of sub sounds into a set of frequency component values for a predetermined frequency band, and generates input data corresponding to the current sub sound based on the set of frequency component values. can create In an example, the processor 130 may convert the current sub sound into a set of frequency component values for a predetermined frequency band through Fast Fourier Transform (FFT).

In one example, the processor 130 determines the first set of classification values according to a difference between the first arrangement for the set of frequency component values and the second arrangement for the frequency component values of the base sound, the first arrangement and the second arrangement 2 Normalizing the change of each frequency component value of the array to determine a second set of classification values, and determining the input data corresponding to the current sub sound based on at least one of the first set of classification values and the second set of classification values can create Here, the base sound may include at least one sub sound that is earlier than the current sub sound among a series of sub sounds. This will be described later with reference to FIGS. 6 and 7 .

In an example, the processor 130 generates a matrix of feature values for a set of frequency component values, and divides the matrix of feature values according to a predetermined time interval to apply the current sub sound based on a plurality of sub matrices generated. Corresponding input data may be generated. This will be described later with reference to FIGS. 10 and 11 .

On the other hand, the processor 130 may obtain a plurality of execution results of the accident sound learning model, and determine the detection result based on the plurality of execution results. This will be described in detail with reference to FIG. 9 .

The processor 130 may control the operation of the accident processing unit 100b. For example, the processor 130 may be further configured to generate a command for controlling a camera installed in the vicinity of a remote location based on a detection result using the accident sound learning model. In addition, the processor 130 may be further configured to generate a notification message for accident information based on the detection result using the accident sound learning model. This will be described later with reference to FIG. 13 .

Additionally, the accident detection device 100 may further include a running processor (not shown). The learning processor, in conjunction with the processor 130 or independently of the processor 130, learns the accident sound learning model stored in the model storage unit 123 or uses the accident sound learning model to infer whether an accident has occurred. operation can be performed.

4 is a flowchart of an accident detection method according to an embodiment.

The accident detection apparatus 100 may perform an accident detection method of determining whether an accident has occurred based on the input sound acquired from the sound collector 10 .

The accident detection method according to the embodiment includes the steps of obtaining a stream of an input sound sensed through the accident sound collector 10 disposed in the vicinity of the remote location in order to detect an accident occurring at a remote location (S310), the input sound Acquiring a series of sub sounds from the stream of (S320), generating a series of input data for executing a thought sound learning model based on the series of sub sounds (S330), and thinking sounds based on the series of input data It may include a step (S340) of determining a detection result as to whether the input sound includes an accidental sound according to the execution result of the learning model.

In step S310 , with reference to FIG. 3 , the processor 130 may acquire a stream of an input sound sensed through the accident sound collector 10 disposed in the vicinity of the remote location S.

In one example, the remote location S may be any location on the road. In one example, the sound collector 10 may be disposed inside the tunnel to collect sounds generated inside the tunnel. When the accident occurs inside the tunnel, the location of the accident, that is, the remote location S is located inside the tunnel, and the sound collector 10 disposed in the vicinity of the remote location S is an input including the accident sound generated by the accident. get sound. The sound collector 10 transmits the acquired input sound to the accident detection device 100 . For example, the sound collector 10 may transmit a stream of the input sound to the accident detection device 100 in a real-time streaming manner.

In step S310 , the processor 130 receives, through the communication unit 110 , a stream of input sound from the accident sound collector 10 disposed in the vicinity of the remote location S, and stores it in the memory buffer 121 . do. Subsequently or concurrently with this, the processor 130 may continuously read the stream of the input sound from the memory buffer 121 .

In step S320, the processor 130 may obtain a series of sub sounds from the stream of the input sound obtained in step S310.

In step S320, the processor 130 may obtain a series of sub sounds by dividing the stream of the input sound obtained in step S310 according to a predetermined length of time. Here, the predetermined length of time can be set as an environmental variable of the accident detection device 100 . For example, the predetermined length of time may be 0.1 seconds or less. For example, the predetermined length of time may be 10 seconds or more. In an example, the sub sound for the thinking sound learning model based on RNN may be generated with a longer length than the sub sound for the thought sound learning model based on the DNN, but is not limited thereto. The predetermined length of time can be adjusted according to factors such as the installation location of the accident detection device 100, the installation environment, and the noise level. Hereinafter, step S320 will be described with reference to FIG. 5 .

5 is a diagram for explaining a process of acquiring a sub sound according to an exemplary embodiment.

The processor 130 may acquire a series of sub sounds by dividing the stream of the input sound stored in the memory buffer 121 according to a predetermined length of time. The series of sub sounds may include a first sub sound and a second sub sound following the first sub sound.

In an example, the processor 130 may obtain a second sub sound that overlaps with at least a part of the first sub sound from the stream of the input sound stored in the memory buffer 121 .

Referring to FIG. 5 , at least a portion of the input sound stream IN_STRM is stored in the exemplary memory buffer 121 of the first time point T1 in the left box of FIG. 5 . The processor 130 converts the input sound stream IN_STRM stored in the memory buffer 121 according to a predetermined time length to the first block B1, the second block B2, the third block B3, and the fourth block. It can be divided into (B4). For example, the predetermined length of time may be 0.05 seconds. In one example, the processor 130 may obtain the first sub sound by bundling the first block B1 and the second block B2. In an example, the processor 130 may obtain the first sub sound SUB_S1 by bundling two or more blocks (eg, B1, B2, and B3). That is, the time length of the first sub sound SUB_S1 may be 0.1 seconds.

The size of the sub sound may change according to a sampling rate of the input sound. For example, if 48000 bytes are sampled per second, a sub sound with a length of 0.1 seconds has a size of 4800 bytes.

The right box of FIG. 5 shows an exemplary memory buffer 121 at a second time point T2 after the first sub sound SUB_S1 is generated at the first time point T1 . The processor 130 may obtain the second sub sound SUB_S2 overlapping at least a part of the first sub sound SUB_S1 from the input sound stream IN_STRM. In the example shown in FIG. 5 , the second sub sound SUB_S2 corresponds to bundling the second block B2 and the third block B3, and the second sub sound SUB_S2 and the first sub sound SUB_S1 are In the second block B2, they overlap each other. In other words, after generating the first sub sound SUB_S1 including the first block B1 and the second block B2, the first block B1 is deleted from the memory buffer 121, and the second block (B2) may be stored in the memory buffer 121 for the subsequent generation of the second sub sound SUB_S2.

In an example, the processor 130 may acquire the second sub sound from a point where the first sub sound ends in the stream of the input sound stored in the memory buffer 121 .

In other words, the processor 130 may acquire the second sub sound SUB_S2 following the first sub sound SUB_S1 so that there is no overlapping between the first sub sound SUB_S1 and the second sub sound SUB_S2 .

For example, referring to FIG. 5 , in the processor 130 , the first sub sound SUB_S1 includes a first block B1 and a second block B2, and the second sub sound SUB_S2 includes a third block It may be generated to include (B3) and a fourth block (B4).

Returning to FIG. 4 again, the accident detection device 100 acquires a series of sub sounds in step S320 and performs the next step S330.

In step S330 , the processor 130 may generate a series of input data for executing the thought sound learning model based on the series of sub sounds obtained in step S320 .

The thought sound learning model according to the embodiment may be implemented through a deep learning algorithm based on DNN or RNN. Hereinafter, artificial intelligence will be briefly described.

Artificial intelligence (AI) is an artificial intelligence that can think and make decisions on its own or a technology to make it. It refers to a field of intelligence.

An artificial neural network (ANN) is a model used in machine learning, and may be composed of artificial neurons (nodes) that form a network by combining synapses. The artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. An artificial neural network may be defined by a connection pattern between neurons of different layers, a learning process that updates model parameters, and an activation function that generates an output value. In the artificial neural network, each neuron may output a function value of an activation function for input signals input through a synapse, a weight, a bias, and the like. In artificial neural networks, learning is to determine the optimal model parameters that minimize the loss function. Machine learning implemented as a deep neural network (DNN) including a plurality of hidden layers among artificial neural networks is called deep learning.

Step S330 is a step of converting a current sub sound among a series of sub sounds into a set of frequency component values for a predetermined frequency band, and generating input data corresponding to the current sub sound based on the set of frequency component values. may include the step of Detailed steps of step S330 will be described later in detail with reference to FIGS. 6 and 10 according to an embodiment of the thought sound learning model.

Subsequently, in step S340 , the processor 130 may determine the detection result of whether the input sound includes an accident sound according to the execution result of the accident sound learning model based on a series of input data.

In step S340, the processor 130 executes the thought sound learning model based on the series of input data generated in step S330. For example, a thought sound learning model is a learning model implemented through a deep learning algorithm based on DNN or RNN.

Subsequently, the processor 130 determines a detection result of whether the input sound includes an accident sound according to the execution result of the accident sound learning model in step S330 . For example, the detection result may include whether the input sound includes an accident sound, the type of the accident sound, an accident probability, and the strength of the detection result.

Additionally, the accident detection method according to the embodiment may further include generating a command for controlling a camera installed in the vicinity of a remote location where the accident occurred based on the detection result of step S340 .

Additionally, the accident detection method according to the embodiment may further include generating a notification message for accident information based on the detection result of step S340 .

6 is a flowchart of a process of generating input data according to an exemplary embodiment.

6 shows a process of generating input data for a thought sound learning model based on DNN.

Referring to FIG. 4 , step S330 includes converting a current sub sound among a series of sub sounds into a set of frequency component values for a predetermined frequency band (step S331) and a current sub sound based on the set of frequency component values. and generating input data corresponding to the sub sound (steps S332a to S334a).

In step S331, the processor 130 may convert the current sub sound from among the series of sub sounds obtained in step S320 with reference to FIG. 4 into a set of frequency component values for a predetermined frequency band.

In an example, the processor 130 may convert a current sub-sound among a series of sub-sounds into a set of frequency component values for a predetermined frequency band. For example, the processor 130 may convert a current sub sound from among a series of sub sounds into a set of frequency component values for a predetermined frequency band through a fast Fourier transform (FFT).

In one example, the processor 130 may divide the frequency band generated as a result of the conversion into predetermined sections and store the divided frequency bands in an array. The processor 130 may store the generated array as a set of frequency component values. For example, a frequency band generated as a result of the fast Fourier transform may be in the range of about 300 Hz to 7 kHz. For example, the predetermined interval is about 10 Hz, and about 650 to 700 arrays may be generated.

Subsequently, the processor 130 may generate input data corresponding to the current sub sound based on the set of frequency component values obtained in step S331a. This includes steps S332a to S334a.

In step S332a, the processor 130 sets the first set of classification values according to the difference between the first arrangement for the set of frequency component values obtained in step S331a and the second arrangement for the frequency component values of the base sound. to decide Here, the base sound means a sound including at least one sub-sound that is older than the current sub-sound among the series of sub-sounds obtained in step S320.

In one example, the first set of classification values includes each of the second array for frequency component values of the base sound (eg 2 seconds long) and the first array for frequency component values of the current sound (eg 1 second). It is an array consisting of 0 or 1 determined to have a value of 1 when the result of determining whether the difference between the corresponding elements is a predetermined constant multiple or more is true and 0 when false. For example, the predetermined constant may be greater than 1. For example, the predetermined constant may be 1.7, but is not limited thereto. According to the accumulated results of the accident detection method of the accident detection apparatus 100 according to the embodiment, the above-described predetermined constant can be adjusted to an appropriate value.

In step S333a, the processor 130 determines the second set of classification values by normalizing the changes in each frequency component value of the first array and the second array.

In an example, the second set of classification values may be determined based on a change in frequency component values between each corresponding element of the first arrangement and the second arrangement.

For example, the second set of classification values is an array in which a value obtained by dividing a difference in frequency component values between respective corresponding elements of the above-described first and second arrays by a difference in maximum frequency component values is stored as the value of each element to be. In other words, each element of the second set of classification values is determined as a value obtained by normalizing a difference in a frequency component value between each corresponding element of the first arrangement and the second arrangement by a difference in the maximum frequency component value. Here, the difference in the maximum frequency component value means the largest value among the differences in frequency component values between the corresponding elements of the first arrangement and the second arrangement. In an example, since the second set of classification values is obtained as a result of normalization, it may have a value between 0 and 1.

In step S334a, the processor 130 receives an input corresponding to the current sub sound based on at least one of the first set of classification values obtained in step S332a and the second set of classification values obtained in step S333a. create data For example, the processor 130 may generate one array by integrating the array corresponding to the first set of classification values and the array corresponding to the second set of classification values, and use the generated one array as input data. have. For example, the processor 130 may generate one array by linking the array corresponding to the first set of classification values and the array corresponding to the second set of classification values, and use the generated one array as input data. have. For example, the processor 130 may use the first set of classification values as input data. For example, the processor 130 may use the second set of classification values as input data.

7 is a graph exemplarily illustrating classification for generating input data according to an embodiment. In the graphs 710 and 720 , the horizontal axis indicates time (msec) and the vertical axis indicates frequency (Hz).

The graph 710 exemplarily shows the first set of classification values generated in step S332a with reference to FIG. 6 .

The graph 720 exemplarily shows the second set of classification values generated in step S333a with reference to FIG. 6 . It can be seen that the darker the color of the graph 720, the greater the change in the frequency band is, for example, the greater the frequency change between the base sound and the current sub sound in a band around 150 Hz.

8 is a diagram exemplarily illustrating a thought sound learning model according to an embodiment.

8 shows an exemplary thinking sound learning model based on DNN. The thinking sound learning model is stored in the model storage unit 123 .

As input data generated in step S330 of FIG. 4 , specifically, input data F generated through steps S331a to S334a of FIG. 6 corresponds to a one-dimensional vector as an array. For example, the input data F is a 1340x1 vector.

The processor 130 inputs the input data F to the accident sound learning model shown in FIG. 8 as input values, and derives the label values (whether or not an accident and the type of the accident sound), data strength, and probability as output values. For example, label values can include background sound (b0), knocking sound (b1), horn sound (b2), ambulance siren (b3), airbreak sound (b4), crash sound (n1), and skid sound (n2). have.

The processor 130 calculates a weight and an activation function (eg, Sigmoid, Relu, etc.) of nodes along each line of the thought sound learning model shown in FIG. 8 , and finally softmax() Get the output value through the function. The processor 130 predicts what kind of acoustic data the input sound is based on the obtained output value.

9 is a diagram for explaining a process of determining whether an accident has occurred, according to an exemplary embodiment.

It is possible to obtain a series of execution results of the thought sound learning model for a series of sub sounds. Referring to FIG. 9 , a process of determining whether an accident occurs based on a pattern of a series of execution results for the thought sound learning model will be described.

In one example, the processor 130 may obtain a plurality of execution results of the accident sound learning model, and determine a detection result of whether the input sound includes an accident sound based on the plurality of execution results.

Referring to FIG. 9 , for example, a series of execution results of the thinking sound learning model for a series of sub sounds are b, n, b, b, n ?? Assume b. Here, b denotes a case in which the detection result is a normal sound rather than an accidental sound, and n denotes a case in which the detection result is an accidental sound.

The processor 130 may additionally perform a process of determining whether an accident has occurred by using a window that binds a plurality of execution results.

For example, let's say the window size is 3. In the exemplary execution result pattern of FIG. 9 , the window W_Ti at the i-th time point Ti includes three execution results b, n, and b, and the window W_Tj at the j-th time point Tj is Assume that it contains three execution results (n, b, n).

In one example, the processor 130 may determine whether an accident occurs according to the number of times of an accident sound among execution results included in the window (ie, when the execution result is n). For example, since the number of times of an accident sound among execution results included in the window W_Ti having a size of 3 is less than two times, the processor 130 may determine that an accident has not occurred. For example, since the number of times of an accident sound among execution results included in the window W_Tj having a size of 3 is two or more, the processor 130 may determine that an accident has occurred.

In one example, the processor 130 may determine whether an accident occurs according to a ratio of an accident sound among execution results included in the window. In one example, the processor 130 may determine whether or not an accident occurs according to a pattern of an accident sound in the execution result included in the window (when the detection result is n). To this end, the processor 130 may use a function Pattern (W_Tx) that analyzes a pattern of a series of execution results according to a window (W_Tx) at an arbitrary time point (Tx).

In this way, by acquiring a plurality of execution results of the accident sound learning model and determining the detection result of whether the input sound includes an accident sound based on the plurality of execution results, it is possible to analyze the thought sound reflecting the continuity of time becomes That is, the accident detection method by the accident detection device according to the embodiment divides the input sound into a series of sub sounds, and the final result is based on the accident sound analysis results for each sub sound as well as the accident sound analysis results between successive sub sounds. Accuracy is improved by deriving accident detection results.

10 is a flowchart of a process of generating input data according to an exemplary embodiment.

10 shows a process of generating input data for a thought sound learning model based on a recurrent neural network (RNN).

RNN has a structure in which the previous input value affects the next input value when the input value has a sequential property such as time. A thought sound learning model based on RNN can determine whether an accident occurred by considering the state of the previous input value through a cyclic layer.

Referring to FIG. 4 , step S330 includes converting a current sub sound among a series of sub sounds into a set of frequency component values for a predetermined frequency band ( S331 ) and the current sub sound based on the set of frequency component values. and generating input data corresponding to the sound (steps S332b and S333b).

In step S331, the processor 130 may convert the current sub sound from among the series of sub sounds obtained in step S320 with reference to FIG. 4 into a set of frequency component values for a predetermined frequency band. In an example, the processor 130 may perform fast Fourier transform to obtain a set of frequency component values. For example, the processor 130 converts the sub sound in the time domain into a value in the frequency domain through fast Fourier transform.

In step S332b, the processor 130 generates a matrix of feature values for a set of frequency component values converted in step S331. In an example, the processor S332b may extract a matrix of valid feature values for a set of frequency component values using a Mel filter bank, but is not limited thereto, and various other feature extraction methods may be used. can In one example, a matrix of feature values of 150 x 100 may be generated for a sub sound having a length of 10 seconds.

In step S333b, the processor 130 generates input data corresponding to the current sub sound based on a plurality of sub-matrices generated by dividing the matrix of feature values generated in step S332b according to a predetermined time interval. . It will be looked at below with reference to the example of FIG. 11 .

11 is a diagram exemplarily illustrating matrix data divided to generate input data according to an embodiment.

As described above, in step S333b, the processor 130 divides the matrix of feature values generated in step S332b according to a predetermined time interval and generates input data corresponding to the current sub-sound based on a plurality of sub-matrices. create

FIG. 11 exemplarily shows five sub-matrices (x1 to x5) generated by dividing a matrix of feature values of 150×100 for a 10-second sub sound into 2-second sections. In another example, it can be divided into sections longer or shorter than 2 seconds, and the length of a predetermined time section can be adjusted according to the accumulated accident detection results.

12 is a diagram exemplarily illustrating a thought sound learning model according to an embodiment.

12 shows an exemplary thinking sound learning model based on RNN. The thinking sound learning model is stored in the model storage unit 123 .

A thought sound learning model based on a recurrent neural network (RNN) is a hidden state calculated by multiplying an input value by weight and adding a bias, and a hyperbolic tangent (hyperbolic tangent) as an activation function to a hidden state calculated by multiplying the previous hidden state by weight and adding a weight. tangent) becomes the next hidden state, so temporal dependence can be considered.

The hidden state at an arbitrary time point t can be expressed by Equation (1).

Unlike DNN and CNN, which use ReLU as an activation function, RNN uses a hyperbolic tangent function. At this time, the expression of the hyperbolic tangent function is as Equation (2).

12 exemplarily shows a thinking sound learning model in which such a process is schematized for all input values. The processor 130 may determine the accident detection result based on the final output value y15 of the accident sound learning model illustrated in FIG. 12 .

In the above, the operation of the accident analysis unit 100a of the accident detection device 100 according to the embodiment has been reviewed. The accident processing unit 100b determines and performs a response operation according to the detection result of the accident analysis unit 100a. It will be looked at in more detail below.

13 is a diagram illustrating an operation process of an accident processing unit according to an exemplary embodiment.

The accident processing unit 100b may determine a response action based on the accident detection result determined by the accident analysis unit 100a and the input sound collected by the sound collector 10 . Here, the input sound corresponds to the sound received by the accident analysis unit 100a from the sound collector 10 and stored in the storage unit 120 and/or the database 122 .

As a result of the accident detection of the accident analysis unit 100a, when it is determined that an accident has occurred, the operator of the operating device 200 needs to check the situation of the actual accident location through CCTV. The processor 130 may be configured to generate a notification message for accident information based on the accident detection result of the accident analysis unit 100a in order to inform the operator of the operation device 200 of the accident detection result. The generated notification message may be transmitted to the operating device 200 through the communication unit 110 .

On the other hand, the accident processing unit 100b may estimate the location of the accident and control the CCTV disposed in the vicinity of the accident location to automatically show the accident scene screen to the operator of the operating device 200 .

That is, with reference to FIG. 13, the accident processing unit 100b estimates the accident direction estimation and the accident distance, estimates the accident location based on the estimated accident direction and distance, searches the CCTV location near the accident location, and uses the CCTV. can be controlled A series of operations of the accident processing unit 100b as described above are actually performed through the processor 130, and as a result of the execution, the processor 130 generates a control command for controlling the CCTV near the accident location.

1) Estimation of accident direction

In general, when a sound source generated from a sound source arrives at a stereo microphone, a time difference occurs due to the baseline interval of the microphone. By measuring such a time difference (TDA, Time Difference of Arrival), the direction of the sound source can be calculated.

The processor 130 may calculate the TDA of the input sound (ie, stereo sound) collected by the stereo microphone of the sound collector 10 through a cross-correlation operation.

To this end, the processor 130 converts the left sound signal and the right sound signal included in the input sound in the time domain into the frequency domain (eg, performs FFT), and calculates cross-correlation between the two frequency components. The processor 130 converts the cross-correlation result back into the time domain, and determines the time to the maximum value at this time as the TDA of the left and right sound signals included in the input sound.

For example, if the TDA is 0, the sound is generated in a direction perpendicular to the microphone, and if it is different by the baseline interval, it is a sound generated at +90 degrees or -90 degrees. .

where K is determined by the baseline and the velocity of the sound wave.

That is, when the TDA is 0, the processor 130 determines the direction perpendicular to the microphone as the thinking direction, and when the TDA is the difference by the baseline interval, +90 degrees or -90 degrees can be determined as the thinking direction. .

2) Estimation of accident distance

For example, when an accident sound is collected by the two sound collectors 10 , the magnitude (S) of the maximum value of the signal of the accident sound is inversely proportional to the square of the distance from the sound source.

이고 n+1번째 사운드 수집기(10)에서 측정한 사고음 신호의 최대값이

일 때, 두 수집기 사이의 거리가 L이라면 다음 수학식 5에 의해서 프로세서(130)는 Sn에서 Sn+1로 향하는 사고의 위치를 추정할 수 있다.Therefore, the distance between the two sound collectors 10 can be known by the ratio of the interval between the sound collectors 10 and the amplitude of the collected accidental sounds. The maximum value of the accident sound signal measured by the nth sound collector 10 is

and the maximum value of the accident sound signal measured by the n+1th sound collector 10 is

When , if the distance between the two collectors is L, the processor 130 may estimate the location of the accident from Sn to Sn+1 by Equation 5 below.

3) Estimation of accident location

The processor 130 may estimate the accident location by estimating the distance and direction from two or more accident sounds and using the accident location of each accident sound as a weighted average.

4) CCTV location search

When the accident location is determined, the accident processing unit 100b selects an appropriate CCTV to view the accident location. In general, several CCTVs are installed on the road. Therefore, the accident processing unit 100b makes the installation location and operating range of the CCTV as data in advance, searches for the location of the CCTV where the accident location can enter within the operating range, and selects the CCTV that can show the accident location.

5) CCTV Control

If the CCTV to be controlled is selected, the processor 130 generates a control command for controlling the selected CCTV. For example, the control command includes a command to read the setting table of the CCTV corresponding to the accident location and control to move to the location. For example, the control command includes a command for controlling to rotate the field of view of the CCTV according to the calculated rotation angle so that the accident location comes to the center of the field of view of the selected CCTV.

14 is a diagram illustrating a data flow of an accident detection system according to an embodiment.

As described above, the accident detection system includes a sound collector 10 and an accident detection device 100 .

The sound collector 10 is disposed on the road and collects sounds generated in the surroundings. The sound collector 10 may include a plurality of sound collectors.

The accident detection device 100 includes an accident analysis unit 100a and an accident processing unit 100b with reference to FIG. 2 . The accident detection device 100 performs an operation for accident analysis and processing by interconnection between various processes executed by the processor 130 described above with reference to FIG. 3 . 14 illustrates the processes driven in the accident detection device 100 and the data flow therebetween. It will be looked at in more detail below.

The sound collector 10 transmits the collected input sound to the accident detection device 100 .

The accident detection device 100 receives an input sound from the sound collector 10 through the communication unit 110 under the control of the processor 130 .

The received input sound is transferred from the CADS process to the MsgMan process under the control of the processor 130 in step 1400 . For example, the raw data of the input sound is passed from the CADS process to the MsgMan process.

The MsgMan process passes data about the input sound to each process that needs data about the input sound for data processing (eg DASAVE process, streamGW process, ADSS process, etc.).

In step 1401 , the DBSAVE process divides the input sound into predetermined time units under the control of the processor 130 and stores it as a sound file of raw data in the storage unit 120 . Also, in step 1401 , the DBSAVE process divides the input sound into predetermined time units under the control of the processor 130 and stores it in the database 122 through an indexing process. For example, the predetermined time unit may be 1 minute, but is not limited thereto.

In step 1406 , the StreamGW process may transmit the collected input sound to the external server WEB through the communication unit 110 in real time under the control of the processor 130 . The external server WEB may transmit the received input sound to the operating device 200 in real time.

In step 1402 , the ADSS process analyzes the received sound input under the control of the processor 130 to detect an accident, and transmits the detection result to the GWIF process to be described later and stores it in the database 122 .

The GWIF process is responsible for all interfaces of transaction and control information for the sound collector 10 and/or the accident detection device 100 input from an external server under the control of the processor 130 . Here, the external server includes a server that provides operation and control services to the operating device 200 .

In step 1405 , the GWIF process transmits the accident detection result and the accident event notification message and to the external server WEB through the communication unit 110 under the control of the processor 130 .

In step 1403, the GWIF process transmits the accident event confirmation message and/or the accident event misinformation message input by the operator from the operating device 200 under the control of the processor 130 through the external server (WEB) through the communication unit 110 received, and the received message is stored in the database 122 . Step 1403 corresponds to the response to step 1405 .

In operation 1404 , the GWIF process may store control-related setting information, state information, and control history information of the sound collector 10 in the database 122 under the control of the processor 130 .

In summary, when the input sound received from the sound collector 10 is delivered to each process running in the accident detection device 100 , each process stores the received input sound in the storage unit 120 and, if necessary, the operating device It is possible to perform real-time streaming to 200 and perform the function of accident detection as to whether the input sound includes an accident sound. In addition, the accident-detected input sound is stored in the database 122 , and the management server may extract the stored input sound from the database 122 and transmit it to the operating device 200 for the operator.

15 is a table exemplarily showing an accident detection result of an accident detection system according to an embodiment.

15 shows the prediction results of the accident detection method performed by the accident detection device 100 according to the embodiment when all 69 30-second accident (skid or collision) sounds are input to the sound collector 10 as input values. . Of the total 69 accident sounds, 63 were detected as accident sounds, and the detection rate reached 91.30%. There were 6 undetected cases, but the false detection rate was confirmed to be 0.00%.

16 is a table exemplarily showing an accident erroneous detection result of the accident detection system according to an embodiment.

16 shows a prediction result of the accident detection device 100 performing the accident detection method according to the embodiment when acoustic data collected for 24 hours on a specific date (July 20, 2019) is input as an input value. The skid sound was generated twice, and the first skid (n1) was detected correctly, but the second skid (n2) was incorrectly detected.

The above-described embodiment according to the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium includes a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and a ROM. , RAM, flash memory, and the like, and hardware devices specially configured to store and execute program instructions.

Meanwhile, the above-described computer program may be specially designed and configured for the present invention, or may be known and used by those skilled in the art of computer software. Examples of the computer program may include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

In the specification of the present invention (especially in the claims), the use of the term "the" and similar referential terms may be used in both the singular and the plural. In addition, when a range is described in the present invention, each individual value constituting the range is described in the detailed description of the invention as including the invention to which individual values belonging to the range are applied (unless there is a description to the contrary). same as

The steps constituting the method according to the present invention may be performed in an appropriate order unless the order is explicitly stated or there is no description to the contrary. The present invention is not necessarily limited to the order in which the steps are described. The use of all examples or exemplary terms (eg, etc.) in the present invention is merely for the purpose of describing the present invention in detail, and unless limited by the claims, the scope of the present invention is limited by the examples or exemplary terminology. it is not In addition, those skilled in the art will appreciate that various modifications, combinations, and changes may be made in accordance with design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the spirit of the present invention is not limited to the scope of the scope of the present invention. will be said to belong to

In the foregoing, specific embodiments of the present invention have been described and illustrated, but the present invention is not limited to the described embodiments, and those of ordinary skill in the art may make various changes to other specific embodiments without departing from the spirit and scope of the present invention. It will be appreciated that modifications and variations are possible. Accordingly, the scope of the present invention should not be defined by the described embodiments, but should be defined by the technical idea described in the claims.

10: Sound Collector
100: accident detection device
200: operating device

Claims (18)

A device for detecting an accident occurring at a remote location,
a memory buffer for storing a stream of input sound sensed through a sound collector disposed in the vicinity of the remote location; and
at least one processor;
The processor is
obtain a series of sub sounds from the stream of input sounds,
generating a series of input data for executing a thought sound learning model based on the series of sub sounds,
configured to determine a detection result as to whether the input sound includes an accident sound according to an execution result of the accident sound learning model based on the series of input data,
The processor, to generate input data corresponding to the current sound,
Converting a current sub sound among the series of sub sounds into a set of frequency component values for a predetermined frequency band, and a first arrangement for the set of frequency component values and a second arrangement for the frequency component values of the base sound A first set of classification values is determined according to a difference between and generate input data corresponding to the current sub-sound based on the second set of classification values, wherein the base sound includes at least one sub-sound prior to the current sub-sound among the series of sub-sounds. do,
The processor is configured to: a true value (TRUE) or a false value ( FALSE) as each classification value of the first set,
The processor is configured to determine the detection result based on a ratio of an accidental sound in a series of execution results of the accidental sound learning model to determine the detection result,
accident detection device.
The method of claim 1,
The processor is
and dividing the stream of input sound according to a predetermined length of time to obtain the series of sub-sounds;
accident detection device.
The method of claim 1,
the series of sub sounds includes a first sub sound and a second sub sound subsequent to the first sub sound,
The processor is
further configured to obtain, from the stream of input sound, the second sub-sound overlapping at least a portion of the first sub-sound,
accident detection device
The method of claim 1,
the series of sub sounds includes a first sub sound and a second sub sound subsequent to the first sub sound,
The processor is
further configured to obtain the second sub sound from a point where the first sub sound ends in the stream of the input sound,
accident detection device
delete delete delete delete The method of claim 1,
The thought sound learning model is implemented using a deep learning algorithm based on DNN (Deep Neural Network),
accident detection device.
The method of claim 1,
The processor is
Further configured to generate a command for controlling a camera installed in the vicinity of the remote location based on the detection result,
accident detection device.
The method of claim 1,
The processor is
Further configured to generate a notification message for accident information based on the detection result,
accident detection device.
A method for detecting an accident occurring at a remote location, the method comprising:
acquiring a stream of the sensed input sound through a sound collector disposed in the vicinity of the remote location;
obtaining a series of sub sounds from the stream of input sounds;
generating a series of input data for executing a thought sound learning model based on the series of sub sounds; and
determining a detection result as to whether the input sound includes an accident sound according to an execution result of the accident sound learning model based on the series of input data;
The step of generating the series of input data includes:
converting a current sub sound among the series of sub sounds into a set of frequency component values for a predetermined frequency band; and
generating input data corresponding to the current sub sound based on the set of frequency component values;
The step of generating input data corresponding to the current sub sound includes:
determining a first set of classification values according to a difference between a first arrangement of the set of frequency component values and a second arrangement of frequency component values of a base sound;
determining a second set of classification values by normalizing changes in values of respective frequency components of the first array and the second array; and
generating the input data based on the first set of classification values and the second set of classification values;
the base sound includes at least one sub sound prior to the current sub sound among the series of sub sounds;
Determining the first set of classification values comprises:
Determining a true value (TRUE) or a false value (FALSE) as each classification value of the first set according to whether a difference between each corresponding frequency component value of the first array and the second array is a predetermined constant multiple or more including,
The step of determining the detection result is,
Comprising the step of determining the detection result based on the ratio of the accident sound in a series of execution results of the accident sound learning model,
How to detect an accident.
13. The method of claim 12,
The remote location is located inside the tunnel,
How to detect an accident.
delete delete delete 13. The method of claim 12,
Generating a command for controlling a camera installed in the vicinity of the remote location based on the detection result
further comprising,
How to detect an accident.
13. The method of claim 12,
generating a notification message for accident information based on the detection result
further comprising,
How to detect an accident.

Images