KR20230070601A - Sound-based intelligent emergency analysis system and method thereof - Google Patents

Abstract

The present invention relates to a sound-based intelligent emergency analysis system capable of specifically classifying current violent or emergency situations, and a method thereof. According to the present invention, the sound-based intelligent emergency analysis system comprises: an emergency bell device installed in each high-crime area to collect audio data including sounds or voices generated within the high-crime area to detect an event for an emergency situation from the collected audio data and generate an emergency alarm operation signal; and an analysis server receiving the audio data from the emergency bell device when the emergency bell operation signal is received, extracting the voice data from the audio data, analyzing the voice data to classify a conversational voice situation, analyzing the remaining audio data excluding the voice data from the audio data to classify an audio situation, and integrating the classified conversational voice situations and audio situations to determine crime or emergency situations classified into security levels by stage according to preset classification criteria to provide a situation classification result. The analysis server collects conversational voice-related texts corresponding to the crime or emergency situation as learning data, uses the learning data to train an artificial intelligence-based voice analysis model in advance, and extracts the voice data and classifies the conversational voice situations on the basis of the trained voice analysis model. The server collects audio-related sounds corresponding to crime or emergency situations as learning data, uses the learning data to train an artificial intelligence-based audio analysis model, and classifies the audio situation on the basis of the trained audio analysis model.

Description

Sound-based intelligent emergency analysis system and method thereof

The present invention relates to a sound-based intelligent emergency situation analysis system capable of classifying a current violence or emergency situation by comprehensively analyzing voice and acoustic signals.

The information described in this section merely provides background information on an embodiment of the present invention and does not constitute prior art.

In general, crime prevention systems are installed in areas with weak security to report and respond to emergencies such as violence and emergencies. Among the crime prevention systems, the emergency bell for crime prevention is installed in a specific area such as a high-crime area. It is a device.

Surveillance cameras installed like these emergency bells for crime prevention are installed in the upper area on one side of the high-crime area, so that when a dangerous situation occurs, the manager checks the recorded video to help, or to find the criminal later. performs the function of recording. Here, a surveillance camera is generally a Closed Circuit Television (CCTV), but a high-performance camera is also used.

Recently, crimes such as assault, robbery, sexual harassment, and murder frequently occur in spaces (eg, indoor public spaces) where exposure to the outside is blocked while being accessible to the public, such as a bathroom. As a result, the anxiety of users using indoor public spaces is gradually increasing. In particular, since women have lower physical abilities than men, they have greater anxiety and burden about using indoor public spaces.

Accordingly, various studies are being conducted on emergency alarm devices for preventing and simultaneously coping with emergency situations in indoor public spaces. Emergency bells for crime prevention are installed on site due to their simple installation and convenient operation. Since the emergency bell can be pressed only through, it is difficult for a party in an actual emergency to press the emergency bell in the criminal's field of view, and the operation of the emergency bell can be forcibly stopped, making it impossible to respond quickly to an emergency situation.

Due to this problem, a sound-based security technology has been studied to detect an emergency situation by comparing the decibel level of the acoustic signal collected through a microphone with a threshold value. There is a problem of low reliability due to high errors.

Recently, emergency bell devices installed in indoor public places use both a button-type emergency bell and an emergency bell with a sound recognition module. Two to three times a day, security personnel (including the police in charge) are dispatched to the place where the emergency bell device is installed, resulting in a waste of manpower.

In fact, 85.6% of mobilizations caused by emergency bell devices were caused by drunkenness or noise, 13.7% of mobilizations were due to pranks or mistakes, and 99.3% were unnecessary mobilizations in non-criminal situations, not actual crimes. is becoming Due to this, the crime prevention person who is dispatched to the place where the emergency bell device operates is mainly trying to determine the authenticity of whether a crime has occurred rather than coping with a crime situation.

As such, in the prior art, when the emergency bell device operates, first responders are first dispatched to determine the authenticity of a crime, and when an actual crime occurs, response personnel are re-mobilized to deal with the crime. There is a problem that not only occurs, but also makes it difficult to quickly cope with crime.

In order to solve the above problems, the present invention comprehensively analyzes voice and acoustic signals generated in the field according to an embodiment of the present invention to provide a sound-based intelligent emergency situation that can specifically classify the current violence or whip-level situation. It is an object to provide an analysis system and method therefor.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the sound-based intelligent emergency situation analysis system according to an embodiment of the present invention is installed in each high-crime area, and audio data including sound or voice generated in the high-crime area An emergency bell device for generating an emergency bell operation signal by detecting an emergency event in the collected audio data by collecting; and when the emergency bell operating signal is received, audio data is received from the emergency bell device, voice data is extracted from the audio data, voice data is analyzed, and conversational voice situations are classified, and the remaining audio data excluding voice data is classified. An analysis server that analyzes sound data, classifies sound situations, integrates the classified conversational voice situations and sound situations, determines crime or emergency situations classified into security levels for each stage according to preset classification standards, and provides situation classification results. Including, but the analysis server collects texts related to conversational voice corresponding to the crime or emergency situation as learning data, learns an artificial intelligence-based voice analysis model in advance using the learning data, and then learns the learned voice. Based on the analysis model, extracting voice data and classifying conversational voice situations, collecting sound-related sounds corresponding to the crime or emergency situation as learning data, and learning an artificial intelligence-based acoustic analysis model using the learning data Then, the acoustic situation is classified based on the learned acoustic analysis model.

According to one aspect of the present invention, the sound-based intelligent emergency situation analysis system, when the situation classification result is received, determines the security level based on the situation classification result to the security terminal in charge of the high-crime area where the emergency bell operation signal is generated. Accordingly, it further includes a control server providing on-site dispatch information or situation response information.

The analysis server may include a communication unit for receiving audio data of the site from the emergency bell device and performing transmission and reception with an external device; a voice extraction unit extracting voice data from the audio data; a conversational speech situation analysis unit that converts the extracted speech data into Korean-based conversational speech text and analyzes a conversational speech situation based on the conversational speech text; an acoustic situation analyzer configured to analyze an acoustic situation by analyzing the remaining audio data excluding voice data from the audio data; and by integrating the conversational voice situation analyzed by the conversational voice situation analysis unit and the acoustic situation analyzed by the acoustic situation analysis unit to determine crimes or emergency situations classified by step-by-step crime codes according to preset classification criteria and provide situation classification results. It includes a situation judgment unit that does.

The voice extraction unit determines whether or not voice is generated for each language type in the audio data and provides language type information to the conversational voice situation analysis unit, and the conversational voice situation analysis unit converts voice data extracted based on the language type information into text data. It converts and translates the converted text data into Korean-based dialogue voice text.

The voice analysis model is formed as a multi-layer neural network structure including deep neural networks (DNNs) for a voice extraction unit, a dialogue voice situation analysis unit, an acoustic situation analysis unit, and a situation determination unit.

According to the sound-based intelligent emergency situation analysis method according to an embodiment of the present invention, in the sound-based intelligent emergency situation analysis method performed by an analysis server that analyzes an emergency situation in conjunction with a sound-based emergency bell device, a) when an emergency bell operating signal is detected from an emergency bell device installed in a pre-set high crime area, receiving audio data from the site generated within the high crime area; b) extracting voice data from the audio data and analyzing a dialogue voice situation based on the extracted voice data; c) extracting sound data from the audio data and analyzing a sound situation based on the extracted sound data; and d) integrating the dialogue voice situation and the acoustic situation to determine a crime or emergency situation classified into security levels by stages according to a preset classification criterion, and providing a situation classification result.

The sound-based intelligent emergency situation analysis method further comprises: e) providing on-site dispatch information or situation response information based on the situation classification result to a security terminal having jurisdiction over a high crime area where the emergency bell operation signal is generated will be.

In the step b), texts related to conversational voice corresponding to the crime or emergency situation are collected as learning data, an artificial intelligence-based voice analysis model is pre-learned using the learning data, and then the learned voice analysis model is used. Based on this, it is to extract voice data and classify conversational voice situations.

The voice analysis model determines whether a voice is generated for each language type in the audio data, converts the extracted voice data into text data based on the determined language type information, and converts the converted text data into Korean-based dialogue voice text. It is to classify conversational voice situations by translating them.

The speech analysis model includes deep neural networks (DNNs) for determining whether consonants are generated by language type, extracting speech data, converting speech data to text data, translating Korean-based dialogue speech text, and classifying dialogue speech situations. It is formed as a multi-layer neural network structure.

In the step c), after collecting sound-related sounds corresponding to the crime or emergency as learning data, and learning an artificial intelligence-based sound analysis model using the learning data, the acoustic sound analysis model is based on the learned sound analysis model. to classify the situation.

According to the above-described problem solving means of the present invention, the present invention detects voice data from audio data in the field and recognizes a dialogue voice situation, thereby enabling initial recognition of violence or an emergency situation, as well as sound detected from audio data in the field. After analyzing the data and classifying the sound situation, it is possible to classify the crime level for violence or emergency situation by step by integrating the dialogue voice situation and the sound situation, thereby reducing social distrust and deterioration of utilization of the emergency bell service. It has the effect of preventing crime and providing quality crime safety-related services.

1 is a diagram illustrating the configuration of a sound-based intelligent emergency situation analysis system according to an embodiment of the present invention.
2 is a diagram illustrating the configuration of an analysis server according to an embodiment of the present invention.
3 is a diagram for explaining crime codes classified for each crime situation according to an embodiment of the present invention.
4 is a flowchart illustrating a method for analyzing a sound-based intelligent emergency situation according to an embodiment of the present invention.
5 is a diagram illustrating a Wave U-Net structure according to an embodiment of the prior art, and FIG. 6 is a diagram illustrating a voice analysis model of a Nested Wave U-Net structure according to an embodiment of the present invention.
7 is a diagram explaining a process of analyzing a conversational voice situation and an acoustic situation in an analysis server according to an embodiment of the present invention.
8 is a flowchart illustrating a process of determining whether a voice is generated for each voice type according to an embodiment of the present invention.
9 is a flowchart illustrating a process of extracting voice data according to an embodiment of the present invention.
10 is a diagram explaining a process of learning a voice analysis model for classifying conversational voice situations according to an embodiment of the present invention.
11 is a diagram explaining a process of classifying conversational voice situations using a learned voice analysis model according to an embodiment of the present invention.
12 is a diagram explaining a process of classifying acoustic situations using a learned acoustic analysis model according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that it may further include other components, not excluding other components, unless otherwise stated, and one or more other characteristics. However, it should be understood that it does not preclude the possibility of existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

In this specification, a 'terminal' may be a wireless communication device with guaranteed portability and mobility, and may be, for example, any type of handheld-based wireless communication device such as a smart phone, a tablet PC, or a laptop computer. Also, the 'terminal' may be a wired communication device such as a PC capable of accessing other terminals or servers through a network. In addition, a network refers to a connection structure capable of exchanging information between nodes such as terminals and servers, such as a local area network (LAN), a wide area network (WAN), and the Internet (WWW : World Wide Web), wired and wireless data communications network, telephone network, and wired and wireless television communications network.

Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasonic communication, visible light communication (VLC: Visible Light Communication), LiFi, and the like, but are not limited thereto.

The following examples are detailed descriptions for better understanding of the present invention, and do not limit the scope of the present invention. Therefore, inventions of the same scope that perform the same functions as the present invention will also fall within the scope of the present invention.

In addition, each configuration, process, process or method included in each embodiment of the present invention may be shared within a range that does not contradict each other technically.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating the configuration of a sound-based intelligent emergency situation analysis system according to an embodiment of the present invention.

Referring to FIG. 1, the sound-based intelligent emergency situation analysis system according to an embodiment of the present invention includes at least one emergency bell device 100, an analysis server 200, and a control server 300, but is limited thereto. It doesn't work.

The emergency bell device 100 is installed in each high-crime area, collects audio data including sounds or voices generated in the high-crime area, detects an emergency event from the collected audio data, and generates an emergency bell operation signal. do. This emergency bell device 100 may include both a button-type emergency bell and a sound recognition emergency bell including a sound recognition module.

The emergency bell device 100 includes a microphone (not shown) for sound collection, a communication module (not shown) for transmitting an emergency bell operating signal and audio data to the analysis server 200, a memory (not shown), damage or forced It may include a warning device (not shown) generated when the power is turned off, a control module (not shown), and the like.

The emergency bell device 100 stores audio data including all sounds and voices generated in crime-prone areas (public toilets, bus stops, alleys, blind spots of buildings, etc.) in a certain time unit (approximately 10 seconds) in a buffer (not shown). When an emergency event is detected, an emergency bell operation signal is generated, and the audio data recorded for a certain period of time before the emergency bell operation signal is generated is read from the buffer and the analysis server (200 ) is sent to At this time, the emergency bell device 100 may secure a storage capacity equal to or greater than a preset capacity by deleting the audio data stored in the buffer in a first-in-first-out manner.

When the emergency bell operating signal is received from the emergency bell device 100, the analysis server 200 receives audio data from the corresponding emergency bell device 100, extracts voice data from the audio data, and analyzes the voice data to obtain a conversational voice situation. and classifies the acoustic situation by analyzing the remaining audio data except for the voice data in the audio data, and integrating the classified dialogue voice situation and the acoustic situation to classify crimes or emergency situations into step-by-step codes according to preset classification criteria. It determines and provides situation classification results. At this time, the analysis server 200 may also receive and analyze sound information recorded for a certain time before the emergency bell activation signal is generated, so that the current situation can be more accurately grasped.

When the situation classification result is received from the analysis server 200, the control server 300 provides the security terminal 400 in charge of the high-crime area where the emergency bell operation signal is generated according to the crime code based on the situation classification result, on-site dispatch information or situation. Provide coping information.

At this time, the analysis server 200 and the control server 300 may be a computer body for a server in a general sense, and may be implemented in various types of devices capable of performing other server roles. Specifically, the analysis server 200 and the control server 300 may be implemented in a computing device including a communication module (not shown), a memory (not shown), a processor (not shown), and a database (not shown), respectively. , For example, it may be implemented as a mobile phone, TV, PDA, tablet PC, PC, notebook PC, and other user terminal devices.

In addition, the security terminal 400 is a terminal capable of wireless communication in connection with the police station or other institutions to perform notification of whether a security agent is dispatched or not, a crime situation, etc. can

The emergency bell device 100 has unique identification information designated by the control server 300, and the emergency bell operating signal and the situation classification result include the identification information of the corresponding emergency bell device 100. Therefore, the analysis server 200 and the control server 300 can use the identification information of the emergency bell device 100 to check the location information of the place where the emergency bell device 100 is installed, that is, the high crime area, and the corresponding high crime area. Information can be quickly transmitted to the security terminal 400 in charge of the area.

Therefore, the analysis server 200 and the control server 300 store identification information of each emergency bell device 100 and information of the security terminal 400 in charge of each high crime area in the database 210 .

Meanwhile, the emergency bell device 100 may further include at least one or more camera devices 150 for capturing on-site images of high crime areas. For example, when the crime-prone area is a blind spot of a building such as a bus stop, an underground walkway, a roof of a building or a stairway, a camera device 150 such as a CCTV is installed on an upper part of one side of the crime-prone area such as an underground walkway, a building rooftop or stairs. After installation, the field situation may be photographed through the camera device 150 .

When the situation classification result is received, the control server 300 receives the field image in real time through the camera device 150 of the corresponding high-crime area, checks the field image based on the situation classification result, and sets the current situation to a preset security level. , and may generate on-site dispatch information or situation response information according to the classified security level. At this time, the control server 300 may change the security level at any time according to the field video received in real time.

2 is a diagram for explaining the configuration of an analysis server according to an embodiment of the present invention, and FIG. 3 is a diagram for explaining crime codes classified for each crime situation according to an embodiment of the present invention.

2 and 3, the analysis server 200 includes a communication unit 201, a voice extraction unit 202, a dialogue voice situation analysis unit 203, an acoustic situation analysis unit 204, and a situation determination unit 205. Including, but not limited to.

The communication unit 201 receives audio data from the emergency bell device 100, and communication necessary to provide transmission and reception signals between the emergency bell device 100 as well as the control server 300 and user terminals in the form of packet data in conjunction with the communication network. provide an interface. Here, the communication unit 201 may be a device including hardware and software necessary for transmitting/receiving a signal such as a control signal or a data signal through a wired/wireless connection with another network device.

The voice extraction unit 202 extracts human voice data from the audio data, and the conversational voice situation analysis unit 203 converts the extracted voice data into text, and then converts the extracted voice data into text based on the conversational voice situation (threatening situation, assault situation). , money extortion situation, etc.) is analyzed.

The acoustic situation analysis unit 204 analyzes the remaining audio data except voice data from the audio data and analyzes the acoustic situation (animal abuse, hidden camera installation, property damage, drunken person, crying child, adult crying, fierce dog barking, bee swarm, fire alarm, assault) etc.) are analyzed.

The situation determination unit 205 integrates the conversational voice situation analyzed by the conversational voice situation analysis unit 203 and the acoustic situation analyzed by the acoustic situation analysis unit 204 to classify crimes into step-by-step crime codes according to preset classification criteria. Or it determines the emergency situation and provides the situation classification result.

As shown in FIG. 3, the crime codes are divided into five security levels (code 0 to code 4), and it can be seen that the severity of the response time, number of people dispatched, and response to the situation increases from code 4 to code 0. For example, when the emergency bell device 100 is installed in a public toilet, the analysis server 200 classifies the crime code as code 0 when an emergency bell operation signal is detected and a female scream is detected in the public toilet, and the control The server 300 transmits the crime code and the situational analysis result for the crime situation (a situation in which a man enters a women's bathroom, a situation in which a victim sobs under threat, a situation in which an assault occurs, etc.). Then, the control server 300 confirms that the crime code is code 0 through the situation analysis result, and a crime prevention agent such as the police is dispatched within the shortest time, and an ambulance, female police, Field dispatch information or situation response information, such as cooperation with dispatch elements such as police personnel in an adjacent area, may be instructed.

As such, the situation determination unit 205 determines violence or emergency as code 4 when situations such as beehives, fires, and abandoned dogs occur, and judges code 3 when situations such as crying children, adults cry, and barking dogs occur, and 2 It is judged as code 2 when bullying situation occurs, code 1 when animal abuse, property damage, hidden camera installation, etc., and code 0 when money extortion, threats, bullying, violence, etc. occur.

4 is a flowchart illustrating a method for analyzing a sound-based intelligent emergency situation according to an embodiment of the present invention. 5 is a diagram illustrating a Wave U-Net structure according to an embodiment of the prior art, and FIG. 6 is a diagram illustrating a voice analysis model of a Nested Wave U-Net structure according to an embodiment of the present invention.

Referring to FIG. 4, in the sound-based intelligent emergency situation analysis method, when the analysis server 200 detects an emergency bell operation signal from the emergency bell device 100 (S11), it receives real-time on-site audio data (S12). .

The analysis server 200 extracts voice data from on-site audio data (S13), and then analyzes a dialogue voice situation based on the extracted voice data (S14). The analysis server 200 collects conversational voice-related texts corresponding to crimes or emergency situations as learning data, uses the collected learning data to learn an artificial intelligence-based voice analysis model in advance, and then uses the learned voice analysis model. Based on this, it is possible to extract voice data and classify conversational voice situations.

At this time, the voice analysis model uses the deep learning structure of U-Net, which is an end-to-end fully-convolutional network-based model, to configure the algorithm, but as the connection structure of the hidden layer is different, VGGnet, GoogLeNet, Deep learning with various structures such as DenseNet, fully convolutional network, and AlexNet can be used.

In particular, the voice analysis model uses nested wave U-Net to extract voice data and divides two audio data and voice data into audio data in which sound and voice are mixed.

As shown in FIG. 5, the conventional Wave U-Net consists of a 1D convolution module and a downsampling module in succession to perform a contracting path in which the front feature map size is reduced, and upsampling ) module, an Expanding Path that increases the size of the back feature map again is performed. At this time, many high-level feature maps are extracted from the time domain through the 1D Convolution module, down-sampled, follow a specific pattern for the time step, and time samples are ignored. This reduces the time resolution by half.

However, as shown in FIG. 6, the nested wave U-Net structure applied to the speech analysis model of the present invention consists of a contraction step, which is a downward path, and an expansion step, which is an upward path. It captures the information and proceeds with fine-grained localization in the expansion phase. At this time, unlike the existing Wave U-Net structure, the nested wave U-Net structure brings the feature maps of the contracting path corresponding to each step at each upsampling step and concatenates them. In order to fit the size, the feature map in the contraction stage is cropped (cut out) to an appropriate size according to the size of the feature map in the expansion stage, and then concat (attached) to maintain the information of the audio data.

In this way, the voice analysis model can remove noise from field audio data and extract voice data with clear sound quality by using the nested wave U-Net structure.

The analysis server 200 extracts audio data excluding voice data from audio data and analyzes an acoustic situation based on the extracted audio data (S15). The analysis server 200 collects sound-related sounds corresponding to crimes or emergencies as learning data, learns an artificial intelligence-based sound analysis model using the collected learning data, and then uses the learned sound analysis model to perform sound analysis. Situations can be categorized.

The analysis server 200 integrates the analyzed dialogue voice situation and sound situation and classifies whether the situation according to the on-site audio data corresponds to the crime code for each crime/emergency situation (S16).

Meanwhile. Each of the steps in FIG. 4 may be divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted if necessary, and the order of steps may be changed.

FIG. 7 is a diagram illustrating a process of analyzing a conversational voice situation and an acoustic situation in an analysis server according to an embodiment of the present invention, and FIG. 8 is a process of determining whether a voice is generated by voice type according to an embodiment of the present invention. 9 is a flowchart illustrating a process of extracting voice data according to an embodiment of the present invention.

As shown in FIG. 7, the analysis server 200 receives on-site audio data (eg, 10-second on-site sound) (S21), determines whether a voice is generated for each voice type, and then extracts the voice data (S21). S22, S23).

As shown in FIG. 8, the analysis server 200 includes a sound DB in which more than 50 types of sound data such as screams, footsteps, and barking dogs are stored, and voice data in which language types such as English, Korean, Chinese, and Korean-Chinese are stored. Build a DB.

Therefore, the analysis server 200 randomly selects 1 to 3 kinds of sound sample data from the sound DB (S31), randomly selects one type of sound sample data from the sound DB (S32), and then After mixing the voice sample data with each other, an artificial intelligence-based voice analysis model is learned to classify language types using language types as labels (S33 and S34). Accordingly, the analysis server 200 may check the language type in the on-site audio data using the learned voice analysis model.

As shown in FIG. 9, the analysis server 200 randomly selects 1 to 3 kinds of sound sample data (S41), randomly selects one kind of sound sample data from the voice DB (S42), and then After mixing the sample data and the voice sample data, the original voice data selected from the voice DB is used as a label to learn an artificial intelligence-based voice analysis model for voice data extraction (S43 and S44). Accordingly, the analysis server 200 may extract a voice signal using one or more types of languages from field audio data using the learned voice analysis model.

Referring back to FIG. 7 , the analysis server 200 provides voice type information to a STT (Speech To Text) voice processing engine based on Korean characteristics, and the voice processing engine converts human language into text (S24). . Here, STT (Speech To Text) means a process of converting human voice language into text data through computer interpretation as a field of voice recognition. Recently, speech recognition engines for STT are improving accuracy by using sound and language models through deep learning algorithms. In addition to HMM (Hidden Markov Model), which is a traditional speech recognition algorithm, DNN (Deep Neural Network) and RNN (Recurrent Neural Network) techniques, which are widely used as deep learning-based algorithms, are applied to show higher accuracy than in the past.

Therefore, the analysis server 200 converts voice data including at least one language type among Korean, English, Chinese, and Korean-Chinese into text, classifies a text-based conversational voice situation (S25), and is robust against a voice interference signal. After the sound situation is classified, the conversation voice situation and the sound situation are combined to classify crime codes for each violence/emergency situation (S26, S27).

10 is a diagram illustrating a process of learning a voice analysis model for classifying a conversational voice situation according to an embodiment of the present invention, and FIG. It is a diagram explaining the process of classifying situations.

Referring to FIG. 10, the analysis server 200 builds a Korean text DB for various conversation situations such as general conversation, extortion of money, intimidation, and bullying (S51), and changes the order of words in conversation situations including one or more texts, , The learning data is primarily augmented (Data Augmentation) by changing words (Korean, Chinese, English, similar words, profanity, etc.) (S52).

The analysis server 200 constructs an English text augmented DB by translating the primary augmented DB into Korean text into English, and constructs a Chinese text augmented DB by translating the primary augmented DB into Korean text into Chinese (S53).

The analysis server 200 builds a second augmented DB by integrating the first augmented DB with Korean text, the English text augmented DB, and the Chinese text augmented DB (S54), and then the second augmented DB for conversational voice situation classification. A voice analysis model is learned using the DB as learning data (S55).

As shown in FIG. 11, the analysis server 200, when field audio data is received, determines whether a voice is generated for each language type from the field audio data, extracts voice data (S61, S62, S63), and then extracts the extracted voice data. Through the STT, it is converted into text data according to the corresponding language type (S64).

If the converted text data is not in Korean, the analysis server 200 translates the converted text data into Korean text, inputs the Korean-translated conversational speech text into the learned speech analysis model, and the speech analysis model determines how the input conversational speech text is Whether or not it corresponds to the dialogue voice situation is classified and output (S65 to S67).

A dialogue voice situation is analyzed (S14). The analysis server 200 collects conversational voice-related texts corresponding to crimes or emergency situations as learning data, uses the collected learning data to learn an artificial intelligence-based voice analysis model in advance, and then uses the learned voice analysis model. Based on this, it is possible to extract voice data and classify conversational voice situations.

In this way, the voice analysis model can be implemented as a deep learning-based algorithm, which has a multi-layer neural network structure including deep neural networks (DNNs) for language type classification, voice signal extraction, STT, and conversational voice situation classification. can be formed as

12 is a diagram explaining a process of classifying acoustic situations using a learned acoustic analysis model according to an embodiment of the present invention.

As shown in FIG. 12, the analysis server 200 is a sound DB that stores sound data for various situations such as violent situations, animal abuse, property damage, crying children, crying adults, fierce dogs, swarms of bees, fire alarms, and hidden camera installation. is constructed, randomly selects one type of audio sample data from the audio DB, and randomly selects one type of audio sample data from the audio DB (S71 and S72).

The analysis server 200 mixes the acoustic sample data and the voice sample data to learn an acoustic analysis model using the acoustic situation type as a label for acoustic situation classification. Accordingly, the analysis server 200 may input the acoustic data extracted from the field audio data to the learned acoustic analysis model, and classify the acoustic situation corresponding to the field audio data through the learned acoustic analysis model.

The embodiments of the present invention described above may be implemented in the form of a recording medium including instructions executable by a computer, such as program modules executed by a computer. Such recording media includes computer readable media, which can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Computer readable media also includes computer storage media, both volatile and nonvolatile, implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. , including both removable and non-removable media.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

100: emergency bell device
150: camera device
200: analysis server
201: Ministry of Communications
202: voice extraction unit
203: dialogue voice situation analysis unit
204: acoustic situation analysis unit
205: situation judgment unit
210: database
300: control server
400: security terminal

Claims (12)

An emergency bell device installed in each high-crime area to collect audio data including sound or voice generated in the high-crime area, detect an emergency event from the collected audio data, and generate an emergency bell operation signal; and
When the emergency bell activation signal is received, audio data is received from the emergency bell device, voice data is extracted from the audio data, voice data is analyzed to classify conversational voice situations, and the rest of the audio data except voice data is classified. An analysis server that analyzes data to classify acoustic situations, integrates the classified conversational voice situations and acoustic situations, determines crime or emergency situations classified into step-by-step security levels according to preset classification standards, and provides situation classification results. include,
The analysis server,
The conversational voice-related texts corresponding to the crime or emergency situation are collected as learning data, an artificial intelligence-based voice analysis model is learned in advance using the learning data, and voice data is extracted and extracted based on the learned voice analysis model. Classify conversational voice situations,
Acoustic-related sounds corresponding to the crime or emergency situation are collected as learning data, an artificial intelligence-based acoustic analysis model is learned using the learning data, and then the acoustic situation is classified based on the learned acoustic analysis model. , sound-based intelligent emergency analysis system. According to claim 1,
Further comprising a control server for providing on-site dispatch information or situation response information according to a security level based on the situation classification result to a security terminal having jurisdiction over a high-crime area where the emergency bell operation signal is generated when the situation classification result is received An intelligent emergency situation analysis system based on human and sound. According to claim 1,
The analysis server,
A communication unit that receives audio data from the site from the emergency bell device and performs transmission and reception with an external device;
a voice extraction unit extracting voice data from the audio data;
a conversational speech situation analysis unit that converts the extracted speech data into Korean-based conversational speech text and analyzes a conversational speech situation based on the conversational speech text;
an acoustic situation analyzer configured to analyze an acoustic situation by analyzing the remaining audio data excluding voice data from the audio data; and
By integrating the conversational voice situation analyzed by the conversational voice situation analysis unit and the acoustic situation analyzed by the acoustic situation analysis unit, crimes or emergency situations classified into step-by-step crime codes are determined according to preset classification criteria, and situation classification results are provided. A sound-based intelligent emergency situation analysis system that includes a situation determination unit. According to claim 3,
The voice extraction unit determines whether or not a voice is generated for each language type in the audio data and provides language type information to the dialogue voice situation analysis unit;
The sound-based intelligent emergency situation analysis system, wherein the dialogue voice situation analysis unit converts voice data extracted based on the language type information into text data, and translates the converted text data into Korean-based dialogue voice text. According to claim 3,
The voice analysis model is formed of a multi-layer neural network structure including deep neural networks (DNNs) for a voice extraction unit, a conversational speech situation analysis unit, an acoustic situation analysis unit, and a situation determination unit. Intelligent emergency analysis system. In the sound-based intelligent emergency situation analysis method performed by an analysis server that analyzes an emergency situation in conjunction with a sound-based emergency bell device,
a) when an emergency bell operating signal is detected from an emergency bell device installed in a pre-set high crime area, receiving audio data from the site generated within the high crime area;
b) extracting voice data from the audio data and analyzing a dialogue voice situation based on the extracted voice data;
c) extracting sound data from the audio data and analyzing a sound situation based on the extracted sound data; and
d) integrating the dialogue voice situation and the sound situation to determine a crime or emergency situation classified into step-by-step security levels according to a predetermined classification standard, and providing a situation classification result, sound-based intelligent emergency situation analysis method. According to claim 6,
e) providing on-site dispatch information or situation response information to a security terminal having jurisdiction over the high crime area where the emergency bell operation signal is generated based on the situation classification result, sound-based intelligent emergency situation analysis method. According to claim 6,
In step b),
The conversational voice-related texts corresponding to the crime or emergency situation are collected as learning data, an artificial intelligence-based voice analysis model is learned in advance using the learning data, and voice data is extracted and extracted based on the learned voice analysis model. A sound-based intelligent emergency situation analysis method that classifies conversational speech situations. According to claim 8,
The voice analysis model determines whether a voice is generated for each language type in the audio data, converts the extracted voice data into text data based on the determined language type information, and converts the converted text data into Korean-based dialogue voice text. A sound-based intelligent emergency situation analysis method that classifies conversational speech situations by translating them. According to claim 9,
The speech analysis model includes deep neural networks (DNNs) for determining whether consonants are generated by language type, extracting speech data, converting speech data to text data, translating Korean-based dialogue speech text, and classifying dialogue speech situations. A method for analyzing sound-based intelligent emergencies, which is formed as a multi-layer neural network structure. According to claim 6,
In step c),
Acoustic-related sounds corresponding to the crime or emergency situation are collected as learning data, an artificial intelligence-based acoustic analysis model is learned using the learning data, and then the acoustic situation is classified based on the learned acoustic analysis model. , Sound-based intelligent emergency situation analysis method. In the analysis server for analyzing the emergency situation in conjunction with the emergency bell device in conjunction with the sound-based emergency bell device,
Audio data is received from the emergency bell device, voice data is extracted from the audio data using an artificial intelligence-based voice analysis model, the voice data is analyzed to classify conversational voice situations, and an artificial intelligence-based acoustic analysis model is used. The audio data except for the voice data is analyzed to classify the acoustic situation, and the classified conversational voice situation and the acoustic situation are integrated and the crime or emergency situation is divided into step-by-step security levels according to preset classification criteria. to provide a situation classification result by determining
The artificial intelligence-based voice analysis model is pre-learned using learning data collected from conversational voice-related texts corresponding to the crime or emergency situation,
The analysis server, characterized in that the artificial intelligence-based acoustic analysis model is pre-trained using learning data collected from acoustic sounds corresponding to the crime or emergency situation.

Images