KR20190063729A - Life protection system for social disaster using convergence technology like camera, sensor network, and directional speaker system - Google Patents

Abstract

A human life protection system using a camera, a sensor network, and a directional speaker based on fusion technology for a social disaster response allows IoT sensor platform technology for a social disaster response, including a CCTV camera, a fusion sensor, an IoT sensor network (USN, WSN, 6LoWPAN, Wi-Fi, NB-IoT, LTE, and LoRa network), and an IoT gateway sensing a social disaster by using ICT technology, to be developed; provides a mobius server, an IDS integrated control system and an IDS local control system, sensor data monitoring, artificial intelligence-based disaster situation image analysis, situation awareness, or object tracking for an image of an image control system, sensor data monitoring, and a function of detecting an event by screaming sound in a dangerous area using a microphone sensor; provides big data analysis for each disaster type and a disaster response IDS-based human life protection system, evacuation guidance system, and disaster response system; provides disaster alert broadcasting and evacuation guidance technology using a directional speaker; and provides an evacuation guidance system using a sensor network and a directional speaker system capable of variable output, and a guidance application of a smartphone using the non-audible frequency.

Description

[0001] The present invention relates to a camera, a sensor network, and a directional speaker system using directional loudspeakers,

More particularly, the present invention relates to a system and method for protecting human lives from a disaster by using a CCTV camera, a fusion compound sensor, an IoT sensor network (USN, WSN, 6LoWPAN, MQTT, IOT sensor platform technology including IOT gateway, NB-IoT, LTE, LoRa network), and IoT gateway, and developed the artificial intelligence-based disaster of mobi server, IDS integrated control system and IDS local control system, Provides event image analysis / situation tracking / object tracking of images, sensor data monitoring, event detection by screaming sound using microphone sensor in dangerous area, big data analysis of social disaster type, IDS-based IDS- Provides disaster alert broadcasting and evacuation induction technology using directional loudspeakers, and supports sensor network and directional The present invention relates to an evacuation guidance system using a speaker system, a camera based on a fusion technology for social disaster, which provides an induction app utilizing the non-audible frequency of a smart phone, a sensor network, and a life guarding system using directional speakers.

As the frequency and magnitude of social disasters increase, the economic damage caused by indirect damage has increased not only directly but also the economic cost of the state or private sector has been rapidly increasing.

In addition to the complexity of the occurrence of disasters, uncertainty factors such as climate change are increasing, so it is necessary to establish a comprehensive response system using information and communication technologies, and it is urgently required to establish a preventive centered disaster response system .

Regarding the social disaster response system, the decision on safety related policy is based on Article 9 of the "Basic Law on Disaster and Safety Management" and the Prime Minister establishes a "Central Safety Management Committee" to deliberate disaster and safety management policy. For the practical review, a subcommittee of the Safety Policy Coordination Committee and a regional committee for the review of regional disaster safety policies are established.

For large-scale social disasters, including natural disasters, the "National Disaster Safety Headquarters" is established under the Article 14 of the "Basic Act on Disaster and Safety Management", which is the head of the National Security Agency, to oversee disaster countermeasures. The Central and Local Accident Prevention Headquarters have been set up to direct the disaster and safety headquarters of the provincial units and municipal units.

Also. In order to share disaster information, the system is structured to share disaster situation information by linking the central disaster safety situation room with local disaster safety situation room and municipal disaster safety situation room.

To this end, by introducing advanced information and communication technology that uses ICT element technology such as communication, image, and sensor for the early and proactive preparation of social disaster to the disaster safety field in technical aspect, There is a need. Because disaster-related budgets and human resources of the local governments are not able to achieve actual site-based disaster management, it is necessary to construct a disaster response system that effectively manages disaster-hit areas using automation and information technology.

In the event of a disaster, if an appropriate response to the first disaster is not achieved, it often leads to a second disaster, and a disaster management system capable of early warning and early response is needed.

With the increase in the number of multi-use facilities, it is necessary to evacuate a large number of people from a place where people are concentrated, and it is necessary to develop a method of effectively evacuating using ICT technology.

In developed countries such as Japan, USA, and Europe, CCTV, directional speakers, Internet of Things (IOT) based disaster warning and response system, and national disaster prevention system have been constructed in response to disaster damage.

The National Disaster Management System (NDMS) establishes the national disaster management system and the disaster management communication system that is being constructed and utilized by the National Emergency Management Agency (NEMA), which is a national disaster management organization. It also provides systematic prevention, preparation, quick response, Fire, etc., provides information on the whole service life of 119 services to provide disaster safety service for the people. Automatic rainfall alarm system It has sub-system of fire alarm video network, fire alarm video network, disaster security network, fire alarm detection system, and real-time monitoring of disaster and disaster area in the country using Mugunghwa satellite No. 5 The satellite dedicated network is also being built and operated for the purpose.

Evacuation and disaster countermeasures are needed to minimize damage to public facilities in disaster areas where fire and gas explosions, such as high-rise building fires, subway fires, shopping centers, underground shopping malls, etc., are highly likely to cause human casualties.

In the case of floods, storms and tsunamis, where accidents occur more frequently than accidents, typhoons, typhoons, tsunamis, strong winds, floods, floods, floods, The major causes of overturning of large cruise ships such as collapsing, collapsing, landslides, and seahorse were mainly caused during the summer, and rivers, valleys, reefs, camping sites .

Particularly, in the case of flood disasters, 17.2% of deaths occurred in the case of flood disasters, and the rate of deaths is much higher than that of fire (0.6%) and fall (4.4%).

Social disasters occur in various forms and deaths are continuously increasing, but there are huge social loss costs due to insufficient measures.

In addition to safety personnel, it is difficult to observe people in accident areas. In addition, it is difficult to monitor people personally. In case of floods and heavy rains during the summer season, there are frequent occurrences of water sports such as rivers, valleys, It is difficult to cope effectively with patrols mainly due to road traffic, fire, explosion of harmful gas, marine accidents, and fall. Particularly in valleys and campgrounds where people are uncomfortable, it is difficult to recognize the accident immediately. I can not.

The number of social disasters in 2012 is 300,000, which is much higher than natural disasters caused by natural disasters. The total number of social disasters in 2012 was 303,707, resulting in 383,129 deaths (7,322 deaths, 375,807 injuries) .

Table 1 shows the number of incidents by social disaster [Source: National Security Agency, (formerly National Emergency Management Agency Emergency Disaster Prevention Year 2012)].

division 2010 2011 2012 3 year average Road traffic 226,863 221,711 223,656 224,076 fire 41,863 43,875 43,249 42,995 suffering 2,267 2,393 3,954 2,871 Ocean 1,627 1,750 1,632 1,669 fall 1,365 2,699 10,119 4,727 Total number of occurrences 280,607 286,851 303,707 290,388

Table 2 shows the trends in deaths due to social disasters (Source: National Security Agency, National Emergency Management and Emergency Management Statistical Yearbook, 2013).

division 2010 2011 2012 3 year average Road traffic 5,505 5,229 5,392 5,375 fire 304 263 257 274 suffering 360 489 632 493 Ocean 85 38 64 52 fall 104 189 333 208 Total deaths 6,758 6,709 7,322 6,929

It is necessary to establish a social disaster response management system through the development of rescue system and rescue system that can minimize the accident damage by quick initial response.

In order to reduce social accidents, it is necessary to investigate and analyze the types of disaster incidents to develop a system capable of preventing accidents and responding in the early stages. It is necessary to develop an ICT-based social disaster prevention and response system that can detect accidents, detect situations, provide information, and respond to disasters that can occur anywhere, anytime, by converging images and other sensors. A system for prevention and reduction of social accidents is needed by utilizing the combined technology of directional speaker, image and physical sensor as a way to grasp and respond to the situation of the disaster and the disaster site in real time.

[Technology trends at home and abroad]

○ Intelligent fusion system using image sensor

The Integrated Control Center develops image analysis solutions and uses image analysis and intelligent video control center building solutions based on Big Data.

It has a response system that can catch the signs of crime early by using image information and can take preemptive action. It also provides infrastructure to improve public service through various information collection and analysis through surveillance equipment.

Intelligent integrated control and image analysis solutions are developed to contribute to strengthening services in the public safety field through big data including security image information, real time traffic information, weather information, geographical information and building information.

Intelligent video control solution is applied to various fields such as traffic, industry, disaster disaster prevention and crime prevention based on image data analysis.

Intelligent image analysis technology is applied to images transmitted to cameras installed at intersections, pedestrian crossings, roads and piers, etc., to deal with illegal parking regulations and traffic accidents.

It monitors advanced manufacturing processes as well as prevention of industrial accidents such as chemical gas leaks or explosion accidents in industrial sites.

Through intelligent image analysis, we can detect abnormal situation or pre-symptom precisely with only image data.

The crime prevention field is used in a wide range of public and life safety fields such as detection of unauthorized persons or goods, detection of missing persons and demented elderly people.

○ Directional speaker

In some advanced countries, it is being used to combat algae and to protect pirates by utilizing high-powered sound technology.

○ Intelligent image level detection system

Intelligent CCTV is used for water level detection to provide real-time analysis of water levels such as river and river, and a system for managing alarm levels separately.

○ Bridge projected suicide monitoring system

It monitors real-time traffic situation of bridges, operates alert system in case of a situation, and quickly puts rescue teams into operation. It is operated in two places of Hangang Bridge in Seoul.

○ Integrated management system for hazardous chemicals

Realization of real-time weather information and material diffusion simulation analysis are introduced by systematizing hazardous chemical information, etc.

Provides information on chemical substances handled by the workplace, customized information for each workplace, and disaster prevention information for accident response.

As a related art 1, Patent Publication No. 10-2016-0097708 discloses a " system for always-on environment control and disaster response control using an integrated sensor module ".

The continuous environment control and disaster response control system using the integrated sensor module is composed of a sensor module including a sensor node, an inter node, and a sink node installed in a unit building and sensing environmental information in a unit building, A U-City integrated operation center unit for receiving and managing information from the sensor module or the control system unit. The continuous environment control and disaster response control system using the integrated sensor module configured in this way enables integrated sensing by multiple sensing nodes, facilitates quick disaster information exchange and analysis, and integrates U-City through smart building It is possible to build an intelligent and safe integrated platform for the whole city by linking with the central control and central control department, maintenance and disaster control system.

As related art 2, patent registration No. 10-17431380000 discloses "Disaster response system and method based on object intelligence communication".

Disaster response systems based on intelligent communication of objects have been developed to cope with unspecified number of residents and creatures due to widening of damage range and cumulative prolongation due to spread of various human resources accidents such as hazardous chemical gas diffusion, crude oil spill, plant explosion, Based on the Big Data, we continually check for human accidents, which have a profound impact on the environment, before the occurrence of accidents, to systematically establish the management of hazardous chemicals accompanied by high diffusion and high risk, It will minimize the risks that can be caused by promptly responding to and coping with the local residents for a long time. In addition, in the event of actual disaster, local people will recognize the area where the disaster occurred, Moving to the safe area by avoiding the propagating area In addition to minimizing personal injury after a disaster, even if the wireless Internet network between the terminal and the central control system of the persons located in the accident area can not carry out the pretext of the disaster, The present invention relates to a system and method for responding to a disaster based on intelligent communication, which provides a path through which a person can safely escape through the confirmation of the location of his / And transmits it through the TCP / IP-based Internet network using the TCP / IP Internet communication module 130. The object-oriented disaster prevention sensor management member 100 ); The wireless LAN communication module 220 is a terminal owned by a person belonging to the area surrounding the industrial facility equipped with the MOUs 100 and belongs to the area around the industrial facility indicating the position of the belonging party A terminal 200 belonging to an area surrounding an industrial facility to transmit through the wireless Internet network; The area around the industrial facility where the location of the OBT management member 100 is indicated on the map indicating the area surrounding the industrial facility The location of the intelligent disaster prevention sensor The area around the industrial facility where the map information is stored Object intelligence Disaster prevention sensor Location information A DB unit 310 including a map list information DB 316; The disaster analysis engine 320 and the safety movement path generation engine 330. The disaster analysis engine 320 is operated to transmit the disaster analysis sensor management member 100 through the TCP / And transmits the surrounding disaster prevention information to the object disaster prevention sensor management member which has transmitted the detected disaster prevention environment information when the accident is detected 100, receives the location coordinate information of the affiliated area around the industrial facility through the wireless Internet network from the terminal 200 belonging to the area surrounding the industrial facility, operates the safety movement path generation engine 330, The map information of the object intelligence disaster prevention sensor location around the industrial facility is extracted from the information map DB 316 of the object intelligence disaster prevention sensor in the vicinity of the industrial facility, The accident risk radius r1 is set around the object intelligent disaster control sensor management member 100 that has transmitted the disaster prevention surrounding situation information in which the accident has been detected on the surrounding intelligent disaster prevention sensor location information map information, (SZ) indicating a safe area from the object intelligent disaster control sensor managing member (100), which transmits the disaster awareness circumstance information detected on the map information to the extracted intelligence disaster prevention sensor management member (100) (R1) of the object intelligent disaster prevention sensor location map information about the extracted industrial environment surrounding the transferred industrial facility by referring to the position coordinate information of the area around the transferred industrial facility, (MR) based on the danger avoiding route, which is a route that moves from the position of the perceived corresponding persons to the safe zone (SZ) And displays the extracted information on the object intelligence disaster prevention sensor location information map information of the extracted industrial facility surroundings and transmits it to the affiliated terminal 200 of the affiliated area located in the outside of the accident risk radius r1 A pair of position-measuring slave-side positioning point-based positioning terminal 500b (500b) having a BLE communication module 530b, 530c and an RSSI measurement module 520b, 520c, And a short distance location guidance terminal 500a for location measurement master side installation point having a BLE communication module 530a and an RSSI measurement module 520a are installed in the vicinity of the industrial facility, And a pair of position-measuring slave-side positioning point-based positioning terminal 500b, 500b, 500c, 500d, 500d, 500c and position A perimeter map list information DB (for each installation point), which stores perimeter map information for each installation point, which indicates an installation point area including an area formed by connecting the short distance guidance terminal 500a for three measurement points on the master side for measurement The disaster response server 300 further includes a disaster response server 300 for detecting the occurrence of an accident occurrence prediction situation when the analyzed value of the disaster prevention circumstance information corresponds to the detection of an accident occurrence prediction situation, And transmits a wireless Internet network normal operation guidance signal for informing that the wireless Internet network is operating normally. The terminal 200 belonging to the area surrounding the industrial facility transmits a predetermined setting cycle through the wireless Internet network from the disaster response server 300, A wireless disconnection guidance signal including user identification information and server disconnection guidance information when the normal operation guidance signal is not transmitted And transmits the data through the local area network using the Lutus communication module 230 and forms the local location guidance terminal group 500 according to the installation point, The wireless local area information terminal 500b and 500c on the slave side installation point and the local positioning information terminal 500a on the master side installation point for location measurement, The BLE communication modules 530b and 530c of the short distance position guidance terminals 500b and 500c and the position measurement master side 500b and 530c of the pair of position measurement slave side installation points, The BLE communication module 530a of the local positioning information terminal 500a for each installation point receives the wireless disconnection guidance signal, The local positioning information terminals 500b and 500c for measuring measurement slave side installation points operate the RSSI measurement modules 520b and 520c to measure the strength RSSI of the received radio disconnection guidance signal, The slave side strength measurement information of the pair is transmitted to the short distance location guidance terminal 500a for each location-based master side installation point constituting a set via the local area network using the BLE communication modules 530b and 530c, The local positioning guide terminal 500a for each position-measuring master side installation point forming a pair with the slave side installation point-based slave side positioning terminal 500b, 500c for position measurement of the position measuring slave side terminal 500a operates the RSSI measuring module 520a, (RSSI) of the received radio disconnection guidance signal, and based on the measured value, master side strength measurement information and a pair of slave side strength measurement information received via the BLE communication module 530a, The local positioning guidance terminal 500a for the master side installation point and the local positioning guidance terminals 500b and 500c for the slave side installation point for position measurement, which receive the signal, are connected by three points By-installation-point location coordinate information indicating the location of the affiliate terminal 200 in the vicinity of the industrial facility, which has transmitted the radio-disconnection guidance signal in the installation point area, and generates perimeter map list information DB 511a ), Peripheral map information for each installation point including the positional coordinate information of the individual members by the installation point is extracted, and referring to the positional coordinate information of each member by the installation point on the extracted peripheral map information for each installation point, The route is displayed, and the route information is processed into peripheral map information for each installation point of the escape route, and the user identification information is collected in the surrounding map information for each of the processed escape route installation points And the terminal 200 belonging to the area surrounding the industrial facility transmits the local communication network from the local positioning guide terminal 500a for each installation point on the master for position measurement to the local communication terminal 500a And receives the surrounding map information for each escape route through the Bluetooth communication module 230 and performs user identification based on the user identification information collected in the surrounding map information for each escape route of the escape route Provides a system for outputting and displaying surrounding map information for each installation point and a method using the same.

As related art 2, Patent Disclosure No. 10-08000230000 discloses "Disaster response resource management system and method ".

A disaster response resource management system is a disaster response resource management system that manages resources that are put into a discrete area of a disaster site. It is a checkpoint device that wirelessly checks the information held by the resource being input. An input unit for inputting input information including an input mission to an input resource; And a processor for generating and outputting current status information on resources input to the corresponding individual zone based on the information from the check point unit and the input unit. According to the present invention, the state of human and material input resources at the disaster site can be databaseed so as to be able to tactically and strategically judge the level conductor of the situation room in the field and remote places, .

However, in order to analyze the disaster characteristics through the case analysis of social disasters and to develop countermeasure scenarios, the typhoon and tsunami should be prepared to prevent the disaster by the early warning system of the Korea Meteorological Administration, Floods, floods, and valley rapids require a disaster response system using social disaster prevention technology and ICT technology, but there is no system to prevent property and casualties.

Therefore, the development of a social disaster response strategy, intelligent camera and disaster information detection, ICT-based complex sensor node and IoT-based sensor network construction, gateway-related disaster response system, IDS integrated control system and IDS local control system, System intelligence based image analysis / situation awareness / image tracking, analysis of the types of social disasters, rapid IDS-based life-saving scenarios in response to disasters, disaster alert broadcasting using directional hotspots, evacuation evacuation in the event of a disaster Technology became necessary.

Patent Publication No. 10-2016-0097708 (Aug. 18, 2016), "System for continuous control of environment and disaster response using integrated sensor module", Korea Advanced Institute of Science and Technology, Patent Registration No. 10-17431380000 (registered on May 29, 2017), "Disaster response system and method based on intelligent communication of objects", Bain Tech Co., Ltd. Patent Registration No. 10-08000230000 (Date of Registration January 25, 2008), "Disaster Response Resource Management System and Method", National Emergency Management Agency

It is an object of the present invention to solve the problems of the related art by providing a CCTV camera, a fusion compound sensor, an IoT sensor network (USN, WSN, 6LoWPAN, MQTT, Wi-Fi, NB-IoT, LTE , LoRa network), and IoT gateway, and developed mobius server, IDS integrated control system and IDS local control system, artificial intelligence based image management system for disaster situation analysis / situational awareness / Provides object tracking of video objects, sensor data monitoring, event detection by screaming sound using microphone sensor in dangerous area, provides big data analysis of social disaster type, disaster response IDS based protection system, And evacuation guidance system using a sensor network and a directional speaker system capable of variable output, We provide a camera, sensor network and directional loudspeaker system based on fusion technology for social disaster response that provides an inductive app using non-audible frequency.

In order to achieve the object of the present invention, a camera, a sensor network based on fusion and fusion technology for social disaster, and a life guarding system using a directional speaker include at least one camera, a digital video recorder (NVR) CCTV camera system; At least one IoT-based sensor for transmitting sensor data that detects a disaster situation; The sensor packet data detected from the IoT-based sensor is transmitted to a wireless sensor network (USN, WSN, 6LoWPAN, MQTT), a Wi-Fi sensor network using NB-IoT, a mobile communication network LTE, a LoRa network, An IoT gateway which receives the packet through any one of the networks, converts the protocol, and transmits the converted protocol to the server; And an IoT-based sensor for collecting and storing sensor data sensed from the IoT-based sensor through any one of the networks and having sensor data monitoring and sensor node information management and control system, A Mobius server connected to the network, an image analysis server connected to the CCTV camera system and detecting an object of the deep learning based image of the artificial intelligence and tracking the object of the disaster situation image analysis / situation recognition / image, Includes an IDS control system that analyzes big data and takes disaster response measures,

The IDS control system consists of a number of IDS local control systems and an IDS integrated control system. The IDS local control system is interlocked with IDS polls.

According to the present invention, a camera, a sensor network and a directional loudspeaker system using a fusion-type technology based on a fusion technology for preventing social disasters can be applied to a CCTV camera, a fusion compound sensor, an IoT sensor network (USN, IOT sensor platform technology including ITS gateway, WSN, 6LoWPAN, Wi-Fi, NB-IoT, LTE and LoRa network) and IoT gateway. We also developed Mobius server, IDS integrated control system and IDS local control system, System intelligence based image analysis / situation recognition / image object tracking, sensor data monitoring, event detection by screaming sound using dangerous area microphone sensor, social disaster type big data analysis, disaster Provides IDS-based human-life-protection system, provides emergency alert broadcasting and evacuation technology, It provides an evacuation guidance system using a directional speaker system that can generate power, and an induction app that utilizes the non-audible frequency of a smartphone.

In this system, a pilot zone system is selected based on the analysis of disaster type and regional disaster occurrence status, and a test bed system is established. The disaster prevention system is divided into disaster prevention systems And will develop a mobile control system that can be used in the event of a social accident in the future.

We are improving the laws and systems related to social disasters, linking the U-City integration platform with the IDS system, linking central agencies, related agencies and municipal systems to build a national disaster prevention system, Lt; / RTI >

FIG. 1 is an overall configuration diagram showing a disaster response system based on a conventional object intelligence communication.
FIG. 2 is a system diagram of a life guarding system using a camera, a sensor network, and a directional speaker based on a fusion technology for social disaster response according to the present invention.
FIG. 3A is a graph showing the trend of occurrence of social disaster and natural disaster.
FIG. 3B is a screen of the IDS name protection system.
FIG. 4 is a screen showing a method of applying a test bed for each core technology.
FIG. 5A is a diagram illustrating a disaster information system based on IoT.
5B is a block diagram of a mobile vehicle-based disaster response system using a pole and a directional speaker.
5C is a diagram showing a configuration example of a sensor node.
5D is a diagram showing the system configuration of the hybrid sensor node.
FIG. 5E shows a hybrid sensor node-sensor node driver test.
5F is a diagram showing the type and configuration of the sensor node.
5G is a configuration diagram of the hybrid sensor node installation.
5H is a diagram illustrating the functions of the IoT sensor, the sensor network gateway, the Mobi server, the IoT monitoring UI, and the IoT management UI using the MQTT protocol according to the embodiment of the present invention.
6 is a diagram showing the outline of the vapor sensor.
7 is a view showing an optical water level sensor.
8 is a view showing a rainfall sensor.
9 is a view showing a human body detection sensor.
10 is a configuration diagram of a Wi-Fi mesh network.
11 is a diagram showing an NB-IoT network structure.
12 is a diagram comparing LoRa with other technologies (Sigfox, LTE MTC, LTE NB IoT).
13 is a diagram showing a LoRa communication layer.
14A is a sensor data reception log, IoT monitoring system screen graph (left), and sensor data value (right) screen of a Mobi server receiving sensor data from various sensor nodes via a sensor network.
FIG. 14B is a detailed implementation screen of the IoT gateway data processing module through a sensor network that extracts the sensor node ID, the sensor device ID, and the sensor data value and stores the extracted data in the DB in the Mobi server.
14C is a diagram showing an example of an IoT sensor network communication protocol (MQTT protocol) in which a sensor node transmits sensor data to a Mobi server through a gateway.
14D is a diagram of a variable output directional speaker system capable of adjusting the radiation angle and the pan tilt.
14E to 14H are views showing the features of the directional speaker.
14I is a diagram showing the external shape of the directional speaker.
14J is a communication protocol definition file showing transmission information of the sensor, the camera, and the directional speaker.
14K is an analysis image emulator screen.
15 is a diagram showing an example of object detection.
16 is a view showing a Faster RCNN.
17 is a diagram showing a structure of a single shot multi-box detector (SSD).
18 is a diagram showing an example of object extraction according to the feature map size in the SSD.
FIG. 19 is a diagram showing an example in which a model learned only by the COCO database is applied to a CCTV image, and a model in which a built database is further learned is applied to a CCTV image.
20 is a diagram showing a deep learning structure for object tracking.
FIG. 21 is a diagram showing an object tracking result. FIG.
22 is a diagram showing a CNN (Convolutional Neural Network) structure for Person re-identification.
23 is a diagram showing the CUHK-03 database (the same person in the same row).
24 is a screen for acquiring a big data database for object detection.
Fig. 25 is a picture for erasing the erroneous detection area.
26 is a photograph of a river viewed from a CCTV camera.
FIG. 27 is a picture of a CCTV camera taken in a dangerous area for prohibiting entry.
FIG. 28 is a photograph of a CCTV camera taken at a private spot and moving in a danger zone at the sunset.
29 is a photograph including a gas sensor having a solar panel, a water level gauge, a directional speaker and a camera.
FIG. 30 is a photograph in which the emergency alert broadcast directional loudspeakers, rescue workers and security personnel reside.
FIG. 31 is a photograph in which an accident occurrence area, a dangerous accident dangerous area and a specific area are provided with a safety sign and set as a prohibited area.
32A, 32B and 33 are views showing a monitoring UI screen in the control system.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The camera, sensor network and directional loudspeaker system based on fusion compound technology for social disaster response of the present invention can be applied to a CCTV camera, a fusion compound sensor, an IoT sensor network (USN, WSN , IOT sensor platform technology including 6LoWPAN, MQTT, Wi-Fi, NB-IoT, LTE, LoRa network) and IoT gateway, and developed mobius server, IDS integrated control system, IDS local control system, Provides event detection by screaming sound using dangerous area microphone sensor, analysis of disaster situation based on artificial intelligence of control system / situation awareness / image tracking, analysis of big disaster data type, disaster response Provides IDS-based human-life-protection system, provides emergency alert broadcasting and evacuation technology, and sensor network and variable output A directional evacuation guidance system using the speaker system, and provides a guided smartphone ratio utilizing the blue frequency App.

The present invention relates to the construction of a hazard prevention system and a disaster situation propagation system by collecting valid data from various disasters and disasters.

- Analysis of disaster type, cause of accident and characteristics through social data collection and big data analysis

- Technical limitations of image sensors Combining sensors (image, physics, sound) with directional speakers to provide early warning and disaster situation propagation technology

- Intelligent broadcasting and prevention technology using CCTV and broadcasting system

- Safe and quick evacuation guidance system in various disaster situations using directional speaker and sensor network

- CCTV, camera, speaker, microphone, sensor, etc.

- Protection system for people, Evacuation guidance system

- Development of mobile control system that can be used in the event of a social accident

- Integration of U-City integration platform with IDS system

- Connection with established disaster prevention systems by type and characteristics of social disasters

- Selection of pilot area according to disaster type and regional disaster situation analysis and establishment of test bed system

- Linking central government agencies, related agencies and local government systems to establish national disaster prevention systems

- Improvement of law and system

○ First year:

<Design and Utilization Plan of Utilization System for Combined Hybrid Sensor>

· Examination of smart system construction plan using fusion multi sensor, various systems and equipment

· Development of various algorithms such as Seamless Tracking

<Design and Usage of Directional Speaker Utilization System>

· Review of possible disaster prevention range using directional speakers

· Modularization and weight reduction of directional speaker according to various types such as disaster type, area, place, etc.

<Use of CCTV camera>

· Review and analysis of utilization plans according to types such as disaster type, area, and place

 Examination of related laws, systems and certification for applying local government system

○ 2nd year: Development of directional speaker using system based on Seamless Tracking technology using IoT-based fusion hybrid sensor network technology

- Development of integrated management scenario from accident prevention to incident response

    · Development of integrated standard operating procedure (SOP) from prevention to countermeasure in case of circulation accident using directional speaker

· System independent operation and integrated operation scenario development

- Development of Seamless Tracking Technology using IoT based Hybrid Hybrid Sensor Network Technology

<Development of sensing technology and transmission module>

· Development of IP board for compression and transmission

· Camera module development

· Development of MANET based complex sensor

· Development of IoT-based sensor network technology

· Developed Seamless Tracking Technology

· Development of context awareness technology through sensor fusion

<Development of directional speaker technology capable of variable output>

· Variable output conversion directional speaker for rescue through acoustic sensor interlock

· Variable output considering direction of disaster / Directional speaker with adjustable radiation angle

· Lightweight and compact size of directional speaker for rescue

· Development of disaster prevention plan through intelligent unmanned broadcasting technology

· Development of mobile control system for safe square area

· Development of evacuation-induction (audible / non-audible) multi-directional speaker

- Disaster management system, human life protection system, evacuation guidance system design

· Development of risk detection technology and evacuation guidance algorithm for multi-use facilities using sensor network technology

· System design and development based on cognitive function of visual and auditory

○ Third year: Improvement and enhancement of system functions, plan for linking with other control centers

 - Advanced linkage system based on fusion compound sensor (image, physics, sound)

<Development of directional speaker technology>

· Development and advancement of directional speaker capable of multi-audible frequency

Development of directional speaker based on beamforming

<Development of sensing technology>

· Advanced technology of Seamless Tracking technology with fusion compound sensor

· IDS integrated control system server, system development

· IoT-based sensor network integrated system development

- Development of evacuation guidance system using cognitive function based sensor network and directional loudspeaker using visual and auditory

- Development of induction APP using non-audible frequency of smartphone (detailed design of evacuation guidance using audible / non-audible frequency, characteristic analysis of non-audible signal)

□ Sensor node

1. Production of two sensor nodes: Type-1 (Basic type), Type-2 (Proximity type)

2. Waterproof performance: IP67

3. Sensor performance

- Water level: 0.3 to 1.5 m range and 0.2 m resolution

- Weather sensor: Temperature, wind direction wind speed

- Communication: Equipped with Wifi mesh network communication

- Directional microphone: Detection of screaming event (setting of dB relative to ambient noise) detection

- Directional camera: Event-linked still image shooting and transmission

- Sensor node safety sensor: vibration, tilt detection

□ Development of Wi-Fi mesh network and IoT-based sensor network technology

- Effective distance of communication between sensor nodes: 50m or more

- Wi-Fi mesh Network Configuration Number of nodes: 5 or more

□ Development of disaster awareness and seamless tracking technology based on image analysis

1. Development of situational awareness technology through fusion sensor

- Object detection and behavior recognition using video data analysis

- Event detection through object detection and tracking technology

- Detection and detection of anomalous situations by object detection / tracking technology and behavior recognition technology

2. Development of Seamless Tracking Technology

- Realization of image based object detection and tracking technology

FIG. 2 is a system diagram of a life guarding system using a camera, a sensor network, and a directional speaker based on a fusion technology for social disaster response according to the present invention.

A camera, a sensor network, and a life guarding system using directional loudspeakers, which are based on a fusion technology for coping with social disasters according to the present invention, include at least one camera, a network video recorder (NVR) Camera system; At least one IoT-based sensor for transmitting sensor data that detects a disaster situation; The sensor data sensed by the sensor may be transmitted to any one of a wireless sensor network (USN, WSN, 6LoWPAN, MQTT), a Wi-Fi sensor network using NB-IoT, a mobile communication network LTE, a LoRa network, An IoT gateway for receiving the protocol over the network and transmitting the converted protocol to the server; And sensor network through any one of a wireless sensor network (USN, WSN, 6LoWPAN, MQTT), Wi-Fi, sensor network using NB-IoT, mobile communication network (LTE), LoRa network, A Mobi server that is connected to an IoT-based sensor network that collects sensor data and analyzes sensor data and analyzes sensor data, analyzes the sensor data, analyzes the sensor data, An image analysis server connected to the CCTV camera system for detecting the object of the deep learning based on the artificial intelligence and analyzing the disaster situation image / situation awareness / image, and analyzing the big data according to the type of social disaster Includes an IDS control system that takes disaster response measures,

The IDS control system consists of a number of IDS local control systems and an IDS integrated control system, and the IDS local control system is interlocked with IDS poles.

The IDS control system is based on the IoT sensor network. The Mobius server is an image analysis server that detects the objects of the deep learning based image of the artificial intelligence and analyzes the disaster situation image / situation recognition / image object, A disaster response system that provides a disaster alert broadcasting and evacuation induction technology through an evacuation guidance system that transmits a disaster announcement text to a mobile communication terminal near a dangerous area and a directional speaker capable of adjusting a radiation angle and a pan tilt And an IDS control system.

The Mobi server is a part of the IDS local control system and transmits various monitoring data such as sensor data, image, and sound information in cooperation with the IDS integrated control system.

The IDS local control system is equipped with a monitoring module and a control module for monitoring images, sensor data and sound source data of the Mobi server, a time series database (TSDB) and an image analysis server (Video Server), and a control module. , Sensor, sound source, image data collection, analysis and monitoring function, object detection based on deep learning of artificial intelligence by image analysis server, disaster situation image analysis / context tracking / image object tracking, directional speaker control Control functions for camera and infrared IR operation, control function for pan / tilt operation, spherical egg and IDS pole, monitoring function for sensor position confirmation, real-time image confirmation and analysis function obtained from camera, Real-time sensor data received from Bias server, past sensor data retrieval function, (Disaster text transmission) and a disaster response function (moving vehicle disaster warning broadcasting function) in conjunction with an emergency evacuation guidance system for guiding evacuation and a mobile vehicle having a directional speaker and a camera, ), Provides IDS pole interlock function,

The IDS integrated control system is connected to a plurality of installed IDS local control systems distributed in various regions, receives disaster situation information from each IDS local control system and provides an IDS local control system monitoring function,

The IDS pole is equipped with a camera for acquiring real-time image information, an IR LED for night-time imaging, a directional speaker for transmitting alarm broadcast to the old egg, pan / tilt control for targeting the dangerous area around the daily life, and communication middleware It is associated with the Mobius server used as part of the IDS local control system.

1. Research Goals

Review comprehensive and detailed analysis of the current status and causes of social disasters to identify social needs and disaster response capabilities

Developing strategies for the development of human life protection systems and development of the necessary technologies for system development for the effective disaster prevention and human casualty reduction technology utilizing ICT technology

Development of technology and system for the protection of human life, focusing on the evacuation inducement in multi-use facilities, which is highly likely to cause accidents and deaths

3 is a screenshot of the IDS lifeguard system.

FIG. 4 is a screen showing a method of applying a test bed for each core technology.

Implementation of demonstration pilot projects for local governments and related organizations for system suitability and performance verification of disaster response countermeasures system In order to ensure that the system developed through the research can be used in conjunction with disaster and safety management systems of local governments, System connection and extension utilization, and presented as a local government disaster safety management system model

I. Final Research Goals

○ Development of social disaster response strategy and management of pilot project

- Establishment of basic strategy for social disaster type

- Development of IDS-based lifesaving scenario and manual

○ IDS sensor network platform for social disaster monitoring and response

- An intelligent sensor node consisting of sensor combinations that can predict the factors causing social disaster in advance or detect in real time

- Mesh or Mobile Ad-hoc communication network for transmitting the information obtained from the sensor nodes to the server

- IoT sensor network that stores and manages information acquired from sensor nodes. - Establishes and operates a MOBIUS server (MOBIUS server), which is used as a life guarding and disaster response system linked to a network. Connected with the control system

[Core R & D Technology]

○ Intelligent sensor node technology

- In order to predict and detect the occurrence of disasters in accordance with the types of social disasters, the necessary sensors such as sound, water level, flow velocity, rainfall, fire etc. are selected and sensor modules

- Sensor node self-diagnosis and system error control technology to enhance system robustness of sensor node and accuracy and reliability of sensor measurement

- Sensor node intelligence algorithm including event classification and urgent situation recognition through collected information including environment and disaster information collection and transmission function

- Acquisition and transmission of acoustic information for seamless tracking and guidance and evacuation guidance and directional loudspeakers

The sensor node is a human life protection system (disaster response system) interlocked with an IoT-based sensor network including a rainfall sensor, a water level sensor, a flow velocity sensor, an air velocity sensor, an acoustic sensor, a microphone sensor, a gas sensor, Evacuation guidance system should be constructed and used. The sensor provides waterproofing and environmental protection.

In addition, the sensor node provides a still image capture and transmission function of a CCTV camera or a sensor node itself camera, a microphone signal analysis and event detection function in case of a sensor node microphone sensor, and a battery discharge warning function in a sensor node. It works with the Mobi server.

The IoT-based disaster response and human life-protection systems used wired power, battery or emergency power supply system rainfall sensor, water level sensor, flow rate sensor, wind speed sensor, acoustic sensor, gas sensor, fire detection sensor and human body sensor.

The rainfall sensor is used to predict the rainfall of the sensor node installation area in real time.

The water level sensor uses an ultrasonic level sensor or a radar level sensor and measures the level of the river.

The flow velocity sensor uses an electromagnetic wave surface velocity meter and measures the flow velocity of the river and is used together with the water level and the rainfall amount. The flow velocity sensor is a non-contact type flow velocity sensor that uses an electromagnetic wave surface velocity meter and measures the Doppler effect, which is a phenomenon in which the frequency of an electromagnetic wave scattered by an object moving in the water velocity is changed. The Doppler frequency fd = (2v / λ) at a flow rate of cosθ is the v = (λ / cosθ) f d. Here, f _d is the Doppler frequency, v is the velocity of the object, λ is the wavelength of the radio wave, and θ is the angle between the speed direction of the object and the propagation direction of the radio wave.

The wind speed sensor is used for disaster analysis by measuring wind speed.

The acoustic sensor uses at least one high-reliability microphone for driving the directional speaker in the event of an accident.

A fire detection sensor is used to detect a fire in a public facility and induce evacuation.

○ Communication network technology between sensor nodes

- Mesh or MANET (Mobile Ad-hoc Network) sensor network configuration that enables active connection establishment without the help of a coordinator such as a base station, freely joining or leaving the network and configuring the communication network so as to change the network topology frequently

- IoT local control system based on Mobius server and IoT integrated control system configuration connected to it

- Connection with internet Mobius server and integrated control system for collection, storage and utilization of disaster information

Accumulation and provision of information for social disaster big data analysis and forecasting

FIG. 5A is a diagram illustrating a disaster information system based on IoT.

5B is a block diagram of a mobile vehicle-based disaster response system using a pole and a directional speaker.

Vehicle-based disaster response systems using poles and directional loudspeakers are equipped with fixed sensors that are assigned device IDs to the poles in coastal areas, riverfronts, rivers, and valleys, and are installed in multiple locations to measure rainfall, river water level, Sensors 71,72 and 73; A sink node for collecting the device ID, the rainfall amount, the river water level, the flow velocity, and the wind speed information from the sensors 71, 72, and 73, and a device ID, rain water level, A gateway 80 for receiving and transmitting through the sensor network WSN; And the device ID, the rainfall amount, the river water level, the flow rate, and the wind speed information from the gateway 80 via a LAN or a wireless LAN (Wi-Fi) (230), the disaster situation is displayed on the computer monitor (230) of the control center of the disaster safety situation room, and when the rainfall amount, the river water level, the flow rate and the wind speed data are above a predetermined standard value, Transmits emergency alert letters to mobile phone numbers in the cell coverage of the disaster-prone area for emergency evacuation, outputs the remote emergency evacuation broadcast signal to the stationary variable-directional speaker 190 via the communication network, The vehicle is dispatched to control the output of a disaster evacuation broadcast signal to the directional speaker 190 fixed to the pole of the moving vehicle, A disaster response system 210 for providing a cause of accident occurrence and statistical analysis data by analysis, and for responding to a disaster through a big data analysis; And a pole 370 of the moving vehicle 300 and is connected to the disaster response system 210 and the emergency disaster communication network LTE (Wi-Fi) And a variable output directional speaker 190 whose angle is adjusted.

The directional speaker includes a fixed variable directional speaker (fixed directional speaker) 190-1 fixed to the ground pole 330 and a variable output directional speaker fixed to the pole 370 of the moving vehicle 300 ) 190-2,

The directional loudspeaker is capable of adjusting the radiation angle and the pan / tilt angle. The motor control signal receiver and the motor adjust the stepwise radiation angle of the directional loudspeaker, And a directional speaker installed in each room for outputting a band signal (non-audible signal) and a pan / tilt controllable directional speaker. The directional speaker and the directional speaker multi-control system for controlling the pan / Respectively,

The directional speaker multi-control system includes a directional speaker controller for adjusting a radiation angle and a pan / tilt of a PTZ controller equipped with a camera and an IR LED and adjusting a volume of a speaker, a directional speaker controller for outputting sound according to the volume set by the directional speaker controller, A speaker amplifier, and a power supply unit for supplying power.

The moving vehicle may have a camera, a directional speaker capable of adjusting the radiation angle and pan / tilt, and a wireless CCTV camera having a Wi-Fi communication unit or an RF wireless communication unit.

(E.g., a smart phone) connected to the disaster response system 210 through a mobile communication network and equipped with an evacuation guidance app,

When a disaster warning such as a typhoon, a tsunami, a strong wind or heavy rains is expected, a disaster alert character is received from the disaster response system as a disaster preparedness measure, and the disaster alert message is received from the directional speaker 190 fixed to the pole of the moving vehicle, Disaster evacuation voice broadcast is transmitted and the non-audible signal frequency is transmitted to the smart device (smartphone) of the corresponding area from the pole-fixed directional speaker 190 installed in the corresponding area, thereby evacuating from the disaster danger area to the safety area .

Sensors are IoT based sensor nodes (IoT lifesaving sensor module) that use rain sensor, water level sensor, flow sensor, wind speed sensor, microphone sensor, fire sensor, gas sensor and human body sensor (PIR sensor) The river level, the flow rate, the wind speed information, the sound information of the microphone, the fire detection information, the gas leak detection information, and the human body detection information to the disaster response system 210 through the network (WSN) and the gateway.

In addition, disaster information detection and fusion intelligent sensors can be applied to various types of sensors such as rain sensor, water level sensor, flow velocity sensor, wind speed sensor, microphone sensor, The sensor network (USN) detects rainfall, river water level, flow velocity, wind speed information, microphone sound information, fire detection information, gas leak detection information, and human body detection information detected by a fire detection sensor, a gas detection sensor and a human body detection sensor (Life guarding system) 210 of the Mobius server through a network of any one of the wireless LAN, the WSN, the 6LoWPAN, the MQTT, the mobile communication network LTE or the Wi-Fi wireless LAN, the NB- ).

In addition, the sensor captures the still image of the camera in the dangerous area and the accident lot area, develops the post transmission technology to the Mobi server, and provides the event detection function such as screaming sound by using the microphone sensor.

The disaster response system 210 is connected to the IDS integrated control system as an IDS local control system and includes a weather warning system 110 for providing weather information, typhoon information, observation information and snowfall information managed by the national disaster management system 100 ), A flood monitoring system (120) and a water level monitoring system (130) for managing the water level and flow rate of the river according to the rainfall, an earthquake disaster prevention management System 140 and is connected to the national disaster prevention system and the local disaster prevention system operated by the local government. The system receives various disaster detection information and disaster information, and the IDS local control system control center and the IDS integrated control system control center It is displayed on the monitor.

The weather forecasting alarm system 110 is a system for monitoring weather information such as route, typhoon, tidal wave, strong wind, heavy rainfall, snow / snow weather information and observation information measured by meteorological satellites, 210).

The disaster response system 210 is interlocked with the GIS server 220 and displays a map of the disaster area due to floods, floods due to heavy rains or earthquakes, and a flood map.

The disaster response system 210 collects and analyzes disaster related data based on the analysis of disaster related unstructured data, keywords and basic reasoning conditions, and provides analysis data and statistical analysis data on the cause of the accident, It provides statistical analysis on the occurrence of accidents from various multidimensional analysis viewpoints by type, time, region, and scale, and provides analysis of various cause analysis data and history statistical analysis data provided by the institute.

In the disaster safety room of the control center, the PC provides sensor information monitoring on the web-based server and alarm notification and related control solution according to the specific condition that the disaster from the sensor is predicted, and the movement in the cell coverage Emergency disaster warning letters are transmitted to the telephone numbers, and escape evacuation is directed to the safe area through a disaster warning broadcast on a moving vehicle equipped with a directional speaker.

The disaster response system developed a monitoring web for building and managing a service server for administrators and an Android or iOS based smartphone app for alarms and control.

A vehicle-based disaster response method using an IDS pole (IDS pole) and a directional speaker includes the steps of: measuring device ID and rainfall amount, river water level, flow rate, and wind speed information from a plurality of sensors assigned device IDs installed in each area; And receives and stores the device ID, the rainfall amount, the river water level, the flow rate, and the wind velocity information measured from the plurality of sensors through the wireless sensor network WSN or the mobile communication network or the Wi-Fi, And the disaster response system transmits a disaster alarm character to mobile phone numbers in the cell coverage of the disaster-prone area so as to be evacuated when the rainfall amount, the river water level, the flow velocity, and the wind speed data become the predetermined reference value or more, Controls the remote emergency evacuation broadcast data to be output to the fixed variable output directional speaker through the communication network. It controls the disaster evacuation broadcasting to be outputted to the directional speaker of the moving vehicle which is dispatched to the disaster prediction area, collects and analyzes the disaster related data, Providing a cause of occurrence and statistical analysis data; And outputting a disaster announcement announcement broadcast via the disaster response system and the emergency disaster communication network to the fixed directional loudspeaker attached to the received ground poles or outputting the disaster announcement voice through the movable directional loudspeaker attached to the poles of the moving vehicle &Lt; / RTI &gt;

The fixed / mobile directional speaker adjusts the stepwise radiation angle of the directional speaker by the motor control signal receiver and the motor, and adjusts the sound pressure level in each step to adjust an audible band speech signal (audible signal) and a non-audible band signal (non-audible signal) And a directional speaker installed in each room for outputting and a pan / tilt adjustable directional speaker are used.

The method may further comprise: a weather warning system that provides weather information, typhoon information, observation information, and snowfall information; A flood monitoring system and a water level monitoring system for managing the water level and flow rate of the river according to the rainfall amount; And / or an earthquake disaster management system that provides earthquake observation information and manages disaster information and damage information corresponding to an earthquake disaster, receives various observation information and disaster information, displays it on a monitor of the computer of the control center, And outputting a disaster evacuation broadcast to the directional speaker of the moving vehicle dispatched to the disaster prediction area at the time of receiving the disaster information.

The method comprises interfacing with the GIS server displaying the disaster area on the map, displaying a disaster area map according to flood or seismic alert due to typhoon, tsunami, strong wind, heavy rainfall, .

The sensors use a rain sensor, a water level sensor, a flow velocity sensor, a wind speed sensor, a microphone sensor installed in a dangerous area, a fire detection sensor, a gas detection sensor and a human body detection sensor. (Disaster response system) of the Mobius server through the gateway through any one or more of the network (USN, WSN, 6LoWPAN, MQTT), Wi-Fi wireless LAN, LTE-M, NB-IoT, ) Sends rainfall, river water level, flow velocity, wind speed information, microphone sensor information (screaming sound), fire detection information, gas detection information, and human body detection information.

When the wireless communication unit (6LoWPAN, MQTT, Wi-Fi, LTE, NB-IoT, LTE-M, LoRa) is installed, the sensors include a rain sensor, a water level sensor, a flow velocity sensor, (WSN, WSN, etc.) to the sensor network (fire sensor, gas sensor, human body sensor, rain water level, flow velocity, wind speed information, microphone sensor data 6LoWPAN, MQTT), Wi-Fi wireless LAN, mobile communication network (LTE-M), NB-IoT and LoRa network through a gateway to the Mobi server, image analysis server, And an IDS local control system having a disaster response system.

The IDS local control system transmits various data including image, voice and sensor data in cooperation with the IDS integrated control system. The IDS local control system includes a mobi server, an image analysis server and a name keeping system, an evacuation guidance system, Disaster response system,

The IDS local control system includes a Mobi server, a time series database (TSDB), an image analysis server (Video Server), a disaster monitoring unit including sensors and sound sources, and a control module. , Sensors and sound sources, image data acquisition and analysis and monitoring functions, object detection based on deep learning of artificial intelligence by image analysis server, disaster scene image analysis / context awareness / image object tracking, Function, control function for camera and infrared IR operation, control function for pan / tilt operation, spherical egg and IDS pole, monitoring function for sensor position confirmation, real time image confirmation and analysis function obtained from camera, Receive real-time sensor data confirmation function, past sensor data inquiry function, (Disaster text transmission) and disaster response function (moving vehicle disaster warning broadcasting) in cooperation with the disaster response system that provides the evacuation guidance system and the moving vehicle disaster warning broadcasting. ,

The IDS integrated control system is connected to a plurality of installed IDS local control systems distributed in various regions, receives disaster situation information from each IDS local control system, performs IDS local control system monitoring,

The IDS pole is equipped with a camera for acquiring real-time image information, an IR LED for night-time imaging, a directional speaker for transmitting alarm broadcast to the old egg, pan / tilt control for targeting the dangerous area around the daily life, and communication middleware It is associated with the Mobius server used as part of the IDS local control system.

The IDS control system is composed of a plurality of IDS local control systems and an IDS integrated control system.

The IDS local control system includes a mobi server, an image analysis server, a life protection system, an evacuation guidance system, a disaster response system, and a directional speaker multiple control system.

The mobius server is connected to a disaster response system and an SMS server through a mobile communication network. The mobius server is connected to the disaster response system, the evacuation guidance system, the disaster response system, and the directional speaker multiplex control system. And transmits a disaster alarm character to the mobile phone numbers in the cell coverage of the disaster prediction area through the SMS server from the disaster response system , A wireless CCTV camera, a directional speaker, and a disaster warning broadcast of a moving vehicle.

An evacuation guidance app for outputting an audible signal / non-audible signal to a speaker of a smart device from a directional speaker linked with the evacuation guidance system, and evacuating to a safe area through a disaster location and a disaster announcement character in the disaster area; To the installed smart device.

The navigation system is connected to the evacuation guidance operating system through a mobile communication network and outputs an audible signal / non-audible signal from the directional speaker coupled with the evacuation guidance system to the speaker of the smartphone. Further comprising a smart device having an evacuation guidance app (App) provided with a guidance path,

The smart device uses a smartphone and receives the announcement of the non-audible broadcast through the smartphone's evacuation-inducing app (the Android-based app for detecting the direction of the sound source based on the directional speaker) from the Mobi server.

5C is a diagram showing a configuration example of a sensor node. 5D is a diagram showing the system configuration of the hybrid sensor node.

The at least one IoT-based sensor

A sensing unit sensing sensing information; A controller for controlling each function for sensor measurement; A storage unit connected to the control unit and storing sensor data; Wireless sensor network that transmits sensor data through a communication protocol of any one of wireless sensor network (WSN, 6LoWPAN, MQTT), Wi-Fi, sensor network using NB-IoT, mobile communication network (LTE), LoRa network, A communication unit; A protocol stack section for providing an IoT protocol (e.g., MQTT protocol), a device driver for driving a device based on an ARM-based Linux operating system, and a power supply section for supplying power to the sensor node, And a sensor adapter for linking the protocol (MQTT, NB-IoT)

The sensor data includes a sensor node ID, a sensor device ID, and a sensor data value.

The normal power supply unit further includes a solar cell unit having a solar cell array, a charge controller, a DC-DC converter, and a battery, and supplying a predetermined rated power using solar heat.

FIG. 5E shows a hybrid sensor node-sensor node driver test.

5F is a diagram showing the type and configuration of the sensor node.

5G is a configuration diagram of the hybrid sensor node installation.

5H is a diagram illustrating the functions of the IoT sensor, the sensor network gateway, the Mobi server, the IoT monitoring UI, and the IoT management UI using the MQTT protocol according to the embodiment of the present invention.

(Example)

* Sensor module

- Microphone sensor: KB500 microphone amplifier module

- Camera sensor: Sony IMX219 image sensor

* Sensor adapter

- Registration of MQTT service for communication connection with oneM2M platform

- Reads the microphone and camera sensor values from the sensor module and stores them in memory.

- Microphone, camera sensor reading cycle: 1 minute (smart algorithm applied)

* Mobius Platform Gateway

- Registering microphone and camera sensor connected to MQTT to Mobius server

- Receive data from microphone, camera device and transmit to Mobius server

* Mobius platform server

- Save, manage and control sensor data of registered microphone and camera device

- External client access sensor data through Open API

* IoT monitoring / analysis GUI

- Save and manage microphone and camera sensor data using TSDB (time series database)

Provides tools to visually monitor and analyze sensor data

* IoT management GUI

- Store and manage information about IoT sensor device in IoT management information DB

- Combine IoT sensor device information with sensor data for monitoring

[Role of technology and system]

○ Analysis of social disaster type and establishment of response strategy

- Establish basic response strategies for social disaster types and provide specific response scenarios

- Develop IDS-based life-saving scenarios and manuals for disaster situations

○ It is installed in the disaster accident site, and it develops fusion and compound sensor node for collection of environment information and real-time monitoring, and provides mesh sensor network and IoT network

- Cost of accidents cost sensor node equipped with sensors such as rainfall, flow rate, water level, sound

○ Real-time information on site information (sound, rainfall, flow rate, etc.) for image analysis and seamless tracking

○ Provide real-time information of field information (sound, gas, fire, etc.) for directional speaker alarm and evacuation guidance

○ Disaster information collected from the sensor node is stored in the Mobius Server and provided with information, linked with the Disaster Control Center

○ Development of complex sensor node based on mesh network

- Sensor network technology based on mesh network, development of IoT based sensor network technology

- Development of dynamic sensor node communication technology based on mesh network

- Implementation of sensor network and internet connection using NB-IoT

- Development of flow velocity and water level sensor

- Sensor node mechanism and firmware redesign and

- Sensor node smart algorithm implementation test and reliability verification

- Sensor node microphone signal analysis and event detection function development

- Sensor node battery discharge warning function development

- Implementation of sensor data acquisition and sensor control system based on Mobius platform

- Sensor data monitoring system development

- Sensor node information management and control system development

○ Situation recognition and image tracking

- Development of context awareness technology through sensor fusion

- Development of Seamless Tracking Technology

○ Development of directional speaker technology capable of variable output

- Development of variable output directional speaker for rescue

- Development of directional speaker multi control system

○ IDS system and pole production

- Development of IDS Integrated Control System

- Development of IDS Local Control System

- IDS pole production for testing

1.2 Sensor node

The control module of the sensor node controls the sensor device, reads the sensor data, and transmits it to the Mobi server through the IoT sensor network.

The control module includes a power supply unit, a PA power supply circuit, an SDRAM memory circuit, a PLL circuit, a GPIO interface, and an LED status display circuit.

* Control module specification

- CPU: ARM Cortex A53 1.2 GHz (Broadcom BCM2837)

- Memory: 1 GB SDRAM

- Storage: 16 GB SD

- Power Supply: 5 V, 300 mA (average)

- Dimension: 85.60 mm x 56.5 mm x 17 mm

- Interfaces: GPIO, I2C, SPI

I. Sensor module

○ Weather sensor: Airmar WeatherStation PB200

6 is a view showing a vapor sensor. The weather sensor is a sensor for collecting various weather information in the water area in real time.

- Vibration sensor and attitude sensor check whether the facility is installed in a stable posture

○ Level sensor: FS-IR02 Optical level sensor

7 is a view showing an optical water level sensor. The water level sensor is a water level monitoring system, which uses a water level sensor installed in the middle of the bridge and installed on the bridge structure in the upstream area near the watering place to measure the water level change.

○ Rain Sensor: Hydreon Rain Gauge Model RG-11

8 is a view showing a rainfall sensor. The rain sensor is a sensor to recognize the local rainfall condition in the watering place because it is dangerous when the rain falls in the prohibited area or the dangerous area where the torrent occurs due to flood, heavy rainfall.

* Input Voltage: 9 ~ 30VDC, 50V surge

  - Reverse polarity protected to 50V

* Current Drain: 15mA nominal. (No outputs on, not raining, no heater)

  - about 1.5 mA in micro-power sleep mode.

  - 50 mA with output on.

  - 55mA - With heater on, 24V dc input.

* Output: Relay closure, Normally Open and Normally Closed contacts.

  - Max load 1A, 24 VDC.

* Operating Temperature range: -40 ℃ to +60 ℃

○ Human body sensor: SEEED PIR Motion Sensor

9 is a view showing a human body detection sensor.

The human body detection sensor is an infrared PIR sensor for recognizing a person entering or leaving a dangerous area.

- Operating Voltage: 3V ~ 5V

- Operating Current (VCC = 3V): 100uA

- Operating Current (VCC = 5V): 150uA

- Measuring Range: 0.1 - 6m

- Default detecting distance: 3m

- Holding Time: 1 25s

- Working Wave Length: 7 ~ 14um

- Detecting Angle: 120 degrees

When the sensor node uses the MQTT protocol, a complex sensor node oneM2M TAS adapter module is implemented.

- Interface module for interworking with oneM2M Thyme gateway based on TAS adapter MQTT protocol

- Manage the data collection cycle in the data acquisition module from the TAS adapter (eg, sensor adapter)

- On the TAS adapter, set the device settings in the device control module and initialize the device driver.

- Sensor data control and management from the TAS adapter to the IoT network according to the smart transport algorithm

- How to install and run the oneM2M TAS adapter

- oneM2M TAS adapter program was developed based on node.js, so you need to install node.js program to run

1.3 IoT based sensor network

1.3.1 Mesh Network Implementation

end. Wi-Fi Mesh network performance test

Raspberry pie is used as a mesh router.

The Raspberry pie not only serves as an access point (AP) but also connects to each other via another wireless network, whereby a mobile device connected to one AP directly communicates with a mobile device connected to another AP, Deliver packets between APs without Internet connection

To do this, each Raspberry pie requires at least two wireless LAN interfaces (wireless LAN card). One connects mobile devices and the other connects mesh routers

10 is a configuration diagram of a Wi-Fi mesh network.

The Wi-Fi mesh network configuration consists of Locale, time, keyboard layout, and resize settings to take full advantage of the full capacity of the SD card for stable use after initial setup of Raspberry Pie.

1.3.2 LoRa network implementation

end. NB IoT (Narrowband Internet of Things)

11 is a diagram showing an NB-IoT network structure.

- NB-IoT terminal (eg smart gas meter, smart water meter):

terminals and an NB-IoT module and communicates with an eNodeB (base station) using the Uu interface using NB-IoT modules

- NB-IoT module: The terminals communicate with the NB-IoT network according to the NB-IoT wireless communication protocol.

- Terminal (eg smart gas meter, smart water meter): UART communication with NB-IoT modules

- eNodeB: The NB-IoT terminal processes the network access message through the Uu interface, manages the cell, and uses the non-access layer (NAS) as an upper layer network element (NE) Stratum data, and NaodeB connects to the IoT EPC via the S1-Lite interface

- IoT EPC: exchanges information with NB-IoT terminals at NAS layer and transfers data related to NB-IoT service to IoT platform for processing

- IoT platform: collects different types of IoT data from a diversified radio access network and sends the required data to the requested service application for processing based on the data type

- Application server: collect IoT data, process data according to customer requirements

LoRa technology is a technical standard established by the LoRa Alliance organization and is a large, low-power, long-distance wireless communication. Theoretically, LoRa has very low power consumption and has a battery life of more than 10 years. The maximum transmission rate of LoRa is 5.47kbps and the cell coverage is 10km. Communication range is 2 ~ 15km in urban area, 30km in area where line of sight is secured, 1 ~ 2km in underground and 2 ~ 3km in indoor.

Bandwidth is 125kHz, maximum transmit power is 14dBm, operates in frequency ISM band, transmission speed is low, and the receive sensitivity is designed to be up to -138dBm.

12 is a diagram comparing LoRa with other technologies (Sigfox, LTE MTC, LTE NB IoT).

13 is a diagram showing a LoRa communication layer.

The physical layer communication is performed between the modem chip between the terminal and the base station, and the MAC layer communication is performed between the terminal and the network server. LoRa uses Chip Spread Spectrum technology as Bandpass Modulation. Chip Spread Spectrum technology is a key part of LoRa technology that enables long range, low power.

LoRa's long range star is best suited for long battery life. In a LoRa network, a node is not associated with a particular gateway, and data transmitted by each node is typically received by multiple gateways.

Each gateway forwards the received packet from the sensor node to the cloud-based Mobi server via the cellular network, Ethernet, and Wi-Fi.

In addition, intelligence and complexity are pushed to the network server (mobi server), which manages the network, inspects duplicate packets, performs security checks, and performs scheduling to the optimal gateway.

In terms of network capacity, a gateway needs to be capable of receiving messages from large nodes in order to make the Long Range Star network operational.

The high network capacity of the LoRa network is achieved by using a multi-channel, multiple-modem transceiver at the gateway to receive multiple channels of simultaneous messages.

Important factors affecting network capacity are the number of concurrent channels, data rate (latency), payload length, and node transmission frequency. Of these, in terms of data rate, the gateways can receive at several different data rates simultaneously on the same channel. If a node is a good link and is close to a gateway, it always uses the lowest data rate and occupies the available spectrum for longer than needed. By increasing the data rate, the latency is shortened and the potential space that other nodes can transmit is widened.

This capability allows the LoRa network to have a very large network capacity, expand the network, and build the network with minimal infrastructure.

The LoRa network is asynchronous, capable of communicating when data is ready to be sent anywhere, whether it is event-driven or scheduled, and recently compared various technologies dealing with the LPWAN space in the GSMA. As a result, Which is 3 ~ 5 times higher than that of Korea.

1.3.3 Sensor Monitoring System Implementation

end. Monitoring sensor node information

14A is a sensor data reception log, IoT monitoring system screen graph (left), and sensor data value (right) screen of a Mobi server receiving sensor data from various sensor nodes via a sensor network.

The microphone sensor data stores the data in the influx DB and verifies the various graphs through the Grafana-based IoT monitoring system.

FIG. 14B is a detailed implementation screen of the IoT gateway data processing module through a sensor network that extracts the sensor node ID, the sensor device ID, and the sensor data value and stores the extracted data in the DB in the Mobi server.

14C is a diagram showing an example of an IoT sensor network communication protocol (MQTT protocol) in which a sensor node transmits sensor data to a Mobi server through a gateway.

When the MQTT protocol is used as the IoT sensor network protocol, the complex sensor node registers the MQTT service (oneM2M service registration) to the Mobi server through the gateway, informs the sensor registration with the IoT management GUI, And stores the sensor-related information in the IoT management information DB.

Complex sensor node -> Gateway -> Mobi server -> Time series DB (TSDB) Sensor data is transmitted and stored. IoT monitoring analysis software or IoT management GUI acquires TSDB sensor data and monitors sensor data.

The Mobi server provides sensor state information to the IoT GUI.

The IoT GUI sends a sensor control request to the Mobi server -> gateway -> multiple sensor nodes, and the sensor settings are changed at the multiple sensor nodes.

14D is a diagram of a variable output directional speaker system capable of adjusting the radiation angle and the pan tilt. In order to utilize directional loudspeaker as a variable output directional loudspeaker in IDS system, we developed an acoustic sensor interlocking and directional loudspeaker multiplex control system. In addition, IDS poles were fabricated and a camera system, a directional speaker, an infrared IR LED, and a PTZ controller (Pan / Tilt controller) were integrated into one system in the IDS local control system (Mobius server).

For example, the sensor packet data sensed from the IoT-based sensor 100 such as a rainfall sensor, a flow velocity sensor, an air velocity sensor, a water level sensor, and an acoustic sensor is transmitted to a wireless sensor network (USN, WSN, 6LoWPAN, MQTT) An IoT gateway for receiving a signal through a sensor network using NB-IoT, a mobile communication network (LTE), a LoRa network, or an Ethernet and converting the protocol to transmit to a server; And a control system 710 of an IDS local control system 700 for collecting and storing sensor packet data sensed by the IoT based sensor through any one of the networks and for managing sensor data and managing and controlling sensor node information In addition,

The IDS local control system 700 comprises an IDS pole including a control system 710, an IDS poll controller enclosure 720 and an IDS poll rescue box 730,

The control system 710 includes an IDS local control system server 711, a streaming server 712, and an image analysis server 714 interlocked with the CCTV camera system,

The IDS pole controller housing 720 includes a directional speaker controller 723 for adjusting the volume of the directional speaker, a directional speaker amplifier 724 for outputting sound according to the set volume, and a power supply unit for supplying power,

The IDS pole saving unit 730 is connected to the IDS local control system server 711 and includes a PTZ controller connected to the directional speaker controller 723 and a camera 731 connected to the PTZ controller and an IR LED 732,

And a directional speaker output portion 733 interlocked with the directional speaker amplifier 724 of the IDS pole controller housing 720.

The directional speaker multi-control system includes a directional speaker controller for adjusting the volume of the speaker, a directional speaker amplifier for outputting sound according to the set volume, and a power supply unit for supplying power.

14E to 14H are views showing the features of the directional speaker.

* Main functions of the system

end. Directional speaker system

- automatic sound pressure level control considering the ambient noise situation by acquiring sound sensor information

- Automatic radiative directional angle control, packet tilt control according to the number of objects by acquiring image information and sensor information

I. Mobius server based IDS local control system

- Mobius server and time series database (TSDB), video analysis server (Video Server), disaster monitoring section including sensor and sound source, video, and control module. It is linked with IDS pole enclosure via communication middleware.

- Sensor, sound source, image data collection and analysis, monitoring function

- Control function for directional speaker control

- Control function for camera and infrared IR operation

- Control function for Pan / Tilt operation

- Monitoring function to check the position of the old egg and IDS pole and sensor

- Real-time image check function obtained from camera

- Ability to view real-time sensor data coming from the Mobius server platform

- Past sensor data inquiry function

- Evacuation guidance function (Disaster text transmission) and disaster response function (Moving vehicle disaster alert broadcasting)

All. IDS Integrated Control System

- Connected with installed IDS local control system distributed in various regions

- IDS local control system monitoring function by integrated operation personnel

la. IDS Paul

The IDS pole is equipped with a camera for acquiring real-time image information, an IR LED for night-time imaging, a directional speaker for transmitting alarm broadcast to the old egg, pan / tilt control for targeting the dangerous area around the daily life, and communication middleware It is connected to the Mobius server used as the IDS local control system.

14I is a diagram showing the external shape of the directional speaker.

14J is a communication protocol definition file showing transmission information of the sensor, the camera, and the directional speaker.

System-Specific User Scenarios

1) IDS Integrated Control System

(1) Peak mode

[IDS Integrated Control System Operator]

[IDS Integrated Control System] determines whether or not [System Administrator] is dispatched according to whether [IDS Local Control System] located at each site of [IDS Integrated Control System] is faulty or not.

[IDS Integrated Control System] monitors the operation status of [IDS Local Control System].

The location of the [IDS Local Control System] is displayed as POI on the map of the site that the [IDS Integrated Control System] has jurisdiction. Monitoring is performed through the image and sensor data values of the [IDS camera] operating in [IDS Local Control System]. Make reports on regular and irregular salvage records. Reports are made based on past data (sensor data, image data, salvage records) regularly / irregularly. Directional loudspeaker sources, and disaster response procedures manuals.

(2) Situation generation mode

[IDS Integrated Control System Operator]

If a disaster occurs on the site where the [IDS Local Control System] is installed, the [IDS Integrated Control System Operator] is informed of the location of the [IDS Local Control System] displayed on the map of the [IDS Integrated Control System] The color is changed and the alarm is displayed.

Normal is green, and when a situation occurs, it is indicated by a red blinking.

Click on the POI where the alarm is displayed to identify the situation through the [IDS camera] image of [IDS Local Control System] and share information on the occurrence of the disaster situation to the relevant authorities.

[IDS Integrated Control System Operator] conducts requests for cooperation of relevant organizations and requesting the operation of additional rescue equipment according to the situation.

[IDS Local Control System] When there is a situation that needs to be controlled more than the system, the [IDS Integrated Control System Operator] conducts rescue activities by remote access to [IDS Local Control System].

The remote control of [IDS Local Control System] by [IDS Integrated Control System Operator] takes direct control over the web and takes control of the system and operates it.

The manual of the system control operation according to the emergency situation follows the disaster response procedure manual according to the situation.

(3) System management mode

[system administrator]

[IDS Integrated Control System] manages the control log of [IDS Local Control System] and log / sensor data / image data of [IDS Local Control System]. Perform DB administration (backup and deletion) periodically / irregularly.

Checks and compensates for network problems between [IDS Integrated Control System] and [IDS Local Control System]. [IDS Integrated Control System Operator] performs the inspection and repair work of [IDS Local Control System] at the request of [IDS Integrated Control System Operator].

2) IDS Local Control System: Unattended

(1) Peak mode

[IDS Local Control System]

Continuously shoot the surveillance area through [IDS Camera] of [IDS Paul].

[IDS Camera] automatically takes a certain time to turn left and right by [Pan / Tilt] and shoots the surveillance area.

[Image Analysis Server] uses the sensor data coming from the [Mobius Server Platform] to divide the risk classification of the monitoring area into three stages: [Safety Zone], [Attention Zone] and [Hazard Zone]. do.

It recognizes whether the object moves out of the safe zone into the attention zone or the danger zone according to the risk classification, and when the entering object occurs, the information about the occurrence of the event situation is transmitted to the [control server].

After receiving information on the occurrence of an event situation, the [control server] starts the situation response according to the [disaster response procedure manual]. The [Disaster Response Procedures Manual] provides pre-defined disaster-specific actions by [IDS Integrated Control System Operators].

(2) Situation generation mode

[IDS Local Control System]

Through the IDS camera, it is confirmed that the object enters the [Attention Zone] from [Safety Zone] according to the risk classification of the monitoring region set by the [Image Analysis Server] according to the sensor data received from the [Mobius Server Platform] The captured information is recognized by [image analysis server] and information about event occurrence is transmitted to [local control server]. Then, the object is classified as [old egg] and individual ID is classified.

[Old egg] is positioned at the center of the image of [IDS camera], and is automatically placed in line with [Directional speaker]. When the [oocyte] moves downstream due to torrents, water, etc., the camera automatically rotates along the [oocyte].

In the [local control server] receiving the event occurrence information, proceed according to the [disaster response procedure manual] defined beforehand.

[Local control server] sends an alarm to [IDS Integrated Control System] after receiving event occurrence information.

Control the HW in the [Local Control Server] to operate the pan tilt. [IDS camera] and [Old egg] are aligned so that the alarm broadcast is sent from [directional speaker].

In the [local control server], the rescue operation is carried out according to the [disaster response procedure manual], and a log of all activities from the beginning is stored in the DB.

In the [local control server], it switches to the normal mode after the situation ends.

The end of the situation is the time when the [old egg] is moved to the [safe zone] according to the risk classification of the surveillance area.

[Old egg]

In case of a disaster situation, check the information of the alarm broadcasting through [Directional Speaker] and take evacuation action to [Safety Zone].

(3) System management mode

[system administrator]

Included in [IDS Integrated Control System]

3) Image analysis server

(1) Peak mode

[Image analysis server]

Based on the sensor information received from the [Mobius platform], the risk classification of the monitoring area is divided into [safety zone], [attention zone], and [hazard zone]. [IDS camera] to analyze the event occurrence through the incoming video.

(2) Situation generation mode

[Image analysis server]

[IDS Camera] detects the occurrence of an event through the incoming video, and transmits the information to [local control server].

It recognizes the situation of the object entering the [attention zone] from the safe zone and transmits the information to the [local control server].

It recognizes the situation of an object entering [danger zone] in [attention zone] and transmits the information to [local control server]. The object ID of the detected image and the location information of each ID are also transmitted.

The object ID and location information is received from the [local control server] to display the location of [old egg] on the map.

(3) System management mode

[system administrator]

Included in [IDS Integrated Control System]

32A, 32B and 33 are views showing a monitoring UI screen in the control system.

4) IDS Paul

(1) Peak mode

[CCTV] - Included in [IDS Local Control System]

[Directional Speaker] - Included in [IDS Local Control System]

[Pan Tilt Control] - Included in [IDS Local Control System]

(2) Situation generation mode

Included in [CCTV] [IDS Local Control System]

[Directional Speaker] Included in [IDS Local Control System]

[Pan / Tilt Control] Included in [IDS Local Control System]

(3) System management mode

The system administrator is included in the IDS integrated control system.

end. IDS Local Control System

- Visualize 2D maps of sites with IDS local control system

- Local site hardware location information visualization

- Indicate the position of hardware on 2D map by ID of hardware

- Local site hardware property information visualization

- If you select hardware ID that is visualized on map, attribute information of selected hardware is output

- Visualization of hardware connection status information

- Provides information on the network connection status of the hardware installed in the local site.

- Visualization of old eggs and rescue activities

- Obtain the location information of the rescuer by the camera due to the occurrence of the old egg and visualize the location of the rescuer on the local site map with POI etc.

- Streaming video visualization

- Real-time visualization through camera

- Sensor data visualization

The sensor data received through the Mobius server platform can be visualized according to the user's choice, classified by period, type, and sensor, visualized in the form of tables and charts, tables or charts visualized, (In Excel format)

- Sound source selection information visualization

- Visualize how many sound sources are being emitted from the directional speakers at present, and visualize them by assigning numbers to the sound sources stored in the sound source DB

- Radiation-oriented angle selection visualization

- Visualization of the currently selected directional loudspeakers at several levels of radial orientation

- Sound pressure level selection visualization

- Visualize the sound pressure level of the selected directional speaker

- Appropriate disaster response procedure manual visualization

- According to the situation recognition information that comes from the image analysis module, the disaster response procedure manual stored in DB is selected and visualized according to the event situation and magnetic field suitable manual

2) Software control

- Playback: Playback the past video information stored in the video DB according to the operator's choice

- Sensor data inquiry: sensor can be selected to visualize the sensor data coming from the Mobius server platform by classifying by period, type and sensor

- Disaster response procedure manual setting

- Can modify the situation-related disaster response procedure manual and create new ones

3) External link service

- Integration of IDS local control system and IDS integrated control system

- The system is interlocked so that the local sites of the IDS local control system distributed in various regions can be monitored and controlled by the IDS integrated control system.

4) Hardware control

- Pan / Tilt Control: You can control pan / tilt of camera / directional speaker by using camera / pan / tilt control.

- Directional loudspeaker multi-control: control so that separate sound sources can be output from each of the directional speakers installed at the local site.

- Sound pressure level control: The system can send a sound pressure regulation signal to the amplifier to adjust the sound pressure level of the directional speaker

- Radiation-oriented angle control: can transmit signals to adjust the radiated angle of the directional loudspeaker in the local site system

- Sound source control: After the type of sound source to be transmitted is determined in accordance with the manual disaster response procedure manual, the sound source is transmitted from the directional speaker

- IDS Camera Control

IDS camera zoom in / out can be controlled from the IDS local control system

I. IDS Integrated Control System

1) Monitoring

- Integrated monitoring of IDS local control system

- POI is displayed where the local control system is installed on the map of the area managed by the IDS Integrated Control System

- IDS local control system video monitoring

- If you select the local control system displayed on the map (marked as POI), the image of the IDS camera of the site appears

- Monitoring local hardware connection status

- Monitor the network connection status of the hardware installed in the selected local site

- Local site sensor data monitoring

- Monitoring the sensor data collected using the Mobius platform at the selected local site

- Local site playback monitoring of IDS local control system

- Past video information stored in video database of selected local site can be checked as needed

- Local site disaster response manual monitoring

- Monitor disaster response procedures manuals currently operated at selected local sites

14K is an analysis image emulator screen.

The Mobius server is linked to a GIS server that displays a map of the disaster area on the map and displays a map of the disaster area due to flood or seismic alert due to typhoons, do.

And a smart device (mobile communication terminal, smart phone) connected to the disaster response system through a mobile communication network and equipped with an evacuation guidance app,

If a catastrophic event such as typhoon, tsunami, heavy rains or floods is expected, a disaster warning letter is sent from the disaster response system to the mobile phone numbers of the expected disaster area, It emits a disaster evacuation broadcast of 20Hz ~ 20kHz audio band, transmits the non-audible signal frequency from the fixed directional speaker of the area to the smart device in the area, and evacuates to the safe area through the non-audible signal frequency.

The Mobius server is equipped with a life guard system, an evacuation guidance system, and a disaster response system. The Mobius server is connected to the disaster response system and the SMS server of the mobile communication network. (Smart phone, mobile communication terminal) installed in the disaster-prone area, and outputs a disaster alarm character from the disaster response system to the mobile phone numbers of the smart device in the cell coverage of the disaster- send.

For example, the directional speaker is a speaker amplifier built-in AMP redesigned to improve the sound pressure level output, sound pressure level below 30dB, 750m maximum reach improvement, 50m voice reach improvement, maximum ± 90 ° radiation angle, Manual adjustment of emission angle, image information and sensor connection system, waterproof, dustproof and vibration reduction technology are applied.

1.4 Situational awareness and image tracking

1.4.1 Development of context awareness technology through sensor fusion and hybridization

Object detection and tracking

- Object detection in computer vision is a technique to find out what kind of object is in an image. Therefore, we have to solve two problems of classification and localization.

15 is a diagram showing an example of object detection.

The image analysis server is interfaced with the CCTV camera system, and uses feature map information of the CNN structure among the deep learning techniques for tracking objects in the action-based image, And receives the area information of the object of the previous image as an input, outputs the position information of the current object as a result, and tracks the object in the image.

In order to extract the objects of the image and track the objects of the behavioral cognitive model, the image analysis server extracts the feature maps including the object location area and the category information in the middle several convolutional layers of the CNN structure, Since the size of the map is small, object location area information is extracted from feature maps of different sizes. Therefore, it is possible to detect objects that are robust against scale, and to be associated with CNN learning data stored in the behavior awareness model of the video object. Since the real-time and fast processing speed are important, the SSD (Single Shot Multi-box Detector) or the Faster RCNN algorithm is used.

The image analysis server is interfaced with the CCTV camera system, and uses feature map information of the CNN structure among the deep learning techniques for tracking objects in the action-based image, And receives the area information of the object of the previous image as an input, outputs the position information of the current object as a result, and tracks the object in the image.

In order to extract the objects of the image and track the objects of the behavioral cognitive model, the image analysis server extracts the feature maps including the object location area and the category information in the middle several convolutional layers of the CNN structure, Since the size of the map is small, object location area information is extracted from feature maps of different sizes. Therefore, it is possible to detect objects that are robust against scale, and to be associated with CNN learning data stored in the behavior awareness model of the video object. Since the real-time and fast processing speed are important, the SSD (Single Shot Multi-box Detector) or the Faster RCNN algorithm is used.

The image analysis server uses a SSD (Single Shot Multi-box Detector) or a Faster RCNN algorithm with a high processing speed of object detection in an image.

The image analysis server extracts an object of the image and extracts a feature map including the object location area and the category information in the middle several convolutional layers of the CNN structure in order to track the object of the behavior aware model, In this paper, we propose a method to extract the object location area information from feature maps of different size and to detect the objects that are robust against the scale. And generates an alarm by extracting an abnormal behavior object for a predetermined time.

The IDS control system consists of a number of IDS local control systems and an IDS integrated control system,

The IDS local control system extracts and tracks the object by applying deep learning technology of artificial intelligence by a mobi server and an image analysis server. The IDS local control system traces the object and displays the dangerous accident / drowning accident, It is defined in the learning database that the behavior in the context of the local abnormal situation event is defined in the learning database and the object in the image moves to the accident area or the dangerous area or disappears within the dangerous area and runs, The presence or absence of an object in the image of a dangerous area,

Disaster response The IDS-based human life-protection system sends emergency text messages to security personnel to take safeguards.

Deep learning model for object detection (deep learning model)

- Since AlexNet in 2012, Deep Learning-based object classification technology has continued to evolve and develop to a level far superior to human classification ability. Therefore, it is important to localize the object in object detection. In the initial object detection, all candidate regions in the image were extracted by selective search and sliding window, and the region was classified by passing through the deep learning structure. However, since this approach is significantly slower, a methodology for improving the throughput in object detection has been studied. There are Faster RCNN and Single Shot Multi-box Detector (SSD) as a good deep-running model for object detection.

Both models have a structure in which the localization process is extended or integrated in the process of extracting image features. This structure enables real-time object detection using deep running. In general, the detection performance is slightly better with Faster RCNN and the processing speed is faster with SSD.

- Faster RCNN (Region based Convolutional Neural Network)

16 is a view showing a Faster RCNN.

A region proposal network that extracts candidate regions from the last convolutional layer of the composite neural network (CNN) that extracts features of the image follows. We consider k anchor boxes considering the scale and aspect ratio of an object in one sliding window.

The Faster RCNN is composed of RPN (Region Proposal Network) and merged with the existing Fast RCNN by sharing the convolution feature. Fast RCNN deduces all of the object estimation area through one CNN computation on one image.

Fast RCNN receives all images and objects and acquires CNN feature map for the whole image. ROI (Region of Interest) The pooling layer extracts a fixed length feature vector from the feature map for each entity. Each feature vector becomes a sequence through a FC (Fully Connected) layer and outputs the probability estimate via softmax and the position of the bounding box.

Pooling is a subsampling process that can reduce image resolution by aggregating feature statistics at various locations. Pooling improves robustness to image variations such as rotation, noise, and distortion, while pooling includes maximum- Pooling Two methods are used.

One CNN classifier repeats the convolution layer and the pooling layer, and depending on the structure, additional functional layers can be added. The feature extracted through convolution and pooling for the input image can be classified by applying various classifiers (eg SVM classifier).

Single Shot Multi-box Detector (SSD)

In the middle of the CNN structure, a feature map including object location area and class information is extracted from several convolutional layers. As the size of the feature map becomes smaller as it passes through the pooling layer, the object location information is extracted from feature maps of different sizes, which makes it possible to detect robust objects on a scale (small objects, small feature maps Extracting large objects).

FIG. 17 shows a structure of a single shot multi-box detector (SSD), and FIG. 18 shows an example of object extraction according to feature map sizes in an SSD. (b) an 8 x 8 feature map, and (b) a 4 x 4 feature map in the image of the ROI candidate region of (a).

○ Object detection result - Human object detection

COCO, Pascal The VOC database is a static image and contains relatively large objects in the image. In the CCTV image, the size of the human object is small and the blur due to the movement of the human object occurs. Therefore, additional learning data is needed to compensate. Therefore, we have built up our own database and learned it to make up for its drawbacks.

Since the real-time nature of disaster detection and response is very important, SSD (Single Shot Multi-box Detector) structure is used in order to speed up object detection.

FIG. 19 is a diagram showing an example in which a model learned only by the COCO database is applied to a CCTV image, and a model in which a built database is further learned is applied to a CCTV image.

Object tracking

Object tracking technology is a technology that detects the changed position by following the object detected through object detection technology over several hours.

In the previous research, most of the techniques were used to extract the feature using the region information of the previous frame and to find the region most similar to the feature in the next frame.

Deep Learning Model for Object Tracking

Recently, a method of using feature map information of a CNN structure composed of a multi-layer structure of an input layer, a hidden layer and an output layer as feature information of an object of deep learning technology has been studied . The overall deep-running structure receives area information of an object of a previous image, a current image, and a previous image, and outputs the position information of the current object as a result.

It currently consists of a 5-layer convolution layer and a 3-layer fully_connected layer. In order to learn object tracking, we use a database for object tracking and calculate and calculate the error between the correct position and the output position of the network in the next frame.

The network that has completed the learning can receive the information of the object of the previous frame in cooperation with the object detection network and output the position of the object in the current frame.

20 is a diagram showing a deep learning structure for object tracking.

Object tracking results

FIG. 21 is a diagram showing an object tracking result. FIG.

1.4.2 Development of Seamless Tracking Technology

end. Re-identification technology result

There is a need for a technique for identifying the identity of the detected objects in order to detect and respond to the disasters.

○ CNN structure for person re-identification

- Students learn to identify whether they are the same person or another person by inputting two pairs of images.

- Use a tied convolutional layer with common learning parameters.

- Through the cross-input neigh- borhood differences process using the differences between the feature maps that pass through each tied convolutional layer, features that indicate whether two images are the same or not are extracted.

22 is a diagram showing a CNN (Convolutional Neural Network) structure for Person re-identification.

For reference, the problem of existing Multi-Layer Neural Network can cause three problems such as number of variables, network size, and learning time. For example, if we make a network that recognizes a 16 * 16 size handwriting using MLNN, if the hidden layer is one layer and has 100 neurons, the weight and bias needed for this network A total of 28,326 pieces are required. If you use more hidden layers, the number of parameters becomes very large. In addition, if the character is moved by 2 pixels as a whole, there is a problem that the learning data should be processed as new learning data. In addition, if the size, rotation, and deformation of characters occur, new learning data must be input. Since the raw data is directly processed without considering the shape of characters, a large amount of learning data is required, which leads to a long learning time.

On the contrary, the structure of CNN (Convolutional Neural Network) differs from that of Fully-Connected Layer. In the Fully-Connected Layer, it is implemented as an Affine layer. In CNN, a convolutional layer and a pooling layer are added.

* Convolutional Layer (Conv Layer)

To solve this problem of MLNN, it is a layer of the product multiplication. Image data such as handwritten data and MNIST data are three-dimensional data such as channel, portrait, and landscape. In the Affine layer, the three-dimensional data is input by replacing the one-dimensional data with the one-dimensional data (784 = 28 * 28), but the composite product can maintain the shape by inputting the three-dimensional data (1, 28, 28) . In CNN, such input / output data is called a feature map (Feautre Map).

* Composite Product Layer - Operation

Operations are performed in the product multiply layer. The shape of the data and the filter (height, width) is called window. For example, the input data is (4, 4), the filter is (3, 3), and the filter is the Conv layer weight.

The composite product operation is performed by moving the window of the filter (window, window size example (3, 3)) at regular intervals. The resultant product is multiplied by the corresponding elements between the input data and the filter, and the sum is obtained. This is called Fused Multiply-Add (FMA). The deflection is added after the filter is applied.

* Composite Product Layer - Padding

Padding refers to filling the input data with a specific value before performing the composite product operation. Padding is mainly used to adjust the spatial size of the output data. When padding, the value to be filled can be determined by the hyperparameter. It mainly uses zero-padding.

The reason for using padding is that if the padding is not used, the spatial size of the data becomes smaller every time passing through the Conv layer, so the padding is used because the edge information is lost.

* Composite Product Layer - Stride

Stride is the interval at which the filter moves, when it applies the filter to the input data. The stride is also used to adjust the size of the output data. Stride usually works better with smaller values such as 1, and when Stride is 1, the spatial size of the input data can be adjusted only in the pooling layer.

* Composite Product Layer - Output Size Calculation

Apply the padding and stride, and obtain the size of the output data when the size of the input data and the filter is given.

* Pooling Layer

The composite product layer was studied. Another layer of CNN, the Pooling Layer, is described.

The pooling layer is used to reduce the spatial size of data such as padding and stride of the composite product layer. Mainly, in the Conv layer, the size of the output data is kept as the input data size, and the size is adjusted only in the pooling layer (Pool: Max-Pooling and Average pooling. In the pooling layer, the window size and the stride value of the pooling are set to the same value, and the average value of the window size and the stride value of the pooling are set to the same value .

○ CUHK-03 database experiment result

It has an accuracy of 84.6% for randomly extracted validations.

23 is a diagram showing the CUHK-03 database (the same person in the same row).

1.4.3 Big Data Acquisition and Testing

24 is a screen for acquiring a big data database for object detection.

- Use of public databases for object detection:

Common Objects in Context Dataset (COCO), Pascal Visual Object Classes (VOC)

- Acquisition of database using black box or CCTV video:

- Correction after object detection using a pre-trained deep learning model: elimination of erroneous detection area and designation of undetected area

Fig. 25 is a picture for erasing the erroneous detection area.

* When acquiring the image, it is a movement accident

a. Disappearance Action demonstration (seating, right side, rear side, front side) 5 or more

- disappeared when swimming

- Thigh level suddenly disappeared after walking slowly (not seen by diving)

- Breast water level slowly disappeared (not seen by diving)

b. Flooding

- Situations in which the face floats while being submerged

- A situation where the face is immersed in water while immersed in water

- Situations where the face is floating in the sky

c. Tube submersion situation

- Suspension of the surface with the tube inserted into the body

- Situation in which the tube is attached to the body and the frontal submerged arm is moved

- Situation in which the tube is put on the body and the upper body is locked and only the lower body moves

d. One person suffers flooding after two waters)

e. Tube submergence situation after two waters

f. One danger zone wandering (by each person in each direction)

- ankle wandering

- waist circumference

- Chest Wandering

g. More than four people each wandering in danger zone (overlapping, one dive disappears)

- 1 wandering ankle suddenly disappeared

- 1 waist wandering suddenly disappeared

- One person suddenly disappeared.

26 is a photograph of a river viewed from a CCTV camera.

FIG. 27 is a picture of a CCTV camera taken in a dangerous area for prohibiting entry.

FIG. 28 is a photograph of a CCTV camera taken at a private spot and moving in a danger zone at the sunset.

* Define behavior in various situations according to abnormal situation event, move to dangerous area

* Selection of a deep running model to recognize actions such as disappearing, running, walking, and falling within a hazardous area

- CNN + LSTM based model

- 3D CNN based model

- Performance test of behavior awareness model

* Added detailed description for re-identification development

- Feature extraction is performed by passing the human region extracted through the object detector of the CCTV camera image to the input of the CNN. Using the extracted features, the presence of the same person is estimated through learning.

* Development of base technology for Seamless Tracking technology

- Deep learning model for object detection

- Accuracy priority model: InceptionResnetV2 based FasterRCNN

- Speed Priority Model 1: MobileNet-based SSD

- Speed Priority Model 2: YOLO_V2

- Deep learning model development and testing

- Learning object detector using existing data

- Camera interlock and detection speed measurement

   InceptionResnetV2 based FasterRCNN: about 800ms

   MobileNet-based SSD: about 35ms

   YOLO_V2: about 40ms

1.5 Social disaster utilization strategy of IDS system: Water safety accident

We use CCTV, sensors, and directional loudspeaker systems to prevent accidents in typhoons, tsunamis, tsunamis, floods, water accidents, drowning accidents, hazardous area and accidents. It monitors and monitors the environment and the situation of the accident occurrence site using CCTV camera, and collects and analyzes information of Mobius server using various sensors through IoT-based sensor / sensor network. It monitors and warns the danger zone of torrential rain, Mobius server analyzes the big data based on the collected data to classify the type of disaster, analyzes existing cases of accidents, and analyzes camera image information and sensor data. It will be used as a disaster response system and a human life protection system by quickly responding to disaster situations by analyzing big data and identifying the characteristics of the disaster site.

1.5.1 Analysis of water safety accident management status

end. Characteristics of water safety accident

According to the Ministry of Public Administration and Security, there have been 285 water-related accidents and 29,382 drowning cases in Korea during the past seven years (2009 ~ 2015). Especially, water safety accidents are concentrated in June ~ August, and other drowning accidents occur in July and August. In other words, it is more important to focus on summer management. In order to evaluate the effective management of water hazard accidents, we conducted an opinion survey on the local 119 center personnel, medical staff, and aquaculture specialists who are highly relevant to the actual accident victims. As a result, in the case of a succumbing accident, if a drowning person does not receive first aid within two to three minutes, the risk of death is very high. In addition, if the 119 emergency crews arrive at the scene after receiving the notification, the time required for the arrival of the emergency crew is at least 10 minutes at the earliest. In addition, the water depth at which drowning accidents occurs is unexpectedly high, and it is pointed out that it is difficult to distinguish between the submersion and the water of the general public. The details of the interviews were examined as shown in Table 4.

Recently, municipalities and regional disaster headquarters have been concentrating on manpower and equipment centering on accidents involving frequent accidents in the summer. In addition, there are cases where people who are in short supply are being supported in connection with local volunteer organizations. As a result of placing workers first in the area with frequent accidents, the number of deaths due to recent water-related accidents has decreased significantly compared with the past. Recently, 30 to 40 people.

The most frequent occurrence of water-related accidents in summer is Gapyeong-gun, Gyeonggi-do, followed by Namyangju-si, Gyeonggi-do. The two areas are geographically close to Seoul, and many resorts, including mountains, rivers, and valleys, have beautiful scenery. The Gapyeong - gun decreased drastically the number of deaths and deaths due to various efforts to reduce the number of waterfalls and accidents, resulting in only eight deaths per year. It is necessary to manage the accident management system of the local governments and disaster response. Every year, there were deaths, and in Gapyeong-gun, it was designated as a no-access zone, preventing accidents.

As a result, the rescue experts point out that it is very difficult to escape because of the water flow that pulls down even if the swimming ability is good. Therefore, although there is a difficulty in changing the river bed every year, investigating such dangerous areas in advance and setting up signs and mapping them will greatly help prevent and manage accidents.

29 is a photograph including a gas sensor having a solar panel, a water level gauge, a directional speaker and a camera. FIG. 30 is a photograph in which the emergency alert broadcast directional loudspeakers, rescue workers and security personnel reside.

FIG. 31 is a photograph in which an accident occurrence area, a dangerous accident dangerous area and a specific area are provided with a safety sign and set as a prohibited area.

Water rescue equipments are equipped with life saving rescue, watering hazard area sign, water prohibition (area) sign, water warning sign, death area sign, drowning hazard area sign, danger zone designation sign, The local governments are subdividing the areas where the accidents have occurred and the dangerous areas, and signs are installed to inform the people of the risk of watering.

In addition to water-related accidents, there are many accidents occurring during activities such as body-wrecking and fishing, and when there are many rapids due to mountains and valleys, many accidents occur across rivers. In such places, CCTV cameras are installed and monitored because there are many difficulties in management by alarm system that warns of danger of occurrence of rapids.

* IDS system development and utilization direction

○ In the National Park Management Corporation and Gapyeong-gun which investigated in the examples, various signboards were installed to prevent water-related accidents and speaker systems for alarm broadcasting were installed and operated. In addition, although CCTV camera system is used in places where some monitoring is needed in accidents and dangerous areas, it is rare that CCTV is used as a total cost of accidents. In addition, up to now, the system applied to the field has not been able to use bidirectional communication and control or similar system using video, sensor and directional speaker like IDS system.

  Through expert advice, it was found that it was very difficult to clearly identify the person who was in the water by using only the image information, but patterns that could identify some submerged persons were investigated. For example, if a person is located below the water surface for more than 10 seconds, it can be judged that the possibility of an accident is high. Therefore, it is effective to enhance functions such as warning about access to accident area and dangerous area, and judgment of accident occurrence through object tracking based on cognitive behavior.

Even if an accidenter is found, rescue and emergency measures must be taken promptly. In the case of a watering accident, the time from accident to death is very short. Therefore, it is necessary for the IDS control system to monitor the accident in the field and notify the danger, and to take active warning broadcast and early disaster response measures.

In order for the IDS control system to be applied to the field, it is necessary to solve the problem that existing CCTV and sensors are subject to a great restriction on the installation and operation according to the protection of personal information. Also, in case of installing in a specific area, regulatory reforms such as improvement of related laws and regulations and legal system are necessary for local governments and related organizations to apply themselves according to disaster safety and necessity such as Natural Park Law.

Embodiments according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The computer-readable recording medium may be any type of storage medium such as magnetic media such as storage, hard disk, floppy disk and magnetic tape, optical media such as CD-ROM, DVD, magnetic recording media such as floppy disks, Any type of hardware device configured to store and perform program instructions, such as magneto-optical media, and ROM, ROM, flash memory, and the like, may be included. Examples of program instructions may include what is produced by the compiler, machine code, as well as high-level language code that may be executed by a computer using an interpreter. Such hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood that various modifications and changes may be made without departing from the scope of the present invention.

100: Sensor 300: Mobius server
700: IDS Local Control System 711: IDS Local Control Server
712: Streaming server 714: Image analysis server
720: IDS pole controller enclosure 723: directional speaker controller
724: Directional speaker amplifier 730: IDS pole housing
731: Camera 732: IR LED
733: Directional speaker output section

Claims (15)

A CCTV camera system having at least one camera and a digital video recorder (NVR) and image analysis software;
At least one IoT-based sensor for transmitting sensor data that detects a disaster situation; The sensor packet data sensed by the IoT-based sensor is transmitted to a sensor network using any one of a wireless sensor network (USN, WSN, 6LoWPAN, MQTT), a Wi-Fi sensor network using NB-IoT, a mobile communication network (LTE) An IoT gateway for receiving the protocol over the network and transmitting the converted protocol to the server; And
An IoT-based sensor for collecting and storing sensor data sensed from the IoT-based sensor through sensor network and collecting sensor data and analyzing big data, An image analysis server connected to the network and connected to the CCTV camera system and detecting an object of a deep learning based image of artificial intelligence to track a disaster situation image analysis / situation recognition / image object, And an IDS control system that analyzes the type-by-type big data to take disaster response measures,
The IDS control system consists of a number of IDS local control systems and an IDS integrated control system. The IDS local control system is composed of a camera, sensor network and directional speakers The guard system. The method according to claim 1,
The sensors use a sensor network (USN, WSN, 6LoWPAN, MQTT), a Wi-Fi sensor, a water level sensor, a flow velocity sensor, a wind speed sensor, a microphone sensor installed in a dangerous area, A water level, a river level, and a level of a sensor data to a name-keeping system and a disaster response system of the Mobius server through a gateway through at least one of a Wi-Fi wireless LAN, a mobile communication network (LTE-M), an NB- Speed, velocity information, microphone sensor information (screaming sound), fire detection information, gas detection information, and human body detection information,
The Mobius server is a camera and sensor network based on fusion technology for social disaster response, which provides an event detection function based on a screaming sound using a microphone sensor, and a human life protection system using a directional speaker. The method according to claim 1,
The at least one IoT-based sensor
A sensing unit sensing sensing information;
A controller for controlling each function for sensor measurement;
A storage unit connected to the control unit and storing sensor data;
Sensor data is transmitted through a communication protocol of any one of wireless sensor networks (USN, WSN, 6LoWPAN, MQTT), Wi-Fi, sensor network using NB-IoT, mobile communication network (LTE), LoRa network and Ethernet A wireless communication unit;
A protocol stack section for providing an IoT protocol; And
Device driver; And a power supply unit for supplying power to the sensor node,
And may further include a sensor adapter for linking the IoT-based sensor network communication protocol,
The sensor data includes a camera based on fusion compound technology for social disaster response including a sensor node ID, a sensor device ID, and a sensor data value, a sensor network, and a life guarding system using a directional speaker. The method of claim 3,
The continuous power supply unit includes a solar-battery-based camera for coping with social disasters, which includes a solar cell array, a charge controller, a DC-DC converter, and a battery, and further includes a solar cell unit that supplies a predetermined rated power using solar heat. Human - security system using sensor network and directional speaker. The method according to claim 1,
The directional loudspeaker comprises a fixed variable directional loudspeaker (fixed directional loudspeaker) fixed to a ground pole and / or a variable directional loudspeaker (mobile directional loudspeaker) fixed to a pole of a moving vehicle,
The directional loudspeaker is capable of adjusting the radiation angle and the pan / tilt angle. The motor control signal receiver and the motor adjust the stepwise radiation angle of the directional loudspeaker, And is a directional speaker capable of adjusting a radiation angle and a pan / tilt angle provided in each room for outputting a band signal (non-audible signal)
And a directional loudspeaker multiplex control system for controlling the angle of the directional loudspeaker and the pan / tilt control system. 6. The method of claim 5,
The directional speaker multi-control system includes a directional speaker controller for adjusting a radiation angle and a pan / tilt of a PTZ controller equipped with a camera and an IR LED and adjusting a volume of a speaker, a directional speaker controller for outputting sound according to the volume set by the directional speaker controller, A speaker amplifier, and a power supply unit for supplying power, a camera and a sensor network based on a fusion technology for social disaster response, and a life guarding system using a directional speaker. The method according to claim 1,
The directional speaker is designed as a speaker amplifier built-in type AMP. It improves the sound pressure level output, so it has a sound pressure level of 90dB or less, 750m maximum reachable distance, 80m improvement in voice reach distance, maximum ± 90 ° radiation angle, automatic radiation angle adjustment system, , A camera system based on a fusion technology for social disaster response, and a human life protection system using a sensor network and a directional speaker, in which voice information linked with image information and sensor information is provided and a transparent waterproof cover is used. The method according to claim 1,
The image analysis server is interfaced with the CCTV camera system and uses feature map information of a CNN structure among deep learning techniques for tracking an object in a motion recognition based image, , Camera and sensor network based on fusion technology for social disaster response, which receives area information of object of previous image as an input and outputs the position information of current object as a result to track an object in image, The guard system. 9. The method of claim 8,
The image analysis server extracts an object of the image and extracts a feature map including the object location area and the category information in the middle several convolutional layers of the CNN structure in order to track the object of the behavior aware model, Feature The size of the map is small, but because it extracts the object location area information from feature maps of different sizes, it detects a strong object on the scale. It is linked with the CNN learning data stored in the behavior awareness model of the image object, And correspondence is important because it is real time and fast processing speed. Therefore, we use SSD (Single Shot Multi-box Detector) or Faster RCNN algorithm, Life-saving system using network and directional speakers. The method according to claim 1,
The IDS control system comprises a plurality of IDS local control systems and an IDS integrated control system,
The IDS local control system includes a mobius server, an image analysis server, a IDS-based IDS-based life support system, an evacuation guidance system for transmitting disaster announcement texts to nearby mobile communication terminals and a directional speaker , A directional loudspeaker multiplex control system,
Receiving an un-audible signal from a directional speaker connected to the disaster response system and an SMS server of a mobile communication network, and receiving and outputting the non-audible signal to a smart device equipped with an evacuation guidance app, A camera and sensor network based on fusion technology for social disaster response, and a human life protection system using directional loudspeakers, in which a disaster alert character is transmitted to mobile phone numbers in cell coverage of the disaster prediction area through a server. 11. The method of claim 10,
An evacuation guidance app for outputting an audible signal / non-audible signal from the directional speaker linked with the evacuation guidance system to the speaker of the smart device and guiding the evacuation to the safe area through the location of the disaster area and the disaster announcement character A camera and sensor network based on fusion technology for social disaster response, transmitted to a smart device, and a life guarding system using directional speakers. 11. The method of claim 10,
The navigation system is connected to the evacuation guidance operating system through a mobile communication network and outputs an audible signal / non-audible signal from the directional speaker coupled with the evacuation guidance system to the speaker of the smartphone. Further comprising a smart device having an evacuation guidance app (App) provided with a guidance path,
The smart device uses a smartphone and receives a notification of evasion from the mobius server through an evacuation-inducing app for a smartphone (an Android-based app for detecting a directional speaker-based sound source direction detection) A camera and sensor network based on fusion technology for disaster response and a human life protection system using directional loudspeakers. 11. The method of claim 10,
The Mobi server is a part of the IDS local control system and transmits various data including video, voice and sensor data in cooperation with the IDS integrated control system,
The IDS local control system includes a monitoring module and a control module for monitoring video, sensor data and sound source data of the Mobi server, a time series database (TSDB), and an image analysis server (Video Server) , A sensor, a sound source, an image data collection and analysis and monitoring function, an object of a deep learning based image of the artificial intelligence by the image analysis server, a disaster situation image analysis / Control function for directional speaker control, control function for camera and infrared IR operation, control function for pan / tilt operation, spherical egg and IDS pole, monitoring function for sensor position confirmation, real-time image confirmation and analysis obtained from camera Function to check real-time sensor data received from Mobius server, display past sensor data Function and disaster prevention function (disaster text transmission) and disaster response function (moving vehicle disaster) in conjunction with the emergency evacuation guidance system which induces the evacuation by transmitting the disaster letter and the disaster response system which provides the moving vehicle emergency alert broadcasting through the directional speaker Alarm broadcast), provides IDS pole interlock function,
The IDS integrated control system is connected to a plurality of installed IDS local control systems distributed in various regions, receives disaster situation information from each IDS local control system, and provides an IDS local control system monitoring function. A camera and sensor network based on fusion technology, and a human - life - protection system using directional speakers. 14. The method of claim 13,
The IDS pole includes a camera for acquiring real-time image information, an IR LED for night image capturing, a directional speaker for transmitting an alarm broadcast to the old egg, pan / tilt control for targeting a dangerous area around the daily life, A camera and sensor network based on a fusion technology for social disaster response, and a life guarding system using directional loudspeaker, connected to the Mobi server which is used as a part of the IDS local control system. 15. The method of claim 14,
The IDS local control system includes the IDS pole including the control system, the IDS pole controller housing and the IDS pole housing,
The control system includes an IDS local control system server (Mobius server), a streaming server, and the image analysis server interlocked with the CCTV camera system,
The IDS pole controller includes a directional speaker controller for controlling a PTZ controller mounted with a camera and IR LED and controlling a radiation angle and a pan tilt, a directional speaker controller for adjusting a volume of the directional speaker, a directional speaker amplifier for outputting sound according to the set volume, And a power supply unit for supplying the power supply unit,
The IDS pole saving unit is connected to the IDS local control system server (Mobius server), the PTZ controller connected to the directional speaker controller, the camera connected to the PTZ controller and the IR LED,
And a directional speaker output unit interlocked with the directional speaker amplifier of the IDS pole controller enclosure.

Images