KR20210144455A - Convergence monitoring system based on real-time visualization technology of sensor data and image data - Google Patents

Abstract

The present invention provides a convergence monitoring system based on a real time visualization technology of sensor data and image data, comprising: a sensor unit (30) having a plurality of sensors; an image monitoring apparatus (10) having a camera (110) and an on-screen display (OSD) module (120) that overlays and outputs detection data of the sensor unit (30) on an image captured by the camera (110); a control server (40) that receives, from the image monitoring apparatus (10), video monitoring information overlaid with the detection data of the sensor unit (30) and analyzes the received detection data to detect whether a disaster situation occurs, and to issue a warning; a disaster prevention facility control panel (20) connected to the image monitoring device (10) by wire to control a disaster prevention facility; and a mobile terminal (50) receiving a disaster situation warning from the control server (40) and outputting the warning. The present invention makes management and maintenance easy by enabling a control server to directly identify an operation state of a disaster prevention facility.

Description

CONVERGENCE MONITORING SYSTEM BASED ON REAL-TIME VISUALIZATION TECHNOLOGY OF SENSOR DATA AND IMAGE DATA}

The present invention relates to a convergence control system and method based on real-time visualization technology of sensor data and image data.

In the case of a general control system, by installing a sensor together with CCTV, etc., the administrator can check the image and the detection value detected by the sensor, thereby monitoring a specific area as a whole and preventing accidents such as fire and crime.

In such a control system, the manager must directly check the collected data, and a manpower who checks the control system must always be there.

However, even if the manpower is resident and the control system is checked in real time, concentration is not always maintained, and the reality is that accidents continue to occur.

That is, in the related art, as a communication network between a monitoring image device and a control system, and a sensor and a control system, is separately constructed, false detection of a disaster situation such as a fire may occur.

Therefore, conventionally, since the sensor network and the surveillance video network are separately constructed, there is a high possibility of false detection of the daughter, and there is a problem in that labor costs for hiring a manpower and cost for management and maintenance are additionally required to operate it.

Republic of Korea Patent Publication No. 10-2012-0050073 (published on May 18, 2012)

Therefore, the present invention has been devised to solve this problem, and an object of the present invention is to integrate and manage a sensor network and an image network, thereby preventing a false detection rate based on real-time visualization technology of sensor data and image data. An object of the present invention is to provide a convergence control system and a method therefor.

In addition, an object of the present invention is to intuitively visualize sensor data on image data so that visibility can be expanded, and propagation of a disaster situation can be quickly made, so that a quick response in case of a disaster situation is possible in real time of sensor data and image data. It is to provide a convergence control system and method based on visualization technology.

The present invention includes the following examples to achieve the above object.

An embodiment of the present invention is an image monitoring apparatus having a sensor unit having a plurality of sensors, a camera, and an OSD (On Screen Display) module for overlaying and outputting sensing data of the sensor unit on an image captured by the camera And, a control server that receives video monitoring information overlaid with sensor data from the video monitoring device, and analyzes the received detection data to detect and alert the presence of a disaster, and a video monitoring device wired to connect to disaster prevention It includes a disaster prevention facility control panel that controls the facility and a mobile terminal that receives and outputs a disaster alert from a control server, wherein the control server compares the change rate of the measurement sensor and the measurement value included in the detection data, and the measurement value and the external environment data A data analysis unit that compares and calculates a result by comparing the measured values of one or more linked sensors linked to the measurement sensor with the change rate of the linked sensor, a visualization module that graphically processes the analysis result of the data analysis unit, and the graphic Disaster prevention according to the analysis result of the interlocking control unit that maps the processed analysis result to the location information of the corresponding sensor and the image, the image display control unit that outputs the image monitoring information including the graphic processing analysis result of the visualization module to the display, and the data analysis unit It may include a convergence control system based on real-time visualization technology of sensor data and image data including a diagnostic processing unit that transmits an operation command to the facility control panel and receives and outputs operation state information of the disaster prevention facility through an image monitoring device. .

Therefore, according to the present invention, the sensor network and the image network can be managed in an integrated manner, and the operation status of the disaster prevention facility can be checked directly from the control server, so that the management and maintenance are easy.

In addition, the present invention analyzes the detection data of the sensor overlaid on the image and expands the visualization range by processing the analysis result graphically, so that control and monitoring are easy.

1 is a diagram showing a schematic configuration of a convergence control system based on real-time visualization technology of sensor data and image data according to the present invention.
FIG. 2 is a block diagram illustrating the OSD module of FIG. 1 .
3 is a flowchart illustrating a convergence control method based on real-time visualization technology of sensor data and image data according to the present invention.

The terms or words used in the present specification and claims are not to be construed as limited in their ordinary or dictionary meanings, and on the principle that the inventor can appropriately define the concept of the term to describe the user's invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention.

Throughout the specification, when a part "includes" a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, the “… wealth", "… energy", "… However, terms such as “module”, “device” and the like mean a unit that processes at least one function or operation, which may be implemented as a combination of hardware and/or software.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing a schematic configuration of a convergence control system based on real-time visualization technology of sensor data and image data, which is an embodiment of the present invention, and FIG. 2 is a block diagram illustrating an OS module.

1 and 2 , a convergence control system based on real-time visualization technology of sensor data and image data includes an image monitoring device 10 having a camera 110 and an OSD (On Screen Display) module 120 and , a sensor unit 30 composed of a plurality of sensors, a control server 40 for alerting whether a disaster has occurred using data received from the video monitoring device 10, and a control server 40 and/or video monitoring and a mobile terminal 50 for receiving the alert and monitoring information transmitted from the device 10 .

The sensor unit 30 includes at least one of heterogeneous sensors, for example, a sound sensor, an illuminance sensor, a human body sensor, a temperature and humidity sensor, an ultrasonic sensor, a proximity sensor, a flame sensor, and a gas sensor. Here, each of the sensors may be composed of a fixed sensor built into the camera 110 , a USB type sensor that is equipped with a USB and detachable, and a communication type sensor that can communicate via wire or wirelessly.

The video monitoring device 10 transmits monitoring information in which the video data and the sensor data are fused from the remote control server 40 as a plurality. Specifically, the image monitoring apparatus 10 fuses the sensor data sensed by the sensor unit 30 and the image data captured by the camera 110 to visualize the sensor data in the image and transmit it to the control server 40 .

In addition, the video monitoring apparatus 10 may be connected to communication with the disaster prevention facility control panel 20 to output an operation command or request status information for confirming the operation state of the disaster prevention facility.

To be more specific, the video monitoring apparatus 10 includes a camera 110 and an OSD module 120 that fuses sensor data and image data, and the OSD module 120 is built-in to the camera 110 and includes a sensor. The sensor data of the unit 30 is overlaid on the image data of the camera 110 and transmitted to the control server.

The OSD module 120 may visualize the sensor's detection data in the image captured by the camera 110 and transmit it to a remote control system. Specifically, the sensor interface 123 for amplifying and converting the sensed data of the sensor unit 30 and the protocol converter for converting the sensed data of the sensor unit 30 output from the sensor interface 123 into standard packets ( 124), a packet data storage unit 126 for storing packet data, an image interface 122 for receiving and amplifying images of the camera 110, and converting data converted into standard protocol packets into text to capture images The overlay control unit 125 for displaying on the screen, the communication unit 128 for outputting the OSD converted image overlaid with text, the control unit 121 for controlling each configuration, and the disaster prevention facility control panel 20 to provide detection information and an external interface 127 that outputs.

The image interface 122 receives and converts image information captured by the camera 110 and outputs it to the overlay controller 125 and/or the communication unit 128 under the control of the controller 121 .

The packet data storage unit 126 stores the packet data converted by the protocol converter 124 . The stored packet data may correspond to sensor data converted into text.

The control unit 121 controls the on/off of the power, grouping a plurality of sensors and setting a channel for each group for displaying data of the sensors for each group, and a channel according to a set command or a command received from the outside and/or a set condition. The overlay control unit 125 and the communication unit 128 are controlled to convert

In addition, the control unit 121 sets at least one sensor among the plurality of sensors as one group, sets channels for each group, and converts and encodes only data of sensors belonging to the corresponding group for each channel by OSD (On Screen Display). to control to transmit to the control server (40).

Here, in the image for each channel, only data of sensors belonging to a corresponding group are expressed in the same image. For example, the images of the first channel and the second channel are the same, but the sensor data displayed on the screen are different. That is, in the first channel, data of sensors belonging to a set first group are displayed, and in the second channel, only data of sensors belonging to a set second group are displayed. Therefore, the monitoring apparatus 10 of the present invention outputs the same image as two channels, and each channel includes different data according to a sensor belonging to a corresponding group.

The control server 40 receives the video for each channel in real time, stores it and/or outputs it to the monitor. For example, the control server 40 classifies and stores the received image for each channel, but outputs the image of the selected channel.

It is preferable that the channel setting of the controller 121 is performed whenever new additions and changes of a plurality of sensors are performed after initialization. Therefore, in the present invention, even if a plurality of sensors are added, different channels are set for each group and the OSD image including different data for each channel is output on the screen, so the screen is not complicated and the expansion and reduction of sensors is easy. have

The sensor interface 123 converts (A/D or D/D conversion) and amplifies the sensed data received from the fixed sensor, the USB type sensor, and the wireless sensor, and outputs it.

The protocol converter 124 signal-processes and corrects the digital signal input from the sensor interface 123, converts it into a standardized protocol, generates a standard protocol packet, and overlays the sensed data signal processed into the standard protocol packet to the overlay control unit 125 can be sent to

Specifically, the protocol converter 124 signals the sensed data received from each sensor as a standardized protocol to generate a standard protocol packet. For example, each sensor outputs data having different protocols according to manufacturers and specifications. Accordingly, the protocol converter 124 generates a standardized standard protocol packet by changing data of different protocols into a standardized protocol so that the data have the same rule.

Preferably, the protocol converter 124 transmits the generated standard protocol packet to the overlay control unit 125 through serial communication. That is, the protocol converter 124 may create a plurality of channels to form a channel for each group.

The overlay control unit 125 encodes the received sensor detection data for each channel to be displayed on the image of the camera 110 and outputs it to the communication unit 128 . Here, the overlay control unit 125 designates the display position of data in the image of the camera 110, converts the sensor's sensed data into a bitmap, and displays the data at the designated position.

The control unit 121 transmits the image including the text of the overlay control unit 125 to the control server 40 , and when abnormal data detection information is transmitted from the control server 40 , the disaster prevention facility control panel through the external interface 127 . The error information is transmitted to (20).

That is, the present invention can be connected to communication with the disaster prevention facility control panel 20 through the OSD module 120, and the disaster prevention facility operation command to the disaster prevention facility control panel 20 is output from the control server 40 and the mobile terminal 50 It is also possible to output a disaster prevention facility operation command from the OSD control unit.

The disaster prevention facility control panel 20 is, for example, a device for controlling any one of a fire extinguishing device, a ventilation device, a water supply device, an alarm device, an elevator driving device, a lighting device, and a door locking device, and an operation command from the OSD module 120 When this is received, the disaster prevention facility is controlled.

In addition, the disaster prevention facility control panel 20 can also transmit the operating states of the connected devices to the control unit 110 . The operation state information of such disaster prevention facilities may be transmitted to the control server 40 and/or the mobile terminal 50 by the control unit 110 .

The control server 40 analyzes the sensor data and the image data received from the plurality of image monitoring devices 10 to analyze an abnormal situation and issue an alarm. To this end, the control server 40 interlocks the data analysis unit 410 for analyzing the sensor data and the image data, the visualization module 430 for graphic processing the analyzed sensor data, and the mapping of the analysis result and the image and the electronic map. It includes a control unit 440, an image display control unit 460 for controlling image display, a DB 420 for storing information, and a diagnosis processing unit 450 for diagnosing the operating states of sensors and disaster prevention facilities.

The data analysis unit 410 analyzes the sensor data and the image data received from the image monitoring apparatus 10 to calculate a change rate. The rate of change includes the range of maximum and minimum measured values at steady state by accumulating and averaging the measured data. Here, the sensor unit 30 measures and transmits not only indoor but also ambient environment data. Accordingly, the data analysis unit 410 may calculate a change rate by accumulating indoor and outdoor environmental data.

The data analysis unit 410 checks the analysis result to determine whether there is an abnormality, and outputs an operation command of the disaster prevention facility and an alarm command signal.

Here, the data analysis unit 410 sets the measurement factor in association with other factors, checks both the related factors and the measured factors, and determines whether there is an abnormality. For example, fire can be judged by combining all three factors: flame, smoke, and temperature. For example, if the temperature is higher than the set condition as the fire changes the temperature, it is checked whether smoke and flame are detected as a factor related to the temperature, and the rate of change calculated through the accumulated measurement values of each entertainment factor is compared. to calculate the result.

The diagnosis processing unit 450 receives and stores the operation state information of the sensor and the operation state of the disaster prevention facility, and controls to output the result to the display and transmit the result to the mobile terminal 50 .

The visualization module 430 expresses information such as a change rate in the image monitoring information of the OSD module 120 in which text is overlaid on the image as a bar-shaped or curved graph, or an abnormal situation occurs in the diagnosis processing unit 450 . According to the pop-up function for an alarm message according to , or the set range of the measured value of sensor data, graphic processing is performed so that it can be expressed in different colors.

The interlocking control unit 440 checks the location information of the video monitoring device 10 to map the electronic map and image information. For example, the interlocking controller 440 maps the location where the image is captured on the electronic map, and maps the graphic-processed analysis result of the visualization module 430 to the image including the corresponding sensor.

The image display control unit 460 outputs the graphic processing result of the warning and visualization module 430 according to the occurrence of an abnormal situation on the display. Here, the image display control unit 460 provides an image display application installed in the mobile terminal 50 to output monitoring information.

DB 420 includes sensor information including sensor IDs installed by space/location, a table for pre-processing and storing data detected by the sensor, a table for classifying message types, and raw data collected from the human body sensor. A sensor management DB that includes a table for storing data, a statistics DB that stores statistical values of data (variation rate, period, month, and time), a device management DB that stores disaster prevention facility information, and map data It may include the stored map data DB.

In addition, the control server 40 performs a network communication module for communication with a plurality of video monitoring devices 10 and a plurality of mobile terminals 50, a statistical analysis module for statistical analysis, and inquiry and search of information. It may further include a monitoring application that can implement a search module for, visualization of statistical data and a web page of the server.

Hereinafter, a convergence control method based on real-time visualization technology of sensor data and image data according to the present invention using the above configuration will be described with reference to the accompanying flowchart.

3 is a flowchart illustrating a convergence control method based on real-time visualization technology of sensor data and image data according to the present invention.

Referring to FIG. 3 , the present invention provides a step S100 for receiving monitoring information and environmental data from a plurality of video monitoring devices 10 , a step S200 for comparing a sensor value change rate and external environmental data with a measured value, and a linkage factor It includes a step S300 of comparing the detection value and the change rate of the sensor value, a step S400 of mapping the analysis result to an image, a step S500 of analyzing whether a disaster has occurred, and a step S600 of issuing an alert.

Step S100 is a step of receiving detected data from the plurality of video monitoring apparatuses 10 . Here, the control server 40 receives both indoor and surrounding environment data from the plurality of video monitoring devices 10 .

Step S200 is a step of checking the rate of change of the measured sensor value and comparing it with the measured value. The data analysis unit 410 compares the measured value sensed by the sensor with the rate of change of the measured value in a normal state, and compares both the measured value and external environmental data. For example, the data analysis unit 410 checks whether the measured temperature does not fall within the range of the rate of change, or is different from the external temperature (or the temperature of another surrounding space) by more than a set reference.

In step S300, if the measured value does not match the rate of change and the environmental data, the control server 40 measures the linked factor linked to the measured value, and compares both the measured value and the rate of change. For example, if the measured value of the temperature is different from the set change rate and the external or ambient temperature, the data analysis unit 410 checks the measured values of flame and smoke linked to the temperature. And the data analysis unit 410 compares the fluctuation rates of the flame sensor and the smoke sensor and the measured values of the flame and the smoke.

Step S400 is a step of mapping the measurement result to the image transmitted from the video monitoring apparatus 10 . Here, the visualization module 430 graphically processes the analysis results of steps S200 and S300 as graphs such as a bar type or a curve type. In this case, the interlocking control unit 440 displays information on the location where the corresponding sensors are installed on the electronic map, and maps the graphic processing information of the visualization module 430 to an image including the location of the corresponding sensor. The image display control unit outputs graphic processing information of the visualization module 430 to the display.

Step S600 is a step in which the data analysis unit 410 detects whether a disaster has occurred by checking the analysis result. The data analysis unit 410 compares the rate of change of the measured values of the sensors, the actual measured values, and the external environment data, and rechecks the measured values of the factors associated with the initially measured factors to detect a disaster situation.

Step S700 is a step in which the control server 40 issues an alert according to the analysis result of the data analysis unit 410 . Here, the alert is displayed as a pop-up function on the display or transmitted to the mobile terminal 50 .

In addition, the diagnosis processing unit 450 transmits a disaster prevention facility operation command to the video monitoring device 10 according to the alarm of the occurrence of a disaster situation for the above data analysis, and through the received operation state information, whether each disaster prevention facility has a failure or a normal operation detect whether Here, the diagnosis processing unit 450 may also check whether the sensor operates at each set period.

As described above, the configuration and operation of a convergence control system and method based on real-time visualization technology of sensor data and image data according to an embodiment of the present invention can be made, and on the other hand, in the description of the present invention, specific embodiments are described. However, various modifications may be made without departing from the scope of the present invention.

Although the present invention has been described above with reference to limited embodiments and drawings, the present invention is not limited thereto, and various modifications and variations are possible by those skilled in the art to which the present invention pertains.

Those of ordinary skill in the art related to this embodiment will understand that it can be implemented in a modified form within a range that does not deviate from the essential characteristics of the above description. Therefore, the disclosed methods should be considered in an illustrative and not a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

10: video surveillance device
20: Disaster prevention facility control panel
30: sensor unit
40: control server
50: mobile terminal
110: camera
120: OSD module
410: data analysis unit
420 : DB
430: visualization module
440: interlocking control unit
450: diagnostic processing unit
460: video display control unit

Claims (2)

a sensor unit 30 having a plurality of sensors;
A video monitoring apparatus 10 having a camera 110 and an OSD (On Screen Display) module 120 that overlays and outputs the detection data of the sensor unit 30 on the image captured by the camera 110 . ;
a control server 40 that receives the video monitoring information overlaid with the detection data of the sensor unit 30 from the video monitoring device 10, and analyzes the received detection data to detect and alert whether a disaster situation exists;
a disaster prevention facility control panel 20 connected to the video monitoring device 10 by wire to control the disaster prevention facilities; and
a mobile terminal 50 for receiving and outputting a disaster alert from the control server 40; including,
The control server 40
Comparing the rate of change of the measurement sensor and the measured value included in the detected data, comparing the measured value with external environmental data, and comparing the measured value of one or more linked sensors linked to the measuring sensor with the rate of change of the linked sensor to produce a result data analysis unit 410;
a visualization module 430 that graphically processes the analysis result of the data analysis unit 410;
an interlocking control unit 440 for mapping the graphic-processed analysis result of the visualization module 430 to the location information and the image of the corresponding sensor;
an image display control unit 460 for outputting image monitoring information including a graphic-processed analysis result of the visualization module 430 to a display; and
a diagnosis processing unit 450 for transmitting an operation command to the disaster prevention facility control panel 20 according to the analysis result of the data analysis unit 410 and receiving and outputting operation state information of the disaster prevention facility through the video monitoring device 10; A convergence control system based on real-time visualization technology of sensor data and image data, including
a) receiving, from a plurality of image monitoring devices 10, image monitoring information overlaid with detection data measured from sensors, and external environment data;
b) comparing the measured values of the sensors and the rate of change of the measured values of the sensors in the video monitoring information, and comparing external environmental data;
c) if there is a difference between the measured value and the rate of change, and the measured value and the external environmental data, comparing the measured value and the rate of change of one or more sensors linked to the measuring sensor;
d) graphic processing of the analysis result as a bar-type or curved graph, and mapping the graphic-processed analysis result to an image including a corresponding sensor and outputting;
e) detecting a disaster situation by checking the analysis result;
f) when a disaster situation is detected, outputting an alert message on the display and transmitting an alert to the mobile terminal 50; and
g) outputting a disaster prevention facility operation command to the disaster prevention facility control panel 20 connected to the video monitoring device 10 , and receiving operation state information from the disaster prevention facility control panel 20 ; A convergence control method based on real-time visualization technology of sensor data and image data, including:

Images