KR20230032087A - Abnormal symptom data visualization system based-on deep learning engine and method thereof - Google Patents

Abstract

An embodiment of the present invention may provide a deep learning engine-based abnormal sign data visualization system comprising: a sensor unit (100) having a plurality of sensors; an imaging unit (200) including a plurality of cameras; an event detection unit (300) collecting detection information of the sensor unit (100) and image information of the imaging unit (200) to predict an abnormal sign; an invisible information visualization unit (400) modeling an actual physical space into a virtual three-dimensional space and visualizing the detection information of the sensor unit (100) in the modeled virtual space; and a control unit (500) processing information output from the event detection unit (300) and the invisible information visualization unit (400) to generate control information and providing a control information editing tool for decision-making. An abnormal sign can be easily identified in a dead zone to perform a rapid countermeasure.

Description

Abnormal symptom data visualization system based-on deep learning engine and method thereof}

The present invention relates to a system and method for visualizing anomaly data based on a deep learning engine.

In a situation where awareness of safety due to social and natural disasters has been gradually amplified since the past, damage from accidents and accidents is increasing as social density and technology are advanced.

For example, in Korea, due to the Sungnyemun fire that occurred in February 2008, a disaster occurred in which cultural assets, the national symbol, were burned down within 5 hours. occurred, resulting in large-scale accidents such as 679 stores being burned/an incident. In addition, in the case of foreign countries, terrorism or accidents that threaten social safety due to various conflicts are spreading.

In this way, social and natural disasters have a great impact on the economy as well as the social impact. Accordingly, various video surveillance systems have been proposed at home and abroad for the purpose of identifying the cause of the accident, preventing it, and preventing it.

However, as shown in FIG. 1, the conventional video surveillance system provides sensed data (a) together with image information (b) in an enumerated manner, and detects objects such as people and vehicles at a moving position to obtain fixed A controller in the form of a frame is provided. In particular, in the prior art, as shown in FIG. 1, CCTV images and sensing values are provided separately from visualization information (image information), and thus there is an unintuitive problem.

In addition, the prior art is a method of playing back by time zone, so when a specific situation occurs, the time required to identify the cause is high, and only the CCTV and sensor for each model owned are compatible, so the equipment is compatible. There was a difficult problem.

Republic of Korea Patent Registration No. 10-1987241 (2019.06.03)

In order to solve the above conventional problems, the present invention provides a system and method for visualizing anomaly data based on a deep learning engine capable of predicting anomalies in advance by intuitively providing non-visible information by fusion of image information and sensor information. is intended to provide

The present invention may include the following embodiments in order to achieve the above object.

An embodiment of the present invention is a sensor unit having a plurality of sensors, an imaging unit having a plurality of cameras, an event detection unit that predicts an anomaly by collecting detection information of the sensor unit and image information of the imaging unit, and a real physical The space is modeled as a virtual 3D space, and the information output from the non-visual information visualization unit and the event detection unit and the non-visual information visualization unit that visualizes the sensing information of the sensor unit in the modeled virtual space is processed to generate control information In addition, it is possible to provide a visualization system of anomaly data based on a deep learning engine including a control unit providing an editing tool for control information for decision making.

Another embodiment of the present invention includes an event detection step of an event detection unit for predicting an event by collecting image information received from a plurality of imaging units and sensing information of a sensor unit, and the event detection step of the event detection unit is a) the imaging unit Collecting image information from, b) extracting an image pattern including a set action or object among the collected images, and c) using the detection information of the sensor unit installed in the field included in the image pattern as a variable used in the image pattern. , and applying weights set differently for each situation and site, d) calculating the event occurrence probability and the predicted value of the event influence through the use variables set as different weights, and e) event occurrence probability and event A visualization method of anomaly data based on a deep learning engine including calculating an event risk index using an influence value may be provided.

The present invention provides control information expressed in a virtual 3D space in which the sensing information of the sensor unit expresses the actual physical space, so that it is easy to grasp abnormal symptoms in the shooting blind spot, so that quick response is possible to prevent loss of property and life. can

In addition, the present invention sets different weights for each site and situation, and calculates the risk index through the influence and smell diffusion probability of each event using the weighted usage variable to the image pattern extracted by the deep learning technique. There is an effect that can predict the risk index.

1 is a diagram showing a conventional output screen of image information and sensor information.
2 is a block diagram showing a system for visualizing anomaly data based on a deep learning engine according to the present invention.
3 is a block diagram illustrating an event detection unit.
4 is a block diagram illustrating an invisible information visualization unit.
5 is a block diagram showing a control unit.
6 is a flowchart illustrating an anomaly prediction step in a method for visualizing anomaly data based on a deep learning engine according to the present invention.
FIG. 7 is a flowchart illustrating step S150 of FIG. 6 .
8 is a flowchart illustrating a visualization process of invisible information according to the present invention.

The terms or words used in this specification and claims are not limited to the usual or dictionary meanings, and the inventor can properly define the concept of the term in order to explain his or her invention in the best way. Based on this, it should be interpreted as a meaning and concept consistent with the technical spirit of the present invention.

Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated. In addition, terms such as “… unit”, “… unit”, “module”, and “device” described in the specification mean a unit that processes at least one function or operation, which is implemented as a combination of hardware and/or software. It can be.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

2 is a block diagram showing a system for visualizing anomaly data based on a deep learning engine according to the present invention.

Referring to FIG. 2 , the present invention includes a sensor unit 100, an imaging unit 200, an event detection unit 300, an invisible information visualization unit 400, a control unit 500, and a monitoring unit 600. ) and a playback unit 700.

The dual sensor unit 100 includes at least one of sensors capable of detecting phenomena in space such as gas, smoke, temperature, humidity, current, resistance, and illuminance, and motion sensors capable of detecting the human body and human actions or movements. It consists of a plurality of sensors including one or more.

The imaging unit 200 may include a plurality of cameras.

Here, the sensor unit and the imaging unit may transmit signals through an internal/external network in a plurality of sites.

In addition, the sensor unit 100 and the image unit 200 may be installed in separate locations as independent devices. Alternatively, the sensor unit 100 and the image unit 200 may be integrally formed and installed at the same location.

The event detection unit 300 calculates abnormal symptoms through information of the sensor unit 100 and the imaging unit 200 . Such an event detection unit 300 will be described with reference to FIG. 3 .

3 is a block diagram illustrating an event detection unit.

Referring to FIG. 3 , the event detection unit 300 includes an image DB 310, an image extraction means 320, a feature DB 330, a sensor DB 340, a use variable setting means 350, , may include an event prediction means 360.

The image DB 310 stores images captured by the imaging unit 200, and the feature DB 330 stores set or extracted image patterns. Here, the image patterns stored in the feature DB 330 refer to images including features including the same or similar actions, phenomena, or objects. The set features may include actions and/or phenomena (smoke, gas, motion, noise) of objects (eg, people, equipment, animals) in the image.

The image extraction means 320 processes the image information stored in the image DB 310 and compares the image information corresponding to the set condition with the image pattern including the existing action or object stored in the feature DB 330.

The specified action may include the moving direction and motion of the object, possession of an object of a set shape (eg, a person with a weapon, a person or animal passing through an object in a prohibited direction or movement prohibited), and the like. there is.

In addition, the specified phenomenon may include physical and chemical phenomena such as noise, smoke, flame, collision, and fall.

The image extraction unit 320 may extract images including normal actions or objects and abnormal actions or objects as image patterns when images including new features are accumulated and stored and features within the same range are repeated.

Here, the image extractor 320 can minimize the load on the network, automatically adjusts the resolution so that sensor detection information and mapped image information can be abbreviated and transmitted, and the bandwidth is reduced (for example, 10 Mbps level Bandwidth), large-capacity transmission can be advantageously processed.

In addition, the image extraction means 320 extracts image information including a set pattern (hereinafter referred to as an image pattern) from the image information stored in the feature DB 330. That is, the image extractor 320 collects prognostic symptoms such as incidents/accidents indicating abnormal behavior of people or mechanical devices as a data pre-processing process.

The use variable setting unit 340 sets variables of the extracted image pattern. As the use variable, detection information of the sensor unit 100 set in association with the image patterns extracted by the image extraction unit 320 is set as a use variable. Here, the sensing information of the sensor unit 100 may correspond to information stored in the sensor DB 350 or real-time sensing information received from the sensor unit 100 .

That is, the use variable setting unit 340 maps information set in a correlation among image information and sensing information of the sensor unit 100 .

The use variable setting means 340 may set weights of use variables for each site and/or situation in order to differentially adjust the risk of a precursor symptom according to the situation of each site.

The event prediction unit 360 predicts the risk through the image pattern itself through a machine learning learning process and calculates the result. For this reason, the event prediction means 360 predicts the event occurrence probability by autonomously tuning the weights according to the used variables using, for example, XGBoost, an open source software library that provides a gradient boosting framework, and DNN ( It predicts (calculates) the event risk index by predicting the event impact through learning through Deep Neural Network) analysis.

The invisible information visualization unit 400 visualizes invisible information (eg, sensing information of the sensor unit 100). The invisible information visualization unit 400 will be described with reference to FIG. 4 .

4 is a block diagram illustrating an invisible information visualization unit.

Referring to FIG. 4 , the invisible information visualization unit 400 may include an information collecting unit 410, a data classifying unit 420, a correlation setting unit 430, and a visualization unit 440. .

The information collection unit 410 collects information from the sensor unit 100 and/or the imaging unit 200 . Here, the information collecting unit 410 may collect image information stored in the image DB 310 or feature DB 330 of the event detection unit 300 or images taken by the imaging unit 200 in real time. Also, the information collection unit 410 may collect sensing information of the sensor unit 100 in real time.

The data classification unit 420 classifies the non-visible information collected by the information collection unit 410 into attribute information and 3D information. To this end, the data classification unit 420 classifies the image unit 200 and the basic information management and detection information of the sensor, monitoring information, and classifies it as information of the actual physical space. To this end, the data classification means 420 may include an attribute DB 421 and a three-dimensional DB 422.

The attribute DB 421 stores detection information of the sensor unit 100 and management information such as the position and properties of the imaging unit 200 . The information stored in the attribute DB 421 is control information including the imaging unit 200 and sensor type, model, installation date, resolution, purpose, location, shooting range, shooting time, control department, name, and contact information. , manager information, construction site and floor-by-floor location information, operation and observation conditions of the imaging unit 200 and sensor unit 100, and event status monitoring information may be included.

The 3D DB (422) stores the shape or external information of the location, area and height of the actual physical space, and objects (eg, columns, walls, doors, machinery, flower beds, pots, fences) existing in the space. do. Here, the 3D information may include 2D or 3D images.

The correlation setting unit 430 sets a correlation between image information, attribute information, and 3D information. Here, correlation means visually mapping the mutual relationship of abstract data in a virtual space. That is, the correlation setting unit 430 may designate a position or display where the sensing information of the sensor unit is output in the modeled virtual space by matching the virtual space coordinates with the sensing information of the sensor.

Here, the correlation setting unit 430 may match and output various characters, symbols, etc. inputted through a set input device to positional coordinates of the virtual space in addition to the detected information of the sensor unit 100 .

The visualization means 440, for example, models a virtual space as a 3D engine, and the location and attribute values of the image unit 200 matched by the correlation setting means 430 in the modeled virtual space, Invisible information is visualized by expressing detected information of the sensor unit 100 .

That is, the visualization means 440 implements the location or image information without image information in a 3D virtual space, and matches the information detected by the sensor unit 100 in the implemented 3D virtual space to obtain invisible information. It is implemented as visible information and output.

Therefore, the present invention can be implemented as visual information by expressing unidentified or uncertain blind spots or spaces where cameras are not installed as a virtual 3D space and matching sensor information, which is non-visual information, within the virtual 3D space. can

The control unit 500 generates control information by processing information output from the event detection unit 300 and the invisible information visualization unit 400, provides a control information editing tool for decision-making, and provides a data-based Processes and outputs control information that enables decision-making.

To this end, the control unit 500 supports various control information formats (eg, video, HTML, Excel) such as image information and sensor unit 100 detection information, and forms a database 590 that can be analyzed. It analyzes and processes the correlation and threshold of each information to provide risk situations with a data-based decision-making model. A detailed configuration of the control unit 500 will be described with reference to FIG. 5 .

5 is a block diagram showing a control unit 500.

Referring to FIG. 5 , the control unit 500 includes a user function module 510, a visualization chart module 520, an external linkage module 530, a data analysis module 540, and a distributed sorting module 550. , a data collection module 560 , a language analysis module 570 , a search module 580 , and a database 590 .

The user function module 510 may provide real-time analysis by function, dashboard, real-time alarm, search results, and table list. In addition, the user function module 510 may provide various menus for visualization charts, collection source management, dictionary resource management, and task management.

The double real-time alarm may output an alarm according to output information of the event detection unit 300 and the invisible information visualization unit 400 .

Real-time analysis by function is interlocked with the invisible information visualization unit 400 to display real-time analysis results along with detection information of the sensor unit 100 related to the function of each device expressed through a virtual space implemented according to the purpose of the device. A method of outputting through may be made.

That is, the control unit 500 drives the invisible information visualization unit 400 to output real-time analysis results as visualized information.

The visualization chart module 520 implements the collected information as a visualization chart. For example, the visualization chart module 520 converts the collected data into a line chart. It provides functions such as realization and graphic interlocking functions with charts of various shapes such as bar pie charts and danograms. The functions of the visualization chart module 520 may be output by the user function module 510 .

The external linkage module 530 provides an app, an external linkage API, and an external data collection function. That is, the external interworking module 530 enables communication with external servers and devices through wired/wireless networks and can receive external data.

The data analysis module 540 provides an analysis algorithm for analyzing the sensed information of the sensor unit 100 and query generation and query optimization operation functions. The analysis algorithm may be interlocked with the event detection unit 300, and the analyzed result is transmitted to an external device (eg, manager's mobile terminal, control server) set through the dash mode and/or external interlock module 530. can do.

The data analysis module 540 may statisticize the information distributed and/or collected by the non-visualized information visualization unit and the event detection unit 300 and aggregate the distribution results to output to the user function module 510 . Also, the data analysis module 540 may provide a scheduler for task management.

The distributed sorting module 550 includes keyword search information (eg, keyword, search number, search result, search date) and scan information, image pattern of the event detection unit 300 and external dictionary search information, and calculation information. It is distributed sorted and stored in the database 590.

The data collection module 560 collects data of the external storage and local storage having the set format of the substrate. Here, the data collection module 560 may search and collect data corresponding to a keyword entered through the dashboard.

The language analysis module 570 extracts topics and morphemes from various electronic documents and files stored in metadata, external and local storage, external storage and local storage. Here, the language analysis module sets standards including at least one of a set word, a combination of words and words, and a sentence, and extracts an electronic document or file having contents meeting the standard.

The search module 580 provides complex search query processing, distributed data search, and distributed high-speed indexing functions. For example, the search module 580 sets and automatically provides an answer according to a set query. In addition, the search module searches distributed data (eg, the image DB 310, the attribute DB 421, the 3D DB 422, the database 590 of the control unit 500), and the user function module ( 510) can be output to the dashboard. In addition, the search module may selectively assign and set indexes to distributed and stored information according to set criteria.

The database 590 includes a metadata DB for storing metadata, an external device data DB for storing information collected from external devices, a calculation result DB for storing calculation results, and an index information DB for storing index information. can include

The monitoring unit 600 monitors the real-time analysis result of the control unit 500 and the event detection unit 300 and transmits the corresponding information through an internal or external network. For example, the control unit 500 sets identification information such as information, IP, and ID of a terminal for which the right to receive monitoring information is set, and the monitoring unit 600 sets the identification information set through the control unit 500. Monitoring information can be transmitted in real time or periodically to an external device having the information. At this time, the monitoring unit 600 may selectively transmit the monitoring result according to the set authority.

The playback unit 700 reproduces images stored in the image DB 310 . Here, the playback unit 700 may reproduce image information related to each event and situation in association with information (data) related to meta data stored in the database 590 of the control unit 500 . In addition, the playback unit 700 may support thumbnail, date and time search, snapshot search, and event search functions.

For example, the playback unit 700 may perform a date and time search for images for each of a plurality of cameras included in the imaging unit 200 through date and time. In addition, the playback unit 700 may implement a thumbnail search so that a changed appearance of a stored image can be confirmed and searched.

In addition, the playback unit 700 supports a fast image search function by previewing snapshots for each time period when searching for image information.

In addition, the playback unit 700 may provide a search function for each event to specify and search for past problem situations based on event data.

The search function of the reproducing unit 700 can provide an effect of immediately checking abnormal situations and singularities of monitoring situations extracted from various sensed information of the sensor unit 100, and detecting various types of sensors in a specific situation. It is possible to check information or quickly check surrounding images related to the situation specific to the detection information of a specific sensor. Therefore, the reproduction unit 700 makes it easier to identify the cause. Here, the playback unit 700 may be included in the control unit 500.

The present invention includes a method for visualizing anomaly data based on a deep learning engine implemented by the above configuration. The method for visualizing anomaly data based on a deep learning engine according to the present invention may include an anomaly prediction step and a visualization step of invisible information. The double anomaly prediction step will be described with reference to FIGS. 6 and 7 .

6 is a flow chart showing an anomaly prediction step in the method for visualizing anomaly symptom data based on a deep learning engine according to the present invention, and FIG. 7 is a flowchart showing step S150 in FIG. 6 .

6 and 7, the anomaly prediction step in the present invention comprises the steps of S110 for receiving image information, step S120 for image processing, step S130 for extracting a pattern, step S140 for setting use variables, A step S150 of predicting an event may be included.

Step S110 is a step of collecting image information in the event detection unit 300 . The imaging unit 200 stores captured images in the image DB 310 in real time, and the image extraction means 320 collects captured images of the imaging unit 200 stored in the image DB 310 in real time.

Step S120 is a step of compressing the image collected by the event detector 300, adjusting the bandwidth to have a bandwidth suitable for large-capacity processing, and compressing the image.

Step S130 extracts an image including a set action or object from among the images collected by the event detection unit 300 and stores it as a feature DB 330, and within a set range with the images stored in the existing feature DB 330. This is the step of extracting the pattern to which it belongs.

Step S140 is a step of setting use variables in the event detection unit 300 . The use variable setting unit 340 of the event detection unit 300 is a step of setting a use variable including a weight according to the field situation of the corresponding image when a newly captured image corresponds to an existing image pattern. Here, the variable used is sensing information (sensing data) of the sensor unit 100 installed in the field. Therefore, the use variable setting means 340, for example, prior to comparing the noise detected at the site of the captured image with the noise value in the abnormal situation in the image pattern, according to the level of noise at the site at ordinary times, Set the weight for the noise value.

Step S150 is a step of predicting an event in the event detection unit 300, and the event risk index is calculated using the step S151 of predicting the event occurrence probability, the step of S152 of predicting the event influence, and the event occurrence probability and the event influence value. This is the calculation step.

Step S151 is a step of predicting an event occurrence probability through weight setting of a use variable assigned to an image pattern in the event predicting unit 360 . Here, the event prediction unit 360 may predict an event occurrence probability using XG Boost.

Step S152 is a step of predicting event influence through DNN analysis in the event prediction means. Event influence predicts what kind of influence, for example, how a flame or temperature generated at the site can affect nearby devices or people according to the weight of the use variable generated at the site.

Step S153 is a step of calculating the risk index through the event occurrence probability and the calculated value of the event influence in the event prediction unit 360 . The event prediction unit 360 may calculate a risk index and output event detection information to the control unit 500 when the calculated risk index value corresponds to a set criterion. When event detection information is received, the control unit 500 may drive an alarm.

8 is a flowchart illustrating a visualization step of invisible information according to the present invention.

Referring to FIG. 8, the visualization of invisible information includes information collection S210, data conversion and processing S220, information classification S230, correlation setting and modeling S240, A step S250 of outputting the visual information as 3D information may be included.

Step S210 is a step of collecting real-time information from the sensor unit 100 and the imaging unit 200 in the invisible information visualization unit 400 . Here, the sensor unit 100 and the imaging unit 200 store information detected in real time as the sensor DB 350 and the image DB 310, and the invisible information visualization unit 400 collects the stored information, or It is also possible to collect information by directly connecting without going through the sensor DB 350 and the image DB 310.

Step S220 is a step of converting and processing data. The information collecting unit 410 may convert the collected information into a set format and process it to be suitable for mass transmission.

Step S230 is a step of classifying information. The data classification means 420 classifies the collected information into actions, objects, and/or phenomena, stores it in the attribute DB 421, and classifies and stores information related to the 3D space into the 3D DB 422.

Step S240 is a step of setting a correlation between each piece of information stored in the attribute DB 421 and the 3D DB 422 and modeling the actual physical space into a virtual space. The correlation setting unit 430 sets a correlation between information stored in the attribute DB 421 and data stored in the 3D DB 422. For example, the correlation sets an image and sensing information of a sensor installed at an actual location of a captured image, or sets sensing information of the sensor unit 100 to be associated with 3D information of an actual physical space.

And the visualization unit 440 models the actual physical space into a virtual 3D space using 3D data.

Step S250 is a step of expressing the sensed information of the sensor in the modeled virtual space. The visualization unit 440 may express the information set by correlation in the modeled virtual space. This models the space where the imaging unit 200 is not installed as a virtual space, and expresses the sensing information of the sensor unit 100 in the modeled virtual space, thereby accurately determining the situation of the blind spot where the image is not secured. make it possible

The information visualized in the 3D virtual space is displayed by the control unit 500 . In addition, the administrator can easily grasp the on-site situation using the invisible information visualization unit 400 as described above, and through this, the detection rate for abnormal symptoms can be increased.

The above description is merely an example of the technical idea of the present invention, and those skilled in the art can make various modifications, changes, and substitutions without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: sensor unit 200: imaging unit
300: event detection unit 310: image DB
320: image extraction means 330: feature DB
340: use variable setting means 350: sensor DB
360: event prediction means 400: invisible information visualization unit
410: information collection means 420: data classification means
421: attribute DB 422: three-dimensional DB
430: correlation setting means 440: visualization means
500: control unit 510: user function module
520: visualization chart module 530: external linkage module
540: data analysis module 550: distributed sorting module
560: data collection module 570: language analysis module
580: search module 590: database
600: monitoring unit 700: playback unit

Claims (6)

A sensor unit 100 having a plurality of sensors;
an imaging unit 200 having a plurality of cameras;
an event detection unit 300 that predicts abnormal symptoms by collecting detection information of the sensor unit 100 and image information of the imaging unit 200;
an invisible information visualization unit 400 that models a real physical space into a virtual 3D space and visualizes the sensing information of the sensor unit 100 in the modeled virtual space; and
a control unit 500 that generates control information by processing information output from the event detection unit 300 and the invisible information visualization unit 400 and provides an editing tool for the control information for decision-making; Visualization system of anomaly data based on deep learning engine including
The method according to claim 1, the event detection unit 300
image extraction means 320 for extracting a set image pattern from image information;
a use variable setting means 340 for setting detection information of the sensor unit 100 detected at the site of the extracted image pattern as a use variable and setting different weights according to the site; and
an event predicting unit 360 for calculating a danger sign by calculating an event occurrence probability and an event influence according to the weight set by the use variable setting unit 340; Visualization system of anomaly data based on deep learning engine including
The method according to claim 1, the invisible information visualization unit 400
information collection means 410 for collecting information from the sensor unit 100 and the imaging unit 200;
a data classification unit 420 for classifying the information collected by the information collection unit 410 into sensing information of the sensor unit 100 according to actions, objects, and phenomena, and information of an actual physical space;
Correlation setting means 430 for setting a correlation so that the detected information of the sensor unit 100 and the information of the actual physical space are linked; and
a visualization unit 440 that models the actual physical space as a virtual 3D space and expresses the sensing information of the sensor unit 100 in the modeled virtual 3D space; Visualization system of anomaly data based on deep learning engine including.
The method according to claim 1, Reproducing unit (700) for reproducing the image stored in the image DB (310); Including more,
The playback unit 700
Reproducing video information related to each event and situation in conjunction with information related to metadata stored in the database 590, and supporting one or more of thumbnail, date and time search, snapshot search, and event search; A visualization system of anomaly data based on a deep learning engine.
An event detection step of an event detection unit 300 for predicting an event by collecting image information received from a plurality of imaging units 200 and sensing information of the sensor unit 100;
The event detection step of the event detection unit 300 is
a) collecting image information from the imaging unit 200;
b) extracting an image pattern including a set action or object among collected images;
c) setting detection information of the sensor unit 100 installed in the field included in the image pattern as a use variable of the image pattern, and applying weights differently set for each situation and site;
d) calculating an event occurrence probability and a predicted value of an event influence through use variables set as different weights; and
e) calculating an event risk index using an event occurrence probability and an event influence value; Visualization method of anomaly data based on deep learning engine including.
The method according to claim 4, further comprising; visualizing the invisible information by the invisible information visualization unit 400,
The visualization step of the invisible information visualization unit 400 is
a-1) collecting real-time information from the sensor unit 100 and the imaging unit 200;
b-1) classifying the collected information into attribute information including at least one of actions, objects, and phenomena, and 3D information including real physical space information;
c-1) setting a correlation between attribute information and 3D information and modeling a real physical space into a virtual space; and
d-1) visualizing invisible information by expressing sensor information sensed by a correlation set in a modeled virtual space; Visualization method of anomaly data based on deep learning engine including.

Images