KR20230107002A - 3d visualization control system and method for managing safety of workers in industrial sites - Google Patents

Abstract

An image capturing device installed in a preset industrial field area, a sensing device installed in a preset industrial field area, and receiving image data from the image capturing device, receiving sensing data from the sensing device, and receiving the image data and the image data. Based on the sensing data, generating worker abnormal occurrence information and disaster occurrence information, and generating a three-dimensional visualization image based on the image data, the sensing data, the worker abnormal occurrence information, and the disaster occurrence information. Including a server A 3D visualization control system is provided.

Description

3D VISUALIZATION CONTROL SYSTEM AND METHOD FOR MANAGING SAFETY OF WORKERS IN INDUSTRIAL SITES}

The present invention relates to a 3D visualization control system and method, and more particularly to a 3D visualization control system and method based on deep learning image analysis and IoT data for worker safety management at industrial sites.

According to the 2019 occupational safety accident occurrence status announced by the Ministry of Employment and Labor, the total number of injured persons was 109,242, an increase of 6,937 from the previous year, and the accident rate was 0.58%, which also increased by 0.4%. The increase in safety accidents is an issue that requires urgent measures considering not only economic loss but also enormous social loss. In particular, in the case of industrial sites where safety accidents frequently occur, the probability of large-scale accidents such as explosions or fires in case of equipment damage, defect, or failure is high due to the complexity, diversification, and aging of facilities. Therefore, it is very important to establish a next-generation situation control system capable of advanced safety management and accident response/prediction that can identify and quickly eliminate the causes of safety accidents, such as insufficient safety training, neglect of safety management, carelessness of workers, and non-compliance with regulations. It is important.

Conventional systems have problems such as GPU heat and system overload when analyzing multiple video channels at once because they do not have enough learning data to detect criminal behavior and industrial accidents and use a lot of hardware resources.

Conventional systems have a problem of listing inefficient control information out of the control situation in a method of providing image information or sensor information through text or 2D images.

Conventional systems have a problem in that, since it is difficult to determine correlations with each other, it is difficult to easily identify causes and countermeasures for various disasters and disaster situations that have occurred.

Accordingly, there is a need for a technology capable of preventing false detection of dangerous situations and predicting dangerous situations in advance by interlocking meta data generated by a deep learning-based behavioral analysis algorithm, and real-time 3D visualization to respond in advance.

The technical problem to be achieved by the present invention is a 3D visualization control system capable of preventing false detection of dangerous situations and predicting dangerous situations in advance by linking metadata generated by a deep learning-based behavior analysis algorithm, and responding in advance by visualizing 3D in real time. and to provide a method.

According to one embodiment, a 3D visualization control system is provided. The 3D visualization control system receives image data from an image capturing device installed in a preset industrial field area, a sensing device installed in a preset industrial field area, and the image capturing device, and receives the sensing data from the sensing device. and generates worker abnormal occurrence information and disaster occurrence information based on the image data and the sensing data, and creates a 3D visualization image based on the image data, the sensing data, the worker abnormal occurrence information, and the disaster occurrence information. contains a server that creates

The server includes an image data collection unit receiving the image data from the image capturing device, a sensing data collection unit receiving the sensing data from the sensing device, and a pre-stored artificial intelligence having the image data and the sensing data as input variables. The image data, the sensing data, the worker abnormal occurrence information, and the disaster occurrence information using an artificial intelligence analysis unit that generates the worker abnormal occurrence information or the disaster occurrence information using an intelligent algorithm, and a pre-stored algorithm It may include a visualization unit for generating a 3D visualization image based on.

The worker abnormal occurrence information may include information on whether work clothes are normally worn, information on whether a working radius is observed, and information on worker body abnormalities.

The disaster occurrence information may include information on whether or not a gas has leaked and the gas leak area, whether a fire has occurred and information on the area where the fire occurred, whether or not there is a facility abnormality, and information on the location of the facility.

According to one embodiment, a 3D visualization control method of a server of a 3D visualization control system is provided. The 3D visualization control method includes receiving image data from an image capturing device installed in a preset industrial site area, receiving sensing data from a sensing device installed in a preset industrial site area, the image data and the sensing Generating worker abnormal occurrence information and disaster occurrence information based on data, and generating a 3D visualization image based on the image data, the sensing data, the worker abnormal occurrence information, and the disaster occurrence information do.

It is possible to prevent false detection of dangerous situations and predict dangerous situations in advance by linking metadata generated by deep learning-based behavior analysis algorithms, and to respond in advance by visualizing real-time 3D.

By visualizing invisible information in 3D identical to the real-time on-site situation, it enables quick on-site response along with intuitive situation judgment.

It is possible to accurately predict the risk index by setting different weights for each site and situation and calculating the risk index through the probability of occurrence for each event using the used variable with weight applied to the image pattern extracted by the deep learning technique. .

1 is a block diagram of a 3D visualization control system according to an embodiment.
2 is a block diagram of a server according to one embodiment.
3 is a block diagram of an artificial intelligence analysis unit according to an embodiment.
4 is a block diagram of a visualization unit according to an exemplary embodiment.
5 to 8 are diagrams for explaining operation contents of a visualization unit according to an exemplary embodiment.
9 is a flowchart of a 3D visualization control method according to an embodiment.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

The terms used in the embodiments of the present invention have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, etc. . In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding embodiment. Therefore, the term used in the present embodiments should be defined based on the meaning of the term and the overall content of the present embodiment, not a simple name of the term.

In an embodiment of the present invention, terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

Also, in the embodiments of the present invention, singular expressions include plural expressions unless the context clearly indicates otherwise.

In addition, in the embodiments of the present invention, terms such as "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or other features, numbers, steps, operations, components, parts, or combinations thereof, or any combination thereof, is not precluded from being excluded in advance.

Also, in an embodiment of the present invention, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented by at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

1 is a block diagram of a 3D visualization control system according to an embodiment. 2 is a block diagram of a server according to one embodiment. 3 is a block diagram of an artificial intelligence analysis unit according to an embodiment. 4 is a block diagram of a visualization unit according to an exemplary embodiment. 5 to 8 are diagrams for explaining operation contents of a visualization unit according to an exemplary embodiment.

Referring to FIG. 1 , a 3D visualization control system according to an embodiment includes an image capturing device 100, a sensing device 200, a server 300, a manager terminal 400, and a worker terminal 500.

As an example, the image capturing device 100 may be installed in a preset industrial field area (eg, inside or outside a power plant).

As an example, there may be a plurality of image capturing devices 100, and each image capturing device 100 may be a CCTV camera or a thermal imaging camera (infrared camera).

As an example, the image capture device 100 may include a communication module that transmits and receives image data to and from the server 300 through an IoT network.

As an example, the image data may include real image image data and/or thermal image image data.

As an example, the sensing device 200 may be installed in a preset industrial site area (eg, inside or outside a power plant).

As one embodiment, there may be a plurality of sensing devices 200, and each sensing device 200 is any one of a gas leak detection sensor, a heat detection sensor, a smoke detection sensor, a fire detection sensor, a sound detection sensor, and a motion detection sensor. can be

As an example, the sensing device 200 may be attached to a worker's body, and may include a location measurement sensor (eg, a GPS sensor) and a body information detection sensor.

As an example, the sensing device 200 may include a communication module that transmits and receives sensing data to and from the server 300 through an IoT network.

As an example, the sensing data may include gas (LPG, butane gas, propane gas, methane gas, alcohol, hydrogen gas) concentration data, heat detection data, smoke detection data, fire detection data, sound data, and motion detection data. can

As an example, the sensing data may include data of a worker's blood pressure, pulse rate, and body temperature.

As an embodiment, the server 300 receives image data from the image capturing device 100 and sensing data from the sensing device 200, and receives worker abnormal occurrence information and disaster occurrence information based on the image data and the sensing data. Information may be generated, and a 3D visualization image may be generated based on image data, sensing data, worker abnormal occurrence information, and disaster occurrence information.

Referring to FIG. 2 , the server 300, as an embodiment, includes an image data collection unit 310, a sensing data collection unit 320, an artificial intelligence analysis unit 330, a visualization unit 340, and a storage unit 350. ), a notification unit 360 may be included.

The image data collection unit 310 may receive image data from the image capturing device 100 as an example.

The image data collecting unit 310, as an example, may include a communication module that transmits and receives data to and from the image photographing device 100 using wired/wireless communication.

The sensing data collection unit 320 may receive sensing data from the sensing device 200 as an example.

As an example, the sensing data collection unit 320 may include a communication module that transmits and receives data with the sensing device 200 using wired/wireless communication.

As an example, the artificial intelligence analysis unit 330 may generate worker abnormal occurrence information or disaster occurrence information using a pre-stored artificial intelligence algorithm using image data and sensing data as input variables.

As an example, the worker abnormal occurrence information may include information on whether work clothes (eg, hard hat, dust mask, etc.) are normally worn, information on whether a working radius is observed, and information on worker body abnormalities.

Disaster occurrence information, as an example, may include information on whether gas has leaked and the gas leak area, whether a fire has occurred and information on the area where the fire occurred, whether there is an abnormality in the facility and information about the location of the abnormal facility.

As an example, the pre-stored artificial intelligence algorithm may be a machine learning algorithm or a deep learning algorithm.

As an example, the artificial intelligence analysis unit 330 may generate a machine learning model based on learning data.

As an example, the learning data may include gas concentration data, heat detection data, smoke detection data, fire detection data, sound data, motion detection data, and reference data.

Referring to FIG. 3, the artificial intelligence analysis unit 330, as an embodiment, includes a feature point extraction module 331, an image pattern extraction module 332, a use variable setting module 333, and an event prediction module 334. can do.

As an example, the feature point extraction module 331 may extract feature points from image data using a pre-stored feature point extraction algorithm.

As an example, the image pattern extraction module 332 may extract image pattern information by comparing feature points with previously stored reference data.

As an example, the image pattern extraction module 332 may extract image pattern information by comparing feature points with reference data including specific actions or phenomena stored in the storage unit 350 .

A specific action is an example, such as the moving direction and motion of an object, possession of an object of a set shape (for example, a person with a weapon, a person or animal passing through a prohibited direction or object that is prohibited from moving), etc. can include

As an example, the specific phenomenon may include physical or chemical phenomena such as noise, smoke, flame, collision, and fall.

The image pattern extraction module 332, as an embodiment, extracts as reference data when features within the same range are repeated as images including both normal actions or objects and abnormal actions or objects and new features are accumulated and stored. can

As an example, the image pattern extraction module 332 may automatically adjust the resolution and reduce the bandwidth (for example, a bandwidth of 10 Mbps) so that image data and sensing data can be reduced and transmitted. Through this, the load of the network can be minimized.

As an example, the use variable setting module 333 may set image pattern information and sensing information as use variables, and set different weights according to industrial site zones.

As an example, the use variable setting module 333 may map information set to a correlation among image pattern information and sensing information using previously stored map data.

As an example, the event prediction module 334 may calculate an event occurrence probability and an event risk index according to the weight set in the use variable setting module 333 .

As an example, the event prediction module 334 may predict the risk based on the image pattern by itself through a machine learning learning process and calculate the result. As an embodiment, the event prediction module 334 calculates an event occurrence probability by autonomously tuning the weights according to the used variables using XGBoost, an open source software library that provides a gradient boosting framework, and calculates the probability of event occurrence, and DNN (Deep Neural Network ) event risk index can be calculated through learning through analysis.

As an example, the visualization unit 340 may generate a 3D visualization image based on image data, sensing data, worker abnormal occurrence information, and disaster occurrence information using a pre-stored algorithm.

Referring to FIG. 4 , the visualization unit 340, as an embodiment, includes a data classification module 341, an attribute database 342, a 3D database 343, a correlation setting module 344, and a visualization module 345. can include

As an example, the data classification module 341 may classify data received from the image photographing device 100 into location data, attribute data, and image data by using a pre-stored algorithm.

As an example, the data classification module 341 may classify data received from the sensing device 200 into location data, attribute data, and sensing data by using a previously stored algorithm.

As an example, the attribute database 342 includes attribute data including location, type, model, installation date, resolution, purpose, shooting range, and shooting time of the image capturing device 100, and an image of the image capturing device 100. Data, attribute data including the location, type, model, installation date, purpose, and sensing cycle of the sensing device 200, sensing data of the sensing device 200, control information including the name and contact information of the control department, and manager It is possible to store information, construction site and floor-by-floor location information.

The 3D database 343, as an embodiment, includes 3D spatial information, that is, the location, area and height of the actual physical space, and objects existing in the space (eg, columns, walls, doors, machinery, flower beds, flower pots). , fence) can store the shape or external information.

As an example, the correlation setting module 344 may set a correlation between image data and attribute data of the image capturing apparatus 100 and 3D space information using previously stored map data.

As an example, the correlation setting module 344 may set a correlation between the sensing data and attribute data of the sensing device 200 and the 3D spatial information using previously stored map data.

As an embodiment, the correlation setting module 344 matches virtual space coordinates and image data using previously stored map data to output image data of the image capturing device 100 in the modeled virtual space. You can specify a location or display.

As an example, the correlation setting module 344 matches the coordinates of the virtual space with the sensing data using previously stored map data, and the location where the sensing data of the sensing device 200 is output within the modeled virtual space. I can designate a mark.

Referring to FIG. 5 , as an example, the visualization module 345 may model a real physical space into a virtual 3D space and display correlation-based sensing information in the modeled virtual 3D space.

6 and 7, the visualization module 345, as an embodiment, models a real physical space into a virtual 3D space, and displays worker abnormal occurrence information and disaster occurrence information in the modeled virtual 3D space. can

Referring to FIG. 8 , as an example, the visualization module 345 may model a real physical space into a virtual 3D space and display (overlay) image data on the modeled virtual 3D space.

As an example, the visualization module 345 may output correlation-based sensing data, worker abnormal occurrence information, disaster occurrence information, and image data to the screen of the manager terminal 400 using wired/wireless communication.

The storage unit 350, as an example, may store image data and sensing data.

As an example, the storage unit 350 may store image patterns representing images including features including the same or similar actions, phenomena, or objects.

As an example, the storage unit 350 may store image patterns including actions and/or phenomena (smoke, gas, motion, and noise) of objects (eg, people, equipment, and animals) in the image.

As an example, the notification unit 360 transmits a notification message containing details of the occurrence of an abnormal worker to the manager terminal 400 and the worker terminal 500 when it is determined through the artificial intelligence analysis unit 330 that an abnormal worker has occurred can do.

Notification unit 360, as an embodiment, when it is determined that a disaster has occurred through the artificial intelligence analysis unit 330, a notification message including disaster occurrence history and disaster occurrence area to the manager terminal 400 and worker terminal 500 can send

As an embodiment, the notification unit 360 may include a communication module that transmits and receives data with the manager terminal 400 and the worker terminal 500 using wired/wireless communication.

As an example, the manager terminal 400 may be connected to the server 300 through an app or the web, and may output video data, sensing data, worker abnormal occurrence information, disaster occurrence information, and video data to a screen. Through this, it is possible to respond promptly in the event of a worker's abnormality or a disaster.

The manager terminal 400 includes a memory means such as a mobile communication terminal, a desktop computer, a notebook computer, a workstation, a palmtop computer, a personal digital assistant (PDA), a web pad, and the like, as an embodiment. It may be a digital device equipped with a microprocessor and equipped with an arithmetic capability.

As an embodiment, the worker terminal 500 may be connected to the server 300 through an app or the web, and output image data, sensing data, worker abnormal occurrence information, disaster occurrence information, and image data to the screen. Through this, it is possible to respond promptly in the event of a worker's abnormality or a disaster.

As an embodiment, the worker terminal 500 includes a memory means such as a mobile communication terminal, a desktop computer, a notebook computer, a workstation, a palmtop computer, a personal digital assistant (PDA), and a web pad. It may be a digital device equipped with a microprocessor and equipped with an arithmetic capability.

9 is a flowchart of a 3D visualization control method according to an embodiment.

Referring to FIG. 9, the 3D visualization control method of the server 300 according to an embodiment includes receiving image data from the image capturing device 100 installed in a preset industrial site area (S100), Receiving sensing data from the sensing device 200 installed in the field area (S200), generating worker abnormal occurrence information and disaster occurrence information based on image data and sensing data (S300), image data, sensing data , generating a three-dimensional visualization image based on information on the occurrence of abnormal workers, and information on occurrence of disasters (S400).

Steps S100 to S400 are the same as the operations of the server 300 described above, so detailed descriptions thereof are omitted.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. that fall within the scope of the right.

Those skilled in the art related to this embodiment will be able to understand that it can be implemented in a modified form within a range that does not deviate from the essential characteristics of the above description. Therefore, the disclosed methods are to be considered in an illustrative rather than a limiting sense. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent scope will be construed as being included in the present invention.

Claims (5)

An imaging device installed in a preset industrial site area,
A sensing device installed in a preset industrial site area, and
Receives image data from the image capturing device, receives sensing data from the sensing device, generates worker abnormal occurrence information and disaster occurrence information based on the image data and the sensing data, and generates the image data and the sensing data. , Server for generating a three-dimensional visualization image based on the worker abnormal occurrence information and the disaster occurrence information
3-dimensional visualization control system containing a. In paragraph 1,
The server,
an image data collecting unit receiving the image data from the image capturing device;
a sensing data collection unit receiving the sensing data from the sensing device;
An artificial intelligence analysis unit for generating the worker abnormal occurrence information or the disaster occurrence information using a pre-stored artificial intelligence algorithm using the image data and the sensing data as input variables, and
A visualization unit for generating a 3D visualization image based on the image data, the sensing data, the operator abnormal occurrence information, and the disaster occurrence information using a pre-stored algorithm.
Including, three-dimensional visualization control system. In paragraph 2,
The worker abnormal occurrence information,
A three-dimensional visualization control system that includes information on whether or not work clothes are normally worn, information on whether or not the working radius is observed, and information on worker's body abnormalities. In paragraph 3,
The disaster occurrence information,
A three-dimensional visualization control system that includes information on gas leakage and gas leakage zones, fire occurrence and fire zone information, facility abnormality and abnormal facility location information. As a 3D visualization control method of a server of a 3D visualization control system,
Receiving image data from an image capturing device installed in a preset industrial site area;
Receiving sensing data from a sensing device installed in a preset industrial site area;
Generating worker abnormal occurrence information and disaster occurrence information based on the image data and the sensing data, and
Generating a 3D visualization image based on the image data, the sensing data, the worker abnormal occurrence information, and the disaster occurrence information
3-dimensional visualization control method comprising a.

Images