KR20210131581A - Workplace Safety Management Apparatus Based on Virtual Reality and Driving Method Thereof - Google Patents

Abstract

The present invention relates to a workplace safety management apparatus based on virtual reality (VR) and a driving method thereof. The workplace safety management apparatus based on VR according to an embodiment of the present invention includes: a communication interface unit which receives a photographed image of a workplace from a camera of the workplace; and a control unit which uses the received photographed image to generate a VR background image of the workplace, makes an object into an image based on a deep learning result of the object extracted from the received photographed image, and generates a VR image of the workplace by synthesizing the generated VR background image and the imaged object, to output the VR image.

Description

Workplace Safety Management Apparatus Based on Virtual Reality and Driving Method Thereof

The present invention relates to a virtual reality-based workplace safety management device and a driving method of the device, for example, an object (eg, object/person/equipment, etc.) for which a signal for location tracking is not detected in a worksite such as tunnel construction. B. It relates to a virtual reality-based workplace safety management device that can easily monitor the workplace through virtual reality (VR) images, even in the event of a signal error, and a method of driving the device.

In general, in construction, civil engineering, and plant sites, the safety of each worker is required as the top priority. However, up to now, most of the individual safety of these workers is conducted prior to training, and when they are put into an actual work site, it is common for each individual to take responsibility for their own safety. Recently, due to the development of smart devices and wearable devices, some systems have been proposed that allow the location and body information of workers to be checked even on a central control server.

In a conventional monitoring system in a work space such as a tunnel, it is common to monitor an image displayed through CCTV (Closed Circuit Television) and to manage and supervise it in a control center. When monitoring the work site only with such CCTV, it was difficult to check the location and information of the exact object in real time.

In order to compensate for this, it is conventionally possible to check information about an object in a certain space using a beacon or a beacon scanner. There are many cases where it is not possible to properly check the situation information inside.

Korean Patent Publication No. 10-2074684 (2020.02.03) Korean Patent Publication No. 10-1944823 (2019.01.28) Korean Patent Publication No. 10-2014652 (2019.08.20) Korean Patent Publication No. 10-2020-0030784 (2020.03.23) Korean Patent Publication No. 10-2020-0031312 (2020.03.24)

An embodiment of the present invention provides a virtual reality that can easily monitor a work site through a virtual reality (VR) image even when an object or signal for which a signal for location tracking is not detected in a work site such as tunnel construction is in error. An object of the present invention is to provide a work-site safety management device based on and a method of driving the device.

A virtual reality-based workplace safety management device according to an embodiment of the present invention includes a communication interface unit for receiving a photographed image of the workplace from a camera of the workplace, and the workplace using the received photographed image. Create a virtual reality (VR) background image for the appearance, image the object based on the deep learning result of the object extracted from the received captured image, and synthesize the generated VR background image with the imaged object It includes a control unit that generates and outputs a VR image of the work site.

The control unit extracts a unit frame image of the same time from each captured image received from a plurality of cameras installed at different locations in the work site and stitches each extracted unit frame image to the VR background You can create an image.

The controller may analyze the received captured image to acquire depth information of the extracted object, and apply the acquired depth information to synthesize the imaged object with the VR background image.

The controller may further use a distance value calculated by beacon information received in relation to the extracted object in order to acquire depth information of the extracted object.

The controller may switch to the VR image control mode when an error is detected in an event or an object at the work site.

In addition, the driving method of the virtual reality-based workplace safety management apparatus according to an embodiment of the present invention comprises the steps of: a communication interface unit receiving a photographed image of the work site from a camera of the work site; and the control unit, the A virtual reality (VR) background image of the appearance of the work site is generated using the received captured image, the object is imaged based on the deep learning result of the object extracted from the received captured image, and the generated VR background and synthesizing the image and the imaged object to generate and output a VR image of the work site.

The generating and outputting step includes extracting a unit frame image of the same time from each captured image received from a plurality of cameras installed at different locations in the work site and stitching each of the extracted unit frame images to create the VR You can create a background image.

The generating and outputting may include analyzing the received captured image to obtain depth information of the extracted object, and applying the obtained depth information to synthesize the imaged object with the VR background image.

The generating and outputting may further use a distance value calculated by beacon information received in relation to the extracted object to obtain depth information of the extracted object.

In the generating and outputting step, when an error is detected in an event or the object at the work site, the VR image control mode may be switched.

According to an embodiment of the present invention, when an error occurs in an object without a signal such as a beacon or a signal such as a beacon at the work site (or work space), furthermore, variously converted real-time situations or situation information in the field in VR You will be able to check it quickly and accurately through the video.

In addition, according to an embodiment of the present invention, it is possible to monitor the work site in real time with VR images if necessary, so that it is possible to more quickly respond to incidents and accidents.

1 is a view showing a virtual reality-based workplace safety management system according to an embodiment of the present invention;
Figure 2 is a block diagram illustrating the detailed structure of the construction site safety management device of Figure 1;
3 is a block diagram illustrating the structure of the VR image processing unit of FIG. 2;
4 is a flowchart showing a driving process of a virtual reality-based workplace safety management device according to an embodiment of the present invention, and
5 is a flowchart illustrating another driving process of a virtual reality-based workplace safety management device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a diagram illustrating a virtual reality-based workplace safety management system according to an embodiment of the present invention.

1, the virtual reality-based workplace safety management system 90 according to an embodiment of the present invention includes a user device 100, field devices 113, 115, 117, a communication network 120 and ( Virtual reality-based) includes a part or all of the construction site safety management device (130).

Here, "including some or all" means that some components such as the scanner device 115 constituting the field devices 113, 115, and 117 are omitted so that the system 90 is configured, or a construction site safety management device Some or all of the components constituting the communication network 130 are integrated into a network device (eg, a wireless switching device, etc.) constituting the communication network 120 and may be configured to help a sufficient understanding of the invention. In order to be described as including all.

The user device 100 is, for example, a beacon device provided in a work cap worn by a worker or supervisor of a construction site (or work site), such as a tunnel block, or a terminal device such as a smart phone possessed by the worker, furthermore, the location of the worker It includes various devices such as a dedicated terminal equipped with a trackable module (eg, short-range communication module). Of course, it may include a wearable device such as a domestic S company's Galaxy Gear that the worker wears on his or her wrist. In an embodiment of the present invention, it is preferable to use a beacon device that transmits a beacon signal in consideration of aspects such as construction cost without threatening the safety of the work, but it will not be particularly limited thereto. That is, if location tracking is possible within the field, various communication devices capable of short-range wireless communication may be used.

The field devices 113 , 115 , and 117 include a camera 113 , a scanner device 115 , and a communication relay device 117 , and may further include various devices capable of detecting the inside of the construction site. Representatively, it may further include a detection sensor such as a mobile robot that moves within the construction site, monitors through shooting, etc. and transmits a warning broadcast, or a gas sensor that senses the internal environment of the construction site. Of course, the detection sensor can be mounted on the mobile robot. The camera 113 may include various types of cameras, such as a 3D camera, a CCTV, and an IP camera. For example, as shown in FIG. 1 , a plurality of cameras may actually be installed in the tunnel curtain wall. In addition, a plurality of cameras are installed at different positions to facilitate the creation of VR images inside the curtain to photograph the interior and provide the captured images to the construction site safety management device 130 through the communication relay device 117 . For example, in the case of a 3D camera, a 3D photographed image may be provided as a photographed image, but when a plurality of 3D cameras are installed, each 3D camera may transmit a 3D photographed image, respectively. Of course, each camera transmits information about the time taken or the identification information (ID) of the camera 113 when providing the captured image. The 3D camera may generate and further provide depth information on objects, ie, objects in the captured image by itself. However, since the 3D camera is expensive, it is preferable to apply CCTV in the embodiment of the present invention, but it will not be particularly limited thereto.

The scanner device 115 communicates with the surrounding user device 100 . The scanner device 115 is, for example, a beacon scanner, which receives a beacon signal of a beacon device mounted on a worker's hat and the like and provides it to the communication relay device 117 . Of course, the beacon scanner 115 may transmit its own device identification information (ID), device identification information of the received beacon device, and information on signal reception strength when providing the received beacon signal. The information is provided to the construction site safety management device 130 via the communication relay device (117). Of course, the scanner device 115 according to an embodiment of the present invention may include various types of communication devices capable of short-range communication if the location of a worker or equipment can be tracked by performing communication with the user device 100 in addition to the beacon scanner. can The beacon can operate at a distance of up to 50m, and it can be very useful at the construction site because it uses wireless communication technologies such as infrared and RF to communicate at a short distance of less than 100m. A plurality of devices may be installed in the scanner device 115 or the construction site, for example, the construction site safety management device 130 measures the distance based on which scanner device 115 strongly detects a signal of a specific beacon device. location can be confirmed. Of course, scattering or reflection of beacon signals may occur inside the construction site, and various methods of measurement such as setting a plurality of scanner devices 115 as a group to increase accuracy may be performed. For example, if the signal is the strongest from the scanner device 115 of a specific group and the signal is continuously received, the position can be calculated based on the signal.

Communication may not be smooth in a closed area such as tunnel construction. For example, GPS communication is not possible. Therefore, the communication relay device 117 according to an embodiment of the present invention can be installed at the entrance of the makjang to relay so that smooth communication is made between the user device 100 outside the makjang and inside the makjang. By adjusting the signal radiation direction of the antenna, it may be possible to transmit and receive signals to the inside. The communication relay device 117 may of course be an access point (AP). Accordingly, it may be included in the concept of the communication network 120 . However, in that the communication relay device 117 can be directly installed and used by a construction company, etc., it was intended to be distinguished from the communication network 120 installed by a key communication company.

The communication network 120 includes both wired and wireless communication networks. For example, a wired/wireless Internet network may be used or interlocked as the communication network 120 . Here, the wired network includes an Internet network such as a cable network or a public telephone network (PSTN), and the wireless communication network includes CDMA, WCDMA, GSM, Evolved Packet Core (EPC), Long Term Evolution (LTE), and Wibro networks. meaning to include Of course, the communication network 120 according to the embodiment of the present invention is not limited thereto, and may be used, for example, in a cloud computing network under a cloud computing environment, a 5G network, and the like. For example, if the communication network 120 is a wired communication network, the access point in the communication network 120 may connect to a switching center of a telephone company, etc., but in the case of a wireless communication network, it accesses the SGSN or GGSN (Gateway GPRS Support Node) operated by the communication company to transmit data. or by connecting to various repeaters such as Base Station Transmission (BTS), NodeB, and e-NodeB to process data.

The communication network 120 may include an access point. Here, the access point includes a small base station such as a femto or pico base station that is often installed in a building. Femto or pico base stations are classified according to the maximum number of communication relay devices 117, etc. that can be connected to the classification of small base stations. Of course, the access point may include a short-distance communication module for performing short-distance communication such as the communication relay device 117 and the like, such as Zig-bee and Wi-Fi. The access point may use TCP/IP or Real-Time Streaming Protocol (RTSP) for wireless communication. Here, short-range communication may be performed in various standards such as Bluetooth, Zigbee, infrared, radio frequency (RF) such as ultra high frequency (UHF) and very high frequency (VHF), and ultra wideband communication (UWB) in addition to Wi-Fi. Accordingly, the access point extracts the location of the data packet, designates the best communication path for the extracted location, and forwards the data packet to the next device, such as the construction site safety management device 130, along the designated communication path. . The access point may share several lines in a general network environment, and includes, for example, a router, a repeater, and a repeater.

The construction site safety management device 130 may include various devices for monitoring the workers of the construction site. Representatively, it may include a server for processing data provided at a construction site, and may further include, for example, a computer body or a monitor that a control agent performs control. Of course, it may include a VR device such as a controller or manager's smartphone or a head mounting device (HMD). On the monitor, the internal appearance of the construction site and the location of a worker performing work from the inside may be displayed. Various additional information may be displayed. Such a monitoring screen may be called a 'dashboard'. Of course, the monitoring screen may be configured in various forms and implemented on the screen. Usually, the inside of a construction site is monitored through CCTV, but it is difficult to check the location and information of an accurate object (eg, object/person, etc.) in real time through this. To compensate for this, information on an object in a certain space can be checked using a beacon or a beacon scanner. However, even in this case, when an error occurs in an object in which a signal is not detected or a signal detection, as well as variously changing real-time situation information in the field, there are many cases.

To this end, the construction site safety management device 130 according to the embodiment of the present invention provides a photographed image of the camera 113 provided from the field devices 113, 115, and 117 installed in the tunnel or the scanner device 115, etc. A VR image may be generated based on scan data of the user device 100 , for example, a beacon device, and it may be switched to a VR image mode to be monitored. For example, when it is determined that an incident or accident occurs at the construction site, or an error has occurred in the beacon device, etc., when an object with no beacon signal is detected, and also at the request of the controller, it can be switched to the VR video mode. . The VR image is preferably a 360-degree VR image. For example, a 3D image displayed based on each point where the camera 113 is installed may be used. In addition, when the controller designates a specific location, a 3D image may be displayed on the screen based on the location. In this case, for example, a beacon device in which a beacon signal is not detected can be more easily identified.

The construction site safety management device 130 may produce a VR background image for the inside of the construction site by using, for example, a captured image obtained when the camera 113 is initially installed at the construction site to generate a VR image, and may be stored in advance. . Here, the background refers to a portion excluding the object when viewed in the captured image. Therefore, it can be seen as a part excluding the object from the internal appearance of the construction site. Since the appearance of the inside of the construction site changes as time elapses through blasting work, etc., the construction site safety management device 130 may generate the background image as a VR image in real time or at regular time intervals in consideration of this. For example, the construction site safety management device 130 may install a 3D camera to observe the interior of the construction site in 3D. However, to implement 360-degree VR, the construction site safety management device 130 stitches unit frame images of captured images received from a plurality of cameras at the construction site by time, more precisely, by stitching unit frame images of the same time to each other. ) to create a real-time VR image. For example, a VR image may be generated by stitching unit frame images corresponding to the same time with respect to time t1. For example, in the captured image received from camera 1, a unit frame image corresponding to time t1 (eg, 1/60 second) and a unit frame image corresponding to the same time t1 received from camera 2 provided at another location on the construction site can be obtained and stitched together. Of course, in this process, a calibration operation for correcting an image may be performed. Through this, it is possible to convert images taken from multiple viewpoints into a panoramic image for multiple viewpoints, and the image can be checked by changing the viewpoint according to the user's intention. Since the VR background image may be generated using streaming data transmitted in real time, the embodiment of the present invention will not be limited to any one form. Of course, the VR background image may be made intermittently when there is an event, such as a blasting operation, in the construction site in order to reduce the data calculation throughput.

In addition, the construction site safety management device 130 according to an embodiment of the present invention determines the distance based on the captured image received from the plurality of cameras 113 and the communication signal between the user device 100 and the scanner device 115 . can be measured The scanner device 115 may also be installed in various locations, and accurate distance measurement may be performed by triangulating the user device 100 . Of course, the measurement distance can be calculated using the received signal strength. For example, when a 3D camera is used as the camera 113 , since the operator also provides depth information about the equipment, that is, a VR image can be generated using this. The VR image corresponds to an image formed by overlapping a VR background image and an object. In the case of an IP camera or CCTV other than a 3D camera, depth (or depth) information can be calculated through image data, ie, pixel analysis. For example, a distance may be measured using pixels in a unit frame image of a captured image, but, for the depth information, for example, beacon information, that is, a distance calculated by a signal reception direction or reception signal strength may be used as depth information. Here, the depth information may be position information on which position the object is located in the 3D space. Accordingly, the construction site safety management device 130 may generate three-dimensional information by further using information such as beacons, if two-dimensional information is obtained through pixel analysis.

For example, when the VR video control mode is switched to the VR video control mode under a specified condition (eg, when an incident or accident occurs) in order to monitor the construction site, the construction site safety management device 130 generates a pre-generated or real-time VR The VR image of the construction site created by overlapping (or synthesizing) objects (or object images, image files, etc.) in the construction site on the background image can be displayed on the monitor. Here, the mode may mean that when a specified condition is met (or when there is a selection), a specified operation is automatically operated, or a predetermined series of operations are sequentially operated. In the process of generating the VR image of the construction site, the construction site safety management device 130 performs a deep learning operation to increase the accuracy of the overlapping objects in relation to the overlapping objects, and is finally determined by the deep learning result. Objects can be viewed as overlapping. For example, when the user device 100 that was sensed in real time is suddenly not detected, a clear determination as to why the user device 100 is not detected may be made by switching to the VR image control mode. This allows prompt action to be taken. In addition, in the space where there is a blasting operation, new objects related to the blasted fragments can be accurately detected through deep learning and implemented in a VR image to further increase the accuracy of monitoring the inside of the construction site.

FIG. 2 is a block diagram illustrating a detailed structure of the construction site safety management device of FIG. 1 .

As shown in FIG. 2 , the construction site safety management device 130 according to an embodiment of the present invention includes a communication interface unit 200 , a control unit 210 , a VR image processing unit 220 , and a part of the storage unit 230 . or all inclusive.

Here, "including some or all" means that some components such as the storage unit 230 are omitted, or some components such as the VR image processing unit 220 are integrated into other components such as the control unit 210. As meaning that can be configured and the like, it will be described as including all in order to help a sufficient understanding of the invention.

The communication interface unit 200 may communicate with the user device 100 via the communication network 120 and the communication relay device 117 of FIG. 1 . In the process of performing such a communication operation, the communication interface unit 200 may perform operations such as modulation/demodulation, encoding/decoding, muxing/demuxing, and scaling for converting resolution. In this regard, since it is obvious to those skilled in the art, further description will be omitted.

The communication interface unit 200 receives, as monitoring data of the construction site, typically image data for a captured image of the camera 113 and scan data provided by the scanner device 115 , and provides it to the control unit 210 . The scan data may be, for example, location tracking information for tracking the location of the user device 100 such as a beacon device or a smart phone.

In addition, the communication interface unit 200 may perform an operation for controlling the inside of the construction site with a VR image, for example, when an event (eg, an event, an accident) occurs in the construction site or there is a request from a control agent or supervisor. . To this end, VR image data can be transmitted so that the VR image is displayed on the computer or monitor of the control personnel or the electronic display of the control center.

The control unit 210 may perform overall control operations of the communication interface unit 200, the VR image processing unit 220 and the storage unit 230 of FIG. 2 constituting the construction site safety management device 130 of FIG. 1 . . Representatively, the control unit 210 receives the image data for the photographed image of the construction site received through the communication interface unit 200 and the scan data for sensing the position of the worker or equipment, and temporarily stores it in the storage unit 230 . After that, if necessary, it may be called and provided to the VR image processing unit 220 .

The controller 210 may, for example, generate a VR background image of the construction site by using the captured image of each camera when the camera 113 is initially set at the construction site. In addition, the pre-generated VR background image can be updated in real time or periodically whenever the internal appearance of the construction site is changed. For this, of course, it may operate in conjunction with the VR image processing unit 220 . Also, when the controller 210 is switched to the VR image control mode of the construction site, the controller 210 may output a VR image generated by overlapping objects on a pre-generated VR background image.

The VR image processing unit 220 generates a VR image for monitoring the inside of the construction site with real-time VR. Here, the VR image is composed of a VR background image excluding an object in the interior of the construction site and an object or object images overlapping the background image. Of course, the object here can be observed all objects whose location is not tracked by a beacon or the like. For example, in the case of a conventional CCTV, it may be difficult to check the exact location or information of the objects in real time. Typically, it may be a problem of resolution. In addition, even when an IP camera, a 3D camera, etc. are used, it is inevitable to observe the inside of the construction site by limiting the installation location aside from the expensive installation cost.

However, the VR image processing unit 220 according to the embodiment of the present invention provides a VR image based on the location of the construction site using the captured images obtained from a plurality of cameras. 360 degree VR video is possible. For example, unit frame images corresponding to time t1 may be acquired from the captured images of the camera 1 and the camera 2, respectively. Also, an interpolated image located in the middle may be generated using two unit frame images. In this way, a 360-degree VR image can be created by creating an interpolated image for each angle and then stitching all the images.

Also, the VR image processing unit 220 may set and store coordinate values in the generated VR image when the VR image is generated, so that the VR image can be controlled based on the set coordinate value. Of course, such coordinates, that is, the view to be controlled, are provided as two-dimensional coordinate values. For example, although a two-dimensional coordinate value is requested by the controller, it is converted into a three-dimensional coordinate value and the VR image is controlled based on this. can make it go away Of course, in order to perform such an operation, the two-dimensional coordinate value and the three-dimensional coordinate value may be matched and stored in a lookup table (LUT), etc., and then the VR image may be output based on this.

Above all, the VR image processing unit 220 accurately distinguishes an object by a deep learning operation in order to insert or overlap an accurate object in the VR background image. For example, assuming that a blasting operation is performed within a construction site, it may be difficult to accurately identify an object such as a stone newly created by this. In particular, it may be more difficult to distinguish through the captured images obtained through CCTV. Therefore, in this case, a deep learning operation for object classification may be performed for accurate identification, and the object finally classified by deep learning may be overlapped with the background image. Above all, although depth information of an object is required for overlap, it has been described above that it can be acquired in various forms. When a 3D camera is used, the camera itself provides depth information, so it can be used. If an IP camera or CCTV is used, pixel analysis is used, but scan data scanned by the scanner device 115 may be further used. For example, suppose that the beacon device of worker B is not detected or an error occurs between worker A and worker C in the construction site. In this case, the distance of worker B can be inferred based on the distance between workers A and C. Therefore, the corresponding distance value may be used as depth information.

For example, in the case of specifying camera 1 to control the construction site while watching VR images, and the case of specifying camera 2 to control the construction site while watching VR images, the viewing position, that is, the view is different, so each It can be seen that a VR image is generated and an image thereof is output. For this purpose, the VR image processing unit 220 may pre-generate a VR image corresponding to each location, but it may be possible to generate and display the VR image in real time. Since the VR image generation method has been sufficiently described above, we will replace it with the contents. However, if multiple views are used, data processing for VR image generation increases, so selectable viewpoints can be set as default, in other words, selectable viewpoints can be reduced and one can be selected for viewing. have.

Also, the VR image processing unit 220 may transmit the VR image to a separate VR device that implements the VR image in conjunction with the controller 210 . Therefore, the VR image processing unit 220 outputs the existing photographed images or other images of the construction site to a dashboard such as an electric sign board, but the VR image may be controlled through a VR device such as a head mounting device (HMD). have. The VR device may be provided with a panel such as a (micro) lenticular lens capable of realizing 3D, so that it may be possible to implement a VR image.

The storage unit 230 may temporarily store various types of information or data processed under the control of the control unit 210 , and may output the information when there is a request from the control unit 210 . The output data may be provided to the VR image processing unit 220 . Here, it is meaningless to distinguish the meaning of information or data, and it is common to mix them. The information may mean a control command such as a request (REQ) or a response (ACK), and the data may mean image data of a captured image or scan data of the scanner device 115 . Also, when viewed through the data packet, the header portion includes additional information and the like, and the payload portion includes pixel data and the like. Although it can be understood to this extent, in the embodiments of the present invention, the concept of such terms is not particularly limited.

Meanwhile, as another embodiment of the present invention, the control unit 210 of FIG. 2 includes a CPU and a memory, and the CPU and the memory may be configured as a single chip. The control unit 210 may include a control circuit, an operation unit (ALU), a command interpretation unit, a registry, and the like, and the memory may include a RAM. The control circuit may perform a control operation, the operation unit may interpret the operation processing of binary bit information, and the instruction interpreter may interpret whether the input binary bit information is related to the performance of any operation. Here, the analysis may be that after matching the data on the left and right sides of the lookup table (LUT), the control circuit finds the data on the right based on the data on the left and performs the operation. The registry is involved in software data storage. According to the above configuration, the CPU copies the program of the VR image processing unit 220 when the construction site safety management device 130 is initially driven, loads it into the memory, and executes it to quickly increase the data processing speed.

3 is a block diagram illustrating the structure of the VR image processing unit of FIG. 2 .

The VR image processing unit 220 ′ of FIG. 3 is a modified example of the VR image processing unit 220 of FIG. 2 , and includes a VR background generation unit 300 , an object deep learning unit 310 , an object image generation unit 320 , And the (background and object) VR image generating unit 330 includes some or all.

Here, “including some or all” means that some components such as the object image generator 320 are omitted, or the VR background generator 300 or the object image generator 320 is a VR image generator As meaning that can be configured by being integrated in (330), it will be described as including all in order to help a sufficient understanding of the invention.

The VR background generating unit 300, the object deep learning unit 310, the object image generating unit 320, and the VR image generating unit 330 of FIG. 3 are a hardware (H/W) module, a software (S/W) It may be configured by a module or a combination thereof.

The VR background generating unit 300 generates a VR background image of the interior of the construction site based on the captured image received from the camera 113 . Here, the VR background image may use a 3D captured image received through a 3D camera, or may use a captured image captured through an IP camera or CCTV. Of course, in the case of a 3D photographed image, it may be easier to create a VR image. A 360 VR background image can be obtained by generating a VR image for each angle using the received 3D captured image.

In addition, as mentioned above, the VR background generating unit 300 may generate a VR background image by stitching unit frame images corresponding to the same time to the captured images of the cameras 113 installed at different locations. . For example, with respect to camera 1, camera 2 corresponds to a position having a specific angle. Accordingly, by combining the unit frame image of the corresponding camera 2 with the unit frame image of the camera 1, a VR image for each angle can be generated. Of course, by creating and combining an interpolated image between the two images, it will be possible to realize a smoother, that is, a flexible VR image in many angle changes.

The object deep learning unit 310 may include, for example, a template-based Deep Convolution Neural Network (DCNN) classifier for object classification, and use the classification result to increase the object identification accuracy through deep learning such as artificial intelligence (AI). can For example, a stone can be typically detected when there is a blasting operation at a construction site, but various objects can be detected, so it can be clearly classified through deep learning and implemented in a VR image.

The object image generator 320 calculates depth information of an object from a received captured image or, since depth information is provided from a 3D captured image, it may be used. Also, the object image generating unit 320 may extract an object in the form of an image from a unit frame image of the received captured image and manage it. Imaging may include operations such as graphic processing or correction of the extracted object. The object image generator 320 may provide the extracted object image to the background and object VR image generator 330 to insert the extracted object image into the VR background image according to the depth information.

The VR image generating unit 330 creates a VR image by overlapping the object image (or object image) on the VR background image. The overlap may mean that the layer of the VR background image and the layer of the object image are different from each other like in augmented reality, and the image of the object image is implemented by maintaining a transparent state in the remaining area except for the object. However, in the embodiment of the present invention, it will not be particularly limited to such a method. Through this, it is possible to obtain the same effect as controlling the internal space at a specific location on the construction site.

4 is a flowchart illustrating a driving process of a virtual reality-based workplace safety management device according to an embodiment of the present invention.

Referring to FIG. 4 together with FIG. 1 for convenience of explanation, the (virtual reality-based) construction site safety management device 130 of FIG. 1 according to an embodiment of the present invention is a photographed image of a work site from a camera of the work site. is received (S400).

In addition, the construction site safety management device 130 generates a virtual reality (VR) background image of the appearance of the work site using the received captured image, and based on the deep learning result of the object extracted from the received captured image, the object image, and synthesize (or combine, overlay) the generated VR background image and the imaged object to generate and output a VR image of the worksite (S410).

Here, imaging an object may mean extracting an object from a unit frame image and performing GUI graphic processing on the extracted object. Alternatively, by analyzing the pixel value of the extracted object, it can be implemented in the VR background image.

In addition, the workplace safety management device 130 may perform various operations, and since the other detailed information has been sufficiently described above, it will be replaced with the contents.

5 is a flowchart illustrating another driving process of a virtual reality-based workplace safety management device according to an embodiment of the present invention.

Referring to FIG. 5 together with FIG. 1 for convenience of explanation, the workplace safety management device 130 of FIG. 1 according to an embodiment of the present invention provides an image provided by photographing the workplace by, for example, the camera 113 of the workplace. Receive (S500).

In addition, the workplace safety management device 130 generates a VR background image using the received captured image (S510). In order to generate a VR background image, the workplace safety management device 130 may generate an image for each angle of the received captured image, and may generate a VR background image in the form of combining them. Since this has been sufficiently explained above, we will replace it with those contents.

The workplace safety management device 130 extracts an object from the unit frame image of the received captured image, and performs learning on the extracted object to define the object (S520, S530). The object may typically be a worker (eg, beacon information) or equipment (eg, equipment data), but may be, for example, a stone newly introduced during various construction works or newly discovered during a blasting operation. You can search the learning DB for object definition.

In addition, the workplace safety management device 130 may generate depth information in order to insert the above extracted object into the VR background image. For example, in the case of using a 3D camera, since the camera provides depth information, the depth information can be utilized to apply it to a 360-degree VR image based on this. If it is difficult to calculate depth information from a unit frame image of a captured image, such as a CCTV or an IP camera, the depth information may be generated by using beacon information or the like. For example, when a specific object that is not tracked is located between the position of the worker A and the position of the worker B, for example, when the pixel analysis result is close to the position of the worker A, depth information may be generated based on this. In this way, when the depth information is generated, beacon information may be used. The coordinate value of the corresponding object can be set in memory as three-dimensional coordinates (x, y, z).

Furthermore, the workplace safety management device 130 images the object through pixel analysis of the extracted object, and creates a VR image by overlapping the generated VR background image (S540, S550). When overlapping the object image on the VR background, the above depth information may be reflected to enable the realization of a 360-degree VR image for each angle. For example, even in the case of an object, it may be implemented three-dimensionally through three-dimensional imaging.

In addition, the workplace safety management device 130 displays the VR image (S560). VR images can be implemented on electronic display boards in the control center, but it will be possible to watch them through a VR device that is separately worn by the control personnel or supervisors monitoring the construction site.

In addition to the above, detailed information related to the workplace safety management device 130 has been sufficiently described above, so it will be replaced with the contents.

On the other hand, even though it has been described that all components constituting the embodiment of the present invention are combined or operated in combination, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining one or more. In addition, although all of the components may be implemented as one independent hardware, some or all of the components are selectively combined to perform some or all functions of the combined components in one or a plurality of hardware program modules It may be implemented as a computer program having Codes and code segments constituting the computer program can be easily deduced by those skilled in the art of the present invention. Such a computer program is stored in a computer-readable non-transitory computer readable media, read and executed by the computer, thereby implementing an embodiment of the present invention.

Here, the non-transitory readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, memory, etc. . Specifically, the above-described programs may be provided by being stored in a non-transitory readable recording medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and it is common in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Various modifications may be made by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: user device 113, 115, 117: field installation device
120: communication network 130: construction site safety management device
200: communication interface unit 210: control unit
220: VR image processing unit 230: storage unit
300: VR background generation unit 310: object deep learning unit
320: object image generation unit 330: VR image generation unit

Claims (10)

a communication interface unit for receiving a photographed image of the work site from a camera of the work site; and
A virtual reality (VR) background image for the appearance of the work site is generated using the received captured image, the object is imaged based on the deep learning result of the object extracted from the received captured image, and the generated VR A control unit for generating and outputting a VR image of the work site by synthesizing the background image and the imaged object;
Virtual reality-based workplace safety management device including. According to claim 1,
The control unit extracts a unit frame image of the same time from each captured image received from a plurality of cameras installed at different locations in the work site and stitches each extracted unit frame image to the VR background A virtual reality-based workplace safety management device that generates images. 3. The method of claim 2,
The control unit analyzes the received captured image to acquire depth information of the extracted object, and applies the acquired depth information to synthesize the imaged object in the VR background image. management device. 4. The method of claim 3,
The control unit further uses a distance value calculated by beacon information received in relation to the extracted object in order to acquire depth information of the extracted object. 4. The method of claim 3,
The control unit, a virtual reality-based workplace safety management device for switching to a VR image control mode when an error is detected in an event or the object in the workplace. receiving, by the communication interface unit, a photographed image of the work site from a camera of the work site; and
The control unit generates a virtual reality (VR) background image for the appearance of the work site using the received captured image, and images the object based on the deep learning result of the object extracted from the received captured image, A step of synthesizing the generated VR background image and the imaged object to generate and output a VR image of the work site;
A driving method of a virtual reality-based workplace safety management device including. 7. The method of claim 6,
The step of generating and outputting the above is,
Based on virtual reality that extracts unit frame images of the same time from each captured image received from a plurality of cameras installed at different locations in the work site and stitches each extracted unit frame image to generate the VR background image The operation method of the workplace safety management device of 8. The method of claim 7,
The step of generating and outputting the above is,
A method of driving a virtual reality-based workplace safety management device that analyzes the received captured image to acquire depth information of the extracted object, and synthesizes the imaged object with the VR background image by applying the acquired depth information. 9. The method of claim 8,
The step of generating and outputting the above is,
A driving method of a virtual reality-based workplace safety management device further using a distance value calculated by beacon information received in relation to the extracted object in order to acquire depth information of the extracted object. 9. The method of claim 8,
The step of generating and outputting the above is,
A driving method of a virtual reality-based workplace safety management device for switching to a VR image control mode when an error is detected in an event or the object in the workplace.

Images