KR20240028081A - Smart Quality Control System of Construction Sites using Augmented Reality - Google Patents

Abstract

The smart quality management system at the construction site based on augmented reality can provide a new concept quality management platform that can check the process status and quality of the construction site and easily convey information based on digital twin and AR technology.
The present invention provides a platform for smart process and quality management at construction sites based on digital twin and AR technology, which has the effect of quickly and accurately supporting HMAT quality control and process management at large-scale construction sites.
The present invention enables efficient process management through real-time interaction between platforms, and has the effect of significantly improving economic efficiency, such as reducing major defects, combinations, loss costs, and required costs at construction sites.

Description

Smart Quality Control System of Construction Sites using Augmented Reality}

The present invention relates to a smart quality management system at a construction site. More specifically, it can provide a new concept quality management platform that can check the process status and quality of a construction site and easily transmit information based on digital twin and AR technology. This is about a smart quality management system at construction sites based on augmented reality.

At construction sites, supervision is required to ensure that buildings, building facilities, or structures are constructed according to the design documents and to supervise quality control, construction management, and safety management in accordance with the Building Act.

Recently, smart construction technology has been introduced in various fields of the construction industry to improve productivity, quality control, and safety of construction work in accordance with the 4th Industrial Revolution.

Our country's construction has low productivity, the construction workforce is aging and the number of skilled workers is rapidly decreasing, and there is a need for digitalization and automation of construction technology.

In response to these changes in the construction situation, construction companies are rapidly pursuing automation and digitalization in areas such as quality management.

In the case of large-scale construction sites, there is a problem that it is difficult for workers to individually check all information (location, status, etc.), including the construction process and quality status. At construction sites, there is a lack of quality managers and limitations in information management based on 2D drawings.

Korean Patent No. 10-2402845

In order to solve this problem, the present invention is an augmented reality-based construction site that can provide a new concept quality management platform that can check the process status and quality of the construction site and facilitate information transfer based on digital twin and AR technology. The purpose is to provide a smart quality management system.

The smart quality management system for augmented reality-based construction sites according to the characteristics of the present invention to achieve the above purpose is,

Using a laser distance meter, 3D scanning is performed along the scanning path inside or outside the construction site to generate 3D scanning data, and the 3D scanning data is converted into points by preset 3D coordinates (x, y, z). A 3D scanning device that labels 3D point cloud data, which is pointized 3D scanning data;

Video information is obtained by filming the interior and exterior of the construction site in video form along the same scanning path scanned by the 3D scanning unit, and a 3D video frame, which is video information, is converted into a 2D still image of the section at each section change on the scanning path. A video processing device that converts each video frame into video frames;

A construction site history database unit that stores construction site data in a database, including at least one of construction site material information, drawing information, equipment manual information, and site view data, which are actual objects in physical space; and

Generating 3D image information overlapped on point cloud data through 3D coordinate matching between 2D image frames received from the video processing device and 3D point cloud data received from the 3D scanning device, and generating the overlapped 3D image information and a digital twin platform that generates a 3D digital twin using the construction site data.

The digital twin platform further includes a virtual content database unit that stores virtual content data by matching the virtual coordinates of the 3D virtual space of the 3D digital twin to implement an Augmented Reality (AR) environment, and provides on-site video or /And providing an interface for generating the virtual content data corresponding to the physical space of the field based on the captured image of the field user to the user terminal.

With the above-described configuration, the present invention provides a platform for smart process and quality management at construction sites based on digital twin and AR technology, which has the effect of quickly and accurately supporting smart quality management and process management at large-scale construction sites.

The present invention enables efficient process management through real-time interaction between platforms, and has the effect of significantly improving economic efficiency, such as reducing major defects, combinations, loss costs, and required costs at construction sites.

Figure 1 is a diagram showing the configuration of an augmented reality-based smart quality management system at a construction site according to an embodiment of the present invention.
Figure 2 is a block diagram briefly showing the internal configuration of a 3D scanning device according to an embodiment of the present invention.
Figure 3 is a block diagram briefly showing the internal configuration of a video processing device according to an embodiment of the present invention.
Figure 4 is a diagram showing the configuration of a drone device according to an embodiment of the present invention.
Figure 5 is a block diagram briefly showing the internal configuration of a digital twin platform according to an embodiment of the present invention.
Figure 6 is a diagram showing a data flow diagram for smart process and quality control at a construction site according to an embodiment of the present invention.
Figure 7 is a diagram showing the visualization of AR navigation based on location information and AR status information where quality status information can be viewed according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components.

Terms such as first, second, A, B, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term “and/or” includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate overall understanding when describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

Figure 1 is a diagram showing the configuration of a smart quality management system at an augmented reality-based construction site according to an embodiment of the present invention, and Figure 2 is a block diagram briefly showing the internal configuration of a 3D scanning device according to an embodiment of the present invention. , Figure 3 is a block diagram briefly showing the internal configuration of a video processing device according to an embodiment of the present invention.

The smart quality management system 100 for an augmented reality-based construction site according to an embodiment of the present invention includes a 3D scanning device 110, a video processing device 120, a user terminal 130, a digital twin platform 140, and a drone device. Includes (150).

The 3D scanning device 110 includes a 3D scanning unit 111, a scanning data storage unit 112, a scanning processing unit 113, and a first communication unit 114.

The 3D scanning unit 111 performs 3D scanning along a scanning path inside or outside the construction site using a laser distance meter to generate 3D scanning data and stores it in the scanning data storage unit 112. Here, the scanning path establishes a specific point inside or outside the construction site where 3D scanning is performed.

The 3D scanning unit 111 generates 3D scanning data by performing 3D scanning at a specific point of the scanning path. When generating 3D scanning data, coordinate information is also generated and stored in the scanning data storage unit 112.

A laser range finder emits laser light toward a subject, which is a structure, and receives laser light reflected from the subject to generate 3D scanning data including depth information.

The scanning processing unit 113 controls the overall operation of the 3D scanning device 110 and is electrically connected to the 3D scanning unit 111, the scanning data storage unit 112, and the first communication unit 114 to control the operation.

The scanning processing unit 113 points the 3D scanning data by preset 3D coordinates (x, y, z) for the entire shape, including the boundary lines, planes, and curved surfaces of the whole and part of the three-dimensional shape, and then points the 3D scanning data. , 3D point cloud data, which is pointized 3D scanning data, is labeled and stored in the scanning data storage unit 112.

Here, the 3D point cloud may be a set of a plurality of points belonging to a 3D coordinate system. In a 3D coordinate system, points can be defined as (x-coordinate, y-coordinate, z-coordinate), and each of the points can be distributed along the surface of the 3D object to be expressed.

A point cloud is a group of numerous points that are densely emitted from a laser rangefinder, reflected from an object, and returned to the receiver. Each group has x, y, and z position coordinates, so it targets the spatial distance to each point based on the central coordinate system. This refers to information provided in the form of a point cloud, a large number of points containing the corresponding 3D location through dense sampling of the surface. Visualizing the point cloud data in 3D form enables free movement of viewpoints and enables measurement of space. Information can be provided.

The scanning processing unit 113 transmits the 3D point cloud data stored in the scanning data storage unit 112 to the digital twin platform 140 through the network 101 using the first communication unit 114.

The network 101 is a communication network that is a high-speed backbone network of a large communication network capable of high-capacity, long-distance voice and data services, and may be a next-generation wired or wireless network to provide the Internet or high-speed multimedia services. If the network 101 is a mobile communication network, it may be a synchronous mobile communication network or an asynchronous mobile communication network. An example of an asynchronous mobile communication network may be a WCDMA (Wideband Code Division Multiple Access) type communication network. The network 101 serves to transfer signals and data between the systems of the 3D scanning device 110, video processing device 120, user terminal 130, and digital twin platform 140.

As shown in FIG. 3, the video processing device 120 according to an embodiment of the present invention includes a camera unit 121, an image frame input unit 122, a first frame switching unit 123, and a second frame switching unit ( 124), a control unit 125, an image capture unit 126, and an image frame storage unit 127.

The camera unit 121 captures the interior and exterior of the construction site in video format along the same scanning path scanned by the 3D scanning unit 111.

The video frame input unit 122 receives a video file of the construction site from the camera unit 121 and obtains a video frame.

The first frame conversion unit 123 divides each image frame into macroblocks having a size of N×N pixels and converts the divided macroblocks into a black-and-white image that indicates a boundary line.

The first frame conversion unit 123 converts each image frame into an edge image using an edge detection algorithm for a plurality of macro blocks.

The edge detection algorithm means that only the section in which the pixel value of the boundary line is found to have a larger difference than the preset reference pixel value is identified as an edge. As another embodiment, the edge detection algorithm may detect edges using the rate of change of brightness values between pixels using the Sobel operation method.

The first frame conversion unit 123 divides each image frame into macro blocks with a size of 16 × 16 pixels, converts the divided macro blocks into a black and white image indicating a boundary line, and uses an edge detection algorithm. To detect a section transition using , the edge image of the N-1th video frame and the edge image of the Nth video frame are generated 1/24 second ago.

The first frame switching unit 123 calculates the difference value between the macro blocks at the same position in the N-1th image frame and the N-th image frame, and when the difference value between macro blocks is less than the preset first threshold, the similar It is determined to be a macro block, and if the difference value between macro blocks is greater than the preset first threshold (T1), the scene is determined to be a different macro block.

The first frame switching unit 123 adds up the number of all similar macro blocks in the video frame, and if the summed number of similar macro blocks is greater than the preset second threshold T2, it determines that section switching has occurred. .

The video of the N-1th video frame and the video of the Nth video frame have a temporal difference of 1/24 second, and are similar because they have locality characteristics.

The second frame conversion unit 124 measures the peak signal-to-noise ratio (PSNR) for each video frame, and measures the signal ratio between the video signal of the N-1th video frame and the video signal of the Nth video frame. If the difference in noise ratio (PSNR) is greater than the preset third threshold (T3), it is determined that the section change has occurred.

PSNR represents the power of noise relative to the maximum power that a signal can have. It is used when evaluating picture quality loss information in video or video loss compression and is defined as follows in Equation 1.

Here, MAX is the maximum value of the corresponding video, and can be obtained by subtracting the minimum value from the maximum value of the corresponding channel. For example, in the case of an 8-bit gray scale image, it is 255 (255 - 0). MSE (Mean Square Error) is the mean square error and measures the average of the squared errors, that is, the mean square difference between the estimated value and the actual value.

When the difference from the reference original image is large, the MSE value increases (or the PSNR value decreases).

The control unit 125 determines the error range of the first section change point where the section change is made in the first frame change unit 123, and determines the error range of the second section change point where the section change is made in the second frame change unit 124 (113). Determine the error range of the point. The error range represents the range when the section change point is narrow or very wide beyond the normal section change point.

The control unit 125 may select and detect a normal section change point, except for cases in which the first frame change unit 123 or the second frame change unit 124 falls within an error range.

In another embodiment, the control unit 125 calculates the average value of the first section change point at which the section change is made in the first frame change unit 123 and the second section change point at which the section change is performed in the second frame change unit 124. It can be selected as the final section transition point.

The image capture unit 126 finds the section transition point and then calculates the duration of each section.

The video capture unit 126 counts the number of video frames from the video frame in which the previous section change occurred to the next section change, and when receiving a section change specific signal from the control unit 125, stores the counted number of video frames. .

The image capture unit 126 calculates the duration of each section using the counted number of image frames and the preset number of frames per second according to the input image.

For example, in the case of a movie, assuming 24 frames/sec, if the next section transition occurs in the 100th video frame from the video frame in which the previous section transition occurred, it means that the previous scene was maintained for 4.16 (100/24) seconds. it means.

The image capture unit 126 captures and extracts video frames in which section transitions have occurred using the calculated duration of each section.

Here, section switching refers to when a building site is photographed at one specific point of the aforementioned scanning path and then moved to another specific point.

The control unit 125 labels the image frames extracted from the image capture unit 126, stores them in the image frame storage unit 127, and transmits the stored image frames to the digital twin platform 140 through the second communication unit 128. do. The video processing device 120 converts a 3D image frame, which is video information, into a 2D image frame at each section change on the scanning path.

Here, the section represents the point at which image information is acquired by filming at a specific point of the construction site.

In the case of large-scale construction sites, it is difficult for site and construction managers to manually take pictures using cameras due to the high interior ceiling height of high-rise buildings or buildings.

In this case, the manager obtains image information by using the drone device 150 to photograph a location where it is difficult to obtain image information.

Figure 4 is a diagram showing the configuration of a drone device according to an embodiment of the present invention.

The drone device 150 according to an embodiment of the present invention includes a flying robot body 152 forming a skeleton, and a quadrotor portion 153 formed across the flying robot body 152 in the east, west, north, south, and south directions.

The flying robot body 152 and the quadrotor unit 153 are made of lightweight and durable aluminum alloy steel, and propellers 154 and 155 are installed on the upper surfaces, respectively. The propellers 154 and 155 may include propellers of a certain shape, and an airflow in the downward direction is generated by the operation of the propellers 154 and 155, which creates a lift force that allows the drone device 150 to fly. This happens.

The flying robot body 152 has a camera device 156 and a lidar sensor unit 157 installed on its lower surface to photograph the outer wall of a high-rise building or a high-positioned object inside the building.

The processor (not shown) is formed on one side of the flying robot body 152 and generates coordinate information when photographing an object using the camera device 156, labels and stores it together with the image information, and stores the image information including the coordinate information. It is transmitted to the user terminal 130 through a third communication unit (not shown).

Each of the plurality of drone devices 150 may be connected to the user terminal 130 through wireless communication through the network 101 and may operate according to a control signal received from the user terminal 130.

Each of the plurality of drone devices 150 is equipped with a camera device 156, and can transmit captured images through the camera device 156 to the user terminal 130.

Each of the plurality of drone devices 150 is equipped with a 3-axis acceleration sensor, gyroscope, magnetometer, ultrasonic sensor, pressure gauge, etc., and can detect movement information such as current location, altitude, movement speed, and direction of movement. Movement information can be transmitted to the user terminal 130.

The user terminal 130 may be implemented as a computing device with a communication function, for example, but is not limited to a mobile phone, desktop PC, laptop PC, tablet PC, smartphone, etc., and may be implemented as an external device. It can also be implemented as various types of communication devices that can be connected to a server.

The user terminal 130 is connected to a plurality of drone devices 150 through the user terminal 130, and transmits a control signal corresponding to the command input by the user to the plurality of drone devices 150 to control the plurality of drone devices 150. Movement can be controlled to (150).

The user terminal 130 is equipped with a display unit (not shown) and can display and output various image information through the display unit. For example, the user terminal 130 may output a captured image received from a first drone device through the display unit, and may output a captured image received from a second drone device through the display unit.

The user terminal 130 transmits image information captured from each of the plurality of drone devices 150 to the digital twin platform 140.

The LiDAR sensor unit 157 generates a laser pulse signal and transmits it forward, and detects a reflected wave signal reflected from a building wall, etc.

The LiDAR sensor unit 157 receives a detection signal from a laser diode detection unit (not shown) that detects the driving signal of a laser diode (not shown) and calculates the departure time of the LiDAR time of flight (ToF). do.

The LiDAR sensor unit 157 receives the reflected wave signal reflected from the building wall and calculates the arrival time of the LiDAR flight time.

The lidar sensor unit 157 calculates the flight time from detection of the laser pulse signal to detection of the reflected wave signal using the departure time and arrival time.

The lidar sensor unit 157 calculates distance information by measuring the time difference between departure time and arrival time.

A processor (not shown) periodically receives distance information from the LiDAR sensor unit 157.

The processor (not shown) determines whether the distance information received from the lidar sensor unit 157 falls within the error range of the preset reference distance value, and if it falls within the error range, controls the camera device 156 to determine whether the distance information received from the lidar sensor unit 157 is within the error range of the preset reference distance value. When photographing an object located high on an external wall or inside a building, and it is outside the error range, the camera device 156 of the drone device 150 adjusts the distance between the drone device 150 and the object until it falls within the error range of the reference distance value. Control to stop filming. The reference distance value is a value set at a distance at which the resolution of the captured image or the shooting range of the object is optimized.

Figure 5 is a block diagram briefly showing the internal configuration of a digital twin platform according to an embodiment of the present invention.

The digital twin platform 140 according to an embodiment of the present invention includes a wireless communication unit 141, a digital twin storage unit 142, a construction site history database unit 143, a server control unit 144, and a three-dimensional digital twin unit 145. ), a virtual content database unit 146, and an augmented reality implementation unit 147.

The wireless communication unit 141 stores the 3D point cloud data received from the 3D scanning device 110 in the digital twin storage unit 142, and stores the image frame received from the video processing device 120 in the digital twin storage unit 142. ), and the image information received from the drone device 150 is stored in the digital twin storage unit 142.

The wireless communication unit 141 receives 2D image information captured from each of the plurality of drone devices 150 from the user terminal 130 and stores it in the digital twin storage unit 142.

The construction site history database unit 143 forms a database and stores construction site data including at least one of construction site material information, drawing information, equipment manual information, and site view data, which are actual objects in physical space.

The server control unit 144 converts the 2D image information received from the drone device 150 into 3D point cloud data corresponding to the coordinate values of the image information.

The server control unit 144 provides 3D image information overlapped on the point cloud data through 3D coordinate matching between the 2D image frame received from the video processing device 120 and the 3D point cloud data received from the 3D scanning device 110. is created and stored in the digital twin storage unit 142.

The server control unit 144 may convert 2D image information captured from each of the plurality of drone devices 150 into 3D point cloud data corresponding to the coordinate values of the image information, and then separately store and manage the data.

In addition, the server control unit 144 converts the 2D image information captured from each of the plurality of drone devices 150 into 3D point cloud data corresponding to the coordinate value of the image information, and then uses the coordinate value to scan the 3D scanning device ( It can be stored and managed along with the 3D point cloud data received from 110).

The server control unit 144 forms the 3D point cloud data of the 3D scanning device 110 in order according to the scanning path, and the 3D point cloud data of the drone device 150 also determines the corresponding position of the scanning path based on the coordinate value. You can insert and save it.

The 3D digital twin unit 145 is linked with the digital twin storage unit 142 to store overlapping 3D image information, 3D point cloud data of the drone device 150, respective image coordinates, 3D modeling coordinates, A 3D digital twin is created using construction site data. Here, the 3D digital twin may be one in which the state of the structure is reflected in the 3D modeled structure.

The three-dimensional modeling space may include at least one spatial grid, and the spatial grid may correspond to a specific actual location of the construction site.

The image information received from the drone device 150 is used to fill in the 3D modeling portion of the blank portion due to the absence of a captured image of the construction site.

In other words, a 3D digital twin connects real objects in a physical space to a 3D virtual space based on images taken of the physical space of the field and spatial information (depth data) that scans the positions of real objects located in the physical space. It models the corresponding virtual objects.

The 3D digital twin unit 145 matches and models material information, drawing information, and equipment manual information corresponding to each component included in the construction site structure.

The virtual content database unit 146 stores virtual content data for implementing an augmented reality (AR) environment.

The virtual content database unit 146 may further include a 3D virtual image modeling a virtual object for constructing a 3D digital twin.

This virtual content database unit 146 can match virtual content to real objects or spatial coordinates and store it as virtual content data. Here, virtual content data is virtual content generated by a computing device and includes labels, text information, images, drawing objects, and three-dimensional entities, and is augmented content created in response to physical space or images taken of physical space. .

Additionally, the virtual content database unit 146 may store virtual content data by matching it to virtual coordinates of the 3D virtual space of the 3D digital twin.

The augmented reality implementation unit 147 provides an augmented reality environment to the user terminal 130 through the wireless communication unit 141 with virtual content data and spatial information data. As a result, the server control unit 144 can provide the user terminal 130 with an interface for generating virtual content data corresponding to the physical space of the field based on the field image or/and the image captured by the field user, and can be generated in this way. The virtual content may be created and displayed in a location corresponding to the physical space.

The augmented reality environment is created by inserting related virtual content into the physical space around the user or communication partner, and refers to an environment in which information provided to the on-site manager (user terminal 130) is augmented.

The user terminal 130 has a built-in augmented reality application, can transmit and receive quality control information based on the interface provided by the digital twin platform 140, and can modify or add various information of the digital twin platform 140. It may be possible.

Figure 6 is a diagram showing a data flow diagram for smart process and quality control at a construction site according to an embodiment of the present invention, and Figure 7 shows AR navigation and quality status information based on location information according to an embodiment of the present invention. This is a diagram showing the visualization of viewable AR status information.

Site and construction managers and managers of construction specialized companies can connect to the digital twin platform 140 through the user terminal 130 to perform smart quality management functions at the construction site.

The user terminal 130 runs a mobile scanning app to obtain field images and/or images captured by field users and transmits them to the digital twin platform 140.

The digital twin platform 140 generates a 3D digital twin, which is 3D model information, and transmits it to the user terminal 130.

As shown in FIG. 7 through the mobile quality control AR app, the user terminal 130 provides AR navigation ((a) of FIG. 7) showing image information of a construction site 3D modeled in the user terminal 130, and AR The process and quality status of the construction site ((b) in Figure 7) can be confirmed through the visualization of the status information.

The user terminal 130 checks the quality status and inspection information through the built-in mobile quality control AR app, and if there is an abnormality, generates an abnormality control signal and transmits it to the digital twin platform 140. Map whether there is an abnormality in the quality control inspection information.

The digital twin platform 140 includes the location and status information of quality control inspection information where an error occurred and provides it to the mobile recovery task AR app built into the user terminal 130.

The user terminal 130 generates recovery and response operation information through the built-in mobile recovery operation AR app and transmits it to the digital twin platform 140.

Operations according to embodiments of the present specification can be implemented as a computer-readable program or code on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data that can be read by a computer system. Additionally, computer-readable recording media can be distributed across computer systems connected via a network, so that computer-readable programs or codes can be stored and executed in a distributed manner.

When the embodiment is implemented in software, the above-described techniques may be implemented as modules (processes, functions, etc.) that perform the above-described functions. Modules are stored in memory and can be executed by a processor. Memory may be internal or external to the processor and may be connected to the processor by a variety of well-known means.

Additionally, computer-readable recording media may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Program instructions may include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc.

Although some aspects of the invention have been described in the context of an apparatus, it may also refer to a corresponding method description, where a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method may also be represented by corresponding blocks or items or features of a corresponding device. Some or all of the method steps may be performed by (or using) a hardware device, such as a microprocessor, programmable computer, or electronic circuit, for example. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

In embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In embodiments, a field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can do it.

100: Smart Quality Management System 101: Network
110: 3D scanning device 111: 3D scanning unit
112: scanning data storage unit 113: scanning processing unit
114: First communication unit 120: Video processing device
121: Camera unit 122: Video frame input unit
123: first frame switching unit 124: second frame switching unit
125: control unit 126: video capture unit
127: video frame storage unit 128: second communication unit
130: User terminal 140: Digital twin platform
141: wireless communication unit 142: digital twin storage unit
143: Construction site history database unit 144: Server control unit
145: 3D digital twin unit 146: virtual content database unit
147: Augmented reality implementation unit 150: Drone device
152: Flying robot body 153: Quadrotor part
154, 155: propeller 156: camera device
157: Lidar sensor unit

Claims (5)

Using a laser distance meter, 3D scanning is performed along the scanning path inside or outside the construction site to generate 3D scanning data, and the 3D scanning data is converted into points by preset 3D coordinates (x, y, z). A 3D scanning device that labels 3D point cloud data, which is pointized 3D scanning data;
Video information is obtained by filming the interior and exterior of the construction site in video form along the same scanning path scanned by the 3D scanning unit, and a 3D video frame, which is video information, is converted into a 2D still image of the section at each section change on the scanning path. A video processing device that converts each video frame into video frames;
A construction site history database unit that stores construction site data in a database, including at least one of construction site material information, drawing information, equipment manual information, and site view data, which are actual objects in physical space; and
Generating 3D image information overlapped on point cloud data through 3D coordinate matching between 2D image frames received from the video processing device and 3D point cloud data received from the 3D scanning device, and generating the overlapped 3D image information A smart quality management system at a construction site including a digital twin platform that creates a 3D digital twin using the construction site data. In claim 1,
A drone device that is equipped with a camera device on one side of the flying robot body that forms the skeleton, photographs an object at a high position at the construction site, generates image information including coordinate information when photographing the object, and transmits it to the digital twin platform. It further includes,
The digital twin platform converts the 2D image information received from the drone device into 3D point cloud data corresponding to the coordinate value of the image information, and combines the overlapped 3D image information, the 3D point cloud data of the drone device, and , A smart quality management system at a construction site that generates the 3D digital twin using each image coordinate, 3D modeling coordinate, and construction site data. In claim 2,
The drone device further includes a lidar sensor unit installed near the camera device to periodically detect the distance to the front where images are to be captured,
The drone device determines whether the distance information received from the lidar sensor unit falls within the error range of the preset reference distance value, and if it falls within the error range, controls the camera device to control the distance information received from the lidar sensor unit. A smart quality control system at a construction site that photographs an object at a location and, when it falls outside the error range, controls the camera device to stop filming until the distance between the drone device and the object falls within the error range of the reference distance value. . In claim 1,
The digital twin platform further includes a virtual content database unit that stores virtual content data by matching virtual coordinates of the 3D virtual space of the 3D digital twin to implement an Augmented Reality (AR) environment, and provides on-site video Or/and a smart quality management system at a construction site that provides an interface for generating the virtual content data corresponding to the physical space of the site based on images captured by the site user to the user terminal. In claim 4,
The user terminal checks the quality status and inspection information through the built-in mobile quality control AR app, and if there is an abnormality, generates an abnormality control signal and transmits it to the digital twin platform,
The digital twin platform is a smart quality management system at a construction site that includes location and status information of quality control inspection information where abnormalities occurred and provides it as a mobile recovery task AR app built into the user terminal.

Images