KR102451336B1 - building digital-twin blockchain system for smart city by use of multi-layer video matching and rendering - Google Patents

Abstract

BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates generally to a technology that enables image-based evaluation of building quality. In particular, the present invention stores a database of exterior images, plumbing images, and non-destructive inspection images for each building in a smart city, and digitally matches and displays these images in multi-layers based on building architectural drawing (CAD) images, markers, NFC tags, and image feature points. It is about building digital twin block chain technology that technically supports to enable twin-style building quality evaluation and manages building-related information with high reliability on a distributed ledger through block chain. According to the present invention, by matching and displaying architectural drawing images, exterior images, piping images, and non-destructive inspection images in multi-layers for a building, people can see the quality of the building in augmented reality (AR) or virtual reality (VR). It has the advantage of being able to implement a digital twin of a building in a smart city by providing it. In addition, according to the present invention, there is an advantage in that it is possible to secure trust in the building quality evaluation result by managing various data for realizing the building digital twin of the smart city with the distributed ledger of the block chain.

Description

Building digital-twin blockchain system for smart city by use of multi-layer video matching and rendering}

BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates generally to a technology that enables image-based evaluation of building quality.

In particular, the present invention stores a database of exterior images, plumbing images, and non-destructive inspection images for each building in a smart city, and digitally matches and displays these images in multi-layers based on building architectural drawing (CAD) images, markers, NFC tags, and image feature points. It is about building digital twin block chain technology that technically supports to enable twin-style building quality evaluation and manages building-related information with high reliability on a distributed ledger through block chain.

Recently, there has been a lot of discussion about smart cities. A smart city refers to a city that acquires information using various types of electronic data collection sensors by information and communication technology (ICT), and manages and utilizes resources efficiently by this information. The purpose of a smart city is to solve traffic problems, environmental problems, housing problems, and facility inefficiencies caused in city life by using information and communication technology, and to enable citizens to enjoy a convenient and pleasant life.

As smart cities are expected to not only solve various problems of existing cities, but also to preemptively respond to the 4th industrial revolution and create new growth engines, countries around the world are actively building smart cities. When a smart city is built, traffic information can be obtained in real time, reducing the travel distance, making life easier for residents, such as remote work, and ultimately reducing carbon dioxide emissions.

In such a smart city, the management of buildings also needs to be done efficiently by information and communication technology. In general, there is no way to examine a building in detail, and whenever a serious problem occurs or a special necessity is recognized, a specialized company is commissioned to inspect the inside and outside of the building, and the method is checked while comparing it with the architectural design one by one. have. It is necessary to remove this inefficiency and conveniently perform building evaluation using information and communication technology.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a technology capable of evaluating the quality of a building in general based on an image.

In particular, an object of the present invention is to store a database of exterior images, piping images, and non-destructive inspection images for each building in a smart city, and match and display these images in multi-layers based on building architectural drawing (CAD) images, markers, NFC tags, and image feature points. The goal is to provide a building digital twin block chain technology that provides technical support to enable digital twin-style building quality evaluation and reliable distributed ledger management of building-related information through block chain.

On the other hand, the problem to be solved of the present invention is not limited to these matters, and other problems to be solved can be understood from the description of the present specification.

In order to achieve the above object, the building digital twin block chain system according to the present invention is an architectural drawing image, an outdoor or indoor exterior image, a piping image of a pipe endoscopy, and a non-destructive inspection image in order to configure a digital twin for a building. Image information generating apparatus 100 for generating a plurality; a cloud image server 200 for database management by receiving a plurality of architectural drawing images, exterior images, piping images, and non-destructive inspection images from the image information generating device 100 for constructing a digital twin of a building; A plurality of architectural drawing images, exterior images, piping images, and non-destructive inspection images for a specific building are provided from the cloud image server 200, and these images are multi-layered with a CAD mapping number based on markers and feature points to match the corresponding building. Image registration display server 300 for generating a matched image 301 for; A stereoscopic display device 400 that receives the matched image 301 from the image matching expression server 300 and displays it three-dimensionally to the user in response to the physical space of the building in virtual reality (VR) or augmented reality (AR); As a device that provides image evaluation for individual buildings to the user, accesses the image matching expression server 300 to request image evaluation for a specific building, and accordingly provides a matched image 301 for the building and block chain An image evaluation verification device 500 configured to directly check the reliability of the matched image 301 by accessing the architecture information distributed ledger 510 through; may be configured to include.

At this time, the cloud image server 200 and the image matching expression server 300 participate in the block chain and use the image history information, mapping history information, and timestamp as transaction data when each building image is created and changed. It is configured to process the blockchain to be reflected in 510 .

In the present invention, the cloud image server 200 includes: an architectural drawing storage unit 210 for storing design drawing information for individual buildings in the form of file data or image data; an exterior image storage unit 220 for storing an exterior image that is an image of an exterior or interior exterior of the corresponding building; A piping image storage unit 230 for storing identification codes for a plurality of markers shown in the piping image and the piping image taken with the camera unit (C) in the endoscopy inspection of the piping installed in the building; The non-destructive inspection image storage unit 240 for storing the non-destructive inspection images taken in the non-destructive inspection of the building; may be configured to include.

In addition, the image matching expression server 300, a multi-image receiving unit 310 that receives a plurality of architectural drawing images, indoor and outdoor exterior images, piping images, and non-destructive inspection images for a specific building from the cloud image server 200; The CAD mapping number included in the architectural drawing image and the image feature point identified in the exterior image and non-destructive inspection image are mutually mapped to map which point in the architectural drawing image corresponds to each playback point of the exterior image and non-destructive inspection image. a mapping unit 320; A code mapping unit that maps the CAD mapping number included in the architectural drawing image and the marker identification code identified in the piping image or exterior image to which point in the architectural drawing image corresponds to each playback position of the piping image or exterior image (330); Based on the marker identification code and image feature points, the architectural drawing images, indoor and outdoor exterior images, piping images, and non-destructive inspection images of individual buildings are matched and arranged, and these images are combined based on the physical space of the building by synthesizing and rendering them as multi-layer images. a multi-layer matching unit 340 that generates a registered image 301; The multi-layer matching unit 340 may include a registered image display unit 350 for outputting the generated registered image 301 to the stereoscopic display device 400 .

In the present invention, the cloud image server 200 is an architectural drawing image of an individual building, an exterior or indoor exterior image, a piping image of a pipe endoscopy, and a non-destructive inspection image through block chain processing that shares the architectural information distributed ledger 510. It may further include a; blockchain image server unit 250 that processes the image history and mapping history corresponding to the creation and change of one or more of the cloud image server 200 to share in real time. In addition, the image matching expression server 300 is an architectural drawing image of an individual building, an outdoor or indoor exterior image, a piping image of a pipe endoscopy, a non-destructive inspection image through block chain processing that shares the architectural information distributed ledger 510 It may further include; a block chain matching server unit 360 that processes the image history and mapping history corresponding to the creation and change of one or more of the image matching expression server 300 to share in real time.

In addition, the image history of the blockchain transaction data includes one or more of image identification information, shooting meta information, and image integrity information related to the generated or changed image, and the mapping history of the blockchain transaction data includes markers and It consists of at least one of mapping history information, mapping meta-information, and mapping change information related to the mutual mapping state of feature points, and the timestamp of the blockchain transaction data is the creation time or block chain insertion time for each blockchain transaction data. It may be configured to include information indicating

On the other hand, the computer program according to the present invention is stored in the medium to operate the building digital twin block chain system based on the multi-layer image matching expression for the smart city as described above in combination with hardware.

According to the present invention, by matching and displaying architectural drawing images, exterior images, piping images, and non-destructive inspection images in multi-layers for a building, people can see the quality of the building in augmented reality (AR) or virtual reality (VR). It has the advantage of being able to implement a digital twin of a building in a smart city by providing it.

In addition, according to the present invention, there is an advantage in that it is possible to secure trust in the building quality evaluation result by managing various data for realizing the building digital twin of the smart city with the distributed ledger of the block chain.

[FIG. 1] is a diagram showing the overall configuration of a digital twin block chain system for buildings based on multi-layer image matching expression according to the present invention.
2 is a diagram conceptually showing an image database for building image inspection in the present invention.
3 is a view showing the concept of acquiring an image of a pipe inside a building through a pipe endoscopy in the present invention.
[FIG. 4] is a view showing an example of an image of a pipe inside a building obtained through a pipe endoscopy in the present invention.
5 is a diagram conceptually illustrating multi-layer image registration and display in the present invention.
6 is a diagram illustrating the concept of using markers and feature points for multi-layer image registration in the present invention.
[Fig. 7] is a diagram showing an example of performing change record management through a block chain in a building digital twin according to the present invention.
[Fig. 8] is a view showing an example of a result of performing a building quality image inspection in a building digital twin according to the present invention.

Hereinafter, the present invention will be described in detail with reference to the drawings.

[Fig. 1] is a diagram showing the overall configuration of a building digital twin block chain system based on multi-layer image matching expression according to the present invention.

The present invention relates to a method for ensuring the reliability of building evaluation by managing various data for this in a block chain as a technology that enables the evaluation of buildings to be performed in a digital twin method. In general, digital twin refers to a technology that creates an object (twin) identical to the real thing in a virtual space and verifies it through various simulations. The present invention achieves the evaluation of individual buildings as a digital twin, which is suitable for smart cities. In other words, for individual buildings, exterior images, piping images, and non-destructive inspection images are matched based on architectural drawings (CAD), and these are laminated in multi-layers to express them in virtual reality (VR) or augmented reality (AR). It is possible to evaluate the building through images without exploring the real thing.

The building digital twin block chain system according to the present invention for achieving this purpose is an image information generating device 100, a cloud image server 200, an image matching expression server 300, a stereoscopic display device 400, and image evaluation verification. It is operated by the cooperative operation of the device 500 . Among them, the cloud image server 200, the image matching expression server 300, and the image evaluation verification device 500 participate in the block chain, and other devices may or may not be configured to participate in the block chain. . Meanwhile, in FIG. 1 , two or more devices may be combined to form one device, or a separate device may be configured by removing some of the functions of these devices.

In the present invention, the history of creation and change of the building image and its mapping history are managed through the block chain. Information on how these images were created or changed (updated) and how the mutual mapping between them was created and changed is managed by a blockchain, which eliminates the possibility of manipulating multi-layer images to ensure the overall reliability of building evaluation. it is for In consideration of the complexity of blockchain processing, it is desirable to store the image file itself in the cloud image server 200 and manage image-related information with a blockchain.

Referring to [FIG. 1], the block chain transaction information includes an image history and a mapping history related to multiple images of a building. It is preferable that a timestamp is also included in the blockchain, and the timestamp is information indicating when each transaction data was created or inserted into the blockchain.

In the blockchain transaction data, the image history includes image identification information and shooting meta information (time, place, filming company, etc.) related to the image created or changed at that time, and may include image integrity information. The image identification information is information for identifying which image is related to among a plurality of images stored in the cloud image server 200 , and the shooting meta information is information for describing the image itself. The image integrity information is information that guarantees that the corresponding image is original, unmanipulated, and may be implemented as a hash number or checksum.

In blockchain transaction data, the mapping history represents information related to how markers and feature points are mapped to each other between multiple images at a given point in time. The mapping history may include mapping history information, mapping meta information (time, event, etc.), mapping change information (change information of markers and feature points), and the like. When any one of several images is updated as a design change for a building is changed, outdoor or indoor shooting, pipe endoscopy, or non-destructive inspection is performed, the mapping relationship between them is analyzed and the content is reflected in the mapping history.

In general, a block chain has the concept of an open digital transaction ledger in which transaction information generated on a network is replicated and distributed as many as the number of nodes shared among network participants. Blockchain technology is currently being actively applied centering on cryptocurrency, but in the present invention, it is intended to utilize blockchain technology to manage various data forming a digital twin of a building.

In [Fig. 1], the image information generating device 100 is a device for generating various types of images for constituting a digital twin of a building, for example, a drone, a 360-degree camera, a pipe endoscope inspection device, and a building non-destructive inspection device ( For example: thermal imaging camera, X-RAY imaging device), etc. can be implemented. In addition, an apparatus for providing an architectural drawing image by obtaining an architectural drawing (CAD data) from a computer device of a construction CAD company of the building may also be included in the image information generating apparatus 100 .

The cloud image server 200 is a device for storing and managing data (images) for displaying multi-layer images of a building digital twin from the image information generating device 100 as an image database.

The image matching expression server 300 is an architectural drawing image, an exterior image, a piping image (piping endoscopy image), a non-destructive inspection image (eg, a thermal image inspection image, an X-RAY inspection image) for a building from the cloud image server 200 ) and multi-layer matching of these images. An image generated by the image registration display server 300 by multi-layer registration of several images is called a registered image 301 .

The stereoscopic display device 400 is a device that receives the matched image 301 from the image matching expression server 300 and displays the stereoscopic display to the user in the form of, for example, virtual reality (VR) or augmented reality (AR).

The image evaluation verification apparatus 500 is a device operated by a person who intends to perform image evaluation on a building. The image evaluation verification apparatus 500 may access, for example, the image registration expression server 300 to request image evaluation for a specific building, and may receive a matched image 301 for the corresponding building accordingly. The image evaluation verification apparatus 500 operates cooperatively with the stereoscopic display device 400 , and may be implemented in hardware with the stereoscopic display device 400 in consideration of operational efficiency. It is preferable that the image evaluation verification device 500 is configured to directly check the reliability of the matched image 301 by accessing the distributed ledger 510 of the building information through the block chain.

[Fig. 2] is a diagram conceptually showing an image database for building image inspection in the present invention.

In the present invention, the exterior image, the pipe image, and the non-destructive inspection image are multi-layered as a three-dimensional image for individual buildings and rendered and displayed in VR or AR format by matching the architectural drawing image. Data for display is provided from the image information generating apparatus 100 and stored and managed as an image database.

Referring to FIG. 2 , the cloud image server 200 includes an architectural drawing storage unit 210 , an exterior image storage unit 220 , a piping image storage unit 230 , a non-destructive inspection image storage unit 240 , and a block chain. It is configured to include an image server unit (250).

First, the architectural drawing storage unit 210 is a component for storing design drawings (CAD drawings) for individual buildings. The architecture drawing storage unit 210 may be implemented in various ways to store the design drawings, for example, it may be stored as a CAD file or it may be saved as an image of a CAD drawing or a PDF file, and preferably an architectural drawing for rendering efficiency. It can also be converted and stored in the form of an image.

The exterior image storage unit 220 is a component that stores an image of the exterior or interior of a building, that is, an exterior image. The exterior of the building can be photographed with a drone, for example, and the interior can be photographed with a 360-degree camera. Markers (video markers, wireless tag markers) like those described with reference to [Fig. 6] can be installed at special checkpoints on the indoor and outdoor exteriors of buildings, and these markers are installed on the exterior image taken with a drone or 360-degree camera. reflected as it is. Therefore, it is possible to analyze the exterior image or obtain the marker identification information for the checkpoint of the building by the reading function of a drone or a camera, and use it for image matching, as will be described later.

The pipe image storage unit 230 is a component for storing an image captured by a camera in an endoscopy of a pipe installed in a building, that is, a pipe image. The inside of the pipe can be inspected using the pipe endoscope device as in [FIG. 3] to check whether the pipe construction is done properly for the building and whether an abnormality has occurred in the pipe, and the pipe image is a shot image of this process .

At this time, it is preferable that the pipe image storage unit 230 stores not only the image but also various data related thereto. For example, as will be described later with reference to [Fig. 6], all or part of the identification code values of the image markers M1 and M2 and the radio tag marker A installed in the piping pipe 10 are stored in the piping image storage unit. 230 may be stored. It is also possible to configure the pipe endoscope device to analyze the image markers M1 and M2 in real time while photographing the inside of the pipe and insert it as metadata in the pipe image.

In addition, the operator can read the identification code value of the radio tag marker A using a tag reader (not shown) during the pipe endoscopy, and it is also possible to insert these read values as metadata in the pipe image. During the plumbing construction process of a building, a plumber attaches a wireless tag marker (A) to multiple points of the pipe, for example, at a pipe connection point, and includes the information in the building completion data file. Inserting the read identification code value of the radio tag marker (A) may help to determine the shooting point.

In addition, the camera unit (C) of the pipe endoscope device may be provided with an acceleration sensor (eg, a gyro sensor), the acceleration data generated by the camera unit (C) during pipe endoscopy is performed as metadata in the pipe image. Insertion is also possible. The acceleration data is helpful in determining the position and operation (eg, refraction) of the camera unit C when reproducing and analyzing the pipe image.

The non-destructive inspection image storage unit 240 is a component for storing images taken while performing various non-destructive inspections on a building. For example, the non-destructive inspection image storage unit 240 may store a thermal image inspection image or an X-RAY inspection image performed on the corresponding building.

The block chain image server unit 250 is a component that shares the image history and mapping history of the building with the cloud image server 200 in real time through block chain processing that shares the building information distributed ledger 510 .

[Fig. 3] is a view showing the concept of acquiring an image of a pipe inside a building through a pipe endoscopy in the present invention, and [Fig. 4] is an example of an image of a pipe inside a building obtained through a pipe endoscopy in the present invention It is a drawing showing

In general, a pipe is composed of a multi-refraction pipeline inside a building, and the pipe endoscopy is performed using an endoscope device as shown in [FIG. 3], and a pipe image is obtained through this. The camera unit (C) is put into the inside of the piping pipe (10) to take an image inside the pipe, and provides the captured image to the DVR equipment outside the pipe through an image signal cable. Here, it is preferable to provide a separate LED lighting unit (not shown) around the camera unit (C) because the environment in which the endoscope device operates is dark inside the multi-refractive pipeline.

The video signal cable is connected to the rear end of the camera unit (C) to supply operating power to the camera unit (C) and the LED lighting unit, while the camera unit (C) is connected to the camera unit (C) and the DVR equipment to each other. The captured video inside the pipe is transmitted to the DVR equipment, and forward, backward, and rotation control signals are transmitted from the outside to the camera unit (C) from the inside of the pipe. The stainless spring constitutes a kind of push-rod, and when the camera unit (C) advances forward, the stainless spring performs a function of pushing it in the longitudinal direction, and when it is pulled back, the inner deflection of the multi-refractive pipeline It prevents damage by lowering the frictional force of the part that is in contact with the part that has been made.

As such, the endoscope device smoothly moves the multi-refractive pipeline by the guide action of the stainless spring, and in the process, the camera unit (C) captures the inside of the pipe and transmits the pipe image to the DVR device. At the bottom of [Fig. 3], several examples of such a piping image are presented. The DVR device uploads the piping image to the cloud image server 200 through the network, and the piping image is provided for inquiry and analysis through a smartphone or computer.

[Fig. 4] shows an example of an image of a pipe inside a building obtained through such a pipe endoscopy in the present invention. Referring to [FIG. 4], for the piping image, for a specific building (eg, a specific apartment located in Yongin-si, Gyeonggi-do), not only matters related to the entire building but also related to a specific lake (eg, No. 2001) are arranged. In addition, the piping images for various piping related matters (eg, upright pipe, sewage pipe, bit inside, horizontal transverse pipe) are organized by item. In addition, in the piping image, not only the overall image but also the image of the part where unusual matters (eg, breakage of outdoor outlet pipe, blockage of underground pipe, foreign matter in the negotiating pipe, backflow of household) occurs are specified. In addition, the marker number and the CAD mapping number for the point of occurrence of the singularity are identified. The marker number and the CAD mapping number will be described later with reference to [Fig. 5] and [Fig. 6].

[Fig. 5] is a diagram conceptually illustrating multi-layer image registration and display in the present invention, and [Fig. 6] is a diagram illustrating the concept of using markers and feature points for multi-layer image registration in the present invention.

5, the image matching expression server 300 is an architectural drawing image, indoor/outdoor exterior image, piping image (pipe endoscopy image), non-destructive inspection image (eg: A thermal image inspection image, an X-RAY inspection image) are provided, and the images are multi-layered to generate a registered image 301 . Through this, images corresponding to the physically same point of the building in multiple images are matched and overlapped and synthesized. The matched image 301 is rendered and displayed in three dimensions in response to the physical space of the building, for example, navigation. For this, virtual reality (VR) or augmented reality (AR) is preferable.

In the present invention, markers and feature points may be adopted as reference points (matching points) for matching multiple images. Markers and feature points can be adopted in various ways, and as the markers, image markers (eg barcodes, QR codes) and wireless tag markers (eg NFC tags, RFID tags) are possible, A standardized pillar or corner of an apartment can be used as a characteristic point. Markers and feature points will be described with reference to FIG. 6 .

With reference to [Fig. 6], the markers and feature points used for multi-layer image registration in the present invention will be described. (a) of [Fig. 6] shows the image marker and NFC tag installed on the piping pipe, (b) of [Fig. 6] is an example of setting the CAD mapping number on the piping pipe design drawing, [Fig. ] (c) is a diagram conceptually showing that image feature points can be derived through image analysis.

A plurality of piping pipes 10 are interconnected in a building to be in charge of piping construction. Referring to (a) of [Fig. 6], in the present invention, a marker is installed to distinguish and identify each point of the piping pipe 10. do. The marker may be composed of an image marker (M1, M2) and a radio tag marker (A). The image markers M1 and M2 are markers for identification through image processing, for example, barcodes or QR codes. These image markers M1 and M2 may be printed during the manufacturing process of the piping pipe 10 or may be attached as stickers during the piping construction process. The radio tag marker (A) is a marker for identification by short-range wireless communication technology, and may include, for example, an NFC tag or an RFID tag.

Referring to (b) of [Fig. 6], CAD mapping numbers (eg, B11-1 to E11-2) are assigned to design drawing data (CAD data) of a building. These CAD mapping numbers are assigned to distinguish and identify each point on the design drawing at the time of designing the building. The markers M1, M2, and A mounted on each inner side or connection point of the piping pipe 10 in the process of piping construction are matched and stored in the CAD mapping number by the constructor. Since unique identification codes are assigned to the markers M1, M2, and A, it is possible to identify where each point in each image is located on the design drawing by mapping the identification code to the design drawing. In the pipe endoscopy, by image processing the pipe image taken through the camera unit (C) to identify the image markers (M1, M2), it is also possible to identify which part on the design drawing the current playback point of the pipe image corresponds to.

In (c) of FIG. 6, an external image and a non-destructive inspection image (thermal image, X-RAY inspection image) are shown in relation to the pipe part. The photographer of these exterior images and non-destructive inspection images stores where the shooting point is in the building, and this information can be stored matching the CAD mapping number of the design drawing. In addition, by analyzing the exterior image and the non-destructive inspection image, it is possible to recognize a feature part, for example, a standardized column or corner part of an apartment, compare it with a design drawing, and match it with a CAD mapping number and store it. Each area of the apartment is in charge of its own function, and since those characteristics are reflected in the design and construction, the characteristic parts can be recognized and matched with the design drawings. In this specification, such a feature part is called an image feature point.

5, the image registration display server 300 includes a multi-image receiving unit 310, a feature point mapping unit 320, a code mapping unit 330, a multi-layer matching unit 340, a registered image display unit ( 350), and a block chain matching server unit 360 is configured.

The multi-image receiver 310 is configured to receive an architectural drawing image, an indoor/outdoor exterior image, a piping image, and a non-destructive inspection image (eg, thermal image inspection image, X-RAY inspection image) for a specific building from the cloud image server 200 . is an element

The feature point mapping unit 320 is a component that maps CAD mapping numbers of architectural drawing images and image feature points of exterior images and non-destructive inspection images (eg, standardized pillars of an apartment, corner recognition, etc.). The feature point mapping unit 320 analyzes the exterior image and the non-destructive inspection image to recognize a feature part, for example, a standardized column or corner part of an apartment, and match it with a CAD mapping number in comparison with the design drawing. Since each area of the building is responsible for its own function and its characteristics are reflected in the design and construction, there is a point where the characteristic parts can be recognized and matched with the design drawings.

As a result, with respect to each reproduction point of the exterior image and non-destructive inspection image (eg, thermal image inspection image, X-RAY inspection image), it is possible to map which point in the architectural drawing (architecture image) it relates to.

The code mapping unit 330 is a component that maps the CAD mapping number of the architectural drawing image and the marker identification code of the piping image to each other. In the piping construction process, information about which piping pipe 10 is installed at which point of the building is input by the constructor and included in the building completion data file, and the code mapping unit 330 is an image marker ( Based on the identification codes of M1 and M2), the current playback position of the piping image is mapped where it corresponds in the architectural drawing image of the building. Similarly, the code mapping unit 330 matches the identification code of the radio tag marker (A) inserted in the piping image with the building completion data file to map where the current playback position of the piping image corresponds to the architectural drawing image of the building. .

At this time, the code mapping unit 330 is the specification (eg, length, bending position) of the piping pipe 10 stored in the architectural drawing storage unit 210 and the positions of the markers M1, M2, A, and between these markers. From the time taken to move, the moving speed between the markers of the camera unit C can be calculated. And, it may be configured to determine the position of the piping pipe 10 with respect to the current regeneration point based on the moving speed. In this case, it may be assumed that the camera unit C moves at a constant velocity between the marker and the marker.

On the other hand, the code mapping unit 330 may determine the position of the piping pipe 10 corresponding to the reproduction point of the piping image based on the acceleration data generated by the acceleration sensor of the camera unit (C) and inserted into the piping image. have. This is because, when moving between markers, if acceleration data is integrated on the time axis, the velocity is calculated, and if integrated again on the time axis, the distance moved between the markers can be known.

As a result, with respect to each reproduction point of the pipe image, it is possible to relatively accurately map which point the camera unit C of the pipe endoscope device captures in the architectural design drawing (architecture drawing image).

Meanwhile, the code mapping unit 330 may map the marker identification information obtained from the exterior image to the CAD mapping number of the architectural drawing image. The configuration of installing markers at checkpoints for each checkpoint on the indoor and outdoor exterior of a building was previously described. In this case, the code mapping unit 330 maps the marker identification information obtained from the exterior image to the CAD mapping number of the architectural drawing image. Can be used.

In addition, the code mapping unit 330 may map the inspection point identification information of the non-destructive inspection image to the CAD mapping number of the architectural drawing image. Previously, the feature point mapping unit 320 described a configuration in which the image feature point of the non-destructive inspection image is mapped to the CAD mapping number of the architectural drawing image, and the code mapping unit 330 converts the inspection point identification information into the CAD mapping number of the architectural drawing image. The result of mapping to can also be used as a complement.

The multi-layer matching unit 340 includes an architectural drawing image, an indoor/outdoor exterior image, a piping image, and a non-destructive inspection image (eg, thermal image inspection image, X- It is a component that registers and aligns RAY inspection images) and synthesizes them into a multi-layer image.

Architectural drawing images, indoor/outdoor exterior images, piping images, and non-destructive inspection images (eg thermal imaging inspection images, X-RAY inspection images) are all images created for the same building (eg, Hanmaeum Apartment No. 2001 in Yongin-si, Gyeonggi-do). to be. Accordingly, if one point is unreasonably set for the building, there are parts of the architectural drawing image, the indoor/outdoor exterior image, the piping image, and the non-destructive inspection image corresponding thereto. In the present specification, matching or matching means matching these images based on a physical space, which is achieved through markers and feature points.

In addition, in the present invention, configuring the registered image 301 as a multi-layer means that these plurality of images are combined while maintaining their respective identities. In the present invention, the matched image 301 is displayed through a stereoscopic display device 400 such as virtual reality (VR) or augmented reality (AR), which means that the images are displayed together while maintaining their originality.

To this end, it is preferable that the multimedia matching unit 340 maps the exterior image, the piping image, and the non-destructive inspection image to the architectural drawing image (ie, the CAD image) based on the markers and the feature points to perform multi-layer rendering. This mapping information is generated by the feature point mapping unit 320 and the code mapping unit 330, and the multimedia matching unit 340 uses this mapping information to align and align an architectural drawing image, an indoor/outdoor exterior image, a piping image, and a non-destructive inspection image. and synthesized into a multi-layer image.

The registered image display unit 350 displays the registered image 301 generated by the multi-layer matching unit 340 through the external stereoscopic display device 400 . At this time, the matched image display unit 350 is an augmented reality (AR) or virtual reality (VR) method in a state in which the design drawing, the exterior image, the pipe image, the thermal image inspection image, and the X-RAY inspection image are accurately mapped. It is preferably configured to provide

The block chain matching server unit 360 is a component that shares the image history and mapping history of the building with the image matching expression server 300 in real time through block chain processing that shares the building information distributed ledger 510 .

[Fig. 7] is a diagram showing an example of performing change record management through a block chain in a building digital twin according to the present invention.

Referring to [Fig. 7], the concept of building a digital twin for a new house (eg, apartment 101, No. 102) and managing it in a block chain to ensure the reliability of these information is shown. In this case, the content of multi-layer stacked records of image mapping and the display of matched images can be managed by the block chain.

In the present invention, the location of the corresponding shooting point, the pipe, the steel frame, and the energy state are matched and stored based on the marker and the feature point at the time of shooting the building multi-layer image (eg, exterior image, pipe image, non-destructive inspection image). Preferably, whenever each building image is generated, image history information, mapping history information, and timestamps are put into the block chain to be reflected in the building information distributed ledger 510 .

For each building, the images taken by the construction company or supervisory company at each construction stage and the images taken during various inspection processes constitute a digital twin for the building, and based on this, image evaluation of the building can be performed. have. In the present invention, it is impossible to remove or replace individual images by inputting and managing the history and mutual mapping information for these images in the block chain, thereby ensuring the reliability of the digital twin.

[Fig. 8] is a diagram showing an example of a result of a building quality image inspection in a building digital twin according to the present invention.

Referring to [Fig. 8], the image-based results of the building quality inspection for 101-dong 2001 and 2002 of a specific apartment are shown. First, looking at the inspection results for No. 2001, exterior images, piping images, thermal imaging camera images, and X-ray images are stored as inspection results. By creating and analyzing the multi-layer stacked images for these images based on the mapping data, the construction and management quality for the 2001 can be specifically determined, and the manager or the 2001 resident can use the three-dimensional display device 400 to You can check the layer matching image directly. It is guaranteed by the blockchain that these quality inspection results are reliable.

Meanwhile, the present invention can be implemented in the form of computer-readable codes on a computer-readable non-volatile recording medium. Various types of storage devices exist as such non-volatile recording media. For example, hard disks, SSDs, CD-ROMs, NAS, magnetic tapes, web disks, cloud disks, etc. A form in which it can be implemented and executed can also be implemented. In addition, the present invention may be implemented in the form of a computer program stored in a medium to execute a specific procedure in combination with hardware.

100: image information generating device
200: cloud video server
210: architectural drawing storage unit 220: exterior image storage unit
230: piping image storage unit 240: non-destructive inspection image storage unit
250: block chain video server unit
300: image matching expression server
301: registered image 310: multi-image receiver
320: feature point mapping unit 330: code mapping unit
340: multi-layer matching unit 350: registered image display unit
360: block chain matching server unit
400: stereoscopic display device
500: image evaluation verification device
510: distributed ledger of architectural information

Claims (6)

an image information generating apparatus 100 for generating a plurality of architectural drawing images, outdoor or indoor exterior images, piping images of a piping endoscopy, and non-destructive inspection images to construct a digital twin for a building;
a cloud image server 200 for receiving a plurality of architectural drawing images, exterior images, plumbing images, and non-destructive inspection images for constructing a digital twin of a building from the image information generating device 100 and managing the database;
A plurality of architectural drawing images, exterior images, piping images, and non-destructive inspection images for a specific building are provided from the cloud image server 200, and these images are multi-layered with a CAD mapping number based on markers and feature points to match the corresponding building Image registration display server 300 for generating a matched image 301 for;
A stereoscopic display device 400 that receives the matched image 301 from the image matching expression server 300 and displays it three-dimensionally to the user in response to the physical space of the building in virtual reality (VR) or augmented reality (AR);
consists of,
The cloud image server 200 and the image matching expression server 300 participate in the block chain and use the image history information, mapping history information, and timestamp as transaction data when each building image is created and changed. 510) is configured to process the blockchain to be reflected,
The cloud image server 200,
an architectural drawing storage unit 210 for storing design drawing information for individual buildings in the form of file data or image data;
an exterior image storage unit 220 for storing an exterior image that is an image of an exterior or interior exterior of the corresponding building;
A piping image storage unit 230 for storing identification codes for a plurality of markers indicated in the piping image and the piping image taken by the camera unit (C) in the endoscopy of the piping installed in the building;
Non-destructive inspection image storage unit 240 for storing the non-destructive inspection image taken in the non-destructive inspection of the building;
A building digital twin block chain system based on multi-layer image matching expression for a smart city, characterized in that it comprises a.
delete The method according to claim 1,
The image matching expression server 300,
a multi-image receiving unit 310 that receives a plurality of architectural drawing images, indoor/outdoor exterior images, piping images, and non-destructive inspection images for a specific building from the cloud image server 200;
The CAD mapping number included in the architectural drawing image and the image feature points identified in the exterior image and non-destructive inspection image are mapped to which point in the architectural drawing image corresponds to each reproduction point of the exterior image and non-destructive inspection image a feature point mapping unit 320 that maps
By mapping the CAD mapping number included in the architectural drawing image and the marker identification code identified in the piping image or the exterior image to each other, each reproduction position of the piping image or the exterior image corresponds to which point in the architectural drawing image a code mapping unit 330 for mapping
Based on the marker identification code and image feature points, the architectural drawing images, indoor and outdoor exterior images, piping images, and non-destructive inspection images of individual buildings are aligned and synthesized and rendered as a multi-layer image. a multi-layer matching unit 340 that generates a matched matched image 301;
a registered image display unit 350 for outputting the registered image 301 generated by the multi-layer matching unit 340 to the stereoscopic display device 400;
A building digital twin block chain system based on multi-layer image matching expression for a smart city, characterized in that it comprises a.
4. The method of claim 3,
The cloud image server 200,
Through blockchain processing that shares the architectural information distributed ledger 510, it responds to the creation and change of one or more of architectural drawing images of individual buildings, exterior or indoor exterior images, piping images of pipe endoscopy, and non-destructive inspection images. a block chain image server unit 250 for processing image history and mapping history to be shared by the cloud image server 200 in real time;
Consists of further comprising,
The image matching expression server 300,
Through blockchain processing that shares the architectural information distributed ledger 510, it responds to the creation and change of one or more of architectural drawing images of individual buildings, exterior or indoor exterior images, piping images of pipe endoscopy, and non-destructive inspection images. a block chain matching server unit 360 that processes the image history and mapping history so that the image matching expression server 300 shares in real time;
A building digital twin block chain system based on multi-layer image matching expression for a smart city, characterized in that it further comprises a.
5. The method according to claim 4,
The image history of the blockchain transaction data is configured to include one or more of image identification information, shooting meta information, and image integrity information related to the generated or changed image,
The mapping history of the block chain transaction data is composed of at least one of mapping history information, mapping meta information, and mapping change information related to the mutual mapping state of markers and feature points at the time point,
The timestamp of the block chain transaction data is a multi-layer image matching expression-based building digital twin for a smart city, characterized in that it includes information indicating the creation time or block chain insertion time for each block chain transaction data blockchain system.
6. The method of claim 5,
As a device for providing image evaluation for individual buildings to the user, accessing the image matching expression server 300 to request image evaluation for a specific building and receiving a matched image 301 for the corresponding building according to the block an image evaluation verification device 500 configured to directly check the reliability of the matched image 301 by accessing the building information distributed ledger 510 through a chain;
A building digital twin block chain system based on multi-layer image matching expression for a smart city, characterized in that it further comprises a.

Images