CN114331194A - Building measurement management method and system based on digital twins - Google Patents
Building measurement management method and system based on digital twins Download PDFInfo
- Publication number
- CN114331194A CN114331194A CN202111681378.9A CN202111681378A CN114331194A CN 114331194 A CN114331194 A CN 114331194A CN 202111681378 A CN202111681378 A CN 202111681378A CN 114331194 A CN114331194 A CN 114331194A
- Authority
- CN
- China
- Prior art keywords
- action
- degree
- degree live
- live
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a building measurement management method and a system based on digital twins, wherein the method comprises the following steps: acquiring three-dimensional 360-degree live-action image data; the 360-degree live-action image data comprises 360-degree live-action video and 360-degree live-action photos; the 360-degree live-action pictures are shot by workers carrying 360-degree cameras towards fixed directions at various focus points of a construction site, and the 360-degree live-action videos are shot by the workers carrying the 360-degree cameras when the workers walk between the focus points of the construction site; aligning the 360-degree live-action picture with a preset three-dimensional building model; establishing a well-aligned linkage relation between the 360-degree live-action photo and the three-dimensional building model; and comparing the 360-degree live-action photo with the three-dimensional building model at the same visual angle. Can be in high efficiency with each 360 degrees outdoor scene photos with preset three-dimensional building model alignment to can also guarantee that the 360 degrees outdoor scene photos that the position corresponds is paid close attention to the customer can satisfy customer's definition requirement.
Description
Technical Field
The invention relates to the field of building engineering supervision, in particular to a building measurement management method and system based on digital twins.
Background
According to the definition of wikipedia, building measurement refers to building construction supervision, survey, maintenance management, safety inspection, fire prevention inspection, insurance assessment, facility management, space planning and the like. Namely, monitoring and managing buildings from different dimensions. The current popular practice in the technology is as follows: monitoring personnel go to the scene to monitor, find the matters that need to record, then shoot and fill in relevant form and record through mobile device end App. The manager of the office checks the recorded data through the PC end web page and arranges follow-up. And follow-up personnel follow-up according to the data collected on site.
In order to realize remote building measurement, a chinese patent (application publication No. CN112581618A) previously applied by the present applicant discloses a method and system for comparing a three-dimensional building model with a real scene in the building engineering industry. However, when the applicant applies the above method to practice, the applicant finds that the above method still has the following points which need further improvement:
1. when the visual angle which is the same as the initial position in the 360-degree live-action picture is positioned in the preset three-dimensional building model, 6 object positions which are easy to mark need to be found at random in the visual angle of the initial position in the 360-degree live-action picture manually, and the two-dimensional coordinates of a screen are recorded by clicking the object positions; meanwhile, manually finding a visual angle capable of seeing the 6 object positions in the three-dimensional building model, and clicking the object positions to record three-dimensional coordinates; and subsequently, calculating the position and the posture of a camera which sees the same visual angle as the 360-degree live-action picture in the three-dimensional building model according to the 6 two-dimensional coordinates and the 6 three-dimensional coordinates. Obviously, the method requires that coordinate points are found in the 360-degree live-action photos manually, and the two-dimensional coordinates and the three-dimensional coordinates are selected manually and intuitively, so that the precision is low; if the collected real-scene photos are 360 degrees, the operations of respectively selecting the two-dimensional coordinates and the three-dimensional coordinates need to be manually repeated for each photo, so that the precision is low, the time consumption is high, and the efficiency is low.
2. In the method, if a 360-degree live-action video is collected, although the operation of manually selecting the two-dimensional coordinates and the three-dimensional coordinates is only required to be performed on the first key frame, the alignment of the first key frame is realized by manually selecting the two-dimensional coordinates and the three-dimensional coordinates respectively, and the problem of low two-dimensional and three-dimensional alignment precision also exists; furthermore, although the alignment relationship of the subsequent key frames can be deduced through the operation with the first key frame, the efficiency is greatly improved, as the video is shot in the moving process, the fuzzy condition is inevitable, and the definition of the video frame corresponding to the concerned position fed back by the client often cannot meet the definition requirement.
In view of the above-mentioned disadvantages, the applicant has filed the present application based on a prior application.
Disclosure of Invention
The invention aims to solve at least one of the technical problems in the prior art, and provides a building measurement management method and system based on digital twin, which can align each 360-degree live-action photo with a preset three-dimensional building model efficiently, and can ensure that the 360-degree live-action photo corresponding to the attention position of a client can meet the definition requirement of the client.
In a first aspect, the present invention provides a digital twin-based building survey management method, the method comprising:
acquiring three-dimensional 360-degree live-action image data; the 360-degree live-action image data comprises 360-degree live-action video and 360-degree live-action photos; the 360-degree live-action pictures are shot by workers carrying 360-degree cameras towards fixed directions at various focus points of a construction site, and the 360-degree live-action videos are shot by the workers carrying the 360-degree cameras when the workers walk between the focus points of the construction site;
aligning the 360-degree live-action picture with a preset three-dimensional building model;
establishing a well-aligned linkage relation between the 360-degree live-action photo and the three-dimensional building model, so that the 360-degree live-action image data and the visual angle of the three-dimensional building model are kept consistent;
and comparing the 360-degree live-action photo with the three-dimensional building model at the same view angle so as to carry out building measurement management.
In a first aspect, the present invention provides a digital twin based construction survey management system, the system comprising:
the acquisition module is used for acquiring three-dimensional 360-degree live-action image data; the 360-degree live-action image data comprises 360-degree live-action video and 360-degree live-action photos; the 360-degree live-action pictures are shot by workers carrying 360-degree cameras towards fixed directions at various focus points of a construction site, and the 360-degree live-action videos are shot by the workers carrying the 360-degree cameras when the workers walk between the focus points of the construction site;
the alignment module is used for aligning the 360-degree live-action photo with a preset three-dimensional building model;
the linkage module is used for establishing a linkage relation between the aligned 360-degree live-action photo and the three-dimensional building model, so that the view angle of the 360-degree live-action image data is consistent with that of the three-dimensional building model;
and the comparison module is used for comparing the 360-degree live-action photos with the three-dimensional building model at the same visual angle so as to carry out building measurement management.
Compared with the prior art that the definition cannot meet the requirement due to the fact that key frames are extracted from 360-degree live-action videos and used as photos corresponding to the customer points of interest, the building measurement management method and system based on the digital twin, provided by the invention, are used for taking pictures at the customer points of interest to obtain the 360-degree live-action photos capable of meeting the definition requirement, and when workers wear safety helmets to walk between the points of interest on a construction site, the 360-degree live-action photos are switched to the 360-degree live-action videos, so that sufficient identical feature points can be provided for the subsequent use of an ORB (ordered FAST and rotad BRIEF) feature point detection algorithm to calculate the positions of cameras between two adjacent 360-degree live-action photos in a world coordinate system, and the accuracy of the calculated camera positions is improved.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The invention is further described below with reference to the accompanying drawings and examples;
FIG. 1 is a block diagram of a digital twin based construction survey management system in one embodiment.
Fig. 2 is a flow chart of a digital twin-based building survey management method in one embodiment.
FIG. 3 is a flow diagram of a digital twin-based building survey management method in one embodiment.
FIG. 4 is a flow diagram of a digital twin-based building survey management method in one embodiment.
FIG. 5 is a flow diagram of a digital twin-based building survey management method in one embodiment.
FIG. 6 is a block diagram of a computer device in one embodiment.
Fig. 7 is a diagram illustrating the effect of split screen display in a digital twin-based building surveying management method according to an embodiment.
Fig. 8 is a diagram illustrating the effect of a mixed display in a digital twin-based building surveying management method according to an embodiment.
Fig. 9 is a diagram illustrating the effects of the overlay display in a digital twin-based building survey management method according to an embodiment.
Reference numerals:
110. a terminal; 111. a photo taking switching module; 112. a video shooting switching module; 120. a server; 121. an acquisition module; 122. an alignment module; 123. a linkage module; 124. and a comparison module.
Detailed Description
Reference will now be made in detail to the present preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.
FIG. 1 is a block diagram of a digital twin based construction survey management system in one embodiment. Referring to fig. 1, the digital twin-based building surveying management method is applied to a digital twin-based building surveying management system. The digital twin-based building measurement management system includes a terminal 110 and a server 120 connected through a network. The terminal 110 may specifically be a 360-degree camera or other electronic device integrated with the 360-degree camera, for example, the terminal 110 may specifically be a mobile terminal 110, and the mobile terminal 110 may specifically be a mobile phone integrated with the 360-degree camera, a tablet computer, and the like. The server 120 may be implemented as a stand-alone server 120 or as a server cluster of multiple servers 120.
It should be noted that the present invention is based on the chinese patent (application publication No. CN112581618A) previously applied by the applicant, and the patent name is an improved scheme performed by a method and a system for comparing a three-dimensional building model with a real scene in the building engineering industry.
Example 1:
in one embodiment, as shown in fig. 2, a method for digital twin-based building survey management, the method comprises:
step S202: the server 120 acquires three-dimensional 360-degree live-action image data; the 360-degree live-action image data comprises 360-degree live-action video and 360-degree live-action photos; the 360-degree live-action pictures are shot towards a fixed direction at each point of interest of a construction site by a worker carrying a 360-degree camera, and the 360-degree live-action videos are shot when the worker carrying the 360-degree camera walks between the points of interest of the construction site.
It is understood that, after the 360-degree camera captures the 360-degree real image data, the 360-degree real image data is transmitted to the server 120 through the network, the server 120 stores the 360-degree real image data in the local storage, and the server 120 may acquire the 360-degree real image data from the local storage to perform the method of the embodiment when necessary.
Step S204: the server 120 aligns the 360 degree live view with the pre-set three-dimensional architectural model.
As one example, the three-dimensional building model may employ a BIM model.
As shown in fig. 3, the aligning the 360-degree live-action photograph with the preset three-dimensional building model includes:
step S2041: the server 120 calculates world coordinates corresponding to the cameras when shooting other 360-degree live-action pictures according to the 360-degree live-action image data with the world coordinates of the cameras when shooting the first 360-degree live-action picture as an origin.
As shown in fig. 4, specifically, the calculating the world coordinates corresponding to the cameras when taking other 360-degree live-action pictures according to the 360-degree live-action image data with the world coordinates of the camera when taking the first 360-degree live-action picture as the origin includes:
step A: the server 120 divides a plurality of video frame photos obtained by splitting a 360-degree live-action video captured between a focus point corresponding to a camera and an adjacent focus point when the first 360-degree live-action photo is captured, and takes the 360-degree live-action photos captured at the two focus points as a group of photo data sets.
For example, if the camera is at point E when the first 360 degree live-action photograph is taken, the shooting time is 14:25: 42; when a second 360-degree live-action picture is shot, the camera is at the point F, and the shooting time is 14:25: 53; the corresponding time of the 360-degree live-action video shot by the worker from the point E to the point F is 14:25:43 to 14:25:52, and the video length is 10 seconds; the frame rate of the 360-degree live-action video is 30 frames per second, and 1 second is equal to 1000 milliseconds, so that a video frame photo is formed every about 33 milliseconds in the process of shooting the 360-degree live-action video, and the video frame photo can be split into 300 video frame photos in total, and at this time, a group of photo data sets can be 302 photos in total.
And B: the server 120 sequentially calculates the change of the camera position corresponding to each photo according to the shooting time sequence of each photo in the group of photo data sets until the world coordinates corresponding to the attention point camera adjacent to the first 360-degree live-action photo are obtained.
And C: the server 120 sequentially calculates world coordinates corresponding to the cameras when taking other 360-degree live-action pictures according to the above steps a to B.
It can be understood that, after calibrating the 360-degree camera, the mapping relationship between the pictures taken by the 360-degree camera and the world coordinate system can be obtained, and when two pictures are taken by the same 360-degree camera at different positions and different angles for the same object in the world coordinate system, an orb (organized FAST and Rotated brief) feature point detection algorithm can be used to extract the feature points of each frame of the 360-degree live-action video, and then the displacement change of the same feature points in each frame of the picture is tracked, so as to reversely deduce the displacement and angle change of the world coordinate system between the camera positions during two times of taking pictures, thereby determining the position and posture of the camera in the world coordinate system in each frame of the picture. How to determine the position change of the camera in the actual coordinate system according to the photo may specifically be implemented by calling a cv2. findEsentialMat () function and a cv2.recoverPose () function provided in OpenCV, which is not described herein again.
Because the orb (organized FAST and Rotated live) feature point detection algorithm is used to extract the feature points of each frame of picture in the 360-degree live-action video, and then the displacement change of the same feature points in each frame of picture is tracked, so as to calculate the position and the posture of the camera in the world coordinate system in each frame of picture, it is required that there are enough same feature points between two adjacent frames of pictures, if there are too few same feature points in two adjacent frames, the accuracy of the algorithm is not high, and even the calculation fails. In the invention, as the worker can shoot the 360-degree live-action video between two adjacent points of interest when walking between the two adjacent points of interest, a plurality of adjacent frames with enough same characteristic points are obtained, and the positions of the two adjacent points of interest can be calculated by the algorithm no matter how far the distance between the two adjacent points of interest is. Meanwhile, the distance between adjacent attention points can be not limited, so that the path for photographing by workers is not required to be planned in advance, attention points can be added according to the temporary requirements of customers in the process of photographing the photos, and the 360-degree live-action photos corresponding to other photographed or non-photographed attention points are not influenced. Meanwhile, the method is convenient for the workers to select the most appropriate route for taking the picture according to the specific construction condition on the site.
Step S2042: the server 120 determines a first mapping relationship between the world coordinates and the three-dimensional coordinates of the three-dimensional building model according to at least three groups of preset corresponding coordinate point pairs.
It will be appreciated that points at three door openings, distributed O, P and Q points, may be selected in the three-dimensional building model, and the three-dimensional coordinates of these three points may be derived from the three-dimensional building model. Regardless of the construction progress of the site, when the worker arrives at the construction site, the three door opening points O ', P ' and Q ' can be easily and accurately found and taken as the points of interest (i.e., the 360-degree live-action pictures are taken at the three door opening points by using a 360-degree camera). The world coordinates corresponding to the three door opening points O ', P' and Q 'can be determined through the above step S2041, so that three groups of preset corresponding coordinate points of the point O' and the point O, the point P 'and the point P, and the point Q' and the point Q are used for determining a first mapping relationship between the world coordinates and the three-dimensional coordinates of the three-dimensional building model, and a method for establishing a mapping relationship between two coordinate systems belongs to the prior art and is not repeated here. It should be noted that after the first mapping relationship between the world coordinate and the three-dimensional coordinate of the three-dimensional building model is established, it can be ensured that the relative positions of all other points of interest in the three-dimensional building model are accurate, and it is not necessary to artificially select 6 two-dimensional coordinates and 6 three-dimensional coordinates to calculate the position and posture of the camera viewing the same view angle as the 360-degree live-action picture in the three-dimensional building model, respectively, for each picture, thereby improving the data processing efficiency.
Step S2043: the server 120 determines, according to the fixed direction and the first mapping relationship, a position and a posture of a camera that is seen in the three-dimensional model from the same viewing angle as each of the 360-degree live-action photos, so as to align each of the 360-degree live-action photos with a preset three-dimensional building model.
In this embodiment, each 360-degree live-action picture is taken by the worker in a fixed direction, for example, the 360-degree live-action picture is taken by the worker wearing a safety helmet in a fixed direction at each focus point on the site, that is, the posture of each 360-degree live-action picture is fixed, so that the viewing angle of the three-dimensional building model can be set in advance according to the fixed direction, and once the viewing angle of the three-dimensional building model is adjusted, subsequent adjustment is not needed, and the data processing speed is further increased.
Compared with the prior art, the method has the advantages that the problems of large workload and low alignment precision caused by the fact that the position and the posture of a camera which sees the same visual angle as a 360-degree live-action photo in a three-dimensional building model need to be deduced by manually and intuitively selecting 6 two-dimensional coordinates and 6 three-dimensional coordinates respectively. The invention appoints at least three groups of mark points (such as door opening, upright post and the like) which can be accurately positioned in a construction site and a three-dimensional building model, establishes a first mapping relation between world coordinates and three-dimensional coordinates of the three-dimensional building model by utilizing the three groups of mark points which can be accurately positioned, and can ensure that the positions of points of interest appointed by customers in the three-dimensional building model and the position relation between the points of interest can be accurately reflected in the three-dimensional building model as long as the three groups of mark points are accurately positioned.
Step S206: the server 120 establishes a linkage relationship between the aligned 360-degree live-action photograph and the three-dimensional building model, so that the 360-degree live-action image data and the perspective of the three-dimensional building model are kept consistent.
It is understood that the specific implementation of this step can refer to the chinese patent (application publication No. CN112581618A) previously applied by the applicant, and is not described herein again.
Step S208: under the same view angle, the server 120 compares the 360-degree live-action photograph with the three-dimensional building model so as to perform building measurement management.
Specifically, for the convenience of user analysis, the method provides three view modes: split screen, mix, overlap. Referring to fig. 7, split screen is to divide the screen into left and right sides, each side being the complete scene. The whole looks like two identical scenes. Referring to fig. 8, blending divides the screen into left and right sides, one side is a three-dimensional building model, and the other side is a 360-degree live-action video, and the whole looks like a scene. Referring to fig. 9, the overlay, i.e. overlaying the 360 degree live-action video and the three-dimensional building model together, can change the overlay effect by changing the transparency, only see the 360 degree live-action video when the transparency is 0, and only see the three-dimensional building model when the transparency is 100. If the transparency is 50, the effect of overlapping the 360-degree live-action video and the three-dimensional building model can be seen.
Compared with the prior art that the definition cannot meet the requirement due to the fact that key frames are extracted from 360-degree live-action videos and used as photos corresponding to the customer points of interest, the building measurement management method and system based on the digital twin, provided by the invention, are used for taking pictures at the customer points of interest to obtain the 360-degree live-action photos capable of meeting the definition requirement, and when workers wear safety helmets to walk between the points of interest on a construction site, the 360-degree live-action photos are switched to the 360-degree live-action videos, so that sufficient identical feature points can be provided for the subsequent use of an ORB (ordered FAST and rotad BRIEF) feature point detection algorithm to calculate the positions of cameras between two adjacent 360-degree live-action photos in a world coordinate system, and the accuracy of the calculated camera positions is improved.
As shown in fig. 5, in one embodiment, before the acquiring 360 degrees live-action image data in three dimensions, the method further comprises:
step S302: the server 120 obtains a two-dimensional CAD drawing corresponding to a preset three-dimensional building model.
Step S304: the server 120 performs coordinate matching on the two-dimensional CAD drawing and the three-dimensional building model, and determines a second mapping relation between any coordinate on the two-dimensional CAD drawing and a ground three-dimensional coordinate in the three-dimensional building model.
Step S306: the server 120 marks each of the points of interest on the two-dimensional CAD drawing.
Compared with the method for directly checking the position of the focus point needing to shoot the 360-degree live-action picture in the three-dimensional building model, the method for checking the focus point needing to shoot the 360-degree live-action picture in the three-dimensional building model has the advantages that the focus point needing to be shot in the three-dimensional building model can be marked on the two-dimensional CAD drawing, and therefore workers can conveniently check the focus point needing to be shot in a construction site by using a mobile phone.
In one embodiment, the method further comprises:
and performing data conversion processing and storage on the two-dimensional CAD drawing so as to browse through a Web browser.
And carrying out lightweight processing and storage on the three-dimensional building model so as to be capable of browsing through a Web browser.
In the invention, because the worker needs to take pictures and videos at the construction site, the two-dimensional CAD drawing and the three-dimensional building model need to be checked by using the portable terminal 110, and the two are converted into a format which can be browsed by a Web browser, thereby greatly facilitating the operation of the worker at the site.
In one embodiment, the method further comprises:
when the 360-degree camera detects that the button is switched to the triggered state from the non-triggered state, the 360-degree camera is switched to the 360-degree live-action photo shooting mode.
When the 360-degree camera detects that the button is switched from a triggered state to a non-triggered state, switching to a 360-degree live-action video shooting mode; wherein the button comprises a hardware button or a software button.
In the invention, considering that the time taken by the staff for taking the picture is far less than the time for taking the video, the two modes of taking the picture and taking the video are switched conveniently for the staff, the 360-degree live-action picture taking mode of the camera can be switched only when the staff triggers the button by hands, and when the staff walks between two focus points, the staff does not need to be triggered by hands, thereby reducing the burden of the staff. Further, when switching between the 360-degree live-action video shooting mode and the 360-degree live-action photo shooting mode, the consistency between the current 360-degree live-action photo and the last frame of the 360-degree live-action video shot before the current 360-degree live-action photo and the first frame of the 360-degree live-action video shot after the current 360-degree live-action photo may be poor due to the fact that the worker forgets to switch the corresponding mode in time. In order to solve the problem, the same camera is adopted to shoot videos and photos simultaneously, and the 360-degree live-action photo shooting mode can be switched only under the condition that a worker triggers the button, so that when the worker adjusts the camera to a fixed direction to prepare for shooting, the 360-degree live-action photo shooting mode can be entered only by triggering the button, the 360-degree live-action photo obtained by shooting and the 360-degree live-action video shot by the previous frame (namely the last frame of the 360-degree live-action video) can be ensured to be good in visual angle consistency, when the worker finishes shooting, the button is released to enable the button to be converted from a triggered state to a non-triggered state, and the first frame shot in the 360-degree live-action video shooting mode is good in visual angle consistency. Good consistency means that two frames before and after the 360-degree live-action photo have more identical feature points, which can ensure the accuracy of calculating the position of each camera by the orb (original FAST and Rotated bright) feature point detection algorithm.
Example 2:
to facilitate an understanding of the present invention, the following provides a preferred embodiment illustrating the complete flow of a digital twin-based construction survey management system of the present invention:
1) and uploading the CAD drawing corresponding to the building, and performing data conversion processing and storage on the CAD drawing so as to browse through a Web browser.
2) Uploading a BIM model corresponding to the building, and carrying out lightweight processing and storage on the BIM model so as to enable the BIM model to be browsed through a Web browser.
3) And (3) carrying out coordinate matching on the CAD drawing and the BIM model, specifically, calculating a conversion matrix from a CAD coordinate system to the BIM coordinate system through n (n is more than or equal to 3) corresponding point coordinates (x, y), and calculating the ground coordinates corresponding to the BIM model through the conversion matrix for any coordinate on all subsequent CAD drawings.
4) According to business needs, places needing to be photographed are planned and marked on a CAD drawing, tree-shaped structure data is finally formed, the levels are region-multi-span-layer-point, the tree-shaped structure data are stored in a database and are provided for a mobile phone App to use, and photographing is mainly facilitated for photographing personnel according to preset positions and photographing sequences. If the BIM model is directly used for position marking, the operation of field shooting personnel is troublesome, and after all, the complexity of field operation 3D is much greater than that of 2D.
5) The shooting personnel can fix the panoramic camera (namely a 360-degree camera) on the safety helmet when taking the safety helmet to the scene for shooting, and can also fix the panoramic camera on the tripod for shooting, wherein the specific mode is selected by the staff according to specific requirements, and the method is not limited here. After the panoramic camera is prepared, a worker logs in the mobile phone App and connects the panoramic camera to enter a corresponding project space, the pre-planned photographing point data (region-multi-field-layer-point) is automatically acquired, a certain layer is selected to start a photographing guide, and then the photographing process is completed according to the voice prompt of the App and the position of a photographing point map displayed on a mobile phone screen. In the process of going from one photo site to another, the App will automatically record video data using panoramic recording. After all the shots are finished, the App automatically uploads the panoramic photos and videos to the server 120.
6) After the server 120 obtains all the panoramic photos and videos, the world coordinates (x, y, z) of each photo in the photographing coordinate system (i.e., the world coordinate system) are obtained through calculation. The method specifically comprises the following steps: dividing each panoramic video into a plurality of photos (generally 30 or 25 depending on the FPS of the video), finding the same characteristic points for every two photos by a characteristic point algorithm, calculating the displacement of the characteristic points to calculate the change of the positions of the cameras for twice photographing to obtain relative coordinates, and finally calculating the world coordinates of all the photos by taking the first photo as an origin. The reason why the calculation is not performed directly using the panoramic photograph in the present embodiment is: if the distance between the two photos is too far, enough characteristic points cannot be found for operation, and the photographing points concerned by the client are generally far, so that the two photographing points are connected through the panoramic video, the video is split into a plurality of photos for operation, the distance between the two photos participating in the operation is short, enough characteristic points are available for operation, and the operation is more accurate. The reason why the video is not directly shot and the video and the picture are needed in the embodiment is that the picture quality derived from the video is not good than that of the picture to be shot, and the customer has higher requirements on color restoration and fineness of the picture.
7) Matching the photographing coordinate system with a CAD coordinate system, specifically, calculating a conversion matrix from the photographing coordinate system to the CAD coordinate system through n (n is more than or equal to 3) corresponding point coordinates (x, y), then calculating coordinates of all photographing points in the CAD, calculating ground coordinates of the photographing points in the BIM through the conversion matrix from the CAD to the BIM, finally converting the ground coordinates into units in the BIM according to the unit of a z value in the photographing coordinate system as a meter, and finally obtaining coordinates (x, y, z) of each photo in the BIM to generate a digital twin model; the reason why the photographing coordinate system and the BIM coordinate system are not directly matched in this embodiment is that the requirement on the user is always high when the operation is performed in 3D, and especially the alignment work is performed, so that the operation in 2D lowers many thresholds, and a common clerk can perform the operation. This is another reason why CAD needs to be introduced.
8) And the detection personnel measure the building through the digital twin model. And entering a live-action mode, freely switching to different places and visual angles through mark points on the CAD drawing or shooting points of a tree structure, and directly roaming in the display of the panoramic image. If the design data is compared, the mode can be switched to the mixed mode or the superposition mode to find whether the design is in accordance with the specification. The problem which needs to be recorded can be found by creating a marking point on the panoramic image or the BIM model and filling a corresponding form for recording.
9) And (5) repeating the operation (8) by the rechecking personnel to check defects and repair leakage.
10) The management personnel can visually see the spatial positions and the states of all monitoring data in the whole building by checking the detection data and using a pure virtual mode, enter a certain monitoring record, can assign subsequent processing personnel, and the related personnel can receive new task reminders in the form of Email, short messages, App pushing and the like.
11) The subsequent processing personnel receives a new task, and knows the specific position to be processed, the condition to be processed, the tool to be processed and the like through the digital twin model, so that the more sufficient preparation is made before starting.
12) And after the follow-up processing personnel finish the task, photographing to record the finish condition and updating the task state. And the manager receives the task updating notice immediately and can check the specific completion condition by clicking.
13) With the lapse of time, the periodical shooting enables a plurality of panoramic photos to exist in each planned shooting point, and all the key moments of the buildings from the absence to the existence are recorded. Managers can look up or compare data at different time points as required, and tracing requirements of monitoring problems are met.
In summary, the present embodiment has at least the following advantages:
1) when the field data is collected, 360-degree panoramic photo shooting is used for replacing common photo shooting. By doing so, not only is all-around data recorded, but also shooting can be completed by ordinary personnel with little training. Therefore, the field expert can monitor remotely without going to the site;
2) when the field data is uploaded to the server 120, not only the data is stored, but all 360 panoramic photos, relevant CAD two-dimensional drawings and BIM three-dimensional models are associated to generate a digital twin model (virtual and real three-dimensional space are completely overlapped);
3) when monitoring personnel monitor, the field monitoring is changed into remote monitoring, the digital twin model is utilized, the remote monitoring can be carried out at a distance, if the remote monitoring is carried out in the real world in the close place, the visual angle of different positions of a building can be quickly switched, all field details under the visual angle can be checked at 360 degrees, in addition, the reality and the design can be conveniently compared by means of the design data of the digital twin model, and the problems can be quickly positioned by using three modes of left-right comparison, mixed comparison and overlapped comparison. And finally, establishing a space mark point on the digital twin model, and recording the space mark point in cooperation with filling of the form.
4) When checking the monitoring data, a manager does not check fragmented list data, but directly and visually checks the spatial distribution condition, the subsequent follow-up state and the like of all the monitoring data on the digital twin model, and freely switches between a real mode and a virtual mode, and the real mode can also compare field changes of different shooting times.
5) When the follow-up person receives the follow-up task, the follow-up person can not know the field situation through the plane photo any more, and can view the task on the digital twin model instead. Therefore, the site situation can be known more fully, and the early preparation of task execution is well made.
Example 3:
as shown in fig. 1, in one embodiment, there is also provided a digital twin-based building measurement management system, the system comprising:
the acquisition module 121 is configured to acquire three-dimensional 360-degree live-action image data; the 360-degree live-action image data comprises 360-degree live-action video and 360-degree live-action photos; the 360-degree live-action pictures are shot by workers carrying 360-degree cameras towards fixed directions at various focus points of a construction site, and the 360-degree live-action videos are shot by the workers carrying the 360-degree cameras when the workers walk between the focus points of the construction site;
the alignment module 122 is configured to align the 360-degree live-action photograph with a preset three-dimensional building model;
the linkage module 123 is configured to establish a linkage relationship between the aligned 360-degree live-action photograph and the three-dimensional building model, so that the 360-degree live-action image data and the perspective of the three-dimensional building model are kept consistent;
and the comparison module 124 is used for comparing the 360-degree live-action photo with the three-dimensional building model at the same view angle so as to perform building measurement management.
In one embodiment, the alignment module 122 includes:
the world coordinate calculation unit is used for calculating world coordinates corresponding to the cameras when the other 360-degree live-action pictures are shot according to the 360-degree live-action image data by taking the world coordinates of the cameras when the first 360-degree live-action picture is shot as an origin;
the first mapping relation determining unit is used for determining a first mapping relation between the world coordinate and the three-dimensional coordinate of the three-dimensional building model according to at least three groups of preset corresponding coordinate point pairs;
and the alignment unit is used for determining the position and the posture of a camera which sees the same visual angle as each 360-degree live-action picture in the three-dimensional model according to the fixed direction and the first mapping relation so as to align each 360-degree live-action picture with a preset three-dimensional building model.
In one embodiment, the world coordinate calculation unit includes:
the photo data set generating subunit is used for splitting a plurality of video frame photos obtained by splitting a 360-degree live-action video shot between an attention point corresponding to the camera and an adjacent attention point when the first 360-degree live-action photo is shot and the 360-degree live-action photos shot at the two attention points as a group of photo data set;
the world coordinate calculation subunit is used for sequentially calculating the change of the camera position corresponding to each photo according to the shooting time sequence of each photo in the group of photo data sets until the world coordinate corresponding to the attention point camera adjacent to the first 360-degree live-action photo is obtained;
and the circulation subunit is used for sequentially calculating the world coordinates corresponding to the cameras when the other 360-degree live-action pictures are taken according to the steps A to B.
In one embodiment, the system further comprises a 360 degree camera, the 360 degree camera comprising:
the photo shooting switching module 111 is used for switching to a 360-degree live-action photo shooting mode when detecting that the button is switched from a non-triggered state to a triggered state;
the video shooting switching module 112 is configured to switch to a 360-degree live-action video shooting mode when detecting that the button is switched from a triggered state to a non-triggered state; wherein the button comprises a hardware button or a software button.
It is understood that the system embodiment of the present invention and the method embodiment in embodiment 1 are based on the same inventive concept, and are not described herein again.
FIG. 6 is a diagram illustrating an internal structure of a computer device in one embodiment. The computer device may specifically be the terminal 110 (or the server 120) in fig. 1. As shown in fig. 6, the computer apparatus includes a processor, a memory, a network interface, an input device, and a display screen connected through a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program that, when executed by the processor, causes the processor to implement a digital twin-based building measurement management method. The internal memory may also have stored therein a computer program that, when executed by the processor, causes the processor to perform a digital twin-based building measurement management method. Those skilled in the art will appreciate that the configuration shown in fig. 6 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing devices to which aspects of the present invention may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, the digital twin-based building measurement management system provided herein may be implemented in the form of a computer program that is executable on a computer device such as that shown in fig. 6. The memory of the computer device may store various program modules constituting the digital twin-based construction survey management system, such as the acquisition module 121, the alignment module 122, the linkage module 123, and the comparison module 124 shown in fig. 1. The computer program of each program module causes the processor to execute the steps of the digital twin-based building measurement management method according to each embodiment of the present application described in the present specification.
For example, the computer device shown in fig. 6 may perform the step of acquiring three-dimensional 360-degree live-action image data through the acquisition module 121 in the digital twin-based construction survey management system shown in fig. 1. The step of aligning the 360 degree live view with the preset three-dimensional building model is performed by the alignment module 122. And executing a step of establishing the well-aligned linkage relationship between the 360-degree live-action photo and the three-dimensional building model through the linkage module 123, so that the 360-degree live-action image data and the three-dimensional building model keep consistent visual angle. The step of comparing the 360-degree live-action photo with the three-dimensional building model at the same view angle is executed by the comparison module 124, so as to perform building measurement management.
In one embodiment, there is provided an electronic device including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the program performing the steps of the above-described digital twin-based building measurement management method. Here, the steps of the digital twin-based building survey management method may be the steps of the digital twin-based building survey management method of each of the above embodiments.
In one embodiment, a computer-readable storage medium is provided that stores computer-executable instructions for causing a computer to perform the steps of the above-described digital twin-based building measurement management method. Here, the steps of the digital twin-based building survey management method may be the steps of the digital twin-based building survey management method of each of the above embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRA), Rambus Direct RAM (RDRA), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
Claims (10)
1. A method for building survey management based on digital twinning, the method comprising:
acquiring three-dimensional 360-degree live-action image data; the 360-degree live-action image data comprises 360-degree live-action video and 360-degree live-action photos; the 360-degree live-action pictures are shot by workers carrying 360-degree cameras towards fixed directions at various focus points of a construction site, and the 360-degree live-action videos are shot by the workers carrying the 360-degree cameras when the workers walk between the focus points of the construction site;
aligning the 360-degree live-action picture with a preset three-dimensional building model;
establishing a well-aligned linkage relation between the 360-degree live-action photo and the three-dimensional building model, so that the 360-degree live-action image data and the visual angle of the three-dimensional building model are kept consistent;
and comparing the 360-degree live-action photo with the three-dimensional building model at the same view angle so as to carry out building measurement management.
2. The method for managing building survey based on digital twin as claimed in claim 1, wherein said aligning a 360 degree live view with a preset three-dimensional building model comprises:
taking the world coordinates of the camera when the first 360-degree live-action picture is shot as an origin, and calculating the corresponding world coordinates of the camera when other 360-degree live-action pictures are shot according to 360-degree live-action image data;
determining a first mapping relation between the world coordinate and the three-dimensional coordinate of the three-dimensional building model according to at least three groups of preset corresponding coordinate point pairs;
and determining the position and the posture of a camera which sees the same visual angle as each 360-degree live-action picture in the three-dimensional model according to the fixed direction and the first mapping relation so as to align each 360-degree live-action picture with a preset three-dimensional building model.
3. The building measurement management method based on the digital twin as claimed in claim 2, wherein the calculating the world coordinates corresponding to the cameras taking the other 360-degree live-action pictures according to the 360-degree live-action image data with the world coordinates of the camera taking the first 360-degree live-action picture as an origin comprises:
step A: taking a plurality of video frame photos obtained by splitting a 360-degree live-action video shot between an attention point corresponding to a camera and an adjacent attention point when a first 360-degree live-action photo is shot and the 360-degree live-action photos shot at the two attention points as a group of photo data sets;
and B: sequentially calculating the change of the camera position corresponding to each photo according to the shooting time sequence of each photo in a group of photo data sets until the world coordinates corresponding to the attention point camera adjacent to the first 360-degree live-action photo are obtained;
and C: and D, sequentially calculating world coordinates corresponding to the cameras when other 360-degree live-action pictures are taken according to the steps A to B.
4. The method for digital twin-based architectural survey management according to claim 1, wherein prior to said acquiring three-dimensional 360 degree live-action image data, the method further comprises:
acquiring a two-dimensional CAD drawing corresponding to a preset three-dimensional building model;
coordinate matching is carried out on the two-dimensional CAD drawing and the three-dimensional building model, and a second mapping relation between any coordinate on the two-dimensional CAD drawing and a ground three-dimensional coordinate in the three-dimensional building model is determined;
and marking the attention points on the two-dimensional CAD drawing.
5. The method of claim 4, wherein the method further comprises:
performing data conversion processing and storage on the two-dimensional CAD drawing so as to browse through a Web browser;
and carrying out lightweight processing and storage on the three-dimensional building model so as to be capable of browsing through a Web browser.
6. The method of claim 1, wherein the method further comprises:
when the 360-degree camera detects that the button is switched from a non-triggered state to a triggered state, switching to a 360-degree live-action photo shooting mode;
when the 360-degree camera detects that the button is switched from a triggered state to a non-triggered state, switching to a 360-degree live-action video shooting mode; wherein the button comprises a hardware button or a software button.
7. A digital twin-based construction survey management system, the system comprising:
the acquisition module is used for acquiring three-dimensional 360-degree live-action image data; the 360-degree live-action image data comprises 360-degree live-action video and 360-degree live-action photos; the 360-degree live-action pictures are shot by workers carrying 360-degree cameras towards fixed directions at various focus points of a construction site, and the 360-degree live-action videos are shot by the workers carrying the 360-degree cameras when the workers walk between the focus points of the construction site;
the alignment module is used for aligning the 360-degree live-action photo with a preset three-dimensional building model;
the linkage module is used for establishing a linkage relation between the aligned 360-degree live-action photo and the three-dimensional building model, so that the view angle of the 360-degree live-action image data is consistent with that of the three-dimensional building model;
and the comparison module is used for comparing the 360-degree live-action photos with the three-dimensional building model at the same visual angle so as to carry out building measurement management.
8. The digital twin based construction survey management system of claim 7 wherein the alignment module comprises:
the world coordinate calculation unit is used for calculating world coordinates corresponding to the cameras when the other 360-degree live-action pictures are shot according to the 360-degree live-action image data by taking the world coordinates of the cameras when the first 360-degree live-action picture is shot as an origin;
the first mapping relation determining unit is used for determining a first mapping relation between the world coordinate and the three-dimensional coordinate of the three-dimensional building model according to at least three groups of preset corresponding coordinate point pairs;
and the alignment unit is used for determining the position and the posture of a camera which sees the same visual angle as each 360-degree live-action picture in the three-dimensional model according to the fixed direction and the first mapping relation so as to align each 360-degree live-action picture with a preset three-dimensional building model.
9. The digital twin-based construction survey management system of claim 8, wherein the world coordinate calculation unit comprises:
the photo data set generating subunit is used for splitting a plurality of video frame photos obtained by splitting a 360-degree live-action video shot between an attention point corresponding to the camera and an adjacent attention point when the first 360-degree live-action photo is shot and the 360-degree live-action photos shot at the two attention points as a group of photo data set;
the world coordinate calculation subunit is used for sequentially calculating the change of the camera position corresponding to each photo according to the shooting time sequence of each photo in the group of photo data sets until the world coordinate corresponding to the attention point camera adjacent to the first 360-degree live-action photo is obtained;
and the circulation subunit is used for sequentially calculating the world coordinates corresponding to the cameras when the other 360-degree live-action pictures are taken according to the steps A to B.
10. A digital twin based construction survey management system according to claim 7 further comprising a 360 degree camera, the 360 degree camera comprising:
the photo shooting switching module is used for switching to a 360-degree live-action photo shooting mode when detecting that the button is switched from a non-triggered state to a triggered state;
the video shooting switching module is used for switching to a 360-degree live-action video shooting mode when detecting that the button is switched from a triggered state to a non-triggered state; wherein the button comprises a hardware button or a software button.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111681378.9A CN114331194B (en) | 2021-12-30 | 2021-12-30 | Building measurement management method and system based on digital twins |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111681378.9A CN114331194B (en) | 2021-12-30 | 2021-12-30 | Building measurement management method and system based on digital twins |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114331194A true CN114331194A (en) | 2022-04-12 |
| CN114331194B CN114331194B (en) | 2022-09-09 |
Family
ID=81022862
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111681378.9A Active CN114331194B (en) | 2021-12-30 | 2021-12-30 | Building measurement management method and system based on digital twins |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114331194B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008102205A2 (en) * | 2006-08-09 | 2008-08-28 | Fotonation Vision Limited | Detection of airborne flash artifacts using preflash image |
| US20190096092A1 (en) * | 2017-09-27 | 2019-03-28 | Arcsoft (Hangzhou) Multimedia Technology Co., Ltd. | Method and device for calibration |
| CN111696216A (en) * | 2020-06-16 | 2020-09-22 | 浙江大华技术股份有限公司 | Three-dimensional augmented reality panorama fusion method and system |
| CN112015275A (en) * | 2020-08-29 | 2020-12-01 | 南京翱翔智能制造科技有限公司 | Digital twin AR interaction method and system |
| CN112396686A (en) * | 2019-07-31 | 2021-02-23 | 鸿富锦精密电子(天津)有限公司 | Three-dimensional scene engineering simulation and live-action fusion system and method |
| CN112581618A (en) * | 2020-12-23 | 2021-03-30 | 深圳前海贾维斯数据咨询有限公司 | Three-dimensional building model and real scene comparison method and system in building engineering industry |
| CN112819943A (en) * | 2021-01-15 | 2021-05-18 | 北京航空航天大学 | Active vision SLAM system based on panoramic camera |
| CN113327319A (en) * | 2021-06-03 | 2021-08-31 | 中德（珠海）人工智能研究院有限公司 | Complex scene modeling method and device, server and readable storage medium |
-
2021
- 2021-12-30 CN CN202111681378.9A patent/CN114331194B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008102205A2 (en) * | 2006-08-09 | 2008-08-28 | Fotonation Vision Limited | Detection of airborne flash artifacts using preflash image |
| US20190096092A1 (en) * | 2017-09-27 | 2019-03-28 | Arcsoft (Hangzhou) Multimedia Technology Co., Ltd. | Method and device for calibration |
| CN112396686A (en) * | 2019-07-31 | 2021-02-23 | 鸿富锦精密电子(天津)有限公司 | Three-dimensional scene engineering simulation and live-action fusion system and method |
| CN111696216A (en) * | 2020-06-16 | 2020-09-22 | 浙江大华技术股份有限公司 | Three-dimensional augmented reality panorama fusion method and system |
| CN112015275A (en) * | 2020-08-29 | 2020-12-01 | 南京翱翔智能制造科技有限公司 | Digital twin AR interaction method and system |
| CN112581618A (en) * | 2020-12-23 | 2021-03-30 | 深圳前海贾维斯数据咨询有限公司 | Three-dimensional building model and real scene comparison method and system in building engineering industry |
| CN112819943A (en) * | 2021-01-15 | 2021-05-18 | 北京航空航天大学 | Active vision SLAM system based on panoramic camera |
| CN113327319A (en) * | 2021-06-03 | 2021-08-31 | 中德（珠海）人工智能研究院有限公司 | Complex scene modeling method and device, server and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114331194B (en) | 2022-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5093053B2 (en) | Electronic camera | |
| KR101181967B1 (en) | 3D street view system using identification information. | |
| CN108958469B (en) | Method for adding hyperlinks in virtual world based on augmented reality | |
| WO2023093217A1 (en) | Data labeling method and apparatus, and computer device, storage medium and program | |
| KR100545048B1 (en) | System for drawing blind area in aerial photograph and method thereof | |
| JP5854951B2 (en) | Panorama image generator | |
| CN112383745A (en) | Panoramic image-based digital presentation method for large-scale clustered construction project | |
| CN110751728A (en) | Virtual reality equipment and method with BIM building model mixed reality function | |
| KR101259550B1 (en) | Method And Apparatus Contrasting Image Through Perspective Distortion Correction | |
| KR20190109094A (en) | Real time remote system and method for observing tunnel working face | |
| CN114331194B (en) | Building measurement management method and system based on digital twins | |
| US20210209844A1 (en) | Computer Vision Systems and Methods for End to End Image Inspection | |
| JP2005310044A (en) | Apparatus, method and program for data processing | |
| CN114048541B (en) | Asset space marking method and system based on digital twins | |
| CN111064947A (en) | Panoramic-based video fusion method, system, device and storage medium | |
| CN114565849B (en) | Asset AI (artificial intelligence) identification and positioning method and system based on digital twin | |
| JP4208980B2 (en) | Photo shooting management method | |
| CN113836337B (en) | BIM display method, device, equipment and storage medium | |
| JP3859794B2 (en) | Photo shooting management method | |
| León-Vega et al. | Fisheye lens image capture analysis for indoor 3d reconstruction and evaluation | |
| CN112581618B (en) | Three-dimensional building model and real scene comparison method and system in building engineering industry | |
| CN116310237A (en) | Method, system, electronic equipment and storage medium for constructing online three-dimensional map | |
| Anastasiou et al. | Holistic 3d Digital Documentation of a Byzantine Church | |
| CN112581618A (en) | Three-dimensional building model and real scene comparison method and system in building engineering industry | |
| CN116828315A (en) | Method for simultaneously controlling multiple trolleys to shoot panoramic pictures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |