TWM565860U - Smart civil engineering information system - Google Patents

Abstract

The present invention relates to a smart civil engineering information system providing for a civil engineering project including a plurality of modeling information and a plurality of on-site information associated with a building which is located on a construction site, comprising: an information server storing the plurality of modeling information and the plurality of on-site information; a remote device connected with the information server via an internet and comprising a information processing platform providing a first user to access and process the plurality of modeling information and the plurality of on-site information; and a handheld device connected with the information server via the internet, acquiring an instant image for the construction site and comprising an augmented reality platform, wherein the augmented reality platform accesses and correspondingly matches up the plurality of modeling information with the instant image to form an augmented reality image, and a second user makes the plurality of on-site information in reference with the augmented reality image and uploads the plurality of on-site information to the information server instantly.

Description

Smart Engineering Information System

This creative department is about a smart engineering information system, especially a platform that includes action augmented reality, which can be used to obtain real-time engineering entity information at the construction site, and can be instantly transmitted back to the central server, and corresponding to the virtual model information. Combine, provide relevant personnel for immediate inspection, or other intelligent application information systems of various multi-applications.

With the evolution of the times, the domestic construction industry is facing the impact of internationalization. Foreign manufacturers rely on superior technology, rich experience, huge capital and excellent management capabilities to successively undertake many domestic projects. If the domestic construction industry cannot upgrade its own construction technology and strengthen it. Organizational management, improvement of construction efficiency, and reduction of engineering costs are bound to be incapable of competing with foreign manufacturers, and operation automation must be one of the key projects for technical improvement.

Today, the domestic construction industry is engaged in engineering construction or work item inspection. The measurement work is mostly done with basic ranging and angle measurement with low automation. However, in recent years, the architectural design of the building has become more complicated. The use of plan drawings is difficult to design, construct and check. It must be analyzed by 3D model design, and the workload and complexity of measurement are increasing.

In recent years, the construction industry has begun to develop the Building Information Modeling (BIM) integrated management system, in addition to using the three-dimensional model image, let the designer and the owner understand the overview of the project, the three-dimensional model objectization plus the time axis to form a four-dimensional model Using BIM and supplementing with construction visualization, this information can be used for planning and simulation of construction scheduling, and it is clear in advance. After the function and impact, reduce the mistakes in application and cost, and greatly improve the quality of the project.

Computer-assisted intervention is required to improve automation. Currently, the functions of computer-aided design used in the construction industry are often limited to the design plan before implementation. In addition to the change design during construction, the construction monitoring supervision of the construction project in the medium or late stage Manual inspections are used for inspections of work items. Although there are construction specifications to be observed, there is no reliable efficiency and quality. However, it depends on the professional and inspection experience of the construction personnel, so it is easy to cause a big gap between design and construction.

If we can combine the information received by the surveying and mapping technology with the building information model, we will be able to analyze and utilize the key stages of real physical space output and BIM virtual information, creating more new value for BIM in practical applications. The system applies the limited dilemma during the construction phase, and allows the mapping technology to exert greater utility in the construction industry, so that the life cycle of the entire building can be more fully planned, maintained and managed.

In recent years, the productivity of the construction industry is low, and its industrial automation is not the main reason for this phenomenon. The traditional construction technology relies on the professional and inspection experience of the construction personnel, and it is difficult to rely on effective and objective numerical conditions to judge to maintain reliable efficiency and quality. When conducting construction inspection, the on-site engineer will carry out inspection according to the inspection project, inspection method and design specification. The items to be inspected are complicated, the schedule pressure is large, and the data is huge. The inspection personnel are required to make a zero mistake in the judgment. There are practical difficulties, which may cause the construction errors of non-checking projects or the problems that are inconsistent with the drawings to be easily ignored, resulting in an increase in construction costs in the later period.

The job is the reason, the conventional surface fitting or the full-surface fitting technology, in fact, there are still many shortcomings and a lot of room for improvement, in view of the shortcomings of the conventional technology, and to improve the flatness of the mechanism during the fitting, the pressure of the fit Uniform distribution, enhanced adhesion, reduced colloidal thickness, reduced overall cost, improved bonding quality, etc., the applicant has been carefully tested and researched, and has a perseverance In the spirit, I finally conceived the "Smart Engineering Information System" of this case, which can overcome the shortcomings of the above-mentioned conventional technologies. The following is a brief description of the creation.

In order to improve the application level of the current Building Information Modeling (BIM) on the construction site, this author proposes an image application technology that can provide instant visual feedback and will be numerically analyzed and computerized. The three-dimensional spatial information obtained from surveying and mapping, combined with the three-dimensional design model as a check record, attempts to visually present the difference between the actual site construction conditions and the design drawings. This can strengthen the judgment basis of the on-site inspection personnel, and shorten the reflection time of construction errors, to avoid the cost increase caused by subsequent incorrect operations, or to speed up the revision of the engineering change design to reduce the lack of personnel experience. Loss, and delays in engineering caused by excessive inspection data or documents, improve the efficiency and reliability of on-site construction monitoring.

The image application technology proposed by this creation can meet the needs of construction supervision: the concept of combining the simple and immediate characteristics of the forward project, and the processing of real-world spatial information, and the concept of refining the appearance of the three-dimensional urban application model. It is applied to the indoor scenes of the construction site, simplifying the model data for visual 3D rendering and analysis, assisting the BIM system in the construction work, and making up the actual data in the construction operation phase to compensate for the lack of actual data in the construction operation phase, and establishing the old inspection of the engineer. In the nuclear mode, the simulation of the finished product presented by the handheld smart device is used to further reduce the time of the mistaken discovery and further effectively reduce the cost of dismantling and the manpower of the inspection.

This creation mainly carries out non-destructive and rough inspection during construction during the construction process by means of simple measurement, effectively mapping the image to the model space coordinate system and converting The image information is the material of the model component, and finally the BIM system is used to complete the three-dimensional model of image application for subsequent comparison of the inspection records, or computer-assisted judgment of the actual progress of the construction, review of construction progress problems or performance evaluation and evaluation.

According to this work, a smart engineering information system is proposed, which is supplied for engineering projects, which includes complex model information related to the building and a plurality of on-site information, which is located at the project site, and includes: an information server, which stores The model information and the on-site information; the remote device, which is connected to the information server via the network, and includes an information processing platform for providing the first user to access and process the model information through the information processing platform And the on-site information; and the handheld device, which connects to the information server through the network, captures the live image of the project site, and includes an augmented reality platform, and the augmented reality platform accesses the model information. And combining with the real-time image to form an augmented reality image, and the second user records the live information with reference to the augmented reality image at the project site, and immediately uploads the information to the information server.

Preferably, the first user performs an object image correspondence analysis program through the information processing platform, and the scene information is corresponding to the model information, and the object image correspondence analysis program includes geometric intersection analysis, spatial resection analysis, and A collinear analysis of one of them.

Preferably, the first user performs a material image production process through the information processing platform, and the two-dimensional image information included in the on-site information is orthorectified, standardized, and collaged into a material image. The production process includes one of shadow judgment analysis, object capture analysis, image orthorectification, geometric correction, coordinate normalization, and image collage.

Preferably, the first user performs the object image correspondence analysis program and the material image production program on the remote device by operating the information processing platform, and the site is processed after the processing. Information is applied to the model information.

Preferably, the first user accesses the model information and the on-site information through the information processing platform to import, export, store, view, copy, edit, modify, merge, superimpose, and collage. , calibrating, capturing, stitching, scaling, segmenting, counting, analyzing, inspecting, auditing, or managing information about such models and such on-site information.

Preferably, the augmented reality platform accesses the model information to the information server to virtually display the digital three-dimensional model of the building on the augmented reality platform of the handheld device.

Preferably, the handheld device includes a gyroscope and a geomagnetic meter, and the handheld device transmits the model information and the real image through the gyroscope and the geomagnet, respectively, and completes the correspondence between the model information and the instant image. Combining the model information with the real-time image to form the augmented reality image.

Preferably, the second user is located at the engineering site, and according to the model information in the augmented reality image, the real-time image is recorded as the live information at the corresponding location, and is immediately uploaded to the information servo. Device.

Preferably, the first user is an engineer, a manager or a supervisor, and the second user is an inspector or a field member.

Preferably, the information processing platform is provided through a webpage system and includes a webpage platform interface, and the first user can directly operate the information processing platform through an internet browser.

Preferably, the engineering project is a project on construction, construction or public works.

100‧‧‧Smart Engineering Information System

110‧‧‧Information Server

120‧‧‧First remote unit

130‧‧‧Second remote unit

140‧‧‧Handheld smart device

Figure 1 is a schematic diagram showing the system architecture of the Creative Intelligence Engineering Information System.

Figure 2 is a schematic diagram showing the architecture between the creative information server and the handheld smart device.

Figure 3 is a schematic diagram showing the operational state of the augmented reality platform of the present handheld smart device at the engineering site.

Figure 4 is a schematic diagram showing the augmented reality operation interface of the augmented reality platform of the present handheld smart device.

Fig. 5A and Fig. 5B are diagrams showing the corresponding matching and positioning of the model information and the field information in the indoor space when the augmented reality platform is indoors.

FIG. 6A is a schematic diagram showing a photographing interface of the augmented reality platform of the present handheld smart device.

Figure 6B shows a schematic diagram of the recording interface of the augmented reality platform of the present handheld smart device.

Figure 7A shows a schematic diagram of the architecture between the remote device and the handheld smart device, and the first processing flow for the engineering related information of the creative smart engineering information system.

Figure 7B reveals the second processing flow chart of the engineering-related information in the Creative Intelligence Engineering Information System.

Figure 8 is a schematic diagram showing the resection of a single photo space in this creation.

Fig. 9A, Fig. 9B, and Fig. 9C show a schematic diagram of projection and masking judgment of the present object.

Figure 10 is a schematic diagram showing the intersection of the spatial plane and the camera geometry image to obtain the object coordinates of the image point coordinates projected onto the plane.

Figure 11 is a schematic diagram showing the definition of two mutually perpendicular unit vectors in a three-dimensional coordinate space.

Figure 12 is a schematic diagram showing the conversion of three-dimensional coordinates into two-dimensional coordinates in a linear combination.

Figure 13 is a diagram showing the interpolated pixels after resampling by nearest neighbor interpolation.

Figure 14 reveals the on-site information of the original image of the building captured by the hand-held smart device at the engineering site.

Figure 15 shows the image of the material of the object formed by the orthorectification of the original image.

Figure 16 reveals a material image that is geometrically corrected and positioned in the correct position.

Figures 17, 18, and 19 show material images applied to a digital BIM 3D model.

The present invention will be fully understood by the following examples, so that those skilled in the art can do so, but the implementation of the present invention may not be restricted by the following implementation cases; the pattern of the creation is The size, size and scale are not included. The size, size and scale of the original implementation of this creation are not limited by the schema of the creation.

The term "preferred" as used herein is non-exclusive and should be understood as "preferably, but not limited to", and any steps described or recited in any specification or claim can be performed in any order, and are not limited to the claim The order of the present invention should be determined only by the accompanying claims and its equal schemes, and should not be determined by the examples of the embodiments; the term "comprising" and its variations appear in the specification and claims. An open term is not intended to be limiting, and does not exclude other features or steps.

FIG. 1 is a schematic diagram showing the system architecture of the creative intelligent engineering information system; the intelligent engineering information system 100 of the present invention includes an information server 110, a first remote device 120, a second remote device 130, a handheld smart device 140, and the like. Each device is connected to each other through a network. The network can use a wired network or a wireless network. The wireless network can be a 3G network, a 4G network, a Bluetooth network, or a Wi-Fi network. The first remote device 120 and the second far The end device 130 can be a desktop computer, a notebook computer, a tablet computer, a smart phone, or a tablet phone. The handheld smart device 140 can be a handheld smart device, a tablet computer, a smart phone, or a tablet phone, in this embodiment. The first remote device 120 is preferably a desktop computer and is operated by an engineer. The second remote device 130 is preferably a tablet computer and is operated by a manager. The handheld smart device 140 is preferably a smart phone and is inspected by a technician. operating.

Figure 2 is a schematic diagram showing the architecture between the creative information server and the handheld smart device; the information server 110 is provided with an information processing platform, which includes an image processing module, a building model module, and a database module. And the management module, and the architectural model module preferably uses a SketchUp module or a Revit module, and the information processing platform 110 refers to the platform-as-a-service (PaaS) technology, and includes a web-based platform interface, which can communicate with the Internet through the Internet. The webpage system is provided to the user for operation, that is, the user can directly operate the information processing platform on the information server 110 through the Internet browser on the first remote device 120 or the second remote device 130.

FIG. 3 is a schematic diagram showing the operation state of the augmented reality platform of the present invention in the engineering field; FIG. 4 is a schematic diagram showing the augmented reality operation interface of the augmented reality platform of the present handheld smart device; The 140 is equipped with a Unity Augmented Reality Creation Tool Platform, which is a visualized physical 3D animation engine with Augmented Reality (AR) function for 3D augmented reality display before collecting local data. Through the Unity BIM 3D model of the building, and the real-time image matching of the building at the project site, as shown in Figures 3 and 4, to build the Augmented Reality (AR) of the handheld smart device. The scene serves as an interactive platform that dynamically simulates the virtual vision of the building. The 3D model is rendered through the screen of the mobile vehicle and is automated or Manually adjust the model perspective so that the auditor can quickly confirm the difference between the fabrication and design of the component. In addition to confirming the construction status and collecting the live photo as a record, this creation uses the image sensor of the mobile vehicle as the site data collection. The receiving end obtains the image of the surface material of the object, records the relative position of the shooting location and the model, and stores the data of the gyroscope and the accelerometer of the mobile vehicle, so the image, the relative position, and the posture of the vehicle are the augmented reality platform data. Collect important tasks and provide important sources of follow-up.

5A and 5B are diagrams showing the corresponding matching positioning of the model information and the field information in the indoor space when the augmented reality platform is indoors; the display of the augmented reality platform of the handheld smart device 140 is displayed. The building model, that is, the BIM information of the building, is actually the model information in the virtual space. It needs to be coordinately matched with the physical building in the physical space to form augmented reality images. The gyroscope and the geomagnetic meter on the smart device 140 first find out the direction in the indoor space, and then manipulate the model perspective on the handheld smart device 140 to augment the function of the visual platform to provide a visual (Filed of View, FOV) measurement. The left and right translations can be adjusted to match the relative position of the model and the real object, and the corresponding position of the user in the model is obtained. Then, the photo taken by the subsequent handheld smart device will be coordinated according to the model operation, and the model will be executed. Apply, join entities and virtual spaces.

FIG. 6A is a schematic diagram showing a photographing interface of the augmented reality platform of the present handheld smart device; FIG. 6B is a schematic diagram showing a recording interface of the augmented reality platform of the present handheld smart device; As shown in FIG. 6A, recording, text memo recording, etc. can also be performed at the engineering site. As shown in FIG. 6B, when the handheld smart device 140 completes spatial information collection at the engineering site, the data is transmitted back to the back end. The first remote device 120 performs data check and performs object image correspondence analysis to establish an object image correspondence relationship to a single photo. Space resection will refine the external orientation parameters of the image, then import the object information in the Revit project, use the number, attribute and vector data provided by the object to carry out the material image production process, and automatically shorten the inspection data and production system. The working hours of the material image, and the database is built for information storage, and finally the three-dimensional model of the material image is applied. The application model can display and check the difference with the real scene in the computer, and can also put the application model back into the real-world scene, and continue to augment the reality to the augmented reality platform of the handheld smart device as a three-dimensional Reference material for the model.

This creation mainly uses non-destructive and rough inspection during construction during the construction process, effectively mapping the image to the model space coordinate system, and converting the image information into the material of the model component, and finally combining BIM. The system completes the three-dimensional model of image application for subsequent comparison of the inspection records, or computer-assisted judgment of the actual progress of the construction, review of construction progress problems or performance verification and evaluation of pricing.

This creation uses the BIM model to compare and check the indoor scenes of the building through augmented reality, and records the image space information for subsequent production of the three-dimensional application model to facilitate engineering check and record. In order to verify the feasibility and effectiveness of this operation process, the sixth floor of the Civil Engineering Building of the University of Taiwan, which was established by the completion method, is used as the implementation area.

This creation is based on the sixth floor of the National Civil Engineering Building of NTU. It is divided into two parts in the BIM model data processing steps, which are the three-dimensional model FBX file and the rendezvous object coordinate data. The object coordinate data processing is based on the facet of the model object, and the information to be transferred must include: object number, facet number, facet normal vector, starting point three-dimensional coordinates, and end point three-dimensional coordinates. Use Revit Lookup with your own C# program to export the model data according to the above data format, and then merge it into object-oriented format based on the line segment data.

Unity is a cross-platform visual creation physics engine. Developers can define their scene environment with preset parameters, connect C# scripts to implement other computing logic, and provide developers to publish their completed applications to handheld smart devices. Platform, so this creation chooses this platform Unity, which can shorten the overall development time and enable 3D scenes and augmented reality to be quickly implemented through this visual creation physics engine.

Set the lens angle of view of the screen, so that the output of the APK file allows the mobile phone to display the image of the lens, and at the same time, the FBX 3D model is used to represent the interior of the building, allowing the user to drag the picture and adjust its viewing angle to achieve the most In accordance with the attitude angle of the current lens image, and recording the image information and the position and rotation posture of the camera in the three-dimensional model system, the subsequent processing program can be operated.

In the actual scene, the data is checked and the image data is collected manually, and then the image is mapped by a single image, and the relationship between the image and the 3D model is integrated, supplemented by automated processing, and the data is extracted and the material image is constructed. The 3D application model and other project analysis, improve the inspection results and quickly feedback to the BIM system as a record.

Figure 7A shows a schematic diagram of the architecture between the remote device and the handheld smart device, and a first processing flow for the engineering-related information of the creative intelligent engineering information system; Figure 7B reveals the engineering-related information of the creative intelligent engineering information system. The second processing flow chart; in summary, the creative intelligent engineering information system, the second user, such as a checker at the construction site, receives the augmented reality platform on the mobile device side of the handheld smart device 140. The BIM three-dimensional model data is then superimposed and matched with the physical structure, and the mobile device is used to augment the real-world scene, and then the smart device 140 is used to start capturing the scene information such as the image and posture of the building, and return to the center. Information server.

The first user may be an engineer or a supervisor, etc., and the remote device 120, 130 obtains the on-site information from the central information server, and begins to perform object image correspondence analysis and material image production between the site information and the model information. Process, finally store and update the material image and its location information, you can apply the on-site information to the 3D model, so that the latest development and current status of the construction site can be instantly presented on the BIM 3D information for internal supervisors and engineering related The personnel immediately check, check, compare, confirm, etc., or use computer-aided inspection, and the updated BIM 3D model information will be fed back to the augmented reality platform of the handheld smart device as a reference material for the 3D model. The corresponding image analysis also includes steps of obtaining the coordinates of the end face of the model face, obtaining the image coordinates of the corresponding point on the image, and analyzing the resection of the single photo space. The material imaging system also includes: shadow judgment analysis, image capture, geometric correction. Wait for steps.

(A) Image correspondence analysis

The orientation of the image can be divided into an interior orientation and an outer orientation. The inner orientation parameter is a parameter of the relative relationship between the camera perspective projection center and the image in the expression space, and describes the relationship between the photographing center and the photograph when the camera is photographed. The relative position size can be obtained by the camera rate method to obtain the value of the internal orientation parameter before shooting. The outer orientation parameter contains the perspective projection center ( X _L , Y _L , Z _L ) of the image and the triaxial attitude parameter ( ω , , κ ), used to describe the geometric relationship of the object image when shooting.

Figure 8 is a schematic diagram showing the resection of a single photo space in this creation; a space resection of a single image space based on the collinear equation, using known ground control point data and pixel observation Solving the external azimuth parameters, each image point observation can establish two observational collinear equations, and each image contains 6 unknown external azimuth parameters, and the total number of image observation equations must satisfy the number of unknown parameters. Solve, as shown in Figure 8, the collinear equation is as follows:

In the formula (1), f : like the main distance; ( x , y ): the coordinates of the image point on the photo; ( X , Y , Z ): the coordinates of the object point of the object space; ( X _L , Y _L , Z _L ): perspective center position parameter; △ x , △ y is the image point offset; m ₁₁ , m ₁₂ , L m ₃₃ ; rotation matrix element, by attitude parameter ( ω , , κ ) composition, attitude parameter equation as in equation (2):

Since the collinearity is a nonlinear equation, an initial external azimuth reference value is needed to achieve a linearized solution calculation in an iterative manner. In this creation, the mobile vehicle completion rate is first determined to obtain the orientation parameter of the camera module, and the inertial component information recorded at the time of shooting is converted into the external orientation reference value, and the known model three-dimensional coordinate point is used as the The image control data includes the full control points of the X, Y, and Z components. By solving the external orientation parameters by more than three sets of image control points, it is possible to obtain more accurate image position and posture than the inertial component information recorded during shooting. For the subsequent steps of material imaging production.

(B) Material imaging system

This creation aims at the image of the automated production of the material. Therefore, the image orientation parameter obtained by the object image is used as an important reference for this step. Using known internal and external orientation parameters, The three-dimensional model is used for perspective projection, and the process of converting the three-dimensional object into two-dimensional information can know the overlapping relationship between the objects on the projection plane, thereby judging whether the target object is obscured by other objects on the image, and looking for The object coordinates of the object end point and its corresponding image point coordinates. Finally, the ortho-beam operation is performed on the object body, and the image is geometrically deformed and corrected, and the material veneer of the target object is produced after the drawing.

Because this creation uses the veneer of image-mounting objects as an auxiliary tool for construction inspection, the data format is based on each surface of the building. In the process of model image production, when the objects overlap, the current viewing angle is reflected in the shadow between the planar objects. When the occlusion condition occurs, the object coordinates are reversed by the image coordinates of the overlapping faces, and the object is judged by the distance from the camera.

Fig. 9A, Fig. 9B, and Fig. 9C show a schematic diagram of the projection and occlusion judgment of the created object; when performing the occlusion judgment, first find out the overlapping objects in the model image, as shown in Fig. 9A, which are two mutually perpendicular walls. Projecting the contours of the three planes A, B, and C to obtain the 9B map. The three polygons after projection are known to overlap each other, and any two overlapping objects are selected to find the central image point coordinates of the overlapping region, such as The 9C map finds the coordinates of the central image point of the overlapping area after the A and C projections, and uses the principle of spatial plane and camera geometry to intersect.

Figure 10 is a schematic diagram showing the intersection of the spatial plane and the camera geometry image to obtain the coordinates of the object coordinates projected onto the plane; the known object plane equation and the geometric position of the camera are used as the control conditions of the system, and the calculation is performed. The puncture point coordinates are used as the object point coordinates corresponding to the image pixels, as shown in Fig. 10.

This method is a single image-based object image space coordinate conversion mode, which uses the equation (3) to represent a known plane in the object space, and the parameter { a , b , c } in the equation (3) is a plane method. The vector, and substituting this plane equation into a collinear form, can derive equation (4) to represent the coordinates of the intersection of the camera position and the vector of the pixel with the plane. By comparing the distances of the two wires, it is possible to discriminate the object of the shadow, and the object surface of the mask removes the overlapped area to be shielded for subsequent analysis work.

aX + bY + cZ = 1 ( 3)

Image geometry correction, or Image Rectification, is designed to eliminate the geometric distortion caused by the perspective projection during shooting and to present the corrected projection projections in orthophotos. In the imaging geometry of the telemetry image, all the beams will first pass through the center of the image and then intersect on the image plane. The coordinates of the points with the same horizontal position but different vertical heights will be imaged at different positions of the image. In ortho-projection geometry, the projected beam is perpendicular to the horizontal reference plane, so the height change does not have different imaging results on the orthographic projection.

When the aerial image is produced by orthophoto, the object coordinates, the normal vector and the horizontal reference plane on the same level model are used to obtain the corresponding image coordinates, and the perspective projection geometry is removed. The height difference displacement. The ground point coordinates ( X , Y , Z ) of the DEM and the model surface normal vector are known, and the above steps are performed to perform backprojection (Backward Projection) to solve the image point coordinates ( x , y ).

In this creation, the orthorectification of the image is performed using known three-dimensional coordinates. In the 3D model, the plane of each object is not nearly the same, and the plane normal vector is not necessarily parallel to the coordinate axis. If the three-axis coordinate is only taken as the two-dimensional orthophoto image coordinate, the ignored axial value may cause The projection is deformed so that the orthoimage has incorrect correction information. In addition to the two-dimensional and three-dimensional coordinates of the endpoint, the information connected to the plane of the model image also has a plane normal vector. However, in the three-dimensional coordinate system, if the plane is rotated by the angle between the two normal vectors, the rotation angle will be A variety of combinations result in the inability to find the correct rotation matrix for the plane. Therefore, in this creation, the concept of linear combination of vectors is used to obtain the orthophoto image coordinates.

Figure 11 is a schematic diagram showing the definition of two mutually perpendicular unit vectors in a three-dimensional coordinate space; Figure 12 is a schematic diagram for converting a three-dimensional coordinate into a two-dimensional coordinate in a linear combination; the first step is to define a coordinate frame of the projection target Define the appropriate coordinate axis direction according to the plane to be projected. For example, Figure 11 defines two mutually perpendicular unit vectors in the three-dimensional coordinate space, and then uses the linear combination of vectors (5) to set the three-dimensional coordinates (X , Y , Z). Converting to (a , b), as shown in Figure 12, and setting (a , b) its unit length, can represent the concept of two-dimensional coordinates (x , y), convert each object coordinate to get Corresponding ortho coordinates.

Back-projection of the ground point coordinates ( X , Y , Z ) to obtain the image coordinate system ( x , y ) with the origin of the photo center. The calculated image coordinates must be converted to the photo coordinates with the origin of the photo in the upper left corner. System, and set the size of the pixel to adjust the image size required for the process, the conversion relationship between the two coordinate systems is shown in the equation (6), where ( x _i , y _i ) is expressed as a known image coordinate (col, row) is the corresponding photo coordinate, s _p is the pixel size, and w and h are the length of the image and the length of the banner.

The geometric space transformation is used to correct the pixel spatial relationship in the image, and the stretching is performed according to certain rules. In numerical image processing, geometric transformation consists of two basic operations: coordinate transformation of coordinates, interpolation of pixel values.

The coordinate transformation can be expressed by the following formula (7): ( x , y ) = T {( v , w )} (7)

Where ( v , w ) is the cell coordinate in the original image, and (x, y) is the pixel coordinate after geometric transformation. Affine conversion is the most commonly used space coordinate conversion method. The general conversion formula is as shown in equation (8):

After the cell is repositioned, the DN value (Digital Number) of the cell needs to be re-added. However, when the rule mesh is filled with image information, the problem that the image coordinates are not integers is generated, and interpolation is required. Re-sampling to fill in the appropriate DN value. Common interpolation methods are Nearest Neighborhood, Linear Interpolation, and Double Cube. Bicubic Interpolation, this method is based on the nearest neighbor interpolation method with the shortest operation time, and each new position is given to the nearest neighbor DN value in the original image, although it is more bilinear interpolation, double cube Interpolation tends to cause some linear edges to lose detailed information and severe distortion, but greatly reduces the computational complexity of interpolation operations.

Figure 13 is a schematic diagram showing the interpolated pixels after re-sampling by nearest neighbor interpolation; the nearest neighbor interpolation is to assign the weight of the DN value in a distance manner, as shown in Fig. 13 for an image. A partial area of ×4, and the cell coordinates of the new image are as shown by the red slash square (Row=619.71, Col=493.39), and the nearest integer value of the coordinates is (Row=620, Col=493), the same The DN value after resampling is 56.

First, set the working range of the orthophoto image, select the corresponding model plane, and then perform the Inverse Mapping, and convert the 2D plane projection to the 3D space for image positioning. The intersection method is used to solve the orientation parameters of the image to link the 3D model information. Calculate the coordinates of the three-dimensional object point in the object space by linear transformation, and then back-project the selected plane to obtain the ortho-image frame, and connect the captured image to re-sample to obtain the positive of the plane. Shoot material images.

Figure 14 reveals the scene information of the original image of the built-in smart device at the engineering site; the 15th image reveals the material image of the original image after orthorectification; the 16th image reveals the geometric correction The material image in the correct position; the 17th, 18th, and 19th images reveal the material image applied to the digital BIM 3D model; the original image is as shown in Figure 14, according to the above process. The image of each plane is converted into an orthographic material image as shown in Fig. 15, which proves that the process proposed by the creation is feasible, and then the material image is translated to the correct position of the model according to the geometric relationship of the model plane, as shown in Fig. 16, Profit The fusion of multiple images and material image application, the results of the three-dimensional model application of the actual scene are as shown in Fig. 17, Fig. 18 and Fig. 19.

Further provide more of this creative implementation such as:

Embodiment 1: A smart engineering information system is applied to an engineering project, the engineering project includes complex model information associated with the building and a plurality of on-site information, the building is located at the engineering site, and includes: an information server, which stores the same Model information and such on-site information; a remote device connected to the information server via a network and including an information processing platform for providing a first user to access and process the model information through the information processing platform and And the on-site information; and the handheld device, which connects to the information server through the network, captures the live image of the project site, and includes an augmented reality platform, and the augmented reality platform accesses the model information and The real-time image is matched and positioned to form an augmented reality image, and the second user records the live information with reference to the augmented reality image at the project site, and immediately uploads the information to the information server.

Embodiment 2: The smart engineering information system of embodiment 1, wherein the first user implements an object image correspondence analysis program through the information processing platform, and the field information is corresponding to the model information.

Embodiment 3: The smart engineering information system of embodiment 1, wherein the object image correspondence analysis program comprises one of geometric intersection analysis, spatial resection analysis, and collinear analysis.

The smart engineering information system of the first embodiment, wherein the first user performs a material image production process through the information processing platform, and orients the two-dimensional image information included in the live information. Correct, standardize, and tile into material images.

Embodiment 5: The intelligent engineering information system according to Embodiment 1, wherein the material imaging production program comprises one of a shadow judgment analysis, an object capture analysis, an image orthorectification, a geometric correction, a coordinate standardization, and an image collage. .

The smart engineering information system of the first embodiment, wherein the first user performs processing on the remote device by operating the information processing platform to perform an object image correspondence analysis program and a material image production program. The on-site information is then applied to the model information.

The smart engineering information system of the first embodiment, wherein the first user accesses the model information and the on-site information through the information processing platform for importing, exporting, storing, and viewing. Copy, edit, modify, merge, overlay, tile, correct, capture, stitch, scale, segment, count, analyze, examine, audit, or manage information about such models and such on-site information.

Embodiment 8: The smart engineering information system of embodiment 1, wherein the augmented reality platform accesses the model information to the information server for virtual display on the augmented reality platform of the handheld device A digital three-dimensional model of the building.

Embodiment 9: The smart engineering information system of embodiment 1, wherein the handheld device comprises a gyroscope and a geomagnetic meter, and the handheld device transmits the model information and the instant image through the gyroscope and the geomagnet, respectively. After the correspondence between the model information and the real-time image is completed, the model information is combined with the real-time image to form the augmented reality image.

Embodiment 10: The smart engineering information system of embodiment 1, wherein the second user is located at the engineering site, and the live image is recorded at a corresponding location according to the model information in the augmented reality image. As such on-site information, and instantly uploaded to the information server.

Embodiment 11: The smart engineering information system of embodiment 1, wherein the first user is an engineer, a manager or a supervisor, and the second user is a checker or a field person.

Embodiment 12: The smart engineering information system of embodiment 1, wherein the first user and the second user each have different rights, and the model information and the field information are graded for different rights. A user accesses the second user.

Embodiment 13: The smart engineering information system as described in embodiment 1, wherein the information processing platform is provided through a webpage system and includes a webpage platform interface, and the first user can directly operate the webpage through an internet browser. Information processing platform.

Embodiment 14: The intelligent engineering information system as described in Embodiment 1, wherein the engineering project is a project related to construction engineering, construction engineering or public engineering.

Embodiment 15: The intelligent engineering information system as described in Embodiment 1, wherein the information processing platform comprises an image processing module, a building model module, a database module, and a management module.

Embodiment 16: The smart engineering information system of embodiment 1, wherein the information processing platform comprises one of a SketchUp module and a Revit module.

Embodiment 17: The smart engineering information system of embodiment 1, wherein the augmented reality platform is selected from the Unity platform.

Embodiment 18: The intelligent engineering information system as described in Embodiment 1, wherein the model information includes a building information model, a three-dimensional sketch of the building, a plan view, an elevation view, a section view, a detail view, a three-dimensional view, a perspective view, Material sheet, wiring diagram, piping diagram, electromechanical diagram, layer, building component properties, number of building components, supplier information, project progress, engineering model, engineering statistics, procurement information, latitude and longitude information, space coordinates information, location information, and positioning Capital One of them.

Embodiment 19: The smart engineering information system as described in Embodiment 1, wherein the live information includes voice information, recorded information, annotation information, text information, two-dimensional image information, latitude and longitude information, coordinate information, location information, and positioning information. one of them.

Embodiment 20: The smart engineering information system of embodiment 1, wherein the network is a wired network or a wireless network, wherein the wireless network is selected from the group consisting of a 3G network, a 4G network, a 5G network, and a blue network. One of the bud network and the Wi-Fi network.

Embodiment 21: The smart engineering information system of embodiment 1, wherein the remote device is one selected from the group consisting of a desktop computer, a notebook computer, a tablet computer, a smart phone, and a tablet phone.

Embodiment 22: The smart engineering information system of embodiment 1, wherein the handheld device is one selected from the group consisting of a handheld smart device, a tablet computer, a smart phone, and a tablet phone.

Embodiment 23: The smart engineering information system of embodiment 1, wherein the model information is a building information model and includes a digital three-dimensional model of the building.

Claims (23)

A smart engineering information system, which is applied to an engineering project, the engineering project includes a plurality of model information associated with a building and a plurality of on-site information, the building is located at a project site, and includes: an information server that stores the same Model information and such on-site information; a remote device connected to the information server via a network and including an information processing platform for providing a first user access, processing and processing the model information and the like Field information; and a handheld device connected to the information server through the network, capturing an instant image of the project site, and including an augmented reality platform, the augmented reality platform accessing the model information And combining with the real-time image to form an augmented reality image, and a second user records the live information with reference to the augmented reality image at the project site, and immediately uploads the information to the information server. The smart engineering information system of claim 1, wherein the first user implements an object image correspondence analysis program through the information processing platform, and the field information is corresponding to the model information. The smart engineering information system of claim 1, wherein the object image correspondence analysis program comprises one of geometric intersection analysis, one spatial resection analysis, and one collinear analysis. The smart engineering information system of claim 1, wherein the first user implements a material image production process through the information processing platform, and the one or two included in the on-site information Dimensional image information is orthorectified, normalized and tiled into a material image. The smart engineering information system of claim 1, wherein the material image production program comprises a shadow judgment analysis, an object capture analysis, an image orthorectification, a geometric correction, a standardization of a standard, and an image mosaic. Post one of them. The smart engineering information system of claim 1, wherein the first user performs processing on the remote device by operating the information processing platform to perform an object image analysis program and a material image production program. The on-site information is then applied to the model information. The smart engineering information system of claim 1, wherein the first user accesses the model information and the on-site information through the information processing platform for importing, exporting, storing, viewing, copying, Edit, modify, merge, overlay, tile, correct, capture, stitch, scale, segment, count, analyze, examine, audit, or manage information about such models and such on-site information. The smart engineering information system of claim 1, wherein the augmented reality platform accesses the model information to the information server to virtually display the augmented reality platform of the handheld device A digital three-dimensional model of the building. The smart engineering information system of claim 1, wherein the handheld device comprises a gyroscope and a geomagnetic meter, and the handheld device respectively positions the model information and the real image through the gyroscope and the geomagnet. After the correspondence between the model information and the real-time image is completed, the model information is combined with the real-time image to form the augmented reality image. The smart engineering information system of claim 1, wherein the second user is located at the project site, and the live image is recorded at the corresponding location according to the model information in the augmented reality image. Wait for live information and upload it to the news server instantly. The smart engineering information system of claim 1, wherein the first user is an engineer, a manager or a supervisor, and the second user is a checker or a field member. The smart engineering information system of claim 1, wherein the first user and the second user each have different rights, and the model information and the field information are graded for the first use of different rights. And access by the second user. The smart engineering information system of claim 1, wherein the information processing platform is provided through a webpage system and includes a webpage platform interface, and the first user can directly operate through an internet browser. The information processing platform. The smart engineering information system as described in item 1 of the claim, wherein the engineering project is a project relating to a construction project, a construction project or a public project. The smart engineering information system of claim 1, wherein the information processing platform comprises an image processing module, a building model module, a database module, and a management module. The smart engineering information system of claim 1, wherein the information processing platform comprises a One of the SketchUp modules and a Revit module. The smart engineering information system of claim 1, wherein the augmented reality platform is selected from a Unity platform. The smart engineering information system of claim 1, wherein the model information comprises an architectural information model, a three-dimensional schematic of a building, a plan, an elevation, a section, a detail, and a three-dimensional View, a perspective view, a material table, a wiring diagram, a piping diagram, an electromechanical diagram, a layer, a building component property, a number of building components, a supplier information, a project schedule, an engineering model, a project One of statistical data, one procurement information, one latitude and longitude information, one space coordinate information, one location information, and one location information. The smart engineering information system of claim 1, wherein the live information comprises a voice message, a recorded message, a note information, a text message, a two-dimensional image information, a latitude and longitude information, a landmark information, and a One of location information and a location information. The smart engineering information system of claim 1, wherein the network is a wired network or a wireless network, wherein the wireless network is selected from a 3G network, a 4G network, and a 5G network. One of the roads, a Bluetooth network, and a Wi-Fi network. The smart engineering information system of claim 1, wherein the remote device is selected from the group consisting of a desktop computer, a notebook computer, a tablet computer, a smart phone, and a tablet phone. The smart engineering information system of claim 1, wherein the handheld device is selected from the group consisting of a handheld smart device, a tablet computer, a smart phone, and a tablet phone. The smart engineering information system of claim 1, wherein the model information is a building information model and includes a digital three-dimensional model of the building.