CN115240140A

CN115240140A - Equipment installation progress monitoring method and system based on image recognition

Info

Publication number: CN115240140A
Application number: CN202210831579.0A
Authority: CN
Inventors: 秦旷宇; 钟彬; 史健勇; 曾远帆; 吕征宇; 周亮
Original assignee: Shanghai Jiaotong University; State Grid Shanghai Electric Power Co Ltd
Current assignee: State Grid Shanghai Electric Power Co Ltd
Priority date: 2022-07-14
Filing date: 2022-07-14
Publication date: 2022-10-25

Abstract

The invention provides an equipment installation progress monitoring method and system based on image recognition, wherein the method comprises the following steps: respectively constructing a BIM structure model and a BIM equipment model, and adding the BIM equipment model to a required position in the BIM structure model to obtain the BIM model; scanning a construction environment to obtain a construction environment image, and overlaying a BIM (building information modeling) model to the construction environment image to obtain a virtual-real combined scene image; and respectively acquiring a scene image and a construction environment image according to a set speed, identifying the position relation between the BIM equipment model in the scene image and the real equipment in the construction environment image, and automatically counting the equipment installation condition. According to the invention, real-time data acquisition is carried out on the engineering project without depending on specific equipment, the statistical equipment installation progress process can be automatically completed by the system, the working efficiency is improved, the visualization and information interaction of the engineering construction progress are realized, and the real-time monitoring on the installation progress of the engineering field equipment is further realized.

Description

Equipment installation progress monitoring method and system based on image recognition

Technical Field

The invention relates to the technical field of engineering progress management, in particular to an equipment installation progress monitoring method and system based on image recognition, and meanwhile provides corresponding equipment and a computer readable storage medium.

Background

Progress management is carried out on the capital construction project, the project is ensured to be completed as expected, and the method is an important management index concerned by construction units and construction units. In order to guarantee the project progress and achieve the project progress target, all units track and manage the progress during construction. The conventional method is to take the already-compiled cross-road map and network map plan as a baseline plan. And collecting data related to the progress as the project progresses, and updating the original plan. And judging the speed of the project progress through the comparison of the new plan and the baseline plan, and then adjusting the progress in a deviation rectifying mode. The conventional method for managing the progress of the engineering project has many advantages. But the biggest drawback is the lack of visualization and information interaction.

At present, no explanation or report of the similar technology of the invention is found, and similar data at home and abroad are not collected.

Disclosure of Invention

In view of the above-mentioned shortcomings in the prior art, the present invention provides a method and a system for monitoring the installation progress of a device based on image recognition, and also provides a corresponding device and a computer-readable storage medium.

The invention is realized by the following technical scheme.

According to one aspect of the invention, an equipment installation progress monitoring method based on image recognition is provided, and comprises the following steps:

respectively constructing a BIM structure model and a BIM equipment model, and adding the BIM equipment model to a required position in the BIM structure model to obtain the BIM model;

scanning a construction environment to obtain a construction environment image, and overlaying the BIM model to the construction environment image to obtain a virtual-real combined scene image;

and respectively acquiring the scene image and the construction environment image according to a set speed, identifying the position relation between the BIM equipment model in the scene image and the real equipment in the construction environment image, and automatically counting the equipment installation condition.

Optionally, the respectively constructing a BIM structure model and a BIM device model, and adding the BIM device model to a required position in the BIM structure model to obtain the BIM model includes:

according to CAD drawings, according to a model and construction site 1:1, respectively creating a BIM structure model and a BIM equipment model, and adding the BIM equipment model into the BIM structure model according to the placed real position to obtain a primary BIM model;

converting the preliminary BIM model into a three-dimensional model file, and respectively configuring the material of the BIM structure model and the material of the BIM equipment model to obtain a final BIM model; wherein:

the material of BIM structure model is fixed material, the material of BIM equipment model sets up according to equipment type, equipment model and the three dimension of equipment name difference.

Optionally, the scanning the construction environment to obtain a construction environment image, and overlaying the BIM model onto the construction environment image to obtain a virtual-real combined scene image, including:

scanning a construction site to obtain a construction environment image;

creating a punctuation on the BIM to obtain a virtual space coordinate, and obtaining a point corresponding to the punctuation in the construction environment image to obtain a construction environment space coordinate;

and registering the virtual space coordinates and the construction environment coordinates, so that the BIM is overlapped with the construction environment image to obtain a virtual-real combined scene image.

Optionally, the obtaining the scene image and the construction environment image respectively according to a set rate, identifying a position relationship between a BIM device model in the scene image and real devices in the construction environment image, and automatically counting device installation conditions includes:

respectively acquiring the scene picture and the construction site image according to a set rate, and identifying a BIM (building information modeling) equipment model in the scene picture and real equipment in the construction site image according to the scene picture and the construction site image which are in one-to-one correspondence according to a timestamp;

and judging whether the BIM equipment model and the real equipment are overlapped or not by utilizing the material of the BIM model:

if so, further judging whether the BIM equipment model and the real equipment are of the same equipment type, equipment model and equipment name, if so, marking the real equipment as installed equipment; if not, searching whether real equipment with the same equipment type, equipment model and equipment name as the BIM structural model exists in the material designated pixel point range, and if so, marking the real equipment as installed equipment;

otherwise, marking the real equipment as uninstalled equipment;

and completing automatic statistics of the equipment installation condition.

Optionally, the determining, by using the material of the BIM model, whether the BIM device model overlaps with the real device includes:

setting the BIM structural model as a semitransparent material; setting the BIM equipment model as different monochromatic materials according to three dimensions of equipment type, equipment model and equipment name;

acquiring all non-semitransparent materials in a current picture, and acquiring two-dimensional coordinates of a BIM equipment model in the current picture according to the corresponding relation between a preset monochromatic material and the BIM equipment model; and meanwhile, identifying real equipment in the current picture, obtaining two-dimensional coordinates of the real equipment, and judging whether the BIM equipment model is overlapped with the real equipment or not according to the two groups of coordinates.

Optionally, the searching whether the real device with the same device type, device model, and device name as the BIM structure model exists within the material-specified pixel point range includes:

acquiring length and width pixel values of the BIM structural model from the scene image, drawing a circle or a rectangle by taking x times of the pixel values as diameters, and acquiring a specified pixel point range, wherein x can be an adjustable parameter;

and when real equipment with the same equipment type, equipment model and equipment name as the BIM equipment model exists in the specified pixel point range, the real equipment is considered to be installed.

According to another aspect of the present invention, there is provided an image recognition-based device installation progress monitoring system, including:

the BIM model processing module is used for respectively constructing a BIM structure model and a BIM equipment model, and adding the BIM equipment model to a required position in the BIM structure model to obtain the BIM model;

the positioning module is used for scanning a construction environment to obtain a construction environment image, and the BIM model is superposed to the construction environment image to obtain a virtual-real combined scene image;

the image processing module is used for respectively acquiring the scene image and the construction environment image according to a set speed, identifying a BIM equipment model in the scene image and real equipment in the construction environment image, and associating the BIM equipment model with the real equipment in the construction environment image;

the progress recognition module is used for judging the position relation between the BIM equipment model in the scene image and the real equipment in the construction environment image, carrying out installation progress recognition and realizing automatic statistics on equipment installation conditions;

and the network module is used for being responsible for communication among the modules.

Optionally, the BIM model processing module includes:

a model construction unit for constructing, according to a CAD drawing, a model and job site 1:1, respectively creating a BIM structure model and a BIM equipment model, and adding the BIM equipment model into the BIM structure model according to the placed real position to obtain a primary BIM model;

the format conversion unit is used for converting the preliminary BIM into a three-dimensional model file;

the attribute exporting unit is used for exporting the model attributes of the preliminary BIM into a text document according to components, wherein the model attributes comprise model types, affiliated systems, models, names and size data, and can be used for subsequent material setting of the preliminary BIM;

and the material unit is used for endowing the preliminary BIM model with different materials according to three dimensions of equipment type, equipment model and equipment name to obtain a final BIM model.

Optionally, the positioning module includes:

the visual inertia SLAM unit is used for creating a punctuation on the BIM model, obtaining a virtual space coordinate, and obtaining a point corresponding to the punctuation in the construction environment image to obtain a construction environment space coordinate; registering the virtual space coordinates and the construction environment coordinates to ensure that equipment in the construction environment keeps stable when moving, and obtaining a virtual-real combined scene image;

and the coordinate input unit is used for outputting initial coordinates and providing an initial pose for the coordinate matching of the virtual space and the real space, and the initial pose is used for improving the coordinate matching accuracy.

Optionally, the image processing module includes:

the scene recording unit is used for respectively acquiring the scene picture and the construction site image according to a set rate;

the sensor data recording unit is used for recording each acquired scene picture and each acquired construction site image;

and the data processing unit is used for corresponding the scene pictures to the construction site images one by one according to time stamps and associating the BIM equipment model in the scene images with real equipment in the construction environment images.

Optionally, the progress identification module includes:

the image identification unit is used for identifying the type and the position of the BIM equipment model appearing in each frame of image and the type and the position of the real equipment appearing in each frame of image;

the image matching unit is used for judging whether the BIM equipment model is overlapped with the real equipment or not, if so, further judging whether the BIM equipment model and the real equipment are of the same equipment type, equipment model and equipment name or not, and if so, marking the real equipment as installed equipment; if not, searching whether real equipment with the same equipment type, equipment model and equipment name as the BIM structural model exists in a material designated pixel point range, and if so, marking the real equipment as installed equipment; otherwise, marking the real equipment as the uninstalled equipment.

Optionally, the system further comprises any one or more of the following modules:

-a logging module for storing data generated during operation of the system;

-a progress model viewing module for showing the progress of the installation of the device.

According to a third aspect of the present invention, there is provided an apparatus comprising: the system comprises a computer and a field scanning device connected with the computer; wherein:

the field scanning apparatus includes: the camera and the inertial sensor are used for scanning the construction environment, acquiring a construction environment image and sending the construction environment image to the computer;

the computer includes: a display, a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the program being operable to perform the method of any of the above, or to operate the system of any of the above.

Optionally, the field scanning apparatus may adopt a mobile intelligent device, for example: mobile phones, tablet computers, and the like.

According to four aspects of the invention, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, is operable to perform a method, or to run a system, as described in any of the above.

Due to the adoption of the technical scheme, compared with the prior art, the invention has the following beneficial effects:

the invention is independent of specific equipment, can realize field data scanning through most common marketable mobile intelligent equipment (such as a mobile phone, a tablet personal computer and the like), can perform all calculations on a server (computer) in the operation process, can provide required field image data and inertial sensor data by the intelligent equipment, and reduces the requirement on the equipment.

The statistical equipment installation progress process can be automatically completed by the system, and the working efficiency is improved.

The invention is particularly directed at the equipment installation link in the engineering construction, can carry out visual archiving and automatic statistics of the installation progress in the equipment installation process, and realizes the visualization and information interaction of the engineering construction progress.

The invention realizes the field data acquisition through the mobile intelligent device with the high-definition camera and other devices supporting the network, can acquire real-time data of the engineering project, obtains the installation progress of the engineering field device and further realizes the real-time monitoring of the installation progress of the engineering field device.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a flowchart illustrating an overall method for monitoring an installation progress of a device based on image recognition according to an embodiment of the present invention;

FIG. 2 is a functional block diagram of an apparatus installation progress monitoring system based on image recognition according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating an overall method for monitoring the installation progress of a device based on image recognition according to a preferred embodiment of the present invention;

FIG. 4 is a functional block diagram of an installation progress monitoring system for an apparatus based on image recognition according to a preferred embodiment of the present invention;

FIG. 5 is a schematic diagram of an initial interface in an embodiment of the present invention;

FIG. 6 is a schematic diagram of a scene data collection interface in an exemplary embodiment of the present invention; wherein, (a) is a BIM model interface, and (b) is a construction environment image interface;

fig. 7 is a schematic diagram of an apparatus identification process in an embodiment of the present invention.

Detailed Description

The following examples illustrate the invention in detail: the embodiment is implemented on the premise of the technical scheme of the invention, and gives a detailed implementation mode and a specific operation process. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.

An embodiment of the invention provides an equipment installation progress monitoring method based on image recognition.

As shown in fig. 1, the method for monitoring the installation progress of a device based on image recognition according to this embodiment may include the following steps:

s1, respectively constructing a BIM structure model and a BIM equipment model, and adding the BIM equipment model to a required position in the BIM structure model to obtain the BIM model;

s2, scanning a construction environment to obtain a construction environment image, and overlaying the BIM into the construction environment image to obtain a scene image combining virtuality and reality;

and S3, respectively acquiring the scene image and the construction environment image according to a set speed, identifying the position relation between the BIM equipment model in the scene image and the real equipment in the construction environment image, and automatically counting the equipment installation condition.

In a preferred embodiment of S1, respectively constructing a BIM structure model and a BIM device model, and adding the BIM device model to a desired position in the BIM structure model to obtain the BIM model, may include the following steps:

s101, according to a CAD drawing, according to a model and construction site 1:1, respectively creating a BIM structure model and a BIM equipment model, and adding the BIM equipment model into the BIM structure model according to the placed real position to obtain a primary BIM model;

s102, converting the preliminary BIM model into a three-dimensional model file, and respectively configuring the material of the BIM structure model and the BIM equipment model to obtain a final BIM model; in a specific application example, the three-dimensional model file can be an FBX file or an OBJ file; wherein:

the BIM structure model is made of fixed materials, and the BIM equipment model is set differently according to three dimensions of equipment type, equipment model and equipment name. In a specific application example, the material of the BIM structural model can be selected from a semitransparent material for adding; BIM equipment model materials are distinguished according to equipment types, different equipment of the same type also need to be distinguished, and monochromatic materials can be selectively added.

In a preferred embodiment of S2, scanning the construction environment to obtain a construction environment image, and superimposing the BIM model onto the construction environment image to obtain a virtual-real combined scene image, may include the following steps:

s201, scanning a construction site to obtain a construction environment image;

s202, creating a punctuation on the BIM to obtain a virtual space coordinate, and obtaining a point corresponding to the punctuation in the construction environment image to obtain a construction environment space coordinate;

and S203, registering the virtual space coordinates and the construction environment coordinates, so that the BIM model is overlapped with the construction environment image to obtain a virtual-real combined scene image.

In a preferred embodiment of S3, the method respectively acquires a scene image and a construction environment image according to a set rate, identifies a positional relationship between a BIM device model in the scene image and a real device in the construction environment image, and automatically counts a device installation condition, and may include the following steps:

s301, respectively acquiring a scene picture and a construction site image according to a set rate, corresponding the scene picture and the construction site image one by one according to a timestamp, and identifying a BIM (building information modeling) equipment model in the scene picture and real equipment in the construction site image;

s302, judging whether the BIM equipment model and the real equipment are overlapped or not by using the material of the BIM model:

if the equipment type, the equipment model and the real equipment are the same, further judging whether the BIM equipment model and the real equipment are the same, and if so, marking the real equipment as the installed equipment; if not, searching whether real equipment with the same equipment type, equipment model and equipment name as the BIM structural model exists in the material designated pixel point range, and if so, marking the real equipment as installed equipment;

otherwise, marking the real equipment as the uninstalled equipment;

and S303, completing automatic statistics of the equipment installation condition.

In a preferred embodiment of S302, determining whether the BIM device model overlaps with the real device by using a material of the BIM model includes:

setting a BIM structural model as a semitransparent material; setting BIM equipment models to be different monochromatic materials according to three dimensions of equipment types, equipment models and equipment names;

acquiring all non-translucent materials in a current picture, and acquiring two-dimensional coordinates of a BIM equipment model in the current picture according to a preset corresponding relation between a monochromatic material and the BIM equipment model; meanwhile, real equipment in the current picture is identified, two-dimensional coordinates of the real equipment are obtained, and whether the BIM equipment model and the real equipment are overlapped or not is judged according to the two groups of coordinates.

In a preferred embodiment of S302, searching whether a real device in the same device type, device model, and device name as the material-specified pixel point range of the BIM structure model exists may include the following steps:

s3021, acquiring length and width pixel values of the BIM structural model from a scene image, drawing a circle or a rectangle by taking x times of the pixel values as diameters, and acquiring a specified pixel point range, wherein x can be an adjustable parameter;

and S3022, when the real equipment with the same equipment type, equipment model and equipment name as the BIM equipment model exists in the specified pixel point range, determining that the real equipment is installed.

In some embodiments of the invention:

and scanning the construction environment space by adopting equipment with a camera and an inertial sensor, and displaying the BIM structural model and the BIM equipment model in a scanning image of the construction environment space in an overlapping manner. The superimposed display may be displayed by a display.

Creating a BIM model requires the following:

the BIM is converted into a three-dimensional model file (for example, a file with a format such as FBX, OBJ and the like) according to a BIM structure model and a BIM equipment model which are modeled by a CAD drawing, and the material of the BIM structure model and the material of the BIM equipment model are replaced by different types of materials. After format conversion and material replacement, the material is imported into a development tool (e.g., U3D software). And the material replacing operation can be carried out in the development software.

The BIM structure model is fixed in material, for example, a semitransparent material can be added, the BIM equipment model material is distinguished according to the equipment type, different equipment of the same type (which can be distinguished by the equipment name) also needs to be distinguished, and a monochromatic material can be added.

Adopt the equipment of taking camera and inertial sensor, scan construction environment space, with BIM structural model and BIM equipment model together, the stack shows in construction environment space's scanning image, specifically can be:

and creating punctuation points on the BIM model, and performing coordinate registration with points in the construction environment, so that the virtual space coordinates are overlapped with the construction environment space coordinates.

When the virtual space coordinates are overlapped with the construction environment space coordinates, a scanning image and a screen picture (namely a scene image) of the construction environment are collected, the images are stored at a set speed (for example, a speed of 30 frames per second), and the equipment installation progress recognition is carried out. Further, a video (for obtaining a scanned image of a construction environment) and a recording screen (for obtaining a scene image) are photographed at the same frame rate; acquiring equipment data in the video through image identification, wherein the equipment data comprises equipment types, equipment models, equipment names, identification accuracy, time, positions and camera Transform attributes; and acquiring the equipment data of the BIM equipment model in the recording screen, and obtaining the equipment position information through the material color numerical value and the range.

Through the function of parameters of image recognition in the past, the type and the model of equipment are used for marking the equipment in the result, the recognition accuracy is used for judging whether the image recognition is correct (the accuracy is greater than a specified value and is regarded as accurate), and the three parameters of time, position and camera Transform attribute are used for confirming whether virtual data and real data are synchronous.

Monochrome blocks representing different devices exist in image data obtained by recording a screen, and specific RGB values of the monochrome blocks and two-dimensional coordinates in a current picture can be obtained by inquiring GPU rendering process data.

By reading each frame of image, the device installation condition is automatically counted, and the counting mode is as follows:

and identifying whether the monochromatic material of the BIM equipment model in the scene image is overlapped with the real equipment in the identified construction environment image, if so, further judging whether the two kinds of equipment are of the same type and are the same type, and if so, marking the real equipment as the installed equipment. If not, searching whether the semi-transparent material is in the designated pixel point range and has the same type and the same type of equipment (namely the equipment type, the equipment model and the equipment name are all the same), if so, marking the real equipment as the installed equipment, otherwise, marking the real equipment as the uninstalled equipment. And finally obtaining the installation progress of the equipment by comparing the equipment data acquired from the video with the equipment data acquired from the recording screen.

Searching the range of the appointed pixel points of the semitransparent material specifically comprises the following steps:

acquiring the length and width pixel values of a BIM structural model from a scene image, and drawing a circle or a rectangle by taking x times of the pixel values as the diameters; x may be an adjustable parameter.

And when the real equipment with the same equipment type, the same equipment model and the same equipment name as the BIM equipment model exists in the specified pixel point range, the installed real equipment is considered.

The process gives an allowable deviation value and when the difference in the position of the equipment is less than the allowable deviation value, the equipment is considered to have been installed.

An embodiment of the invention provides an equipment installation progress monitoring system based on image recognition.

As shown in fig. 2, the system for monitoring the installation progress of a device based on image recognition according to this embodiment may include the following modules:

the positioning module is used for scanning the construction environment to obtain a construction environment image, and the BIM model is superposed to the construction environment image to obtain a virtual-real combined scene image;

the image processing module is used for respectively acquiring a scene image and a construction environment image according to a set speed, identifying a BIM equipment model in the scene image and real equipment in the construction environment image, and associating the BIM equipment model with the real equipment in the construction environment image;

the progress recognition module is used for judging the position relation between the BIM equipment model in the scene image and the real equipment in the construction environment image, carrying out installation progress recognition and realizing automatic statistics on the equipment installation condition;

and the network module is used for being responsible for communication among the modules, so that data interaction work is smoothly carried out.

In a preferred embodiment, the BIM model processing module may include the following units:

a model construction unit for, according to a CAD drawing, constructing a model of the construction site 1:1, respectively creating a BIM structural model and a BIM equipment model, and adding the BIM equipment model into the BIM structural model according to the placed real position to obtain a preliminary BIM model; in a specific application example, the model building unit can be existing three-dimensional modeling software or a plug-in of the modeling software;

the format conversion unit is used for converting the preliminary BIM model into three-dimensional model files in FBX, OBJ and other formats;

an attribute deriving unit for deriving the model attribute as a text document by component; in a specific application example, the model attribute includes a model type (device type), an affiliated system, a model type (device model), a model name (device name), and size data, and can be used for subsequent setting of a preliminary BIM model material;

Further, there are multiple shaders in a texture unit, each of which can control one or more textures.

In a preferred embodiment, the positioning module may include the following units:

the visual inertia SLAM unit is used for creating a punctuation on the BIM model, obtaining a virtual space coordinate, and obtaining a point corresponding to the punctuation in the construction environment image to obtain a construction environment space coordinate; registering the virtual space coordinates and the construction environment coordinates to ensure that equipment in the construction environment keeps stable when moving, and obtaining a scene image combining virtuality and reality;

and the coordinate input unit is used for outputting initial coordinates and providing an initial pose for matching the virtual space and the real space, and the initial pose can be used for improving the accuracy of coordinate matching.

In a preferred embodiment, the image processing module may include the following units:

the scene recording unit is used for respectively acquiring a scene picture and a construction site image according to a set rate;

the sensor data recording unit is used for recording each frame of scene picture and construction site image; in a specific application example, when each frame of image is recorded, synchronously recording data of real equipment in the construction site image at the time, wherein the data comprises acceleration, angular velocity and the like;

the data processing unit is used for corresponding the scene images and the construction site images one by one according to the timestamps and associating the BIM equipment model in the scene images with real equipment in the construction environment images; in a specific application example, the pose of the real equipment when each frame of image is acquired can be calculated according to the data (acceleration and angular velocity) of the real equipment in the construction site image.

Furthermore, the real pose is calculated in real time to obtain the moving track of the equipment, and the moving track is synchronized with the camera in the BIM model, so that the stable fusion of the BIM model and the construction environment can be realized.

In a preferred embodiment, the progress identification module may include the following units:

the image matching unit is used for judging whether the BIM equipment model and the real equipment are overlapped, if so, further judging whether the BIM equipment model and the real equipment are the same equipment type, equipment model and equipment name, and if so, marking the real equipment as the installed equipment; if not, searching whether real equipment with the same equipment type, equipment model and equipment name as the BIM structural model exists in a material designated pixel point range, and if so, marking the real equipment as installed equipment; otherwise, the real device is marked as an uninstalled device.

In a preferred embodiment, the system further comprises any one or more of the following modules:

-a recording module for storing a series of data generated during the operation of the system;

It should be noted that, the steps in the method provided by the present invention may be implemented by using corresponding modules, devices, units, and the like in the system, and those skilled in the art may implement the composition of the system by referring to the technical solution of the method, that is, the embodiment in the method may be understood as a preferred example for constructing the system, and will not be described herein again.

An embodiment of the present invention provides an apparatus, which may include: the system comprises a computer and a field scanning device connected with the computer; wherein:

a field scanning apparatus comprising: the camera and the inertial sensor are used for scanning the construction environment, acquiring a construction environment image and sending the construction environment image to the computer;

the computer includes: a display, a memory, a processor and a computer program stored in the memory and executable on the processor, the processor being operable to perform the method of any one of the above or to operate the system of any one of the above when the processor executes the program.

In a specific application example, the field scanning apparatus may employ a mobile intelligent device, such as: the mobile phone and tablet computer which comprise the camera and the inertial sensor can also adopt the portable equipment such as a notebook computer externally connected with the camera with the inertial sensor.

An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, is operable to perform any of the methods described above, or to run any of the systems described above.

Optionally, a memory for storing a program; a Memory, which may include a volatile Memory (RAM), such as a Random Access Memory (SRAM), a Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), and the like; the memory may also comprise a non-volatile memory, such as a flash memory. The memories are used to store computer programs (e.g., applications, functional modules, etc. that implement the above-described methods), computer instructions, etc., which may be stored in partition on the memory or memories. And the computer programs, computer instructions, data, etc. described above may be invoked by a processor.

The computer programs, computer instructions, etc. described above may be stored in partitions in one or more memories. And the computer programs, computer instructions, data, etc. described above may be invoked by a processor.

A processor for executing the computer program stored in the memory to implement the steps of the method according to the above embodiments. Reference may be made in particular to the description relating to the previous method embodiments.

The processor and the memory may be separate structures or may be an integrated structure integrated together. When the processor and the memory are separate structures, the memory, the processor may be coupled by a bus.

In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be further clearly and completely described below with reference to a specific application example and the accompanying drawings.

In the following specific application examples, the adopted device with the display screen, the camera and the inertial sensor may be a smart phone or a tablet computer including these hardware, or a tablet computer or a notebook computer externally connected with a camera with an inertial sensor. Depending on the device with the display screen, the camera and the inertial sensor, the function of superimposing the virtual model on the real model in the following specific application examples has a hardware basis for implementation.

The specific application example provides a method for monitoring the installation progress of the equipment based on image recognition for explanation. As shown in fig. 3, the method for monitoring the installation progress of the device based on image recognition provided by this specific application example includes the following steps:

step 100: and respectively creating a BIM structure model and a BIM equipment model according to the CAD drawings by using modeling software. And importing the model into a development tool, such as U3D and UE4, and adding different materials for different types of models.

Step 200: and matching the virtual space coordinates with the real space through a positioning function, and then displaying the BIM model in a screen, namely realizing the virtual-real fusion.

Step 300: the method comprises the steps of obtaining a scene image, synchronously reading data of inertial sensors, and corresponding the image and the data of the sensors one by one according to time stamps.

Step 400: and acquiring the BIM equipment model in the picture and the equipment information of the construction site frame by frame to obtain the equipment installation information.

Wherein, step 100 corresponds to a BIM model processing module, step 200 corresponds to a positioning module, step 300 corresponds to an image processing module, and step 400 corresponds to a progress recognition module. The network module is responsible for information transmission interaction when different modules are installed on different devices.

In step 100, revit software is used as modeling software, U3D software is used as a development tool, and C # is used as a programming language. Part of the model import tool is developed secondarily based on Revit software, and the main function is to export information into a text document component by component, wherein the text document is named according to component Family type Family and component number Element-ID. The model import tool is based on U3D development, the main function is to associate the property text document file with the model FBX.

Further, different materials are added to different types of models, specifically: adding M _ columns made of the same material for all structural column models, adding M _ outer walls made of the material for outer walls, adding M _ nozzles made of the material for fire-fighting nozzles, adding M _ PT50KV of 50KV transformers, adding M _ PYDuct to smoke exhaust pipelines and the like. All materials are pure colors, and different materials are distinguished by different RGB values, so that the subsequent image recognition process is facilitated. All materials are colored by two passes, the first Pass only renders the back, and Cull is set as Front; the second Pass renders only the front, with fill set to Back.

In step 200, matching the virtual space coordinates with the real space, specifically implementing the process as follows: two points are selected in the BIM model top view, and the coordinates of the two points in the model space, which are plane coordinates (the vertical direction value is 0), are temporarily recorded. Then, a camera is started, the ground plane of the construction site is identified, and then two anchor points are added at corresponding positions. And overlapping two coordinates in the model space with two virtual punctuations on the ground plane of the construction site, namely realizing the matching of the virtual space coordinates and the real space.

Further, the method for identifying the ground plane of the construction site is operated by, for example, univeyengine, xr, ARFoundation, arpanenschangedeventargs in an ARFoundation plug-in U3D. And identifying the operation of adding the anchor point after the construction site plane is identified, and then adopting UnityEngine. As for the matching mode of the anchor point and the plane anchor point of the virtual space, the specific process is as follows:

two point coordinates X1 and X2 in a virtual space are assumed; anchor point coordinates A1 and A2, virtual space coordinate system E, real space coordinate system E, translation vector T and rotation vector R, then

Calculating T, R values such that E = Re + T;

wherein

Further, after the virtual space coordinates are matched with the real space coordinates, the BIM model needs to be stably superimposed on the real space as the device moves, and the process is implemented by using the SLAM technology. In the field of robotics, this technique is defined as simultaneous localization and mapping (SLAM), which can be described as: the robot starts to move from an unknown position in an unknown environment, self-positioning is carried out according to position estimation and a map in the moving process, and meanwhile an incremental map is built on the basis of self-positioning, so that autonomous positioning and navigation of the robot are realized. For a SLAM technology, there are mainly three solutions, depending on the sensors used: SLAM based on RGB-D cameras, SLAM based on laser point clouds, and SLAM based on vision. The vision-based SLAM is called a Visual-Inertial odometer (VIO) if it incorporates a VIO, and may be specifically classified into a monocular camera, a binocular camera, and an RGB-D camera. The three implementation schemes have advantages and disadvantages respectively:

the VIO-SLAM scheme based on the monocular camera is low in cost, but the space scale is uncertain, and the problem of initialization is solved;

the VIO-SLAM scheme of the binocular camera can acquire space scale, but the configuration is complex and the cost is high;

the VIO-SLAM scheme of the RGB-D camera can also obtain the space scale, has low cost and is easily interfered by illumination.

For the specific application example, both the monocular camera scheme and the RGB-D scheme may be adopted according to different field situations, and the monocular scheme is adopted in the specific application example.

In step 300, acquiring a screen image, synchronously reading inertial sensor data, and corresponding the image and the sensor data one to one according to a timestamp, specifically including:

the screen recording function is implemented using a unityngine.recorder, the recording frame rate is 30, and the screen recording data set is: recorderSet30. Acquiring camera authority by using unity.application.requestuserrauthorization (), and recording a camera data set as: webCamTexturSet30; ACCELERATION sensor data is acquired using a sensor, type _ acceerometer, a direction sensor, type _ ORIENTATION, a GYROSCOPE sensor, type _ gyro, a LINEAR ACCELERATION sensor, type _ LINEAR _ ACCELERATION, a GRAVITY sensor, type _ GRAVITY, totaling VIOSET. Timestamp was obtained using system. TimeSET. All data are packed into a data set DATASET according to 30 groups per second, and then are sent to the progress identification module through the network module.

Further, the inertial sensor data set may be used for camera Transform verification, not necessarily.

Further, when the screen recording function is running, the transparency of the structural model material in the BIM model can be set to 0.

In step 400, the BIM device model in the picture and the device information of the construction site are obtained frame by frame to obtain the device installation information, which specifically includes: obtaining a set of DATASET, first identifying the material color block of the BICM device model in the RecorderSet30, and obtaining the device type, the number, and the relative pixel coordinate C in the picture _BIM . The type of device present in WebCamTexturSet30 is then identified, along with the relative pixel coordinates C _R And counting the number thereof.

Further, in this specific application example, in order to implement the image recognition function, the image recognition database paddlepaddlele is referred to. The data sets are collected from equipment actually used in the construction project.

Further, the image recognition result is recorded as serialized information, and multiple pieces of device data are obtained from each image, including the device type, the recognition accuracy, the Time, the position (relative coordinates in the image), the Transform attribute of the camera at that Time, and m (family _R Transform); and intercepting 20 pieces of data before and after the maximum lambda value as a progress monitoring data set and recording the data as mSet (family]In the case of less than 20, the maximum value is obtained.

Further, screen recording data, specifically, a time period image set corresponding to the mSet [ family.

Further, the type and position of the device in the screen recording data are obtained by the specific process:

searching the material of the target equipment in the picture, specifically, color parameters set in advance, obtaining all RGB value pixel points to obtain boundary numerical values, then calculating a midpoint according to the boundary numerical values, wherein a midpoint coordinate is the equipment position C _BIM 。

Further, in the actual construction operation process, the installation position of the equipment is usually slightly deviated from the drawing due to various reasons, so that when the installation progress of the equipment is identified, an allowable deviation value D is given, and the requirements are met:

D≥|C _BIM -C _R |

namely, the device is considered to be installed at the installation position C _R 。

Where D = F (X, Y, L), X and Y are the dimensions of the device in the image, and L is the BIM device model distance from the camera, which can be calculated from m.transform and mSet [ family.name ]. Transform.

And (4) after the circulation operation, all the equipment is detected to one side, and finally the installation progress information of the construction project equipment is obtained. All the calculation results generated in the above steps are stored in the recording module.

The equipment installation condition can be checked through the progress model checking module.

Specifically, in this specific application example, seven functional modules are defined: the system comprises a BIM model processing module, a positioning module, an image processing module, a progress recognition module, a network module, a recording module and a progress model viewing module. The BIM model processing module builds a model on the Revit platform and provides model information to be exported to FBX, the model attribute information is exported to be text stable, the FBX is loaded on the Unity platform, and the material of the model is set. The positioning module is used for processing the matching of the virtual space coordinates and the real space, so that the BIM model can be displayed in a construction scene in an overlapping mode. The image processing modules are divided into three functional groups: the scene recording unit acquires a camera image and a screen image, the sensor data recording unit and the data processing unit and packs a plurality of data sets according to each frame. And the progress identification module is used for respectively identifying the equipment information in the camera data set and the screen recording data set and comparing the equipment information to obtain equipment installation information. The network module is responsible for communication between the various modules. The recording module is responsible for storing all data generated in the operation process of the system. And the progress model checking module is responsible for displaying the monitoring result to the user.

In some examples, the communication between the modules may adopt a mobile phone as a front-end acquisition device, and a cloud server as a back-end processing device: except the positioning module and the image processing module, the other three modules are all arranged in the back-end server.

In some examples, when the standalone monocular camera + tablet computer is used as hardware, the rest of the hardware is disposed in the backend server except for the scene recording unit of the image processing module.

The following describes the composition of the system for monitoring installation of equipment according to this embodiment.

As shown in fig. 4, the system for monitoring equipment installation provided in this embodiment can be used to execute the method for monitoring equipment installation progress based on image recognition, which includes the following modules:

and the BIM processing module 1 is used for converting the format of the model, modifying the material of the model and counting the information of the model.

And the positioning module 2 is used for matching the coordinates of the virtual space and the real space to realize the virtual-real fusion effect.

And the image processing module 3 is used for acquiring a screen image, acquiring inertial sensor data of the equipment and associating the inertial sensor data with the image.

And the progress recognition module 4 is used for recognizing whether the equipment model is installed in the construction scene.

And the network module 5 is responsible for communication among the modules.

And the recording module 6 is responsible for all data generated in the system operation process.

And the progress model viewing module 7 is responsible for displaying the installation progress of the equipment.

Further, the BIM model processing module 1 includes:

the model building unit 10 is used for building a BIM structure model and a BIM equipment model and adding the BIM equipment model into the BIM structure model according to the placed real position;

and the format conversion unit 11 is used for processing the BIM into formats such as FBX, OBJ and the like.

The attribute deriving unit 12 derives the model attribute as a text document on a component-by-component basis.

And a material unit 13 for assigning the model to the unused material according to the type.

Further, the positioning module 2 is characterized by comprising:

and the visual inertia SLAM unit 21 is used for acquiring the position of the equipment in the space, matching the coordinates of the virtual space and the real space and keeping the equipment stable when the equipment moves.

And the coordinate input unit 22 is used for outputting initial coordinates and providing an initial pose for matching the virtual space and the real space.

Further, the image processing module 3 includes:

and after the virtual space coordinates are matched with the real space coordinates, the scene recording unit 31 records images of the BIM model which are superposed and displayed on the real scene, and meanwhile, the camera records scene images without the BIM model.

The sensor data recording unit 32 synchronously records inertial sensor data including acceleration, angular velocity, and the like at the time when each frame image is recorded.

And the data processing unit 33 calculates the pose of the device when each frame of image is acquired according to the inertial sensor data.

Further, the progress identification module 4 includes:

and an image identification unit 41, configured to identify the type and location of the BIM device model appearing in each frame of image, and the type and location of the real device model appearing in each frame of image.

And the image matching unit 42 is used for calculating whether the homogeneous models from different sources are present in the specified pixel range.

Further, the homogeneous models from different sources refer to: a BIM equipment model and a real-existing equipment model of a construction site. ,

the following further describes the method and system for monitoring the installation progress of the device provided in the specific application example, using an interface during actual operation.

As shown in fig. 5, a specific example of the practical runtime use interface corresponding to the method and the system for monitoring the installation progress of the device in the specific application example is described as follows:

fig. 5 is an executed initial interface which is also a scene selection interface, in this specific application example, a software interface is simplified to the greatest extent, and only functions related to the technical solution in this specific application example are reserved. Fig. 6 is a scene data collection interface, and fig. 7 is an example of device identification.

The application mode is as follows:

(1) When entering a new construction environment, firstly selecting a BIM structure model corresponding to the construction environment on a selection PROJECT interface, and then creating two positioning points on a model top view. The positioning points preferably select four corner points of the pillar, so that the subsequent registration accuracy is improved. And finally, clicking a left button to enter a scene data acquisition interface.

(2) After entering a scene acquisition interface, the mobile device acquires ground information as much as possible to establish a working plane. And then clicking a button to create two anchor points, wherein the anchor points are the corresponding positions of the positioning points in the BIM model on the construction site. Clicking the button "register zoom" or "register no zoom" will superimpose the BIM model on the job site. Further, two buttons do not affect the implementation of the functions of the present invention, and the identification of the BIM device is based on color blocks.

(3) After the models are overlapped, a 'collecting' button is clicked, a user starts to slowly move in a construction scene with the equipment, and the site is comprehensively collected. Take fire control shower nozzle as an example: fig. 7 (a) shows a camera capture screen, (b) shows a screen display screen, and (c) shows an example of device identification.

It should be noted that, the specific interface interaction may be adjusted according to the user preference and the actual engineering condition, and this embodiment is only an example of the method demonstration and is not used to limit the present invention.

In this embodiment, the method and system for monitoring the installation progress of the device can be implemented by a computer-readable storage medium storing computer code, and when the computer code is executed, the method is executed. Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disks, and the like.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (P RAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium (tr index media), such as a modulated data signal and a carrier wave.

It should be noted that the software program can be executed by a processor to implement the above steps or functions. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Additionally, some of the steps or functionality of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various functions or steps. The method disclosed by the embodiment shown in the embodiment of the present specification can be applied to or realized by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of this specification may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present specification may be embodied directly in a hardware decoding processor, or in a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

The systems, devices, modules or units according to the above embodiments of the present invention may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices. Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRA M), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically erasable programmable read only memory (EEP ROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory computer readable media (trans entity me d a), such as modulated data signals and carrier waves. It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on fixed or removable recording media and/or transmitted via a data stream in a broadcast or other signal bearing medium and/or stored in a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention herein comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.

The method and the system for monitoring the installation progress of the equipment based on image recognition provided by the embodiment of the invention use the equipment with the camera and the inertial sensor to scan the construction environment space, superpose and display the BIM model in the construction environment, synchronously acquire the data of the camera and the screen image, automatically judge the installation condition of the equipment through the image recognition system, realize the field data scanning through most common mobile intelligent equipment (such as a mobile phone, a tablet personal computer and the like) in a market, perform all calculations on a server (a computer) in the operation process, provide required field image data and inertial sensor data by the intelligent equipment, reduce the requirement on the equipment, automatically complete the equipment installation progress process through the system, improve the working efficiency, particularly aim at the equipment installation link in engineering construction, perform visual archiving and automatic statistics in the equipment installation process, realize the visualization and information interaction of the engineering construction progress, realize the field data acquisition through the mobile intelligent equipment with the camera and other equipment supporting a network, obtain the real-time data of the engineering project, further obtain the installation progress of the engineering, and further realize the real-time monitoring of the installation progress of the engineering.

The above embodiments of the present invention are not exhaustive of the techniques known in the art.

The foregoing description has described specific embodiments of the present invention. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.

Claims

1. An equipment installation progress monitoring method based on image recognition is characterized by comprising the following steps:

respectively constructing a BIM structure model and a BIM equipment model, and adding the BIM equipment model to a required position in the BIM structure model to obtain a BIM model;

2. The method for monitoring the equipment installation progress based on image recognition according to claim 1, wherein the steps of respectively constructing a BIM structure model and a BIM equipment model, and adding the BIM equipment model to a required position in the BIM structure model to obtain the BIM model comprise:

according to a CAD drawing, according to a proportion of a model to a construction site 1, respectively creating a BIM structure model and a BIM equipment model, and adding the BIM equipment model to the BIM structure model according to a placed real position to obtain a preliminary BIM model;

the material of BIM structure model is fixed material, the material of BIM equipment model distinguishes the setting according to the three dimensions of equipment type, equipment model and equipment name.

3. The method for monitoring the equipment installation progress based on image recognition according to claim 1, wherein the step of scanning a construction environment to obtain a construction environment image, and the step of superimposing the BIM model on the construction environment image to obtain a virtual-real combined scene image comprises the steps of:

scanning a construction site to obtain a construction environment image;

creating punctuations on the BIM to obtain virtual space coordinates, and obtaining points corresponding to the punctuations in the construction environment image to obtain construction environment space coordinates;

and registering the virtual space coordinates and the construction environment coordinates, so that the BIM model is overlapped with the construction environment image to obtain a virtual-real combined scene image.

4. The method for monitoring the equipment installation progress based on image recognition according to claim 1, wherein the steps of respectively acquiring the scene image and the construction environment image according to a set rate, recognizing a position relationship between a BIM equipment model in the scene image and real equipment in the construction environment image, and automatically counting equipment installation conditions comprise:

if the equipment type, the equipment model and the real equipment are overlapped, further judging whether the BIM equipment model and the real equipment are the same equipment type, equipment model and equipment name, and if so, marking the real equipment as installed equipment; if not, searching whether real equipment with the same equipment type, equipment model and equipment name as the BIM structural model exists in a material designated pixel point range, and if so, marking the real equipment as installed equipment;

otherwise, marking the real equipment as uninstalled equipment;

and completing automatic statistics of the equipment installation condition.

5. The image recognition-based equipment installation progress monitoring method according to claim 4, further comprising any one or more of the following:

-said determining whether the BIM device model overlaps with the real device by using the material of the BIM model comprises:

acquiring all non-semitransparent materials in a current picture, and acquiring two-dimensional coordinates of a BIM equipment model in the current picture according to the corresponding relation between a preset monochromatic material and the BIM equipment model; meanwhile, identifying real equipment in the current picture, obtaining two-dimensional coordinates of the real equipment, and judging whether the BIM equipment model is overlapped with the real equipment or not according to the two groups of coordinates;

-said searching whether there is a real device of the same device type, device model and device name as the BIM structure model within the range of the material-specified pixel points, comprising:

6. An equipment installation progress monitoring system based on image recognition is characterized by comprising:

7. The image recognition-based equipment installation progress monitoring system of claim 6, further comprising any one or more of the following:

-the BIM model processing module comprising:

the model building unit is used for respectively building a BIM (building information model) structure model and a BIM equipment model according to a CAD (computer-aided design) drawing and a proportion of the model to a construction site 1, and adding the BIM equipment model to the BIM structure model according to a placed real position to obtain a preliminary BIM model;

the attribute exporting unit is used for exporting the model attributes of the preliminary BIM into a text document according to components for subsequent material setting of the preliminary BIM;

the material unit is used for endowing the preliminary BIM model with different materials according to three dimensions of equipment type, equipment model and equipment name to obtain a final BIM model;

-the positioning module comprising:

the coordinate input unit is used for outputting initial coordinates and providing an initial pose for the coordinate matching of a virtual space and a real space;

-the image processing module comprising:

the scene recording unit is used for respectively acquiring the scene picture and the construction site image according to a set speed;

the data processing unit is used for corresponding the scene images and the construction site images one by one according to timestamps and associating the BIM equipment model in the scene images with real equipment in the construction environment images;

-said progress identification module comprising:

the image matching unit is used for judging whether the BIM equipment model and the real equipment are overlapped, if so, further judging whether the BIM equipment model and the real equipment are the same equipment type, equipment model and equipment name, and if so, marking the real equipment as installed equipment; if not, searching whether real equipment with the same equipment type, equipment model and equipment name as the BIM structural model exists in the material designated pixel point range, and if so, marking the real equipment as installed equipment; otherwise, marking the real equipment as the uninstalled equipment.

8. The image recognition-based equipment installation progress monitoring system according to claim 6 or 7, further comprising any one or more of the following modules:

-a logging module for storing data generated during operation of the system;

9. An apparatus, comprising: the system comprises a computer and a field scanning device connected with the computer; wherein:

the field scanning device comprises: the camera and the inertial sensor are used for scanning the construction environment, acquiring a construction environment image and sending the construction environment image to the computer;

the computer includes: a display, a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the program being operable to perform the method of any one of claims 1 to 5 or to execute the system of any one of claims 6 to 8.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 5 or to carry out the system of any one of claims 6 to 8.