WO2019137814A1

WO2019137814A1 - Method for monitoring a construction site

Info

Publication number: WO2019137814A1
Application number: PCT/EP2018/097042
Authority: WO
Inventors: Darno Alexander KETTERER; Andrew Allen; Kai Liu; Kai Bohne; Udo Schulz; Achim Brenk; Heiko KLEINEDER; Jens Koenig; Elmar STAUDACHER
Original assignee: Robert Bosch Gmbh
Priority date: 2018-01-09
Filing date: 2018-12-27
Publication date: 2019-07-18
Also published as: DE102018200221A1

Abstract

The invention relates to a method for monitoring at least one construction project provided at a construction site while carrying out a construction operation on the at least one construction project. In particular, the at least one construction project is detected (105) visually by means of an imaging sensor system (205); at least one feature of a structural part of the construction project is detected (110) from the detected (105) image data; and on the basis of the detection (110) of the at least one feature of the structural part of the construction project, a change of a building plan or an intervention in the course of the construction operation is carried out (120). The at least one feature detected in this manner preferably forms the basis of a building information modeling (BIM) process.

Description

description

title

Site monitoring procedure

The invention relates to a method for monitoring at least one building object present on a construction site according to the preamble of claim 1. The present invention also relates to a computer program, a machine-readable data carrier for storing the computer program and an electronic control unit, by means of which the method according to the invention can be carried out.

State of the art

The well-known method of "Building Information Modeling" (BIM = Building Information Modeling) enables optimized planning, execution and management of buildings by means of data processing. All relevant building data are digitally modeled, combined and recorded. A building is geometrically visualized as a virtual building model by means of a computer model, whereby a distinction is made between a parametric building model and an intelligent building model. In the parametric building model, all elements (walls, ceilings, dimensions, labels, objects, intersection lines, etc.) can be interdependent, while in the intelligent building model, intelligence is limited to individual objects. The BIM method is used in building construction and construction on a construction site as well as in the later building or property management.

DE 10 2015 010 726 A1 discloses a method for monitoring the current state of a construction site having at least one working machine, in which by means of the work machine arranged on

Monitoring means the environment of the working machine is observed in real time. In this case, corresponding observation data are generated, which by means of a computing unit for evaluating the current site condition are evaluated. Suitable monitoring means are cameras or scanner devices arranged on a suitable work machine of the construction site. The observation data comprise building trades, construction site objects, persons and work machines located in the vicinity of the machine, the current construction progress being determined or monitored on the basis of this data in accordance with existing, digitized construction site planning. The

Observation data represent two- or three-dimensional pictorial representations of the machine environment.

Disclosure of the invention

The invention is based on the idea to capture by means of fixed or movably arranged sensors on a construction site structural parts of at least one building site present on site and to recognize by machine vision empirically definable features of the structural parts of the respective building object. On the basis of the features thus identified, the correctness of the construction measures carried out on the object, the quality of construction, the construction process and / or the construction progress can be compared with correspondingly available plans.

The proposed method can additionally or supportively also be used for the detection of working on the structural parts machinery or construction equipment. As a result, it is additionally possible to provide a logical and, in particular, computer-based plausibility check of a recognized construction measure.

The proposed method for monitoring the construction site or at least one building object provides, in particular, that the at least one building object is captured by means of a sensor that evaluates the captured image of the building object by means of machine vision on the basis of at least one predefinable constructional feature of the building object and that the results in the evaluation,

object-related data in a building object comprehensive

Building Information Modeling (BIM) of an overall or overall

Construction project. The BIM may also provide that on the basis of the resulting, object-related Data a change of the building planning or object planning or the construction process or an intervention in the construction process or the construction or

Processing of the building object is made.

Likewise, the proposed evaluation of building objects provides support in the construction billing and / or billing. In doing so, data of a building modeling, a project planning system with financial systems, e.g. an Enterprice Resource Planning System (ERP). Such an approach is often summarized under the term "BIM 5D", namely, if in addition to the building data mentioned above also project and calculation data are combined to form a planning system.

The advantage of the proposed method lies in a continuous and, compared to the state of the art, automated or automatable, machine-based and still considerably more comprehensive construction control. The thus made possible visualization of the exact construction process and / or the automatic warning or instructions to the staff and / or the automatic intervention and correction options in the construction process is thus ultimately a significant improvement in construction quality and a significant reduction in construction times and construction costs. In addition, including historical data of a construction project and the

Real-time acquisition of a construction process a more accurate prediction or prediction of the construction process possible, whereby the construction times and construction costs can be further improved.

Based on the proposed method can thus be carried out an optimization of the construction process in the construction or processing of a building object.

In the proposed method, it can further be provided that, during the execution of a construction measure on the at least one building object, at least one image is detected or generated by means of a preferably imaging sensor, that a specifiable constructive feature of the construction object is evaluated from the acquired or generated image and that, in the evaluation, resulting, object-related data are compared with data or values of the corresponding feature determined in advance in the preliminary field, and with a recognized deviation of the compared characteristics by one empirically definable threshold value, a change in the building design or the construction process or an intervention in the construction process is made. Such an intervention in the construction process may be that information or warnings to appropriate persons, such as site supervisor, passed. Likewise, one or more operating variables of a construction machine or a construction vehicle can be changed or changed to the respective one

Construction measure to the possibly changed state of construction of the building

adapt.

In the imaging sensor systems are known per se

Detection of objects by means of an optical camera and / or a

Radar antenna and / or a laser beam into consideration. Such sensor systems are particularly useful in the field of automotive engineering for detecting objects, e.g. from people, for various assistance functions to automated driving become known.

Alternatively or additionally, methods known per se may be used

Object detection are used, e.g. Appearance-based methods that capture the characteristic of an object type (e.g., a person) based on a training data set preferably formed of image data. Such a training data set contains different characteristics of a single object type and expresses the possible, different expressions of a single object. In the development and application phase, in one training step, i. "offline" as it were, generates characteristic features from corresponding training images. A collection of such features is then summarized into an overall description of the object type, a so-called feature vector. To get one

To be able to uniquely assign the feature vector to an object type, a so-called "classifier" must be trained. A collection of such

Feature vectors can then be used to describe the appearance of an object in a predeterminable image plane.

For the object recognition, the mentioned feature vectors or feature numbers of the classifier describing the respective object can be converted into interlaced and automatically generated weights into a number which indicates the probability for the object to be classified in a current search window. In contrast to the prior art In the present case, not all object examples must be trained, whereby a correspondingly rigid structure of the preprocessing or object recognition on the basis of fixed filter masks and weights is avoided.

For the recognition of a desired object type, e.g. a particular tool, vehicle or person, a number of image examples can be used, which represent different characteristics of the respective object. This allows machine learning of characteristic, image-based properties of a given object. In addition, negative pictorial counter-representations of the respective object can also be taken as an exclusion criterion, so to speak, in order to further increase the recognition quality.

The computer program according to the invention is set up to carry out each step of the method, in particular if it runs on a computing device or a control device. It allows the implementation of the

inventive method in an electronic control unit without having to make structural changes to this. For this purpose, the machine-readable data carrier is provided on which the computer program according to the invention is stored. By loading the computer program according to the invention on an electronic control unit is the

obtained according to the invention electronic control unit, which is set up to monitor a construction site and / or to control a work machine present at the construction site.

The likewise proposed electronic control unit which cooperates with a named sensor system comprises in particular a recognizer which recognizes at least one feature of a structural part of the building object from the image data provided by the sensor system. By means of an intended model calculator, the corresponding feature of the

Building object determined in advance. By means of a comparator, the sensory data and the previously calculated data are compared. The result of the comparison is fed to an agent who is connected to a said BIM system or to a construction machine

Construction vehicle works together. By means of the agent, a change in the building planning or the construction process or an intervention in the construction sequence may be made. Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.

It is understood that the features mentioned above and those yet to be explained below can be used not only in the particular combination indicated, but also in other combinations or in isolation, without departing from the scope of the present invention.

Brief description of the drawings

Fig. 1 shows an embodiment of the method according to the invention

using a flowchart.

Fig. 2 shows an application of the method according to the invention

furnished electronic control unit based on a schematic block diagram.

Description of exemplary embodiments

In the embodiment of a method shown in Fig. 1 for

Monitoring a construction site having at least one building object to be monitored requires that sensors for the detection of pictorial representations of the building object are arranged in a stationary or movable or detachable manner. The stationary sensors may be camera systems or the like, which in a conventional manner to buildings,

Light masts or fixed to stationary base stations can be installed. These camera systems are preferably pivotable and / or focusable. Non-stationary sensors can be those which are mounted on or on mobile objects such as vehicles, machines, drones, etc., and / or are carried or operated by persons.

After the start 100 of the routine shown in FIG. 1 triggered, for example, by the start of a construction measure, a building object individually taken into the monitoring in the present embodiment is imaged 105 by means of imaging sensors mentioned or described below. From the image data acquired in this way is determined by means of the known method of the Computer-aided or machine vision, at least one empirically predeterminable feature of a constructive part of the building 110 recognized.

One possible embodiment of a building is a building construction in which finished building elements, e.g.

Precast reinforced concrete parts or steel construction site, which by a construction machine or by a crane from a storage place or transport vehicle to the

Installation point are positioned to be detected by sensors. At the

Construction machine, the crane or arranged near the installation imaging sensors can recognize the precast concrete element. The prefabricated reinforced concrete elements usually have structural features, e.g. specific

System connections, electrical boxes, pipe feedthroughs, door or

Window sections, chamfers, etc., on the basis of which a categorization of the respective building object is possible. In the case of externally very similar objects, the application of a specific feature adapted to the size of the object, e.g. a QR code, a data matrix or a bar code, for a one-to-one identification of the object possible.

Prior to this, the feature or the corresponding features of the building object were determined 115 by means of a model calculation. The aforementioned 3D data including the specific features are mostly found in a BIM database or a collaboration platform, e.g. a "cloud", available.

120. Due to the three-dimensional or spatial sensory detection of the building object, 120 corresponding perspectives have to be considered in this comparison and the digitized building object or the corresponding features of the building object possibly simulate digitally.

If the comparison 120 results in a deviation between these comparison data, the deviation preferably within an empirically predeterminable one

Threshold value, then return to the beginning of the routine, e.g. before step 105, jumped back. However, if the comparison 120 shows that the deviation is outside the specified threshold value, then a change 125 of the present building design or of the

Construction process or an intervention in the construction process, which may consist in that one or more operating variables of a construction machine or a construction vehicle is / are changed in order to adapt the respectively affected construction measure to the possibly changed construction state of the monitored construction object. Similarly, the change 125 may correspond to a message or warning from the responsible personnel, such as a site manager.

The sensory detection of the image data of the building object to be monitored can take place in a manner known per se by means of optical sensors. By means of algorithms known per se in the field of computer-aided or

Machine image processing can use the image data obtained thereby object features such. Edges, corners, surfaces etc. are extracted. With the mentioned object features of the building object, it can be e.g. one

Reinforced concrete wall, wherein a feature therein, such. a

Electrical installation box, in a BIM database mentioned as a separate, three-dimensional object is deposited. With this, further objects can be detected within a recognized object. In addition, the spatial extent of an object can be calculated more accurately. In addition, not only can we classify coarse geometric objects, e.g. Bricks, wall elements, pipes or stair elements, but also individual objects such as a named electrical installation box.

The results of the feature recognition of the respective building object can also be stored in a construction progress documentation of the BIM (Building Information Modeling) system. This is particularly advantageous for such building objects, which are no longer visible from the outside after their construction or processing. An essential advantage of the described method is thus the enabling of an automated and precisely documented billing of building activities, e.g. towards a client.

It should be noted that in the described method step for

Model calculation of object-related characteristics is required that information on construction planning or construction site planning of the respective

Building object, e.g. Construction drawings, construction process, construction stages and a construction schedule in terms of the BIM system.

From the results of the aforementioned target / actual comparison, both the degree of completion of the respective construction project can be determined. Also can Based on a detected target-actual deviation in advance, ie predicated recognizing whether the schedule of the construction process can be met. The results of the target-actual comparison of the construction progress can also be visualized by means of methods of virtual reality known per se, for example in a stereoscopic 3D display system according to the prior art.

Based on the available, digitized information on construction planning or construction site planning, predictions and thus hints can be made

Construction participants, e.g. with regard to the materials to be used. Thus, on the basis of findings from said target / actual comparison and the planned construction process, the materials to be used in a subsequent operation as well as corresponding parts of the construction plan may be immediately passed to the site manager or operator of a construction vehicle or construction machine, e.g. by means of visual augmented reality on a display, together with a real video or image recording of the building object as well as synthetically inserted construction objects. As a result, incorrect use of the material can be detected immediately and appropriate countermeasures can be taken immediately.

The method described can be realized in the form of a control program for an electronic control unit for controlling a building measure or the sensor system mentioned here. Such, for carrying out the method described above electronic control unit is shown schematically in Fig. 2.

The control device shown in FIG. 2 can also be connected to a data interface to the database of a BIM system, a database of a project management, an enterprise resource planning (ERP) system and / or further

Be connected to data sources, which provide additional context information. The data interface may also provide a cellular connection to facilitate data exchange between the controller and said databases, e.g. the cloud.

The aforementioned sensors for image acquisition may be communication-based or data-technically networked, stationary and non-stationary environmental sensors which are present on the construction site and which preferably supply a digital 3D image of the construction object to be monitored. As stationary sensors come those in consideration, which are installed on buildings, on light poles or fixed base stations stationary. In this case, the sensors can be designed to be pivotable and focusable. Non-stationary sensors are those which are arranged on movable objects, such as vehicles, machines, people, drones, etc. From the of

different images supplied to each sensor, the respective

Fields of view assembled and merged with respect to the image information, so that a total of a digital 3D image of an entire building object or possibly multiple building objects is present.

By means of algorithms known per se for image processing, predefinable features of the respective building object are extracted from the sensory image information obtained, such as, for example, Edges, Corners, Surfaces, etc. From the characteristics thus obtained, the building objects can be classified, e.g. as walls, windows, doors, pipes, pipes, breakthroughs, etc. In addition, textures or the surface texture, the colors and possibly captions of the building objects can be recognized, whereby the object classification can be made even more precise or possibly even more object Attributes.

It should be noted that currently available algorithms for image-evaluating object classification are so robust executable that even unfinished objects can be safely classified. Thus, e.g. an unfinished wall or roof not fully covered can be identified and classified.

If global measurement points or georeference points and / or GPS receivers are stationary or not stationary at the respective construction site, the features and objects extracted by means of the algorithms mentioned can additionally be provided with a global position. This makes it possible to check whether the building objects are being erected globally in the right position. Additional sensors, e.g. for detecting the spatial position of an object, or additional feature transmitters, e.g. Passive or active RFID identifiers allow an additional plausibility check and specification of the respective building object.

The described image processing results in a georeferenced 3D overall object model, eg a building, including the recognized, georeferenced building objects, such as foundation plates, walls in the various floors, windows, stairs, etc. The fact that the aforementioned environmental sensors are present from the beginning of construction at the site, all parts of the overall object are recognizable and georeferenced in the 3D overall object model available, and In particular, even if these parts in the course of construction progress are no longer visible or visible from the outside.

In addition to the above-mentioned image information, the building objects may additionally be provided with a time stamp with regard to their first recognition in the 3D overall object model. For this purpose, individual images or video sequences with a e.g. provided by a Goelokalisationssystem time stamp, are stored in a database. Also, the degree of completion of the respective building object can be recorded as a function of time. According to the present construction schedule, at a certain geo-referenced location, e.g. planned a vertical and in itself flat wall, but this wall only partially bricked, then the object recognition of this building object "wall" with a lower image acquisition rate so to speak, in "time lapse" through and be used the resulting image data to be used, the image repetition rate of this wall to be interpreted as a construction progress. The object recognition can be done by means of stationary sensors and camera (s).

With the mentioned algorithms for machine vision or for

Object recognition also identifies (classifies) people, machines, tools and other work equipment. This makes it possible to detect whether and, if so, how many persons, which machines, etc., are georeferenced on which part of the overall object. It can also be seen whether these tools are in motion or active. In addition, it can be recognized in good time whether the machines, tools, persons, etc. required for the next construction steps or for the next work step are already on-site or have been ordered in due time to the place of work. The information thus determined may then be in the 3D object model

Taking into account, for the respective steps in the

Construction section and construction process necessary machinery, tools, persons, etc. are also available in digitized form. In the case of the object recognition mentioned, the characteristic of an object type, for example a person or a construction vehicle, can be based on a training data record generated in advance. This data record contains different characteristics of a single object type and expresses the different characteristics of a single object. In the

Development and application phase "offline" in a training step from respective training images characteristic features generated. A

Collection of such characteristics is then summarized and gives a total description of the object type. In order to be able to unambiguously assign a corresponding feature vector to an object type, a classifier provided for the named classification must be trained. A collection of said features is then used to describe the appearance of an object in a given image plane.

To recognize a desired object type, e.g. a tool, a vehicle or a pedestrian a plurality of image examples are required for the respective object type, representing different manifestations of the object, preferably in all environmental conditions occurring. Thus, through machine learning, the characteristic, image-based

Properties of a desired object can be determined, in addition a plurality of counterexamples of non-possible expressions can be based.

In carrying out the object recognition, the respective building object descriptive feature numbers of a said classifier on a

A plurality of nested and automatically generated weights converted into a number indicating the probability of the object to be classified in a current search window. This becomes the requirement of a

Preprocessing or object detection on the basis of fixed filter masks and weights effectively avoided.

In the method described, it is assumed that information on the construction planning of the respective building object, eg architectural drawings, the planned construction process, the intended construction phases and the construction schedule are available in digitized form. For example, the corresponding data is created as a parametric building model, using a CAD system and suitable planning tools from an existing BIM software, so that it can be used with fused and classified information of said sensors in a said 3D object model are comparable and thus a said target-actual comparison can be performed. Both the correctness with regard to the presence or the degree of completion of the building object, with respect to the global position of the object, the dimensions and

Surface texture of the object, etc. are checked. Also, it can be detected by means of a target-actual deviation, whether the schedule of

Construction process is complied with. The results of the target-actual comparison of the construction progress can also be visualized in a manner known per se by means of virtual reality methods, so that the processor can better oversee the entire construction project.

The free spaces available for the movement of vehicles, machines and persons from the 3D (total) object model depending on the progress of the construction can be made available to them, and e.g. serve as spatial automatic limits for tool and machine movements.

If high-precision sensors are used in the aforementioned sensors, e.g. LIDAR systems or stereoscopic cameras, then highly accurate measurements of the respective construction measure can be made in real time, so to speak. In this case, components detected by means of machine vision can be dimensioned simultaneously in the manner described. To increase the

Measuring accuracy in the measurements mentioned geolocalized measured reference points can be used. The resulting dimensions can then be compared with corresponding nominal dimensions from the respective blueprints and with a detected deviation outside an empirically predeterminable threshold value or a corresponding permissible tolerance, e.g. visualized by means of "augmented reality". In this case, those parts of the building object with impermissible tolerances, e.g. highlighted in color and / or by flashing etc.

The electronic control unit 200 shown schematically in FIG. 2, which is indicated by the dashed line, cooperates with a described, imaging sensor 205 or receives corresponding sensor data from the sensor system 205. By means of a recognizer 210, at least one feature of a constructive part of the building object is recognized from the image data captured by the sensor system 205. By means of a model calculator 215 is the corresponding characteristic of the building object determined in advance. By means of a comparator 220, the sensorially recognized data and the previously calculated data are compared with one another. The result of the comparison is supplied to an agent 225 which cooperates with a said BIM system or with a construction machine or a construction vehicle 230. By means of the

Agent 225 is made if necessary, a change in the building design or the construction process or an intervention in the construction process.

Claims

claims

1. A method for monitoring at least one on a construction site

When a construction project is carried out on the at least one construction object, the at least one construction object is image-captured (105) by means of an imaging sensor (105) in that at least one feature of the constructional part of the (105) image data Building object is detected (110), and that on the basis of the detection (1 10) of the at least one feature of the structural part of the building a change in a building design or an intervention in a construction process of the construction measure is made (120).

2. The method according to claim 1, characterized in that the at least one recognized (1 10) feature of the constructive part of the building object in a building data modeling (BIM) of a construction project comprehensive building project is based (1 15)

3. The method according to claim 1 or 2, characterized in that the

recorded (105) image data by means of the method of computer-assisted vision for the detection (110) of the at least one feature of the constructive part of the building object to be evaluated.

4. The method according to any one of the preceding claims, characterized

characterized in that during the execution of the construction measure by means of the imaging sensor (205) the at least one building object is detected (105) that from the acquired (105) image data the at least one feature of the constructive part of the building object is recognized (110) the at least one recognized (1 10) feature of a constructive part of the building object is compared (120) with computationally determined (1 15) data of the corresponding feature of the constructive part of the building object, and if a deviation of the compared data is detected a change in the building design or Intervention in the construction progress of the construction work, based on building data modeling (BIM) (120).

5. The method according to claim 4, characterized in that the change in the building design or an intervention in the construction process of the construction measure in a detected deviation of the compared data is made by a predetermined threshold (120).

6. The method according to any one of claims 3 to 5, characterized in that by means of the imaging sensor (205), the outer shape of the building object and / or the surface condition of the building object are detected and evaluated or will.

7. The method according to any one of the preceding claims, characterized

characterized in that an object recognition is carried out by means of the sensor system (205), wherein the characteristic of an object type is detected on the basis of a training data record.

8. The method according to claim 7, characterized in that for a

Object type characteristic features are generated and from a feature vector is formed, for which a trainable classifier is provided.

9. The method according to claim 8, characterized in that an object

descriptive feature vectors are converted by automatically generated weights into a value indicating the probability of the object to be classified in a current search window.

A computer program configured to perform each step of a method according to any one of claims 1 to 9.

1 1. Machine-readable data carrier on which a computer program

is stored according to claim 10.

12. Electronic control unit (200) which is set up to control a system for building data modeling (BIM) and / or a construction machine (230) by means of a method according to one of claims 1 to 9.

13. Electronic control device according to claim 12, characterized by a recognizer (210) for detecting at least one feature of a

constructive part of the building object based on at least one sensor signal provided by an imaging sensor (205), a A model calculator (215) for calculating the at least one feature of the constructive part of the building object, a comparator (220) for comparing the data detected by the recognizer (210) and calculated by the model calculator (215), and an agent (225) for performing Modification of the building design or the construction process or an intervention in the construction process, depending on the result of the comparison.