Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/11—Region-based segmentation

Definitions

• the present invention relates to an in-building structure recognition system and an in-building structure recognition method, and in particular, to an in-building structure recognition system that recognizes a structure placed inside a building such as a building using deep learning using a neural network.
• the present invention relates to a recognition system, a method and program for recognizing structures inside buildings.
• Another idea is to perform machine learning and create a trained model using a rendered image of a completed 3D model of the construction site that closely resembles the actual appearance, rather than photos of the actual construction site. It will be done.
• rendered images are mainly created for commercial purposes of buildings, and their production costs are high, making it difficult to prepare a sufficient number of rendered images as learning images. Further, the work of annotating structures included in rendered images becomes enormous and requires time and effort to perform manually.
• the trained model created thereby is required to be able to recognize structures with high accuracy.
• the system is capable of recognizing with high accuracy site images that include noise components such as wire mesh, protective nets and sheets, temporarily installed iron fences and poles, garbage, and materials existing at the construction site. is required.
• noise components such as wire mesh, protective nets and sheets, temporarily installed iron fences and poles, garbage, and materials existing at the construction site.
• the existence status of such noise components changes from moment to moment as they are moved or added to suit the construction situation, so it is necessary to It is recommended to use a model. In this case, it is desirable to be able to minimize the time and cost of regenerating a model tailored to the site.
• Non-Patent Document 1 discusses the problem of the huge amount of point cloud data in as-built modeling, which creates 3D models based on 3D measurements of existing large-scale equipment, and describes the problem of ⁇ Measurement used in as-built modeling of large-scale equipment''. It should be noted that the measurement principle of this device is different from that of point cloud measurement devices for small parts.For point cloud measurement of small parts, triangulation is generally performed using a laser output device and a CCD camera. However, with this method, as the size of the object increases, the equipment becomes larger.Also, when measuring small parts, the measured point cloud is often only a few million points at most, but in the case of large equipment, It has been pointed out that "a large number of point clouds are required for modeling.”
• Patent Document 1 "digitized data of existing parts of a building acquired from existing drawings is converted into 3D CAD data, and created from point cloud data acquired by a 3D laser scanner or the point cloud data.
• Existing partial survey means is stored together with various field survey data including the 3D polygon model that has been constructed, and newly constructed part objects are selected from member objects stored in the member library in advance for the 3D polygon model.
• a CPU functioning as a member construction position output means for searching and outputting a member object corresponding to a unique ID together with its construction position information from a three-dimensional CAD model designed by the construction member design means; and an automatic position pointing device that indicates the construction position of the member in the existing part based on the construction position information of the member object output by the member construction position output means.
• an image acquisition unit that acquires an input image generated by imaging a real space using an imaging device, and based on the position of one or more feature points reflected in the input image, a recognition unit that recognizes a relative position and orientation between the real space and the imaging device; an application unit that provides an augmented reality application using the recognized relative position and orientation; and the recognition unit. and a display control unit that superimposes a guiding object that guides a user operating the imaging device on the input image according to the distribution of the feature points so that the recognition processing performed by the image processing device is stabilized. is disclosed.
• Patent Documents 1 and 2 both disclose techniques for grasping a three-dimensional space or an object in a three-dimensional space, in particular, three-dimensional point clouds in large-scale facilities such as buildings and factories are used. It did not solve the problem of huge amounts of data, and it was not suitable for automating the recognition of structures in images in order to quickly understand the situation at a construction site in the middle of construction. .
• Patent Documents 1 and 2 also apply to site images that include noise components such as wire mesh, protective nets and sheets, temporarily installed iron fences and poles, garbage, and materials existing at the construction site. There was no consideration given to recognizing structures with high accuracy or retraining the model to match the latest site conditions.
• the present invention solves the above-mentioned problems, and also handles site images that include noise components such as wire mesh, protective nets and sheets, temporarily installed iron fences and poles, garbage, and materials existing at the construction site.
• the present invention provides an in-building structure recognition system and an in-building structure recognition method that can recognize target structures with high accuracy and can recognize structures with high precision in accordance with the latest site conditions. be.
• the present invention also provides a program for causing a computer to execute each step of the method for recognizing structures inside a building.
• the present invention provides an in-building structure recognition system for recognizing in-building structures using a machine learning model, which uses BIM (Building Information Modeling) data and a first site image.
• a first machine learning model generation unit that performs machine learning using a first machine learning model to generate a first machine learned model, and an image of a noise construct that does not have BIM data for the first machine learned model a second machine learning model generation unit that performs relearning using a second on-site image containing images to generate a second machine learned model; a scanning unit that scans the interior of the building and acquires three-dimensional point cloud data and a third on-site image; and a noise component that removes an image of the noise constituent from the third on-site image acquired by the scanning unit.
• BIM Building Information Modeling
• a structure recognition system Provides a structure recognition system.
• the first machine learning model generation unit uses an image generated from BIM data as correct data and an image generated by rendering the BIM data as a first site image.
• the method is characterized in that machine learning is performed using an image obtained by processing using information obtained from observation data as observation data to generate a first machine learned model.
• the second machine learning model generation unit is configured to generate correct data and observation data for the second site image that includes an image of a noise component that does not have BIM data.
• the method is characterized in that the set is input to a first machine-learned model and re-learning is performed to generate a second machine-learned model.
• the scanning unit acquires images inside the building, and at least one corresponding reference point or reference structure exists between successive frames. If at least one corresponding reference point or reference structure does not exist, an alert prompting rescanning is notified.
• the noise component removal unit extracts an image of the noise construct from the third on-site image acquired by the scanning unit by stereo matching, and generates a mask image of the noise construct.
• the method is characterized in that it generates an image, reconstructs the image so as to interpolate a portion of the mask image, and generates an image from which noise components have been removed.
• the third machine learning model generation unit generates a correct answer for the image from which noise components have been removed from the third on-site image obtained by the noise component removal unit.
• the method is characterized in that a set of data and observed data is input to a second machine-learned model and re-learning is performed to generate a third machine-learned model.
• the in-building structure recognition unit inputs a third on-site image to a third machine-learned model to identify inside buildings included in the third on-site image.
• the feature is that it recognizes the structure of.
• the present invention provides an in-building structure recognition method for recognizing structures in a building using a machine learning model, which performs machine learning using BIM (Building Information Modeling) data and a first site image. and retraining the first machine learned model using a second site image that includes an image of a noise composition that does not have BIM data. scanning the building while determining the success or failure of scanning the structures inside the building, and generating 3D point cloud data of the structures inside the building and images inside the building.
• BIM Building Information Modeling
• a method for recognizing a structure inside a building includes a step of extracting and outputting point cloud data of a structure inside a building recognized by an inside structure recognition unit from the three-dimensional point cloud data acquired in the above.
• the scanning step includes acquiring an image inside the building and determining whether or not there is at least one corresponding reference point or reference structure.
• the method is characterized in that an alert prompting rescanning is notified if at least one or more corresponding reference points or reference structures do not exist.
• the present invention provides a program that causes a computer to execute each step of the above method for recognizing structures inside a building.
• BIM Building Information Modeling
• the accuracy of structure recognition can be improved by relearning the model to match the latest site conditions in response to noise components that change moment by moment.
• FIG. 1 is a schematic diagram showing the entire structure recognition system in a building according to the present invention.
• FIG. 2 is a diagram showing the flow of each process of the building structure recognition system according to the present invention.
• FIG. 3 is a schematic diagram showing the first machine learning model generation section of the present invention.
• FIG. 4 is a schematic diagram showing the second machine learning model generation section of the present invention.
• FIG. 5 is a schematic diagram showing the scanning section of the present invention.
• FIG. 6 is a schematic diagram illustrating the noise construct removal section of the present invention.
• FIG. 7 is a schematic diagram showing the third machine learning model generation section of the present invention.
• FIG. 8 is a schematic diagram showing the intra-building structure recognition section of the present invention.
• FIG. 9 is a diagram showing the overall flow of the method for recognizing structures inside a building according to the present invention.
• FIG. 1 is a schematic diagram showing the entire building structure recognition system 1 according to the present invention.
• the in-building structure recognition system 1 according to the present invention includes a first machine learning model that performs machine learning using BIM (Building Information Modeling) data and a first site image to generate a first machine learned model.
• the generation unit 11 performs relearning on the first machine learned model M1 using a second site image including an image of a noise component that does not have BIM data, and generates a second machine learned model M2.
• a second machine learning model generation unit 12 that generates 3D point cloud data inside the building and a third on-site image are obtained by scanning the inside of the building while determining the success or failure of scanning the structures inside the building.
• a third machine learning model generation unit 13 that performs relearning on the second machine learned model using the image from which noise components have been removed, and generates a third machine learned model M3;
• the building structure recognition unit 40 uses the third machine learned model M3 to recognize the structures in the building, and the building structure recognition unit recognizes the structures from the three-dimensional point cloud data acquired by the scanning unit.
• a point cloud data output unit 50 that extracts and outputs point cloud data of structures in the building.
• the first machine learning model generation unit 11 performs machine learning using BIM (Building Information Modeling) data and the first site image to generate a first machine learned model. Specifically, the first machine learning model generation unit 11 uses the image generated from the BIM data as correct data, and processes the image generated by rendering the BIM data using the information obtained from the first site image. Machine learning is performed using the image obtained by this as observation data to generate a first machine learned model.
• BIM Building Information Modeling
• the second machine learning model generation unit 12 performs relearning on the first machine learned model M1 using a second site image including an image of a noise component that does not have BIM data, and A machine learned model M2 is generated. Specifically, the second machine learning model generation unit 12 generates a set of correct data and observed data for the second site image including an image of a noise component that does not have BIM data, using the first machine learning model generation unit 12. A second machine-learned model is generated by inputting the input into the trained model and performing re-learning.
• the scanning unit 20 scans the inside of the building while determining the success or failure of scanning the structures inside the building, and acquires three-dimensional point cloud data inside the building and a third on-site image.
• the scanning unit 20 acquires an image inside the building, determines whether or not there is at least one corresponding reference point or reference structure, and determines whether there is at least one or more corresponding reference point or reference structure. If a structure does not exist, an alert will be sent prompting a rescan.
• the noise constituent removal unit 30 extracts an image from which noise constituents have been removed from the image inside the building acquired by the scanning unit 20.
• the noise constituent removing unit 30 extracts an image of the noise constituent from the third scene image acquired by the scanning unit 20 by stereo matching, generates a mask image of the noise constituent, and interpolates the portion of the mask image. The image is reconstructed to generate an image from which noise components have been removed.
• the third machine learning model generation unit 13 performs relearning on the second machine learning model using the image from which the noise components extracted by the noise component removal unit have been removed, and generates a third machine learning model. Generate trained model M3. Specifically, the third machine learning model generation unit 13 generates a set of correct data and observed data for the image from which noise components have been removed from the third on-site image, obtained by the noise component removal unit 30. , and performs relearning by inputting it into the second machine learned model to generate a third machine learned model.
• the in-building structure recognition unit 40 recognizes the structures in the building using the third machine learned model M3.
• the intra-building structure recognition unit 40 inputs the third on-site image to the third machine-learned model and recognizes the intra-building structures included in the third on-site image.
• the point cloud data output unit 50 extracts and outputs point cloud data of the structures inside the building recognized by the building structure recognition unit from the three-dimensional point cloud data acquired by the scanning unit.
• FIG. 2 is a diagram showing the flow of each process of the building structure recognition system according to the present invention.
• Figure 2 shows the relationships between machine learning model generation processing, data acquisition/scanning processing, noise component removal processing, and building structure recognition processing among the processes executed in the building structure recognition system.
• the machine learning model generation process is executed by the first machine learning model generation unit 11, the second machine learning model generation unit 12, or the third machine learning model generation unit 13.
• the data acquisition/scanning process is executed by the scanning section 20 or the point cloud data output section 50.
• the pre-processing before scanning may be performed by any external imaging device or scanning device (not shown).
• the noise constituent removal process is executed by the noise constituent removal unit 30.
• the building structure recognition process is executed by the building structure recognition section 40.
• the overall processing flow is roughly divided into pre-processing before scanning and processing after scanning.
• the pre-processing before scanning includes generating a first machine learned model and relearning the first machine learned model to generate a second machine learned model.
• BIM data and a first site image for generating a first machine learned model are acquired.
• a first machine learned model is generated using the acquired BIM data and the first site image.
• the first machine learned model is created assuming an ideal site, and can be used universally for various sites. However, in the actual site, there may be items (wire mesh, protective nets and sheets, temporarily installed iron fences and poles, trash, materials, etc.) that are not included in the BIM data.
• a second site image that includes such noise components that become noise, and added it to the first machine learned model in order to learn the noise components as noise.
• Re-learning will be performed.
• a second machine-learned model matching the actual site situation is obtained.
• the second site image is acquired when inspecting the site before performing actual scanning at the site.
• Processing after scanning involves scanning, removing noise components, generating a third machine learned model, recognizing structures inside the building, and acquiring point cloud data. It includes doing.
• the success or failure of scanning is determined, and if it is necessary to rescan, an alert is sent to prompt rescanning. If an alert prompting a rescan is received, a rescan will be performed. Repeat this until all objects in the building are scanned. Noise components are then removed from the scanned third scene image.
• FIG. 3 is a schematic diagram showing the first machine learning model generation section of the present invention.
• the first machine learning model generation unit 11 uses the image generated from the BIM data as correct data (correct image), and processes the image generated by rendering the BIM data using the information obtained from the first site image. Machine learning is performed using the image obtained by this as observation data (observed image) to generate a first machine learned model.
• the correct image generated from the BIM data shows the structure in the image in a way that distinguishes it from the background.
• the correct image may be one that is manually generated, such as by manually filling in parts of the structure within the image.
• the correct image may be, for example, a binarized image in which a structure part and a background part can be distinguished. Observation images are also generated from BIM data.
• a rendered image is generated by rendering BIM data.
• using information such as textures extracted from the first site image, textures, etc. are added to the rendered image to generate an image that is more similar to the actual photograph of the site, and this is used as an observation image.
• a machine learning model generation process is performed using such a set of the correct image and observed image to generate a first machine learned model M1.
• the first machine learned model M1 can be used as a general-purpose machine learned model for recognizing structures inside a building. In particular, when scanning a building in an ideal environment where no noise components exist and recognizing structures within the building, the first machine-learned model M1 can be used.
• FIG. 4 is a schematic diagram showing the second machine learning model generation section of the present invention.
• the second machine learning model generation unit 12 inputs a set of correct data and observation data regarding the noise component image that does not have BIM data included in the second site image into the first machine learned model M1. Then, relearning is performed to generate a second machine learned model M2.
• the first machine learned model M1 since the correct image and observed image are generated based on BIM data and used for machine learning, the first machine learned model M1 is free of noise components. Suitable for scanning ideal building environments that do not exist. On the other hand, in actual sites, noise components that do not have BIM data often exist.
• a second machine learned model M2 that is capable of is generated.
• a set of correct images and observed images to be used for relearning the first machine learned model M1 is generated from the second on-site image.
• the correct image of the second scene image shows the structure in the image so as to be distinguished from the background.
• the correct image of the second site image may be one that is manually generated, such as by manually filling in a portion of the structure in the image.
• the correct image of the second site image may be, for example, a binarized image in which a structure part and a background part can be distinguished.
• the second on-site image including noise components may be used as is. Further, as the observed image of the second on-site image, an image obtained by pre-processing the second on-site image including noise components as necessary may be used. Using such a set of the correct image of the second on-site image and the observed image, a relearning process is performed on the first machine learned model M1 to generate a second machine learned model M2.
• FIG. 5 is a schematic diagram showing the scanning section of the present invention.
• the scanning unit 20 acquires an image inside the building, determines whether or not there is at least one corresponding reference point or reference structure, and determines whether there is at least one or more corresponding reference point or reference structure. If a structure does not exist, an alert will be sent prompting a rescan.
• the scanning unit 20 may include a scanning success/failure determination unit 203, an alert notification unit 204, and a rescan processing unit 205 for each function.
• the scanning success/failure determination unit 203 determines whether there is at least one corresponding reference point or reference structure.
• the "reference point” is a point that serves as a reference for matching consecutive frames, and for example, a marker or the like may be attached to a structure in a building to serve as the reference point.
• a “reference structure” is a structure that serves as a reference for matching consecutive frames, and for example, a structure that has a straight part, such as a corner of a column, is a structure that serves as a reference. You may also do so.
• the alert notification unit 204 notifies an alert prompting rescanning when at least one corresponding reference point or reference structure does not exist.
• the alert is for notifying the user that rescanning is necessary, and includes the display of an icon or message on the display screen of a scanning device such as the distance measuring scanner 201 or the imaging device 202, a warning sound, etc. It may be.
• the rescan processing unit 205 receives a rescan instruction from the user and performs the rescan.
• the processing in the scanning success/failure determination unit 203, alert notification unit 204, and rescan processing unit 205 is repeated until all scanning of the scan target is completed, and when the scanning is completed, the third site image and the three-dimensional point cloud data are is obtained.
• the third on-site image and three-dimensional point cloud data acquired by the scanning unit 20 may include information on noise components.
• FIG. 6 is a schematic diagram illustrating the noise construct removal section of the present invention.
• the noise constituent removing unit 30 extracts an image of the noise constituent from the third scene image acquired by the scanning unit 20 by stereo matching, generates a mask image of the noise constituent, and interpolates the portion of the mask image. The image is reconstructed to generate an image from which noise components have been removed.
• the noise component removal unit 30 may include a stereo matching unit 301, a mask image generation unit 302, and an image reconstruction unit 303 for each function.
• the stereo matching unit 301 adds two images of the same object taken from different viewpoints (typical (right image and left image) are input, three-dimensional depth is estimated for each pixel, and a distance image is generated in which the estimated depth for each pixel is represented by a gradation.
• Existing machine learned models for performing stereo matching calculate the parallax that represents the disparity between two images of the same object taken from different viewpoints (typically a right image and a left image). It may be a machine-learned model in which a mapping function for obtaining an image is learned using a convolutional neural network (CNN). Additionally, existing machine learned models for performing stereo matching may use recursive refinement to update disparity from coarse to fine, or a hierarchical network with a layered cascade architecture for inference.
• CNN convolutional neural network
• the stereo matching unit 301 may use an existing stereo matching method that does not use a machine learning model instead of the above method that uses an existing machine learned model for stereo matching.
• the three-dimensional depth of each pixel is estimated using two images (typically the right image and the left image) taken from two points, and the estimated depth of each pixel is A distance image represented by a gradation may be generated.
• the mask image generation unit 302 performs threshold processing on the distance image generated by the stereo matching unit 301 to generate a mask image of the noise component. For example, if a wire mesh that is a noise component exists in front of a structure to be recognized, a mask image of the wire mesh portion that is a noise component is generated.
• the image reconstruction unit 303 removes the mask portion of the mask image generated by the mask image generation unit 302 from the original image, and obtains a reconstructed image by interpolating the mask image portion.
• the image reconstruction unit 303 adds the third on-site image containing noise components and the mask image of the noise components generated by the mask image generation unit 302 to an existing machine learned model for image reconstruction.
• a reconstructed image may be obtained such that regions of the mask image are removed and portions of the mask image are interpolated.
• the existing machine learned model for reconstructing an image may be an existing machine learned model generated by deep learning using a neural network. Thereby, the image reconstruction unit 303 obtains an image from which noise components have been removed from the third on-site image.
• FIG. 7 is a schematic diagram showing the third machine learning model generation section of the present invention.
• the third machine learning model generation unit 13 generates a set of correct data and observed data for the image from which noise components have been removed from the image inside the building extracted by the noise component removal unit 30.
• the input is input to the trained model and relearning is performed to generate a third machine learned model.
• the correct image of the second on-site image containing noise components and the observed image obtained in pre-processing before scanning are used for re-learning.
• noise components were removed from the third field image that contained the noise components and was acquired during the actual scanning.
• a third machine-learned model M3 is generated that is capable of recognizing structures further suited to the target site.
• the set of correct images and observed images to be used for relearning the second machine learned model M2 is generated from the image obtained by the noise component removal unit 30 from which noise components have been removed from the third on-site image. be done.
• the correct image of the image from which the noise components have been removed is one that shows the structures in the image in a way that distinguishes them from the background.
• the correct image of the image from which the noise components have been removed may be one that is manually generated, for example, by manually filling in portions of the structure in the image.
• the correct image of the image from which noise components have been removed may be, for example, a binarized image in which a structure part and a background part can be distinguished.
• the image from which the noise components have been removed may be used as is.
• the observed image of the image from which the noise components were removed was processed by preprocessing as necessary on the image from which the noise components were removed from the third on-site image containing the noise components. Images may also be used. Using the set of the correct image and observed image from which such noise components have been removed, relearning processing is performed on the second machine learned model M2, and the third machine learned model M3 is generate.
• FIG. 8 is a schematic diagram showing an intra-building structure recognition section and a point cloud data output section of the present invention.
• the intra-building structure recognition unit 40 inputs the third on-site image to the third machine-learned model M3 and recognizes the intra-building structures included in the third on-site image.
• the output from the third machine learned model M3 shows the structure inside the building in the image so as to distinguish it from the background.
• the output from the third machine-learned model M3 may be a binarized image that can distinguish between the components inside the building and the background.
• the third scene image may include noise constructs that were present during the scanning of the scene.
• the third machine learned model M3 is obtained by additionally learning an image in which noise components are removed from a third on-site image that includes noise components, and when the third on-site image includes noise components, Structures inside a building can be recognized with high accuracy even when
• the point cloud data output unit 50 is a scanning unit that generates three-dimensional point cloud data of a portion corresponding to the recognized structure in the building based on the image information of the structure in the building recognized by the structure recognition unit 40 in the building. It is extracted from the three-dimensional point cloud data scanned in step 20 and output as point cloud data. As a result, 3D point cloud data of the structures inside the building is obtained, and by performing rendering etc. on the obtained 3D point cloud data, it is used to generate 3D CAD data of the inside of the building including the structure, etc. It becomes possible to use it for.
• FIG. 9 is a diagram showing the overall flow of the method for recognizing structures inside a building according to the present invention.
• the indoor structure recognition method according to the present invention includes step S901 of performing machine learning using BIM (Building Information Modeling) data and a first site image to generate a first machine learned model; Step S902 of performing re-learning on the learned model M1 using a second site image including images of noise components that do not have BIM data to generate a second machine-learned model M2; A scanning step S903 in which the building is scanned while determining the success or failure of scanning of the structure in the building, and three-dimensional point cloud data of the structure in the building and an image inside the building are obtained; Step S904 of removing the image of the noise component from the image of Step S905 of generating the third machine learned model M3; Step S906 of recognizing structures in the building using the third machine learned model M3; and S906, from the three-dimensional point cloud
• BIM Building Information Modeling
• the scanning step S903 acquires an image inside the building and determines whether there is at least one corresponding reference point or reference structure. If at least one corresponding reference point or reference structure does not exist, an alert prompting rescanning is sent.
• the present invention provides a program that causes a computer to execute each step of the above method for recognizing structures inside a building.
• An in-building structure recognition system for recognizing structures in a building using a machine learning model in Example 2 performs machine learning using BIM (Building Information Modeling) data and a first site image, and The first machine learning model generation unit 11 generates the first machine learned model M1, scans the building while determining the success or failure of scanning the structures in the building, and generates three-dimensional point cloud data and data in the building.
• BIM Building Information Modeling
• a scanning section 20 that acquires a third on-site image; a noise component removal section 30 that removes an image of noise components from the image inside the building acquired by the scanning section 20; a third machine learning model generation unit 13 that performs relearning on the first machine learned model M1 using the image from which noise components have been removed, and generates a third machine learned model M3;
• the building structure recognition unit 40 recognizes the structures in the building using the third machine learned model M3, and the three-dimensional point cloud data acquired by the scanning unit 20 recognizes the structures in the building.
• the present invention is characterized by comprising a point cloud data output section 50 that extracts and outputs point cloud data of the structures in the building recognized by the section 40.
• the difference from the building structure recognition system 1 of Example 1 is that the first machine-learned model is re-trained using a second site image that includes images of noise components that do not have BIM data. It does not include a second machine learning model generation unit 12 that generates a second machine learned model.
• the third machine learning model generation unit 13 performs relearning on the first machine learned model M1 instead of the second machine learned model M2, and Generate trained model M3. That is, in the second embodiment, the third machine learning model generation unit 13 calculates the difference between the correct data and the observed data for the image from which the noise components extracted from the building image extracted by the noise component removal unit 30 are removed. The set is input to the first machine learned model M1 and relearning is performed to generate the third machine learned model M3.
• a set of correct images and observed images to be used for relearning the first machine-learned model M1 is generated from an image obtained by the noise component removal unit 30 from which noise components have been removed from the third on-site image. be done.
• the correct image of the image from which the noise components have been removed is one that shows the structures in the image in a way that distinguishes them from the background.
• the correct image of the image from which the noise components have been removed may be one that is manually generated, for example, by manually filling in portions of the structure in the image.
• the correct image of the image from which noise components have been removed may be, for example, a binarized image in which a structure part and a background part can be distinguished.
• the image from which the noise components have been removed may be used as is.
• the observed image of the image from which the noise components were removed was processed by preprocessing as necessary on the image from which the noise components were removed from the third on-site image containing the noise components. Images may also be used. Using a set of the correct image from which such noise components have been removed and the observed image, relearning processing is performed on the first machine learned model M1, and the third machine learned model M3 is generate.
• the in-building structure recognition method for recognizing structures in a building using a machine learning model in Example 2 is the in-building structure recognition method for recognizing structures in a building using a machine learning model.
• BIM Building Information Modeling
• Step S906 recognizes the structures inside the building using the third machine learned model, and from the three-dimensional point cloud data acquired by the scanning section, Step S907 of extracting and outputting point cloud data of the structure.
• the difference from the building structure recognition method of Example 1 is that the first machine learned model is retrained using a second site image that includes images of noise components that do not have BIM data. , does not include step S902 of generating the second machine learned model.
• step S905 of generating the third machine learned model performs relearning on the first machine learned model M1 instead of the second machine learned model M2, and 3 machine learned model M3 is generated. That is, in Example 2, the set of correct data and observed data for the image from which noise components have been removed from the third on-site image containing noise components obtained by the noise component removal unit 30 is It is input to the machine learned model M1 and relearning is performed to generate a third machine learned model M3.
• the building structure recognition system and the building structure recognition method according to the present invention described above it becomes possible to measure the shape and position of noteworthy members at a construction site, thereby improving accuracy and speed. can be improved. Furthermore, the amount of components to be managed at a construction site can be reduced, and accordingly, the amount of data handled by the construction site component management system can be significantly reduced.
• wire mesh, protective nets and sheets, temporarily installed iron fences and poles, garbage, materials, etc. existing at the construction site can be used. Even on-site images containing noise components can be recognized with high accuracy.
• the accuracy of structure recognition can be improved by relearning the model to match the latest site conditions in response to noise components that change moment by moment.

Landscapes

• Engineering & Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Image Processing (AREA)

Priority Applications (2)

Applications Claiming Priority (1)

Publications (1)

Family

ID=89941509

Family Applications (1)

Country Status (2)

Cited By (1)

Citations (5)

• 2022
  • 2022-08-18 JP JP2024541348A patent/JPWO2024038551A1/ja active Pending
  • 2022-08-18 WO PCT/JP2022/031247 patent/WO2024038551A1/ja not_active Ceased

Patent Citations (5)

Also Published As

Similar Documents

Legal Events