JP2020149546A - Construction support system and construction support device - Google Patents

Abstract

To improve convenience for construction management.SOLUTION: There is provided a construction support device 3 for supporting construction for fitting a member B forming a building facility K to an architectural structure, comprising: a construction plan information storage part 11 for storing a three-dimensional model; a three-dimensional shape data generation part 22b for generating three-dimensional shape data with advance of a work using a photographic image imaged by a camera 2 fitted to an operator S; a matching processing part 24a for comparing the three-dimensional model with the three-dimensional shape data at a specific time, for determining whether or not there is a newly matched fitting completion member; and a first association processing part 24b for, if there is the newly matched fitting completion member, detecting a region corresponding to the fitting completion member on the three-dimensional shape data at a previous time of the specific time, and associating fitting completion member identification information for identifying the fitting completion member to at least any one of the detected region of the three-dimensional shape data and the photographic image in which the region is imaged.SELECTED DRAWING: Figure 1

Description

The present invention relates to a construction support system and a construction support device.

In recent years, when constructing building equipment (for example, air conditioning ducts, sanitary pipes, equipment, etc.), it is common to design using software such as CAD (Computer Aided Design) and BIM (Building Information Modeling). is there. By designing using these software, the same three-dimensional model (three-dimensional model) as the real thing can be reproduced on a computer.

Conventionally, it has been considered to use a three-dimensional model for construction of building equipment and maintenance after completion, and related technologies include, for example, the following.
Patent Document 1 describes a method for installing a new member in an existing part of a building. In the technique of Patent Document 1, a three-dimensional polygon model is created from the point cloud data acquired by the three-dimensional laser scanner, and the construction position of the member is calculated by arranging the member element to be newly constructed on the three-dimensional polygon model. To do. Then, the construction position calculated by the visible laser irradiation is pointed out, and the member is attached to the indicated position.

Further, Patent Document 2 describes a method of recording a construction state of a member at a construction site in association with design data. In the technique of Patent Document 2, the attribute data changed, deleted, and added at the construction site and the image data of the members are sent by e-mail to the CAD device, and the information is imported into the CAD device to record the construction state (for example). , Paragraphs [0069] to [0071]).

Japanese Unexamined Patent Publication No. 2013-149119 Japanese Unexamined Patent Publication No. 2012-226525

However, in the conventional technique, there has not been proposed a technique capable of managing members and workers in association with the construction process (carry-in-temporary placement-installation, etc.) at the construction site. Therefore, for example, there are the following problems.
First, if traceability (traceback back to the construction process) is not possible for each member that composes the construction object and a construction error is discovered after completion, the worker or cause responsible for the construction of the member can be identified. It could not be identified early.
Secondly, it was not possible to judge whether or not the workers were able to perform efficient work.
As a result, the conventional technology does not provide sufficient support for construction management, and is not always convenient for managers and workers.

From this point of view, the present invention provides a construction support system and a construction support device that can improve the convenience of construction management.

In order to solve the above problems, the construction support system according to the present invention is a construction support system that supports construction in which members constituting building equipment are attached to a building.
This construction support system includes a camera attached to the worker and a construction support device that supports the construction, and the camera captures the work contents of the worker who works in the work space.
The construction support device includes a storage unit, a three-dimensional shape data generation unit, a matching processing unit, and a first associating processing unit.
The storage unit stores a three-dimensional model which is information indicating a state in the work space after the member is attached.
The three-dimensional shape data generation unit uses the captured image taken by the camera to generate three-dimensional shape data in which the work space is three-dimensionally shaped as the work progresses.
The matching processing unit compares the three-dimensional model with the three-dimensional shape data at a specific time, and determines whether or not there is a newly matched mounting complete member.
When there is a newly matched attachment complete member, the first association processing unit detects a region corresponding to the attachment complete member in the three-dimensional shape data earlier than the specific time, and detects the three-dimensional shape. The attachment completed member identification information that identifies the attachment completed member is associated with at least one of the shape data area and the captured image in which the area is captured.

Further, the construction support device according to the present invention is a construction support device that supports the construction of attaching the members constituting the building equipment to the building.
This construction support device includes a storage unit, a three-dimensional shape data generation unit, a matching processing unit, and a first associating processing unit.
The storage unit stores a three-dimensional model which is information indicating a state in the work space after the member is attached.
The three-dimensional shape data generation unit uses a photographed image taken by a camera mounted on the worker to generate three-dimensional shape data obtained by shaping the work space into a three-dimensional shape as the work progresses.
The matching processing unit compares the three-dimensional model with the three-dimensional shape data at a specific time, and determines whether or not there is a newly matched mounting complete member.
When there is a newly matched attachment complete member, the first association processing unit detects a region corresponding to the attachment complete member in the three-dimensional shape data earlier than the specific time, and detects the three-dimensional shape. The attachment completed member identification information that identifies the attachment completed member is associated with at least one of the shape data area and the captured image in which the area is captured.

In the construction support system and the construction support device according to the present invention, by searching the past three-dimensional shape data and the photographed image of the member using the installation completed member identification information, the three-dimensional during construction of the member after the work is completed. You can check the shape data and captured images. Therefore, when a construction error is discovered after completion, the worker in charge of construction of the member and the cause can be identified at an early stage.

The configuration may further include a movement route calculation unit for calculating the position of the worker with the passage of time and a second association processing unit.
When there is a newly matched attachment complete member, the second association processing unit detects a region corresponding to the attachment complete member in the three-dimensional shape data earlier than the specific time, and detects the tertiary. The attachment completed member identification information is associated with the movement path of the worker who is around the area of the original shape data.

In this way, it is possible to confirm the movement route of the member during construction after the work is completed by searching using the attachment completed member identification information. Therefore, the worker in charge of the construction of the member and the cause can be identified at an early stage.

The movement path calculation unit receives information obtained by performing three-dimensional composition processing (eg, SfM (Structure from Motion)) of the captured image by photogrammetry technology or information output from a three-dimensional laser scanner separately attached by the worker. It may be calculated using the information processed by the point cloud.

The member identification information that can identify the member is described in the member, and the matching processing unit reads and reads the member identification information included in the photographed image or the three-dimensional shape data. The information should be used to identify the comparison target. By doing so, it is possible to reduce mistakes in the matching process and the linking process.

The member identification information that can identify the member is displayed on the member, and the matching processing unit reads and reads the member identification information included in the photographed image or the three-dimensional shape data to identify the member. The information may be used to correct the position of the three-dimensional shape data. In this way, the accuracy of comparing the 3D model and the 3D shape data by the matching process is improved.

Further, the construction support system according to the present invention is a construction support system that supports the construction of attaching the members constituting the building equipment to the building.
This construction support system includes a camera attached to the worker and a construction support device that supports the construction, and the camera captures the work contents of the worker who works in the work space.
The construction support device includes a three-dimensional shape data generation unit and a movement route calculation unit.
The three-dimensional shape data generation unit uses the captured image taken by the camera to generate three-dimensional shape data in which the work space is three-dimensionally shaped as the work progresses.
The movement route calculation unit calculates the position of the worker from the three-dimensional shape data by using the three-dimensional composition processing by the photogrammetry technique.

In this way, it is possible to identify the movement route of the worker during construction. Therefore, for example, it is possible to compare the movement route of an excellent worker and the movement route of a poor worker, and it is possible to analyze the work efficiency.

Further, the construction support system according to the present invention is a construction support system that supports the construction of attaching the members constituting the building equipment to the building.
This construction support system includes a camera attached to the worker and a construction support device that supports the construction, and the camera captures the work contents of the worker who works in the work space.
The construction support device includes a storage unit, a three-dimensional shape data generation unit, a matching processing unit, a third association processing unit, and a learning data collection unit.
The storage unit stores a three-dimensional model which is information indicating a state in the work space after the member is attached.
The three-dimensional shape data generation unit uses the captured image taken by the camera to generate three-dimensional shape data in which the work space is three-dimensionally shaped as the work progresses.
The matching processing unit compares the three-dimensional model with the three-dimensional shape data at a specific time, and determines whether or not there is a newly matched mounting complete member.
When there is a newly matched attachment completion member, the third association processing unit detects a region corresponding to the attachment completion member in the three-dimensional shape data earlier than the specific time, and detects the three-dimensional shape. The classification information for grouping the mounting completed members is associated with the captured image in which the shape data area is captured.
The learning data collection unit collects the captured image for each of the classification information.

In this way, data for machine learning can be easily collected.

Further, the construction support system according to the present invention is a construction support system that supports the construction of attaching the members constituting the building equipment to the building.
This construction support system includes a three-dimensional laser scanner mounted on a worker and a construction support device that supports the construction, and the three-dimensional laser scanner acquires point cloud data in the work space.
The construction support device includes a storage unit, a three-dimensional shape data generation unit, a matching processing unit, and a first associating processing unit.
The storage unit stores a three-dimensional model which is information indicating a state in the work space after the member is attached.
The three-dimensional shape data generation unit uses the point cloud data acquired by the three-dimensional laser scanner to generate three-dimensional shape data obtained by shaping the work space into a three-dimensional shape as the work progresses.
The matching processing unit compares the three-dimensional model with the three-dimensional shape data at a specific time, and determines whether or not there is a newly matched mounting complete member.
When there is a newly matched attachment complete member, the first association processing unit detects a region corresponding to the attachment complete member in the three-dimensional shape data earlier than the specific time, and detects the three-dimensional shape. The mounting completed member identification information that identifies the mounting completed member is linked to the shape data area.

In the construction support system according to the present invention, it is possible to confirm the three-dimensional shape data during construction of the member after the work is completed by searching the past three-dimensional shape data of the member using the mounting completed member identification information. it can. Therefore, when a construction error is discovered after completion, the worker in charge of construction of the member and the cause can be identified at an early stage.

According to the present invention, the convenience of construction management can be improved. The effects of one aspect of the present invention are shown below.
According to one aspect of the present invention, the position information of the camera (that is, the position information of the worker who wears the camera) is acquired from the three-dimensional shape data even indoors where the radio wave from GPS (Global Positioning System) hygiene does not reach. be able to. As a result, the work analysis (flow line analysis) of the worker can be performed and utilized for improving the construction efficiency.
In addition, by accumulating this three-dimensional shape data over time during construction, it becomes possible to record various construction processes at the construction site. As a result, for example, traceability of the construction process becomes possible, and it can be used for validation (recording of the construction process for construction quality control). In addition, the cause of the defect can be found by analyzing the construction process data accumulated when a defect occurs after the construction.

It is a schematic block diagram of the construction support system which concerns on 1st Embodiment of this invention. It is an image diagram of a work space where a worker constructs a building facility. It is a flowchart which shows the operation (registration process of construction information) of the construction support system which concerns on 1st Embodiment of this invention. It is an image diagram of a three-dimensional model. It is a figure which shows the construction work of a building facility. It is a figure which shows the construction work of a building facility. It is a figure for demonstrating the associating process. It is a figure which shows the construction work of a building facility. It is a figure which shows the construction work of a building facility. It is a schematic block diagram of the construction support system which concerns on 2nd Embodiment of this invention.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings as appropriate. Each figure is only schematically shown to the extent that the present invention can be fully understood. Therefore, the present invention is not limited to the illustrated examples. Further, in the embodiment, description may be omitted for a configuration that is not directly related to the present invention or a well-known configuration. Further, in each figure, common components and similar components are designated by the same reference numerals, and duplicate description thereof will be omitted.

[First Embodiment]
<Structure of construction support system according to the first embodiment>
The configuration of the construction support system 1 according to the first embodiment will be described with reference to FIG. The construction support system 1 is a system that supports the construction of building equipment, the recording of the construction process, and the maintenance after completion. Building equipment refers to various types of equipment provided in a building, such as an air conditioning duct. In this embodiment, it is assumed that the building equipment is attached to the frame of the building.

The construction support system 1 shown in FIG. 1 is a construction support that executes various processes using a camera 2 mounted on a worker S in charge of construction of building equipment K and an image (or an image) taken by the camera 2. It mainly includes a device 3. The number of cameras 2 (that is, the number of workers S) is not particularly limited, and for example, a plurality of cameras 2 may exist in the work space. The camera 2 and the construction support device 3 can always or temporarily communicate with each other via some kind of communication means (for example, a wireless LAN (Local Area Network) or a communication cable). The image captured by the camera 2 is recorded on a computer-readable recording medium (for example, a memory card), and the recording medium is attached to the construction support device 3 as needed to display the image on the construction support device 3. You may import it.

The camera 2 photographs the work performed by the worker S regarding the building equipment K, for example, an object mounted on the head of the worker S and in the front direction (line-of-sight direction) of the worker S ( See FIG. 2). The camera 2 here has a resolution capable of recognizing the shape of the building equipment K and the tag T attached to each part (hereinafter, referred to as “member B”) constituting the building equipment K. On the tag T, member identification information for identifying the member B by means such as a bar code, a QR code (registered trademark), and a handwritten character string is described. The type and shape of the tag T are not particularly limited as long as they can be attached to the member B. The camera 2 is, for example, a digital video camera, and captures several tens of images (also referred to as frames) per second.

The construction support device 3 is a device that supports the construction and maintenance of building equipment after completion. The construction support device 3 is, for example, a management personal computer (PC) installed in the management room of the construction site, or a server located away from the construction site. The construction support device 3 may be a part of a cloud server or a cloud system. The construction support device 3 mainly includes a storage means 10 and a control means 20. Although the details will be described later, the storage means 10 stores the information obtained by performing the three-dimensional composition processing by the photogrammetry technique. In the present embodiment, SfM (Structure from Motion) is illustrated as an example of the three-dimensional composition processing by the photogrammetry technique, and the SfM information storage unit 13 is an example of the storage unit that stores the information obtained by performing the three-dimensional composition processing. Is illustrated.

The storage means 10 is composed of, for example, a storage medium such as a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), and a flash memory.
The storage means 10 stores information necessary for processing executed by the construction support device 3.
The storage means 10 here mainly includes a construction plan information storage unit 11, a captured image storage unit 12, an SfM information storage unit 13, a worker information storage unit 14, and a member information storage unit 15.

Information about the construction plan is stored in the construction plan information storage unit 11. The information stored in the construction plan information storage unit 11 includes a three-dimensional model showing the state after the construction is completed. The three-dimensional model is created using software such as CAD and BIM. A three-dimensional model is a collection of elements (also called "objects"). Assuming the construction of installing the air conditioning duct, each element is, for example, each part (for example, a straight pipe or an elbow) constituting the air conditioning duct. For example, attribute information (element information) such as an element ID (Identification) for identifying an element, a three-dimensional shape, a product number, a dimension, a material, and a position is registered in each element.

The captured image storage unit 12 stores an image captured by the camera 2. The captured image storage unit 12 stores, for example, an image acquired from the camera 2 in real time. Attribute information (shooting information) such as a camera ID for identifying the camera 2, a shooting time, and a shooting position is registered in each image (frame) constituting the video. When the shooting location of the camera 2 is indoors, the GPS (Global Positioning System) receiver generally provided in the camera 2 is GPS.
Since the radio waves transmitted from the satellite cannot be received, the shooting position cannot be specified. Since the construction of building equipment is generally performed indoors, it will be described below assuming that the shooting position cannot be specified using GPS. The captured image storage unit 12 may store information (angle of view, resolution, frame rate, etc.) of the camera 2 as information other than video.

The SfM information storage unit 13 stores information obtained by SfM processing the image stored in the captured image storage unit 12. SfM is a method of simultaneously restoring the geometric shape of an object reflected in an image and the movement of the camera (camera position in time series). The information stored in the SfM information storage unit 13 includes the three-dimensional shape data of the object (here, each part of the building equipment K in particular) reflected in the image and the movement of the camera 2.

The worker information storage unit 14 stores information about the worker S who is in charge of the construction of the building equipment K. The information stored in the worker information storage unit 14 includes information such as a worker ID for identifying the worker S, a camera ID of the camera 2 worn by the worker S, and a movement route of the worker S. The movement path of the worker S corresponds to the movement of the camera 2 calculated by the SfM technique with the worker S.

The member information storage unit 15 stores information about each part (member B) constituting the building equipment K. The information stored in the member information storage unit 15 includes information such as a product number, dimensions, and material of the member B.

The control means 20 is realized by, for example, a program execution process by a CPU (Central Processing Unit), a dedicated circuit, or the like. When the control means 20 is realized by a program, the program can be stored and provided on a computer-readable recording medium (eg, CD-ROM). The program can also be provided through a network such as the Internet.

The control means 20 includes an image acquisition unit 21, an SfM processing unit 22, a movement route calculation unit 23, a linking processing unit 24, a tag processing unit 25, a display control unit 26, and a learning data collection unit 27. , Is mainly provided.

The image acquisition unit 21 is a function of acquiring an image captured by the camera 2 of the worker S. The image acquired by the image acquisition unit 21 may be such that the shape of the member B and the movement of the camera 2 can be calculated by the SfM process described later. The time when the image acquisition unit 21 acquires the image of the camera 2 is not particularly limited, and the image may be acquired in real time, or the images of the construction taken after the construction is completed may be collectively acquired. The image acquisition unit 21 should exclude images (frames) that are not suitable for SfM processing. The unsuitable image (frame) is, for example, a blurred image when the worker S vigorously moves his head, or an image that is too close to determine the shape of the object to be photographed.

The SfM processing unit 22 is a function of performing SfM processing on the image captured by the camera 2.
The SfM processing unit 22 includes a camera position / orientation estimation unit 22a and a three-dimensional shape data generation unit 22b. Since the SfM technique is a known technique, detailed description thereof will be omitted here.

The camera position / posture estimation unit 22a estimates the position and posture of the camera 2 from the acquired image as the work progresses. Here, "estimate with the progress of work" includes a case where the position and posture of the camera 2 are continuously estimated with the passage of time, and a case where the position and posture of the camera 2 are discretely estimated for each event. It is an intention.

The three-dimensional shape data generation unit 22b generates three-dimensional shape data of the object (here, in particular, each part of the building equipment) reflected in the acquired image as the work progresses. Here, "generated with the progress of work" is intended to include a case where three-dimensional shape data is continuously generated with the passage of time and a case where three-dimensional shape data is discretely generated for each event.

The movement route calculation unit 23 associates the camera 2 with the worker S, and calculates the movement of the camera 2 calculated by the SfM technique as the movement route of the worker S.

The association processing unit 24 is a function of retroactively associating information with the member B when the member B is newly attached to the building. The tying processing unit 24 has a matching processing unit 24a, a first tying processing unit 24b, a second tying processing unit 24c, and a third tying processing unit 24d.

The matching processing unit 24a determines whether or not there is a member B newly attached to the building. This determination method by the matching processing unit 24a is not particularly limited. For example, the matching processing unit 24a compares the three-dimensional model showing the state after the completion of construction with the latest three-dimensional shape data, and when some object is arranged at the position corresponding to the element, the member B is newly added. Judge that it is attached. Further, at the timing of the completion of the mounting of the member B, the worker S operates a mobile terminal (not shown) to input information indicating the completion of mounting, or gives a signal in front of the camera 2, and the matching processing unit 24a thereof It may be determined that the member B is newly attached by detecting the information or the signal.

Further, the matching processing unit 24a determines whether or not the member B is attached as designed by comparing the shape and volume of the corresponding element with the shape and volume of the corresponding portion of the three-dimensional shape data. To do. When the member B is not attached as designed, the matching processing unit 24a identifies the worker S around the attached member B as the builder who performed the construction of the member B, and notifies (not shown). Notify the contractor or supervisor of the construction error by using means. Regarding the determination of whether or not the members are attached as designed, the 3D model and the 3D shape data may not completely match depending on the shooting conditions by the camera 2 and the accuracy of generating the 3D shape data. is assumed. In that case, for example, an allowable value may be provided, and if the deviation between the three-dimensional model and the three-dimensional shape data is within a predetermined range, it may be determined that the member B is attached as designed.

The first associating processing unit 24b associates each member B constituting the building equipment K with the three-dimensional shape data and the photographed image. Specifically, the first associative processing unit 24b refers to the three-dimensional shape data in the past (may include the time when the member B is attached) from the time when the member B is newly attached to the building, and each of them is referred to. A region corresponding to the member B of the three-dimensional shape data is extracted. The first association processing unit 24b extracts a region corresponding to the member B based on, for example, a shape or a color. The first associative processing unit 24b acquires information (for example, three-dimensional shape, product number, dimensions, material, etc.) regarding the attached member B with reference to the construction plan information storage unit 11, and the acquired information. The region may be extracted from the three-dimensional shape data using. Then, the first link processing unit 2
In 4b, information (for example, element ID) for identifying the member B is associated with the extracted region. Further, the first associating processing unit 24b associates information (for example, element ID) for identifying the member B with the photographed image in which the extracted area is photographed. The first associative processing unit 24b may have an image recognition model that recognizes the member B from the captured image, and specifies a captured image in which a region extracted using this image recognition model is captured. May be good. Further, the first association processing unit 24b may associate information other than the information for identifying the member B (for example, the element ID) with the three-dimensional shape data or the captured image. In addition, the information for identifying the attached member B may be particularly referred to as "attachment completed member identification information".

The second associating processing unit 24c associates each member B constituting the building equipment K with the movement path of the worker S. Specifically, the second association processing unit 24c extracts a region corresponding to the member B from the past three-dimensional shape data by the same method as the first association processing unit 24b. Then, the second associating processing unit 24c conveyed the member B to the worker S (the worker S who attaches the camera 2 at a position close to the extracted area) around the extracted area. It is identified as a worker, and information (for example, element ID) that identifies the member B is associated with the movement path of the specified worker S. Here, when the member B is delivered, the information for identifying the member B is associated with the movement path of the different worker S with the time of delivery as the boundary.

The third associating processing unit 24d associates the classification information for grouping the member B with the photographed image. Specifically, the third association processing unit 24d extracts a region corresponding to the member B from the past three-dimensional shape data by the same method as the first association processing unit 24b. Then, the third associating processing unit 24d associates the photographed image in which the extracted region is photographed with the classification information (for example, the product number) for grouping the member B.

The tag processing unit 25 is a function of performing various processes using the tag T. The tag processing unit 25 has a member identification unit 25a and a position correction unit 25b.

The member identification unit 25a reads the member identification information (for example, element ID) described in the tag T from the image taken by the camera 2 or the three-dimensional shape data, and is included in the member B or the three-dimensional shape data displayed in the image. Identify member B. Since the technique related to reading the tag T (character recognition technique) is a known technique, detailed description thereof will be omitted here. The member identification information specified by the member identification unit 25a is used for associating the information by the association processing unit 24. For example, the matching processing unit 24a identifies the comparison target, and the first association processing unit 24b, the second association processing unit 24c, and the third association processing unit 24d extract the region of the member B and select the captured image. Used for. That is, the first association processing unit 24b, the second association processing unit 24c, and the third association processing unit 24d use the member identification information specified by the tag T to obtain the member B from the three-dimensional shape data and the captured image. May be extracted. As a result, mistakes in the matching process and the linking process can be reduced.

The position correction unit 25b has an image or a three-dimensional shape of a tag T (hereinafter, may be referred to as a “reference tag T”) attached to a fixed point (for example, a sleeve provided on a wall or the like) taken by the camera 2. Read from the data and set a reference point at the relevant position in the 3D shape data. Here, the position where the reference tag T is attached is also set as the reference point in the three-dimensional model showing the state after the completion of construction, and the position correction unit 25b sets the reference point of the three-dimensional model and the three-dimensional shape data. By aligning with the reference point of, the position of the three-dimensional shape data is corrected. As a result, the accuracy of comparing the three-dimensional model and the three-dimensional shape data by the matching processing unit 24a is improved. The position correction unit 25b may correct the position of the three-dimensional shape data by using the tag T attached to the member B.

The display control unit 26 displays various information regarding the construction of the building equipment K. Display control unit 26
Displays, for example, switching the three-dimensional shape data to the figure below. In addition, the display control unit 26 displays information on the progress of work and the finished product. Further, the display control unit 26 emphasizes and displays the inconsistent portion (member) when the member B is not attached as designed, and also displays the time when the construction error occurred and the worker who performed the work. .. Further, the display control unit 26 has a linking information display unit 26a for displaying information and the like linked by the linking processing unit 24.

The association information display unit 26a receives, for example, input of information for identifying the member B (for example, an element ID), and extracts three-dimensional shape data including the input member B and a captured image as a basis thereof. indicate. Further, the association information display unit 26a displays the input movement path of the member B.

Further, the association information display unit 26a accepts, for example, input of information for identifying the worker S (for example, a worker ID) and displays the input movement route of the worker S. The movement routes of the plurality of workers S may be displayed side by side or overlapped on one screen so that the movement routes of the workers S can be compared. Thereby, for example, it is possible to compare the movement route of the excellent worker S and the movement route of the poor worker S, and it is possible to analyze the work efficiency. Further, the linking information display unit 26a may display the time required for attaching the member B, the amount of construction work in a predetermined time, and the like for each worker S.

The learning data collection unit 27 is a function of collecting data for machine learning. Here, the third associating processing unit 24d associates the photographed image with classification information (for example, a product number) for grouping the members. The learning data collection unit 27 receives, for example, input of classification information (for example, a product number) for grouping member B, and collects captured images based on the information. The collected captured image can be used as, for example, supervised learning data, and it can be expected that the automatic recognition ability of the member B will be improved by performing learning using the learning data.

<Operation of construction support system according to the first embodiment>
The operation (registration process of construction information) of the construction support system 1 according to the first embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing the operation (registration process of construction information) of the construction support system according to the first embodiment.

(Registration process of construction information)
First, the image acquisition unit 21 of the construction support device 3 acquires an image taken by the camera 2 of the worker S (step S1). The image acquisition unit 21 excludes images (unnecessary images) that are not suitable for SfM processing (step S2).

Next, the three-dimensional shape data generation unit 22b generates three-dimensional shape data of the object (here, in particular, the member B constituting the building equipment K) reflected in the image by SfM processing as the work progresses (step). S3). Further, the camera position / posture estimation unit 22a estimates the position and posture of the camera 2 as the work progresses by using the three-dimensional shape data acquired by the SfM processing (step S3). Then, the camera position / posture estimation unit 22a registers the estimated position (movement) of the camera 2 as the position (movement path) of the worker S in the worker information storage unit 14 (step S4). Further, the three-dimensional shape data generation unit 22b registers the generated three-dimensional shape data in the SfM information storage unit 13 (step S4). The timing of estimating the camera position / orientation and the timing of generating the three-dimensional shape data may be different. Further, the number of times the camera position / orientation is estimated and the number of times the three-dimensional shape data is generated within a predetermined period may be different.

Next, the matching processing unit 24a determines whether or not a new member B is attached to the building (step S5). When the new member B is not attached (“No” in step S5), the control means 20 returns the process to step S1 and continues to acquire the image. On the other hand, when a new member is attached (“Yes” in step S5), the matching processing unit 24a determines whether or not the member B is attached as designed (step S6). The matching processing unit 24a compares, for example, the three-dimensional model with the three-dimensional shape data, and determines whether or not the design is correct depending on whether or not the shapes and volumes of the corresponding portions match.

When the member B is not attached as designed (“No” in step S6), the matching processing unit 24a is a builder who constructs the member S around the attached member B. (Step S7), the worker S and the supervisor are notified of the construction error by using a notification means (not shown). On the other hand, when the member B is attached as designed (“Yes” in step S6), information is associated retroactively in steps S8 to S10. For example, the first linking processing unit 24b performs the first linking process of associating the information (for example, the element ID) that identifies the member B constituting the building equipment K with the three-dimensional shape data or the captured image (step S8). .. In addition, the second linking processing unit 24c performs the second linking process of associating the information (for example, the element ID) that identifies the member B constituting the building equipment K with the movement path of the worker S (step S9). Further, the third linking processing unit 24d performs a third linking process of associating the classification information for grouping the member B with the captured image (step S10).

Next, the matching processing unit 24a determines whether or not all the members B are attached to the building (step S11). When all the members B are not attached (“No” in step S11), the control means 20 returns the process to step S1 and continues to acquire an image. On the other hand, when all the members B are attached (“Yes” in step S11), the process ends.

[Example]
As shown in FIG. 2, the operation of the construction support system according to the first embodiment will be described by taking as an example the construction in which three ducts Ba, Bb, and Bc are attached to the duct B0. In FIG. 2, the state after mounting is represented by a virtual line. The three-dimensional model E in the construction of FIG. 2 is shown in FIG. The three-dimensional model E has an element e1 corresponding to the duct B0 (see FIG. 2) and elements e2, e3, e4 corresponding to the ducts Ba, Bb, Bc (see FIG. 2).

As shown in FIG. 5, it is assumed that the worker S1 who was on the right side of the paper at the time t1 moved to the left side of the paper and moved near the duct Ba at the time t3. Further, as shown in FIG. 6, it is assumed that the worker S1 lifts the duct Ba at the time t4 and attaches the duct Ba to the duct B0 at the time t6.

It is assumed that the captured image of the camera 2 in this case, the generated three-dimensional shape data, and the comparison result between the three-dimensional model and the three-dimensional shape data are as shown in FIG. As shown in FIG. 7, the duct Ba is not reflected in the captured images F11 and F12 at times t1 and t2, and the duct Ba is reflected in the captured images F13 to F16 at times t3 to t6. Further, the three-dimensional shape data G1 at times t1 and t2 does not include the duct Ba, and the three-dimensional shape data G3 to G6 at times t3 to t6 include the duct Ba. Further, as for the comparison result between the three-dimensional model and the three-dimensional shape data, the comparison result of the element e2 does not match at the times t1 to t5 before the duct Ba is attached, and the time t6 when the duct Ba is attached is the time t6 of the element e2. The comparison results are in agreement.

In this case, the first associating processing unit 24b refers to the three-dimensional shape data G1 to G6 before the time t6 when the duct Ba is attached to the duct B0 (including the time t6), and each three-dimensional shape data. The region corresponding to the duct Ba is extracted from G1 to G6. Here, since the duct Ba is included in the three-dimensional shape data G3 to G6, the corresponding region of the three-dimensional shape data G3 to G6 is extracted. Then, the first associating processing unit 24b associates the information (element ID “e2”) for identifying the duct Ba with the region extracted from the three-dimensional shape data G3 to G6. Further, the first association processing unit 24b associates information (element ID “e2”) for identifying the duct Ba with the photographed images F13 to F16 in which the extracted area (that is, the duct Ba) is photographed.

Further, the second tying processing unit 24c extracts a region corresponding to the duct Ba from the past three-dimensional shape data G1 to G6 by the same method as the first tying processing unit 24b. Then, the second associating processing unit 24c identifies the worker S1 around the extracted region of the three-dimensional shape data G3 to G6 as the worker who carried the duct Ba, and identifies the worker S1. The information for identifying the duct Ba (element ID "e2") is associated with the movement path of.

Further, the third association processing unit 24d extracts a region corresponding to the duct Ba from the past three-dimensional shape data G1 to G6 by the same method as the first association processing unit 24b. Then, the third linking processing unit 24d links the classification information (for example, "air conditioning duct") for grouping the ducts Ba into the captured images F13 to F16 in which the extracted regions of the three-dimensional shape data G3 to G6 are photographed. wear.

After that, as shown in FIG. 8, when the worker S1 attaches the duct Bb to the duct Ba, the process of associating the information about the duct Bb with the past three-dimensional shape data and the captured image is performed. Further, in the same manner, as shown in FIG. 9, when the worker S1 attaches the duct Bc to the duct Bb, the process of associating the information about the duct Bc with the past three-dimensional shape data and the captured image is performed.

As described above, the construction support system 1 according to the first embodiment is constructing the member B after the work is completed by searching the past three-dimensional shape data and the photographed image using the information for identifying the member B. You can check the 3D shape data and captured images of.
Further, the construction support system 1 according to the first embodiment can confirm the movement route of the member B during construction after the work is completed by searching using the information for identifying the member B.
Therefore, when a construction error is discovered after completion, the worker S in charge of the construction of the member B and the cause can be identified at an early stage.

[Second Embodiment]
<Structure of construction support system according to the second embodiment>
The configuration of the construction support system 101 according to the second embodiment will be described with reference to FIG. The construction support system 101 is changed to the three-dimensional laser scanner 102 to be attached to the worker S. Further, in the construction support device 103, the storage means 10 includes the scanner information storage unit 113, and the control means 20 includes the scanner information processing unit 122.

The three-dimensional laser scanner 102 irradiates the object (here, each part of the building equipment in particular) with the laser beam, and the object around the worker S is determined by the time when the reflected light returns and the laser irradiation angle. Acquire 3D coordinates (point cloud data).

The scanner information storage unit 113 stores information processed by processing the three-dimensional coordinates (point cloud data) of the object acquired by the three-dimensional laser scanner 102. The information stored in the scanner information storage unit 113 includes three-dimensional shape data of an object (here, each part of a building facility in particular) around the worker S and the movement of the three-dimensional laser scanner 102.

The scanner information processing unit 122 is a function of processing the three-dimensional coordinates (point cloud data) of the object acquired by the three-dimensional laser scanner 102. The scanner information processing unit 122 has a scanner position / orientation estimation unit 122a and a three-dimensional shape data generation unit 122b.
The scanner position / orientation estimation unit 122a estimates the position and stillness of the three-dimensional laser scanner 102 from the acquired three-dimensional coordinates (point cloud data) of the object as the work progresses.
The three-dimensional shape data generation unit 122b generates three-dimensional shape data as the work progresses by performing point cloud processing on the acquired three-dimensional coordinates (point cloud data) of the object.

As described above, the construction support system 101 according to the second embodiment can also achieve substantially the same effect as that of the first embodiment.

[Modification example]
Although the embodiments of the present invention have been described above, the present invention is not limited to this, and can be carried out within a range that does not change the gist of the claims. Modifications of each embodiment are shown below, for example.

In the construction support system 1 of the first embodiment, a case where a video camera is assumed as the camera 2 and several tens of images (also referred to as frames) are taken per second has been illustrated. However, the shooting method by the camera 2 is not limited to this. For example, markers may be attached to both hands of the worker S, and automatic shooting (event-driven shooting) may be performed when the two markers attached to both hands of the worker S are AND-recognized. This shooting method is based on the idea that "both hands of the worker are reflected in the image" is when some work (event) is being performed. In this case, there is an advantage that the photographic data does not become too large.

Further, in the first embodiment, the first associating processing unit 24b associates the extracted area with the photographed image in which the member B is identified (for example, the element ID). However, the position information in which the extracted region is shown together with the information for identifying the member B (for example, the element ID) may be associated with the captured image. As a result, when a plurality of members B are shown in one captured image, it becomes easy to identify the corresponding member B.

It is also possible to combine the techniques of the construction support system 1 described in the first embodiment and the construction support system 101 described in the second embodiment. For example, the worker S may be equipped with both the camera 2 and the three-dimensional laser scanner 102.

1,101 Construction support system 2 Camera 3,103 Construction support device 11 Construction plan information storage unit 12 Photographed image storage unit 13 SfM information storage unit 14 Worker information storage unit 15 Member information storage unit 21 Image acquisition unit 22 SfM processing unit 22a Camera position / orientation estimation unit 22b 3D shape data generation unit 23 Movement path calculation unit 24 Linking processing unit 24a Matching processing unit 24b First linking processing unit 24c Second linking processing unit 24d Third linking processing unit 25 tags Processing unit 25a Member identification unit 25b Position correction unit 26 Display control unit 26a Linking information display unit 27 Learning data collection unit 102 Three-dimensional laser scanner

Claims (9)

It is a construction support system that supports the construction of attaching the members that make up the building equipment to the building.
The camera attached to the worker and
Equipped with a construction support device that supports the construction
The camera captures the work contents of the worker who works in the work space.
The construction support device is
A storage unit that stores a three-dimensional model that is information indicating a state in the work space after the member is attached, and a storage unit.
A three-dimensional shape data generation unit that generates three-dimensional shape data obtained by shaping the work space into a three-dimensional shape as the work progresses, using the captured image taken by the camera.
A matching processing unit that compares the three-dimensional model with the three-dimensional shape data at a specific time and determines whether or not there is a newly matched mounting completed member.
When there is a newly matched mounting completed member, a region corresponding to the mounting completed member in the three-dimensional shape data earlier than the specific time is detected, and the detected region of the three-dimensional shape data and the region are A first associating processing unit for associating attachment completion member identification information for identifying the attachment completion member with at least one of the captured images to be photographed is provided.
A construction support system characterized by this. A movement route calculation unit that calculates the position of the worker over time, and
When there is a newly matched attachment completed member, a region corresponding to the attachment completed member in the three-dimensional shape data earlier than the specific time is detected, and around the detected region of the three-dimensional shape data. It is provided with a second linking processing unit that links the mounting completed member identification information to the moving path of the worker.
The construction support system according to claim 1, wherein the construction support system is characterized in that. The movement path calculation unit calculates using information obtained by three-dimensionally synthesizing the captured image by photogrammetry technology or information output from a three-dimensional laser scanner separately attached by the worker, using point cloud processing information. ,
The construction support system according to claim 2, wherein the construction support system is characterized in that. Member identification information that can identify the member is described in the member.
The matching processing unit reads the member identification information included in the captured image or the three-dimensional shape data, and uses the read member identification information to identify the comparison target.
The construction support system according to any one of claims 1 to 3, wherein the construction support system is characterized in that. Member identification information that can identify the member is displayed on the member.
The matching processing unit reads the member identification information included in the captured image or the three-dimensional shape data, and uses the read member identification information to correct the position of the three-dimensional shape data.
The construction support system according to any one of claims 1 to 4, wherein the construction support system is characterized in that. It is a construction support device that supports the construction of attaching the members that make up the building equipment to the building.
A storage unit that stores a three-dimensional model, which is information indicating the state in the work space after the member is attached,
A three-dimensional shape data generation unit that generates three-dimensional shape data obtained by shaping the work space into a three-dimensional shape as the work progresses, using images taken by a camera attached to the worker.
A matching processing unit that compares the three-dimensional model with the three-dimensional shape data at a specific time and determines whether or not there is a newly matched mounting completed member.
When there is a newly matched mounting completed member, a region corresponding to the mounting completed member in the three-dimensional shape data earlier than the specific time is detected, and the detected region of the three-dimensional shape data and the region are A first associating processing unit for associating attachment completion member identification information for identifying the attachment completion member with at least one of the captured images to be photographed is provided.
A construction support device characterized by this. It is a construction support system that supports the construction of attaching the members that make up the building equipment to the building.
The camera attached to the worker and
Equipped with a construction support device that supports the construction
The camera captures the work contents of the worker who works in the work space.
The construction support device is
A three-dimensional shape data generation unit that generates three-dimensional shape data obtained by shaping the work space into a three-dimensional shape as the work progresses, using the captured image taken by the camera.
It is provided with a movement route calculation unit that calculates the position of the worker from the three-dimensional shape data by using a three-dimensional composition process by photogrammetry technology.
A construction support system characterized by this. It is a construction support system that supports the construction of attaching the members that make up the building equipment to the building.
The camera attached to the worker and
Equipped with a construction support device that supports the construction
The camera captures the work contents of the worker who works in the work space.
The construction support device is
A storage unit that stores a three-dimensional model that is information indicating a state in the work space after the member is attached, and a storage unit.
A three-dimensional shape data generation unit that generates three-dimensional shape data obtained by shaping the work space into a three-dimensional shape as the work progresses, using the captured image taken by the camera.
A matching processing unit that compares the three-dimensional model with the three-dimensional shape data at a specific time and determines whether or not there is a newly matched mounting completed member.
When there is a newly matched mounting completed member, a region corresponding to the mounting completed member in the three-dimensional shape data earlier than the specific time is detected, and the detected region of the three-dimensional shape data is photographed. A third linking processing unit that links the captured image with classification information that groups the mounting completed members,
A learning data collecting unit that collects the captured image for each of the classification information is provided.
A construction support system characterized by this. It is a construction support system that supports the construction of attaching the members that make up the building equipment to the building.
A 3D laser scanner mounted on the worker and
Equipped with a construction support device that supports the construction
The three-dimensional laser scanner acquires point cloud data in the work space and
The construction support device is
A storage unit that stores a three-dimensional model that is information indicating a state in the work space after the member is attached, and a storage unit.
Using the point cloud data acquired by the three-dimensional laser scanner, a three-dimensional shape data generation unit that generates three-dimensional shape data obtained by shaping the work space into three dimensions as the work progresses.
A matching processing unit that compares the three-dimensional model with the three-dimensional shape data at a specific time and determines whether or not there is a newly matched mounting completed member.
When there is a newly matched attachment complete member, a region corresponding to the attachment complete member in the three-dimensional shape data earlier than the specific time is detected, and the attachment completion is performed in the detected three-dimensional shape data region. A first linking processing unit for linking member identification information is provided.
A construction support system characterized by this.

Images