JP2021047715A - Learning support system, learning support method, and learning support program - Google Patents

Abstract

To provide a learning support system, a learning support method, and a learning support program for efficiently creating teacher data used for machine learning for determining a status in construction work or the like.SOLUTION: A support server 20 includes: a BIM information storage section 22 which stores design information of a structure constituted by a three-dimensional element model including construction elements associated with each step; a teacher information storage section 23 which stores teacher data used for machine learning; and a control section 21 which generates a prediction model for predicting a progress status of each step. The control section 21 sequentially arranges the three-dimensional model obtained by combining element models of the construction elements used in each step in a model space using the design information, generates a two-dimensional image obtained by photographing the three-dimensional model from a viewpoint in the model space and stores it in the teacher information storage section 23, and generates the prediction model for predicting the progress status for each step from the photographed image using the teacher data stored in the teacher information storage section 23.SELECTED DRAWING: Figure 1

Description

The present invention relates to a learning support system, a learning support method, and a learning support program for creating teacher data used for machine learning for determining a construction status and a progress status in a building work or the like.

At construction sites such as construction and civil engineering, photographs may be taken before, after, or during construction. These photographs are used to manage the progress of construction work and to confirm the construction status. Therefore, a technique for efficiently determining the progress of building work or the like is being studied (see, for example, Patent Document 1). In the technique disclosed in this document, the control unit of the support server identifies the architectural element included in the captured image recorded in the site image information recording unit. Next, the building element information storage unit is used to identify the process associated with the building element, and the specified process is output to the manager terminal.

Further, in machine learning, supervised learning using teacher data may be used. A technique for reducing the burden of creating such teacher data is also being studied (see, for example, Patent Document 2). In the technique disclosed in this document, the arithmetic unit of the video display system has a prediction model in which a plurality of results simulated for a plurality of states of the manufacturing apparatus are machine-learned as teacher data. Then, at least a part of the first state acquired by the information acquisition unit is input to the prediction model, the second state inside the manufacturing apparatus is calculated, and the video information is generated from the second state.

JP-A-2017-107443 JP-A-2018-169818

However, the materials and fittings used for construction differ depending on the construction site. Then, as the situation changes greatly according to the progress of the construction, it becomes a burden to prepare the teacher data. It is also possible to perform machine learning using images for each material and simulation images. However, it is difficult to determine the exact situation unless the materials used for construction and the temporarily placed materials can be distinguished from each other depending on the site conditions.

The learning support system for solving the above problems is used for machine learning and a design information storage unit that records design information of a structure composed of a three-dimensional element model including building elements associated with each process. It includes a teacher information storage unit that records teacher data and a control unit that generates a prediction model that predicts the progress of each process. Using the design information, the control unit sequentially arranges a three-dimensional model that combines element models of building elements used in each process in the model space, and from the viewpoint in the model space, the three-dimensional model. A two-dimensional image obtained by photographing the model is generated, recorded in the teacher information storage unit, and the teacher data recorded in the teacher information storage unit is used to predict a prediction model for predicting the progress of each process from the photographed image. Generate.

According to the present invention, teacher data used for machine learning for determining a situation in a building work or the like can be efficiently created.

Explanatory drawing of the system of this embodiment. The explanatory view of the hardware configuration of this embodiment. It is explanatory drawing of the data recorded in the storage part of this embodiment, (a) is a BIM information storage part, (b) is a teacher information storage part, (c) is a learning result storage part, (d) is a field. The image information storage unit, (e) is an explanatory diagram of the progress information storage unit. The explanatory view of the processing procedure of the teacher data generation processing of this embodiment. It is explanatory drawing of the processing procedure of this embodiment, (a) is the learning process, (b) is the explanatory diagram of the prediction process. It is explanatory drawing of the teacher data used in this embodiment, (a) is an image of the completion of wall foundation construction, (b) is an image of gypsum board construction, and (c) is an image of outputting material information of each part. Explanatory drawing. The explanatory view of the processing procedure of the generation processing of the surface model of another embodiment. Explanatory drawing of teacher data used in another embodiment, (a) is an image in which a material is temporarily placed in a first position, (b) is an image in which a material is temporarily placed in a second position, and (c) is a material. Is temporarily placed in the third position, (d) is an image in which the material is temporarily placed at the first angle, (e) is an image in which the material is temporarily placed at the second angle, and (f) is an image in which the material is temporarily placed at the third angle. Explanatory drawing of the temporarily placed image.

Hereinafter, one embodiment will be described with reference to FIGS. 1 to 6. In this embodiment, it will be described as a learning support system for determining the progress of the process at the construction site of the construction work. In this embodiment, teacher data is created from a CG image generated by using BIM (Building Information Modeling). In this BIM, it is assumed that attribute data such as cost, finish, and management information can be added to a digital model of a three-dimensional building created on a computer.
In this embodiment, as shown in FIG. 1, a user terminal 10 and a support server 20 are used.

(Explanation of hardware configuration)
The hardware configuration of the information processing apparatus H10 constituting the user terminal 10 and the support server 20 will be described with reference to FIG. The information processing device H10 includes a communication device H11, an input device H12, a display device H13, a storage unit H14, and a processor H15. This hardware configuration is an example, and can be realized by other hardware.

The communication device H11 is an interface that establishes a communication path with another device and executes data transmission / reception, such as a network interface card or a wireless interface.

The input device H12 is a device that receives input of various information, such as a mouse, a keyboard, a camera (photographing device), and the like. The display device H13 is a display or the like that displays various information.
The storage unit H14 is a storage device that stores data and various programs for executing various functions of the user terminal 10 and the support server 20. An example of the storage unit H14 is a ROM, a RAM, a hard disk, or the like.

The processor H15 controls each process in the user terminal 10 and the support server 20 by using the programs and data stored in the storage unit H14. Examples of the processor H15 include a CPU, an MPU, and the like. The processor H15 expands a program stored in a ROM or the like into a RAM and executes various processes for each process.

The processor H15 is not limited to the one that performs software processing for all the processing executed by itself. For example, the processor H15 may include a dedicated hardware circuit (for example, an integrated circuit for a specific application: ASIC) that performs hardware processing for at least a part of the processing executed by the processor H15. That is, the processor H15 is [1] one or more processors that operate according to a computer program (software), [2] one or more dedicated hardware circuits that execute at least a part of various processes, or [2] 3) It can be configured as a circuitry including a combination thereof. The processor includes a CPU and a memory such as a RAM and a ROM, and the memory stores a program code or an instruction configured to cause the CPU to execute a process. Memory or computer-readable media includes any available medium accessible by a general purpose or dedicated computer.

(System configuration)
Next, each function of the learning support system will be described with reference to FIG.
The user terminal 10 is a computer terminal provided with an imaging device used for photographing a construction site. The user terminal 10 is carried by the person in charge at the site and is used to photograph the site as needed in order to confirm the progress of the construction work. Then, the user terminal 10 uploads the captured image to the support server 20.

The support server 20 is a computer system that supports management of the progress of construction work by using captured images of the construction site. The support server 20 includes a control unit 21, a BIM information storage unit 22 (design information storage unit), a teacher information storage unit 23, a learning result storage unit 24, a site image information storage unit 25, and a progress information storage unit 26.

The control unit 21 performs a process described later (a process including a modeling stage, a teacher data generation stage, a learning stage, a prediction stage, a determination stage, and the like). By executing the program for each process for this purpose, the control unit 21 functions as a modeling unit 210, a teacher data generation unit 211, a learning unit 212, a prediction unit 213, a determination unit 214, and the like.

The modeling unit 210 executes a process of generating a three-dimensional model of the structure using the BIM data.
The teacher data generation unit 211 uses the modeling unit 210 to execute a process of generating teacher data to be used for machine learning.

The learning unit 212 executes a process of generating a prediction model for predicting the architectural elements included in the captured image by machine learning using the teacher data.
The prediction unit 213 executes a process of predicting the architectural element included in the captured image by using the prediction model.
The determination unit 214 executes a process of determining the progress (process) of the construction using the predicted building element.

As shown in FIG. 3A, the BIM information storage unit 22 records the BIM data 220 created by the BIM. This BIM data 220 is recorded when a building is designed using three-dimensional CAD. The BIM data 220 includes a project, an element model, and an arrangement. Further, in the present embodiment, information on attributes, processes, and scheduled dates is included in association with the element model.

The project information includes information on the name of the building, longitude / latitude, building orientation, and the like.
The element model information is information about a three-dimensional model (BIM object) of each building element used in a building.
The placement information includes information about the coordinates at which each element model is placed.
The attribute information includes information about the attributes of each element model. This attribute includes information about the element type, quantity, architectural element, and the like. Element type information includes information that identifies walls, slabs, columns, beams, roofs, stairs, zones, and the like. The quantity information includes information on length, area, volume and the like. The architectural element information includes information for identifying reinforcing bars, concrete, urethane, light iron, gypsum board, floor panels, cloth, fittings, flooring and the like. With the architectural element information, it is possible to display the texture such as the visual color, the homogeneity of brightness, and the unevenness of the surface of the material of the architectural element.

The process information includes information for identifying the process using the building element. This step includes, for example, "before concrete placement", "before heat insulation", "after heat insulation", "before board pasting", "after board pasting", "finishing completion" and the like. And, depending on the process, the architectural elements that can be confirmed at the site differ. For example, "before concrete placement", the reinforcing bars can be visually recognized. Concrete can be visually recognized if it is "before heat insulation", and urethane can be visually recognized if it is "after heat insulation".
Scheduled date information includes information about the scheduled date of this process.

As shown in FIG. 3B, the teacher information storage unit 23 records the teacher data 230 used for machine learning. This teacher data 230 is recorded when the teacher data creation process described later is executed. The teacher data 230 includes data on outdoor and indoor perspectives (teacher images generated by two-dimensional CG) of each place and each process generated by using BIM, and architectural elements contained therein. Will be done. Here, each building element included in the perspective is associated with the attribute information (building element information) included in the BIM data.

As shown in FIG. 3C, the learning result storage unit 24 records a prediction model 240 for predicting architectural elements included in the captured image. This prediction model 240 is recorded when machine learning is performed using the teacher data. The prediction model 240 is composed of a network (intermediate layer) that inputs a photographed image and outputs a ratio of building elements included in the photographed image.

As shown in FIG. 3D, the site image information storage unit 25 records image management data 250 for captured images of the construction site. The image management data 250 is recorded when a captured image is acquired from the user terminal 10. The image management data 250 is configured to include an image code, a shooting date and time, a place, and information about a shot image.

The image code information includes information regarding an identifier for identifying each photographed image of the construction site.
The shooting date / time information includes information on the date and time when the shot image was shot.
The location information includes information for identifying the location of the construction site where the captured image was taken.
The photographed image information includes a photographed image of the construction site.

As shown in FIG. 3E, progress management data 260 for managing the progress of construction work is recorded in the progress information storage unit 26. This progress management data 260 is recorded when the process determination process is executed. The progress management data 260 is configured to include information about the location, image code, process, and situation.
The location information includes an identifier for identifying each location on the construction site.
The image code information includes an identifier for identifying a captured image of this location.
The process information includes an identifier for identifying the specified process based on the captured image. This process information further includes information on the proportion of architectural elements contained in the captured image.
The status information includes information indicating the status of the construction work schedule for this process.

Next, with reference to FIGS. 4 to 6, a processing procedure for determining the progress of construction work on the support server 20 configured as described above will be described.
(Teacher data creation process)
First, the teacher data creation process will be described with reference to FIG.
Here, the control unit 21 of the support server 20 executes the BIM data acquisition process (step S1-1). Specifically, the teacher data generation unit 211 of the control unit 21 acquires the BIM data 220 of the building for which progress management is performed from the BIM information storage unit 22.

Next, the control unit 21 of the support server 20 sequentially specifies each process of the construction work as a processing target, and repeats the following processing.
The control unit 21 of the support server 20 executes the surface model generation process (step S1-2). Specifically, the teacher data generation unit 211 of the control unit 21 specifies the architectural element used up to the process to be processed in the BIM data. Then, the teacher data generation unit 211 instructs the modeling unit 210 to create a surface model of the building by using the three-dimensional model of all the specified building elements. In this case, the modeling unit 210 generates a three-dimensional model using the building elements associated with the processes up to the process to be processed and arranges them in the model space (three-dimensional space). For example, the architectural element that can be seen in the state of "wall base construction completed" is "concrete / LGS", and the architectural element that can be seen in the state of "plasterboard completed" is "concrete / gypsum board".

Here, the control unit 21 of the support server 20 executes the rendering condition specifying process (step S1-3). Specifically, the teacher data generation unit 211 of the control unit 21 specifies the scheduled date (month and day) of the BIM data 220 recorded in the BIM information storage unit 22. Next, the teacher data generation unit 211 identifies a plurality of time zones in which the person in charge may take a picture. The teacher data generation unit 211 specifies the position of the sun in the scheduled date and time zone, and specifies the sunshine condition for the building. In this case, an opening is specified in the surface model of the building, and the sunlight from this opening is used as a light source. Furthermore, if the installation of a luminaire is registered in the space, this luminaire is included as a light source. Then, the teacher data generation unit 211 specifies a plurality of rendering conditions using the specified light source.

Next, the control unit 21 of the support server 20 sequentially uses the specified plurality of rendering conditions to execute the rendering process (step S1-4). Specifically, the modeling unit 210 of the control unit 21 executes rendering of the three-dimensional model using the rendering conditions.

Next, the control unit 21 of the support server 20 executes the process of specifying the viewpoint position (step S1-5). Specifically, the teacher data generation unit 211 of the control unit 21 specifies a shooting position (viewpoint) in the model space. This viewpoint is set in consideration of the position where the person in charge takes a picture in the model space. For example, at a position in the model space where the person in charge can stand upright, at a position separated from the material by a predetermined distance, a plurality of viewpoint positions are specified by the height when standing upright. The viewpoint position may be a position where the building material can be seen, and is not limited to a plurality of viewpoint positions at a position separated by a predetermined distance and at a height when standing upright.

Then, the control unit 21 of the support server 20 repeats the following processing at each specified designated position.
The control unit 21 of the support server 20 executes a CG image generation process (step S1-6). Specifically, the teacher data generation unit 211 of the control unit 21 instructs the modeling unit 210 of the viewpoint position. In this case, the modeling unit 210 generates a perspective image (CG image) at the viewpoint position. This perspective image includes each architectural element (attribute information) included in the image.

The image 500 shown in FIG. 6A is a perspective generated in the process “completion of wall base construction”. In this image 500, a base material 501, a floor material 502, and a wall material 503 are arranged as building elements. Further, the image 510 shown in FIG. 6B is a perspective generated in the process “during gypsum board construction”. In this image 510, gypsum board 511 is arranged in addition to the base material 501, the floor material 502, and the wall material 503. The image 520 shown in FIG. 6 (c) includes the architectural elements included in the image 510. Specifically, the architectural element information 521 to 524 is set corresponding to the base material 501 to gypsum board 511. Then, the ratio of each architectural element can be calculated by using the area of each architectural element.

Next, the control unit 21 of the support server 20 executes the recording process of the teacher data (step S1-7). Specifically, the teacher data generation unit 211 of the control unit 21 acquires a perspective image from the modeling unit 210. Next, the teacher data generation unit 211 calculates the ratio of the area of each building element in the perspective image. Then, the teacher data generation unit 211 records the teacher data in which the area ratio (progress of the process) of each building element is set in association with the perspective image in the teacher information storage unit 23.
Then, the control unit 21 of the support server 20 repeats the above-described processing for each viewpoint position, rendering condition, and process.

(Learning process)
Next, the learning process will be described with reference to FIG. 5A.
First, the control unit 21 of the support server 20 executes the call processing of the teacher data (step S2-1). Specifically, the learning unit 212 of the control unit 21 acquires the teacher data 230 from the teacher information storage unit 23. The teacher data 230 records a ratio for each architectural element included in the perspective image.

Next, the control unit 21 of the support server 20 executes the machine learning process (step S2-2). Here, as machine learning, deep learning that connects a plurality of nodes included in the input layer, the intermediate layer (hidden layer), and the output layer is used. Specifically, the learning unit 212 of the control unit 21 uses a perspective image for the input layer and uses the area ratio of the building element for each node of the output layer. Then, a prediction model (building element determination model) consisting of intermediate layer nodes and connections that connect the nodes of the input layer to the nodes of the output layer is generated. Then, the learning unit 212 records the generated prediction model in the learning result storage unit 24.

(Process judgment processing)
Next, the process determination process will be described with reference to FIG. 5 (b).
First, the control unit 21 of the support server 20 executes the acquisition process of the captured image (step S3-1). Specifically, the construction site of the construction work is photographed using the user terminal 10. In this case, the user terminal 10 generates a captured image of the site. Then, the user terminal 10 acquires the shooting date and time from the system timer of the user terminal 10. In addition, the user terminal 10 acquires an identifier that identifies the shooting location by manual input using the user interface or by a position specifying means such as GPS (Global Positioning System). Then, the user terminal 10 stores the generated captured image in association with an identifier that specifies the shooting date and time and the shooting location. Further, the user terminal 10 uploads the stored photographed image to the support server 20. This captured image is uploaded with information about the capture date and time and location associated with it. The prediction unit 213 of the control unit 21 assigns an image code to the uploaded captured image and records it in the on-site image information storage unit 25.

Next, the control unit 21 of the support server 20 sequentially identifies the captured images recorded in the on-site image information storage unit 25, and repeats the following processing for each processing target image.
First, the control unit 21 of the support server 20 executes the architectural element prediction process (step S3-2). Specifically, the prediction unit 213 of the control unit 21 inputs the image to be processed into the input layer of the prediction model, and predicts the ratio of each building element in the output layer.

Next, the control unit 21 of the support server 20 executes the process prediction process (step S3-3). Specifically, the determination unit 214 of the control unit 21 uses the BIM information storage unit 22 to specify an element model in which each building element predicted by the prediction model is recorded in the attribute. In this case, the element model of each architectural element is specified in the order in which the predicted ratios are close to each other. Further, the determination unit 214 searches for the process associated with the specified element model. Then, the process extracted by the search is specified. For example, if a process in which a building element is associated with "concrete" or "urethane" is searched and "after heat insulating material is applied" is extracted as a process associated with these building elements, this "after heat insulating material is applied". To identify. The "light iron" and "floor panel" that are not included in the photographed image are included in the process "before board pasting" and the process "after board pasting", respectively, but are not included in "after applying heat insulating material". .. On the other hand, if the building elements included in each process of the BIM information storage unit 22 have excess or deficiency in the building elements included in the captured image, it is determined that the process cannot be specified.

Here, when it is determined that the process cannot be specified, the determination unit 214 specifies a process including at least one of the building elements included in the photographed image as a process candidate. Then, the determination unit 214 outputs the specified building candidate to the user terminal 10 of the administrator. In this case, the process candidates are output in the order in which the building elements recorded in the BIM information storage unit 22 and the building elements included in the captured image have the most common building elements. Then, the determination unit 214 acquires the process designated by the administrator from the predicted process candidates via the user terminal 10.

Next, the control unit 21 of the support server 20 executes the process recording process (step S3-4). Specifically, the determination unit 214 of the control unit 21 records the progress management data 260 recording the ratio of the building elements included in the captured image in the progress information storage unit 26 together with the specified process. Further, the determination unit 214 acquires BIM data 220 in which the shooting location and the process are recorded from the BIM information storage unit 22. Next, the determination unit 214 compares the scheduled date recorded in the BIM data 220 with the shooting date recorded in the on-site image information storage unit 25, and calculates the difference number of days. Then, the determination unit 214 records the situation in the progress management data 260. Here, when the shooting date is later than the scheduled completion date, the "delay of the difference days" is recorded in the status data area. On the other hand, if the shooting date is earlier than the scheduled completion date, or if the shooting date and the scheduled completion date match, "as scheduled" is recorded in the status data area.
The above process is repeated until all captured images are completed.

According to this embodiment, the following effects can be obtained.
(1) In the present embodiment, in the teacher data creation process, the control unit 21 of the support server 20 performs a BIM data acquisition process (step S1-1), a surface model generation process (step S1-2), and a CG image. The generation process (step S1-6) and the teacher data recording process (step S1-7) are executed. Usually, the burden of preparing teacher data for machine learning is large. Further, even if images of individual architectural elements can be prepared as teacher data, it is difficult to prepare images of the construction site itself as teacher data. In the present embodiment, by using BIM data, teacher data used for machine learning can be efficiently generated.

(2) In the present embodiment, the control unit 21 of the support server 20 executes the rendering condition specifying process (step S1-3) and the rendering process (step S1-4). As a result, a perspective image close to reality can be generated and used as teacher data according to the time when the process is performed.

(3) In the present embodiment, the control unit 21 of the support server 20 executes the viewpoint position specifying process (step S1-5). As a result, teacher data used for machine learning can be generated in consideration of the shooting situation by the person in charge.

(4) In the present embodiment, the control unit 21 of the support server 20 executes the architectural element prediction process (step S3-2) and the process prediction process (step S3-3). Thereby, the progress status of the process can be determined according to the ratio of the building elements included in the photographed image.

This embodiment can be modified and implemented as follows. The present embodiment and the following modified examples can be implemented in combination with each other within a technically consistent range.
-In the above embodiment, the BIM data 220 recorded in the BIM information storage unit 22 is used to generate a perspective image and use it as teacher data. The elements included in this perspective image are not limited to architectural elements. For example, temporary storage materials placed at the construction site may be included. In this case, the BIM information storage unit 22 stores a three-dimensional model of the shape when temporarily placed for each material used for construction. For example, when temporarily placing gypsum boards, a three-dimensional model in which a plurality of gypsum boards are stacked flat is recorded.

Therefore, the process of generating the surface model including the temporarily placed material (step S1-2) will be described with reference to FIG. 7. Here, a three-dimensional model of the temporary storage material is arranged in the model space.
First, the control unit 21 of the support server 20 executes the architectural element placement process (step S4-1). Specifically, the teacher data generation unit 211 of the control unit 21 uses the BIM data 220 recorded in the BIM information storage unit 22 to specify the architectural elements set up to the process to be processed. Then, the teacher data generation unit 211 instructs the modeling unit 210 to create a surface model of the building by using the three-dimensional model of the specified building element.

Next, the control unit 21 of the support server 20 executes the process of specifying the temporary storage material (step S4-2). Specifically, the teacher data generation unit 211 of the control unit 21 uses the BIM information storage unit 22 to identify materials that may be temporarily placed at the time when the process to be processed is performed. For example, the processes before and after the process to be processed are specified, and the building element (material) used in this process is specified as a temporary storage material.

Next, the control unit 21 of the support server 20 executes the temporary storage material placement process (step S4-3). Specifically, the teacher data generation unit 211 of the control unit 21 acquires a three-dimensional model of the temporary storage material from the BIM information storage unit 22. Then, the teacher data generation unit 211 arranges a three-dimensional model of the temporary placement material at a random position on the floor in the model space of this process.

Here, as shown in FIG. 8A, the three-dimensional model 601 of the temporary storage material is arranged at the first position with respect to the architectural element 600. Further, as shown in FIGS. 8 (b) and 8 (c), the three-dimensional model 601 of the temporary storage material is arranged at the second and third positions in the model space. Further, as shown in FIGS. 8 (d), 8 (e), and (f), the three-dimensional model 601 of the temporary storage material is rotated at arbitrary first to third angles in the model space.
In this case, a label (attribute) indicating that the material is temporarily placed is added to the perspective image of the teacher data.

Then, in the learning process, machine learning is performed using the perspective image in which the temporary placement material is arranged. In the process determination process, the ratio of building elements is calculated by excluding the temporary placement material in the photographed image.
As a result, even when the temporary storage material is arranged at the construction site, the progress of the process can be determined by the building element except for the temporary storage material.
Further, the progress may be determined by using the material of the temporarily placed material excluded. In this case, when the temporary storage material used in the next subsequent process is detected, it is determined that the subsequent process is being prepared.

-In the above embodiment, the user terminal 10 and the support server 20 are used. Then, the support server 20 executes the process determination process. The hardware that executes the process determination process is not limited to the support server 20. For example, the user terminal 10 may be provided with the learning result storage unit 24 to the progress information storage unit 26, and the user terminal 10 may execute the process determination process.

-In the above embodiment, the control unit 21 of the support server 20 executes the learning process. In this learning process, a verification process may be executed. In this case, machine learning is performed using a part of a plurality of perspective images created in the teacher data creation process. Then, in the verification process, the control unit 21 executes the process prediction process (step S3-3) using the perspective image that is not used for machine learning. In this verification process, the control unit 21 could not correctly determine the process according to the ratio of the case where the predicted process matches the process recorded in the BIM information storage unit 22 and the case where the predicted process does not match (error). Rate) is calculated.

When the error rate is less than the reference value, the control unit 21 records the generated prediction model in the learning result storage unit 24. On the other hand, when the error rate is equal to or higher than the reference value, the control unit 21 re-executes the teacher data generation process, adds the teacher data, and re-executes the learning process using the added teacher data.

-In the above embodiment, a perspective image is used as the teacher data. Teacher data is not limited to perspective images. For example, a 360-degree panoramic image in all directions, up, down, left, and right may be used. In this case, a panoramic image from the viewpoint is generated in the model space. Then, when shooting the site, a spherical camera capable of shooting a panoramic image is used.
Further, the teacher data is not limited to the CG image using the BIM data. For example, an image (Mixed Reality image, etc.) consisting of a live-action image and a CG image may be used. In this case, a live-action image can be used as the background, or an image in which a live-action architectural element image is superimposed on the BIM model can be used.

-In the above embodiment, the control unit 21 of the support server 20 executes the machine learning process (step S2-2). Here, a perspective image is used for the input layer, and the area ratio of the building element is used for each node of the output layer as information indicating the progress of the process. Information that can predict the process may be set in the output layer. For example, a perspective image may be used for the input layer, and architectural elements may be used for each node of the output layer. In this case, the progress of the process is estimated by the combination of the building elements predicted by using the photographed image.

-In the above embodiment, the control unit 21 of the support server 20 executes the rendering condition specifying process (step S1-3). Here, the rendering conditions are specified using the scheduled date and time zone. Rendering conditions are not limited to the case where the scheduled date and time zone are used. For example, an image processed image that reduces the influence of the light source and characterizes the visible building material may be used. In this case, the same image processing is performed on the captured image at the time of the process determination process.

-In the above embodiment, the control unit 21 of the support server 20 executes the rendering condition specifying process (step S1-3). Here, the surrounding environment of the construction site may be taken into consideration. In this case, the BIM information storage unit 22 stores a three-dimensional model of the surrounding building. Then, when generating the perspective image, the arrangement of the light source is determined in consideration of the surrounding buildings. Further, the surrounding building seen from the opening of the structure may be included in the perspective image as information other than the architectural element. As a result, machine learning can be performed using CG images that are close to reality.

-In the above embodiment, the control unit 21 of the support server 20 executes the teacher data recording process (step S1-7) and the machine learning process (step S2-2). Then, in the process determination process, the control unit 21 of the support server 20 executes the architectural element prediction process (step S3-2). Here, the teacher data generation unit 211 performs machine learning using the teacher data in which the area ratio of each building element is set in association with the perspective image. The teacher data is not limited to the case of using the area ratio. For example, the process may be predicted directly from the image. In this case, machine learning is performed using the teacher data in which the process information is set in association with the perspective image. Then, a perspective image is used for the input layer, and a prediction model using process information is generated for the output layer. In this case, the control unit 21 of the support server 20 executes the process prediction process using this prediction model (step S3-3).

-In the above embodiment, the control unit 21 of the support server 20 executes the process recording process (step S3-4). Here, the scheduled date recorded in the BIM data 220 and the shooting date recorded in the on-site image information storage unit 25 are compared to calculate the difference number of days, and the situation is recorded in the progress management data 260. The recorded contents of the process are not limited to this. For example, when the scheduled date is not recorded in the BIM data 220, the process information is recorded in association with the shooting date.

10 ... user terminal, 20 ... support server, 21 ... control unit, 210 ... modeling unit, 211 ... teacher data generation unit, 212 ... learning unit, 213 ... prediction unit, 214 ... process determination unit, 22 ... BIM information storage unit, 23 ... Teacher information storage unit, 24 ... Learning result storage unit, 25 ... Site image information storage unit, 26 ... Progress information storage unit.

Claims (9)

A design information storage unit that records design information of a structure composed of a three-dimensional element model including architectural elements associated with each process, and a design information storage unit.
A teacher information storage unit that records teacher data used for machine learning,
It is a learning support system equipped with a control unit for generating a prediction model that predicts the progress of each process.
The control unit
Using the design information, a three-dimensional model combining element models of architectural elements used in each process is sequentially arranged in the model space.
A two-dimensional image obtained by photographing the three-dimensional model is generated from the viewpoint in the model space and recorded in the teacher information storage unit.
A learning support system characterized in that a prediction model for predicting the progress of each process is generated from a captured image using the teacher data recorded in the teacher information storage unit. The learning support system according to claim 1, wherein the control unit generates a prediction model for predicting architectural elements included in the two-dimensional image. The learning support system according to claim 1 or 2, wherein the control unit performs rendering using a light source arranged in the model space. The learning support system according to claim 3, wherein the control unit sets the light source under lighting conditions based on the shooting time. Any one of claims 1 to 3, wherein the control unit determines the positions of a plurality of different viewpoints in the model space and generates a plurality of two-dimensional images taken from the positions of the different viewpoints. The learning support system described in the section. Any one of claims 1 to 5, wherein the control unit arranges a three-dimensional model of the temporary placement material in the model space for each teacher data and generates the two-dimensional image. The learning support system described in. The learning support system according to claim 6, wherein the control unit identifies a building element used in a process before and after creating teacher data and uses it as the temporary storage material. A design information storage unit that records design information of a structure composed of a three-dimensional element model including architectural elements associated with each process, and a design information storage unit.
A teacher information storage unit that records teacher data used for machine learning,
It is a method of providing learning support using a learning support system equipped with a control unit for generating a prediction model that predicts the progress of each process.
The control unit
Using the design information, a three-dimensional model combining element models of architectural elements used in each process is sequentially arranged in the model space.
A two-dimensional image obtained by photographing the three-dimensional model is generated from the viewpoint in the model space and recorded in the teacher information storage unit.
A learning support method characterized in that a prediction model for predicting the progress of each process is generated from a captured image using the teacher data recorded in the teacher information storage unit. A design information storage unit that records design information of a structure composed of a three-dimensional element model including architectural elements associated with each process, and a design information storage unit.
A teacher information storage unit that records teacher data used for machine learning,
It is a program for providing learning support using a learning support system equipped with a control unit for generating a prediction model that predicts the progress of each process.
The control unit
Using the design information, a three-dimensional model combining element models of architectural elements used in each process is sequentially arranged in the model space.
A two-dimensional image obtained by photographing the three-dimensional model is generated from the viewpoint in the model space and recorded in the teacher information storage unit.
A learning support program characterized in that it functions as a means for generating a prediction model for predicting the progress of each process from a captured image by using the teacher data recorded in the teacher information storage unit.

Images