JP7095232B2 - Work support system, work support method and work support program - Google Patents

Description

The present invention relates to a work support system, a work support method, and a work support program that support work at a construction site of a structure.

Techniques for estimating the amount of work required for building construction, work time, cost, materials, materials, amount of materials and materials, type and size of required equipment, etc. are being studied (for example). See Patent Document 1). In the techniques described in these documents, the construction assisting device calculates the area of a predetermined area of the construction target based on the image of the construction target, and stores the peripheral information of the construction target. Then, the construction element, which is an element necessary for the construction of the construction target, is stored, and the construction characteristics of the construction target are calculated based on the area of the predetermined area, the peripheral information, and at least two or more of the construction elements.

In addition, various materials and equipment are used at construction sites. Techniques for recognizing such materials and equipment by images are also being studied (see, for example, Non-Patent Document 1). In the technique described in this document, the material / equipment is specified by recognizing the image taken by the material / equipment as a subject by machine learning.

Japanese Unexamined Patent Publication No. 2013-88895

Obayashi Corporation, "Promote unmanned construction site of Robotics Construction 02", [online], [Search on April 7, 2017], Internet <https://www.obayashi.co.jp/technofair2016/03robotics/ 03_02.html>

As described in Patent Document 1, the amount of work and the contents of work differ depending on the area of the construction target range. However, in the technique described in this document, it is necessary to specify the range for calculating the area, which is troublesome. Further, in the technique described in Non-Patent Document 1, the purpose is to specify the type of materials and equipment carried in, and it is assumed that the amount of work and the contents of work using such materials and equipment are evaluated. Not.

-The work support system that solves the above problems is a control unit connected to a shooting unit that generates a captured image of the subject and a management information storage unit that records management information for evaluating work using materials. And prepare. Then, the control unit acquires a photographed image of the work site from the photographing unit, identifies the material to be worked included in the photographed image, and identifies the area where the material is photographed in the photographed image. The actual area of the area is calculated, and the work evaluation information of the work using the material is calculated based on the actual area and the management information recorded in the management information storage unit. This makes it possible to calculate the area based on the material reflected in the photographed image and efficiently evaluate the work related to this material.

-In the work support system, it is preferable that the control unit acquires the shooting conditions of the shot image and calculates the actual area of the region using the shooting conditions. This makes it possible to calculate the actual area where the material is constructed based on the photographed image.

-In the work support system, the control unit generates a divided image obtained by dividing the captured image using the divided roughness according to the material, and specifies the captured material for each divided image. Is preferable. As a result, it is possible to generate a divided grid for specifying the material area in consideration of the recognition accuracy of the material.

-In the work support system, it is preferable that the control unit calculates the progress of work using the material as the work evaluation information based on the actual area. This makes it possible to grasp the progress of the work based on the captured image.

-In the work support system, it is preferable that the control unit calculates the evaluation result of comparing the actual area of the material and the actual area of the area in the finished map of the work target range as the work evaluation information. .. As a result, the work status can be grasped by comparing with the model image.

-In the work support system, the control unit compares the material for each pixel in the finished map of the work target range with the area where the material is photographed in the photographed image, and the position information of the pixels of the materials that do not match. Is preferable to output. This makes it possible to confirm the accuracy of the arrangement of materials.

-In the work support system, it is preferable that the control unit calculates the area of the material specified in the work target range and outputs warning information when the area is smaller than the determination threshold value. This makes it possible to confirm the accuracy of the construction work based on the area where the materials are arranged.

-In the work support system, it is preferable that the control unit calculates the work load using the material in the actual area as the work evaluation information. As a result, the workload can be predicted efficiently.

-In the work support system, it is preferable that the control unit identifies the deterioration state of the material in the photographed image, and further calculates the work load according to the deterioration state. This makes it possible to predict the workload for repairing the deterioration of the material.

-In the work support system, it is preferable that the control unit specifies the difficulty level of the work in the captured image, and further calculates the work load according to the difficulty level. This makes it possible to predict the workload in consideration of the difficulty of the work.

According to the present invention, it is possible to support work at a construction site of a structure.

Explanatory drawing of the work support system in 1st Embodiment. It is explanatory drawing of the information stored in each storage part of the management system in 1st Embodiment, (a) is a learning information storage part, (b) is a material information storage part, (c) is a work information storage part. , (D) is an explanatory diagram of the evaluation information storage unit. Explanatory drawing of the processing procedure in 1st Embodiment. It is explanatory drawing in 1st Embodiment, (a) is a building to work, (b) is a photographed image, (c) is an explanatory diagram of photography conditions. Explanatory drawing of photographed image in 1st Embodiment. Explanatory drawing of the processing procedure in 2nd Embodiment. Explanatory drawing of the processing procedure in 2nd Embodiment. It is explanatory drawing in 2nd Embodiment, (a) shooting condition, (b) photographed image, (c) is explanatory drawing of the grid with different materials.

(First Embodiment)
Hereinafter, using FIGS. 1 to 5, the first work support system, work support method, and work support program for calculating an estimated estimated amount according to the work area at the construction site of a building (structure) are embodied. An embodiment will be described. In this embodiment, the wall surface of an existing building is photographed, and the area of the work target area on this wall surface is calculated. Further, the estimated estimated amount of repair work (construction work using materials) is calculated according to this area. Here, a specific material (for example, an outer wall tile attached to the wall surface of a building) is used as a work target on the wall surface.

As shown in FIG. 1, the work support system of the present embodiment includes a management server 20 connected to a user terminal 10.
The user terminal 10 is a computer terminal used by a person in charge of calculating an estimated estimated amount of money, and includes a camera 11, a touch panel display 12, and a communication unit 13.

The camera 11 functions as a shooting unit for shooting a subject. In the present embodiment, a photographed image of an area (wall surface) to be repaired is generated for an existing building.
The touch panel display 12 functions as an input unit for inputting various information and also as an output unit for outputting various information acquired from the management server 20.

The communication unit 13 transmits the information input to the touch panel display 12 to the management server 20, and also receives various information from the management server 20.
The management server 20 is a computer system for supporting the work of the person in charge. The management server 20 includes a control unit 21, a learning information storage unit 22, a material information storage unit 23, a work information storage unit 24, and an evaluation information storage unit 25. In the present embodiment, the material information storage unit 23 and the work information storage unit 24 function as the management information storage unit.

The control unit 21 includes a CPU, RAM, ROM, and the like, and performs processes described later (each process such as a work support stage, an image acquisition stage, a material prediction stage, and an area management stage). By executing the work support program for that purpose, the control unit 21 functions as a work support unit 211, an image acquisition unit 212, a material prediction unit 213, and an area management unit 214.

The work support unit 211 executes a process of supporting work using the user terminal 10.
The image acquisition unit 212 executes a process of acquiring a captured image captured by the camera 11 of the user terminal 10. The image acquisition unit 212 holds a distortion pattern for correcting image distortion in the captured image.

The material prediction unit 213 executes a process of predicting the material of each region included in the captured image. In this embodiment, the materials included in the captured image are predicted by deep learning. The material prediction unit 213 predicts the material contained in the captured image by using the estimation model machine-learned according to the division roughness.
The area management unit 214 executes a process of calculating the area of the area in which the material to be repaired is shown in the photographed image.

As shown in FIG. 2A, the learning information storage unit 22 records the discriminative model data 220 used for predicting the material. This discriminative model data 220 is recorded when learning is performed on the material image. The discriminative model data 220 is configured to include data relating to the division roughness and the discriminative model.

In the division roughness data area, data regarding the size (number of pixels) of dividing the captured image into a plurality of grids is recorded.
The discriminative model data area is a model for recognizing materials, and learning results in machine learning (deep learning) are recorded. In this machine learning, a discriminative model for determining a subject is generated using the labeled teacher data. By using the discriminative model, the material type (material code) is predicted for the unidentified subject.

As shown in FIG. 2B, the material information storage unit 23 records a material management record 230 for the material to be repaired. This material management record 230 is recorded when the material to be repaired is registered. The material management record 230 is configured to include data related to a material code and a material unit price.

In the material code data area, data regarding an identifier for identifying each material is recorded.
In the material unit price data area, data regarding the price (unit price) per unit area of this material is recorded.

As shown in FIG. 2 (c), the work information storage unit 24 records a work management record 240 for the material to be repaired. This work management record 240 is recorded when the work required for repair is registered. The work management record 240 is configured to include data related to a material code, a work type, and a work unit price.

In the material code data area, data regarding an identifier for identifying each material to be repaired is recorded.
In the work type data area, data regarding an identifier for identifying the work required for repairing each material is recorded. For example, in the case of tiles, work contents such as "replacement" are recorded.
In the work unit price data area, data regarding the work unit price per unit area required for this work is recorded.

As shown in FIG. 2D, the evaluation information storage unit 25 records an evaluation management record 250 for the material to be repaired. This evaluation management record 250 is recorded when the work required for repair is registered. The evaluation management record 250 is configured to include data regarding an evaluation target, an evaluation date, materials, an area, and an estimated amount of money.

In the evaluation target data area, data regarding an identifier for identifying the repair work is recorded.
In the evaluation date data area, data regarding the date on which the estimated amount is calculated is recorded.

In the material data area, data related to an identifier (material code) for identifying the material to be repaired is recorded.
In the area data area, data regarding the area of the area to be repaired is recorded.
In the estimated amount data area, data regarding the estimated estimated amount of repair work is recorded.

(Estimation estimation processing)
Next, the estimation estimation process executed in the work support system will be described with reference to FIGS. 3 to 5.
Here, as shown in FIG. 4A, it is assumed that the tile 52 on the wall surface 51 of the building 50 is repaired.
First, as shown in FIG. 3, the control unit 21 of the management server 20 executes a shooting process of the target range (step S1-1). Specifically, the person in charge uses the user terminal 10 to take a picture of the building 50 to be repaired. In this case, the user terminal 10 transmits the captured image captured by the camera 11 to the management server 20 via the communication unit 13. Then, the image acquisition unit 212 of the control unit 21 of the management server 20 acquires the captured image from the user terminal 10.
Here, the captured image 500 shown in FIG. 4B is acquired. The captured image 500 includes an image of the tile 52 on the wall surface 51 of the building 50 as a subject.

Next, the control unit 21 of the management server 20 executes the image distortion correction process (step S1-2). Specifically, the image acquisition unit 212 of the control unit 21 corrects the image distortion in the captured image by the lens of the camera 11 by using the distortion pattern.

Next, the control unit 21 of the management server 20 executes a determination target material selection process (step S1-3). Specifically, the work support unit 211 of the control unit 21 outputs a material selection screen to the touch panel display 12 of the user terminal 10. This screen is provided with an input field for materials to be detected in the captured image. For example, when repairing the tile 52 on the wall surface 51 of the building 50, "tile" is set in the input field of the material selection screen.

Next, the control unit 21 of the management server 20 executes the input processing of the calculation target range (step S1-4). Specifically, the work support unit 211 of the control unit 21 outputs a range input screen to the touch panel display 12 of the user terminal 10. This range input screen is provided with shooting condition input fields for the distance from the shooting location to the calculation target range (here, the wall surface of the building as the subject), the angle of the subject with respect to the line-of-sight direction, and the width of the subject. Regarding the shooting conditions (distance, viewing angle, width and length), the remaining one variable may be calculated by inputting two of the three variables.

FIG. 4 (c) is a view of the building 50 as viewed from above. Here, it is assumed that the wall surface 51 is photographed from the position 55. In this case, the distance L1 from the shooting location to the calculation target range (subject), the angle r1 of the calculation target range with respect to the line-of-sight direction, and the width length L2 of the calculation target range are input in the shooting condition input field.

Next, the control unit 21 of the management server 20 executes the division roughness specifying process (step S1-5). Specifically, the work support unit 211 of the control unit 21 outputs a division roughness screen to the touch panel display 12 of the user terminal 10. This division roughness screen is provided with an input field for setting the size of the grid for dividing the captured image. For example, if the material can be recognized in a relatively small area, the division roughness is reduced. In this case, the grid that divides the captured image can be made smaller so that the grid can be made to correspond even at the end of the material area, and the error in calculating the area can be reduced. On the other hand, if the material can be recognized accurately by using a relatively large area, the division roughness is increased. In this case, the deviation from the grid becomes large at the end of the material area, and the error in calculating the area becomes large, but the recognition accuracy of the material can be improved. Then, the work support unit 211 acquires the division roughness input to the division roughness screen.

Next, the control unit 21 of the management server 20 executes the estimation model selection process according to the division roughness (step S1-6). Specifically, the material prediction unit 213 of the control unit 21 divides the captured image into a grid composed of the number of pixels corresponding to the designated division roughness. Then, the material prediction unit 213 selects an estimation model according to the division roughness in the learning information storage unit 22.

As shown in FIG. 5, in the captured image 500, a plurality of grids 501 are generated with a specified division roughness. Then, in each grid 501, the material image included in each grid 501 is estimated using the selected estimation model.

Next, the control unit 21 of the management server 20 executes the process of specifying the material range included in the image (step S1-7). Specifically, the material prediction unit 213 of the control unit 21 compares the estimated material specified by using the estimation model with the determination target material designated by the user terminal 10 for each grid included in the captured image. Then, the grid in which the estimated material and the judgment target material match is specified, and the material range is specified. In the captured image 500 shown in FIG. 5, the material range is configured by the grid 502.

Next, the control unit 21 of the management server 20 executes the calculation process of the calculation target area (step S1-8). Specifically, the area management unit 214 of the control unit 21 counts the number of grids in the specified material range. Further, the area management unit 214 multiplies the area of one grid and the number of grids based on the distance input on the range input screen and the angle with respect to the line-of-sight direction to obtain the actual area of the material to be determined (calculation target area). ) Is calculated.

Next, the control unit 21 of the management server 20 executes an estimation process based on the material unit price (step S1-9). Specifically, the work support unit 211 of the control unit 21 acquires the material unit price associated with the material code of the determination target material from the material information storage unit 23. Further, the material code of the judgment target material and the work unit price associated with the work type are acquired from the work information storage unit 24. Then, the work support unit 211 calculates the estimated estimated amount of repair (work evaluation information) by multiplying the total amount of the material unit price and the work unit price by the calculation target area.

Next, the control unit 21 of the management server 20 executes the output processing of the estimation result (step S1-10). Specifically, the work support unit 211 of the control unit 21 generates an evaluation management record 250 including data related to the evaluation target, the evaluation date, the material, the area, and the estimated amount of money, and records the evaluation management record 250 in the evaluation information storage unit 25. Further, the work support unit 211 outputs an evaluation result screen to the touch panel display 12 of the user terminal 10. This evaluation result screen includes the estimated amount of the calculated work estimate.

According to this embodiment, the following effects can be obtained.
(1-1) In the present embodiment, the control unit 21 of the management server 20 takes a picture of a target range (step S1-1), a process of selecting a material to be determined (step S1-3), and a range of materials included in the image. (Step S1-7), the calculation process of the calculation target area (step S1-8) is executed. Thereby, the material included in the photographed image can be specified, and the area of the work target can be calculated based on this material.

(1-2) In the present embodiment, the control unit 21 of the management server 20 executes an estimation process (step S1-9) based on the material unit price and an output process (step S1-10) of the estimation result. As a result, it is possible to efficiently calculate a rough estimate in consideration of the material to be worked.

(1-3) In the present embodiment, the control unit 21 of the management server 20 executes the image distortion correction process (step S1-2). As a result, the image may be distorted depending on the image area due to the influence of the camera lens, but an accurate area can be calculated in consideration of the image distortion.

(1-4) In the present embodiment, the control unit 21 of the management server 20 executes the input processing of the calculation target range (step S1-4). This makes it possible to calculate the absolute value of the subject included in the image.

(1-5) In the present embodiment, the control unit 21 of the management server 20 performs a division roughness specifying process (step S1-5) and an estimation model selection process (step S1-6) according to the division roughness. Run. Thereby, the calculation accuracy of the area can be adjusted according to the material.

(Second embodiment)
Next, a second embodiment in which the work support system, the work support method, and the work support program are embodied will be described with reference to FIGS. 6 to 8. In the first embodiment described above, a case where the area to be repaired is calculated using the photographed image and the work support for estimating the estimated amount of the repair cost according to the area is performed has been described. In this embodiment, a case where the work is used for confirming the finished product as a work using materials will be described.

Here, at the construction site inside the building (construction site), LGS (Light Gauge Steel: lightweight steel frame), PB (plaster board), glass, wood, metal panel, control panel, piping material, glass wool, concrete, etc. Materials are used. Then, when the interior wall is constructed, the interior is photographed and the finished shape is confirmed.
In this case, as the camera 11 of the user terminal 10, a spherical camera capable of capturing a spherical image having a viewing range of 360 degrees is used.

Further, as shown in FIG. 6, the work information storage unit 24 records the finished product management record 241. This finished product management record 241 is recorded when the finished product map registration process is executed. The finished product management record 241 includes data related to the process code, shooting range, division roughness, confirmation method, and finished product map.

In the process code data area, data regarding an identifier for identifying a process for confirming a finished product is recorded.
In the shooting range data area, data for specifying the shooting range for checking the finished shape is recorded at the place where the finished shape is confirmed. For example, when photographing the finished shape of the wall of a room, data on the width, depth, and height of the room are recorded.

In the division roughness data area, data regarding the size when the captured image is divided into a plurality of grids is recorded.
In the confirmation method data area, data regarding a flag for specifying a method for confirming a material is recorded. In this embodiment, as the confirmation method, either confirmation by material type or confirmation by material area is used. Therefore, in the present embodiment, in the case of confirmation by the material area, the area confirmation flag is recorded in the confirmation method data area.
In the finished map data area, data regarding the arrangement of materials of each subject included in the celestial sphere image (model image) in the preceding construction is recorded.

(Complete map registration process)
A completed map registration process as a work support process of the present embodiment will be described with reference to FIGS. 6 to 8.
First, in order to create a finished map, a model room is built in advance. In this pre-construction, the finished product map registration process is executed in each process where it is necessary to confirm the finished product.

In this finished map registration process, first, the control unit 21 of the management server 20 executes a shooting direction confirmation process (step S2-1). Specifically, the work support unit 211 of the control unit 21 outputs a shooting range setting screen for inputting the size of the room for confirming the finished shape to the user terminal 10. This shooting range setting screen includes a process for confirming the finished shape, and input fields for shooting range information regarding the width, depth, and height of the room. Then, the work support unit 211 acquires the shooting range information input in the user terminal 10.

As shown in FIG. 8A, the width L11, the depth length L12, and the height L13 of the room 60 are input to the shooting confirmation screen. Then, the work support unit 211 records the input shooting range information in the work information storage unit 24 in association with the process code.

Next, the control unit 21 of the management server 20 executes the shooting process of the target range (step S2-2). Specifically, the image acquisition unit 212 of the control unit 21 outputs a shooting instruction to the user terminal 10. In this case, the user terminal 10 uses the camera 11 to generate a 360-degree spherical image of the inside of the room 60. Then, the image acquisition unit 212 acquires the spherically captured image from the user terminal 10.

Next, the control unit 21 of the management server 20 executes the division roughness specifying process in the same manner as in steps S1-5 (step S2-3). Further, the work support unit 211 associates the acquired division roughness with the process code and records it in the finished product management record 241 of the work information storage unit 24. A predetermined fixed value may be used for the division roughness.

Next, the control unit 21 of the management server 20 executes the estimation model selection process according to the division roughness as in step S1-6 (step S2-4).
In this case, as shown in FIG. 8B, in the captured image 600, a plurality of grids 601 are generated with the specified division roughness. Here, in the spherical image, a subject (wall) having a width, depth, and height specified in the shooting range information is specified, and the subject image is divided into a plurality of grids 601 with a division roughness.

Next, the control unit 21 of the management server 20 executes the material determination process (step S2-5). Specifically, the material prediction unit 213 of the control unit 21 specifies the material for each grid included in the spherically photographed image.

Next, the control unit 21 of the management server 20 executes the process of creating the finished map (step S2-6). Specifically, the work support unit 211 of the control unit 21 generates a room model based on the spherical image, and creates a finished map to which the specified material is assigned to each wall in the room model. Generate.

Next, the control unit 21 of the management server 20 executes the recording process of the finished map in association with the process (step S2-7). Specifically, the work support unit 211 of the control unit 21 outputs a confirmation method setting screen to the touch panel display of the user terminal 10. On this confirmation method setting screen, it is possible to select either confirmation of the material type or confirmation by the material area in the finished product. Then, the work support unit 211 acquires the confirmation method input to the confirmation method setting screen, associates this confirmation method (flag) with the process code of the preceding process, and works information storage unit 24 together with the generated finished map. Record in.

(Production confirmation process)
Next, the finished product confirmation process will be described with reference to FIG. 7.
Here, the control unit 21 of the management server 20 executes a process of reading the finished map of the target range (step S3-1). Specifically, the work support unit 211 of the control unit 21 outputs a confirmation process input screen to the display of the user terminal 10. This confirmation process input screen is provided with an input field for a process for confirming the finished product. Then, the work support unit 211 acquires the input process code in the user terminal 10, and acquires the completed map associated with the acquired process code from the work information storage unit 24.

Next, the control unit 21 of the management server 20 executes a shooting direction confirmation process (step S3-2). Specifically, the work support unit 211 of the control unit 21 acquires the photographing range information associated with the process code from the work information storage unit 24.

Next, the control unit 21 of the management server 20 executes the shooting process of the target range in the same manner as in step S2-2 (step S3-3).
Next, the control unit 21 of the management server 20 executes the division roughness specifying process in the same manner as in step S2-3 (step S3-4). Specifically, the work support unit 211 of the control unit 21 acquires the division roughness associated with the process code from the work information storage unit 24.

Next, the control unit 21 of the management server 20 executes the estimation model selection process according to the division roughness as in step S2-4 (step S3-5).
Next, the control unit 21 of the management server 20 executes the material determination process in the same manner as in step S2-5 (step S3-6).

Next, the control unit 21 of the management server 20 executes a determination process as to whether or not a check based on the area is necessary (step S3-7). Specifically, the work support unit 211 of the control unit 21 confirms in the work information storage unit 24 whether or not the area confirmation flag is recorded for this process. If the area confirmation flag is recorded, it is determined that the area check is necessary.

When the area confirmation flag is not recorded and it is determined that the area check is unnecessary (when "NO" in step S3-7), the control unit 21 of the management server 20 executes the correctness determination process for each grid. (Step S3-8). Specifically, the work support unit 211 of the control unit 21 determines that the material for each grid matches the material recorded on the finished map.

On the other hand, when the area confirmation flag is recorded and it is determined that the area check is necessary (when "YES" in step S3-7), the control unit 21 of the management server 20 calculates the area of the determination target material range. The process is executed (step S3-9). Specifically, the work support unit 211 of the control unit 21 calculates the area of each material included in the spherically captured image based on the imaging range information acquired from the user terminal 10.

Next, the control unit 21 of the management server 20 executes an area comparison process with the determination threshold value (step S3-10). Specifically, the work support unit 211 of the control unit 21 calculates the calculated area of each material. Then, the work support unit 211 determines that there is an error when there is a difference of the determination threshold value or more as compared with the area of the material registered in the finished product map.

Next, the control unit 21 of the management server 20 executes a determination process as to whether or not there is an erroneous portion (step S3-11). Specifically, the work support unit 211 of the control unit 21 is erroneous when there is a material mismatch in step S3-8 or when there is a difference of the determination threshold value or more in the area comparison in step S3-10. Judge that there is a part.

When it is determined that there is an erroneous part (when "YES" in step S3-11), the control unit 21 of the management server 20 executes the display process of the corresponding part (step S3-12). Specifically, the work support unit 211 of the control unit 21 generates work evaluation information in which pixels of materials that do not match (material areas with errors) are displayed in a spherical image, and is displayed on the touch panel display 12 of the user terminal 10. Output.

In this case, FIG. 8C shows a photographed image 610 at the time of the completed form confirmation process corresponding to the photographed image 600 at the time of the completed form map registration process. In this captured image 610, only the grid 611 is displayed for simplification. In this captured image 610, a plurality of grids 611 are generated with a specified division roughness. Then, in the grid 612 shaded so that they can be identified, the materials that do not match are shown.

According to this embodiment, the following effects can be obtained.
(2-1) In the present embodiment, the control unit 21 of the management server 20 executes the process of creating the finished map (step S2-6). As a result, model information can be efficiently generated by the advance construction.

(2-2) In the present embodiment, in the finished map registration process, the control unit 21 of the management server 20 confirms the shooting direction (step S2-1) and specifies the division roughness (step S2-3). To execute. In this case, the shooting range information and the division roughness are recorded in the work information storage unit 24 in association with the process code. Further, in the finished product confirmation process, the control unit 21 of the management server 20 executes a shooting direction confirmation process (step S3-2) and a division roughness specification process (step S3-4). As a result, the conditions in the finished product map registration process can be used in the finished product confirmation process.

(2-3) In the present embodiment, the control unit 21 of the management server 20 executes a determination process as to whether or not a check by area is necessary (step S3-7). When it is determined that the check by area is unnecessary (when "NO" in step S3-7), the control unit 21 of the management server 20 executes the correctness determination process for each grid (step S3-8). This makes it possible to determine accurate construction based on the type of material used.

(2-4) In the present embodiment, when it is determined that the area check is necessary (when “YES” in step S3-7), the control unit 21 of the management server 20 calculates the area of the determination target material range. (Step S3-9), the area comparison process with the determination threshold value (step S3-10) is executed. As a result, accurate construction can be determined based on the amount of material used.

(2-5) In the present embodiment, when it is determined that there is an erroneous part (when "YES" in step S3-11), the control unit 21 of the management server 20 executes the display process of the corresponding part. (Step S3-12). This makes it possible to point out construction mistakes and urge corrections.

(2-6) In the present embodiment, a camera capable of capturing a spherical image is used as the camera 11 of the user terminal 10. This makes it possible to efficiently acquire an image of the entire construction area.

Moreover, the above-mentioned embodiment may be changed as follows.
-In each of the above embodiments, the control unit 21 of the management server 20 executes the selection process of the estimation model according to the division roughness. Specifically, the material prediction unit 213 of the control unit 21 divides the captured image into a grid composed of the number of pixels corresponding to the divided roughness designated on the divided roughness screen. The method of identifying the area where the material is arranged is not limited to the case of using the grid. For example, segmentation for classifying the subject included in the captured image or the edge included in the captured image may be used to specify the range of the subject included in the captured image.

-In each of the above embodiments, the user terminal 10 and the management server 20 are used, but the hardware configuration is not limited to these. For example, the user terminal 10 may be provided with various functions included in the management server 20.

-In each of the above embodiments, the control unit 21 of the management server 20 executes the process of specifying the division roughness. Here, the division roughness may be selected by the control unit 21 based on the distance and the unit size of the material.

-In each of the above embodiments, a discriminative model is generated by machine learning (deep learning). The method of generating the discriminative model is not limited to deep learning. For example, a discriminative model consisting of a distribution (histogram) of features is generated for a cluster in which a plurality of features included in one teacher data are put together. An index (type of subject) of a learning image is given to this discriminative model. Then, a plurality of feature quantities included in the unidentified captured image may be calculated, and the subject included in the captured image may be predicted based on the comparison between the distribution of the feature quantities and the discriminative model.

-In the first embodiment, the control unit 21 of the management server 20 executes a material range specifying process (step S1-7) and a calculation target area calculation process (step S1-8) included in the image. .. Here, the area of the material included in the photographed image is calculated. As long as the area can be calculated based on the materials included in the photographed image, the area is not limited to the case where all the areas in which the calculation target material is photographed are specified as the calculation target range. For example, the area may be calculated based on the deterioration status of the material. Specifically, an area in which the material is photographed as a subject is specified in the photographed image, and the area including the material in the calculation target state is calculated in this material area. The state to be calculated includes, for example, deterioration of materials. This deterioration can be evaluated using the state of the material (discoloration, cracking, dirt, etc.). Further, the state of the material may be evaluated based not only on the surface of the material but also on the inside of the material. For example, the reflectance and absorptance of light having a predetermined wavelength changes depending on the state of adhesion of the tile to the wall surface. Therefore, the internal state may be evaluated in the photographed image of the evaluation object taken by using the light beam of a predetermined wavelength (for example, infrared light), and the working range (calculation target range) may be specified by this. ..

Further, the construction state in the invisible portion may be evaluated based on the captured image. For example, the area may be calculated by predicting the state of the structure hidden behind the subject in front of the captured image. For example, when tiles are applied to the front surface of a structure, the tiles on the side surfaces of the structure are predicted by machine learning based on the arrangement of the tiles, and the area is calculated based on this prediction.

-In the first embodiment, the control unit 21 of the management server 20 executes the input processing of the calculation target range (step S1-4). Further, in the second embodiment, the control unit 21 of the management server 20 executes a shooting direction confirmation process (step S2-1). The information necessary for calculating the area may be calculated based on the shooting conditions. For example, the focal length at the time of shooting may be used to calculate the distance to the subject and the arrangement angle included in the shot image.

Further, a reference object whose size is known in advance may be attached to the work target surface and photographed, and the imaging conditions may be specified based on the size and shape of the reference object reflected in the photographed image. In this case, the distance can be calculated based on the comparison between the original size of the reference object and the size of the reference object image in the captured image. In addition, the inclination can be calculated based on the comparison between the original shape of the reference object and the shape of the reference object image.

-In the first embodiment, the control unit 21 of the management server 20 selects an estimation model according to the division roughness (step S1-6) and specifies a material range included in the image (step S1-). 7) is executed. Here, when a plurality of materials are specified in one grid, the area ratio of each material included in this grid may be calculated and the material range may be specified. For example, if the area ratio exceeds the standard value (for example, half), it is included in the calculation target range. Further, by reducing the reference value of the area ratio, it is possible to suppress the material shortage.

-In the first embodiment, the control unit 21 of the management server 20 executes an estimation process (step S1-9). Instead of the calculation target area, the work unit price based on the repair area requiring repair of the material and the difficulty of repair may be used. Here, when a plurality of work types are recorded for this material in the work information storage unit 24, the work type confirmation screen is output to the touch panel display 12 of the user terminal 10. Then, the work support unit 211 may change the unit price to be multiplied by the area according to the work type input on the work type confirmation screen. For example, for a work type with a high degree of difficulty, the work unit price is increased. For work that does not require material replacement, a rough estimate is calculated based only on the work unit price, not including the material unit price.

-In the second embodiment, a camera capable of capturing a spherical image is used as the camera 11 of the user terminal 10. The means for photographing the surroundings is not limited to the spherical camera. For example, a camera having a limited field of view may be rotated to acquire a captured image. Then, a panoramic image in which these captured images are joined is generated, and this panoramic image is used.

-In the second embodiment, the control unit 21 of the management server 20 executes the reading process of the finished map of the target range (step S3-1). In this case, the process code is input to the user terminal 10. Instead of this, the progress of the work may be specified based on the arrangement of the materials included in the captured image. In this case, the material type and the material area included in the photographed image taken in the finished product confirmation process are specified, and the finished product closest to the arrangement and area of the material is acquired from the work information storage unit 24. As a result, the progress can be predicted based on the captured image.

10 ... user terminal, 11 ... camera, 12 ... touch panel display, 13 ... communication unit, 20 ... management server, 21 ... control unit, 211 ... work support unit, 212 ... image acquisition unit, 213 ... material prediction unit, 214 ... area Management unit, 22 ... Learning information storage unit, 23 ... Material information storage unit, 24 ... Work information storage unit, 25 ... Evaluation information storage unit, 500, 600, 610 ... Photographed images.

Claims (10)

A shooting unit that generates a shot image of the subject, and
A learning information storage unit in which a discriminative model according to the division roughness is recorded,
It is a work support system equipped with a management information storage unit that records management information for evaluating work using materials and a control unit connected to the management information storage unit.
The control unit
From the shooting unit, the shot image and shooting conditions of the work site are acquired.
Using the division roughness according to the material, a divided image obtained by dividing the captured image is generated.
Using the discriminative model according to the division roughness recorded in the learning information storage unit, the material to be worked included in each of the division images is specified.
In each of the divided images, each divided image in which the material was photographed is specified, and the actual area is calculated according to the photographing conditions of the divided images.
A work support system characterized in that work evaluation information of work using the material is calculated based on the actual area and the management information recorded in the management information storage unit. The work support system according to claim 1 , wherein the control unit calculates the progress of work using the material as the work evaluation information based on the actual area. The control unit is characterized in that the evaluation result comparing the area of the material in the finished map of the work target range with the actual area of the divided image in which the material is photographed is calculated as the work evaluation information. The work support system according to claim 1 or 2 . The control unit is characterized in that the material for each pixel in the finished map of the work target range is compared with the area where the material is photographed in the photographed image, and the position information of the pixels of the materials that do not match is output. The work support system according to claim 3 . The work support system according to claim 4 , wherein the control unit calculates an area of a material specified in the work target range, and outputs warning information when the area is smaller than a determination threshold value. .. The work support system according to claim 1 or 2 , wherein the control unit calculates a work load using the material in the actual area as the work evaluation information. The control unit identifies the deterioration state of the material in the photographed image, and determines the deterioration state of the material.
The work support system according to claim 6 , further comprising calculating the workload according to the deterioration status. The control unit identifies the difficulty level of the work in the captured image.
The work support system according to claim 6 or 7 , further comprising calculating the workload according to the difficulty level. A shooting unit that generates a shot image of the subject, and
A learning information storage unit in which a discriminative model according to the division roughness is recorded,
It is a work support method that evaluates the work status using a work support system equipped with a management information storage unit that records management information for evaluating work using materials and a control unit connected to the management information storage unit.
The control unit
From the shooting unit, the shot image and shooting conditions of the work site are acquired.
Using the division roughness according to the material, a divided image obtained by dividing the captured image is generated.
Using the discriminative model according to the division roughness recorded in the learning information storage unit, the material to be worked included in each of the division images is specified.
In each of the divided images, each divided image in which the material was photographed is specified, and the actual area is calculated according to the photographing conditions of the divided images.
A work support method comprising calculating work evaluation information of work using the material based on the actual area and the management information recorded in the management information storage unit. A shooting unit that generates a shot image of the subject, and
A learning information storage unit in which a discriminative model according to the division roughness is recorded,
It is a work support program that evaluates the work status using a work support system equipped with a management information storage unit that records management information for evaluating work using materials and a control unit connected to it.
The control unit
From the shooting unit, the shot image and shooting conditions of the work site are acquired.
Using the division roughness according to the material, a divided image obtained by dividing the captured image is generated.
Using the discriminative model according to the division roughness recorded in the learning information storage unit, the material to be worked included in each of the division images is specified.
In each of the divided images, each divided image in which the material was photographed is specified, and the actual area is calculated according to the photographing conditions of the divided images.
A work support program characterized in that it functions as a means for calculating work evaluation information of work using the material based on the actual area and the management information recorded in the management information storage unit.

Images