JP2019175144A - Building management system, learning device, position determination device, and position determination method - Google Patents

Abstract

To provide a building management system, a learning device, a position determination device, and a position determination method for appropriately managing a building.SOLUTION: A building management system 1 includes: a position determination device 40 that determines a position where a photograph is taken from among photographs taken of a building; and a learning device 10 that generates, by machine learning, a determination criterion used to determine a position by the position determination device 40. A learning device 10 includes: a three-dimensional shape data storage unit that stores three-dimensional shape data modeling the building; a two-dimensional image generation unit that generates a two-dimensional image of the building by rendering the three-dimensional shape data by setting a viewpoint position and a line-of-sight direction; and a learning unit that generates the determination criterion by learning a relationship between extraction information extracted from among two-dimensional images generated by the two-dimensional image generation unit and the viewpoint position set when generating the two-dimensional image. The position determination device 40 includes: a photograph reception unit that accepts photographs of the building; and a determination unit that determines the position where the photograph is taken on the basis of the determination criterion generated by the learning device.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a building management system for managing a building, and a learning device, a position determination device, and a position determination method that can be used in the building management system.

  When constructing a large-scale building such as a plant, it is necessary to accurately grasp the progress of construction at a wide range of construction sites. In addition, it is necessary to confirm in detail whether or not the building is constructed as designed in a wide range of buildings during or after the construction of the building. Progress management and construction confirmation become a large-scale work as the building becomes large-scale, and requires a lot of labor. Therefore, ingenuity has been made to make such work as efficient as possible and reduce man-hours (for example, Patent Document 1).

Japanese Patent No. 2712231

  In order to perform progress management and construction confirmation in a large-scale building, it is first necessary for an operator to accurately grasp his / her position. However, within the building, the reception status of signals from GPS satellites is limited. It may become unstable and it may be difficult to obtain highly accurate position information by positioning using signals from GPS satellites.

  Even in such a situation, there is a need for a technique that enables an operator to perform work for managing a building such as progress management and construction confirmation while accurately grasping his / her position.

  This invention is made | formed in view of such a condition, The objective is to provide the technique which makes it possible to manage a building appropriately.

  In order to solve the above-described problems, a building management system according to an aspect of the present invention includes a position determination device that determines a position where a photograph is captured from a photograph of the building, and the position determination device determines a position. And a learning device that generates a determination criterion to be used by machine learning. The learning device sets a two-dimensional image of the building by rendering a three-dimensional shape data by setting a viewpoint position and a line-of-sight direction, and a three-dimensional shape data storage unit that stores the three-dimensional shape data modeling the building. Learning the relationship between the generated 2D image generation unit, the extracted information extracted from the 2D image generated by the 2D image generation unit, and the viewpoint position set when generating the 2D image And a learning unit that generates a determination criterion. The position determination device includes a photo reception unit that receives a photograph of a building and a determination unit that determines a position where the photo is captured based on a determination criterion generated by the learning device.

  According to this aspect, it is possible to determine with high accuracy the position where the photograph is taken from the photograph taken of the building. Thereby, even when it is difficult to detect a position using GPS or the like, the position can be accurately grasped.

  The learning apparatus may further include a photograph acquisition unit that acquires a photograph of the building and position information indicating a position where the photograph is captured. The learning unit may learn the determination criterion by further using the building photograph and the position information acquired by the photo acquisition unit. According to this aspect, by further using a photograph of an actual building, it is possible to learn a more accurate position determination criterion.

  The 2D image generation unit sets the viewpoint position inside the building and generates a 2D image inside the building, and the learning unit captures the photo from the photo taken inside the building. The photo acceptance unit accepts a photo taken inside the building, and the judgment unit determines the position inside the building where the photo was taken. May be. According to this aspect, even when it is difficult to receive a signal from a GPS satellite inside the building, the position can be determined with high accuracy.

  The position determination device includes a structure information storage unit that stores information on a structure that constitutes a building, and information that indicates a position of the structure in the building, and a position based on the position determined by the determination unit. A structure information presenting unit that determines an imaged structure and presents information about the determined structure. According to this aspect, the person in charge of the operation, management, maintenance, maintenance, security, etc. of the building can easily obtain information on the structure around the current position, thus improving work efficiency. Can be made.

  The position determination device includes a three-dimensional shape data storage unit that stores the three-dimensional shape data of the building, and a progress stage determination unit that determines the progress stage of the building by comparing the photograph and the three-dimensional shape data. And may be further provided. According to this aspect, the person in charge who manages the progress of the construction at the construction site of the building can grasp the progress stage easily and accurately.

  The position determination device includes a three-dimensional shape data storage unit that stores the three-dimensional shape data of the building, and a confirmation unit that checks the quality of the construction of the building by comparing the photograph and the three-dimensional shape data. Further, it may be provided. According to this aspect, it is possible to improve the work efficiency of the person in charge who confirms the quality of the construction at the construction site of the building.

  Another aspect of the present invention is a learning device. This device has a 3D shape data storage unit that stores 3D shape data modeling a building, and renders 3D shape data by setting a viewpoint position and a line-of-sight direction to render a 2D image of the building. Learning the relationship between the generated 2D image generation unit, the extracted information extracted from the 2D image generated by the 2D image generation unit, and the viewpoint position set when generating the 2D image And a learning unit that generates a determination criterion used for determining a position where the photograph is captured from a photograph capturing the building.

  According to this aspect, it is possible to generate a determination criterion capable of determining with high accuracy the position where the photograph is captured from the photograph capturing the building.

  Yet another embodiment of the present invention is a position determination device. This apparatus is extracted from a two-dimensional image generated by rendering a three-dimensional shape data that models a building by setting a viewpoint position and a line-of-sight direction, and a photo reception unit that accepts a photograph of a building. A determination unit that determines a position at which the photograph is captured based on a determination criterion generated by learning the relationship between the extracted information and the viewpoint position set when generating the two-dimensional image. .

  According to this aspect, it is possible to determine with high accuracy the position where the photograph is taken from the photograph taken of the building. Thereby, even when it is difficult to detect a position using GPS or the like, the position can be accurately grasped.

  Yet another embodiment of the present invention is also a position determination device. This apparatus includes a photo reception unit that receives a photograph of an imaging target captured by a captured image, and a two-dimensional image generated by rendering three-dimensional shape data that models the imaging target by setting a viewpoint position and a line-of-sight direction. A determination unit that determines a position where a photograph is taken based on a determination criterion generated by learning a relationship between extraction information extracted from the image and a viewpoint position set when generating the two-dimensional image And comprising.

  According to this aspect, it is possible to determine the position where the photograph is captured from the photograph to be captured with high accuracy. Thereby, even when it is difficult to detect a position using GPS or the like, the position can be accurately grasped.

  Yet another embodiment of the present invention is a position determination method. This method accepts a photograph of an imaging target captured by a captured image, and extracts from a two-dimensional image generated by rendering the three-dimensional shape data modeling the imaging target by setting the viewpoint position and the line-of-sight direction Determining the position at which the photograph was captured based on the determination criterion generated by learning the relationship between the extracted information and the viewpoint position set when generating the two-dimensional image. Let the computer run.

  According to this aspect, it is possible to determine the position where the photograph is captured from the photograph to be captured with high accuracy. Thereby, even when it is difficult to detect a position using GPS or the like, the position can be accurately grasped.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which makes it possible to manage a building appropriately can be provided.

It is a figure which shows the whole structure of the building management system which concerns on embodiment. It is a figure which shows the structure of the learning apparatus which concerns on embodiment. It is a figure which shows the structure of the position determination apparatus which concerns on embodiment. It is a figure which shows the structure of the construction site terminal which concerns on embodiment. It is a figure which shows the example of the building managed by the building management system which concerns on embodiment. It is a figure which shows the example of the three-dimensional shape data which modeled the plant shown in FIG. It is a figure which shows the example of the two-dimensional image produced | generated by rendering the three-dimensional shape data shown in FIG. It is a figure which shows the example of the screen displayed on the display apparatus of the construction site terminal. It is a flowchart which shows the procedure of the position determination method which concerns on embodiment. It is a flowchart which shows the procedure of the position determination method which concerns on embodiment.

  FIG. 1 shows an overall configuration of a building management system according to an embodiment. In this embodiment, an example of managing a plant for producing chemical products, industrial products, and the like as a building will be described. The building management system 1 generates, by machine learning, a position determination device 40 that determines a position where the photograph is captured, and a determination criterion that the position determination device 40 uses to determine the position from the photograph of the building. A learning device 10 that is installed on the construction site 3 of the building, and various devices that are installed on the plant 6. These devices are connected to each other via the Internet 2. In the construction site 3, a robot used for automatically imaging a building and collecting a photograph used by the learning device 10 for learning, a robot used for building construction, operation, maintenance, etc. 5. A device such as a construction site terminal 80 used by a construction person in charge at the construction site 3 is provided. The robot 5 is, for example, an equipment installation robot for installing equipment at the construction site 3 or a welding robot that performs welding at the construction site 3. The plant 6 includes a robot 7 used for the operation of the plant 6, an operator terminal 8 used by an operator of the plant 6, and a person in charge used by a person in charge of maintenance, security and maintenance of the plant 6. A device such as a terminal 9 is provided. The robot 7 is, for example, a driving robot that grasps the operating state of the plant 6 and operates valves and the like. The robot 5, the construction site terminal 80, the robot 7, the operator terminal 8, and the person-in-charge terminal 9 take a photograph of the surroundings in the building under construction or the building after completion to the position determination device 40. By transmitting and acquiring position information from the position determination device 40, the user's own position is grasped.

  The position determination device 40 matches a photograph taken inside a building with a two-dimensional image generated from three-dimensional shape data modeling the building, thereby making a part of the building photographed in the photograph. Is detected in the three-dimensional shape model, and the position and the line-of-sight direction where the photograph is taken are determined with high accuracy. As described above, the signal reception status from the GPS satellite becomes unstable inside the building, and it may be difficult to grasp the position with high accuracy. However, the position determination device 40 according to the present embodiment. Since the position can be grasped with high accuracy even inside the building, the robot 5, the person using the construction site terminal 80, the robot 7, the operator using the operator terminal 8, the person in charge A person in charge who uses the terminal 9 can work while accurately grasping his / her position inside the building.

  When building a large-scale building such as a plant, the details of parts such as valves may not be determined yet when modeling the building and creating 3D shape data at the design stage. Is actually constructed, parts having shapes different from the three-dimensional shape data are used, and the actual building may not exactly match the three-dimensional shape data. In addition, during construction of the building, there is an unfinished or unfinished part, so only a part of the three-dimensional shape data is shown in the photograph, while on the other hand, in parts of the building such as scaffolding and materials Things that don't appear in the photo. In addition, the construction status of the building, the status of the scaffolding and material storage, etc. can change daily as the construction progresses. Even in such a case, when a person discriminates between a photograph of a building and the three-dimensional shape data modeling the building, it is different from the part that is strictly required to match. Although it is possible to accurately discriminate between different parts while discriminating them from allowable parts, it has been considered difficult to automate such operations. Therefore, in the past, such work had to rely on a small number of experts.

  In order to solve such a problem, in the building management system 1 of the present embodiment, a two-dimensional image of a building model is set by rendering the three-dimensional shape data by setting the viewpoint position and the line-of-sight direction. By automatically learning the relationship between the extracted information extracted from those two-dimensional images and the viewpoint position set when generating those two-dimensional images, the positions are automatically extracted from the photos. A criterion for determination is generated. In addition, a large number of photographs of the actually constructed buildings are collected, and the judgment criteria are learned by using these photographs. Thereby, since it is possible to automate the discrimination between the photograph of the building and the three-dimensional shape data of the building, the labor can be significantly reduced.

  The two-dimensional image generated from the three-dimensional shape data can accurately set the viewpoint position and the direction of the line of sight, but as described above, parts that are different from the actually constructed building are modeled. In some cases, the shape of some parts may not be accurate. On the other hand, the photographed image of the building accurately reflects the shape of the part actually constructed, but cannot accurately grasp the imaged position. Therefore, it is possible to learn the relationship between the image and the position more accurately by learning the judgment criteria using both the two-dimensional image generated from the three-dimensional shape data and the photograph of the actually constructed building. And the position can be more accurately determined from the photograph. The person in charge at the construction site 3, the operator of the plant 6 and the person in charge of maintenance, security, and maintenance of the plant 6 can easily and accurately grasp the position, thereby improving the work efficiency and the work load. Can be reduced. In addition, since the robot 5 can perform work that needs to be performed at an accurate position, the number of steps required for construction and operation can be greatly reduced.

  In this figure, for the sake of simplicity of explanation, the learning device 10 is shown as a single device. However, the learning device 10 is realized by a plurality of servers using cloud computing technology, distributed processing technology, or the like. May be. As a result, it is possible to process a large amount of photographs collected from the automatic image pickup device 4 and a large amount of two-dimensional images generated from the three-dimensional shape data at high speed so that the determination criterion can be learned. The time required for improvement can be greatly reduced.

  Hereinafter, a photograph of an actual building and a two-dimensional image generated from three-dimensional shape data modeling the building are collectively referred to as an “image” of the building.

  FIG. 2 shows a configuration of the learning device according to the embodiment. The learning device 10 includes a communication device 11, a control device 20, and a storage device 30.

  The communication device 11 controls wireless or wired communication. The communication device 11 transmits / receives data to / from the position determination device 40, the automatic imaging device 4, the robot 5, the construction site terminal 80, the robot 7, the operator terminal 8 and the like via the Internet 2.

  The storage device 30 stores data and computer programs used by the control device 20. The storage device 30 stores determination reference data 31 and three-dimensional shape data 32. The determination reference data 31 is determination reference data used by the position determination device 40 to determine the position. The three-dimensional shape data 32 is three-dimensional shape data that models a building.

  The control device 20 includes a photo acquisition unit 21, a two-dimensional image generation unit 22, an analysis unit 23, a learning unit 24, and a providing unit 25. In terms of hardware components, these configurations are realized by a CPU of a computer, a memory, a program loaded in the memory, and the like, but here, functional blocks realized by their cooperation are illustrated. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The photograph acquisition unit 21 acquires a photograph of a building imaged by the automatic imaging device 4 and information indicating a position where the photograph is captured. The automatic image pickup device 4 moves around and inside a building to automatically pick up images around and inside the building, and uses any position detection technology such as a GPS receiver, an acceleration sensor, a gyro sensor, and a LIDAR. Is used to obtain information indicating the position where the photograph was taken. Although there is a possibility that this position information cannot be acquired with high accuracy, it is used to learn a determination criterion for roughly determining the position where the photograph is taken. The automatic imaging device 4 is an unmanned aerial vehicle equipped with an imaging device and a communication device for transmitting image data captured by the imaging device, or a storage device that stores image data captured by the imaging device. There may be. An unmanned aerial vehicle may be moved around and inside a building by being operated by an operator, or automatically built by an autopilot function using spatial recognition technology such as images captured by an imaging device or radar. You may move around and inside. As will be described later, in order to determine the progress stage of the construction by the position determination device 40, the automatic imaging device 4 automatically moves the entire range of the construction site 3, for example, after one day of construction is completed, A picture of the entire range of the building under construction may be taken.

  The two-dimensional image generation unit 22 generates a two-dimensional image of a building by setting the viewpoint position and the line-of-sight direction and rendering the three-dimensional shape data 32. The two-dimensional image generation unit 22 sets a position in a range in which a device such as the robot 5 or the construction site terminal 80 can move as a viewpoint position, and sets a line-of-sight direction in a range in which a photograph is assumed to be taken from the viewpoint position. The three-dimensional shape data 32 is rendered by setting. When the position determination function by the position determination device 40 is used after the building is completed, only the position of the passage where the robot 5 or the construction site terminal 80 can move may be set as the viewpoint position. When using even during construction of an object, it is preferable to set the viewpoint position over the entire range of the construction site 3. Moreover, it is preferable to set the viewpoint position not only in the range up to the height of the building but also in the range up to the height that can be reached by a crane or a scaffold used for constructing the building.

  The 2D image generation unit 22 renders 3D shape data by setting illumination that reproduces light around the building and light such as the sun in order to reproduce a photograph of an actual building more accurately. May be. For example, a two-dimensional image that reproduces sunlight and shade for each time zone may be generated by setting illumination at the position of the sun for each time zone and rendering three-dimensional shape data.

  As will be described later, in order to enable the position determination device 40 to determine the progress stage of the construction, the two-dimensional image generation unit 22 is incomplete or incomplete at each progress stage of the construction. Is deleted from the three-dimensional shape data, and a two-dimensional image of the building that will be completed in the progress stage is generated. By generating a determination criterion for each progress stage, it is possible to determine the progress stage of the building construction work from the photograph of the building imaged by the automatic imaging device 4 during the construction of the building.

  The analysis unit 23 analyzes the photograph of the building acquired by the photo acquisition unit 21 and the two-dimensional image of the building generated by the two-dimensional image generation unit 22, and extracts information indicating their characteristics. The analysis unit 23 may analyze a photograph of a building and a two-dimensional image using any known image analysis technique.

  The analysis unit 23 may detect a structure such as a component or device existing in the image by using a contour detection technique or a pattern recognition technique, and identify the detected structure. The learning device 10 stores the three-dimensional shape data or the two-dimensional image of the parts constituting the building in the storage device 30 and compares the structure included in the image with the structure detected from the image. You may identify. When analyzing a two-dimensional image generated from three-dimensional shape data, information on the structure is stored as metadata of the three-dimensional shape data, information on the structure included in the three-dimensional shape data, and two The region of the structure rendered in the three-dimensional image may be associated.

  The analysis unit 23 may analyze data of pixels constituting the image and extract information indicating the characteristics of the image. For example, image characteristics may be extracted by statistically analyzing pixel brightness, saturation, chromaticity, and the like. This information is an index that indicates whether or not parts or equipment with specific brightness, saturation, chromaticity, etc. are present in the image, and if so, their type, number, orientation, etc. sell.

  The learning unit 24 learns the relationship between the extraction information extracted by the analysis unit 23 and the viewpoint position set when the two-dimensional image is generated or the position where the photograph of the building is captured, Determination criterion data 31 is generated. The learning unit 24 learns the relationship between the extracted information extracted by analyzing the photograph and the positional information indicating the rough position where the photograph is taken, thereby determining the region determination criterion for determining the rough position. A position criterion for determining a precise position by learning the relationship between the extracted information extracted by analyzing the two-dimensional image and the viewpoint position for generating the two-dimensional image May be generated. These criteria may be generated separately or integrated. The learning unit 24 may learn the relationship between the extracted information, the viewpoint position, and the line-of-sight direction, and generate a determination criterion for determining the position and the line-of-sight direction from the building image.

  The learning unit 24 determines whether the structure detected in the image by the analysis unit 23 is an object for which a strict match is required when matching the photograph with the image whose position is to be determined. Weighting may be performed according to the accuracy required for matching, such as whether the structure is required to match a structure that is not required but has the same type of function. For example, if you have decided to install a valve but are not required to match the type of valve, the criteria will be such that it can be determined that the photo shows another type of valve. You may learn. In addition, if the pipe diameter is fixed but it is not required to match even the color, it is determined that it matches in a photograph that shows another color pipe of the same diameter, and pipes with different diameters are shown. The determination criterion may be learned so that it is determined not to match in the photograph being photographed.

  The learning unit 24 may delete an object that is not a part of the building, such as a scaffold or material that is reflected in a photograph of the building, and then use the learning part 24 for learning while the scaffold or material is still visible. May be used. When deleting scaffolds or materials, the scaffolds and materials may be detected and deleted using contour detection technology, etc., or a photo with distance information is taken using a ranging camera etc. In some cases, scaffolds, materials, and the like may be deleted by deleting pixels within a predetermined distance range.

  The learning unit 24 inputs image data to the input layer, constructs a neural network that outputs position information from the output layer, and adjusts the weight of the intermediate layer of the neural network according to the combination of the image and the position, You may learn the criteria for determining the position where the photograph was imaged. The learning unit 24 inputs image data to the input layer, constructs a neural network that outputs the progress stage from the output layer, and determines the neural network according to the combination of the two-dimensional image generated for each progress stage and the progress stage. A criterion for determining the progress stage may be generated by adjusting the weight of the intermediate layer.

  The learning unit 24 may divide the construction site into a plurality of regions and generate a determination criterion for each region. Moreover, you may produce | generate a criterion for every time slot | zone using the two-dimensional image produced | generated by setting illumination for every time slot | zone.

  The providing unit 25 provides the determination reference data 31 learned by the learning unit 24 to the position determination device 40.

  FIG. 3 shows a configuration of the position determination apparatus according to the embodiment. The position determination device 40 includes a control device 50 and a storage device 70.

  The storage device 70 stores data and computer programs used by the control device 50. The storage device 70 stores determination reference data 71, three-dimensional shape data 72, and a structure information database 73. The determination reference data 71 is determination reference data used by the position determination device 40 to determine the position. The three-dimensional shape data 72 is three-dimensional shape data that models a building. The structure information database 73 stores information related to the structures constituting the building.

  The control device 50 includes a photo reception unit 51, a determination unit 52, a position information transmission unit 55, a progress stage determination unit 56, a progress stage transmission unit 57, a structure information search unit 58, a structure information transmission unit 59, and a line check unit 60. And the confirmation result transmission part 61 is provided. The determination unit 52 includes an area determination unit 53 and a matching unit 54. These functional blocks can also be realized in various forms by hardware only, software only, or a combination thereof.

  The photo accepting unit 51 accepts a photo inside the building transmitted from the robot 5 or the construction site terminal 80. The determination unit 52 determines the position where the photograph accepted by the photograph acceptance unit 51 is captured. As described above, when the determination criterion data 71 is generated as a two-step determination criterion, that is, the region determination criterion and the position determination criterion, the region determination unit 53 determines the region where the photograph is captured according to the region determination criterion. The matching unit 54 determines the position where the photograph was taken according to the position determination criterion in the region determined by the region determination unit 53.

  The position information transmitting unit 55 transmits position information indicating the position where the photograph is taken to the apparatus that has received the photograph. As described above, when the criterion data 71 is generated by learning the relationship between the extracted information extracted from the two-dimensional image of the building and the viewpoint position and the line-of-sight direction when the two-dimensional image is generated. The determination unit 52 determines the position and the line-of-sight direction where the photograph is taken, and the position information transmission unit 55 transmits the position information and information indicating the line-of-sight direction.

  The progress stage determination unit 56 determines the progress stage from the photo received by the photo reception unit 51 in accordance with a determination criterion for determining the progress stage. As described above, the photograph used to determine the progress stage may be received from the automatic imaging device 4, or may be received from the robot 5, the construction site terminal 80, or the like. The progress stage determination unit 56 may determine the progress stage of building construction by comparing the building shown in the photograph received by the photo reception unit 51 with the three-dimensional shape data 72. For example, the progress stage may be determined by calculating the ratio of the volume or surface area of the building represented as the three-dimensional shape data 72 and the building shown in the photograph. As described above, when the determination criterion for determining the progress step learned from the two-dimensional image generated for each progress step is generated, the progress step determination unit 56 proceeds according to the determination criterion. The stage may be determined.

  The structure information search unit 58 determines the structure in the photo received by the photo reception unit 51 based on the position determined by the determination unit 52 or the position and the line-of-sight direction, and the determined structure Information regarding the object is retrieved from the structure information database 73. The structure information transmission unit 59 transmits the information searched by the structure information search unit 58 to the device that has received the photograph.

  The line check unit 60 compares the photograph accepted by the photograph accepting unit 51 with the three-dimensional shape data 72 to confirm whether or not the piping is constructed as designed. The confirmation result transmission unit 61 transmits the confirmation result by the line check unit 60 to the device that has received the photo.

  FIG. 4 shows a configuration of the construction site terminal according to the embodiment. The construction site terminal 80 includes a communication device 81, a display device 82, an input device 83, an imaging device 84, a storage device 85, and a control device 90.

  The communication device 81 controls wired or wireless communication. The display device 82 displays the display image generated by the control device 90. The input device 83 inputs information input from the user to the control device 90. The imaging device 84 images a building around the construction site terminal 80. The storage device 85 stores data and computer programs used by the control device 90.

  The control device 90 includes a photo transmission unit 91, a position information presentation unit 92, a progress stage presentation unit 93, a structure information presentation unit 94, and a confirmation result presentation unit 95. These functional blocks can also be realized in various forms by hardware only, software only, or a combination thereof.

  The photo transmission unit 91 transmits a photo of the building imaged by the imaging device 84 to the position determination device 40. The position information presentation unit 92 displays the position information transmitted from the position determination device 40 on the display device 82. The progress stage presentation unit 93 displays the progress stage transmitted from the position determination device 40 on the display device 82. The structure information presentation unit 94 displays information on the structure transmitted from the position determination device 40 on the display device 82. The confirmation result presentation unit 95 displays the result of the line check transmitted from the position determination device 40 on the display device 82.

  The configuration of the operator terminal 8 and the person in charge terminal 9 is the same as the configuration of the construction site terminal 80. However, the operator terminal 8 and the person-in-charge terminal 9 may not be provided with the progress stage presentation unit 93 and the confirmation result presentation unit 95.

  FIG. 5 shows an example of a building managed by the building management system according to the embodiment. FIG. 5 is a photograph after the completion of a plant as an example of a building.

  FIG. 6 shows an example of three-dimensional shape data obtained by modeling the plant shown in FIG. The three-dimensional shape data is created by CAD or the like.

  FIG. 7 shows an example of a two-dimensional image generated by rendering the three-dimensional shape data shown in FIG. A large number of such two-dimensional images are generated and provided to the learning device 10.

  FIG. 8 shows an example of a screen displayed on the display device of the construction site terminal. The display device 82 displays a photograph of the plant imaged by the imaging device 84, information related to the structure is displayed by the structure information presentation unit 94, and further, the result of the line check is displayed by the confirmation result presentation unit 95. It is displayed. In this way, by superimposing the photograph taken by the image pickup device 84 and displaying the information on the structure and the result of the line check, the information can be easily understood visually using augmented reality (AR). Can be presented. The construction site terminal 80 may acquire information from a progress management system or the like, and further display information related to progress management.

  FIG. 9 is a flowchart illustrating a procedure of the position determination method according to the embodiment. FIG. 9 shows a procedure executed by the learning apparatus 10. The two-dimensional image generation unit 22 generates a two-dimensional image of the building by setting the viewpoint position and the line-of-sight direction and rendering the three-dimensional shape data modeling the building (S10). The analysis unit 23 analyzes the two-dimensional image of the building generated by the two-dimensional image generation unit 22 and extracts information indicating the characteristics of the two-dimensional image (S12). The photograph acquisition unit 21 acquires a photograph of a building imaged by the automatic imaging device 4 or the like and information indicating a position where the photograph is captured (S14). The analysis part 23 analyzes the photograph of the building acquired by the photograph acquisition part 21, and extracts the information which shows the characteristic of a photograph (S16). The learning unit 24 learns the relationship between the extraction information extracted by the analysis unit 23 and the viewpoint position set when the two-dimensional image is generated or the position where the photograph of the building is captured, Criteria data used to determine the position from the photograph of the building is generated (S18).

  FIG. 10 is a flowchart illustrating a procedure of the position determination method according to the embodiment. FIG. 10 shows a procedure executed by the position determination device 40. The photo accepting unit 51 accepts a photo inside the building transmitted from the robot 5 or the construction site terminal 80 (S20). The determination unit 52 determines the position where the photograph accepted by the photograph acceptance unit 51 is captured (S22). The position information transmitting unit 55 transmits position information indicating the position where the photograph is taken to the apparatus that has received the photograph (S24).

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

  The technology of the present invention can be used not only for the plant but also for managing the construction and operation of buildings such as buildings. Moreover, it can be used not only for the inside and the periphery of a building but also for determining a position in an arbitrary object that can be imaged by an imaging device and has three-dimensional shape data.

  DESCRIPTION OF SYMBOLS 1 Building management system, 10 Learning apparatus, 21 Photo acquisition part, 22 Two-dimensional image generation part, 23 Analysis part, 24 Learning part, 25 Provision part, 40 Position determination apparatus, 51 Photo reception part, 52 Determination part, 53 area | region Determination unit, 54 matching unit, 55 position information transmission unit, 56 progress stage determination unit, 57 progress stage transmission unit, 58 structure information search unit, 59 structure information transmission unit, 60 line check unit, 61 confirmation result transmission unit, 80 Construction site terminal.

Claims (10)

A position determination device for determining a position where the photograph is captured from a photograph of a building image;
A learning device that generates, by machine learning, a determination criterion used by the position determination device to determine a position;
With
The learning device
A three-dimensional shape data storage unit for storing three-dimensional shape data modeling the building;
A two-dimensional image generation unit that generates a two-dimensional image of the building by rendering the three-dimensional shape data by setting a viewpoint position and a line-of-sight direction;
The determination criterion is generated by learning the relationship between the extraction information extracted from the two-dimensional image generated by the two-dimensional image generation unit and the viewpoint position set when generating the two-dimensional image. The learning department,
With
The position determination device includes:
A photo acceptance unit that accepts a photograph of the building,
A determination unit configured to determine a position where the photograph is captured based on the determination criterion generated by the learning device;
A building management system comprising: The learning device
A photo acquisition unit for acquiring a photograph of the building and position information indicating a position at which the photograph is captured;
The building management system according to claim 1, wherein the learning unit further learns the determination criterion by further using the photograph of the building and the position information acquired by the photo acquisition unit. The two-dimensional image generation unit generates a two-dimensional image inside the building by setting a viewpoint position inside the building,
The learning unit learns a determination criterion used for determining a position where the photograph is taken from a photograph taken inside the building,
The photo accepting unit accepts a photo taken inside the building,
The building management system according to claim 1, wherein the determination unit determines a position inside the building where the photograph is taken. The position determination device includes:
A structure information storage unit that stores information related to the structure constituting the building, and information indicating a position of the structure in the building;
A structure information presentation unit that determines a structure captured in the photograph based on the position determined by the determination unit, and presents information about the determined structure;
The building management system according to any one of claims 1 to 3, further comprising: The position determination device includes:
A three-dimensional shape data storage unit for storing the three-dimensional shape data of the building;
A progress stage determination unit that determines the progress stage of the building by comparing the photograph and the three-dimensional shape data;
The building management system according to claim 1, further comprising: The position determination device includes:
A three-dimensional shape data storage unit for storing the three-dimensional shape data of the building;
By comparing the photograph and the three-dimensional shape data, a confirmation unit for confirming the quality of construction of the building,
The building management system according to claim 1, further comprising: A three-dimensional shape data storage unit for storing three-dimensional shape data modeling a building;
A two-dimensional image generation unit that generates a two-dimensional image of the building by rendering the three-dimensional shape data by setting a viewpoint position and a line-of-sight direction;
The building was imaged by learning the relationship between the extraction information extracted from the two-dimensional image generated by the two-dimensional image generation unit and the viewpoint position set when generating the two-dimensional image. A learning unit that generates a criterion used to determine a position at which the photograph is captured from a photograph;
A learning apparatus comprising: A photo acceptance unit that accepts photographs of buildings,
Extraction information extracted from the 2D image generated by rendering the 3D shape data modeling the building by setting the viewpoint position and the line of sight, and set when generating the 2D image A determination unit that determines a position where the photograph is captured based on a determination criterion generated by learning a relationship with a viewpoint position;
A position determination apparatus comprising: A photo accepting unit that accepts a photograph of an imaging target captured by the captured image;
Extraction information extracted from the two-dimensional image generated by rendering the three-dimensional shape data modeling the imaging target by setting the viewpoint position and the line-of-sight direction, and was set when generating the two-dimensional image A determination unit that determines a position where the photograph is captured based on a determination criterion generated by learning a relationship with a viewpoint position;
A position determination apparatus comprising: Receiving a photograph of an imaging target captured by the captured image;
Extraction information extracted from the two-dimensional image generated by rendering the three-dimensional shape data modeling the imaging target by setting the viewpoint position and the line-of-sight direction, and was set when generating the two-dimensional image Determining a position at which the photograph is captured based on a criterion generated by learning a relationship with a viewpoint position;
A position determination method characterized by causing a computer to execute.