JP6868515B2 - Structure management device, structure management method, and structure management program - Google Patents

Description

The present invention relates to a structure management device for managing a structure such as a bridge, a structure management method, and a structure management program, and in particular, a technique for effectively grasping progressive deformation in a structure such as a bridge. Regarding.

Structures such as bridges should be managed in a series of steps such as inspection-planning-construction, and there is asset management as such a comprehensive management method. In asset management, it is important to manage the history of inspections and constructions appropriately and in chronological order, and in order to facilitate the management, it is common to use a drawing for structure management called a basic control chart.

To create a basic control chart, it is necessary to measure the bridge structure. Three methods are known as the measurement method: a method using a TS (total station), a method using a ground laser, and a method using a stereo camera. Further, as a management method of the basic control chart, there are a two-dimensional management method and a management method using photographed photographs. The two-dimensional management method is a method of managing a bridge structure with a two-dimensional drawing, and is often used when managing with a paper base or CAD. On the other hand, the management method using photographed photographs is to photograph the bridge structure with a digital camera or the like at the time of measurement or inspection of the bridge structure, and manage the photographed photograph. This management method is suitable when it is desired to record and manage the natural deformation of the bridge structure. Since the number of photographed photographs is enormous, it is common to digitize them and manage them on a computer in a list format or a catalog format.

However, according to such a conventional technique, there is a problem that the measurement of the bridge structure becomes complicated and the cost of creating the basic control chart is high. That is, bridge structures can be measured with high accuracy by using expensive measuring devices such as TS and ground laser, but the basic control chart is usually used as a base map for maintenance inspection work. , It doesn't have to be more accurate than necessary. In addition, the necessary requirement for creating a basic control chart is that there is little on-site measurement work and it is possible to easily create a basic control chart in indoor work. Further, although the method using a stereo camera is effective in terms of reducing the on-site measurement work, it is not feasible to create the bridge structure from scratch because it requires man-hours for indoor work.

Therefore, in order to solve the above problems, the applicant has proposed a structure management drawing creation device capable of easily creating a structure management drawing suitable for time series management (see Patent Document 1). ). According to the structure management drawing creation device disclosed in Patent Document 1, photographed photographs and deformed / repaired figures can be managed in time series, so that structure management drawings suitable for time series management can be easily obtained. It becomes possible to create.

Japanese Patent No. 479090

By the way, in the inspection work, from the viewpoint of preventive maintenance, it is important to appropriately record how the deformation such as cracks is progressing and manage the history in chronological order. Since the inspection work is a complicated and time-consuming work, it is required to be able to perform it efficiently with a simple operation. Specifically, as a request for continuous inspection, there is a request to improve the efficiency of the second and subsequent inspection work (to realize a data update method that anyone can easily perform). There is also a demand for accurate recording (to realize a time-series and consistent recording method) in order to grasp the progress of deterioration.

The present invention has been made to solve the above problems, and provides a structure management device, a structure management method, and a structure management program capable of effectively grasping progressive deformation in a structure. The purpose is to do.

The structure management device according to one aspect of the present invention is a structure management device for managing a structure, and includes a registration unit for registering structural parameters of the structure and a photograph of the structure, and the structure. A state in which a 3D model creation unit that creates a 3D model of an object using the structural parameters, a 3D model created by the 3D model creation unit, and a previously photographed photograph registered by the registration unit are associated with each other. Then, when the photograph taken this time is registered by the registration unit, the corresponding point acquisition unit that acquires the corresponding points between the previously photographed photograph and the photograph taken this time by the feature amount matching method and the corresponding point acquisition unit acquire the corresponding points. A correspondence point selection unit that selects the corresponding points, and a 3D space arrangement unit that arranges the photograph taken this time in the 3D space in association with the 3D model based on the correspondence points selected by the correspondence point selection unit. The gist is that we have prepared.

The structure management method according to one aspect of the present invention is a structure management method for managing a structure, which includes a registration step for registering a structural parameter of the structure and a photograph of the structure, and the structure. A state in which a 3D model creation step of creating a 3D model of an object using the structural parameters, a 3D model created in the 3D model creation step, and a previously photographed photograph registered in the registration step are associated with each other. Then, when the photograph taken this time is registered in the registration step, the corresponding point acquisition step of acquiring the corresponding point between the photograph taken last time and the photograph taken this time by the feature amount matching method and the corresponding point acquisition step are acquired. A corresponding point selection step for selecting the corresponding points selected, and a 3D space arrangement step for arranging the photograph taken this time in the 3D space in association with the 3D model based on the corresponding points selected in the corresponding point selection step. The gist is to include it.

The gist of the structure management program according to one aspect of the present invention is to make a computer function as the structure management device.

According to the present invention, it is possible to provide a structure management device, a structure management method, and a structure management program capable of effectively grasping progressive deformation in a structure.

It is a block diagram of the drawing making apparatus for structure management in a basic technique. It is an overview diagram of a stereo photographing apparatus in a basic technique. It is a figure which shows the basic control chart creation method in the basic technology. It is a figure which shows the member dimension measurement method in the basic technique. It is explanatory drawing of the mathematical formula in the basic technique. It is explanatory drawing of the determination function of the dimension measurement condition in a basic technique. It is explanatory drawing of the measurement accuracy determination function and the calibration function of the stereo photographing apparatus in the basic technique. It is a figure which shows the photograph | photograph management method in the basic technique. It is a figure which shows the secular change management method in the basic technology. It is a figure which shows the position calculation method of the deformed part in the basic technique. It is a figure which shows the display result at the time of removing the lens distortion and the like. It is a figure which shows the display result when the lens distortion and the like are not removed. It is explanatory drawing of the distortion (error) factor caused by the internal mechanism of a digital camera. It is a figure which shows the inspection, repair management (deformation measurement) method in a basic technique. It is a figure which shows the principle of accurately measuring the position and quantity of a deformed part. It is a flowchart which shows the operation of the drawing creation apparatus for structure management in a basic technique. It is a flowchart which shows the operation of the drawing creation apparatus for structure management in a basic technique. It is explanatory drawing of the bridge 3D model reconstruction processing in a basic technique. It is explanatory drawing of various data necessary for bridge 3D model reconstruction processing in basic technology. It is explanatory drawing of various data necessary for bridge 3D model reconstruction processing in basic technology. It is explanatory drawing of the 3D model of the bridge (Rahmen viaduct) in the basic technology. It is explanatory drawing of the 3D model of the bridge (Rahmen viaduct) in the basic technology. It is a flowchart which shows the bridge 3D model reconstruction processing in a basic technique. It is a flowchart which shows the bridge 3D model reconstruction processing in a basic technique. It is a top view of the skeleton member of a bridge column, beam, and deck in the basic technology. It is a top view of the skeleton member of a bridge column, beam, and deck in the basic technology. It is a top view of the skeleton member of a bridge column, beam, and deck in the basic technology. It is explanatory drawing of the development drawing creation process in a basic technique. It is explanatory drawing of the development drawing creation process in a basic technique. It is explanatory drawing of the development drawing creation process in a basic technique. It is a flowchart which shows the development drawing creation process in a basic technique. It is explanatory drawing of the deformation measurement processing in a basic technique. It is explanatory drawing of various data necessary for deformation measurement processing in a basic technique. It is a flowchart which shows the deformation measurement processing in a basic technique. It is explanatory drawing of the deformation measurement processing in a basic technique. It is explanatory drawing of the deformation measurement processing in a basic technique. It is explanatory drawing of the deformation measurement processing in a basic technique. It is explanatory drawing of the time series management method of the photograph and the deformed / repaired figure in the basic technique. It is explanatory drawing of the time series management method of the photograph and the deformed / repaired figure in the basic technique. It is a figure which shows the specific example 1 of the time series management of the photograph taken in the basic technique. It is a figure which shows the specific example 2 of the time series management of the photograph taken in the basic technique. It is a figure which shows the specific example 1 of time-series management of a deformation / repair part in a basic technique. It is a figure which shows the specific example 2 of the time series management of the deformation / repair part in the basic technique. It is a figure which shows the specific example 3 of the time series management of the deformation / repair part in the basic technique. It is explanatory drawing of the field of application of correspondence between the conventional photographs. It is explanatory drawing of the outline of this invention. It is explanatory drawing of the relationship determination in this invention. It is explanatory drawing of the relationship determination in this invention. It is explanatory drawing of the 1st sorting method in this invention. It is explanatory drawing of the 2nd and 3rd sorting methods in this invention. It is a block diagram of the drawing making apparatus for structure management in this invention. It is explanatory drawing of various data necessary for the progressive deformation grasping process in this invention. It is a flowchart which shows the progressive deformation grasping operation in this invention. It is a flowchart which shows the progressive deformation grasping operation in this invention. It is a flowchart which shows the progressive deformation grasping operation in this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same or similar parts are designated by the same or similar reference numerals. However, it should be noted that the drawings are schematic, and the relationship between the thickness and the plane dimension, the ratio of the thickness of each layer, etc. are different from the actual ones. Therefore, the specific thickness and dimensions should be determined in consideration of the following explanation. In addition, it goes without saying that the drawings include parts having different dimensional relationships and ratios from each other.

≪Basic technology≫
The present invention is an improved invention of the technique disclosed in Patent Document 1 (hereinafter, referred to as "basic technique"). Therefore, the contents of the basic technology will be explained first.

FIG. 1 is a configuration diagram of a drawing creation device 10 for structure management in the basic technique. The structure management drawing creation device 10 is a device for creating a management drawing of a structure such as a bridge (hereinafter, may be referred to as a “basic control chart”), and is registered as shown in FIG. Unit 11, basic control chart creation unit 12, output unit 13, storage unit 14, 3D (dimension) model creation unit 15, 3D spatial arrangement unit 16, 3D model reconstruction unit 17, and deformation measurement. It is provided with a part 18.

The registration unit 11 registers various information such as structural parameters of the structure and photographed photographs of the structure. As the photograph of the structure, a stereo photograph (right, left) taken by the stereo photographing device 20 can be used. The storage unit 14 is a memory for storing various types of information. The 3D model creation unit 15 creates a 3D model of the structure using structural parameters and the like. The 3D space arrangement unit 16 arranges the photographed photograph in the 3D space (X, Y, Z). The 3D model reconstructing unit 17 updates the member dimensions of the 3D model using photographed photographs, and reconstructs the 3D model using the updated member dimensions. The basic control chart creation unit 12 creates a control drawing using the reconstructed 3D model. The output unit 13 outputs the created management drawing. The deformation measuring unit 18 measures the position and quantity of the deformed portion on the photographed photograph. Hereinafter, each function will be described in detail.

(How to create a basic control chart)
As described above, according to the prior art, there is a problem that the measurement of the bridge structure becomes complicated and the cost of creating the basic control chart is high. Therefore, in the basic technology, the following method is adopted so that a basic control chart suitable for time series management can be easily created.

FIG. 2 is an overview view of the stereo photographing apparatus 20 in the basic technique. As shown in FIG. 2, the stereo photographing apparatus 20 is provided with a stereo pedestal 22 at one end of a monopod 21, and two digital cameras 23 are attached to the stereo pedestal 22 at predetermined intervals. A remote shutter cable 24 is connected to the two digital cameras 23, enabling simultaneous shutter release of the two cameras.

FIG. 3 is a diagram showing a method of creating a basic control chart in the basic technology. First, a desired bridge 3D model is selected from the bridge 3D models for each bridge structure. For example, among bridge 3D models by bridge structure such as "Rahmen high bridge", "Concrete single T girder (Gerber girder)", "Concrete single plate girder (Slab girder)", "PC structure (I type girder, T type girder)" When "Rahmen girder bridge" is selected, a screen for inputting structural parameters of the rigid frame girder bridge is displayed as shown in FIG. 3 (A). Next, when the structural parameters of the rigid frame viaduct are input, the initial bridge 3D model of the rigid frame viaduct is displayed as shown in FIG. 3 (B). When the initial bridge 3D model is displayed, the member dimensions are measured using stereo photographs. The details of this measurement method will be described later, but as shown in FIG. 3C, it is only necessary to indicate the portion to be measured on the photographed photograph. When the member dimensions are measured, as shown in FIG. 3D, a bridge 3D model reconstructed to the measured member dimensions is completed. When the output is instructed in this state, a basic control chart (for example, a developed view) is created using the completed bridge 3D model as shown in FIG. 3 (E), and the created basic control chart is output.

FIG. 4 is a diagram showing a member size measuring method in the basic technique. First, as shown in FIG. 4A, when the part (vertical beam) to be measured is selected, the corresponding photographed photograph (corresponding two left and right images) is displayed. Here, as shown in FIG. 4 (B), when two dimensional measurement points are specified on the left image, the positions on the right image corresponding to these two points are automatically set as shown in FIG. 4 (C). To be searched for. As a result, the XYZ coordinates of the two points are calculated by [Equation 1] to [Equation 3], and the dimensional value (6805 mm) is calculated based on these XYZ coordinates. Here, [Equation 1] is a relational expression of a stereo camera (model), [Equation 2] is _{a calculation expression of model coordinates (X, Y, Z) M} of a stereo model, and [Equation 3] is a calculation expression. , Real coordinates of the stereo model (X, Y, Z) _G calculation formula (see FIG. 5). In FIG. 5B, an orthogonal coordinate system is set for the measurement target, and the origin of the XYZ coordinates is set at the left camera position.

FIG. 6 is an explanatory diagram of a determination function of dimensional measurement conditions in the basic technique. In order to ensure the measurement accuracy, the following conditions must be satisfied according to the specifications of the stereo photographing apparatus 20.
(1) Shoot within a shooting distance of 25 m (recommended within 20 m). With a focal length of 24 mm (35 mm camera equivalent), the shooting angle of view is wide, so it is possible to shoot close to the target.
(2) Take a picture facing the measurement target. However, if it is unavoidable to shoot from an angle, the temperature should be within ± 30 degrees (recommended within ± 15 degrees).

Therefore, [Equation 4] and [Equation 5] are calculated at the time of measuring the member dimensions. [Equation 4] is a formula for calculating the distance L. The | vector OS | and | vector OE | in [Equation 4] are the distances from the shooting position O to the stations S and E, respectively. [Equation 5] is a calculation formula for the angle θ. The angle θ is calculated from the inner product of the vector MO and the vector MS, and the shooting angle | θ-90 | is obtained.

Here, it is determined whether or not the distance L is within 25 m (recommended within 20 m), and whether or not the shooting angle | θ-90 | is within 30 degrees (recommended within 15 degrees). If this dimensional measurement condition is not satisfied, a message indicating that the measurement accuracy may exceed a predetermined limit is displayed, and measures such as prompting the measurement using another photograph are taken.

FIG. 7 is an explanatory diagram of a measurement accuracy determination function and a calibration function of the stereo photographing apparatus 20 in the basic technique. That is, the structure management drawing creation device 10 has an accuracy determination function for determining whether or not the measurement using the stereo photographing device 20 satisfies a predetermined measurement accuracy and a predetermined measurement in order to surely secure the measurement accuracy. It has a calibration function when the accuracy is not met.

For example, as shown in FIG. 7A, when the size of the signboard is measured at the site with a measure, it is assumed that the length x width = 91 cm x 60 cm. In this case, as shown in FIG. 7B, when the measured value is input in mm units, it is determined whether or not the input measured value satisfies a predetermined measurement accuracy, and the determination result is displayed. Here, since it is assumed that the predetermined measurement accuracy is satisfied, “◯” is displayed as the determination result. On the other hand, when the predetermined measurement accuracy is not satisfied, as shown in FIG. 7C, the photographing parameters of the stereo photographing apparatus 20 are adjusted so that the error of the measured value is minimized. The judgment criteria are not particularly limited, but here, the case where the error is within 10 mm or the error rate is within 1% is marked with “◯”.

As described above, according to the basic technology, since the method of reconstructing the bridge 3D model using the photographed photograph and creating the basic control chart using the reconstructed bridge 3D model is adopted, the time series is adopted. It is possible to easily create a basic control chart suitable for management.

(Photograph management method)
According to the prior art, there is a problem that the management of photographed photographs is complicated and the positional relationship with the bridge structure cannot be grasped. In other words, in the measurement and inspection of bridge structures, it is important to know which part of the photograph was taken from which position. However, it is a very troublesome task to manage the photographed bridge portion and the photographed position in association with each other, and it is more difficult to grasp the positional relationship of each photographed photograph. Therefore, in the basic technology, the following method is adopted so that the positional relationship of the bridge structure can be visually grasped.

FIG. 8 is a diagram showing a photograph management method in the basic technique. First, in the structure management drawing creation device 10, photographed photographs taken for each bridge structure are registered. In this state, as shown in FIG. 8 (A), when the registered photograph is dragged and dropped to the sectioned portion such as the span of the target bridge or the column, the photograph is taken at the position in the dropped 3D space. The photo is placed. At this point, the layout is rough, but by associating the 3D model of the bridge with the photographed photograph, as shown in FIG. Correspondence becomes possible.

As described above, according to the basic technology, since the photograph taken is managed in the 3D space, it is possible to visually grasp the positional relationship of the bridge structure. Further, since the photographed photograph can be arranged at a free position in the 3D space, for example, it is possible to manage the photographed photograph of the bridge structure from the bottom to the top. Further, since the photographed photograph can be managed as a stereo, it is possible to measure the distance, the angle, the area, etc. using the photographed photograph. In addition, since it is possible to search for photographs taken by bridge structure, it is possible to manage photographs taken by bridge structure in chronological order.

(Aging management method)
According to the prior art, since the secular change of the bridge structure such as the repair history is managed by the two-dimensional drawing, there is a problem that it is difficult to imagine the actual bridge structure and it is difficult to grasp the secular change of the deformed part. Therefore, in the basic technology, the following method is adopted so that the secular change of the deformed part can be easily grasped.

FIG. 9 is a diagram showing a secular change management method in the basic technique. The above-mentioned photograph management method in 3D space can also be utilized for bridge maintenance and inspection work. That is, by managing a plurality of photographs taken of the target bridge in chronological order and for each bridge structural part, it is possible to realize the secular change management of the deformed part. Specifically, as shown by the rectangle drawn by the dotted line in FIG. 9 (A), the structure part of the part where the secular change is to be seen is selected in the bridge 3D model of the entire bridge structure. As a result, as shown in FIG. 9B, the aging viewer is displayed for the selected structural part. When displaying this aging viewer, it is possible to display a screen for describing / editing the shooting date of the shot photo and a memo at the time of inspection, and also to display the target shot photo as a thumbnail.

As described above, according to the basic technology, it is possible to manage the secular change in the 3D space using the bridge 3D model, so that it is easy to imagine the 3D structure and easily grasp the secular change of the deformed part. It becomes possible.

(Deformity measurement method)
According to the prior art, there is a problem that it is not easy to accurately and safely measure the position and quantity of the deformed portion. That is, in order to measure the position and quantity of the deformed part occurring in the actual bridge structure, it is necessary to approach the target part with an aerial work platform or the like and measure with a scale. Has problems in terms of the workload of preparatory work and the safety of workers. It is also possible to perform inspection work with a zoom camera, but in this case, the measurement of the position and quantity of the deformed part lacks accuracy. Further, when the result is described in the basic control chart, the description itself becomes a schematic diagram. Therefore, in the basic technology, the following method is adopted so that the position and quantity of the deformed part can be measured accurately and safely.

FIG. 10 is a diagram showing a method of calculating the position of a deformed portion in the basic technique. That is, as shown by points 1 to 4 in FIG. 10, the bridge 3D model and the photograph taken are associated with each other. This makes it possible to accurately calculate the position of the deformed portion based on the bridge 3D model in consideration of the lens distortion of the photographed photograph. Hereinafter, the difference between the case where the lens distortion and the like are removed and the case where the lens distortion is not removed will be described in detail.

FIG. 11 is a diagram showing a display result when lens distortion and the like are removed. That is, when lens distortion and the like are removed, as shown in FIG. 11A, a photograph taken after the shooting position is accurately calculated is arranged. Then, when the arranged photographed photograph (for example, the photographed photograph surrounded by a circle) is double-clicked, this photographed photograph is displayed superimposed on the bridge 3D model as shown in FIG. 11B. As is clear from this figure, the lens distortion on the barrel has been removed, so the bridge 3D model and the photograph taken are exactly the same.

FIG. 12 is a diagram showing a display result when lens distortion and the like are not removed. Here, as in FIG. 11B, the photographed photograph is shown in a state of being superimposed on the bridge 3D model. However, since the lens distortion on the barrel is not removed, as is clear from FIG. 12, the gap between the bridge 3D model and the photograph taken is large in the peripheral portion.

FIG. 13 is an explanatory diagram of distortion (error) factors caused by the internal mechanism of the digital camera. That is, the digital cameras on the market for consumer use are not manufactured for photogrammetry. Therefore, in the basic technology, since a commercially available digital camera for consumer use is used for measurement, the nominal focal length correction amount (Δf), the lens center position deviation (xp ₀ , yp ₀ ), and the lens distortion amount ( The radial direction (barrel type) lens distortion coefficient k1, k2) is accurately tested and calculated. This work is called camera calibration in photogrammetry. Therefore, the focal length correction amount (Δf), the lens center position deviation (xp ₀ , yp ₀ ), and the radial direction lens distortion coefficient (k1, k2) are referred to as camera calibration data. In addition, the tangier direction (pincushion type) lens distortion coefficient (p1, p2) may be added. The formula for calculating the amount of distortion correction (Δx, Δy) including the lens distortion coefficient in the tangier direction is shown below. ^{r 2 = x '2 + y} ' 2, is the (x, y) photo coordinate.

Δx = (x / f) Δf + x _p0 − x (k ₁ r ² + k ₂ r ⁴ ) − (p ₁ (r ² + 2 x ² ) + p ₂ xy)
Δy = (y / f) Δf + y _p0 − y (k ₁ r ² + k ₂ r ⁴ ) − (p ₁ (r ² + 2y ² ) + p ₂ xy)
FIG. 14 is a diagram showing an inspection and repair management (deformation measurement) method in the basic technique. That is, in the basic technique, a deformation measurement method using a photograph taken in a 3D space is adopted. Specifically, as shown in FIG. 14A, a bridge 3D model created from the bridge set is selected, and feature points are associated between the bridge 3D model and a single photograph. As a result, the shooting position is automatically acquired, and the single photograph is automatically arranged in the 3D space. Next, as shown by the rectangle in FIG. 14 (B), the deformation on the single photograph arranged in the 3D space is traced. As a result, the position and quantity (length, area, etc.) of the traced deformed part are measured, and a figure showing the deformed part is drawn on the bridge 3D model and output on the basic control chart. It is also possible to reflect the measured position and quantity of the deformed portion in a drawing other than the developed view (such as a deformed developed view).

FIG. 15 is a diagram showing a principle of accurately measuring the position and quantity of the deformed portion. That is, according to the basic technology, the positional relationship between the bridge 3D model and the photograph taken is accurately obtained. Therefore, it is possible to determine which surface of the member constituting the bridge 3D model the deformed portion on the photographed photograph corresponds to. For example, as shown in FIG. 15, the points A, B, C, and D on the photograph L correspond to the points A', B', C', and D'on the corresponding member surfaces L', respectively. .. As a result, as described with reference to FIG. 14, when a deformed portion is input on a single photograph, the position and quantity of the deformed portion can be accurately measured.

As described above, according to the basic technology, it is possible to associate the bridge 3D model in consideration of the lens distortion of the photographed photograph with the photographed photograph, so that the position and quantity of the deformed part can be measured accurately and safely. It becomes.

(Operation of drawing creation device 10 for structure management)
FIG. 16 is a flowchart showing the operation of the structure management drawing creation device 10 in the basic technique. Hereinafter, an operation of creating a structure management drawing will be described with reference to FIG.

First, the target bridge information is reserved and registered (step S1). Specifically, enter the bridge name, bridge type, route name, bridge code, set number, start / end station name, km, bridge type, planned shooting person, scheduled shooting date, and structural parameters for each target bridge structure. , Instruct "reservation registration". As a result, various information of the target bridge is reserved and registered in the project DB on the storage unit 14.

Next, the target bridge is photographed by the stereo photographing apparatus 20 (step S2).

Next, the target bridge information is officially registered (step S3). Specifically, based on the results of on-site photography, the bridge information registered for reservation is corrected and "main registration" is instructed. As a result, various information of the target bridge is registered in the project DB, and an initial bridge 3D model is created using the registered bridge type and its structural parameters (step S4).

Next, the stereo photograph is registered (step S5). Specifically, the stereo photographing device 20 is connected to the PC, and the photographed photograph is taken into the PC from the recording medium in the digital camera. The recording media in the left and right cameras are given the names of the left camera and the right camera in advance. Therefore, when the captured photos are imported, the shooting times are synchronized and paired, and the left and right photos are automatically listed. If you check the shooting time and the contents of the left and right photos and instruct "Register", the stereo shot photos will be registered.

Next, member information is added (step S6). Specifically, the stereo photograph is associated with the member information and the rough position information. By this step S5, the dimensional measurement operation of the member and the 3D space arrangement operation of the photographed photograph can be smoothly performed.

Next, the member dimensions are measured (step S7). Specifically, after selecting the member to be measured, the stereo photograph of the member is called, and both ends of the member are indicated by the mouse cursor on the photograph. As a result, the positions of the two designated points are automatically searched and the member dimensions are measured.

Next, it is determined whether or not the measurement using the stereo photographing apparatus 20 satisfies a predetermined measurement accuracy (step S8). Here, if the predetermined measurement accuracy is not satisfied, the stereo photographing apparatus 20 is calibrated (steps S8 → S9). On the other hand, when the predetermined measurement accuracy is satisfied, the member dimensions are updated, and the bridge 3D model is reconstructed using the updated member dimensions (steps S8 → S10).

Next, a basic control chart is created and output (steps S11 → S12). Specifically, a basic control chart (for example, a development chart) is created using the completed bridge 3D model, and the created basic control chart is output to a display, a printer, or the like. There are various output formats such as jpg format and dxf format, and the output format is not particularly limited.

FIG. 17 is a flowchart showing the operation of the structure management drawing creation device 10 in the basic technique. Hereinafter, an operation of inputting a deformed portion will be described with reference to FIG.

First, the deformed portion is photographed by the stereo photographing apparatus 20 (step S21). The camera used for this shooting may be a camera with a zoom function, and a single camera may be used.

Next, the target bridge information is opened (step S22). Specifically, the project of the bridge to be input of the deformed part is opened from the project DB.

Next, the photographed photograph is registered and the camera to be used is selected (step S23). Specifically, the photographed photograph is taken into the PC and the photographed photograph is registered. Also, set the information of the camera used.

Next, member information is added (step S24). Specifically, the stereo photograph is associated with the member information and the rough position information. By this step S24, the dimensional measurement operation of the member and the 3D space arrangement operation of the photographed photograph can be smoothly performed. It is also possible to omit this step S24 and execute the next step S25.

Next, the bridge 3D model is associated with the photographed photograph (step S25). Specifically, at least four or more identical points are indicated for each of the bridge 3D model and the photographic image, and the relational expression between the bridge 3D model and the photographic image is calculated. When the association is completed, the shooting position and the shooting direction are accurately obtained, and this information is used to accurately arrange the shot photograph in the 3D space. Since the placed photographed photograph exactly overlaps with the bridge 3D model, the deformed part input on the photographed photograph is projected on the actual coordinate system of the bridge 3D model, and the position and quantity of the deformed part can be accurately measured. It will be possible.

Next, the deformed portion is input (step S26). Specifically, the data of the deformed portion input on the photographic image is converted from the image coordinate system to the real coordinate system on the bridge 3D model, and the position and quantity on the target member are calculated. In addition, when inputting a deformed part, attribute information such as cracks, peeling, and floating of reinforcing bars and judgment classification are registered together with input coordinate value data.

Finally, the deformed portion is combined with the basic control chart and output (step S27). Specifically, the measured deformed part is combined and output on a basic control chart (for example, a developed view) as a separate layer.

(Characteristic processing)
As described above, according to the structure management drawing creation device 10 of the basic technology, it is possible to execute a new process which has not been conventionally performed. Among them, the characteristic processes are listed below.

(1) Bridge 3D model reconstruction process The member dimensions of the initial bridge 3D model are updated using photographed photographs, and the bridge 3D model is reconstructed using the updated member dimensions. At that time, the positions and dimensional values of all the members of the bridge 3D model are updated in real time in consideration of the relationship between the members to be updated and other adjacent members, and the bridge 3D model is deformed and reconstructed. For example, FIGS. 18 (A), (B), and (C) show a bridge 3D model before reconstruction, and FIGS. 18 (D), (E), and (F) show a bridge 3D model after reconstruction. Here, when the two members shown by the arrows in FIG. 18A are updated, various members adjacent to these members are automatically deformed. Specifically, as shown in FIGS. 18 (B) and 18 (E), the shape of the floor slab is deformed from a rectangle to a rectangular shape, and as shown in FIGS. 18 (C) and 18 (F), the pillar shape is also changed from a rectangle. It will be transformed into a rectangular shape.

19 and 20 are explanatory views of various data required for the bridge 3D model reconstruction process. Hereinafter, the bridge 3D model reconstruction process will be described in more detail with reference to these figures.

The basic control chart creation DB 30 is a database stored in the storage unit 14, and includes a project information table 31, a stereo camera information table 32, and a camera information table 33 as shown in FIG. As shown in FIG. 20A, the project information table 31 is a table for storing project information such as "ID", "project name", and "scheduled input person". As shown in FIG. 20B, the stereo camera information table 32 is a table for storing stereo camera information such as "ID", "camera pair name", and "shooting method". As shown in FIG. 20C, the camera information table 33 is a table for storing camera information such as "ID", "camera name", and "serial number". The camera pair ID in the project information table 31 is associated with the camera pair name in the stereo camera information table 32, and the left camera ID and the right camera ID in the stereo camera information table 32 are the camera information table 33. It is associated with the camera name inside.

As shown in FIG. 19, when the project data 34 is set in the project information file 35, the project data 34 is also stored in the project information table 31 (the same applies at the time of updating). The bridge structure and structural parameters set in the project data 34 are described in the bridge 3D model information file 36. The 3D model data folder 37 is a folder for 3D model data of each member constituting the bridge 3D model described in the bridge 3D model information file 36. When the member dimensions are measured, the member dimension values in the 3D model data folder 37 are updated. The development drawing internal file 38 is an internal file created for development drawing output based on the bridge 3D model information. Based on this internal file 38, a development drawing file 39 such as a jpg format or a dxf format is output. In the stereo photographic information file 40, stereo photographic information (for example, an image file name of a pair for each stereo pair, corresponding point information when arranged in 3D space, shooting position information, etc.) is described. The actual photo image is stored in the left photo image data folder "L" 41 and the right photo image data folder "R" folder 42. The measurement data at the time of stereo measurement is temporarily stored in the storage memory, but is not output to a file.

21 and 22 are explanatory views of a 3D model of a bridge (Rahmen viaduct).

As shown in FIGS. 21 (A) and 21 (B), the bridge 3D model is composed of 3D models of each member such as columns, vertical beams, horizontal beams, and floor slabs. The 3D model data of these members are "pillar i-j.x", "vertical i-j.x", "horizontal i-j.x", "floor i-j.x", and the like. These members are updated based on dimensional measurements from stereo photographs to reconstruct the entire bridge 3D model. As shown in FIG. 22, the column position is defined by i (= 1, ..., M) and j (= 1, ..., n), and the column ij, the beam (vertical ij, horizontal ij, Determine the part name of middle vertical / middle horizontal ij). m is the number of pillars in the line direction (direction from the start point to the end point), and n is the number of pillars in the line orthogonal direction (direction from left to right).

FIG. 23 is a flowchart showing the bridge 3D model reconstruction process.

First, when the initial bridge 3D model is created (step S31) and the member dimensions are measured (step S32), the dimensions of the target member are changed (step S33), and the bridge 3D model information connected to the target member is obtained. It is read (step S34). Whether or not it is connected (adjacent) to the target member can be determined based on the contents of the bridge 3D model information file 36.

Next, the influence on the column 3D model data is determined based on the bridge 3D model information (step S35). The column 3D model data is the 3D model data of the columns constituting the bridge.

Here, if it is determined that there is no influence on the column 3D model data, the 3D model data of the target member is updated (step S36). When there is no effect on the pillar 3D model data, (1) when changing the height of the balustrade, (2) when changing the length of the floor slab overhang, (3) when the middle layer vertical (horizontal) beam This refers to the case where the dimensional change of the member itself does not affect other members, such as when changing the height and width.

On the other hand, when it is determined that there is an influence on the column 3D model data, the column 3D model data connected to the target member is extracted (step S37), and the extracted column 3D model data is used to change the dimensions of the target member. It is updated based on (step S38). "Update of column 3D model data" means updating the position XY and dimensions of the column. Further, the 3D model data of the member connected to the column 3D model updated in this way is extracted (step S39), and the 3D model data of the extracted member is similarly updated (step S40).

When the update of the 3D model data of all the connecting members is completed (step S41), the reconstructed bridge 3D model is created (step S42). The above process is repeated every time all member dimensions are measured (steps S43 → S32).

FIG. 24 is a flowchart showing a bridge 3D model reconstruction process, and FIGS. 25, 26, and 27 are plan views of skeleton members of bridge columns, beams, and decks. Hereinafter, using these figures, updating the column 3D model data (FIG. 23, step S38) and updating the member 3D model data (FIG. 23, step S40) will be described in more detail.

First, when the leveling length is changed from L0 to L1 (step S51), it is determined whether or not this dimensional change is applied to all the leveling (step S52). Here, as shown in FIG. 25 (A), when the dimensional change is applied to all horizontal columns, the column i-1 is selected as the reference (fixed) column (step S53), and the reference column i-1 is selected. After moving the reference pillar i-1 by ΔL to determine the center position (step S54), the vertical beam i is moved by ΔL (step S55). ΔL is “L1-L0”. Further, when the dimensions of the sideways i are changed from L0 to L1 (step S56), all the dimensions of the sideways are updated as shown in FIG. 25 (B).

On the other hand, as shown in FIG. 26 (A), when the dimensional change is applied to one horizontal column, the column 1-1 is selected as the reference column (step S57). Then, as shown in FIG. 26 (B), after the posts 1-2 is moved by ΔL in order to determine the center position of the pillars 1-2 (step S58), the linear equation, such as _{l b} and _{l 1} Calculation (step S59) is performed to determine the position of the pillar i-2. As a method for determining the position of the column i-2, determined and pillars 1-2 position after the movement, a straight line l _b passing through the column 4-2 position farthest from the column 1-2 positions , preferably in the respective columns 2-2 positions and posts 3-2 position an intersection of the straight line _{l b} and the straight line _{l 2} and _{l 3.} In this way, the positions of the individual pillars i-2 can be determined accurately and at high speed.

Then, as shown in FIG. 26 (C), calculates the straight line l _b and the angle between the straight line l ₁ theta (step S60), after calculating the column dimensions W'after deformation, the post-vertical beams and horizontal beams The shape of is changed (step S61 → S62). Specifically, as shown in FIG. 26 (D), a parallel line of a straight line l _b than the dimension of the pillars auxiliary linear l _b ⁺ and l _b ^- to change the vertical burr shape seeking. Then, from the dimension of the cross beam is a parallel line of a straight line _{l 1} auxiliary linear _l ^{1 +} and _{l 1} ^- a determined, straight _l ^{b +} a _{l 1} ^- the intersection of the intersection of the straight line _l ^{b +} a _l ^{1 +} And change the horizontal shape. The deformation angle of the _{column can be calculated based on the angle θ formed by the straight lines l b} and l _1, and if the deformation angle of the column can be calculated, the dimension W'after deformation can be calculated from the dimension W of the square or rectangle before deformation. .. When the shapes of all columns, horizontal beams, and vertical beams are changed (step S63), a pseudo-affine transformation formula is calculated (step S64), and as shown in FIG. 26 (E), the pseudo-affine transformation formula is used. The shape of the floor slab is changed (step S65).

Further, a case of changing the vertical beam so as to be shorter will be described. In this case, as shown in FIG. 27 (A), the length is adjusted along the current vertical beam direction with the pillar marked with a circle as the base point. Specifically, as shown in FIG. 27 (B), to secure the pair of pillars, it obtains a straight line l ₁ between the moved column position. Then, the parallel auxiliary straight lines _l ^{1 +} and _{l 1} of the straight line _{l 1} ^- seeking, linear _l ^{b +,} linear _{l b} ^-, linear _l ^{a +,} linear _{l a} ^- Calculating the intersection of the FIG. 27 (C ), The shapes of columns, vertical beams, and horizontal beams can be changed.

(2) Development map creation process A development map, which is a basic control chart, is created from the completed bridge 3D model. At that time, instead of developing all the surfaces of the individual 3D models constituting the bridge 3D model, as shown in FIGS. 28 and 29, the surfaces in contact with other members (surfaces that cannot be seen from the outside) are Do not expand. That is, the surface to be included in the development drawing is determined in consideration of the position and the adjacent relationship of each member. In this way, there is an advantage that the management becomes easy because the development view does not include the surfaces unnecessary for managing the bridge. As shown in FIG. 30, it is also possible to embed a photograph taken corresponding to the member included in the developed view in the developed view.

FIG. 31 is a flowchart showing a development drawing creation process.

First, when the bridge 3D model is created (step S71) and the bridge 3D model information connected to the target member is read (step S72), the surface to be output as a development view is determined based on the bridge 3D model information. (Step S73). Here, when the entire area of the surface is a surface that is not adjacent to other members (step S74) and is a surface that can be seen from the outside (step S75), it is determined that the surface should be output as a developed view. There is. "The entire area of the surface is not adjacent to other members" means that even a part of the surface has an area that is not adjacent to other members.

When the surface to be output as the developed view is determined in this way, the surface is registered in the developed view output memory (step S76), the members are grouped (step S77), and then the surface is arranged and output at the designated position (step S77). Step S78). When the development drawing output is completed for all the members (step S79), the development drawing internal file is output (step S80), and the development drawing is output in jpg format, dxf format, or the like (step S81).

(3) Deformity measurement processing By associating the created bridge 3D model with the photographed photograph, if the deformed part (crack, peeling, exposed reinforcement, etc. of concrete) is input on the photographed photograph, the position and quantity of the deformed part Is converted from the image coordinate system to the real coordinate system of the bridge 3D model, and the length and area of the deformation are measured accurately. As a result, as shown in FIG. 32, when the deformed portion is input on the photograph 1 in which the deformed portion is photographed, the deformed portion is accurately reflected in the developed view 1. When a new deformed part is photographed for the same member in the subsequent inspection work, the new deformed part is input on this photographed photograph 2. As a result, the developed view 2 showing the result of the deformation so far is imported, and the new deformed part is reflected in the developed view 2 (see the developed view 3).

FIG. 33 is an explanatory diagram of various data required for the deformation measurement process. The difference from FIG. 19 is that the deformation measurement information file 43, the deformation measurement internal file 44, and the deformation diagram file 45 are added. The deformation measurement information file 43 is a file that stores measurement information of the deformation location. The deformation measurement internal file 44 is an internal file created for deformed diagram output based on the measurement information of the deformed portion. Based on this internal file 44, a modified figure file 45 such as a jpg format or a dxf format or a modified quantity table 45 in a csv format is output.

FIG. 34 is a flowchart showing the deformation measurement process, and FIGS. 35, 36, and 37 are explanatory views of the deformation measurement process. Hereinafter, the deformation measurement process will be described in more detail with reference to these figures.

First, a photographic image from which lens distortion has been removed is created (step S91), and a corresponding point between the bridge 3D model and the photographic image is acquired (step S92). Next, the shooting position (Xo, Yo, Zoo) and the shooting direction angle (Ω, Φ, Κ) are calculated (step S93), and the deformation (length / area) on the photographic image is measured (step S94). Further, as shown in FIG. 35, the line-of-sight vector Vs from the shooting position (viewpoint) is calculated to extract the 3D model plane that intersects the bridge 3D model (step S95 → S96), and the number of the extracted 3D model planes is calculated. Determine (step S97).

Here, if the 3D model surface is not extracted, a message such as "The target 3D surface does not exist. Please use another photo." Is displayed (step S98). When one 3D model plane is extracted, the intersection coordinates XYZ of the line-of-sight vector and the target 3D model plane are calculated as shown in FIG. 36 (step S99). When two or more 3D model planes are extracted, as shown in FIG. 37, the plane normal vector _VN and the line-of-sight vector distance _SL are calculated (step S100), and the optimum plane is selected (step S101). ). The optimal surface, an optimal surface of the extracted two more 3D model surface, for example, (1) the angle of the V _S and V _N theta is or surface closest to 180 degrees, (2) S _The surface closest to L is selected (step S102). Thus, (1) about the near surface angle θ is 180 degrees of V _S and V _N to the operator are directly opposite, and (2) nearest the surface S _L is looked from the worker Since it is in the foreground, the optimum surface can be selected.

Next, the calculated XYZ coordinates are converted into surface XY coordinates (step S103), and the attribute information of the deformation measurement result is registered (step S104). Then, when the deformation measurement is completed for all the locations, the deformation measurement data is output (steps S105 → S106). If there is a past deformation measurement result, the past deformation measurement data is read, superimposed on the developed view, and output (step S107 → S108 → S109).

(4) Time-series management According to the structure management drawing creation device 10 of the basic technology, photographed photographs, the position of deformed / repaired parts (which member, which position), shape (figure), quantity ( Length and area) can be managed in chronological order. The time-series management of captured photographs is mainly realized by the registration unit 11, the 3D spatial arrangement unit 16, and the stereo photo information file 40. That is, the registration unit 11 describes the attribute information of the photographed photograph (single photograph) in the stereoscopic photograph information file 40, and the 3D space arrangement unit 16 refers to the stereoscopic photograph information file 40 when arranging the photographed photograph in the 3D space. And use the attribute information of the photograph taken. Further, the time-series management of the position, shape, and quantity of the deformation / repair location is mainly realized by the deformation measurement unit 18, the basic control chart creation unit 12, and the deformation measurement information file 43. That is, the deformation measurement unit 18 describes the attribute information of the deformation / repair figure in the deformation measurement information file 43, and the basic control chart creation unit 12 describes the deformation measurement information file 43 when creating the basic control chart. Use the attribute information of the deformed / repaired figure by referring to. Hereinafter, such time series management will be described in detail with reference to the drawings.

FIG. 38 is an explanatory diagram of a time series management method of a photograph and a deformed / repaired figure. First, the construction (photographing) history and the inspection (photographing) history can be set on the stereo photo registration screen as shown in FIG. 38 (A). For example, suppose that a bridge is photographed four times as follows.

(1) 2007/12/10 base drawing creation photography (2) 2009/01/10 normal general inspection photography (3) 2009/12/14 ○○ construction-00-00
(4) Normal general inspection photography on November 26, 2010 All these photographs will be registered in the project of the target bridge. Next, the attributes of the photographed photograph are set. Here, as shown in FIG. 38 (A), the radio button is used to select which of "basic drawing creation", "inspection information", and "construction information" the photographed photograph corresponds to. When "Create base map" is selected, "shooting date and time", "imaging type", "shooting name" and the like can be set as shown in FIG. 38 (B). When "inspection information" is selected, "inspection date and time", "inspection type", "inspection name" and the like can be set as shown in FIG. 38 (C). When "construction information" is selected, "construction date and time", "construction type", "construction name" and the like can be set as shown in FIG. 38 (D). The inspection name is automatically set using the inspection date and time and the inspection type, and the construction name is automatically set using the construction date and time and the construction type. Of course, the inspection name and construction name set in this way can be manually corrected. Next, as shown in FIG. 38 (E), photographed photographs of inspection or construction history are registered and managed. The inspection or construction history of deformed / repaired figures can inherit the construction history of the entered photograph. In the following, the construction history will be mainly illustrated and explained, but the same applies to the inspection history.

FIG. 39 is an explanatory diagram of a time series management method of a photograph and a deformed / repaired figure. First, as shown in FIG. 39 (A), a display condition (display set) is selected on the time series management screen. Here, "all shooting" is selected as the display condition. Other display conditions include "2007/12/10 base drawing creation", "2009/01/10 normal general inspection", "2009/12/14 XX construction-00-00", and "2010/11/26 normal". There are "general inspection photography" and "latest construction". By selecting the display conditions, the On / Off of the check box can be set efficiently. When a desired construction history is selected on this time-series management screen, as shown in FIGS. 39 (C), (D), and (E), a photograph of the target, a deformed / repaired portion, and a developed view are displayed.

FIG. 40 is a diagram showing a specific example 1 of time series management of photographed photographs. First, when the construction history is selected, as shown in FIG. 40 (A), the photographed photograph of the object is displayed on the photograph list screen. Further, as shown in FIG. 40 (B), the photograph of the object is arranged in the 3D space and displayed on the 3D space screen. In this state, when the photograph taken by the circle in FIG. 40 (B) is double-clicked, the information of the photograph is shown in FIG. 40 (C) (for example, "2009/01/10 normal general inspection"). Is displayed on the status bar. Here, when the arrow keys "←", "→", "↑", and "↓" are pressed with the desired member selected, the photographed photographs of the member are sequentially displayed. The display order (movement method) of the photographed photographs is not particularly limited. For example, if you press the "←" and "→" keys, you will move within the same construction, if you press the "↑" key, you will move to the past, and if you press the "↓" key, you will move to a different construction. It has become.

FIG. 41 is a diagram showing a specific example 2 of time series management of photographed photographs. This specific example corresponds to the actual screen of the software shown in FIG. When a desired member is selected on the upper left screen of the figure, a list of thumbnail photographs of the member is displayed as shown by the arrow (a). When a desired thumbnail photo is double-clicked in this state, the management screen of the photographed photo is displayed as shown by the arrow (b). Also, when a deformed figure is selected on the lower left screen in the figure, the management screen of the photograph taken associated with the deformed figure is displayed as shown by the arrow (c). ..

FIG. 42 is a diagram showing a specific example 1 of time-series management of a deformed / repaired portion. First, when the construction history is selected, as shown in FIG. 42 (A), the graphic of the deformed / repaired portion input at the time of the construction is displayed on the 3D space screen. When a photograph (not shown) on the 3D space screen is selected in this state, the information of the photograph is displayed in a pop-up as shown in FIG. 42 (B). Therefore, when a deformed / repaired part is selected, comments and judgment categories are displayed for the deformed / repaired part. Here, you can switch between the photo and the transformation by pressing the space key.

FIG. 43 is a diagram showing a specific example 2 of time-series management of deformed / repaired parts. As shown in FIG. 43 (A), the construction history is also selected on the input photographic image screen of the deformed / repaired portion, and the deformed / repaired portion of the target is displayed. As a result, it is possible to easily grasp what kind of situation the past deformation / repaired part has changed to the present. Further, as shown in FIG. 43 (B), the deformed / repaired portion is similarly displayed for each construction history on the display screen of the developed view. As a result, it is possible to grasp in chronological order how the deformed / repaired parts are distributed and progressing, which can be useful for repair planning.

FIG. 44 is a diagram showing a specific example 3 of time-series management of deformed / repaired parts. As shown in FIG. 44 (A), the construction history is also selected in the aggregate output of the deformed / repaired parts, and the deformed / repaired parts of the target are displayed. As a result, the amount of deformation and the amount of repair for each member can be totaled for each construction history. Further, as shown in FIG. 44 (B), it is possible to grasp the change in the deformed area and the repaired area from the aggregated results at different times.

In addition, although the time-series management and tabulation of the figures of the deformed / repaired parts have been mainly described here, the basic technology is not limited to this. That is, more specifically, the target for time-series management and aggregation of the deformed / repaired figure is the position (which member, which position) of the deformed / repaired part, the shape (figure), and the shape (figure). Quantity (length and area).

As described above, according to the structure management drawing creation device 10 in the basic technology, it is possible to manage photographed photographs and deformed / repaired figures in time series. That is, the photographed photograph can be managed in the 3D space for each construction history. In addition, the photographed photos can be managed for each construction history on the photo list screen and the list list screen. Similarly, the input deformed / repaired figures can be managed for each construction history on the 3D space screen, the developed view screen, and the input photograph screen. In addition, when inputting a photograph, the input deformed / repaired figure can be superimposed and displayed. Further, by designating the construction history in the total output of the deformed / repaired figures, the deformed / repaired parts at that time can be totaled.

Although not specifically mentioned in the above description, the development drawing creation process and the deformation measurement process are not necessarily processes on the premise that the reconstruction process has been performed. That is, even if the initial 3D model is used instead of the 3D model after the reconstruction process, the development drawing creation process and the deformation measurement process may be performed by the same method as described above.

That is, in the deformation measurement process, accurate measurement is possible by accurately associating the 3D model with the photograph taken. In other words, the early 3D model is an unfinished 3D model that differs from the actual dimensions of the bridge structure and lacks accuracy. However, many bridges are of standard type, and for standard type bridges, development drawing processing and deformation measurement processing can be performed without any problem even if an early 3D model is used.

<< Embodiment of the present invention >>
Next, the present invention will be described focusing on the differences from the basic technique. The present invention is an improved invention of the basic technique, and is a technique for effectively grasping progressive deformation by associating photographs with each other. Of course, the present invention includes all the contents of the basic technique.

(Conventional photo-to-photo correspondence application field)
Correspondence between photographs is to find a point (corresponding point) at the same position between two or more photographs.

FIG. 45 is an explanatory diagram of a conventional field of application of correspondence between photographs (Source: Computer Vision State-of-the-art Guide 2). Specifically, FIG. 45A shows an example of image matching (mosaic image creation) by searching for corresponding points. FIG. 45B shows an example of feature point tracking (person tracking). FIG. 45C shows an example of object recognition (road sign recognition) using a specific image. As shown in these figures, many application examples have been introduced so far for feature matching, which is a method of associating photographs with each other.

However, these are application cases of computer vision in image analysis, not application cases in the field of maintenance and management of infrastructure such as bridge structures. In particular, in order to grasp the progress of deformation, it is necessary to accurately associate the photograph taken with the 3D model of the bridge structure. Therefore, in the present invention, a method of effectively grasping the progressive deformation by accurately associating the photographed photograph with the 3D model by using the photographed photograph arranged in the 3D space and associating the photographs with each other. Is adopted.

(Outline of the present invention)
FIG. 46 is an explanatory diagram of an outline of the present invention. Here, the structure management drawing creation device 10 is used to associate photographs with each other to grasp and record progressive deformation. The structure management drawing creation device 10 may be a computer installed indoors, or may be a tablet terminal used in on-site inspection work. For convenience of explanation, the screen is partially enlarged or the area of interest is surrounded by a frame, but of course, these screens are just examples.

First, in FIG. 46 (A), the 3D model 60 in the state of the previous inspection is displayed on the screen, and attention is paid to the deformation 50 shown in the frame F1. Deformation 50 is a deformed figure such as a crack arranged on the 3D model 60.

Next, in FIG. 46 (B), the photograph 70 taken in the previous inspection (hereinafter referred to as “previous photograph”) 70 is displayed on the left side of the screen, and the photograph taken in this inspection (hereinafter referred to as “this time photograph”). ) 80 is displayed on the right side of the screen. It is assumed that the previous photograph 70 is already associated with the 3D model 60. By associating the previous photo 70 with the current photo 80 in this state, the current photo 80 can be accurately arranged in the 3D space.

Next, in FIG. 46 (C), the deformation 50 input in the previous inspection is superimposed and displayed on the photograph 80 this time (see frame F2), and the portion of the deformation 50 is enlarged and displayed (see frame F3). .. At this time, if the deformation of the inspection is progressing this time, the progressing part is additionally input to the deformation 50 (see frame F4). This additional input may be automatic or manual. For example, when the operator confirms that the deformation on the photograph 80 is progressing this time, the worker may edit the figure of the deformation 50 by tracing the progressing portion.

Finally, in FIG. 46 (D), the 3D model 60 in the state of this inspection is displayed on the screen. This screen reflects the editing result of FIG. 46 (C). That is, the leftmost deformation 50 among the deformations 50 shown in the frame F5 is cracked downward.

As described above, according to the present invention, it is possible to improve the efficiency of the second and subsequent inspection work by associating the photographs at different times with each other. In addition, it goes without saying that the progress of deformation can be accurately recorded, and the record of repair work for deformation is appropriately reflected in the cycle of bridge maintenance management in a series of inspection-planning-construction. However, it can also be used when managing history in chronological order. Therefore, using the 3D model 60 as a base diagram for information sharing and management, it is possible to improve the quality and efficiency of inspection records of bridges and promote the overall optimization of maintenance.

(Determining the exact relationship between the photo and the 3D model this time)
47 and 48 are explanatory views for determining the exact relationship between Photo 80 and the 3D model 60 this time. Accurate relationship determination is to accurately determine from which position and in which direction the 3D model 60 was photographed (single-photographing in photogrammetry).

First, as shown in FIG. 47 (A), it is assumed that the XY on the previous photograph 70 and the XYZ on the 3D model 60 are associated with each other. As a result, as shown in FIG. 47 (B), when the previous photograph 70 and the 3D model 60 are superimposed and displayed, the previous photograph 70 and the 3D model 60 can be accurately matched.

At this time, as shown in FIG. 48 (A), the faces of the previous photograph 70 and the 3D model 60 correspond to each other and are in a transparent state. For example, the XY of the point 71 on the previous photograph 70 corresponds to the XYZ on the 3D model 60. Therefore, as shown in FIG. 48 (B), when the "association execution" is instructed while the previous photo 70 and the current photo 80 are displayed, the feature is that the previous photo 70 and the current photo 80 automatically correspond to each other. It is acquired by the quantity matching method, and 4 to 8 points are selected as points to be used for determining the relationship from the corresponding points. As a result, the exact relationship between the XY of the point 81 on the photograph 80 and the XYZ on the 3D model 60 is determined this time.

As described above, in the present invention, the exact relationship between the XY of the point 81 on the photo 80 and the XYZ on the 3D model 60 is determined by using the previous photo 70, which is already associated with the 3D model 60, as an intermediary. I try to decide. As a result, even when the photo 80 and the 3D model 60 are superimposed and displayed this time, the photo 80 and the 3D model 60 can be accurately matched this time.

(Method of selecting corresponding points)
Next, the method of selecting the corresponding points in the present invention will be described in detail. In the present invention, the corresponding points are selected by appropriately combining the three selection methods described below. In the following description, the corresponding points at the stage acquired by the feature amount matching method may be referred to as "acquisition points", and the corresponding points selected from the acquired points may be referred to as "selection points".

The feature amount matching method is a method of finding a corresponding point (acquired point in this case) at the same position between two photographs based on a feature amount such as an edge or a corner point in a photograph. When using the feature matching method, there is an advantage that anyone can easily perform this method by image processing compared to the method of manually finding the same point, and further, the process of extracting the selection points from the acquired points. Has the advantage of being easy to apply.

(1) Sorting method by utilizing the 3D model 60 In order to determine the exact relationship between the photograph 80 and the 3D model 60 this time, acquisition points existing other than the 3D model part are unnecessary. Therefore, in the present invention, the previous photograph 70 and the 3D model 60 are superimposed and displayed by internal processing, and acquisition points existing other than the 3D model portion are excluded by mask processing.

FIG. 49 is an explanatory diagram of a sorting method by utilizing the 3D model 60. According to the prior art, as shown in FIG. 49 (A), since the 3D model 60 is not utilized, corresponding points are acquired even in places (steel tower, house, etc.) unrelated to the 3D model 60. On the other hand, according to the present invention, as shown in FIG. 49B, mask processing is performed using the mask 60M of the 3D model portion. That is, in order to utilize the 3D model 60, the corresponding points acquired at places unrelated to the 3D model 60 are excluded (see frames F11 and F12). As a result, only the acquisition points that are valid in relation to the 3D model 60 are easily selected, and it is possible to accurately associate the photograph 80 with the 3D model 60 this time. Further, by performing the masking process after acquiring the corresponding points, the corresponding points are once acquired even at a portion unrelated to the 3D model 60, so that there is also an effect that the corresponding points can be acquired at the contour portion of the 3D model 60.

(2) Sorting method by utilizing existing photographed photographs Sorted by photographed photographs already arranged in 3D space (the same position is accurately selected between the 3D model and photographed photographs) , Selected corresponding points) is an important point on the 3D model 60. Therefore, among the corresponding points selected by the mask processing, those close to the selected corresponding point positions may be preferentially selected.

The photo on the left side of FIG. 50 (A) is the previous photo 70 already arranged in the 3D space, and the photo on the right side of FIG. 50 (A) is the photo 80 this time to be arranged in the 3D space. Here, since there is an acquisition point near (for example, within 100 pixels) the corresponding point 2 (large circle mark) and the corresponding point 4 (large circle mark) that have been selected, the acquisition points (small circle mark) are selected. ing. As a result, it is possible to prioritize and select important points on the 3D model 60, and it can be determined that the same position can be accurately selected between the 3D model and the photograph taken. It becomes possible to associate the 80 with the 3D model 60.

(3) Sorting method considering even placement and acquisition range When a large number of corresponding points are acquired, it is important to eliminate the bias of the sorting points. Therefore, among the corresponding points selected by the mask processing, those close to the four corners of the rectangle 82 surrounding the acquired points may be preferentially selected.

FIG. 50B shows Photo 80 this time to be arranged in 3D space. Here, since the acquisition points are located near the four corners of the rectangle 82 surrounding the acquisition points (for example, within 100 pixels), the acquisition points are preferentially selected. As a result, not only the bias of the selection points can be eliminated, but also the acquisition points can be selected from the maximum range of the rectangle 82 surrounding the acquisition points. Therefore, the photograph 80 and the 3D model 60 are more accurately associated with each other this time. Becomes possible.

As described above, in the present invention, it is possible to select the corresponding points acquired by the feature amount matching method and accurately associate the photograph 80 with the 3D model 60 this time. As for the execution order of the above three sorting methods, "(1) Sorting method by utilizing the 3D model 60" may be the first, and "(2) Sorting method by utilizing the existing photographed photograph". The execution order of "(3) Sorting method considering uniform arrangement and acquisition range" is not particularly limited.

(Drawing device for structure management)
FIG. 51 is a configuration diagram of the structure management drawing creation device 10 in the present invention. The structure management drawing creation device 10 is a structure management device for managing a structure, and as shown in FIG. 51, the registration unit 11, the basic control drawing creation unit 12, and the output unit 13 , Storage unit 14, 3D model creation unit 15, 3D space arrangement unit 16, 3D model reconstruction unit 17, deformation measurement unit 18, image processing parameter registration unit 101, and time-series link management unit 102. A corresponding point acquisition unit 103 and a corresponding point selection unit 104 are provided.

The image processing parameter registration unit 101 is a processing unit that registers various parameters used in image processing. Specifically, the processing method used in the image processing of the corresponding point acquisition unit 103 and the determination threshold value of the processing result are registered.

The time-series link management unit 102 is a processing unit that manages deformed figures and photographed photographs by linking them in a time-series manner. Specifically, the time series (inspection work history) of the photograph (image) in which the deformed figure is input is linked and managed.

The correspondence point acquisition unit 103 is a processing unit that acquires correspondence points between photographs (for example, correspondence points between the previous photo 70 and the current photo 80) by a feature amount matching method. The feature amount matching method is a matching method in which candidate points (also called KeyPoints) are extracted by feature point extraction and the feature amount of the points is used for similarity determination from the candidate points.

The correspondence point selection unit 104 is a processing unit that selects the correspondence points used by the 3D space arrangement unit 16 from the correspondence points acquired by the correspondence point acquisition unit 103. As the sorting method, (1) a sorting method by utilizing the 3D model 60, (2) a sorting method by utilizing the photographed photograph already arranged, (3) a sorting method considering the uniform arrangement and the acquisition range, etc. shall be adopted. Can be done.

(Various data)
FIG. 52 is an explanatory diagram of various data necessary for effectively grasping the progressive deformation. The difference from FIG. 33 is that the inspection work information file 111, the deformation repair information file 112, and the photo association processing internal file 113 are added.

The inspection work information file 111 is a file for registering information for each inspection and work and performing time series management. Information for each inspection or construction includes inspection construction ID, inspection / construction classification, inspection construction name, inspection construction date, worker, inspection target classification (inner surface / outer surface classification), and the like.

The deformation repair information file 112 is a file in which the graphic information and attribute information of the input deformed figure are registered, and the photographed photograph ID and the inspection work ID are registered. The graphic information includes a graphic division (continuous line segment, rectangle), graphic data (coordinate values on a photograph and on a 3D model 60), and the like. The attribute information includes a deformation ID, a deformation type, a deformation size, a judgment classification, an outline, an article, and the like. The inspection work ID and the photographed photograph ID are associated with the deformation ID as a key. Since the deformation is repaired by the repair work, the repaired information is also managed in the deformation repair information file 112.

The photo association processing internal file 113 is a file for temporarily internally managing parameters and matching results used in the feature amount matching processing.

Note that, in FIG. 52, a 3D model data file 37A, a left photographic image file 41A, and a right photographic image file 42A, which are not shown in FIG. 33, are added, but these are supplementarily added. That is, the 3D model data file 37A is 3D model data stored in the 3D model data folder 37. The left photographic image file 41A is an actual photographic image stored in the left photographic image data folder “L” 41. The right photographic image file 42A is an actual photographic image stored in the right photographic image data folder “R” 42.

(Progressive deformation grasping operation)
53 and 54 are flowcharts showing an operation of grasping a progressive deformation by using the structure management drawing creation device (tablet terminal) 10 in the present invention. In all the steps of FIGS. 53 and 54, four data of the bridge 3D model information file 36, the deformation repair information file 112, the stereophotograph information file 40, and the inspection work information file 111 are used.

First, the transformation list is displayed (step S111). Specifically, the tablet terminal is operated at the site, and the current deformation data is displayed in a list from the deformation repair information file 112 using the deformation ID as a key.

Next, the deformed figure / photograph is displayed (step S112). Specifically, in order to judge the progress of the deformation, the deformation data is selected from the deformation list, the position of the deformation figure of the previous inspection on the 3D model 60, and the photographed photograph corresponding to the deformation ( The previous photo 70) is displayed.

Next, the current state of deformation is determined (step S113). At this time, if the deformation is not progressive, the process proceeds to step S128. On the other hand, if the deformation is progressive, a photograph is taken with a digital camera or the like (step S114) in order to input the update of the progressive deformation, and the photographed photograph (photo 80 this time) is taken with Wi-Fi or the like. Transfer it to a tablet terminal, import it into the project, and register it (step S115).

Next, two photographs, the previous photograph 70 and the current photograph 80, are displayed (step S116). The previous photograph 70 is a photograph taken in the previous inspection (a photograph when an existing deformation is input). Photo 80 this time is a photograph (current photograph) taken in this inspection. The previous inspection may be an inspection one time before the current inspection, or may be an inspection two or more times before.

Next, the photographs are associated with each other (step S117). Specifically, the feature amount matching process is performed, and the corresponding points between the previous photo 70 and the current photo 80 are acquired. This matching result is temporarily recorded in the photo association processing internal file 113.

Next, point extraction is performed based on the association determination threshold value (step S118). Specifically, among the corresponding points acquired in step S117, the determination based on the threshold value is performed, and the corresponding points having a high degree of similarity are extracted. This point extraction result is also temporarily recorded in the photo association processing internal file 113.

Next, the corresponding points are selected (step S119). Specifically, among the corresponding points extracted in step S118, (1) a selection method by utilizing the 3D model 60, (2) a selection method by utilizing an existing photograph taken, (3) even arrangement and acquisition range. Select the corresponding points by a selection method that takes into consideration. The details of this step S119 will be described later.

Next, the 3D coordinate value of the sorting corresponding point is calculated (step S120). Specifically, the 3D coordinate values on the 3D model 60 corresponding to the corresponding points on the image selected in step S119 are calculated.

Next, the photographs are arranged according to the sorting correspondence points (step S121). Specifically, the single-photograph is set using the 3D coordinate values (on the image and on the 3D model 60) calculated in step S120, and the photo 80 is placed in the 3D space this time. Single photo orientation is to calculate the shooting position and shooting direction of a shot photo.

Next, a confirmation display of the progressive deformation is performed (step S122). Specifically, since the relationship between the 3D model 60 and the photo 80 this time was accurately determined by the photo arrangement in step S121, the deformation of the previous inspection was superimposed and displayed on the photo 80 this time, and the progressive deformation was displayed. Check the situation.

Next, the deformed shape is updated according to the progress state (step S123), and it is determined whether or not the graphic division is necessary (step S124). The graphic division is a process of dividing a progressive deformation from a grouped deformation in order to distinguish between a progressive deformation and a non-progressive deformation.

At this time, if the figure division is necessary, the figure division is performed (step S125), and it is determined whether or not the attribute information needs to be updated (step S126). If it is necessary to update the attribute information, the attribute information (for example, the determination type of progressive deformation) is updated (step S127). The length and area of the figure are automatically updated at the same time as the figure shape is updated.

The above process is repeated for all the deformations (NO → S112 → ... in step S128 → S129). When the work for all the deformations is completed (YES in step S128 → S129), the updated input deformation contents are reflected in the deformation list (step S130).

(Corresponding point selection operation)
Hereinafter, the corresponding point selection in step S119 will be described in more detail. In the following description, the case where the selection criteria are satisfied is referred to as "adoption", and the case where the selection criteria are not satisfied is referred to as "non-adoption".

FIG. 55 is a flowchart showing an operation of selecting corresponding points. In all the steps of FIG. 55, four data of the bridge 3D model information file 36, the deformation repair information file 112, the stereo photographic information file 40, and the inspection work information file 111 are used.

First, the mask 60M of the 3D model 60 is created (step S131). Specifically, the 3D model 60 is rotated and moved so as to match the previous photograph 70, and the mask 60M of the 3D model portion is created.

Next, the non-adoption corresponding points due to the mask 60M are removed (step S132). Specifically, if there is a corresponding point in a place other than the mask 60M, the corresponding point is excluded as non-adoption.

Next, the neighborhood points at the four corners are adopted (step S133). Specifically, when there are corresponding points in the vicinity of the four corners of the rectangle 82 surrounding the corresponding points (points at the mask 60M) adopted in step S132, these corresponding points are adopted.

Next, a point near the selected corresponding point is adopted (step S134). Specifically, when there is a corresponding point in the vicinity of the corresponding point selected in the previous photograph 70, this corresponding point is adopted.

As a result, when the number of adopted points reaches 4 points or more and 8 points or less, the corresponding point selection process ends (step S135 → end). On the other hand, if the number of adopted points does not reach 4 points or more and 8 points or less, the same processing is repeated for the next corresponding points (steps S135, S138, S137, S136 → S133 → ...). If the number of adopted points is less than 4 even after repeating the same process for all the adopted points, the corresponding points are manually added (steps S135 → S139).

The order of steps S133 and S134 may be reversed.

As described above, the structure management drawing creation device 10 in the present invention is a structure management device for managing a structure, and is a registration unit for registering structural parameters of the structure and photographed photographs of the structure. 11 corresponds to the 3D model creation unit 15 that creates the 3D model 60 of the structure using the structural parameters, the 3D model 60 created by the 3D model creation unit 15, and the previous photo 70 registered by the registration unit 11. When the photo 80 is registered by the registration unit 11 in the attached state, the corresponding point acquisition unit 103 and the corresponding point acquisition unit 103 that acquire the corresponding points between the previous photo 70 and the current photo 80 by the feature amount matching method. A 3D space arrangement in which the photo 80 is associated with the 3D model 60 and arranged in the 3D space based on the correspondence point selection unit 104 that selects the correspondence points acquired by 103 and the correspondence points selected by the correspondence point selection unit 104. A unit 16 is provided. This makes it possible to effectively grasp the progressive deformation by associating the photographs with each other.

Further, the corresponding point selection unit 104 may superimpose the previous photograph 70 and the 3D model 60 and exclude the corresponding points existing other than the 3D model portion by mask processing. As a result, only the acquisition points that are valid in relation to the 3D model 60 are easily selected, and it is possible to accurately associate the photograph 80 with the 3D model 60 this time. Further, since the corresponding points are once acquired even at a place unrelated to the 3D model 60, there is also an effect that the corresponding points can be acquired at the contour portion of the 3D model 60.

Further, the corresponding point selection unit 104 may preferentially select the corresponding points selected by the mask processing that are close to the selected corresponding point positions in the photographes arranged in the 3D space. As a result, important points can be preferentially selected on the 3D model 60, so that the photograph 80 and the 3D model 60 can be more accurately associated with each other this time.

Further, the corresponding point selection unit 104 may preferentially select the corresponding points selected by the mask process, which are close to the four corners of the rectangle 82 surrounding the corresponding points. As a result, not only the bias of the selection points can be eliminated, but also the acquisition points can be selected from the maximum range of the rectangle 82 surrounding the acquisition points. Therefore, the photograph 80 and the 3D model 60 are more accurately associated with each other this time. Becomes possible.

Further, a 3D space arrangement unit 16 is provided with a deformation measuring unit 18 for inputting a deformed figure on a photographed photograph and a time-series link management unit 102 for managing the photographed photograph with the deformed figure input in association with a time series. May superimpose the deformed figure on the photograph 80 this time when the deformed figure was input on the photograph 70 last time. This makes it possible to easily grasp the progressive deformation by comparing the superimposed deformed figure with Photo 80 this time.

Further, the deformation measuring unit 18 may edit the deformed figure superimposed on the photo 80 this time according to the progress state of the deformation on the photo 80 this time. This keeps the progressive transformation up-to-date so that accurate aging can be managed.

In the above description, the case where the present invention is applied to the structure management drawing creation device 10 has been described as an example, but the function of creating a drawing is not always necessary. That is, the present invention can be applied to all devices for managing structures.

Further, in the above description, the case of managing a bridge has been illustrated and described, but the present invention is not limited thereto. That is, the present invention can be similarly applied when managing structures other than bridges.

Further, the present invention can be realized not only as a structure management drawing creation device 10, but also as a structure management method in which a characteristic processing unit included in the structure management drawing creation device 10 is a step. It can also be realized as a structure management program that causes a computer to function as a drawing creation device 10 for structure management. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

<< Other embodiments >>
As mentioned above, some embodiments have been described, but the statements and drawings that form part of the disclosure are exemplary and should not be understood as limiting each embodiment. This disclosure will reveal to those skilled in the art various alternative embodiments, examples and operational techniques.

As such, each embodiment includes various aspects not described herein.

10 ... Drawing creation device for structure management (structure management device)
11 ... Registration unit 12 ... Basic control chart creation unit 13 ... Output unit 14 ... Storage unit 15 ... 3D model creation unit 16 ... 3D space arrangement unit 17 ... 3D model reconstruction unit 18 ... Deformation measurement unit 20 ... Stereo imaging device 101 ... Image processing parameter registration unit 102 ... Time-series link management unit 103 ... Corresponding point acquisition unit 104 ... Corresponding point selection unit

Claims (10)

A structure management device for inspection and maintenance of structures.
A registration unit for registering structural parameters of the structure and photographed photographs of the structure,
A 3D model creation unit that creates a 3D model of the structure using the structural parameters,
When the current photograph is registered by the registration unit while the 3D model created by the 3D model creation unit is associated with the previously photographed photograph registered by the registration unit, the previous photography is performed. Correspondence point acquisition unit that acquires the correspondence point between the photograph and the photograph taken this time by the feature amount matching method,
A correspondence point selection unit that selects the correspondence points acquired by the correspondence point acquisition unit, and a correspondence point selection unit.
A 3D space arranging unit for arranging the photograph taken this time in 3D space in association with the 3D model based on the corresponding points selected by the corresponding point selection unit is provided .
The structure management device is characterized in that the corresponding point selection unit superimposes the previously photographed photograph and the 3D model, and excludes the corresponding points existing other than the 3D model portion by mask processing. The claim is characterized in that, among the corresponding points selected by the mask processing, the corresponding point sorting unit preferentially selects the corresponding points arranged in the 3D space that are close to the selected corresponding point positions. 1 The structure management device according to 1. The structure management device according to claim 1, wherein the corresponding point sorting unit preferentially sorts the corresponding points selected by the mask processing that are close to the four corners of the rectangle surrounding the corresponding points. Deformation measurement unit that inputs deformed figures on the photograph taken, and
It is provided with a photograph taken by inputting the deformed figure and a time-series link management unit that manages the deformed figure in association with time series.
Any one of claims 1 to 3 , wherein the 3D space arrangement unit superimposes the deformed figure on the photograph taken this time when the deformed figure is input on the photograph taken last time. The structure management device described in the section. The structure management device according to claim 4 , wherein the deformation measuring unit edits a deformed figure superimposed on the photograph taken this time according to a progress state of the deformation on the photograph taken this time. A basic control chart creation unit that creates a 2D basic control chart using the 3D model of the structure generated by the 3D model creation unit.
Further prepare
The structure management device according to claim 1, wherein the 3D model, the photographed photograph, and the basic control chart are associated with each other in a positional relationship of portions included in the 3D model. A 2D basic control chart is created using the 3D model of the structure generated by the 3D model creation unit.
When the deformed figure is input on the photographed photograph by the deformed measuring unit , a basic control chart creating unit for outputting the deformed figure on the basic control chart is further provided.
The structure management device according to claim 4 , wherein the 3D model, the photographed photograph, and the basic control chart are associated with the positional relationship of the deformed figure included in each. The deformation measuring unit inputs a deformed figure on the photographed photograph or the 3D model, and then inputs the deformed figure.
A 2D basic control chart is created using the 3D model of the structure generated by the 3D model creation unit, and the deformation figure is input by the deformation measurement unit on the previous photograph or the 3D model. If so, a basic control chart creation unit for outputting the deformed figure on the basic control chart is further provided.
The structure management device according to claim 4 , wherein the 3D model, the photographed photograph, and the basic control chart are associated with the positional relationship of the deformed figure included in each. It is a structure management method for inspection and maintenance of structures.
A registration step for registering the structural parameters of the structure and a photograph of the structure, and
A 3D model creation step for creating a 3D model of the structure using the structural parameters, and
When the 3D model created in the 3D model creation step and the previously photographed photograph registered in the registration step are associated with each other and the current photographed photograph is registered in the registration step, the previous photographed photograph is taken. Correspondence point acquisition step to acquire the correspondence point between the photograph and the photograph taken this time by the feature amount matching method,
A correspondence point selection step for selecting the correspondence points acquired in the correspondence point acquisition step, and a correspondence point selection step.
Including a 3D space arrangement step in which the photograph taken this time is arranged in the 3D space in association with the 3D model based on the corresponding points selected in the corresponding point selection step .
The structure management method is characterized in that the corresponding point selection step superimposes the previously photographed photograph and the 3D model, and excludes the corresponding points existing other than the 3D model portion by mask processing. A structure management program comprising operating a computer as the structure management device according to any one of claims 1 to 8.

Images