JP7146013B2 - Server device, image processing method, and program - Google Patents

Description

The present invention relates to a server device, an image processing method, and a program for outputting an image for editing a damage diagram showing the damage state of a structure, and in particular to a damage state seen from the side opposite to the imaging side of the structure. It relates to a server device, an image processing method, and a program that make it possible to output an image for accurately and easily editing a damage diagram showing the .

Structures such as bridges and tunnels exist as social infrastructure. These structures are prone to damage, and the damage tends to progress, so regular inspections are required.

An investigator who inspects a structure needs to create an inspection report in a prescribed format as a form showing the results of the inspection, and one of the inspection reports is a damage diagram that illustrates the state of damage to the structure. By looking at the damage diagram created in the prescribed format, even an expert who is different from the inspector who actually inspected the structure can grasp the progress of damage to the structure and formulate a maintenance management plan for that structure. becomes possible.

In Patent Document 1, when creating a diagram showing cracks in the outer wall surface of a building (equivalent to the above-mentioned "damage diagram"), a CAD (computer aided design) drawing showing the building and the outer wall surface of the building are used. When the crack image in the displayed captured image is traced by the operation device, the trace diagram corresponding to the trace operation is mapped on the CAD drawing. that is stated.

In Patent Document 2, when creating a developed view showing the uneven shape of the inner wall surface of the tunnel, the uneven shape figure is generated based on the measurement data obtained by measuring the inner wall surface of the tunnel with a laser scanner. Drawing a pattern of an image on a relief figure is described. This developed view can be created as a view showing the uneven shape when the inner wall surface of the tunnel is seen through the tunnel from above.

JP-A-2005-31044 JP 2012-220471 A

As a form of damage diagram, it may be desired to produce a damage diagram that illustrates the damage state as seen from the opposite side of the structure to the viewing side. For example, an inspector who visually inspects the deck slab of a bridge, even if he/she actually visually inspects the damage state of the deck slab from the underside of the bridge, sketches the state of damage as seen from the top of the bridge in the field notebook and It is necessary to prepare a damage diagram of the form.

In addition, when the floor slab is imaged with an imaging device such as a digital camera instead of visual inspection, according to the image of the floor slab obtained by imaging from the underside of the bridge, the imaging side (the underside of the bridge) and the A damage diagram of a certain type should be prepared to illustrate the damage condition as seen from the opposite side (which is the top side of the bridge).

However, the shape and position of the damage look different when viewed from the bottom side of the bridge and when viewed from the opposite side, the top side of the bridge. It takes For example, when creating a damage diagram showing the state of cracks, it is required not only to enter numerical values such as the width of cracks and the distance between cracks, but also to accurately enter the shape and position of cracks in the damage diagram. be done. Here, if the actual cracking state is not accurately reflected in the damage diagram, it will be difficult to accurately grasp the progress of damage to the structure, making it difficult to formulate an appropriate maintenance plan.

According to the technique described in Patent Document 1, an investigator traces a crack image in a captured image displayed on a computer device to map a trace diagram corresponding to the crack image on a CAD drawing. However, it is not possible to map the damage state seen from the side opposite to the imaging side of the structure on the CAD drawing. In the embodiment of Patent Document 1, since the object to be inspected is the exterior wall surface of the building, it would have been sufficient to create a diagram of the state of cracks as seen from the imaging side (outside of the building).

According to the technique described in Patent Document 2, it is possible to create a developed view showing the uneven shape when the inner wall surface of the tunnel is seen through the tunnel from above, but the uneven shape of the inner wall surface of the tunnel is measured with a laser scanner. Therefore, if a laser scanner is not used, it is not possible to create a developed view showing the uneven shape of the inner wall surface when viewed from above the tunnel.

The present invention provides a server device, an image processing method, and a program that make it possible to output an image for accurately and easily editing a damage diagram illustrating a damage state seen from the side opposite to the imaging side of a structure. intended to provide

In order to achieve the above-described object, the damage diagram editing device according to the first aspect of the present invention includes an image input unit for inputting an image of a structure to be inspected, and a mirror movie by mirror-reversing the input image. A mirror reversing unit that acquires an image, a display unit, a display control unit that displays a mirrored image on the display unit, an operation unit that receives a user's operation, and an operation while the mirrored image is displayed on the display unit. an editing unit that edits a damage diagram showing the state of damage to the structure, which corresponds to the mirrored image, according to the operation received by the unit.

According to this aspect, the reflected image is acquired by mirror-reversing the input image, and the structure is damaged according to the operation accepted by the operation unit while the reflected image is displayed on the display unit. Since the damage diagram corresponding to the mirrored image is edited, the user can obtain the damage diagram corresponding to the mirrored image by performing editing operations while viewing the mirrored image. can. In other words, it becomes possible to accurately and easily compile a damage diagram illustrating the damage state seen from the side opposite to the imaging side of the structure.

A damage diagram editing device according to a second aspect of the present invention includes an image synthesizing unit that synthesizes images, the image input unit inputs a plurality of divided images of a structure as an image, and the mirror reversing unit inputs a plurality of divided images of the structure. and the image synthesizing unit synthesizes the plurality of mirror-inverted split images. According to this aspect, even in the case of split imaging, it is possible to accurately and easily edit the damage diagram illustrating the damage state of the structure as viewed from the side opposite to the imaging side.

A damage diagram editing device according to a third aspect of the present invention includes an image synthesizing unit that performs image synthesis, the image input unit inputs a plurality of divided images of a structure as an image, and the mirror reversing unit inputs the image A synthesized image obtained by synthesizing a plurality of divided images by a synthesizing unit is mirror-inverted. According to this aspect, even in the case of split imaging, it is possible to accurately and easily edit the damage diagram illustrating the damage state of the structure as viewed from the side opposite to the imaging side.

In the damage diagram editing device according to the fourth aspect of the present invention, the image synthesizing unit performs at least one of image processing of scaling, tilt correction, and rotation on each of the plurality of divided images, so that the plurality of Correction is made for the mismatch in the distance or angle between the surface to be imaged of the structure across the divided images and the image capturing device that captured the image. According to this aspect, it is possible to generate a highly accurate damage diagram even when there is an influence of a mismatch in the distance or angle between the surface to be imaged of the structure and the imaging device over a plurality of divided images.

The damage diagram editing device according to the fifth aspect of the present invention comprises a diagram input unit for inputting a diagram showing a structure, and the display control unit superimposes a mirror image on the diagram and displays it on the display unit. According to this aspect, it is possible to edit the damage diagram while looking at the superimposed and displayed drawing and the mirrored image, so that it is possible to easily create an accurate damage diagram.

In the damage diagram editing device according to the sixth aspect of the present invention, the operation unit accepts a trace operation for the damage image in the mirrored image while the mirrored image is displayed on the display unit, and the display controller receives the trace operation. The damage graphic corresponding to the operation is superimposed on the damage image in the mirrored image and displayed on the display section, and the editing section adds the damage graphic corresponding to the tracing operation to the damage graphic. According to this aspect, it is possible to easily create an accurate diagram of damage by tracing the damage image in the mirrored image.

A damage diagram editing apparatus according to a seventh aspect of the present invention includes a damage detection unit that detects a damage image from a mirrored image or an image before mirror reversal, and the display control unit responds to the detected damage image. The damage figure is superimposed on the damage image in the mirrored image and displayed on the display unit, and the editing unit adds the damage figure to the damage figure. According to this aspect, the damage diagram corresponding to the damage image is automatically generated and automatically added to the damage diagram. can be obtained.

In the damage diagram editing device according to the eighth aspect of the present invention, the operation unit receives input of inspection result information for the damage while the mirror image is displayed on the display unit, and the display control unit receives the input inspection result information. The result information and the mirror image are displayed on the display unit, and the editing unit adds the input inspection result information to the damage diagram.

In the damage diagram editing apparatus according to the ninth aspect of the present invention, the editing unit measures the characteristic amount of damage based on the mirrored image or the image before reflection reversal, and transmits information indicating the measured characteristic amount to the damage diagram. Add to

A damage diagram editing device according to a tenth aspect of the present invention comprises a damage diagram input unit for inputting a damage diagram showing a state of damage seen from the side opposite to the imaging side of the structure, wherein the display control unit inputs The damage diagram and the reflected image are displayed on the display unit.

The damage diagram editing device according to the eleventh aspect of the present invention includes: a damage diagram reversing unit for converting the damage diagram to a reversed damage diagram showing the state of damage seen from the imaging side of the object, and the display control unit displays the reversed damage diagram and the image on the display unit.

In the damage diagram editing device according to the twelfth aspect of the present invention, the damage diagram showing the state of damage seen from the opposite side of the imaging side of the structure is reversed to show the state of damage seen from the imaging side of the structure. A damage diagram reversing unit for converting to a damage diagram is provided, the operation unit receives a reversing operation regarding the damage diagram, and the display control unit displays a state in which the damage diagram is displayed on the display unit and a reversed damage diagram in response to the reversing operation. Switches between the states displayed on the display unit.

In the damage diagram editing device according to the thirteenth aspect of the present invention, the reflection reversing section analyzes the input image to determine the direction of the reflection reversal axis.

In the damage diagram editing device according to the fourteenth aspect of the present invention, the reflection reversing unit has the coordinate axes of the first coordinate system based on the structure and the second coordinate system based on the imaging device that captured the image. The direction of the axis of mirror inversion is determined based on the angle formed by the coordinate axes of .

In the damage diagram editing apparatus according to the fifteenth aspect of the present invention, the reflection reversing unit determines the direction of the reflection reversal axis based on the azimuth angle of the imaging device that captured the image, with the structure as a reference.

In the damage diagram editing method according to the sixteenth aspect of the present invention, there is provided a step of inputting an image of a structure to be inspected, a step of mirror-reversing the input image to obtain a mirrored image, a step of displaying a mirrored image; and a damage diagram showing a state of damage to the structure, corresponding to the mirrored image, in response to an operation received by the operation device while the mirrored image is displayed on the display device. and editing the damaged diagram.

The images can be output for accurate and easy compilation of damage diagrams that illustrate damage conditions as seen from the opposite side of the structure to the imaged side.

FIG. 1 is a perspective view showing the structure of a bridge, which is an example of a structure. FIG. 2 is a block diagram showing a configuration example of the damage diagram editor in the first embodiment. FIG. 3 is a first explanatory diagram used for explaining mirror inversion. FIG. 4 is a second explanatory diagram used for explaining mirror inversion. FIG. 5 is an explanatory diagram showing an example of damage graphics. FIG. 6 is an explanatory diagram showing an example of a damage diagram. FIG. 7 is an explanatory diagram showing an example of an inverted damage diagram. FIG. 8 is a flow chart showing the flow of a first example of damage diagram editing processing in the first embodiment. FIG. 9 is a diagram showing an example of a CAD drawing. FIG. 10 is a diagram showing an example of the displayed image. FIG. 11 is a diagram showing an example of the displayed mirrored image. FIG. 12 is a diagram showing an example of displaying a CAD drawing with a mirror image superimposed thereon. FIG. 13 is a diagram showing a display example of an edit screen relating to floor slab coffers. FIG. 14 is a diagram schematically showing an example of a crack progression model in a floor slab. FIG. 15 is a flow chart showing the flow of the second example of damage diagram editing processing in the first embodiment. FIG. 16 is a flow chart showing the flow of a third example of damage diagram editing processing according to the first embodiment. FIG. 17 is a block diagram showing a configuration example of a damage diagram editor according to the second embodiment. FIG. 18 is an explanatory diagram for explaining the variation of mirror inversion, and is a diagram showing an example of an image for each coffer of the floor slab. FIG. 19 is an explanatory diagram showing an example of an imaging device mounted on a suspended robot device. FIG. 20 is an explanatory diagram for explaining variations of mirror inversion.

Hereinafter, embodiments for implementing a server device, an image processing method, and a program according to the present invention will be described with reference to the accompanying drawings.

[Structure example]
FIG. 1 is a perspective view showing the structure of a bridge, which is an example of a structure to which the present invention is applied, and is a perspective view when the bridge is viewed from below. A bridge 1 in this figure has a main girder 2 , a cross girder 3 , a tilting structure 4 and a horizontal structure 5 . Above the main girder 2, a floor slab 6, which is a member made of concrete, is provided. The main girder 2 is a member that spans between abutments or piers and supports the load on the floor slab 6 . The horizontal girder 3 is a member that connects the main girder 2 so that the load is supported by the main girder 2 . The counter leaning structure 4 and the horizontal structure 5 are members that connect the main girder 2 in order to particularly resist lateral loads.

Incidentally, the bridge 1 shown in FIG. 1 is an example of a structure introduced for use in explaining the present invention, and the "structure" in the present invention is not limited to such a bridge. A structure according to the invention may be, for example, a tunnel.

[First embodiment]
FIG. 2 is a block diagram showing a configuration example of the damage diagram editor in the first embodiment.

The imaging device 10 is configured by an imaging device that images a structure to be inspected. Examples of imaging devices include digital cameras and smart phones. Imaging devices other than these may be used.

The damage diagram editing device 20 includes a display unit 22 capable of displaying images, an operation unit 24 for receiving user operations, and an image input unit 26 for inputting an image obtained by imaging a structure to be inspected by the imaging device 10. , a drawing input unit 28 for inputting a CAD (computer aided design) drawing showing the structure to be inspected, a CPU (central processing unit) 30 for executing the program, and storing the program and information necessary for executing the program. and a storage unit 50 for storing data.

The display unit 22 is configured by a display device such as a liquid crystal display device. The operation unit 24 can be composed of a keyboard and a mouse. The operation unit 24 may be configured by an operation device other than these, such as a touch panel. The image input unit 26 and the drawing input unit 28 can be configured by input devices for inputting digital signals. As such an input device, for example, a communication device that performs wireless communication or wired communication may be used. An interface device with a storage medium such as a memory card may be used as an input device. The CPU 30 may be composed of a plurality of CPUs. The storage unit 50 is configured by a memory device.

The CPU 30 includes a display control unit 32 that controls the display unit 22, an image correction unit 34 that performs image correction, a reflection reversal unit 36 that obtains a reflected image by mirror reversing an image, and an image that performs image synthesis. A synthesizing unit 38, a damage detection unit 40 that detects a damage image from the mirrored image (or the image before the reflection is reversed), and a damage diagram showing the state of damage to the structure and corresponding to the mirrored image. and an editing unit 42 for editing.

The display control unit 32 causes the display unit 22 to superimpose the mirrored image on the CAD drawing. In addition, the display control unit 32 causes the display unit 22 to display the mirrored image and the diagram of the damage.

The image correcting unit 34 performs necessary image processing of scaling, tilt correction, and rotation on the image. In other words, correction is performed regarding the mismatch of the distance or angle between the surface of the structure to be imaged and the imaging device 10 over a plurality of divided images. The “scaling” in this example is scale correction related to the discrepancy in the distance between the imaging device 10 and the floor slab 6 (which is the surface to be imaged) over a plurality of divided images. The image correction unit 34 of this example performs image processing to match the scales of a plurality of images by enlarging or reducing the images. The “tilt correction” in this example is a correction related to the mismatch of the tilt angle of the floor slab 6 (which is the surface to be imaged) with respect to the imaging direction of the imaging device 10 over a plurality of divided images. The image correction unit 34 of this example performs image processing to reduce or match the distortion of the image according to the tilt angle of the floor slab 6 with respect to the imaging direction. "Rotation" in this example is a correction for the mismatch of the azimuth angle of the imaging device 10 with respect to the floor slab 6 (which is the surface to be imaged) over a plurality of divided images. The image correction unit 34 of this example performs image processing for matching the angles of the floor slabs 6 in the images between a plurality of images. Parameters for each correction can be input from the operation unit 24 . For example, the operation unit 24 receives inputs such as distance information, tilt angle information, and azimuth angle information. Correction parameters may be acquired from the imaging device 10 . For example, the image correction unit 34 acquires coordinate values in the local coordinate system based on the imaging device 10, converts them into coordinate values in the global coordinate system based on the bridge 1, and based on the coordinate values in the global coordinate system, Determine correction parameters. When correction parameters are added to the image, image correction may be performed using the correction parameters added to the image. Correction parameters may be derived based on the image. For example, the edge of the coffer can be detected from the image, the shape, size and angle of the edge region in the image can be calculated based on the image, and the correction parameters can be derived based on the calculation result. When combining the above-described "scaling", "tilt correction" and "rotation", the image processing can be performed once. Known projective transformation can be used as the image processing.

The mirror inversion unit 36 performs mirror inversion, which is image processing for converting the image input by the image input unit 26 into a mirror image. A "mirror image" is also called a "mirror image." "Reflection reversal" is also called "mirroring". The direction of the axis that is the center of mirror reversal (hereinafter referred to as the "mirror reversal axis") is not limited to the vertical direction of the image. That is, mirror inversion is not limited to so-called left-right inversion. Mirror inversion includes inversion other than horizontal inversion (for example, vertical inversion). The mirror reversal in the present invention can be said to be image processing for reversing an image around the mirror reversal axis. The direction of the axis of reflection reversal can be determined, for example, by image analysis of the input image. Further, the mirror inversion of the present invention includes a mode in which the direction of the axis of mirror inversion is fixed and a mode in which the direction of the axis of mirror inversion is variable. The latter directional aspect will be described in detail later in this specification.

Such mirror inversion will be described with reference to FIGS. 3 and 4. FIG. As shown in FIG. 3, if the letter "F" drawn on the upper surface of the floor slab 6 is seen through from below the floor slab 6, it actually looks like an inverted "F". This is because "F" has an asymmetric shape. Similarly, general damage (for example, cracks) has an asymmetrical shape. It can be seen that the shape of the damage when the floor slab 6 is seen through from the figure is in a relationship of mirror inversion. Therefore, as shown in FIG. 4, an image IMG1 obtained by imaging from the lower side of the bridge 1 is converted into a reflected image IMG2 corresponding to an image viewed from the upper side of the bridge 1 by reflection inversion. Then, the shape of the damage changes from the shape when viewed from the bottom of the bridge 1 (the shape of "F" in FIG. 4 is inverted upside down) to the shape when viewed from the top of the bridge 1 (the shape of "F" in FIG. ). It can be seen that the position of the damage in the image can also be converted from the position viewed from the imaging side to the position viewed from the side opposite to the imaging side by mirror inversion. Note that the case of split imaging for a plurality of times will be described in detail later.

In the example shown in FIG. 4, since the direction of the coordinate axis Gx of the global coordinate system with the bridge 1 as a reference matches the horizontal direction (x direction) of the image IMG1, the image IMG1 is vertically inverted as a mirror inversion. However, for example, when the direction of the coordinate axis Gx of the global coordinate system and the vertical direction (y direction) of the image IMG1 match, the image IMG1 is left-right reversed as mirror inversion.

When a plurality of images (hereinafter referred to as "divided images") obtained by dividing the structure multiple times are input to the image input unit 26, the image synthesizing unit 38 combines these divided images into a single image. image. Such image synthesizing modes include a first image synthesizing mode of synthesizing a plurality of mirror-reversed split images (reflected images), and a plurality of split images before mirror-reversal (non-mirror movies). There is a second image synthesis mode for synthesizing the image). Details of each synthesis mode will be described later.

In the first image synthesizing mode, the mirror reversing unit 36 mirrors each of the plurality of divided images. Further, in the first image synthesizing mode, the display control unit 32 superimposes each mirror-reversed split image on the imaged area in the CAD drawing showing the structure seen from the side opposite to the imaging side. be able to. In the second image synthesizing mode, the mirror reversal unit 36 mirror-reverses the synthesized image obtained by synthesizing the plurality of divided images by the image synthesizing unit 38 . Further, in the second image composition mode, the display control unit 32 can superimpose each divided image before reflection reversal on the captured area in the CAD drawing showing the structure viewed from the imaging side.

The damage detection unit 40 detects a damage image from the mirror image (or the image before mirror reversal). For example, from the mirrored image of the floor slab 6 of the bridge 1, damage images (damage images) such as cracks, peeling, exposure of reinforcing bars, free lime, and water leakage are detected, and the detected damage images are extracted from the mirrored image.

The editing unit 42 has a damage graphic generation function of generating a damage graphic corresponding to the damage image detected by the damage detection unit 40 . The display control unit 32 causes the display unit 22 to display the damage figure corresponding to the detected damage image so as to be superimposed on the damage image in the mirrored image. For example, damage figures such as cracks, spalling, rebar exposure, free lime, water leakage, etc. shown in FIG. 5 are generated. The shape of each damage figure corresponds to the shape of the damage image in the mirror image. The editor 42 adds the generated damage graphic to the damage diagram.

Also, the editing unit 42 can generate a damage figure corresponding to the tracing operation on the operation unit 24 . The operation unit 24 receives a trace operation for the damage image in the mirrored image while the mirrored image is displayed on the display unit 22 . The display control unit 32 causes the display unit 22 to display the damage graphic corresponding to the trace operation by superimposing it on the damage image in the mirrored image. The editing unit 42 adds damage graphics corresponding to the tracing operation to the damage diagram. Moreover, it is preferable to accept not only addition but also various editing operations such as "delete" and "correction".

Also, the editing unit 42 adds various types of information input from the operation unit 24 to the damage diagram. The operation unit 24 accepts input of inspection result information for damage while the mirrored image is displayed on the display unit 22 . The display control unit 32 causes the display unit 22 to display the input inspection result information and the mirrored image. The editing unit 42 adds the input inspection result information to the damage diagram. For example, when the type of damage and the evaluation classification of the degree of damage (also referred to as "rank information") are input from the operation unit 24, they are added to the damage diagram. The "input" of the inspection result information may be a "selective input" that is an input in which a selection is made from predetermined candidates.

The editing unit 42 also has a feature quantity measurement function for measuring the feature quantity of damage based on the mirrored image (or the image before mirror reversal). The editing unit 42 adds information indicating the measured feature amount to the damage diagram. For example, the width, length, and spacing of crack images in the mirrored image of the floor slab 6 of the bridge 1 are detected as the dimensions of the local coordinate system, and the dimensions of the local coordinate system are converted to the dimensions of the global coordinate system. The display control unit 32 causes the display unit 22 to display the measured feature amount of the damage and the mirror image.

FIG. 6 is a diagram showing an example of a damage diagram. This damage diagram shows damage diagrams showing damage such as cracks, water leakage, and free lime for each coffer, and the name of the member (in this example, "floor slab ”), element numbers (Ds0101 to Ds0104, Ds0201 to Ds0204), types of damage (“cracks”, “leakage”, “free lime”, etc.), and evaluation categories for the degree of damage (also referred to as “rank information”) and characteristic quantities of the damage (for example, width and interval of cracks). In this figure, the degree of damage is classified into five grades from "a" to "e".

Further, the CPU 30 includes a damage diagram reversing unit 44 for reversing the damage diagram. FIG. 7 shows an example of an inverted damage diagram in which the damage diagram shown in FIG. 6 is reversed. As shown in FIG. 7, the damage diagram reversing unit 44 of this example converts the damage diagram of FIG. It is converted to the inverted damage diagram of FIG. 7 showing the state of .

The damage diagram reversing unit 44 in this example mirrors and reverses non-text components such as CAD drawings and damage figures by the mirror reversing unit 36, while the member name, element number, damage type, rank information, damage feature amount This is preferable in that the position in the figure is changed without mirror-reversing the text component such as . However, the present invention encompasses the case where text components are mirrored in the same manner as non-text components. By mirroring the CAD drawing of the non-text component and the damaged figure, it becomes easy to visually recognize whether or not at least the shape and position of the damaged figure are correct. For example, it becomes possible to compare with a sketch of a damaged state when the bridge 1 is viewed from below.

The operation unit 24 receives a reversal operation regarding the damage drawing. The display control unit 32 switches between a state in which the damage diagram shown in FIG. 6 is displayed on the display unit 22 and a state in which the reversed damage diagram shown in FIG. .

Next, an example of damage drawing editing processing to which the damage drawing editing method of the present invention is applied will be described.

FIG. 8 is a flow chart showing the flow of a first example of damage diagram editing processing in the first embodiment. This process is executed by the CPU 30 according to a program stored in the storage unit 50 .

First, the drawing input unit 28 inputs a CAD drawing showing the bridge 1 (step S2). The input CAD drawing is displayed on the display unit 22 by the display control unit 32 (step S4). FIG. 9 shows an example of a CAD drawing. "DsXXXX" (Ds0101 to Ds0104 and Ds0201 to DS0204) in the figure is an element number identifying a coffer which is an element of the floor slab 6 of the bridge 1. FIG. Here, "Ds" is a symbol representing "floor slab", and "XXXX" is the grid arrangement position of the coffers, which is the range surrounded by the main girders 2 and cross girders 3 of the floor slab 6. is a number indicating In other words, the floor slab 6 of this example is composed of a plurality of coffers arranged in a grid pattern, and an element number is assigned to each coffer. The CAD drawing 80 of this example contains the rectangular frame information of the coffers. Element numbers DsXXXX in the drawing may be omitted from the display on the display unit 22 . The coordinate axes Gx, Gy, and Gz of the global coordinate system with the bridge 1 as a reference are described in the drawing for later explanation, and may be omitted from the display on the display unit 22 .

Next, the image input unit 26 inputs the image of the coffer of the floor slab 6 (step S6). That is, an image obtained by imaging the floor slab 6 to be inspected for each crate in a grid-like arrangement by the imaging device 10 is input. In this example, an image is input for each case, but a plurality of images may be input collectively.

Next, the image correction unit 34 performs the necessary image correction among the enlargement/reduction (scale correction), tilt correction, and rotation of the image of the coffer (step S8). In other words, corrections are made for mismatches in distance or angle between the floor slab 6 and the imaging device 10 over a plurality of coffers. Note that this step can be omitted if there is no discrepancy in distance and angle over a plurality of spaces.

Next, the image of the coffer is mirror-reversed by the mirror reversing unit 36 (step S10). FIG. 10 shows the image IMG11 of the crate with the element number Ds0101. This image IMG11 is an image showing the state of damage seen from below the bridge 1, and is an image showing the state of damage between one coffer of the floor slabs 6. FIG. FIG. 11 shows a mirrored image IMG12 obtained by mirror-reversing the image IMG11 of FIG. In this example, the coordinate axis Gx of the global coordinate system with the bridge 1 as a reference matches the horizontal direction (x direction) of the image IMG11. IMG12 is obtained. That is, the direction of the mirror inversion axis MAx in this example is the horizontal direction (x direction) of the image IMG11.

Next, the display control unit 32 causes the display unit 22 to display the mirrored image IMG12 superimposed on the CAD drawing 80 (step S12). FIG. 12 shows an example in which the mirrored image IMG12 shown in FIG. 11 is superimposed on the CAD drawing 80 shown in FIG. It should be noted that the mirrored image IMG12 is trimmed of unnecessary portions. In this example, the crate edges in the mirrored image IMG 12 are aligned with the crate edges in the CAD drawing 80 .

Next, the damage detector 40 analyzes the mirror image IMG12 to detect a damage image from the mirror image IMG12 (step S14). For example, an image of a crack is detected from the mirrored image IMG12.

Next, the editing unit 42 measures the feature amount of the damage based on the mirrored image IMG12 (step S16). For example, the width and spacing of cracks are measured as feature quantities.

Next, the editor 42 edits each element of the damage diagram (step S18). FIG. 13 shows a display example of an edit screen relating to a coffer to which the element number Ds0101 of the floor slab 6 is assigned. In this example, the display unit 22 is caused to display the CAD drawing 80 and the mirrored image IMG12 as well as the damage list 82 . In this example, since a crack was detected in the above-described step S14 in the crate with the element number Ds0101, the "type" (type of damage) "crack" is automatically added to the damage list 82 by the editing unit 42. to be entered. In addition, since the width and interval of the "crack" are measured in the above step S16 as the characteristic amount of the "crack" in the crate of the element number Ds0101, the editing unit 42 adds Filled in automatically. Items that have not been automatically entered by the editing unit 42 can be entered by the operating unit 24 . For example, when water leakage and free lime can be visually recognized in the mirrored image IMG12, the type "water leakage + free lime" can be input through the operation unit 24 .

Further, when an automatically undetected crack can be visually recognized in the mirrored image IMG12, the type and size of the crack can be input through the operation unit 24. FIG. Although not shown in FIG. 13, a scale image for measuring the crack dimension can be displayed on the display unit 22, and the scale image can be used to measure the dimension of an undetected crack. is. In order to reflect undetected cracks in the damage diagram, the operation unit 24 can perform a tracing operation on the damage image in the mirrored image IMG12. For example, if the operation unit 24 is composed of a touch panel, tracing the crack image in the mirrored image IMG12 with a finger or a pen adds the locus of the trace to the damage diagram as a damage figure. Here, the shape and position of the crack image in the mirrored image IMG12 in the CAD drawing 80 correspond to the actual shape and position of the crack when the bridge 1 is viewed from above (the side opposite to the imaging side). So you can easily and accurately enter.

"Category" of the damage list 82 is a column for inputting an evaluation category (also referred to as "rank information") of the degree of damage. FIG. 14 is a diagram schematically showing an example of a crack progression model in the floor slab 6. As shown in FIG. In this progression model, the evaluation category RANK is expressed in five stages from "a" to "e". Correspondence between each evaluation category RANK and crack state STATE is as follows.

a: No damage.

b: A state in which a plurality of cracks are generated in parallel along the lateral direction (transverse direction perpendicular to the vehicle traffic direction). This is the stage where multiple cracks due to drying shrinkage form parallel beams.

c: A crack in the longitudinal direction (longitudinal direction parallel to the direction of travel of the vehicle) and a crack in the lateral direction intersect each other. This is the stage where multiple cracks form a grid due to the live load and the crack density in the grid region increases. In the latter half of the period, cracks penetrate the floor slab in the vertical direction (perpendicular to the bottom surface of the floor slab).

d: A state in which the crack density in the lattice-like region exceeds a specified value, and the fracture surfaces of a plurality of penetrating cracks are smoothed. This is the stage where the floor slab loses shear resistance due to the polishing action.

e: A state in which omission has occurred. Wheel loads exceeding the reduced punch shear strength cause pull-out.

Next, it is determined whether or not editing of all elements has been completed (step S20). In this example, it is determined whether or not all the coffers forming the floor slab 6 (element numbers Ds0101 to Ds0104 and Ds0201 to Ds0204 in this example) have been edited.

If there is an unedited element (NO in step S20), the process returns to step S6. When all elements have been edited (YES in step S20), the entire damage diagram is edited (step S22). In this step, for example, the operation section 24 is used to input the soundness judgment classification for the floor slab 6 as a whole.

Next, the edited damage diagram is output (step S24). The damage diagram can be displayed on the display unit 22 by the display control unit 32, and can also be printed out on a printer (not shown) via the network. For example, the damage diagram shown in FIG. 6 is printed out by a printer. In addition, damage diagrams can be uploaded to a database (not shown) via a network.

Although the damage diagram shown in FIG. 6 does not include a mirrored image, the present invention is not limited to such a case. A mirror image may be included in the damage diagram. Also, a mirror image may be embedded in the damage drawing instead of writing the damage figure. In other words, the contents of the damage diagram edited by the editing unit 42 may be appropriately changed according to the format of the damage diagram required or allowed.

An example of processing when an image for each coffer of the floor slab 6 is input has been described above. Preferably, each coffer is combined into a single composite image. In the following, examples of damage drawing editing processing will be introduced separately for the case of synthesizing a plurality of divided images after mirror reversal and the case of mirror reversing a plurality of divided images after synthesizing.

FIG. 15 is a flow chart showing the flow of the second example of damage diagram editing processing in the first embodiment. This process is executed by the CPU 30 according to the program stored in the storage unit 50 . Note that steps similar to those of the first example of damage diagram editing processing shown in FIG.

In this example, the image input unit 26 inputs a plurality of divided images of the floor slab 6 of the bridge 1 (step S102). Next, the image correction unit 34 performs necessary image correction on each divided image (step S104). Next, each divided image is mirror-inverted by the mirror-inverting unit 36 (step S106). Next, the plurality of mirror-reversed divided images (reflected images) are superimposed on the CAD drawing showing the bridge 1 viewed from the side opposite to the imaging side and displayed on the display unit 22 (step S108). . Next, the image synthesizing unit 38 synthesizes the mirror-inverted divided images (step S110).

FIG. 16 is a flow chart showing the flow of a third example of damage diagram editing processing according to the first embodiment. This process is executed by the CPU 30 according to a program stored in the storage unit 50 . Note that steps similar to those of the first example of damage diagram editing processing shown in FIG.

In this example, the image input unit 26 inputs a plurality of divided images of the floor slab 6 of the bridge 1 (step S202). Next, the image correction unit 34 performs necessary image correction on each divided image (step S204). Next, a plurality of divided images (non-reflected images) are superimposed on the CAD drawing showing the bridge 1 viewed from the imaging side and displayed on the display unit 22 (step S206). Next, the image synthesizing unit 38 synthesizes a plurality of divided images (step S208). Next, the composite image is mirror-inverted by the mirror-inverting unit 36 (step S210). A CAD drawing can be mirrored with the composite image.

[Second embodiment]
FIG. 17 is a block diagram showing a configuration example of a damage diagram editor 200 according to the second embodiment. The same components as those of the damage diagram editing apparatus 20 in the first embodiment shown in FIG.

A damage diagram editing apparatus 200 of the second embodiment includes a damage diagram input unit 60 for inputting a damage diagram showing the state of damage seen from the side opposite to the imaging side of the structure.

The damage diagram reversing unit 44 of this example converts the damage diagram input by the damage diagram input unit 60 into an inverted damage diagram showing the state of damage as seen from the imaging side of the structure.

The display control unit 32 of this example has a first existing damage diagram display function for simultaneously displaying the input damage diagram and the mirrored image on the display unit 22, and displays the reversed damage diagram and the non-reflected image on the display unit 22. It has a second existing damage diagram display function to display at the same time.

According to the first existing damage diagram display function, the past damage state seen from the side opposite to the imaging side of the structure (for example, the upper side of the bridge 1) is compared with the current mirrored image to determine the progress of the damage. You can check the status.

According to the second existing damage diagram display function, the progress of damage can be confirmed by comparing the past damage state seen from the imaging side of the structure (for example, the lower side of the bridge 1) and the current image. can be done.

[Variation of mirror inversion]
In the first and second embodiments described above, in order to facilitate understanding of the present invention, the case where the direction of the axis of reflection reversal is fixed was explained as an example. It is not limited to the case where the direction of the image reversal axis is fixed. The present invention may have a configuration in which the direction of the reflection reversal axis is variable.

The mirror inversion unit 36 has a function of determining the direction of the axis of reflection inversion based on various information. The mirror reversal unit 36 performs mirror reversal centering on the mirror reversal axis in the determined direction.

Firstly, there is a mode in which an image input by the image input unit 26 (which is an image before reflection reversal) is subjected to image analysis, and the direction of the reflection reversal axis is determined based on the result of the image analysis.

For example, assume that an image IMG21 for each coffer of the floor slab 6 shown in FIG. 18 is acquired. This image IMG21 includes an image EG of the edge of the crate (hereinafter referred to as "edge image"). In real space, the shape of the edge of a coffer is a rectangle, and the lengths of two mutually orthogonal sides forming the rectangle are generally different. That is, the edge rectangle has long sides and short sides that are perpendicular to each other. In the real space, the two sides of the edge rectangle that are orthogonal to each other are oriented in the same direction as the two coordinate axes Gx and Gy in the global coordinate system with the bridge 1 as a reference. Therefore, the mirror reversing unit 36 detects an edge image from the image before mirror reversal (a composite image may be used in the case of divided imaging), and after projectively transforming the edge image into a rectangle as necessary, , the direction of the mirror inversion axis can be determined based on the direction of one of the mutually orthogonal long and short sides of the rectangle. However, the imaging target (inspection target) of the present invention is not limited to the crate of the floor slab, and the direction of the reflection reversal axis may be determined based on the geometric features of other imaging targets. Needless to say.

Secondly, the coordinate axes of the global coordinate system (“first coordinate system”) based on the structure and the local coordinate system (“second coordinate system”) based on the imaging device 10 that captured the image ) is used to determine the direction of the axis of reflection inversion.

For example, as shown in FIG. 19, an image captured by the imaging device 10 mounted on the robot device 70 is input by the image input unit 26, and imaging device control information according to the imaging device control of the robot device 70 is input. is input by the image input unit 26 . As shown in FIG. 20, this imaging device control information includes coordinate axes Gx, Gy, and Gz of a global coordinate system with the bridge 1 as a reference, and coordinate axes Lx, Ly, and Lz of a local coordinate system with the imaging device 10 as a reference. It contains angle information indicating the angle between the two. In the local coordinate system of this example, one coordinate axis Lz of the three coordinate axes Lx, Ly, and Lz orthogonal to each other is the imaging direction (also referred to as the optical axis direction) of the imaging device 10, and is the floor slab 6 of the bridge 1. (which is the surface to be imaged). That is, in this example, the direction of one coordinate axis Gz of the global coordinate system and the direction of one coordinate axis Lz of the local coordinate system match.

The robot apparatus 70 of this example is a suspension type robot suspended from a bridge, and FIG. 19 shows only the essential parts related to imaging. The suspended robot may be a robot that suspends from a running body (vehicle) on a bridge instead of suspending from the bridge. Also, the present invention can be applied to a case where an unmanned flying robot such as a drone (unmanned flying object) is used instead of the suspended robot.

The robot device 70 of this example includes an imaging device control mechanism 72 that controls movement and rotation of the imaging device 10 . The image pickup device control mechanism 72 includes an XYZ movement mechanism capable of moving the image pickup device 10 in each of the Gx, Gy, and Gz directions orthogonal to each other, and a rotation mechanism for rotating the image pickup device 10 in each of the pan direction P and tilt direction T. It also serves as a possible pan-tilt mechanism.

The mirror inversion unit 36 determines the direction of the axis of reflection inversion based on the angle information indicating the angle between the coordinate axis Gx (or Gy) of the global coordinate system and the coordinate axis Lx (or Ly) of the local coordinate system. is possible.

Further, when the imaging device 10 is rotated in the panning direction P, that is, when the azimuth angle of the imaging device 10 is variable with respect to the global coordinate system with the bridge 1 as a reference, the reflection inverting unit 36 rotates the global coordinate system Based on the azimuth angle of the imaging device 10 with respect to the coordinate axis Gx or Gy of , it is possible to determine the direction of the mirror inversion axis.

Since there are various modes of setting the coordinate axes Lx, Ly, and Lz of the local coordinate system with respect to the coordinate axes Gx, Gy, and Gz of the global coordinate system, and pan-tilt modes (angle control modes) of the imaging apparatus 10, the local coordinates It is only necessary to obtain angle information according to the setting mode of the coordinate axes of the system and the angle control mode of the imaging device 10, and determine the direction of the reflection reversal axis based on the angle information.

[Client-server type]
In a client-server system, at least one of a client device and a server device may constitute a damage diagram editing device. In other words, the damage diagram editing apparatus of the present invention may be configured by one or a plurality of computer devices in a client-server system.

Although the embodiments for carrying out the present invention have been described above, the present invention is not limited to the above-described embodiments and modifications, and various modifications are possible without departing from the gist of the present invention.

1 Bridge 2 Main girder 3 Cross girder 4 Tilting structure 5 Horizontal structure 6 Floor slab 10 Imaging device 20 Damage drawing editing device 22 Display unit 24 Operation unit 26 Image input unit 28 Drawing input unit 30 CPU
32 display control unit 34 image correction unit 36 reflection reversing unit 38 image synthesizing unit 40 damage detection unit 42 editing unit 44 damage drawing reversing unit 50 storage unit 60 damage drawing input unit 70 robot device 72 imaging device control mechanism 80 CAD drawing 82 damage List 200 Damage diagram editing device Ds0101 to Ds0104, Ds0201 to Ds0204 Element number EG Edge images Gx, Gy, Gz Coordinate axes of the global coordinate system IMG1, IMG11, IMG21 Images IMG2, IMG12 Reflected images Lx, Ly, Lz Of the local coordinate system Coordinate axis MAX Reflection reversal axis P Pan direction RANK Evaluation category STATE State T Tilt direction

Claims (11)

an image input unit for inputting an image obtained by imaging a structure to be inspected with an imaging device;
a mirror reversing unit for mirror reversing the input image to obtain a mirror image;
a damage detection unit that detects a damage image from the mirrored image or the image before mirror reversal;
an image synthesis unit that performs image synthesis;
a storage unit that stores the mirrored image;
an operation unit, an editing unit for editing a damage diagram showing a state of damage to the structure and corresponding to the mirrored image, and a display unit for displaying and outputting the damage diagram edited by the editing unit. a display control unit for outputting the mirrored image stored in the storage unit to a client terminal, the output unit for outputting the mirrored image according to an operation accepted by the operation unit. an output unit ;
with
The image input unit inputs a plurality of divided images of the structure as the images,
The mirror inverting unit mirror inverts each of the plurality of divided images,
The image synthesizing unit synthesizes the plurality of mirror-inverted divided images to generate a synthesized image ,
The reflection inverting unit performs image processing to invert the input image about a reflection inversion axis, thereby reversing the imaged surface of the structure in the direction opposite to the imaging direction. Acquiring an image showing a state captured from the back side of the surface as the reflected image;
Server equipment. an image input unit for inputting an image obtained by imaging a structure to be inspected with an imaging device;
a mirror reversing unit for mirror reversing the input image to obtain a mirror image;
a damage detection unit that detects a damage image from the mirrored image or the image before mirror reversal;
an image synthesis unit that performs image synthesis;
a storage unit that stores the mirrored image;
an editing unit for editing a damage diagram showing the state of damage to the structure and corresponding to the mirrored image; and displaying and outputting the damage diagram edited by the editing unit to a display unit. and a display control unit for outputting the mirrored image stored in the storage unit to a client terminal, the output unit outputting the mirrored image according to an operation accepted by the operation unit. an output unit for
with
The image input unit inputs a plurality of divided images of the structure as the images,
The mirror inverting unit mirror inverts a synthesized image obtained by synthesizing the plurality of divided images by the image synthesizing unit ,
The reflection inverting unit performs image processing to invert the input image about a reflection inversion axis, thereby reversing the imaged surface of the structure in the direction opposite to the imaging direction. Acquiring an image showing a state captured from the back side of the surface as the reflected image;
Server equipment. 3. The server device according to claim 1, wherein the direction of said mirror inversion axis is fixed. 3. The server device according to claim 1, wherein the direction of said mirror inversion axis is variable. 5. The server apparatus according to claim 4, wherein the direction of said mirror inversion axis is determined by image analysis of said input image. The mirror inversion unit detects a geometric feature from the image before the mirror inversion or the synthesized image by the image analysis, and the direction of the reflection inversion axis is determined based on the geometric feature. 6. The server device according to claim 5. The reflection inversion unit provides angle information indicating an angle formed between a coordinate axis of a first coordinate system based on the structure and a coordinate axis of a second coordinate system based on the imaging device that captured the image. 5. The server apparatus according to claim 4, wherein the direction of the axis of mirror reversal is determined based on . an image input step of inputting an image obtained by imaging a structure to be inspected with an imaging device;
a mirror reversal step of mirror reversing the input image to obtain a mirror image;
a damage detection step of detecting a damage image from the mirror image or the image before mirror reversal;
an image synthesis step of performing image synthesis;
a storage step of storing the mirrored image in a storage unit;
an operation unit, an editing unit for editing a damage diagram showing a state of damage to the structure and corresponding to the mirrored image, and a display unit for displaying and outputting the damage diagram edited by the editing unit. an output step of outputting the mirror image stored in the storage unit to a client terminal having a display control unit, wherein the mirror image is output according to an operation accepted by the operation unit. an output step ;
has
The image input step inputs a plurality of divided images of the structure as the images,
The mirror reversal step includes mirror reversing each of the plurality of divided images,
The image synthesizing step includes synthesizing the plurality of mirror-inverted divided images to generate a synthetic image ,
In the mirror reversal step, by performing image processing for reversing the input image about a mirror reversal axis, the surface of the structure to be captured is reversed in the direction opposite to the imaging direction. Acquiring an image showing a state captured from the back side of the surface as the reflected image;
Image processing method. an image input step of inputting an image obtained by imaging a structure to be inspected with an imaging device;
a mirror reversal step of mirror reversing the input image to obtain a mirror image;
a damage detection step of detecting a damage image from the mirror image or the image before mirror reversal;
an image synthesis step of performing image synthesis;
a storage step of storing the mirrored image in a storage unit;
an operation unit, an editing unit for editing a damage diagram showing a state of damage to the structure and corresponding to the mirrored image, and a display unit for displaying and outputting the damage diagram edited by the editing unit. an output step of outputting the mirror image stored in the storage unit to a client terminal having a display control unit, wherein the mirror image is output according to an operation accepted by the operation unit. an output step ;
has
In the image input step, a plurality of divided images of the structure are input as the images;
In the mirror reversal step, the composite image obtained by synthesizing the plurality of divided images in the image synthesizing step is mirror-reversed ,
In the mirror reversal step, by performing image processing for reversing the input image about a mirror reversal axis, the surface of the structure to be captured is reversed in the direction opposite to the imaging direction. Acquiring an image showing a state captured from the back side of the surface as the reflected image;
Image processing method. A mirror reversal function for generating a mirror image by mirror reversing an image obtained by imaging a structure to be inspected with an imaging device;
a damage detection function for detecting a damage image from the mirror image or the image before mirror reversal;
an image synthesizing function for synthesizing a plurality of divided images of the structure to generate a synthesized image;
the mirror reversal function for mirror reversing the composite image;
a storage processing function for storing the mirrored image in a storage unit;
An editing function for editing a damage diagram showing the state of damage to the structure according to an operation received by an operation device of a client terminal, and a display control for displaying and outputting the damage diagram edited by the editing function to a display device. an output function of outputting a damage image corresponding to the mirror image stored in the storage unit to a damage diagram editing program having a function;
A program for realizing on a computer,
The mirror reversal function causes the computer to perform image processing for reversing the image around a mirror reversal axis, thereby reversing the surface of the structure to be captured in the direction opposite to the imaging direction. causing the computer to acquire an image showing a state captured from the back side of the imaging surface as the mirrored image;
program. A mirror reversal function for generating a mirror image by mirror reversing an image obtained by imaging a structure to be inspected with an imaging device;
a damage detection function for detecting a damage image from the mirror image or the image before mirror reversal;
the mirror reversal function of mirror reversing each of a plurality of divided images of the structure as the images;
an image synthesizing function of synthesizing the plurality of mirror-reversed split images;
a storage processing function for storing the mirrored image in a storage unit;
An editing function for editing a damage diagram showing the state of damage to the structure according to an operation received by an operation device of a client terminal, and a display control for displaying and outputting the damage diagram edited by the editing function to a display device. an output function of outputting a damage image corresponding to the mirror image stored in the storage unit to a damage diagram editing program having a function;
A program for realizing on a computer,
The mirror reversal function causes the computer to perform image processing for reversing the image around a mirror reversal axis, thereby reversing the surface of the structure to be captured in the direction opposite to the imaging direction. causing the computer to acquire an image showing a state captured from the back side of the imaging surface as the mirrored image;
program.

Images