JP6829846B2 - Structure inspection system, structure inspection method and program - Google Patents

Description

The present invention relates to a structure inspection system, a structure inspection method and a program, and particularly to a technique for easily associating the state of a deformed portion with coordinates.

Conventionally, in the inspection work of structures such as tunnels, bridges and buildings, the place where a person is deformed or deteriorated (hereinafter, simply referred to as “deformed”) by visual inspection, tapping sound, palpation, etc. of the wall surface (hereinafter, deformed part). ), Marked the deformed part with a writing tool, sketched the marked part, and reflected it in the drawing based on the sketch. For example, in the case of tunnel inspection, the deformed part of the lining surface is marked with chalk, the marking position is sketched on a paper surface, and the sketch is traced by CAD to create a lining development drawing.

However, in the conventional method, it is necessary to sketch the marking position, bring back the sketch and convert it into data, and transfer the information. Such a posting operation is very complicated and takes time and effort. In addition, since the posting work is performed manually by humans, mistakes and errors may be introduced.

That is, conventionally, the correspondence between the information obtained by the inspection (state of the deformed part) and the inspected part (coordinates of the deformed part) has depended on human manual work. In order to prove the relationship between the two, a method such as taking a picture of the inspection status with a digital camera may be taken, but it is only an aid to manual work.

Therefore, if the state and coordinates of the deformed part can be electronically grasped by some method and automatically associated with each other, the efficiency and accuracy of the inspection work should be improved.

In Patent Document 1, a deformed portion marked with a chalk on the tunnel interior lining is photographed (a three-dimensional model with brightness information is created by a laser scanner), and a lining development drawing is created based on the photographed data. A configuration is described in which the inspector visually recognizes and traces the markings on the lining development drawing to plot the coordinates of the deformed portion on the lining development drawing. Patent Document 2 describes a configuration in which a photographed image of an outer wall surface of a structure is acquired together with distance information, and an inspection result is marked with a pen device on a monitor on which the photographed image is displayed.

JP-A-2019-020348 Japanese Unexamined Patent Publication No. 2003-214829

However, in the methods described in Patent Documents 1 and 2, a three-dimensional laser scanner, a fixed laser range finder, or the like is required to obtain the coordinates of the deformed portion, which requires time and effort to install the device, and a location with poor visibility. There is a problem that it is not suitable for work in high places.

The present invention has been made to solve such a problem, and provides an inspection system, a structure inspection method, and a program capable of easily associating the state of a deformed portion with the coordinates. The purpose.

An inspection system according to an embodiment of the present invention recognizes an inspection device that inspects deformation at an inspection position on the surface of a structure, a three-dimensional shape of the structure, and a position of the inspection device, and three-dimensionally recognizes the structure. wherein recognizing space recognition device its own position based on the shape, the moving unit incorporating the said test device and the space recognition apparatus, and detects that the test device has performed the test, when the detection The moving body has an inspection position specifying unit for specifying the inspection position, information on the deformation, and an output unit for outputting the inspection position to a storage area, and the moving body has the inspection device in the past. The inspection position in the inspection performed is received as a target position, and the inspection position is moved toward the target position based on the self-position. The inspection device detects that the target position has been reached and executes the inspection. ..
An inspection system according to an embodiment of the present invention includes an inspection device that inspects deformation by photographing the surface of the structure at an inspection position on the surface of the structure, and a three-dimensional shape of the structure and the inspection device. The terminal device includes a space recognition device that recognizes a position and a terminal device that can communicate with the space recognition device. The terminal device detects that the inspection device has executed the inspection, and the inspection position at the time of the detection. and examination-position specifying unit for specifying a, the output unit for outputting the information in the storage area related to changing the shape, a display unit for displaying the previous SL photographic image, one or more two-dimensional inspector on the image is specified An input unit for acquiring time-series data consisting of coordinates, and a trajectory acquisition for acquiring time-series data consisting of one or more three-dimensional coordinates obtained by projecting the two-dimensional coordinates onto the surface of the structure as information on the deformation. It has a part and.
The inspection system according to one embodiment of the present invention includes an inspection device that inspects deformation at an inspection position on the surface of a structure, a space recognition device that recognizes the three-dimensional shape of the structure and the position of the inspection device, and the inspection. The space recognition device includes an inspection position specifying unit that detects that the inspection device has executed the inspection and specifies the inspection position at the time of the detection, including a plotting device that plots the position, and the deformation. The plotting device has the information about, the inspection position, and an output unit that outputs the inspection position to the storage area, and the plotting device converts the inspection position into a coordinate system defined by the origin set in the real space. It has a conversion coordinate acquisition unit that acquires the original coordinate information and a plot unit that plots the three-dimensional coordinate information in the drawing, and the plot unit does not have the three-dimensional coordinate information on the surface of the structure. In the case, the three-dimensional coordinate information is converted into a projection point on the surface of the structure.
An inspection method according to an embodiment of the present invention includes a space recognition device that recognizes a three-dimensional shape of a structure and a position of an inspection device, and recognizes a self-position based on the three-dimensional shape of the structure, and the inspection device. the equipped with moving body receives the inspection position in the inspection of the inspection apparatus has performed in the past as target position, and moving toward the target position based on the self-position, the inspection apparatus, the moving The step of detecting that the body has reached the target position and inspecting the deformation at the inspection position on the surface of the structure, and the space recognition device detecting that the inspection device has performed the inspection, The step includes a step of specifying the inspection position at the time of the detection, and a step of the space recognition device outputting the information regarding the deformation and the inspection position to the storage area.
The inspection method, which is one embodiment of the present invention, includes a step in which the inspection device inspects the deformation by photographing the surface of the structure at the inspection position on the surface of the structure, and a space recognition device in which the structure is inspected. The step of recognizing the three-dimensional shape and the position of the inspection device, the step of detecting that the inspection device has executed the inspection, and the step of specifying the inspection position at the time of the detection, and the terminal device , The step of displaying the captured image, the step of the terminal device acquiring time series data consisting of one or more two-dimensional coordinates specified by the inspector on the image, and the step of the terminal device performing the two-dimensional coordinates. The step of acquiring time-series data consisting of one or more three-dimensional coordinates obtained by projecting the above on the surface of the structure as information on the deformation, and the terminal device outputs the information on the deformation to the storage area. Including steps.
The inspection method according to one embodiment of the present invention includes a step in which the inspection device inspects the deformation at the inspection position on the surface of the structure, and a space recognition device that recognizes the three-dimensional shape of the structure and the position of the inspection device. , The step of detecting that the inspection device has executed the inspection and specifying the inspection position at the time of the detection, and the space recognition device stores information about the deformation and the inspection position in a storage area. An output step, a step in which the plotting device acquires three-dimensional coordinate information obtained by converting the inspection position into a coordinate system defined by an origin set in the real space, and a step in which the plotting device obtains the three-dimensional coordinate information. Includes a step of converting the three-dimensional coordinate information into projection points on the surface of the structure and plotting them in the drawing if is not on the surface of the structure.
A program according to an embodiment of the present invention is a program for causing a computer to execute the above method .

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide an inspection system, a structure inspection method, and a program capable of easily associating the state of a deformed portion with the coordinates.

It is a figure which shows the functional structure of the space recognition apparatus 1. It is a figure which shows the appearance of the space recognition apparatus 1. It is a figure which shows the functional structure of the inspection system 100 which concerns on Embodiment 1. FIG. It is a figure which shows the hardware configuration example of the inspection system 100 which concerns on Embodiment 1. FIG. It is a figure which shows the functional structure of the inspection system 100 which concerns on Embodiment 2. FIG. It is a figure which shows the hardware configuration example of the inspection system 100 which concerns on Embodiment 2. FIG. It is a figure which shows the functional structure of the inspection system 100 which concerns on Embodiment 3. FIG. It is a figure which shows the hardware configuration example of the inspection system 100 which concerns on Embodiment 3. FIG. It is a figure which shows the functional structure of the inspection system 100 which concerns on Embodiment 4. FIG. It is a figure which shows the functional structure of the inspection system 100 which concerns on Embodiment 5.

Structures such as concrete may be deformed and deteriorated. Deformations include bean boards, cold joints, internal defects, sand streaks, surface bubbles, cracks / floats / peeling, rust juice, efflorescence, stains (discoloration), wear, deflection, deformation, water leakage, paint swelling, etc. Deterioration includes neutralization, salt damage, alkali-silica reaction, frost damage, chemical corrosion, fatigue, weathering / aging, fire and the like. In the present specification, these deformations and deteriorations are collectively referred to as deformations. The inspection system 100 according to the embodiment of the present invention is a system capable of easily associating the state of the deformed portion with the coordinates. Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an example of a functional configuration of the space recognition device 1 used in the embodiment of the present invention. The space recognition device 1 has an inspection position specifying unit 11 and an output unit 13.

The inspection position specifying unit 11 performs a process of specifying three-dimensional coordinates indicating a position (referred to as an inspection position) where the structure is inspected. The output unit 13 performs a process of outputting the inspection position and the like specified by the inspection position specifying unit 11. The detailed operation of these processing units will be described later.

FIG. 2 is a diagram showing an example of the appearance of the space recognition device 1, in which the inspection position specifying unit 11 and the output unit 13 are mounted in the housing of the head-mounted device. The head-mounted device is typically an HMD (Head Mounted Display) having a camera, a depth sensor and a transmissive display, and includes, for example, Hollens® and the like. The depth sensor can scan the real space and acquire three-dimensional point cloud data showing the shape of a short-distance object. The transmissive display allows the user to see the real space and arbitrary information at the same time. This can be achieved, for example, by superimposing arbitrary information on an optical transmission or video transmission display.

The space recognition device 1 acquires three-dimensional point cloud data indicating the shape of the surface of the surrounding structure by the depth sensor. The three-dimensional point cloud data is mapped to the device-specific coordinate space by a predetermined setting operation. When the space recognition device 1 moves, the self-position in the coordinate space is specified by comparing the 3D point cloud data acquired by the depth sensor with the 3D point cloud data mapped in the coordinate space. Can be done.

When a base point such as an AR marker is set in advance in the field of view of the camera, the space recognition device 1 captures an image including the AR marker with the camera and analyzes the angle and size of the AR marker to analyze the AR marker. It is possible to calculate the coordinates of the space recognition device 1 in the coordinate space whose origin is. In this case, the space recognition device 1 can mutually convert the device-specific coordinate space and the coordinate space with the AR marker as the origin.

<Embodiment 1>
FIG. 3 is a diagram showing a functional configuration of the inspection system 100 according to the first embodiment of the present invention. The inspection system 100 includes a space recognition device 1 and an inspection device 2. The space recognition device 1 has an inspection position specifying unit 11 and an output unit 13.

FIG. 4 is a diagram showing a hardware configuration example of the inspection system 100. As the space recognition device 1, the head-mounted device described above can be adopted.

The space recognition device 1 can identify the self-position, and when the inspection device 2 is present in the field of view of the camera, the space recognition device 1 inspects by analyzing the image data or the three-dimensional point group data acquired by the camera or the depth sensor. It is also possible to specify the position of the device 2 and the position of a specific part of the inspection device 2.

The inspection device 2 is a device that inspects deformation at the inspection position on the surface of the structure, for example, tapping sound inspection, infrared inspection, appearance inspection, compression strength inspection of concrete, crack / peeling / cavity inspection, reinforcing bar / cover thickness.・ Perform buried object inspection, rebar corrosion inspection, alkali silica reaction inspection, fire inspection, etc. Further, the inspection device 2 also includes a device for inspecting the ground condition at the inspection position of the ground, and can perform, for example, a geological survey, an in-situ test, a geophysical survey, and the like.

The hitting sound inspection includes a method of hitting with an inspection hammer and determining a deformed part based on the difference in hitting sound, and a method of judging a deformed part based on a difference in the frequency of elastic waves due to hitting. As the inspection device 2 for realizing these methods, there are a hammer with a sound collector, a hammer with a frequency measuring device, and the like.

Infrared inspection is a method of capturing infrared rays radiated from the surface of an object as an image and identifying the deformed part from the difference in temperature shown in the contour diagram. As an inspection device 2 for realizing this method, there is an infrared camera (including a video camera) and the like.

In addition, visual inspection can be carried out with a digital camera or the like. The compressive strength inspection of concrete (strength estimation by core strength, resilience method, etc.) can be carried out by a core drill, rebound hammer, or the like. Crack / peeling / cavity inspection (by thermography, elastic wave, tapping method, etc.) can be performed by an infrared imaging device, elastic wave generator / receiver, AE signal detection device, or the like. Reinforcing bar / cover thickness / buried object inspection (by electromagnetic induction, electromagnetic wave radar, X-ray, etc.) can be carried out by a reinforcing bar probe, electromagnetic wave radar, X-ray device, or the like. Reinforcing bar corrosion inspection (based on neutralization depth, chloride ion content, reinforcing bar corrosion amount, etc.) can be carried out by a drill, a voltmeter, a polarization resistance measuring instrument, an electric resistance measuring instrument, or the like. The alkali-silica reaction test can be performed with an electron microscope. Fire inspection (repulsion test, neutralization test, core extraction test, vibration test, loading test, etc.) can be performed by a rebound hammer, drill, core drill, vibration test device, or the like. In addition, as a device for inspecting the ground condition, a geological survey (based on boring core, N value, drilling energy, etc.) can be carried out by a boring machine, a standard penetration tester, a drilling logging device, or the like. The in-situ test (based on elastic modulus, ground bearing capacity, cone index, rock strength, etc.) can be performed by an in-hole loading tester, a flat plate loading tester, a cone penetration tester, a rock Schmidt hammer, or the like. Geophysical exploration (by elastic wave, resistivity, etc.) can be carried out by elastic wave generator / receiver, electrical exploration device, etc.

The inspection position specifying unit 11 detects that the inspection device 2 has executed the above-mentioned inspection. For example, in the case of a tapping sound inspection, the inspection position specifying unit 11 detects a hit by an inspection hammer included in the inspection device 2. For example, if the space recognition device 1 and the inspection device 2 are connected, the inspection position specifying unit 11 can detect the change in acceleration that occurs at the time of hitting by the acceleration sensor included in the space recognition device 1. Alternatively, the inspection position specifying unit 11 can detect that the tip of the inspection hammer has come into contact with the surface of the structure by analyzing the image data or the three-dimensional point cloud data acquired by the camera or the depth sensor. Alternatively, the inspection position specifying unit 11 can detect a change in sound generated at the time of hitting by the microphone provided in the space recognition device 1. In the case of infrared inspection, the inspection position specifying unit 11 can communicate with the inspection device 2 and detect the shooting execution of the infrared camera included in the inspection device 2.

In addition, the inspection position specifying unit 11 specifies the inspection position when the inspection is executed by the inspection device 2. For example, in the case of tapping sound inspection, the inspection position specifying unit 11 analyzes the image data or the three-dimensional point cloud data acquired by the camera or the depth sensor to obtain the coordinates of the tip of the inspection hammer at the moment when the inspection execution is detected. Can be identified. In the case of infrared inspection, the inspection position specifying unit 11 sets coordinates indicating the shooting range of the infrared camera based on the relative positional relationship between the space recognition device 1 and the inspection device 2, the orientation of the infrared camera, the angle of view, and the like. Can be identified.

The output unit 13 outputs the inspection position acquired by the inspection position specifying unit 11 in association with the inspection result by the inspection device 2. The inspection position is one or more three-dimensional coordinate data. The inspection result is information on the deformation, for example, acoustic data at the time of hitting in the case of tapping sound inspection, imaging data in the case of infrared inspection, or information indicating the inspector's findings based on these data. good. For example, it may be output to a storage area (not shown) in the space recognition device 1, or may be output to an external device via the communication function of the space recognition device 1.

In the first embodiment, the space recognition device 1 observes the state of the structure inspection by the inspection device 2, so that the information on the deformation and the inspection position can be directly acquired as data. This improves efficiency and accuracy compared to traditional manual labor.

<Embodiment 2>
FIG. 5 is a diagram showing a functional configuration of the inspection system 100 according to the second embodiment of the present invention. The inspection system 100 includes a space recognition device 1 and an inspection device 2 similar to those in the first embodiment, and a mobile body 3 equipped with these.

FIG. 6 is a diagram showing a hardware configuration example of the inspection system 100. As the space recognition device 1, a head-mounted device similar to that of the first embodiment can be adopted. Further, as the inspection device 2, a camera mounted on a head-mounted device can be adopted. Of the configurations and effects of the space recognition device 1 and the inspection device 2, those that overlap with the first embodiment will be omitted as appropriate in the present embodiment.

Similar to the first embodiment, the space recognition device 1 can specify the self-position. Further, as in the first embodiment, the inspection position specifying unit 11 detects that the inspection device 2 has executed the inspection, for example, the shooting by the camera mounted on the head-mounted device, and the coordinates indicating the shooting range. The set of is specified and set as the inspection position. The output unit 13 outputs the inspection position in association with the shooting data (image data).

The space recognition device 1 according to the second embodiment further includes a movement instruction unit 14. The movement instruction unit 14 commands the moving body 3 to move based on the self-position recognized by the space recognition device 1 and the movement route designated in advance. Typically, while updating the self-position at predetermined intervals and performing feedback control or the like based on the difference from the movement path specified in advance, a command including the movement direction and speed is output.

The travel path is typically given as a set of one or more 3D coordinate data with sequence or time information. For example, using a PC (Personal Computer) or the like (not shown), on the CAD data of the structure to be inspected, the target point, which is the final destination of the movement, and the coordinates to be passed through the automatically navigating moving body 3 are sequentially set. specify. The set of the three-dimensional coordinate data thus obtained is input to the movement instruction unit 14 as a movement path. The space recognition device 1 can mutually convert the coordinate system of CAD data and the coordinate system recognized by itself. Therefore, the movement instruction unit 14 can replace the movement route specified on the CAD data with a coordinate system recognized by itself, and can command a movement route suitable for the real space. That is, the current position of the moving body 3 equipped with the space recognition device 1 is set as the starting point, and a command is output to move to the coordinates specified first. When it reaches that coordinate, it moves to the next specified coordinate. By repeating this, the moving body 3 can be automatically navigated to the coordinates of the target point.

The moving body 3 is a device that can move in space according to a given command, for example, a drone or a vehicle. The moving body 3 has a camera rig 31 and a movement control unit 32. In the example of FIG. 6, the head-mounted device, which is the space recognition device 1 and the inspection device 2, is fixed to the drone, which is the moving body 3, via the camera rig 31. Further, the moving body 3 may be provided with a gauge for protecting the moving body 3 itself, the space recognition device 1, and the inspection device 2 from an impact caused by a collision with a wall surface or the like.

The movement control unit 32 receives a command output by the movement instruction unit 14, and controls the movement direction, speed, and the like according to the command.

The inspection device 2 detects that the moving body 3 has reached the target point and executes the inspection. For example, the surface of a structure is photographed by a camera mounted on a head-mounted device.

When the movement instruction unit 14 completes the execution of the inspection by the inspection device 2 and the identification of the inspection position by the inspection position identification unit 11, the movement instruction unit 14 sequentially gives a command to follow the outward route in the reverse direction to the movement control unit 32 to reach the starting point. It can navigate autonomously.

In the second embodiment, a space recognition device 1 such as HoloLens is used to autonomously navigate a moving body 3 such as a drone to a predetermined inspection location, execute an inspection at the inspection location, and perform an inspection at the inspection location. Get in association with. This makes it possible to carry out an inspection at an inspection site that is difficult for humans to approach. For example, cracks, damage, peeling, etc. of concrete structures such as embankments, cuts and other soil structures, cracks, deformations, etc., at places where natural disasters occur, facets during tunnel construction, high places such as bridges, etc. It is possible to inspect the geology, weathering and alteration of the face during tunnel construction, crack width, crack interval, spring water, etc., such as exfoliation, water leakage, and deformation. Moreover, since the target point and the like can be set arbitrarily, the inspection device 2 can be brought close to a distance that can be visually observed by a camera, and the deformation can be investigated precisely.

In the conventional drone equipped with the autonomous navigation function by GPS, the accuracy of automatic navigation is greatly affected by the accuracy of GPS, but according to the second embodiment, the moving body 3 can be accurately navigated to the target point. It becomes.

As an application example of the second embodiment, the inspection point previously specified by the method of the first embodiment or the inspection point when the inspector manually inspects is given to the movement instruction unit 14 as a target point. Can be done. In this way, it is possible to continuously observe the progress of cracks, damage, peeling, exfoliation, water leakage, deformation, etc. of concrete structures, such as strata, cracks, and deformation of soil structures such as embankments and cuts. Is possible.

<Embodiment 3>
FIG. 7 is a diagram showing a functional configuration of the inspection system 100 according to the third embodiment of the present invention. The inspection system 100 includes a space recognition device 1, an inspection device 2, a mobile body 3 similar to the second embodiment, and a terminal device 4 capable of communicating with the space recognition device 1. The terminal device 4 includes a display unit 41, an input unit 42, an inspection position specifying unit 43, a locus acquisition unit 44, and an output unit 45.

FIG. 8 is a diagram showing a hardware configuration example of the inspection system 100. As the space recognition device 1, a head-mounted device similar to the first and second embodiments can be adopted. Further, as the inspection device 2, a camera mounted on a head-mounted device can be adopted. The moving body 3 is, for example, a drone equipped with a space recognition device 1 and an inspection device 2 and automatically navigates. The terminal device 4 is an information processing device arranged at a position remote from the space recognition device 1, the inspection device 2, and the mobile body 3. Of the configurations and effects of the space recognition device 1, the inspection device 2, and the moving body 3, those that overlap with the first and second embodiments will be omitted as appropriate in the present embodiment.

The terminal device 4 communicates with the space recognition device 1, and includes image data taken at the current position of the space recognition device 1, three-dimensional point cloud data indicating the shape of the surface of the surrounding structure at the time of shooting, and a camera. Acquire data indicating the position and orientation.

The display unit 41 displays the acquired image data. The input unit 42 acquires an input operation performed by the inspector on this image. For example, the display unit 41 and the input unit 42 may be an integrated device as a touch panel display. In this case, the input unit 42 acquires the coordinates of the position (designated position) touched by the inspector on the image displayed on the display which is the display unit 41.

The inspection position specifying unit 43 converts the coordinates of the designated position on the image into a coordinate system of three-dimensional point cloud data indicating the shape of the surface of the surrounding structure at the time of shooting, and the converted designated position is viewed from the camera viewpoint. The projection point projected on the surface of the structure is specified as the inspection position.

The locus acquisition unit 44 acquires time-series data consisting of one or more inspection positions as a locus. For example, when the inspector moves the fingertip on the touch panel, the trajectory of the fingertip, or more accurately, the time-series data of the projection point of the coordinates of the fingertip on the structure surface is acquired.

The output unit 45 outputs the locus acquired by the locus acquisition unit 44.

In the third embodiment, the inspector traces a deformed part such as a crack on the wall surface on the monitor while watching the image delivered from the space recognition device 1 mounted on the drone or the like to the terminal device 4 on the monitor. Perform various operations. As a result, the shape (including information such as size and angle) and coordinates of the crack can be directly acquired as data such as the shape and coordinates of the locus instructed by the fingertip. At this time, the fingertip does not have to touch the deformed part directly, and the coordinates on the wall surface on the extension of the fingertip can be obtained by tracing on the monitor.

According to the third embodiment, it is possible to perform the inspection from a remote place even at the inspection position where it is difficult for the inspector to directly approach. This eliminates the need for equipment required to bring people closer, such as aerial work platforms. In addition, the safety of the inspector is ensured. Furthermore, since it is not necessary to dispatch engineers with specialized knowledge to the site, limited human resources can be effectively utilized.

As an application example of the third embodiment, a marking device such as a spray or a color ball is mounted on the moving body 3, and the spray or the color ball or the like is fired at a remotely designated inspection position to obtain the surface of the structure. Can be marked directly on. This makes it possible to use the conventional visual inspection and the present invention in combination.

Further, in the third embodiment, since the shape and coordinates of the locus can be acquired by the terminal device 4, the space recognition device 1 does not necessarily have to include the inspection position specifying unit 11 and the output unit 13.

<Embodiment 4>
FIG. 9 is a diagram showing a functional configuration of the inspection system 100 according to the fourth embodiment of the present invention. The inspection system 100 includes a space recognition device 1 and an inspection device 2 as in the first to third embodiments. Further, the mobile body 3 as in the second embodiment and the terminal device 4 as in the third embodiment may be included.

In addition to these, the inspection system 100 has a machine learning unit 5. The machine learning unit 5 has a preprocessing unit 51 and a recognition unit 52. The machine learning unit 5 is typically provided in the space recognition device 1, but may be mounted in other hardware such as the inspection device 2 and the terminal device 4. Alternatively, it may be implemented by an external server (not shown) or a method such as cloud computing, edge computing, or fog computing.

The preprocessing unit 51 generates machine learning data based on the inspection result by the inspection device 2. For example, the feature amount extracted from the image data of the surface of the structure taken by the camera which is the inspection device 2 can be used as the machine learning data. Alternatively, the feature amount can be used as machine learning data from the acoustic data at the time of hitting with the inspection hammer.

The recognition unit 52 inputs machine learning data into the trained model and recognizes the deformation. The trained model can output determination data indicating whether or not it is deformed in response to input of machine learning data. The trained model uses, for example, image data or acoustic data collected in a past inspection and data indicating whether or not the image or sound created by a skilled inspector is deformed as supervised data. It can be generated by supervised learning.

According to the fourth embodiment, for example, the camera or the inspection hammer is operated while moving the moving body 3, and the machine learning unit 5 determines whether or not there is any deformation using the acquired image data or acoustic data. , It can be configured to output the inspection position and the inspection result in association with each other when it is determined to be abnormal. This enables autonomous deformation inspection.

<Embodiment 5>
FIG. 10 is a diagram showing a functional configuration of the inspection system 100 according to the fifth embodiment of the present invention. The inspection system 100 may include an inspection device 2, a mobile body 3, and a terminal device 4 in addition to the space recognition device 1 as shown in the first to fourth embodiments.

In addition to these, the inspection system 100 has a plotting device 6. The plotting device 6 has a converted coordinate acquisition unit 61 and a plotting unit 62.

The conversion coordinate acquisition unit 61 converts the coordinates of the inspection position into a coordinate system defined by the origin set in the real space. That is, when an arbitrary origin is set in the real space, the position of the space recognition device 1 is expressed as a relative position with respect to the origin. In the above-described embodiment, the case where the inspection position output by the space recognition device 1 is represented by the coordinate system peculiar to the space recognition device 1 has been mainly described. The converted coordinate acquisition unit 61 converts the coordinates of such an inspection position into a coordinate system set in the real space, and outputs the coordinates of the converted inspection position. The coordinates are three-dimensional coordinates. When the space recognition device 1 stores a plurality of inspection positions and the order, time, or arbitrary attribute information is added to the inspection positions before conversion, the conversion coordinate acquisition unit 61 performs the conversion after conversion. This information can also be added to the inspection position. Thereby, for example, it is possible to manage what kind of coordinates are formed in what order.

The plotting unit 62 plots the three-dimensional coordinate information generated by the conversion coordinate acquisition unit 61 in the drawing. Typical plotting methods include a method of plotting in CAD data and a method of plotting in a development view.

When plotting on CAD data, the plotting unit 62 converts the three-dimensional coordinates output by the conversion coordinate acquisition unit 61 into the coordinate system of the CAD data, and draws the point coordinates corresponding to the inspection position. When a plurality of inspection positions are output and data such as order or time is added, the plotting unit 62 may draw a locus in which the point coordinates corresponding to the inspection positions are connected in chronological order. When data such as attribute information is added to the inspection position, the plotting unit 62 may display according to the attribute information. For example, the display form (color, size, etc.) of the point coordinates and the locus corresponding to the inspection position may be changed according to the attribute information. Alternatively, characters or symbols indicating attribute information may be drawn.

When plotting on a two-dimensional development view of a structure, the plotting unit 62 draws point coordinates corresponding to each inspection position on the development view after generating a development view in which the structure is two-dimensionally developed.

When generating a development view of a structure having an arc, the plotting unit 62 obtains the curvature of the arc based on the three-dimensional point group data of the surface of the structure acquired by the space recognition device 1, and obtains the curvature of the arc on the surface of the structure. Converts the arc into a straightened XY plane. Thereby, the inspection position is also expressed as a point on the XY surface. The coordinates of the inspection position may not be on the XY plane due to variations in the accuracy of the depth sensor. In this case, the inspection position is converted into an approximate value such as a vertical projection point on the XY plane and then plotted.

When generating a developed view of a structure having no arc, the plotting unit 62 sets the top view and bottom view of the structure based on the three-dimensional point cloud data of the surface of the structure acquired by the space recognition device 1. Generate a side view of the orientation. Then, the inspection position is plotted on each of these figures. The coordinates of the inspection position may not be on the plane of each figure due to variations in the accuracy of the depth sensor. In this case, the inspection position is converted into an approximate value such as a vertical projection point on the nearest plane before plotting.

Even when plotting on a two-dimensional development view of a structure, if a plurality of inspection positions are output and data such as order or time is added, the plotting unit 62 is a point corresponding to the inspection position. A locus in which the coordinates are connected in chronological order may be drawn. When data such as attribute information is added to the inspection position, the plotting unit 62 may display according to the attribute information.

According to the fifth embodiment, by using the space recognition device 1 such as a wearable terminal equipped with a small depth sensor, it is possible to manage the deformed position and generate the development view. Therefore, it is not restricted by installing a fixed device such as a conventional laser scanner. In addition, even when compared with the conventional manual inspection, data that associates the information obtained by the inspection (state of the deformed part) with the inspected part (coordinates of the deformed part) can be output all at once. Therefore, it is excellent in efficiency and accuracy, and it is easy to compare with the deformation recorded in the past.

The present invention is not limited to the above embodiment, and can be appropriately modified without departing from the spirit. For example, the method of implementing the various processing means shown in the above-described embodiment by software or hardware is merely an example, and may be replaced by another method. For example, in the embodiment, it is possible to provide a plurality of processing means to be mounted in the same housing as a plurality of different hardware. Alternatively, a plurality of processing means that are to be implemented in different hardware in the embodiment can be implemented in one hardware. Further, some functions of any processing means or processing means may be realized by technologies such as cloud computing, edge computing, and fog computing.

Further, the calculation method, the sensing method, and the like shown in the above-described embodiment are merely examples, and may be replaced by another method having an equivalent function.

Further, each processing means constituting the present invention may be configured by hardware, or may be realized by causing a CPU to execute a computer program for arbitrary processing. Computer programs can also be stored and supplied to a computer using various types of temporary or non-transitory computer-readable media. Temporary computer-readable media include electromagnetic signals supplied to the computer, for example by wire or wirelessly.

100 Structure inspection system 1 Space recognition device 11 Inspection position identification unit 13 Output unit 14 Movement instruction unit 2 Inspection device 3 Mobile unit 31 Movement control unit 4 Terminal device 41 Display unit 42 Input unit 43 Inspection position identification unit 44 Trajectory acquisition unit 45 Output unit 5 Machine learning unit 51 Preprocessing unit 52 Recognition unit 6 Plot device 61 Conversion coordinate acquisition unit 62 Plot unit

Claims (7)

An inspection device that inspects deformation at the inspection position on the surface of the structure,
A space recognition device that recognizes the three-dimensional shape of the structure and the position of the inspection device and recognizes its own position based on the three-dimensional shape of the structure.
Anda moving unit incorporating the said space recognition device and the inspection device,
An inspection position specifying unit that detects that the inspection device has executed the inspection and specifies the inspection position at the time of the detection.
It has an output unit that outputs information about the deformation, the inspection position, and the storage area.
The moving body receives the inspection position in the inspection performed in the past by the inspection device as a target position, and moves toward the target position based on the self-position.
The inspection device is an inspection system that detects that the target position has been reached and executes the inspection. An inspection device that inspects deformation by photographing the surface of the structure at the inspection position on the surface of the structure.
A space recognition device that recognizes the three-dimensional shape of the structure and the position of the inspection device,
Including a terminal device capable of communicating with the space recognition device,
The terminal device is
An inspection position specifying unit that detects that the inspection device has executed the inspection and specifies the inspection position at the time of the detection.
An output unit for outputting information about the Deformation in the storage area,
A display unit that displays the captured image and
An input unit that acquires time-series data consisting of one or more two-dimensional coordinates specified by the inspector on the image, and an input unit.
An inspection system including a locus acquisition unit that acquires time-series data consisting of one or more three-dimensional coordinates by projecting the two-dimensional coordinates onto the surface of the structure as information on the deformation. An inspection device that inspects deformation at the inspection position on the surface of the structure,
A space recognition device that recognizes the three-dimensional shape of the structure and the position of the inspection device,
Including a plotting device for plotting the inspection position.
The space recognition device is
An inspection position specifying unit that detects that the inspection device has executed the inspection and specifies the inspection position at the time of the detection.
It has an output unit that outputs information about the deformation, the inspection position, and the storage area.
The plotting device is
A conversion coordinate acquisition unit that acquires three-dimensional coordinate information obtained by converting the inspection position into a coordinate system defined by the origin set in the real space.
It has a plot section for plotting the three-dimensional coordinate information in the drawing, and
The plotting unit is an inspection system that converts the three-dimensional coordinate information into a projection point on the surface of the structure when the three-dimensional coordinate information is not on the surface of the structure. A moving body equipped with a space recognition device that recognizes the three-dimensional shape of a structure and the position of an inspection device and recognizes its own position based on the three-dimensional shape of the structure, and the inspection device is the inspection device. a step but which receives an inspection position in the inspection performed in the past as a target position, moves toward the target position based on the self-position,
A step in which the inspection device detects that the moving body has reached the target position and inspects the deformation at the inspection position on the surface of the structure.
The space recognition device detects that the inspection device has executed the inspection, and identifies the inspection position at the time of the detection.
An inspection method in which the space recognition device includes a step of outputting information on the deformation and the inspection position to a storage area. A step in which the inspection device inspects the deformation by photographing the surface of the structure at the inspection position on the surface of the structure.
Space recognition apparatus, comprising: recognizing a three-dimensional shape and position of the inspection device of the structure,
A step in which the terminal device detects that the inspection device has executed the inspection and identifies the inspection position at the time of the detection.
When the terminal device displays the captured image,
A step in which the terminal device acquires time-series data consisting of one or more two-dimensional coordinates specified by the inspector on the image.
A step in which the terminal device acquires time-series data consisting of one or more three-dimensional coordinates obtained by projecting the two-dimensional coordinates onto the surface of the structure as information on the deformation.
The terminal device, the inspection method comprising the steps of outputting information about the Deformation in the storage area. The step that the inspection device inspects for deformation at the inspection position on the surface of the structure,
A step of detecting that the inspection device has executed the inspection by a space recognition device that recognizes the three-dimensional shape of the structure and the position of the inspection device and specifying the inspection position at the time of the detection.
A step in which the space recognition device outputs information on the deformation and the inspection position to a storage area.
A step in which the plotting device acquires three-dimensional coordinate information obtained by converting the inspection position into a coordinate system defined by an origin set in real space.
The plotting device includes a step of converting the three-dimensional coordinate information into a projection point on the surface of the structure and plotting it in the drawing when the three-dimensional coordinate information is not on the surface of the structure. Inspection methods. A program for causing a computer to execute the method according to any one of claims 4 to 6.