JP6650639B1 - Structure inspection apparatus, structure inspection method and program - Google Patents

Abstract

A structure inspection apparatus, a structure inspection method, and a program capable of easily associating a state of a deformed portion with coordinates. An inspection device is a device capable of recognizing a three-dimensional shape of a surrounding structure and a movement of an inspector's body, and the inspection device contacts the structure by a movement of a fingertip of the inspector. An inspection position specifying unit 11 for specifying coordinates of the surface of the structure designated as an inspection position without performing, and a trajectory obtaining unit for obtaining, as a trajectory, time-series data including a plurality of temporally continuous inspection positions. And an output unit 13 for outputting the trajectory. [Selection diagram] Fig. 1

Description

  The present invention relates to a structure inspection apparatus, a structure inspection method, and a program, and more particularly to a technique for easily associating a state of a deformed portion with coordinates.

  2. Description of the Related Art Conventionally, in inspection work of structures such as tunnels, bridges, buildings, and the like, a place where a person is deformed or deteriorated (hereinafter, simply referred to as a deformed state) by visual inspection of a wall surface, a tapping sound, palpation, etc. ), Marking the deformed part with a writing instrument, sketching the marked part, and performing the work of reflecting on the drawing based on the sketch. For example, in the case of a tunnel inspection, a deformed portion on a lining surface is marked with chalk, a marking position is sketched on a paper surface, and the sketch is traced by CAD to create a lining development diagram.

  However, in the conventional method, it is necessary to transfer information such as sketching a marking position, bringing the sketch back and converting it into data. Such a transfer operation is very complicated and requires time and effort. In addition, since the transcription operation is performed manually, there is a possibility that an error or an error is introduced.

  That is, in the related art, the correspondence between the information obtained by the inspection (the state of the deformed portion) and the inspected portion (the coordinates of the deformed portion) has depended on the manual work of a person. In order to prove the relationship between the two, a method such as taking a picture of the inspection status with a digital camera is sometimes used, but it was merely an aid to manual work.

  Therefore, if the state and coordinates of the deformed part can be grasped electronically by some method and automatically associated, the efficiency and accuracy of the inspection work should be improved.

  In Patent Document 1, a deformed portion marked with chalk on a lining in a tunnel is photographed (a three-dimensional model with luminance information is created by a laser scanner), and a lining development diagram is created based on the photographed data. A configuration is described in which an inspector visually recognizes and traces a marking on a lining development view to plot coordinates of a deformed portion on the lining development view. Patent Literature 2 describes a configuration in which a captured image of an outer wall surface of a structure is acquired together with distance information, and an inspection result is marked with a pen device on a monitor on which the captured image is displayed.

  Patent Literature 3 discloses a system that acquires a trajectory drawn in a space with an electronic pen and displays the trajectory on a display device such as an HMD (Head Mounted Display).

JP 2019-020348 A JP 2003-214829 A JP 2013-125487 A

  However, the methods described in Patent Literatures 1 and 2 require a three-dimensional laser scanner or a fixed laser range finder to obtain the coordinates of the deformed portion, which requires time and effort for installation of the apparatus, and places where visibility is poor. There is a problem that it is not suitable for work in high places.

  The method described in Patent Document 3 also has a problem that a special device such as an electronic pen is required. Furthermore, it may be difficult to associate a character or the like drawn in a space with a deformed portion. For example, when marking is performed by directly touching a deformed part that has occurred on a wall or the like with an electronic pen, the coordinates of the deformed position and the trajectory of the electronic pen almost match, so that the deformed part can be identified relatively accurately. it can. However, it is difficult to bring the electronic pen into contact when there is a deformed part in a position where a person is difficult to approach, such as a high place. In this case, a compromise such as marking with a digital pen at a position as close as possible to the deformed portion is considered, but this method cannot grasp the exact coordinates of the deformed portion.

  The present invention has been made to solve such a problem, and provides a structure inspection apparatus, a structure inspection method, and a program that can easily associate a state of a deformed portion with coordinates. The purpose is to do.

An inspection apparatus according to an aspect of the present invention is an apparatus capable of recognizing a three-dimensional shape of a surrounding structure and a movement of an examiner's body, and the structure of the inspection apparatus is determined by movement of a fingertip of the examiner. Locus acquisition for acquiring, as a locus, an inspection position specifying unit that specifies coordinates of the surface of the structure indicated without touching the inspection position as an inspection position, and time-series data including a plurality of temporally continuous inspection positions. And an output unit that outputs the trajectory, wherein the trajectory indicates the shape of the deformed portion of the structure, and the output unit is superimposed on the deformed portion of the structure. And projecting the trajectory obtained in the past on a transmissive display.
An inspection apparatus according to an embodiment of the present invention is an apparatus capable of recognizing a three-dimensional shape of a surrounding structure and movement of an inspector's body. Locus acquisition for acquiring, as a locus, an inspection position specifying unit that specifies coordinates of the surface of the structure indicated without touching the inspection position as an inspection position, and time-series data including a plurality of temporally continuous inspection positions. And an output unit that outputs the trajectory, wherein the trajectory indicates the shape of the deformed portion of the structure, and the output unit is superimposed on the deformed portion of the structure. And projecting the trajectory obtained in the past on a transmissive display.
An inspection method according to an embodiment of the present invention includes a step of specifying coordinates of a surface of a structure without touching the structure by a movement of a fingertip of an inspector; a step of specifying the coordinates as an inspection position; Acquiring a time-series data consisting of a plurality of inspection positions continuous as a trajectory, and outputting the trajectory, wherein the trajectory indicates a shape of a deformed portion of the structure , and the in the step of the output, superimposed on Deformation portion of said structure, characterized by projecting the trajectory obtained in the past on the transmission type display.
An inspection method according to an aspect of the present invention includes a step of designating coordinates of a surface of a structure without touching the structure by a movement of a line of sight of an inspector; a step of specifying the coordinates as an inspection position; Acquiring a time-series data consisting of a plurality of inspection positions continuous as a trajectory, and outputting the trajectory, wherein the trajectory indicates a shape of a deformed portion of the structure , and the in the step of the output, superimposed on Deformation portion of said structure, characterized by projecting the trajectory obtained in the past on the transmission type display.
A program according to one embodiment of the present invention is a program for causing a computer to execute the above method.

  According to the present invention, it is possible to provide a structure inspection apparatus, a structure inspection method, and a program that can easily associate a state of a deformed portion with coordinates.

FIG. 2 is a block diagram illustrating a functional configuration of the inspection device. FIG. 2 is a diagram illustrating an example of an appearance of the inspection device 1. 4 is a flowchart illustrating an operation of the inspection device 1 according to the first embodiment. 4 is a flowchart illustrating an operation of the inspection device 1 according to the first embodiment.

  Structures such as concrete may be deformed and deteriorated. Deformation includes bean boards, cold joints, internal defects, sand streaks, surface bubbles, cracks / floating / peeling, rust juice, efflorescence, dirt (discoloration), abrasion, deflection, deformation, water leakage, paint swelling and the like. Deterioration includes neutralization, salt damage, alkali-silica reaction, frost damage, chemical corrosion, fatigue, weathering / aging, fire, and the like. In the present specification, these deformations and deterioration are collectively referred to simply as deformations. The inspection apparatus 1 according to the embodiment of the present invention is an apparatus that can easily associate a state of a deformed portion with coordinates. Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram illustrating a functional configuration of the inspection apparatus 1 according to the first embodiment of the present invention. The inspection device 1 includes an inspection position identification unit 11, a trajectory acquisition unit 12, and an output unit 13.

  FIG. 2 is a diagram illustrating an example of the external appearance of the inspection apparatus 1, in which an inspection position identification unit 11, a trajectory acquisition unit 12, and an output unit 13 are mounted in a housing of the head-mounted device. The head mounted device is typically an HMD (Head Mounted Display) having a camera, a depth sensor, and a transmissive display, and includes, for example, Hololens (registered trademark). The depth sensor can scan the physical space to obtain three-dimensional point cloud data indicating the shape of a surrounding object. The transmissive display allows a user to visually recognize an object displayed on the display and a real space at the same time. By displaying an object on an optical transmission or video transmission display, the user perceives the information as being superimposed and displayed in the real space. The head mounted device is mounted on the head of a user who performs marking while holding a writing implement or the like.

  The inspection device 1 acquires three-dimensional point group data indicating the shape of the surface of the surrounding structure using the depth sensor. The coordinate space to which the three-dimensional point cloud data is mapped is called world coordinates. When the inspection device 1 moves, the self-position in the coordinate space is determined by comparing the three-dimensional point cloud data sequentially acquired by the depth sensor with the three-dimensional point cloud data already mapped to the world coordinates. Can be identified.

  By setting a base point such as an AR marker in the field of view of the camera, the world coordinates can be easily associated with the local coordinates that are the coordinate system of the object displayed on the display by the inspection device 1. When the position of the AR marker is recognized by the camera, the inspection device 1 displays the object by matching the AR marker with the origin of the local coordinates. Thereby, an object can be arranged at a predetermined position in the real space. The AR marker can include information defining the axial direction and the like of the local coordinates in addition to the origin of the local coordinates.

  The operation of each processing unit included in the inspection apparatus 1 according to the first embodiment will be described with reference to the flowchart of FIG.

  The inspection position identification unit 11 recognizes the coordinates of the surface of the structure specified by the inspector based on the movement of the body (S11). The inspection position identification unit 11 can detect a part (for example, a fingertip) of the user's body by analyzing the image data or the three-dimensional point cloud data acquired by the camera or the depth sensor of the inspection device 1. Since the detection of a body including a fingertip can be realized using various known techniques, a detailed description is omitted in this document. In addition, the inspection position identification unit 11 identifies, as the inspection position, coordinates of a projection point obtained by projecting a point corresponding to a fingertip on the surface of the structure from the viewpoint of the camera position.

  The trajectory acquisition unit 12 can acquire time-series data, that is, a plurality of temporally consecutive inspection positions, by continuously obtaining inspection positions at predetermined intervals over a predetermined time. This time-series data is called a trajectory. For example, if the inspector moves the air with the fingertip and traces the surface of the structure while pointing at the surface of the structure with the fingertip, time-series data of the indicated coordinates, more precisely Time-series data of points obtained by projecting on the surface is acquired as a trajectory (S12).

  At this time, the trajectory acquisition unit 12 can start or end acquisition of the inspection position by detecting a predetermined trigger. For example, acquisition of the inspection position may be started or ended by a voice command. Alternatively, acquisition of the inspection position may be started or ended in accordance with a special movement of the fingertip (stationary time, drawing a specific shape, bending and extending the fingertip, etc.).

  The output unit 13 outputs the trajectory acquired by the trajectory acquisition unit 12 (S13). The trajectory is a set of time-series three-dimensional coordinate data. The output trajectory may be stored, for example, in a storage area (not shown) in the inspection device 1 or may be transmitted to an external device via the communication function of the inspection device 1.

  By using the inspection device 1 of the first embodiment, an inspector can easily electronically mark a deformed portion of a structure. The inspector wears the inspection device 1 such as HoloLens, scans the tunnel lining surface and the like, and performs an operation of tracing a deformed portion such as a crack on the wall surface, for example, a crack with a fingertip in the field of view of the camera. The trajectory obtained in this way directly reflects the three-dimensional coordinates and shape (including information such as size and angle) of the crack. At this time, it is not necessary for the fingertip to directly touch the deformed portion, and by performing an operation of tracing the deformed portion in the air, the coordinates of the wall surface on the extension of the fingertip can be obtained.

  According to the first embodiment, since the marking is not performed directly on the deformed portion, it is not always necessary to approach the deformed portion. Hands-free work is also possible. Conventionally, it was necessary to transfer information, such as sketching the marking position, bringing the sketch back and converting it into data.In this embodiment, however, the shape and coordinates of the deformed portion can be directly converted into data, which improves efficiency. Excellent in accuracy and accuracy. Further, according to the present embodiment, even in a place where GPS radio waves do not reach, such as in a tunnel, or in a place where it is difficult to install a laser scanner, a total station, or the like, position information of a deformed part can be obtained.

  As an application example of the first embodiment, a locus can be projected again on the transmissive display of the inspection apparatus 1 at a site where the locus has been acquired in the past. Thereby, it is possible to compare the current deformed part with the past deformed part.

<Embodiment 2>
The second embodiment is characterized in that the inspection position specifying unit 11 recognizes the coordinates of the surface of the structure ahead of the line of sight by tracking the line of sight of the inspector. Other configurations are the same as those of the first embodiment. Hereinafter, the description will focus on the differences from the first embodiment.

  The inspection apparatus 1 according to the second embodiment includes an eye camera for tracking the line of sight of the inspector. The eye camera is mounted at a position where the examiner can catch his eyes when wearing the HMD, typically inside the display. The eye camera may be capable of capturing visible light or capable of capturing infrared light emitted to the eye.

  The operation of each processing unit included in the inspection apparatus 1 according to the second embodiment will be described with reference to the flowchart in FIG.

  The inspection position identification unit 11 recognizes the coordinates of the surface of the structure specified by the inspector based on the movement of the line of sight based on the image captured by the eye camera (S21). For example, the inspection position identification unit 11 continuously captures the eye of the inspector with an eye camera capable of capturing visible light, and recognizes the inner eye and the iris included in the captured image. Then, the gaze direction is detected based on the relative position between the inner corner of the eye and the iris. Alternatively, the inspection position identification unit 11 performs imaging with an infrared camera while irradiating the examiner's eyes with infrared light, and recognizes corneal reflection and pupils included in the captured image. Then, the gaze direction is detected based on the relative position between the corneal reflection and the pupil. When the gaze direction is detected, the inspection position identification unit 11 identifies a projection point (coordinate of the surface of the structure) in the gaze direction with the camera position as a viewpoint as the inspection position.

  The trajectory obtaining unit 12 obtains the time series data of the inspection position, that is, the trajectory by continuously obtaining the inspection position at a predetermined interval for a predetermined time. For example, if the inspector moves his / her gaze and traces the surface of the structure, the time-series data of the viewpoint of the inspector, more precisely, the time-series data of points obtained by projecting the gaze onto the surface of the structure Is acquired as a trajectory (S22).

  At this time, the trajectory acquisition unit 12 can start or end acquisition of the inspection position by detecting a predetermined trigger. For example, acquisition of the inspection position may be started or ended by a voice command. Alternatively, acquisition of the inspection position may be started or ended in response to a special movement of the eye (eg, staring at a certain point for a certain period of time, placing a viewpoint on an icon or the like displayed on a display, or blinking).

  The output unit 13 outputs the trajectory acquired by the trajectory acquisition unit 12 as in the first embodiment (S23).

  According to the second embodiment, as in the first embodiment, the inspector does not need to approach the deformed portion, can perform hands-free work, and can directly convert the shape and coordinates of the deformed portion into data. It can be used in places where it is difficult to use GPS or various measuring devices. In addition, in the second embodiment, since the deformed portion can be marked electronically without using a hand, the inspector can efficiently perform the marking while performing other work.

  It should be noted that the present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the gist. For example, the method of mounting the various processing units described in the above embodiments by software or hardware is merely an example, and may be replaced by another method. For example, in the embodiment, a plurality of processing units which are mounted in the same housing can be provided as a plurality of different hardware. Alternatively, a plurality of processing units which are implemented in different hardware in the embodiment can be implemented in one hardware. Further, any processing means or some of the functions of the processing means may be realized by a technology such as cloud computing, edge computing, fog computing, or the like.

  Further, the calculation method, the sensing method, and the like shown in the above-described embodiment are merely examples, and may be replaced by another method having an equivalent function.

  In the above-described embodiment, the output unit 13 outputs the trajectory acquired by the trajectory acquisition unit 12, that is, the time-series inspection position. However, the present invention is not limited to this, and the output unit 13 may simply output the inspection position specified by the inspection position specifying unit 11.

  Each processing unit constituting the present invention may be configured by hardware, or may be realized by causing a CPU to execute a computer program to perform an arbitrary process. Also, computer programs can be stored and provided to computers using various types of temporary or non-transitory computer-readable media. Transitory computer readable media includes, for example, electromagnetic signals provided to a computer by wire or wirelessly.

DESCRIPTION OF SYMBOLS 1 Inspection apparatus 11 Inspection position specifying part 12 Trajectory acquisition part 13 Output part

Claims (5)

A device capable of recognizing the three-dimensional shape of the surrounding structure and the movement of the examiner's body,
An inspection position identification unit that identifies the coordinates of the surface of the structure specified without contacting the structure by the movement of the inspector's fingertip, as an inspection position,
A trajectory acquisition unit that acquires time-series data composed of a plurality of inspection positions that are temporally continuous, as a trajectory,
An output unit that outputs the trajectory,
The trajectory indicates a shape of a deformed portion of the structure,
The inspection device, wherein the output unit projects the trajectory acquired in the past on a transmission-type display by superimposing the trajectory on a deformed portion of the structure. A device capable of recognizing the three-dimensional shape of the surrounding structure and the movement of the examiner's body,
By the movement of the line of sight of the inspector, an inspection position specifying unit that specifies coordinates of the surface of the structure specified without contacting the structure as an inspection position,
A trajectory acquisition unit that acquires time-series data composed of a plurality of inspection positions that are temporally continuous, as a trajectory,
An output unit that outputs the trajectory,
The trajectory indicates a shape of a deformed portion of the structure,
The inspection device, wherein the output unit projects the trajectory acquired in the past on a transmission-type display by superimposing the trajectory on a deformed portion of the structure. Indicating the coordinates of the structure surface without touching the structure by the movement of the inspector's fingertip;
Identifying the coordinates as an inspection position;
Acquiring time-series data consisting of a plurality of temporally consecutive inspection positions, as a trajectory,
Outputting the trajectory,
The trajectory indicates a shape of a deformed portion of the structure,
Test method in the step of the output, which is superimposed on the Deformation portion of said structure, characterized in that the trajectory acquired in the past is projected on the transmission type display. Indicating the coordinates of the surface of the structure without touching the structure by the movement of the line of sight of the inspector;
Identifying the coordinates as an inspection position;
Acquiring time-series data consisting of a plurality of temporally consecutive inspection positions, as a trajectory,
Outputting the trajectory,
The trajectory indicates a shape of a deformed portion of the structure,
The inspection method, wherein, in the outputting step, the trajectory acquired in the past is projected on a transmissive display so as to be superimposed on a deformed portion of the structure.   A program for causing a computer to execute the method according to claim 3.

Images