JP6230678B2 - Terminal device, terminal device program, and invisible object displacement state visualization method - Google Patents

Description

  This invention is, for example, when performing repair inspection or partial replacement work of equipment such as water and sewage pipes, electric wires, communication cables, etc., which are invisible objects that are embedded in the road and cannot be directly viewed, The present invention relates to a terminal device, a terminal device program, and a method for visualizing a displacement state of an invisible object.

  Conventionally, when excavating a road for road improvement work or repair work for underground burials at sites where water supply pipes, gas pipes, optical fiber cables, etc., which are underground buried under the roads, are invisible Image acquisition means for acquiring a real space image as an image composition apparatus, an image composition program, and an image composition system capable of intuitively confirming an underground buried object and further ensuring appropriate display position accuracy of a virtual image And a measuring means for measuring the position and orientation of the image acquiring means, a storage means for storing a space model, and an image creation for creating a virtual space image so as to obtain a perspective view that is processed and displayed on a display screen. And a display means for displaying a real space image and a virtual space image displayed in perspective in a superimposed manner on a display screen (for example, Patent Document 1).

JP 2012-133471 A

  However, the technique disclosed in Patent Document 1 displays a virtual image obtained by virtually visualizing an underground object that is an invisible object on a moving image in a real space. In order to prevent deviation due to measurement error, the virtual image is corrected by shifting it back and forth, left and right, but correction in the depth direction is impossible, for example, in repair and inspection work for water leaks in water pipes, To what extent the road should be dug, there is a problem that the technique disclosed in Patent Document 1 does not enable an appropriate response.

  The present invention has been made to solve the above-described problems, and when performing repair inspection and partial replacement work of invisible object facilities embedded in the road, the invisible object facilities are converted into an image of real space. An object of the present invention is to provide a terminal device, a program for a terminal device, and a method for visualizing a displacement state of an invisible object that can make correction work more efficient and labor-saving in the case of superimposed display.

1st invention is a terminal device,
A location information acquisition unit for acquiring location information of the terminal device;
An attitude information acquisition unit for acquiring information on the attitude of the terminal device;
An area displayed on a display unit provided in the terminal device based on the position information and the posture information, by inputting position information of a mark provided on a member covering an invisible object, setting a reference plane A calculation unit for calculating the plane coordinates of
A facility information acquisition unit for acquiring facility information of the invisible object in the plane coordinates from a facility information database;
A virtual image generation unit that generates a virtual image based on the facility information;
A superimposition correction unit that acquires a displacement from the reference plane by an operation on a correction target displayed on the display unit on which an image of a predetermined area photographed by the terminal device is displayed;
By reflecting the displacement on the virtual image, a reference point of the virtual image for superimposition generated by the virtual image generation unit and a reference point displayed on the display unit are superimposed to generate an image of the real space And an image superimposing unit to be provided.
A second invention is a terminal device,
A location information acquisition unit for acquiring location information of the terminal device;
An attitude information acquisition unit for acquiring information on the attitude of the terminal device;
The position information of the member covering the invisible object is input, the reference plane is set, and the plane coordinates of the area displayed on the display unit provided in the terminal device are calculated based on the position information and the posture information An arithmetic unit for performing
A facility information acquisition unit for acquiring facility information of the invisible object in the plane coordinates from a facility information database;
A virtual image generation unit that generates a virtual image based on the facility information;
A superimposition correction unit that acquires a displacement from the reference plane by an operation on a correction target displayed on the display unit on which an image of a predetermined area photographed by the terminal device is displayed;
By reflecting the displacement on the virtual image, a reference point of the virtual image for superimposition generated by the virtual image generation unit and a reference point displayed on the display unit are superimposed to generate an image of the real space And an image superimposing unit to be provided.
A third invention is a program for causing a terminal device to visualize a displacement state of an invisible object,
Obtaining information on the position and orientation of the terminal device;
Input the position information of the member covering the invisible object, set the reference plane, and calculate the plane coordinates of the area displayed on the display unit provided in the terminal device based on the position information and the posture information The steps of
Obtaining facility information of the invisible object in the plane coordinates from a facility information database;
Generating a virtual image based on the facility information;
Displaying an image of a predetermined area photographed by the terminal device;
Obtaining a displacement from the reference plane by an operation on the correction object displayed on the display unit;
Generating a real space image by superimposing a reference point of the virtual image for superimposition generated by reflecting the displacement on the virtual image and a reference point displayed on the display unit; Is.
4th invention acquires the information of the position and attitude | position of a terminal device,
Input the position information of the member covering the invisible object, set the reference plane, and calculate the plane coordinates of the area displayed on the display unit provided in the terminal device based on the position information and the posture information The steps of
Obtaining facility information of the invisible object in the plane coordinates from a facility information database;
Generating a virtual image based on the facility information;
Displaying an image of a predetermined area photographed by the terminal device;
Obtaining a displacement from the reference plane by an operation on the correction object displayed on the display unit;
Generating a real space image by superimposing a reference point of the superimposing virtual image generated by reflecting the displacement on the virtual image and a reference point displayed on the display unit. Is.

  Since the terminal device according to the first, second, and third inventions has the above-described configuration and program, and the fourth invention is the above-described method, it is embedded in a road, for example The position of invisible objects that cannot be seen directly from the outside, such as sewer pipes, communication cables, or reinforcing bars of reinforced concrete equipment, can be displayed on the video display as a real space image, resulting in a bird's-eye view, such as liquefaction due to the occurrence of natural disasters such as earthquakes In the repair work of the rise of the water supply pipe due to the phenomenon, it is possible to accurately grasp the depth displacement from the initial installation of the water supply pipe, setting the repair method, improving the accuracy of the construction period estimate, improving the efficiency of the work on site, and reducing the cost It has the effect of being connected.

1 is a block diagram showing a portable terminal device according to Embodiment 1. FIG. It is a figure which shows the floating state of the waterworks pipe by Embodiment 1. FIG. 4 is a diagram showing a moving image display unit of the portable terminal device according to Embodiment 1. FIG. 3 is a flowchart showing operation steps according to the first embodiment. 6 is a diagram illustrating a virtual screen according to Embodiment 1. FIG.

Embodiment 1 FIG.
Hereinafter, the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a portable terminal device (hereinafter referred to as a terminal) 100 for supporting field work such as road construction according to the first embodiment. In FIG. In the repair work when the water pipe 32, which is an invisible material buried in the building, is partially damaged due to a natural disaster such as an earthquake and a water leakage phenomenon occurs, a field engineer (hereinafter referred to as a user) The displacement from the position at the time of embedding is compared by comparing the holding state information obtained from the first and second position information photographed by 100 with the equipment information database 1 storing the position information at the time of embedding the water pipe 32. Is specified.
An example of the site where this terminal 100 is used is shown in FIG. FIG. 2 shows a state where the water pipe 32 is lifted in the direction of the ground surface due to the ground liquefaction phenomenon due to the occurrence of an earthquake. It is in a state of being lifted together with the manhole 31 and the water pipe 32 due to the chemical phenomenon. The road surface 33 corresponding to the member covering the invisible object is provided with marks (signs) 34 drawn with paint or the like at a plurality of three or more places, and this mark 34 is used to determine a reference plane. The The user 30 holds and uses the terminal 100, and uses the terminal 100 to photograph the road surface 33 within the photographing angle θ within the predetermined area including the mark 34 as shown in FIG. Is performed and the displacement of the water supply pipe 32 is specified.

  Each element which comprises the terminal 100 is demonstrated based on FIG. The location information acquisition unit 2 has a function that enables three-point surveying using radio waves from a GPS (Global Positioning System), a wireless LAN, or a mobile phone base station built in the terminal 100. Get the latitude, longitude and altitude information of the location. The facility information acquisition unit 3 stores facility information at the time of embedding the water supply pipe 32 at the position using the position information acquisition unit 2 and the holding state information acquired by the posture information acquisition unit 4 described later as a key. The facility information 12 is acquired from the facility information database 1. At this time, the facility information acquisition unit 3 is information regarding the facility embedded in the region designated by the coordinates of three or more points, that is, the region where the mark 34 is provided, and in this case, the water pipe in the region Information such as 32 laying start points, end points, pipe dimensions, material, and depth from the road surface 33 is acquired. The posture information acquisition unit 4 acquires the elevation angle, the azimuth angle, and the rotation angle of the terminal 100 using a geomagnetic sensor, an acceleration sensor, and the like built in the terminal 100. The virtual image generation unit 5 generates a virtual image based on the facility information 12. The superimposing correction unit 6 is an image superimposing unit 7 to be described later. When a disguise image and a moving image in the real space are superimposed, there is a case where the superimposition angle, the azimuth angle, and the rotation angle may not be correctly superimposed. And a function that enables the input of a correction amount via the moving image display unit 9. The imaging unit 8 is a camera provided in the terminal 100 and performs still image and moving image imaging. A moving image display unit 9 shown in FIG. 3 displays a moving image captured by the imaging unit 8 and a moving image in which a real space and a virtual image are superimposed. Note that the photographing unit 8 is provided on the back surface of the terminal 100. The calculation unit 10 performs various calculations, and the result is stored in the storage unit 11.

Next, the field work support operation using the terminal 100 will be described based on the next operation ST and FIG.
ST0. A step of entering a plurality of three or more places on the road surface 33 by a user 30 with a mark 34 as a reference point when setting a reference plane to be described later.
ST1. A step of acquiring the latitude, longitude, and altitude information of the current position, which is the first position information of the terminal 100 held by the user 30, by the position information acquisition unit 2.
ST2. A step of acquiring information of elevation angle, azimuth angle, and rotation angle, which is second position information of the terminal 100, by the posture information acquisition unit 4.
ST3. The image capturing unit 8 captures an image of a predetermined area including a plurality of marks 34 written on the road surface 33, and the arithmetic unit 10 that inputs the position information of the marks 34 sets a reference plane, and the data of the reference plane is Steps stored in the storage unit 11.
ST4. The calculation unit 10 displays on the moving image display unit 9 from the holding state information of the terminal 100 obtained from the first position information and the second position information acquired in ST1 and ST2 and the angle of view of the photographing unit 8. Calculating the plane coordinates of the region to be processed.
ST5. The equipment information acquisition unit 3 obtains equipment information 12 such as the installation start point, end point, pipe dimensions, and depth from the road surface 33 of the water pipe below the road surface 33 in the plane coordinates obtained in ST4. Step to get from.
ST6. The virtual image generation unit 5 generates a virtual image based on the facility information 12 of ST5.
ST7. A step of displaying an image of a predetermined area within the photographing angle θ shown in FIG. 2 taken by the user 30 on the moving image display unit 9 as a correction information input screen as shown in FIG.
ST8. It is displayed on the correction information input screen in ST7, and is set in ST3 by touching the part from the road surface 33, which is a target for correction, to the manhole cover 31a that is lifted by a finger or other means. The step of obtaining the floating amount of the manhole cover 31a from the reference plane, and the superimposing correction unit 6 that inputs this value converts it into data by mapping conversion and acquires the correction amount.
ST9. The correction amount obtained in ST8 is reflected in the virtual image created in ST6 in the embedded depth data of the water supply pipe 32 in the facility information 12 acquired in ST5, and the virtual image generator 5 performs the virtual for superimposition. Generating an image;
ST10. The image superimposing unit 7 superimposes the manhole 31 which is the reference point of the virtual image for superimposition generated in ST9 and the manhole 31 which is the reference point displayed on the moving image display unit 9, thereby moving the moving image in the real space. Step to image.

  As described above, according to the first embodiment, the user 30 installs the mark 34 on the road surface 33 and operates on the moving image display unit 9 using the terminal 100, so that the road of the water pipe 32 embedded therein. The depth direction from the surface 33 can be easily corrected, and can be visually recognized on the moving image display unit 9 in ST10. For example, the depth displacement from the initial laying of the water supply pipe 32 which is an underground buried object after an earthquake occurs This makes it possible to accurately grasp the field work, improve the efficiency of on-site work such as setting the work method, estimating the work period, and the like, leading to cost reduction.

Embodiment 2. FIG.
The terminal 100 according to the first embodiment described above is capable of three-point surveying using a GPS built in the terminal 100, a radio wave from a wireless LAN, or a mobile phone station. In the second embodiment, in addition to GPS that can be used on a smartphone or tablet, a position acquired by a positioning method such as GDPS (Differential GPS) or RTK-GPS (Real Time Kinetic-GPS) that is realized by a GPS dedicated terminal. It is good also as a structure which utilizes information as an input of the facility information acquisition part 3. FIG. As a result, more accurate alignment is possible, and image superimposition is more accurate.

Embodiment 3 FIG.
In the first embodiment, the depth information of the position where the water pipe 32, which is an invisible object, is buried is corrected and displayed on the moving image display unit 9 of the terminal 100. In the third embodiment, ST10 described above is used. The facility information acquisition unit 3 for inputting the image information of the moving image display unit 9 obtained in (1) has a function of storing it in the facility information database 1 as the facility information 12, that is, a function of rewriting. In this case, the facility information database 1 may be provided in the terminal 100 or may be located at a location accessible via a communication line.
As described above, according to the third embodiment, the position information of the water pipe 32 whose depth has been corrected is written back to the facility information database 1, so that the latest information is stored and the work efficiency at the time of the next construction and the like is improved. .

Embodiment 4 FIG.
In the first embodiment, the terminal 100 is a so-called portable type as shown in FIG. 2, but in this fourth embodiment, the element function provided in the terminal 100 is divided into two, and one is stored in a glasses type. The rest is stored in a so-called portable type as shown in FIG. That is, among the components shown in FIG. 1, the position information acquisition unit 2, the posture information acquisition unit 4, the imaging unit 8, and the moving image display unit 9 are housed in an integrated eyeglass-type terminal device to acquire facility information. The unit 3, the virtual image generation unit 5, the superimposition correction unit 6, the image superposition unit 7, the calculation unit 10, and the storage unit 11 are housed in a portable terminal device as shown in FIG. 2. In this case, the facility information acquisition unit 3, virtual image generation unit 5, superimposition correction unit 6, image superimposition unit 7, calculation unit 10, and storage unit 11 provided in the portable terminal device are not necessarily arranged on the work site. For example, a system that is arranged in an office adjacent to the work site or in a city hall and can be accessed by communication means may be adopted.
As described above, the two-divided structure improves the possibility, and there is an effect that the user 30 does not need to hold the terminal 100 and the working efficiency is increased.

Embodiment 5. FIG.
In the first embodiment, the case where the mark (sign) 34 is drawn on the road surface 33 with paint or the like has been described. A reference plane may be set by using an object. This improves the convenience of the terminal 100.

Embodiment 6 FIG.
In ST5 of the first embodiment described above, the facility information 12 regarding the water supply pipe 32 acquired by the facility information acquisition unit 3 is the starting point, the end point, the dimension of the pipe, the depth from the road surface 33, and the like. However, in the sixth embodiment, instead of the acquired facility information, the facility drawing stored in the facility information database 1 is used. That is, in addition to the equipment information database 1, equipment drawings that are equipment information of the invisible material are stored, and the user 30 operates the terminal 100 to operate the equipment information database via the equipment information acquisition unit 3. 1, the equipment drawing in the plane coordinates described above is acquired as equipment information 12, and the virtual image generation unit 5 continues to use the equipment drawing obtained in place of the equipment information 12 of ST7 described in the first embodiment. A virtual image is generated. That is, in the field work steps in the first embodiment described above, ST5 and ST6 are changed to the following ST5a and ST6a, and the other steps are the same as in the first embodiment.
ST5a. The facility information acquisition unit 3 acquires from the facility information database 1 as facility information 12 a facility drawing of an invisible object in the plane coordinates obtained in ST4.
ST6a. A step of generating the equipment drawing of ST5a as a virtual screen by the virtual image generation unit 5.
As described above, since the facility drawing stored in the facility information database 1 is acquired as the facility information 12 in the sixth embodiment, the moving image in the real space obtained in ST10 is described in the facility drawing. Other than the water supply pipe 32, other buried objects provided adjacent to the water supply pipe 32 are also displayed, and there is an effect that it is possible to secure a safer habitation when performing repair work. In addition, although the equipment drawing showed the example stored in the said equipment information database 1, you may acquire from the equipment drawing database provided separately.

  In the first to sixth embodiments, the water pipe 32 installed under the road is invisible, but is not limited thereto, and may be a reinforced concrete reinforcing bar or the like in a figurine or the like.

  It should be noted that within the scope of the present invention, the embodiments can be freely combined, or the embodiments can be appropriately modified or omitted.

1 equipment information database, 2 position information acquisition unit, 3 equipment information acquisition unit,
4 posture information acquisition unit, 5 virtual image generation unit, 6 superimposition correction unit, 7 image superimposition unit,
8 shooting section, 9 moving image display section, 10 calculation section, 12 equipment information, 31 manhole,
31a Manhole cover, 32 Water pipe, 33 Road surface, 34 mark, 100 terminal.

Claims (8)

A terminal device,
A location information acquisition unit for acquiring location information of the terminal device;
An attitude information acquisition unit for acquiring information on the attitude of the terminal device;
An area displayed on a display unit provided in the terminal device based on the position information and the posture information, by inputting position information of a mark provided on a member covering an invisible object, setting a reference plane A calculation unit for calculating the plane coordinates of
A facility information acquisition unit for acquiring facility information of the invisible object in the plane coordinates from a facility information database;
A virtual image generation unit that generates a virtual image based on the facility information;
A superimposition correction unit that acquires a displacement from the reference plane by an operation on a correction target displayed on the display unit on which an image of a predetermined area photographed by the terminal device is displayed;
By reflecting the displacement on the virtual image, a reference point of the virtual image for superimposition generated by the virtual image generation unit and a reference point displayed on the display unit are superimposed to generate an image of the real space A terminal device provided with an image superimposing unit. A terminal device,
A location information acquisition unit for acquiring location information of the terminal device;
An attitude information acquisition unit for acquiring information on the attitude of the terminal device;
The position information of the member covering the invisible object is input, the reference plane is set, and the plane coordinates of the area displayed on the display unit provided in the terminal device are calculated based on the position information and the posture information An arithmetic unit for performing
A facility information acquisition unit for acquiring facility information of the invisible object in the plane coordinates from a facility information database;
A virtual image generation unit that generates a virtual image based on the facility information;
A superimposition correction unit that acquires a displacement from the reference plane by an operation on a correction target displayed on the display unit on which an image of a predetermined area photographed by the terminal device is displayed;
By reflecting the displacement on the virtual image, a reference point of the virtual image for superimposition generated by the virtual image generation unit and a reference point displayed on the display unit are superimposed to generate an image of the real space A terminal device provided with an image superimposing unit. The terminal device according to claim 1, wherein an image of the real space generated by the image superimposing unit is stored in the facility information database via the facility information acquisition unit. The position information acquisition unit and the posture information acquisition unit provided in the terminal device are integrated and stored in the eyeglass-type terminal device, and the remaining facility information acquisition unit, the virtual image generation unit, and the superimposition The terminal device according to any one of claims 1 to 3, wherein the correction unit, the image superimposing unit, and the computing unit are housed in a portable terminal device. A program for causing a terminal device to visualize the displacement state of an invisible object,
Obtaining information on the position and orientation of the terminal device;
Input the position information of the member covering the invisible object, set the reference plane, and calculate the plane coordinates of the area displayed on the display unit provided in the terminal device based on the position information and the posture information The steps of
Obtaining facility information of the invisible object in the plane coordinates from a facility information database;
Generating a virtual image based on the facility information;
Displaying an image of a predetermined area photographed by the terminal device;
Obtaining a displacement from the reference plane by an operation on the correction object displayed on the display unit;
Generating a real space image by superimposing a reference point of the virtual image for superimposition generated by reflecting the displacement on the virtual image and a reference point displayed on the display unit; Terminal device program. The step of generating an image of the real space includes
The terminal device program according to claim 5, comprising a step of storing the real space image in the facility information database. Obtaining information on the position and orientation of the terminal device;
Input the position information of the member covering the invisible object, set the reference plane, and calculate the plane coordinates of the area displayed on the display unit provided in the terminal device based on the position information and the posture information The steps of
Obtaining facility information of the invisible object in the plane coordinates from a facility information database;
Generating a virtual image based on the facility information;
Displaying an image of a predetermined area photographed by the terminal device;
Obtaining a displacement from the reference plane by an operation on the correction object displayed on the display unit;
Generating a real space image by superimposing a reference point of the superimposing virtual image generated by reflecting the displacement on the virtual image and a reference point displayed on the display unit. Displacement state visualization method for invisible objects. The step of generating an image of the real space includes
The method for visualizing a displacement state of an invisible object according to claim 7, further comprising the step of storing an image of the real space in the facility information database.

Images