JP6033209B2 - Portable terminal device - Google Patents

Description

  This invention is, for example, when performing repair inspection or partial replacement work of equipment such as water and sewage pipes, electric wires, communication cables, etc., which are invisible objects that are embedded in the road and cannot be directly viewed, The present invention relates to a portable terminal device that visualizes as a moving image of a real space.

  Conventionally, when excavating a road for road improvement work or repair work for underground burials at sites where water supply pipes, gas pipes, optical fiber cables, etc., which are underground buried under the roads, are invisible Image acquisition means for acquiring a real space image as an image composition apparatus, an image composition program, and an image composition system capable of intuitively confirming an underground buried object and further ensuring appropriate display position accuracy of a virtual image And a measuring means for measuring the position and orientation of the image acquiring means, a storage means for storing a space model, and an image creation for creating a virtual space image so as to obtain a perspective view that is processed and displayed on a display screen. And a display means for displaying a real space image and a virtual space image displayed in perspective in a superimposed manner on a display screen (for example, Patent Document 1).

JP 2012-133471 A

  However, the technique disclosed in Patent Document 1 displays a virtual image obtained by virtually visualizing an underground object that is an invisible object on a moving image in a real space. In order to prevent deviation due to measurement error, the virtual image is corrected by shifting it back and forth, left and right, but correction in the depth direction is impossible, for example, in repair and inspection work for water leaks in water pipes, To what extent the road should be dug, there is a problem that the technique disclosed in Patent Document 1 does not enable an appropriate response.

  The present invention has been made to solve the above-described problems, and when performing repair inspection and partial replacement work of invisible object facilities embedded in the road, the invisible object facilities are converted into an image of real space. An object of the present invention is to provide a portable terminal device that can improve the efficiency and labor saving of correction work in the case of superimposed display.

  The present invention is a portable terminal device, a position information acquisition unit that acquires information on latitude, longitude, and altitude of a current position, which is first position information of a portable terminal device held by a user, and a portable terminal Shooting that captures an image of a predetermined area including a posture information acquisition unit that acquires the elevation angle, azimuth angle, and rotation angle of the current position, which is second position information of the apparatus, and a plurality of marks provided on a member that covers an invisible object And the position information of the mark, the reference plane is set, the holding state information of the portable terminal device obtained from the first position information and the second position information, and the angle of view of the photographing unit A calculation unit that calculates a plane coordinate of a region displayed on the moving image display unit, a storage unit that stores data of a reference plane, and a facility that acquires facility information of an invisible object in the plane coordinate from the facility information database Information acquisition unit and By touching a virtual image generation unit that generates a virtual image based on information, a moving image display unit that displays an image of a predetermined area as a correction information input screen, and a correction object displayed on the correction information input screen A superimposition correction unit that inputs the obtained data and performs map conversion to acquire the correction amount from the reference plane, and the virtual image for superimposition by reflecting the correction amount on the virtual image in the data of the correction object in the equipment information Is generated by the virtual image generating unit, and an image superimposing unit that generates an image of the real space by superimposing the reference point of the superimposing virtual image and the reference point displayed on the moving image display unit is provided. It is a thing.

  Since the portable terminal device according to the present invention has the above-described configuration, the position of an invisible object that cannot be directly seen from the outside, such as a water and sewage pipe, a communication cable, or a reinforcing bar of a reinforced concrete equipment, embedded in a road. Can be displayed on the moving image display unit as an image of the real space, and it becomes a bird's-eye view image. Can be accurately grasped, setting repair methods, improving the accuracy of estimation of work schedules, improving the efficiency of on-site work, and thereby reducing costs.

1 is a block diagram showing a portable terminal device according to Embodiment 1. FIG. It is a figure which shows the floating state of the waterworks pipe by Embodiment 1. FIG. 4 is a diagram showing a moving image display unit of the portable terminal device according to Embodiment 1. FIG. 3 is a flowchart showing operation steps according to the first embodiment. 6 is a diagram illustrating a virtual screen according to Embodiment 1. FIG.

Embodiment 1 FIG.
Hereinafter, the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a portable terminal device (hereinafter referred to as a terminal) 100 for supporting field work such as road construction according to the first embodiment. In FIG. In the repair work when the water pipe 32, which is an invisible material buried in the building, is partially damaged due to a natural disaster such as an earthquake and a water leakage phenomenon occurs, a field engineer (hereinafter referred to as a user) The displacement from the position at the time of embedding is compared by comparing the holding state information obtained from the first and second position information photographed by 100 with the equipment information database 1 storing the position information at the time of embedding the water pipe 32. Is specified.
An example of the site where this terminal 100 is used is shown in FIG. FIG. 2 shows a state where the water pipe 32 is lifted in the direction of the ground surface due to the ground liquefaction phenomenon due to the occurrence of an earthquake. It is in a state of being lifted together with the manhole 31 and the water pipe 32 due to the chemical phenomenon. The road surface 33 corresponding to the member covering the invisible object is provided with marks (signs) 34 drawn with paint or the like at a plurality of three or more places, and this mark 34 is used to determine a reference plane. The The user 30 holds and uses the terminal 100, and uses the terminal 100 to photograph the road surface 33 within the photographing angle θ within the predetermined area including the mark 34 as shown in FIG. Is performed and the displacement of the water supply pipe 32 is specified.

  Each element which comprises the terminal 100 is demonstrated based on FIG. The location information acquisition unit 2 has a function that enables three-point surveying using radio waves from a GPS (Global Positioning System), a wireless LAN, or a mobile phone base station built in the terminal 100. Get the latitude, longitude and altitude information of the location. The facility information acquisition unit 3 stores facility information at the time of embedding the water supply pipe 32 at the position using the position information acquisition unit 2 and the holding state information acquired by the posture information acquisition unit 4 described later as a key. The facility information 12 is acquired from the facility information database 1. At this time, the facility information acquisition unit 3 is information regarding the facility embedded in the region designated by the coordinates of three or more points, that is, the region where the mark 34 is provided, and in this case, the water pipe in the region Information such as 32 laying start points, end points, pipe dimensions, material, and depth from the road surface 33 is acquired. The posture information acquisition unit 4 acquires the elevation angle, the azimuth angle, and the rotation angle of the terminal 100 using a geomagnetic sensor, an acceleration sensor, and the like built in the terminal 100. The virtual image generation unit 5 generates a virtual image based on the facility information 12. The superimposing correction unit 6 is an image superimposing unit 7 to be described later. When a disguise image and a moving image in the real space are superimposed, there is a case where the superimposition angle, the azimuth angle, and the rotation angle may not be correctly superimposed. And a function that enables the input of a correction amount via the moving image display unit 9. The imaging unit 8 is a camera provided in the terminal 100 and performs still image and moving image imaging. A moving image display unit 9 shown in FIG. 3 displays a moving image captured by the imaging unit 8 and a moving image in which a real space and a virtual image are superimposed. Note that the photographing unit 8 is provided on the back surface of the terminal 100. The calculation unit 10 performs various calculations, and the result is stored in the storage unit 11.

Next, the field work support operation using the terminal 100 will be described based on the next operation ST and FIG.
ST0. A step of entering a plurality of three or more places on the road surface 33 by a user 30 with a mark 34 as a reference point when setting a reference plane to be described later.
ST1. A step of acquiring the latitude, longitude, and altitude information of the current position, which is the first position information of the terminal 100 held by the user 30, by the position information acquisition unit 2.
ST2. A step of acquiring information of elevation angle, azimuth angle, and rotation angle, which is second position information of the terminal 100, by the posture information acquisition unit 4.
ST3. The image capturing unit 8 captures an image of a predetermined area including a plurality of marks 34 written on the road surface 33, and the arithmetic unit 10 that inputs the position information of the marks 34 sets a reference plane, and the data of the reference plane is Steps stored in the storage unit 11.
ST4. The calculation unit 10 displays on the moving image display unit 9 from the holding state information of the terminal 100 obtained from the first position information and the second position information acquired in ST1 and ST2 and the angle of view of the photographing unit 8. Calculating the plane coordinates of the region to be processed.
ST5. The equipment information acquisition unit 3 obtains equipment information 12 such as the installation start point, end point, pipe dimensions, and depth from the road surface 33 of the water pipe below the road surface 33 in the plane coordinates obtained in ST4. Step to get from.
ST6. The virtual image generation unit 5 generates a virtual image based on the facility information 12 of ST5.
ST7. A step of displaying an image of a predetermined area within the photographing angle θ shown in FIG. 2 taken by the user 30 on the moving image display unit 9 as a correction information input screen as shown in FIG.
ST8. It is displayed on the correction information input screen in ST7, and is set in ST3 by touching the part from the road surface 33, which is a target for correction, to the manhole cover 31a that is lifted by a finger or other means. The step of obtaining the floating amount of the manhole cover 31a from the reference plane, and the superimposing correction unit 6 that inputs this value converts it into data by mapping conversion and acquires the correction amount.
ST9. The correction amount obtained in ST8 is reflected in the virtual image created in ST6 in the embedded depth data of the water supply pipe 32 in the facility information 12 acquired in ST5, and the virtual image generator 5 performs the virtual for superimposition. Generating an image;
ST10. The image superimposing unit 7 superimposes the manhole 31 which is the reference point of the virtual image for superimposition generated in ST9 and the manhole 31 which is the reference point displayed on the moving image display unit 9, thereby moving the moving image in the real space. Step to image.

  As described above, according to the first embodiment, the user 30 installs the mark 34 on the road surface 33 and operates on the moving image display unit 9 using the terminal 100, so that the road of the water pipe 32 embedded therein. The depth direction from the surface 33 can be easily corrected, and can be visually recognized on the moving image display unit 9 in ST11. For example, the depth displacement from the initial laying of the water supply pipe 32 which is an underground buried object after an earthquake occurs This makes it possible to accurately grasp the field work, improve the efficiency of on-site work such as setting the work method, estimating the work period, and the like, leading to cost reduction.

Embodiment 2. FIG.
The terminal 100 according to the first embodiment described above is capable of three-point surveying using a GPS built in the terminal 100, a radio wave from a wireless LAN, or a mobile phone station. In the second embodiment, in addition to GPS that can be used on a smartphone or tablet, a position acquired by a positioning method such as GDPS (Differential GPS) or RTK-GPS (Real Time Kinetic-GPS) that is realized by a GPS dedicated terminal. It is good also as a structure which utilizes information as an input of the facility information acquisition part 3. FIG. As a result, more accurate alignment is possible, and image superimposition is more accurate.

Embodiment 3 FIG.
In the first embodiment, the depth information of the position where the water pipe 32, which is an invisible object, is buried is corrected and displayed on the moving image display unit 9 of the terminal 100. In the third embodiment, ST10 described above is used. The facility information acquisition unit 3 for inputting the image information of the moving image display unit 9 obtained in (1) has a function of storing it in the facility information database 1 as the facility information 12, that is, a function of rewriting. In this case, the facility information database 1 may be provided in the terminal 100 or may be located at a location accessible via a communication line.
As described above, according to the third embodiment, the position information of the water pipe 32 whose depth has been corrected is written back to the facility information database 1, so that the latest information is stored and the work efficiency at the time of the next construction and the like is improved. .

Embodiment 4 FIG.
In the first embodiment, the terminal 100 is a so-called portable type as shown in FIG. 2, but in this fourth embodiment, the element function provided in the terminal 100 is divided into two, and one is stored in a glasses type. The rest is stored in a so-called portable type as shown in FIG. That is, among the components shown in FIG. 1, the position information acquisition unit 2, the posture information acquisition unit 4, the imaging unit 8, and the moving image display unit 9 are housed in an integrated eyeglass-type terminal device to acquire facility information. The unit 3, the virtual image generation unit 5, the superimposition correction unit 6, the image superposition unit 7, the calculation unit 10, and the storage unit 11 are housed in a portable terminal device as shown in FIG. 2. In this case, the facility information acquisition unit 3, virtual image generation unit 5, superimposition correction unit 6, image superimposition unit 7, calculation unit 10, and storage unit 11 provided in the portable terminal device are not necessarily arranged on the work site. For example, a system that is arranged in an office adjacent to the work site or in a city hall and can be accessed by communication means may be adopted.
As described above, the two-divided structure improves the possibility, and there is an effect that the user 30 does not need to hold the terminal 100 and the working efficiency is increased.

Embodiment 5. FIG.
In the first embodiment, the case where the mark (sign) 34 is drawn on the road surface 33 with paint or the like has been described. A reference plane may be set by using an object. This improves the convenience of the terminal 100.

Embodiment 6 FIG.
In ST5 of the first embodiment described above, the facility information 12 regarding the water supply pipe 32 acquired by the facility information acquisition unit 3 is the starting point, the end point, the dimension of the pipe, the depth from the road surface 33, and the like. However, in the sixth embodiment, instead of the acquired facility information, the facility drawing stored in the facility information database 1 is used. That is, in addition to the equipment information database 1, equipment drawings that are equipment information of the invisible material are stored, and the user 30 operates the terminal 100 to operate the equipment information database via the equipment information acquisition unit 3. 1, the equipment drawing in the plane coordinates described above is acquired as equipment information 12, and the virtual image generation unit 5 continues to use the equipment drawing obtained in place of the equipment information 12 of ST7 described in the first embodiment. A virtual image is generated. That is, in the field work steps in the first embodiment described above, ST5 and ST6 are changed to the following ST5a and ST6a, and the other steps are the same as in the first embodiment.
ST5a. The facility information acquisition unit 3 acquires from the facility information database 1 as facility information 12 a facility drawing of an invisible object in the plane coordinates obtained in ST4.
ST6a. A step of generating the equipment drawing of ST5a as a virtual screen by the virtual image generation unit 5.
As described above, since the facility drawing stored in the facility information database 1 is acquired as the facility information 12 in the sixth embodiment, the moving image in the real space obtained in ST10 is described in the facility drawing. Other than the water supply pipe 32, other buried objects provided adjacent to the water supply pipe 32 are also displayed, and there is an effect that it is possible to secure a safer habitation when performing repair work. In addition, although the equipment drawing showed the example stored in the said equipment information database 1, you may acquire from the equipment drawing database provided separately.

  In the first to sixth embodiments, the water pipe 32 installed under the road is invisible, but is not limited thereto, and may be a reinforced concrete reinforcing bar or the like in a figurine or the like.

  It should be noted that within the scope of the present invention, the embodiments can be freely combined, or the embodiments can be appropriately modified or omitted.

1 equipment information database, 2 position information acquisition unit, 3 equipment information acquisition unit,
4 posture information acquisition unit, 5 virtual image generation unit, 6 superimposition correction unit, 7 image superimposition unit,
8 shooting section, 9 moving image display section, 10 calculation section, 12 equipment information, 31 manhole,
31a Manhole cover, 32 Water pipe, 33 Road surface, 34 mark, 100 terminal.

Claims (7)

A portable terminal device, a position information acquisition unit that acquires information on latitude, longitude, and altitude of the current position, which is first position information of the portable terminal device held by a user, and the portable terminal device A posture information acquisition unit that acquires the elevation angle, the azimuth angle, and the rotation angle of the current position, which is the second position information, and an imaging unit that captures an image of a predetermined area including a plurality of marks provided on a member that covers an invisible object; The position information of the mark is input, the reference plane is set, the holding state information of the portable terminal device obtained from the first position information and the second position information, and the image of the photographing unit A calculation unit that calculates a plane coordinate of a region displayed on the moving image display unit from a corner, a storage unit that stores data of the reference plane, and facility information of the invisible object in the plane coordinate From information database A facility information acquisition unit to be obtained; a virtual image generation unit that generates a virtual image based on the facility information; the moving image display unit that displays an image of the predetermined area as a correction information input screen; and the correction information input screen. A superimposition correction unit that receives and converts the data obtained by touching the displayed correction target object to obtain a correction amount from the reference plane, and data of the correction target object in the facility information In addition, a virtual image for superimposition is generated by the virtual image generation unit reflecting the correction amount in the virtual image, and a reference point of the virtual image for superimposition and a reference point displayed on the moving image display unit And an image superimposing unit that generates an image of the real space by superimposing and. The portable terminal device according to claim 1, wherein the facility information acquisition unit acquires a facility drawing of the invisible object in the plane coordinates as the facility information from the facility information database. The acquisition of the first position information and the second position information is performed by a function built in the portable terminal device that enables three-point survey using radio waves from GPS. The portable terminal device according to claim 1 or 2. The acquisition of said 1st position information and said 2nd position information inputs what was acquired by the positioning method of either GDPS or RTK-GPS, The Claim 1 or Claim 2 characterized by the above-mentioned. Portable terminal device. The image of the real space which the said image superimposition part produces | generates is stored in the said equipment information database via the said equipment information acquisition part, The any one of Claims 1-4 characterized by the above-mentioned. Portable terminal device. The position information acquisition unit, the posture information acquisition unit, the photographing unit, and the moving image display unit provided in the portable terminal device are integrated and housed in the eyeglass-type terminal device, and the remaining equipment The information acquisition unit, the virtual image generation unit, the superimposition correction unit, the image superimposition unit, the calculation unit, and the storage unit are housed in a portable terminal device. The portable terminal device according to any one of the above. The portable terminal device according to any one of claims 1 to 6, wherein the plurality of marks photographed by the photographing unit are existing facilities that cover the invisible object.

Images