JP2018028507A - Underground survey device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to acquire accurate and precise data as underground survey data while effectively protecting a sensor for underground survey, and create and display the underground survey data and information on the ground surface as single centralized information.SOLUTION: An underground survey device comprises: a survey vehicle 10 that includes a plurality of wheels including front wheels and rear wheels and can travel on a road surface; a three-dimensional underground radar 20 that is arranged in the survey vehicle 10 and creates underground three-dimensional information on the underground below the road surface on which the survey vehicle 10 travels; and an omnidirectional camera 30 that creates an overground three-dimensional video over the road surface on which the survey vehicle 10 travels. The three-dimensional underground radar 20 is arranged on a bottom face of the survey vehicle 10 between the front wheels and rear wheels or between axles of the plurality of wheels.SELECTED DRAWING: Figure 1

Description

  The present invention uses the physical properties of the ground against seismic waves, electrical resistivity, electromagnetic waves, ultrasonic waves, etc., as represented by underground radar exploration, and conducts non-destructive and indirect exploration of underground conditions. -Regarding the underground exploration technology to be investigated.

In recent years, along with the aging of social infrastructure such as roads, tunnels, bridges, and harbor facilities, it is required to repair those facilities to extend their life and maintain a safe social environment. Underground exploration technology for investigating and diagnosing structures is extremely important.
In particular, there is an increase in underground and structural abnormalities due to earthquakes, abnormal weather, aging, etc. For example, it is an urgent task to accurately and promptly conduct cavity surveys such as deterioration inside concrete and under road surfaces. Yes.
For this reason, the importance of underground exploration technology is increasing.

Underground exploration, for example, in the case of hollow exploration under a paved road, prepare an exploration vehicle equipped with a sensor such as an underground radar that projects electromagnetic waves toward the ground and receives reflected waves from the ground, Record the exploration data obtained by the sensor while driving the exploration vehicle on the road in the same way as a general vehicle, and analyze the recorded exploration data, so that the presence and location of cavities, cracks, buried pipes, etc. that exist in the ground A primary survey is conducted to identify
As a result of analysis of the exploration data obtained in the primary survey, if there is a part that seems to have a cavity in the ground, for example, the surrounding area is investigated in detail, and the position, size, scale, etc. of the cavity A secondary survey is conducted to accurately grasp the situation, and the risk of depression is determined and evaluated. After that, necessary measures such as repair of cavities will be taken by road managers.

The exploration data obtained in the primary survey of such underground exploration is obtained as waveform data of vertical and horizontal sections under the road in the traveling direction of the exploration vehicle, but the underground object indicated in the data For example, it is necessary to specify the position where a cavity or a buried pipe is present from the ground surface.
In conventional underground exploration technology, based on the exploration data obtained in the primary survey, location work using a hand-held sensor, etc., and measurement work such as the shape of structures on the ground and road surroundings Etc., and such position identification work has been a heavy burden.

Here, in order to reduce the burden of the location work based on the underground exploration data in such conventional underground exploration technology, the exploration data obtained in the primary survey is converted to a map or road surface image showing the corresponding ground surface. The positions and sizes of cavities, buried pipes and the like are specified as positions in maps and road surface images by arranging and displaying them together.
As such a technique for displaying the underground exploration data together with a map showing the corresponding ground surface and a road surface image, for example, the “underground exploration method” disclosed in Patent Document 1 is proposed.

JP-A-11-352223

However, the conventional underground exploration technique disclosed in Patent Document 1 simply displays the underground exploration data obtained by the underground radar or the like along with the corresponding map or road surface image. It was difficult to accurately identify and grasp where the cavities, buried pipes, etc. existed on the ground surface.
For this reason, if a secondary survey is performed with the underground survey data only confirmed as the position in the corresponding map or road surface image, even if the ground surface is actually destroyed or excavated in the secondary survey, In such a case, the position is shifted from the cavity or the buried pipe, or in the worst case, the part where the target object such as the cavity does not exist is destroyed.

Therefore, it is extremely difficult to identify the position of cavities, buried pipes, etc. as the exact position of the ground surface simply by displaying the underground exploration data side by side with the corresponding map or road surface image and comparing it visually. It was difficult.
For this reason, more accurate and reliable output / display, etc., than just displaying the underground exploration data side by side with a map, etc. were desired, but no effective technology or solution has been proposed so far. .

In addition, in order to provide position information for underground exploration data and ground surface images, it is common to use GPS information that is provided in an exploration vehicle that performs exploration and imaging. However, the positional information obtained by GPS has an error of several meters to several tens of meters.
For this reason, in order to accurately and accurately correspond the underground exploration data to the position of the ground surface, it is not sufficient to acquire and generate position information using GPS. In the underground exploration technology that needs to be specified within the range, it has been desired to obtain more accurate and reliable position information.

Furthermore, as disclosed in Patent Document 1, in the conventional underground exploration technology, a sensor for acquiring underground exploration data such as a ground penetrating radar is protruded largely in front of or behind the exploration vehicle. In addition, it has a form (air type) for towing a loading platform equipped with a radar or the like.
In such a form, the distance from the sensor such as a ground penetrating radar to the road surface / surface is large, and the sensor cannot follow the movement or vibration of the exploration vehicle. There was a certain limit. In addition, when radar or the like is projected or towed forward or rearward of the vehicle, there is a risk of interference or collision with roadsides or obstacles. In addition, bare radar or the like may rebound foreign matter from the road surface or road surface. There was also a risk of damage due to such factors.
For this reason, further improvements have been desired in the configuration and arrangement of the exploration vehicles and sensors, but no effective solution means has been realized so far.

  The present invention has been proposed in order to solve the problems of the conventional techniques as described above, and provides highly accurate and accurate data as underground exploration data while effectively protecting the sensor for underground exploration. It is possible to obtain the location information of the ground and the ground, and by adding more accurate and high-precision location information without using GPS, the underground exploration data and the ground surface information are unified. It is an object of the present invention to provide an underground exploration device that can be generated and displayed as a single piece of information and can display and output underground exploration data integrally in an accurate ortho image showing the ground surface.

  In order to achieve the above object, an underground exploration device according to the present invention includes a plurality of wheels including a front wheel and a rear wheel, and is disposed on the exploration vehicle that can travel on a road surface, and the exploration vehicle travels. An underground exploration means for generating underground three-dimensional information under the road surface, and the underground exploration means is located between the front wheels and the rear wheels, or between the axles of the plurality of wheels on the bottom surface of the exploration vehicle. It is set as the structure arrange | positioned.

  The underground exploration device of the present invention is disposed on the exploration vehicle and generates ground 3D images on the road surface on which the exploration vehicle travels, and a ground image generated by the underground exploration means. Three-dimensional information unification processing means for generating middle three-dimensional information and three-dimensional information generated by the terrestrial image generation means as three-dimensional information unified from the ground to the ground space by a predetermined data unification process; Are provided.

  Further, in the underground exploration device of the present invention, the ground image generation means automatically extracts a predetermined number of feature points from the image data of the ground 3D image, and the extracted feature points, A feature point correspondence processing unit that automatically tracks within each frame image of a moving image and obtains a correspondence relationship between the frame images, and obtains a three-dimensional position coordinate of the feature point for which the correspondence relationship is obtained. A camera vector calculation unit that obtains a camera vector composed of the three-dimensional position coordinates and the three-dimensional rotation coordinates of the camera corresponding to each frame image, and the three-dimensional information unification processing unit includes the camera Based on the camera vector generated by the vector calculation means and the position coordinate information of the underground 3D information, the underground 3D information and the ground 3D image are integrated. There is a configuration in which reduction.

According to the present invention, it is possible to obtain highly accurate and accurate data as underground exploration data while effectively protecting the sensor for underground exploration, and the GPS is used as position information on the ground and ground. More accurate and highly accurate position information can be given.
As a result, the underground exploration data and the ground surface information are not simply displayed side by side, but the underground exploration data and the ground surface information can be generated and displayed as a single unified information. Underground exploration data indicating buried pipes and the like can be integrally displayed and output in an accurate ortho image showing the ground surface.
Therefore, according to the present invention, an unprecedented accurate and highly accurate underground exploration can be realized.

It is explanatory drawing which shows typically the whole image of underground exploration by the underground exploration apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows typically the operation | movement image of the exploration vehicle which acquires and produces | generates the underground exploration data of the underground exploration apparatus which concerns on one Embodiment of this invention, (a) is sectional drawing of the road surface which an exploration vehicle travels (B) is a plan view of the road surface on which the exploration vehicle travels. It is explanatory drawing which shows typically the positional relationship of the exploration vehicle and sensor of the underground exploration apparatus which concern on one Embodiment of this invention, (a) And (b) is a bottom view of an exploration vehicle, (c) is (a It is a top view of the road surface which shows the search range of the sensor corresponding to () and (b). It is explanatory drawing which shows typically the underground exploration data acquired and produced | generated with the underground exploration apparatus which concerns on one Embodiment of this invention, (a) is an image figure of underground 3D data acquired with a sensor, b) is an image diagram of an output image generated and output based on data acquired by a sensor. It is explanatory drawing which shows typically the imaging range of 360 degree | round | round 360-degree three-dimensional video acquired with the underground exploration apparatus which concerns on one Embodiment of this invention, (a) shows the imaging range in a road surface, a bridge, etc. (B) is front sectional drawing of a road surface which shows the imaging range of the road side slope (slope), respectively. Moreover, (c) is an example of a 360-degree all-around video. It is a flowchart which shows the procedure of the unification process of the underground exploration data and the ground image | video in the underground exploration apparatus which concerns on one Embodiment of this invention. It is an image figure of the image which unified the ground 3D data acquired by the underground exploration device concerning one embodiment of the present invention, and the ground 3D data into a ground ortho image, (a) is a subsurface cavity The data for investigation, (b) is the case of the data for investigation of steel bar corrosion of bridge floor boards. It is a block diagram which shows the basic composition of the CV (camera vector) calculating part which produces | generates the camera vector image produced | generated from the ground 360 degree | times stereoscopic image in the underground exploration apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows the conversion image obtained from a ground 360 degree | times stereoscopic image, (a) is a virtual spherical surface to which a spherical image is affixed, (b) is an example of a spherical image affixed to a virtual spherical surface, (c ) Shows an image obtained by developing the spherical image shown in (b) on a plane according to the Mercator projection. It is explanatory drawing which shows the detection method of the specific camera vector in the CV calculating part which concerns on one Embodiment of this invention. It is explanatory drawing which shows the detection method of the specific camera vector in the CV calculating part which concerns on one Embodiment of this invention. It is explanatory drawing which shows the detection method of the specific camera vector in the CV calculating part which concerns on one Embodiment of this invention. It is explanatory drawing which shows the designation | designated aspect of the desirable feature point in the detection method of the camera vector by the CV calculating part which concerns on one Embodiment of this invention. It is a graph which shows the example of the three-dimensional coordinate of the feature point obtained by the CV calculating part which concerns on one Embodiment of this invention, and a camera vector. It is a graph which shows the example of the three-dimensional coordinate of the feature point obtained by the CV calculating part which concerns on one Embodiment of this invention, and a camera vector. It is a graph which shows the example of the three-dimensional coordinate of the feature point obtained by the CV calculating part which concerns on one Embodiment of this invention, and a camera vector. In the CV calculating part which concerns on one Embodiment of this invention, it is explanatory drawing which shows the case where a some feature point is set according to the distance of a feature point from a camera, and a some calculation is repeated. It is a figure at the time of displaying the locus | trajectory of the camera vector calculated | required by the CV calculating part which concerns on one Embodiment of this invention in a video image | video.

Hereinafter, embodiments of the underground exploration device according to the present invention will be described with reference to the drawings.
Here, in the underground exploration apparatus of the present invention shown below, underground 3D information, ground 3D image generation / output, unification processing, camera vector calculation, etc. are executed by a computer according to a program (software) instruction. Realized by processing, means, and function. The program can send commands to each component of the computer to perform the following predetermined processing and functions according to the present invention.
That is, each process, means, and function in the present invention are realized by specific means in which a program and a computer cooperate.

Note that all or part of the program is provided by, for example, a magnetic disk, optical disk, semiconductor memory, or any other computer-readable recording medium, and the program read from the recording medium is installed in the computer and executed. The
The program can also be loaded and executed directly on a computer through a communication line without using a recording medium. Moreover, it can also be comprised with the single information processing apparatus (for example, one personal computer etc.) with which the underground exploration apparatus which concerns on this invention is equipped, and several information processing apparatuses (for example, a plurality of computer group etc.) Can also be configured.

[Ground exploration equipment]
In FIG. 1, the structure of the underground exploration apparatus which concerns on one Embodiment of this invention is shown.
The underground exploration apparatus according to an embodiment of the present invention shown in FIG. 1 includes an exploration vehicle 10, a three-dimensional underground radar 20 and an omnidirectional camera 30 mounted on the exploration vehicle 10.
As shown in FIG. 1, while the exploration vehicle 10 travels on an arbitrary road surface to be explored, the 3D underground radar acquires and generates underground 3D information below the road surface, and the omnidirectional camera 30. To obtain and generate ground 3D images on the road surface, and then generate and output ground 3D information and ground 3D images as unified single 3D information (see FIG. 7 described later). It is like that.
Hereinafter, each configuration of the underground exploration device will be specifically described.

[Exploration vehicle 10]
The exploration vehicle 10 is a vehicle capable of traveling on a road surface such as a road or the ground, which is a target and range of underground exploration, and is configured by a car such as a passenger car, a truck, a light van, various work vehicles, and a light vehicle. In addition, the specific configuration of the exploration vehicle 10 is not particularly limited as long as it can travel on a desired road surface that is an exploration target / target.
FIG. 2 schematically shows an operation image of the exploration vehicle 10 of the present embodiment.
As shown in FIG. 1A, the exploration vehicle 10 has a GPS (high precision GNSS) 11 as an on-vehicle equipment attached to the top surface of the roof, and an antenna 22 of the three-dimensional ground penetrating radar 20 on the bottom surface of the vehicle. The omnidirectional camera 30 is disposed at the center.
In addition, a radar main body 21 of the three-dimensional underground radar 20 is disposed in the vehicle of the exploration vehicle 10, and a CV calculation unit 40 and a centralization processing unit 50 are provided. Note that the CV calculation unit 40 and the unification processing unit 50 are specifically configured by an information processing apparatus such as a PC on which predetermined software (program) is installed.

[Three-dimensional ground penetrating radar 20]
The three-dimensional underground radar 20 provided in the exploration vehicle 10 is an underground exploration means (sensor) for generating underground three-dimensional information below the road surface on which the exploration vehicle 10 travels. As shown in FIG. The radar main body 21 is mounted on the inside of the exploration vehicle 10 and the antenna 22 is arranged on the bottom surface of the exploration vehicle 10.
The three-dimensional ground penetrating radar 20 according to the present embodiment is a physical exploration radar that searches and detects the presence of underground cavities, buried pipes, and the like using a phenomenon in which electromagnetic waves are reflected at a physical boundary surface in the ground. By moving the exploration vehicle 10 at a predetermined speed (for example, a maximum speed of 60 km) while transmitting and receiving electromagnetic waves from the antenna 22 under the control of the radar main body 21, it is possible to easily map information up to 2 m underground in three dimensions. (See FIG. 4 described later).

Specifically, the three-dimensional ground penetrating radar 20 uses an underground radar antenna (multi-channel) having a maximum of 41 channels manufactured by 3d-Radar, Norway, as an underground CT scanner that uses electromagnetic waves. It is possible to visualize the underground image in 3D while driving.
Here, the underground radar used in the conventional hand-held type sensor or the like has one channel (single channel). The single channel is an antenna system in which electromagnetic wave transmission and reception sensors are paired.

On the other hand, the multi-channel employed in the three-dimensional ground penetrating radar 20 of this embodiment includes a sensor (receiving / transmitting means) including a plurality of transmitters and receivers in the antenna, and acquires a plurality of two-dimensional sections. Since it is possible, the underground state can be explored and generated as three-dimensional information.
Conventionally, a plurality of single-channel antennas are arranged, and information on a two-dimensional cross section is acquired at a pitch of about 0.5 m, for example. On the other hand, the multi-channel antenna according to this embodiment is a high-density at a pitch of 7.5 cm, for example. By arranging in, the underground state can be acquired as high-definition three-dimensional information.

In the present embodiment, the antenna 22 of the three-dimensional underground radar 20 as described above is provided between the front and rear wheels on the bottom surface of the exploration vehicle 10 or a plurality of wheels as shown in FIG. It is arranged between the axles.
By arranging the antenna 22 of the three-dimensional ground penetrating radar 20 on the bottom surface of the exploration vehicle 10 in this way, as shown in FIG. 2 (b), the road surface to be explored is multiple times (for example, twice on the same road). By traveling in the vehicle width direction, underground survey data over the entire width of the road surface can be acquired even on a road that exceeds the irradiation range (irradiation width) of the electromagnetic wave of the antenna 22. FIG. 2B shows a case where the underground exploration data at 7.5 cm intervals can be acquired over the entire range of the road width of 2.5 m by causing the exploration vehicle 10 to travel twice on the same road surface.

Here, the form in which the antenna is arranged on the bottom surface of the vehicle is called a “ground type”, and the “air type” is a form in which a conventional antenna protrudes largely forward or rearward of the vehicle or pulls a loading platform equipped with the antenna. Has many advantages compared to.
The air type is assumed to be used for removing landmines in unpaved areas, and the antenna is measured by floating about 20 to 30 cm from the ground surface. For this reason, it can be easily mounted on the vehicle, such as mounted so that it protrudes greatly in front of the vehicle, or towed by the vehicle, but because the distance between the antenna and the ground surface is large, the electromagnetic wave is attenuated and the exploration depth is increased. There is a disadvantage that it becomes shallow.
As described above, the conventional air type has an antenna characteristic that needs to be lifted from 20 to 30 cm from the ground surface, and the antenna itself has a thickness of about 20 cm, so that the antenna is directly mounted on the vehicle (ground type). Could not be used.

On the other hand, the ground type is based on the assumption that the radar antenna is used at a short distance from the road surface. Specifically, it can be measured from about 3 to 10 cm from the ground surface.
For this reason, electromagnetic waves are not easily attenuated, and the exploration depth is about 1.5 to 2 times deeper than the air type.
In addition, the ground type allows the antenna to be as close to the ground surface as possible, and because the antenna height is as thin as about 10 cm in order to be placed directly on the vehicle, the stability and safety of underground surveys are improved. Excellent.

In the present embodiment, the ground type having a large merit as described above is adopted, and the antenna 22 of the three-dimensional underground radar 20 serving as the underground exploration means is disposed on the bottom surface of the exploration vehicle 10.
Specifically, the above-mentioned ground type antenna manufactured by 3d-Radar in Norway is employed, and the antenna 22 of the three-dimensional ground penetrating radar 20 is attached between the front and rear wheel axles (between the wheel bases) of the exploration vehicle 10. ing.
3d-Radar's ground type antenna should be installed at the most stable position between the front and rear axles of automobiles, etc., because the antenna height is more than half thinner than conventional air antennas and other companies' ground type antennas. Is possible.

Here, installing the antenna 22 of the three-dimensional underground radar 20 between the wheel bases of the exploration vehicle 10 has the following advantages.
-The inner ring difference can be reduced by shortening the total vehicle length of the exploration vehicle 10.
・ Small turning is achieved by shortening the total vehicle length of the exploration vehicle 10.
・ There is no need to worry about the rear as in the towing type.
・ Backward movement is easier than towing type.
・ Since the rear wheel type is installed, the antenna is at a certain height relative to the road surface, and stable data can be acquired.

For ground penetrating radar exploration, it is very important for the accuracy of the data that the antenna height is constant, and the merit of arranging the antenna 22 of the three-dimensional ground penetrating radar 20 between the wheel bases of the exploration vehicle 10 is very high. It will be big.
The position of the antenna 22 disposed between the wheel bases of the exploration vehicle 10 is disposed at an optimal position in the vehicle length direction between the front and rear wheels of the exploration vehicle 10 or between a plurality of wheels. That is, the antenna 22 is arranged at an appropriate and optimal position in the vehicle length direction (front and rear in the vehicle traveling direction) of the exploration vehicle 10 according to the vehicle length of the exploration vehicle 10, the arrangement of front and rear wheels, the output and characteristics of the radar, and the like. .
Therefore, the antenna 22 is preferably configured to be movable / adjustable in the vehicle length direction of the exploration vehicle 10.

In the present embodiment, the antenna 22 of the three-dimensional underground radar 20 as described above is configured to be movable / adjustable in the vehicle width direction (left and right in the vehicle traveling direction) between the wheel bases of the exploration vehicle 10. .
Specifically, as shown in FIGS. 3A and 3B, the antenna 22 of the three-dimensional underground radar 20 can be slid to the left and right in the vehicle width direction of the exploration vehicle 10. It can be fixed at a desired position in the width direction. Accordingly, the antenna 22 can be moved and adjusted according to the size (road width) of the road surface to be searched, and can correspond to road surfaces of various sizes (widths).

In the example shown in FIG. 3, the antenna 22 having the exploration width of 1.8 m is moved so as to protrude 15 cm from the left and right side surfaces of the exploration vehicle 10. The case where underground exploration data can be acquired by traveling is shown.
In this way, by enabling the antenna 22 of the three-dimensional ground penetrating radar 20 to move in the vehicle width direction between the wheel bases of the exploration vehicle 10, one antenna having an antenna width (for example, 1.8 m) is moved to the left and right of the exploration vehicle. By sliding a plurality of times (for example, twice), it is possible to acquire data in a range that exceeds the antenna width (for example, a maximum of 3.5 m) (see FIG. 3).
Also, as shown in FIG. 3, the exploration of the side surface of the exploration vehicle 10 is about 15 cm on one side, for example, compared to a 2.5 m wide antenna used in a conventional air type antenna, for example, The risk of contact accidents with other obstacles or third parties can be suppressed or avoided.

The range in which the antenna 22 can move in the vehicle width direction as described above can be arbitrarily set according to the antenna width, the vehicle width of the exploration vehicle 10, and the like.
Also, the moving structure of the antenna 22 is not particularly limited as long as the antenna 22 can be moved in the vehicle width direction, such as a slide structure or a stage structure in which a plurality of fixing positions using bolts or the like are provided.
Further, the antenna 22 can be arranged in an exposed state exposed on the bottom surface of the exploration vehicle 10, but the entire antenna 22 is prepared in preparation for damage such as rebound or collision of a foreign object or contact with the ground or an obstacle. Alternatively, a part can be covered with a cover or the like. For example, a plastic cover that does not affect the transmission and reception of electromagnetic waves from the antenna 22 can be provided.

FIG. 4 shows the underground exploration data acquired and generated by the three-dimensional underground radar 20 according to the present embodiment as described above.
As shown in FIG. 4A, in the three-dimensional ground penetrating radar 20 that moves as the exploration vehicle 10 travels, the multi-channel antenna 22 uses the two-dimensional ground in a predetermined interval (for example, 7.5 cm pitch). Information is acquired along the traveling direction of the vehicle, and acquired and generated as underground three-dimensional information.
As shown in FIG. 4 (b), such underground three-dimensional information is obtained by using underground data in the arrangement direction of the antennas 22 (vehicle width direction) and the traveling direction of the antennas 22 (vehicle length direction). 3D underground section information of vertical section (antenna direction), vertical section (traveling direction), and horizontal section is shown. By generating these images, it can be output as underground 3D image information. .
The generation and output of these three-dimensional underground information is performed by the radar main body 21 of the three-dimensional underground radar 20, a PC connected to the radar main body 21, or the like.

[Omnidirectional camera 30]
The omnidirectional camera 30 is a terrestrial image generation means that is arranged on the roof surface of the exploration vehicle 10 and generates a terrestrial 3D image on the road surface on which the exploration vehicle 10 travels.
Specifically, the omnidirectional camera 30 is one or a plurality of video cameras that can photograph 360 ° around the traveling exploration vehicle 10, and the three-dimensional underground radar described above as the exploration vehicle 10 travels. A ground three-dimensional image (all-around image) corresponding to the underground data acquired by 20 can be captured.

FIG. 5 shows an example of a 360-degree stereoscopic all-round image acquired and generated by the omnidirectional camera 30 of the present embodiment.
Since the omnidirectional camera 30 can capture a 360-degree all-around video, for example, as shown in FIG. 5A, the ground state / landscape with a radius of 15 m centering on the exploration vehicle 10 traveling on the road surface is imaged. Is done. In the example in the figure, the road width is 3.5m, the road shoulder is 0.5m, the side belt, centering on the exploration vehicle 10 traveling in the center of one side lane on a road or bridge with a total width of 14.55m and an effective width of 13.75m. It can be seen that 360 ° all-around video can be captured and acquired in the range of 0.5 m, sidewalk part 2.0 m, separation zone 0.75, and ground cover 0.4 m.
Further, as shown in FIG. 5B, the ground state / landscape is imaged within a radius of 40 m centering on the exploration vehicle 10 traveling on the road surface, and the slope (slope) next to the road surface is 360 °. It can be seen that all-round video can be captured and acquired.
FIG. 5C is an output example of a 360 ° omnidirectional video imaged by the omnidirectional camera 30.

In this embodiment, a predetermined camera vector calculation is performed on the basis of a 360 ° all-round image acquired and generated by the omnidirectional camera 30, so that no GPS surveying can be performed. Regardless of position information, simply attaching an omnidirectional camera 30 to the exploration vehicle 10 and traveling (for example, a maximum speed of about 60 km / h) enables easy and highly accurate three-dimensional ground mapping.
In general, position information obtained by GPS or a four-directional camera mounted on an automobile or the like constituting an exploration vehicle has an error of, for example, about 0.5 m to several meters depending on the GPS reception status. For this reason, it is difficult to specify the position of an arbitrary abnormal point or the like by itself. In addition, since GPS cannot be received in a tunnel or under a viaduct, the position information is only an image of a four-way camera mounted on the vehicle, and it becomes more difficult to specify the position of an arbitrary location.
Therefore, in the present embodiment, a predetermined camera vector calculation is performed in the camera vector calculation unit 40 based on the 360 ° all-round video acquired by the omnidirectional camera 30, so that it does not depend on the GPS and can be performed locally. High-precision three-dimensional ground mapping is performed without performing surveying and measurement.

[CV calculation unit 40]
The CV (camera vector) calculation unit 40 is a camera vector calculation unit that can calculate and generate high-accuracy position information by calculation from the image data of the terrestrial 3D video imaged by the omnidirectional camera 30 described above. .
Specifically, the CV calculation unit 40 according to the present embodiment automatically tracks a feature point extraction unit 41 that automatically extracts a predetermined number of feature points in each frame image of the moving image and the extracted feature points. A feature point correspondence processing unit 42 for obtaining a correspondence relationship between the frame images, and obtaining a three-dimensional position coordinate of the feature point for which the correspondence relationship is obtained, and obtaining a third order of the camera corresponding to each frame image from the three-dimensional position coordinate. It is configured by means such as a camera vector calculation unit 43 for obtaining a camera vector composed of original position coordinates and three-dimensional rotation coordinates (see FIG. 8 described later).
Camera vector calculation is a technique disclosed in Japanese Patent No. 4446041 related to the present applicant, and feature points are automatically extracted from 360 ° all-around video and tracked in a plurality of adjacent frames. A triangle that is calibrated with the base of the camera movement and the tracking point is constructed, and the tracking data is analyzed to obtain a full-circle CV (camera vector) image having the three-dimensional coordinates of the feature point and the camera position and orientation. It is.

The CV (camera vector) value obtained by the CV calculation represents the position and orientation of the camera (omnidirectional camera 30) with six degrees of freedom. That is, the coefficients of 6 degrees of freedom of x, y, z, θx, θy, θz are determined by calculation for all the frames of the 360 ° all-round video imaged by the omnidirectional camera 30. Relative position information and three-dimensional coordinate information can be generated and attached only by calculation in a 360 ° video without depending on position information obtained by surveying or the like.
As a result, a CV value can be generated / applied for an arbitrary all-round image, and based on this CV value, the all-around image and the three-dimensional underground information acquired by the three-dimensional ground penetrating radar 20 described above. A three-dimensional information unification process is performed for unifying.
Specific details of the CV calculation in the CV calculation unit 40 will be described later with reference to FIGS.

[Unification processing unit 50]
The unification processing unit 50 converts the underground 3D information generated by the 3D underground radar 20 and the ground 3D image generated by the all-around camera 30 from the ground to the ground space by a predetermined data centralization process. It is a three-dimensional information unification processing means that is generated as unified three-dimensional information.
Specifically, the unification processing unit 50 according to this embodiment includes the position coordinate information included in the camera vector generated by the CV calculation unit 40 and the underground 3D information acquired by the 3D underground radar 20. Based on the above, 3D information in the ground and 3D images on the ground are unified.

FIG. 6 is a flowchart showing the procedure of the unification processing of the underground exploration data and the ground image by the unification processing unit 50 of the present embodiment.
As shown in the figure, in the unification processing of the present embodiment, first, the underground exploration data (step 1) acquired by the three-dimensional ground penetrating radar 20 and the entire 360 ° of the ground acquired by the omnidirectional camera 30 are displayed. The peripheral video (step 2) is input to the unification processing unit 50.
At this time, the underground exploration data acquired by the three-dimensional ground penetrating radar 20 includes position information based on position information or time information acquired by the GPS (high precision GNSS) 11 mounted on the exploration vehicle 10. It is.
The 360 ° all-round video of the omnidirectional camera 30 includes position information based on position information and time information acquired by the GPS (high precision GNSS) 11 mounted on the exploration vehicle 10, and the CV described above. Calculation processing is performed in the calculation unit 40 (step 3), and a CV value that is a three-dimensional one coordinate is given.

These underground exploration data and 360 ° all-round video are associated with each other based on the corresponding position information, and the CV value calculated and given by the CV calculation unit 40 corresponds to the 360 ° all-round video. It is given to the medium exploration data and tied by the same position coordinate (step 4).
As a result, the underground exploration data becomes high-accuracy mapped underground three-dimensional information given CV values as position coordinates (step 5).
Further, the 360 ° all-round video is subjected to ortho image processing and is generated as a terrestrial 3D video consisting of a high-precision ortho image to which a CV value is given (step 6). Here, for the generation of the ortho image, a known image generation technique can be used. However, in the present embodiment, a highly accurate ortho image is generated and output based on the all-around video to which the CV value is given. be able to. Specifically, using the CV value as position information, the 360 ° all-round video is converted from the central projection into an orthographic projection image having no distortion as seen from directly above, so that the plural orthographic images are not conspicuous. By joining (mosaic), an integrated ortho image can be generated, and ortho image data divided into a desired range and size can be generated and stored.
Thereafter, based on the corresponding CV value, the ground three-dimensional video composed of the ground three-dimensional information and the ortho image is unified, and is generated and output as ground ground unified data (step 7).

High-accuracy 3D mapping by unification processing of the above ground 3D information and ground ortho image, CV calculation of the 3D position coordinates of the ground and ground data is not affected by the GPS reception status, etc. It can be acquired with an error of about 5 to 15 cm based on the above.
The ortho image is converted to an image without distortion as seen from directly above and is given position information. The ortho image of the road surface is generated from the video of the omnidirectional camera to which the three-dimensional coordinate data is added. be able to. Such an ortho image has an accuracy similar to that of a planar map, and thus is an image suitable for displaying a three-dimensional ground radar result.

FIG. 7 shows an example of an image obtained by unifying the ground three-dimensional data and the ground three-dimensional data into a ground ortho image.
FIG. 7A shows a road surface in which the underground information of the cavities and buried pipes obtained by the three-dimensional ground penetrating radar 20 and the ground information such as road surface cracks are generated from the three-dimensional position information obtained by the CV calculation unit 40. It is the output result unified to the ortho image of. The cracks and depressions on the road surface appearing in the figure can be read by image analysis.
FIG. 7B shows an abnormality (rebar corrosion) inside the bridge floor plate discovered by the three-dimensional ground penetrating radar 20 together with an orthoimage of the road surface generated from the three-dimensional position information obtained by the CV calculation unit 40. It is a thing. With such information, the position of rebar corrosion can be clarified, and time-series data changes after the next survey or repair work can be visually compared and databased. In addition, the creation of a database of abnormal locations becomes useful information useful for maintenance management such as a repair plan.
Such a unified processing image according to the present embodiment can be arbitrarily processed / edited by an information processing apparatus such as a PC, or can be displayed on a display device (display), or can be printed by a printing device such as a printer. can do.

The following effects are obtained by the unified processing image obtained by unifying the ground three-dimensional data obtained in the present embodiment as described above into the ground ortho image.
(1) Since high-accuracy three-dimensional coordinates can be obtained, position identification by a two-dimensional ground penetrating radar becomes unnecessary.
(2) With the position correction device, highly accurate three-dimensional coordinates can be obtained even in places where GPS of tunnels and viaducts cannot be received.
(3) Since high-precision three-dimensional coordinates on the ground can be obtained efficiently, there is no need to measure the shape of the structure or the surrounding objects on the road.
(4) Centralized management is possible by creating a database of 3D underground radar underground information and ground information of the all-around camera 30 and the CV calculation unit 40.
(5) From visual information using high-precision ortho images, time series management of information such as comparison with the next survey and after repair work can be performed.
(6) The ortho image has the same accuracy as the planar map, but is visually superior to the map, so that it can provide effective results to the orderer.
(7) Since highly accurate three-dimensional coordinate information can be easily obtained from the video for efficiency and unification of surveys, it is possible to quickly respond to emergencies such as ensuring the safety of supply roads in the event of a disaster.

[CV calculation processing]
Next, specific contents of the CV calculation executed by the CV calculation unit 40 of the underground exploration apparatus according to the present embodiment described above will be described with reference to FIGS.
As described above, the CV calculation is one of methods for obtaining the CV value, and the result obtained by the CV calculation is referred to as CV value and CV data. The notation CV is an abbreviation for camera vector: CameraVector, and the camera vector (CV) is a value indicating a three-dimensional position and a three-axis rotation posture of a camera such as a video camera that acquires an image for measurement or the like.
The CV calculation acquires a moving image (video image), detects a feature point in the image, tracks it in a plurality of adjacent frames, and creates a triangle formed by the camera position and the tracking locus of the feature point in the image. The three-dimensional position of the camera and the three-axis rotational posture of the camera are obtained by generating a large number of images and analyzing the triangle.

In the CV calculation, it is an important characteristic that, in the process of obtaining the CV value, three-dimensional coordinates are simultaneously obtained for feature points (reference points) in the video at the same time.
Further, the CV value obtained by calculation from the moving image is obtained simultaneously with the three-dimensional camera position and the three-dimensional camera posture corresponding to each frame of the moving image. Moreover, in principle, the characteristic that a CV value is obtained corresponding to an image with a single camera is an excellent feature that can be realized only by CV calculation.
However, the CV value can also be obtained from, for example, measurement means (GPS and gyro, IMU, or the like) using another method. Therefore, acquisition of the CV value is not limited to the case of CV calculation.
In addition, when acquiring a CV value by GPS and a gyro, IMU, etc., in order to acquire each frame of a moving image and its 3D camera position and 3D camera posture at the same time, an image frame and measurement It is necessary to synchronize the sampling time with high accuracy and completely.

CV data obtained by calculation from a moving image is a relative value when not processed, but if it is a short section, three-dimensional position information and three-axis rotation angle information can be acquired with high accuracy.
Also, when CV data is acquired from an image, the acquired data is a relative value, and calibration at a known point is necessary to convert it to an absolute value. It is difficult to realize by other methods that can measure the positional relationship with an arbitrary object in the image by the point three-dimensional position coordinates and orientation, or by the feature point newly specified and acquired in the image. It has excellent characteristics.
In addition, since the CV value corresponding to the image is obtained, the compatibility with the image is good, and as long as the CV value is acquired from the image, there is no contradiction in the image. The CV calculation that can directly determine the camera position and its three-axis rotation posture from is suitable for in-image measurement and in-image survey.
In the present invention, based on the CV value data obtained by the CV calculation, the above-described generation of the ground ortho image and the unified processing image of the ground three-dimensional data and the ground ortho image are generated.

[CV calculation unit]
The CV calculation unit 40 obtains a CV value by performing a predetermined CV calculation process on the video image captured by the above-described omnidirectional camera 30. Specifically, as shown in FIG. A feature point extraction unit 41, a feature point correspondence processing unit 42, a camera vector calculation unit 43, an error minimization unit 44, a three-dimensional information tracking unit 45, and a high-precision camera vector calculation unit 46 are provided.

First, in principle, any video can be used for CV calculation. However, in the case of a video with a limited angle of view, the video is interrupted when the viewpoint direction is moved. It is desirable to treat a peripheral image (see FIGS. 5C and 9) or a wide-angle image as a part of the entire peripheral image. Note that a moving image is similar to a continuous still image and can be handled as a plurality of still images.
In general, a moving image recorded in advance is used as the video, but it is also possible to use a video captured in real time in accordance with the movement of a moving body such as an automobile.

Therefore, in this embodiment, the video used for the CV calculation is an all-round video (see FIGS. 5C and 9) obtained by shooting the entire 360 ° circumference of a moving body such as a vehicle, or an all-round video. By using the near wide-angle video and expanding the entire circumference image in the viewpoint direction, it can be handled as a part of the entire circumference image.
The planar development of the all-around video is to express the all-around video as a normal image in perspective. Here, the term “perspective” refers to the fact that the entire circumference image itself is displayed by a method different from the perspective method, such as Mercator projection and spherical projection projection (see FIG. 9). This is because it can be converted and displayed into a normal perspective image by performing planar development display like the equidistant projection.

In the present embodiment, first, for the purpose of obtaining the CV value data by running the exploration vehicle 10, the omnidirectional camera 30 fixed to the exploration vehicle 10 moves around the exploration vehicle 10 along with the movement of the exploration vehicle 10. A 360 ° all-round image (see FIGS. 5C and 9) is taken.
In addition, for the purpose of acquiring the position coordinates, the exploration vehicle 10 may include, for example, a position measuring device configured by a GPS device alone or an IMU device to which absolute coordinates are acquired.
The omnidirectional camera 30 mounted on the exploration vehicle 10 may have any configuration as long as it captures and acquires a wide range of images. For example, a camera with a wide-angle lens, a fisheye lens, a moving camera, a fixed camera, and the like. There are cameras, cameras with a plurality of cameras fixed, cameras that can rotate around 360 degrees, and the like. In the present embodiment, as described above, one or a plurality of cameras are integrally fixed to the exploration vehicle 10, and the omnidirectional camera 30 that captures a wide range image as the exploration vehicle 10 moves is used. .

Then, according to the omnidirectional camera 30 as described above, by being installed on the top of the roof of the exploration vehicle 10 or the like, it is possible to simultaneously capture images of the entire 360 degrees of the camera with one or a plurality of cameras. As the exploration vehicle 10 travels and moves, a wide range image can be acquired as moving image data.
Here, the omnidirectional camera 30 is a video camera that can directly acquire the entire circumference of the camera. However, if more than half of the entire circumference of the camera can be acquired as an image, it can be used as the entire circumference.
Even in the case of a normal camera with a limited angle of view, the accuracy of the CV calculation is reduced, but it can be handled as a part of the entire peripheral video.

Note that the wide-area video imaged by the omnidirectional camera 30 can be pasted as a single image on a virtual spherical surface that matches the angle of view at the time of imaging.
The spherical image data pasted on the virtual spherical surface is stored and output as spherical image (360 degree image) data pasted on the virtual spherical surface. The virtual spherical surface can be set to an arbitrary spherical shape centered on the camera unit that acquires a wide range image.
FIG. 9A is an appearance image of a virtual spherical surface to which a spherical image is pasted, and FIG. 9B is an example of a spherical image pasted to the virtual spherical surface. FIG. 4C shows an example of an image obtained by developing the spherical image of FIG.

Then, the all-round video image generated / acquired as described above is input to the CV calculation unit 40 to obtain CV value data (see FIG. 8).
In the CV calculation unit 40, first, the feature point extraction unit 41 automatically extracts a sufficient number of feature points (reference points) from the moving image data shot and temporarily recorded by the omnidirectional camera 30.
The feature point correspondence processing unit 42 automatically obtains the correspondence relationship by automatically tracking the feature points automatically extracted in each frame image between the frames.
The camera vector calculation unit 43 automatically calculates a camera vector corresponding to each frame image from the three-dimensional position coordinates of the feature points for which the correspondence relationship has been determined.
The error minimizing unit 44 performs statistical processing so that the solution distribution of each camera vector is minimized by overlapping calculation of a plurality of camera positions, and automatically determines the camera position direction subjected to the error minimizing process. .

The three-dimensional information tracking unit 45 positions the camera vector obtained by the camera vector calculation unit 43 as an approximate camera vector, and a plurality of frame images based on the three-dimensional information sequentially obtained as part of the image in the subsequent process. 3D information is automatically tracked along an image of an adjacent frame. Here, three-dimensional information (three-dimensional shape) is mainly three-dimensional distribution information of feature points, that is, a collection of three-dimensional points, and this collection of three-dimensional points constitutes a three-dimensional shape. To do.
The high-precision camera vector calculation unit 46 generates and outputs a camera vector with higher accuracy than the camera vector obtained by the camera vector calculation unit 43 based on the tracking data obtained by the three-dimensional information tracking unit 45.
The camera vector obtained as described above is input to the unification processing unit 50 described above, and is used for the generation of the ground ortho image and the unification processing of the ground three-dimensional data and the ground ortho image.

There are several methods for detecting a camera vector from feature points of a plurality of images (moving images or continuous still images). In the CV calculation unit 40 of the present embodiment shown in FIG. By automatically extracting a number of feature points and automatically tracking them, a three-dimensional vector and a three-axis rotation vector of the camera are obtained by triangulation geometry and epipolar geometry.
By taking a large number of feature points, camera vector information is duplicated, so that a unit triangle composed of the coordinates of three points can be obtained from many feature points more than ten times, and duplicate information Therefore, it is possible to obtain a more accurate camera vector by minimizing the error.

A camera vector is a vector of six degrees of freedom possessed by a moving camera.
In general, a stationary three-dimensional object has six degrees of freedom of position coordinates (X, Y, Z) and rotation angles (Φx, Φy, Φz) of the respective coordinate axes. Therefore, the camera vector refers to a vector of six degrees of freedom of the camera position coordinates (X, Y, Z) and the rotation angles (Φx, Φy, Φz) of the respective coordinate axes. When the camera moves, the direction of movement also enters the degree of freedom, which can be derived by differentiation from the above six degrees of freedom.
Thus, the detection of the camera vector in the present embodiment means that the camera takes six degrees of freedom values for each frame and determines six different degrees of freedom for each frame.

Hereinafter, a specific camera vector detection method in the CV calculation unit 40 will be described with reference to FIG.
First, the image data acquired by the omnidirectional camera 30 described above is input to the feature point extraction unit 41 of the CV calculation unit 40 indirectly or directly, and the frame image appropriately sampled by the feature point extraction unit 41. A point or a small area image to be a feature point is automatically extracted, and a feature point correspondence processing unit 42 automatically obtains a correspondence relationship between the feature points among a plurality of frame images.
Specifically, more than a sufficient number of feature points that are used as a reference for detecting a camera vector are obtained. An example of feature points between images and their corresponding relationships are shown in FIGS. In the figure, “+” is a feature point that is automatically extracted, and the correspondence is automatically tracked between a plurality of frame images (see correspondence points 1 to 4 shown in FIG. 12).
Here, for extracting feature points, it is desirable to specify and extract a sufficient number of feature points in each image as shown in FIG. 13 (see the circles in FIG. 13). For example, about 100 feature points are extracted. Extract points.

Subsequently, the camera vector calculation unit 43 calculates the three-dimensional coordinates of the extracted feature points, and calculates the camera vector based on the three-dimensional coordinates. Specifically, the camera vector calculation unit 43 includes a sufficient number of feature positions that exist between successive frames, a position vector between moving cameras, a three-axis rotation vector of the camera, and each camera position and feature. Relative values of various three-dimensional vectors such as vectors connecting points are continuously calculated by calculation.
In this embodiment, for example, camera motion (camera position and camera rotation) is calculated by solving an epipolar equation from the epipolar geometry of a 360-degree all-round image. In addition, if this is described as triangulation, the equipment that surveys the azimuth and depression angle elevation angle is mounted on the vehicle, and from the data obtained by measuring the same object with the moving surveying instrument, multiple target survey points, This is the same as the case of obtaining the coordinates of each target point and the trajectory of the surveying instrument.

Images 1 and 2 shown in FIG. 12 are images obtained by developing a 360-degree all-round image with Mercator development. When latitude φ and light θ are assumed, points on image 1 are (θ1, φ1) and points on image 2 are ( θ2, φ2). The spatial coordinates of each camera are z1 = (cos φ1 cos θ1, cos φ1 sin θ1, sin φ1), z2 = (cos φ2 cos θ2, cos φ2 sin θ2, sin φ2). If the camera movement vector is t and the camera rotation matrix is R, z1T [t] × Rz2 = 0 is the epipolar equation.
By providing a sufficient number of feature points, t and R can be calculated as a solution by the method of least squares by linear algebra calculation. This calculation is applied to a plurality of corresponding frames.

Here, it is preferable to use a 360-degree all-round image as an image used for the calculation of the camera vector.
The image used for the camera vector calculation may be any image in principle, but the all-around video is extremely advantageous because the direction of the object is expressed as it is in latitude and longitude. An image using a wide-angle lens such as a 360-degree all-round image shown in FIG. 12 makes it easier to select many feature points and track them over a plurality of frames. Further, even when an image is obtained by a narrow-angle lens, it is effective because the feature point does not escape from the field of view when shooting from various directions while moving around the same object.

Therefore, in this embodiment, a 360-degree all-round image is used for the CV calculation, whereby the tracking distance of the feature points can be increased, and a sufficiently large number of feature points can be selected. It becomes possible to select a feature point convenient for each short distance. In addition, when correcting the rotation vector, the calculation process can be easily performed by adding the polar rotation conversion process. As a result, a calculation result with higher accuracy can be obtained.
In FIG. 12, in order to make it easy to understand the processing in the CV calculation unit 2, a 360-degree all-round spherical image obtained by synthesizing images taken by one or a plurality of cameras is developed by a Mercator projection called map projection. However, in an actual CV calculation, it is not necessarily a developed image by the Mercator projection.

Next, the error minimizing unit 44 calculates and calculates a plurality of vectors based on each feature point by a plurality of calculation equations based on a plurality of camera positions and a plurality of feature points corresponding to each frame, Statistical processing is performed so that the distribution of the position of each feature point and the camera position is minimized to obtain a final vector. For example, the optimal solution of the least square method is estimated by the Levenberg-Marquardt method for multiple frame camera positions, camera rotations, and multiple feature points, and errors are converged to determine the camera position, camera rotation matrix, and feature point coordinates. .
Further, feature points having a large error distribution are deleted, and recalculation is performed based on other feature points, thereby improving the accuracy of computation at each feature point and camera position.
In this way, the position of the feature point and the camera vector can be obtained with high accuracy.

14 to 16 show examples of the three-dimensional coordinates of feature points and camera vectors obtained by CV calculation. FIG. 14 to FIG. 16 are explanatory diagrams showing the vector detection method by the CV calculation of the present embodiment, and showing the relative positional relationship between the camera and the object obtained from a plurality of frame images acquired by the moving camera. FIG.
In FIG. 14, the three-dimensional coordinates of the feature points 1 to 4 shown in the images 1 and 2 in FIG. 12 and the camera vector (X, Y, Z) that moves between the images 1 and 2 are shown.
FIGS. 15 and 16 show a sufficiently large number of feature points, the positions of the feature points obtained from the frame image, and the position of the moving camera. In the figure, a circle mark that continues in a straight line at the center of the graph is the camera position, and a circle mark located around the circle indicates the position and height of the feature point.

Here, in the CV calculation in the CV calculation unit 40, in order to obtain more accurate three-dimensional information of feature points and camera positions, as shown in FIG. Set feature points and repeat multiple operations.
Specifically, the CV calculation unit 40 automatically detects feature points having image characteristics in the image, and obtains the corresponding points of the feature points in each frame image. And the n + m-th two frame images Fn and Fn + m are used as unit calculations, and unit calculations with n and m appropriately set can be repeated.
m is the frame interval, and the feature points are classified into a plurality of stages according to the distance from the camera to the feature point in the image. The distance from the camera to the feature point is set so that m becomes larger. M is set to be smaller as the distance to is shorter. This is because the change in position between images is less as the distance from the camera to the feature point is longer.

Then, while sufficiently overlapping the classification of the feature points by the m value, a plurality of stages of m are set, and as n progresses continuously with the progress of the image, the calculation proceeds continuously. Then, the overlap calculation is performed a plurality of times for the same feature point in each step of n and m.
In this way, by performing unit calculation focusing on the frame images Fn and Fn + m, a precise camera vector is calculated over a long time between frames sampled every m frames (frames are dropped). However, in m frames (minimum unit frames) between the frame images Fn and Fn + m, a simple calculation that can be performed in a short time can be performed.

If there is no error in the precision camera vector calculation for every m frames, both ends of the camera vector of the m frames overlap with the Fn and Fn + m camera vectors that have been subjected to the high precision calculation. Accordingly, m minimum unit frames between Fn and Fn + m are obtained by a simple calculation, and both ends of the camera vector of the m minimum unit frames obtained by the simple calculation are Fn and Fn + m obtained by high precision calculation. The scale adjustment of m consecutive camera vectors can be made to match the camera vectors.
In this way, as n progresses continuously with the progress of the image, the scale adjustment is performed so that the error of each camera vector obtained by calculating a plurality of times for the same feature point is minimized, and integration is performed. A camera vector can be determined. Accordingly, it is possible to speed up the arithmetic processing by combining simple arithmetic operations while obtaining a highly accurate camera vector having no error.

Here, there are various simple calculation methods depending on the accuracy. For example, when (1) many feature points of 100 or more are used in high-precision calculation, the minimum number of simple calculation is about ten. (2) Even if the number of the same feature points is the same as the feature points and camera positions, innumerable triangles are established there, and equations for that number are established. By reducing the number of equations, it can be simplified.
In this way, integration is performed by adjusting the scale so that the error of each feature point and camera position is minimized, distance calculation is performed, and feature points with large error distribution are deleted, and other features are added as necessary. By recalculating the points, the calculation accuracy at each feature point and camera position can be improved.

In addition, by performing high-speed simple calculation in this way, camera vector real-time processing becomes possible.
In real-time processing of camera vectors, calculation is performed with the minimum number of frames that can achieve the target accuracy and the minimum number of feature points that are automatically extracted, the approximate value of the camera vector is obtained and displayed in real time, and then the image is accumulated. Accordingly, the number of frames can be increased, the number of feature points can be increased, camera vector calculation with higher accuracy can be performed, and approximate values can be replaced with camera vector values with higher accuracy for display.

Furthermore, in this embodiment, it is possible to track three-dimensional information (three-dimensional shape) in order to obtain a more accurate camera vector.
Specifically, first, the three-dimensional information tracking unit 45 positions the camera vector obtained through the camera vector calculation unit 43 and the error minimization unit 44 as an approximate camera vector, and the image generated in the subsequent process Based on the three-dimensional information (three-dimensional shape) obtained as a part, partial three-dimensional information included in a plurality of frame images is continuously tracked between adjacent frames to automatically track the three-dimensional shape.
Then, from the tracking result of the three-dimensional information obtained by the three-dimensional information tracking unit 45, a high-precision camera vector calculation unit 46 obtains a higher-precision camera vector.

The feature point extraction unit 41 and the feature point correspondence processing unit 42 described above automatically track feature points in a plurality of inter-frame images, but the number of feature point tracking frames is limited due to disappearance of feature points. May come. In addition, since the image is two-dimensional and the shape changes during tracking, there is a certain limit in tracking accuracy.
Therefore, the camera vector obtained by the feature point tracking is regarded as an approximate value, and the three-dimensional information (three-dimensional shape) obtained in the subsequent process is traced on each frame image, and a high-precision camera vector is obtained from the trajectory. Can do.
The tracking of 3D shapes is easy to obtain matching and correlation accuracy, and the 3D shape does not change in size and size depending on the frame image, so it can be tracked over many frames. The accuracy of the camera vector calculation can be improved. This is possible because the approximate camera vector is already known by the camera vector computing unit 43 and the three-dimensional shape is already known.

  When the camera vector is an approximate value, the error of 3D coordinates over a very large number of frames is small because there are few frames related to each frame by feature point tracking. The error of the three-dimensional shape when a part of the image is cut is relatively small, and the influence on the change and size of the shape is considerably small. For this reason, the comparison and tracking in the three-dimensional shape is extremely advantageous over the two-dimensional shape tracking. In tracking, when tracking with 2D shape, tracking changes in shape and size in multiple frames are unavoidable, so there are problems such as large errors and missing corresponding points. However, in tracking with a three-dimensional shape, there is very little change in shape, and in principle there is no change in size, so accurate tracking is possible.

Here, as the three-dimensional shape data to be tracked, there are, for example, a three-dimensional distribution shape of feature points, a polygon surface obtained from the three-dimensional distribution shape of feature points, and the like.
It is also possible to convert the obtained three-dimensional shape from a camera position into a two-dimensional image and track it as a two-dimensional image. Since the approximate value of the camera vector is known, projection conversion can be performed on a two-dimensional image from the camera viewpoint, and it is also possible to follow a change in the shape of the object due to movement of the camera viewpoint.

The camera vector obtained as described above can be displayed in a superimposed manner in the video image captured by the omnidirectional camera 30.
For example, as shown in FIG. 18, the image from the in-vehicle camera is developed in a plane, the corresponding points on the target plane in each frame image are automatically searched, and the corresponding points are combined to match the target plane. A combined image is generated and displayed in the same coordinate system.
Furthermore, the camera position and the camera direction can be detected one after another in the common coordinate system, and the position, direction, and locus can be plotted. The CV data indicates the three-dimensional position and the three-axis rotation, and the CV value can be observed simultaneously in each frame of the video image by displaying it over the video image. An example of an image in which CV data is displayed superimposed on a video image is shown in FIG.
If the camera position is correctly displayed in the video image, the position in the video image indicated by the CV value is the center of the image, and if the camera movement is close to a straight line, the CV values of all frames are displayed overlapping. Therefore, for example, as shown in FIG. 18, it is appropriate to display a position 1 meter right below the camera position. Alternatively, it is more appropriate to display the CV value at the height of the road surface based on the distance to the road surface.

As described above, according to the underground exploration device of the present embodiment, the following excellent operational effects can be achieved.
First, since high-accuracy three-dimensional coordinates based on CV calculation are obtained, the position specifying step by a hand-held sensor or the like as in the above-described prior art is not required, and only a hollow image is obtained by using an ortho image in which underground data is unified. It is possible to specify underground objects such as buried pipes.
In particular, in the prior art, for example, a position specifying operation based on simple position specification using GPS and a four-way camera that captures the front, rear, left, and right of the vehicle is essential, so the effect of the present embodiment is very significant.
In addition, since high-precision three-dimensional coordinates on the ground can be obtained efficiently by the CV calculation in this way, for example, a process of measuring the shape of a structure on the site, road surroundings, etc. is not necessary for specifying the detection position. As a result, the work efficiency is remarkably improved.

In addition, according to the present embodiment, the omnidirectional information can be continuously acquired for the underground information and the ground information as the search target / range, and the omnidirectional three-dimensional information of the underground and the ground can be centrally managed. Can do.
In addition, the database of underground / ground information managed in a unified manner in this way corresponds to software (applications) related to GIS (Geographic Information System) such as “ArcGIS” (registered trademark) manufactured by ESRI, USA. This makes it possible to manage the database with excellent handling, versatility, and expandability.

In addition, the orthographic image that is generated and output in this embodiment and is unified with the three-dimensional information on the ground and the ground can be stored and managed according to the time series.For example, future survey points can be extracted and compared before and after repair work. And time series management of underground exploration information, such as matching, can be performed easily and quickly without correction work.
In addition, the ortho image obtained in the present embodiment has the same accuracy as, for example, a 1/500 plan view (map), and can visually grasp the local situation based on the video, and visually It is possible to provide more useful results and information with added value that cannot be obtained by a simple plan (map).
Furthermore, according to the underground exploration device of the present embodiment, it is possible to easily obtain highly accurate three-dimensional coordinate information from the omnidirectional continuous unification processing and images of the exploration result only by running the exploration vehicle. In addition to underground exploration such as the above, there is also an excellent effect of being able to promptly respond to emergencies, such as ensuring the safety of emergency transport roads and the like in the event of a disaster.

As described above, according to the underground exploration device according to the embodiment of the present invention, the three-dimensional underground radar 20 serving as the underground exploration sensor is effectively protected, and the underground exploration data is highly accurate and accurate. High-precision data that enables accurate data acquisition, and more accurate and high-accuracy position information that cannot be obtained by GPS as ground and ground position information. 3D information can be obtained.
Therefore, according to the present embodiment, the underground exploration data and the ground surface information are different from those obtained by simply displaying the underground exploration data and the plan view (such as a map) as in the prior art. It is possible to generate and display the data and the ground ortho image as a single 3D information. As a result, underground exploration data indicating cavities, buried pipes, etc. can be identified and grasped in an accurate ortho image showing ground images in an integrated manner, and accurate and highly accurate ground that cannot be obtained with conventional technology. Medium exploration becomes possible.

In addition, although the preferred embodiment was shown and demonstrated about the underground exploration apparatus of this invention, this invention is not limited to embodiment mentioned above, A various change implementation is possible in the scope of the present invention. Needless to say.
For example, in the above-described embodiment, the explanation has been made taking the radar (three-dimensional underground radar 20) using electromagnetic waves as an example of the underground exploration means (sensor) according to the present invention. As long as the three-dimensional information therein can be acquired and generated, the radar is not limited to the radar using electromagnetic waves. Specifically, the underground exploration means may be a sensor using seismic waves, electrical resistivity, ultrasonic waves, radar, etc., in addition to using electromagnetic waves.

  The present invention makes use of the physical properties of the ground against seismic waves, electrical resistivity, electromagnetic waves, ultrasonic waves, etc. to search and investigate underground (underground) conditions indirectly and non-destructively, such as underground radar exploration It can be suitably used for underground exploration technology.

10 Exploration vehicle 11 GPS (High precision GNSS)
20 3D ground penetrating radar
21 Radar body 22 Antenna 30 Omnidirectional camera (ground image generation means)
40 CV (camera vector) calculation unit 50 Unification processing unit (three-dimensional information unification processing means)

In the present embodiment, the antenna 22 of the three-dimensional underground radar 20 as described above is provided between the front and rear wheels on the bottom surface of the exploration vehicle 10 or a plurality of wheels as shown in FIG. It is arranged between the axles.
By arranging the antenna 22 of the three-dimensional ground penetrating radar 20 on the bottom surface of the exploration vehicle 10 in this way, as shown in FIG. 2 (b), the road surface to be explored is multiple times (for example, twice on the same road). By traveling in the vehicle width direction, underground survey data over the entire width of the road surface can be acquired even on a road that exceeds the irradiation range (irradiation width) of the electromagnetic wave of the antenna 22. FIG. 2B shows a case where the underground exploration data at intervals of 7.5 cm can be acquired over the entire range of the road width of 3.0 m, for example , by causing the exploration vehicle 10 to travel twice on the same road surface. .

FIG. 5 shows an example of a 360-degree stereoscopic all-round image acquired and generated by the omnidirectional camera 30 of the present embodiment.
Since the omnidirectional camera 30 can capture a 360-degree all-around video, for example, as shown in FIG. 5A, the ground state / landscape with a radius of about 5 m centered on the exploration vehicle 10 traveling on the road surface. Is imaged. In the example in the figure, the road width is 3.5m, the road shoulder is 0.5m, the side belt, centering on the exploration vehicle 10 traveling in the center of one side lane on a road or bridge with a total width of 14.55m and an effective width of 13.75m. It can be seen that 360 ° all-around video can be captured and acquired in the range of 0.5 m, sidewalk part 2.0 m, separation zone 0.75, and ground cover 0.4 m.
In addition, as shown in FIG. 5 (b), a ground state / landscape is imaged within a radius of about 5 to 25 m with the exploration vehicle 10 traveling on the road surface as a slope (slope) next to the road surface. It can also be seen that a 360 ° all-round image can be captured and acquired.
FIG. 5C is an output example of a 360 ° omnidirectional video imaged by the omnidirectional camera 30.

A camera vector is a vector of six degrees of freedom possessed by a moving camera.
In general, a stationary three-dimensional object has six degrees of freedom of position coordinates (X, Y, Z) and rotation angles (Φx, Φy, Φz) of the respective coordinate axes. Therefore, the camera vector refers to a vector of six degrees of freedom of the camera position coordinates (X, Y, Z) and the rotation angles (Φx, Φy, Φz) of the respective coordinate axes. When the camera moves, the direction of movement also enters the degree of freedom, which can be derived by differentiation from the above six degrees of freedom.
Thus, the detection of the camera vector in the present embodiment means that the camera takes six degrees of freedom for each frame and determines six different coefficients for each frame.

Image 1 and 2 shown in FIG. 12 is an image obtained by developing Mercator 360 degrees all around the image, the latitude phi, when the through degree theta, a point on the image 1 (.theta.1, .phi.1), a point on the image 2 (Θ2, φ2). The spatial coordinates of each camera are z1 = (cos φ1 cos θ1, cos φ1 sin θ1, sin φ1), z2 = (cos φ2 cos θ2, cos φ2 sin θ2, sin φ2). If the camera movement vector is t and the camera rotation matrix is R, z1T [t] × Rz2 = 0 is the epipolar equation.
By providing a sufficient number of feature points, t and R can be calculated as a solution by the method of least squares by linear algebra calculation. This calculation is applied to a plurality of corresponding frames.

Claims (3)

An exploration vehicle having a plurality of wheels including front wheels and rear wheels and capable of traveling on a road surface;
An underground exploration means that is arranged in the exploration vehicle and generates underground three-dimensional information under the road surface on which the exploration vehicle travels,
The underground exploration device is characterized in that the underground exploration means is disposed between the front and rear wheels or between the axles of the plurality of wheels on the bottom surface of the exploration vehicle. A terrestrial image generation means for generating a terrestrial 3D image disposed on the exploration vehicle and traveling on the road surface on which the exploration vehicle runs;
Three-dimensional information in which the underground three-dimensional information generated by the underground exploration means and the ground three-dimensional video generated by the ground image generating means are unified from the ground to the ground space by a predetermined data unification process. The underground exploration device according to claim 1, further comprising: three-dimensional information unification processing means that is generated as follows. The terrestrial video generation means is
A feature point extraction unit for automatically extracting a predetermined number of feature points from the image data of the terrestrial 3D video;
About the extracted feature points, a feature point correspondence processing unit that automatically tracks within each frame image of the moving image and obtains a correspondence relationship between the frame images;
A camera vector calculation unit that obtains the three-dimensional position coordinates of the feature point for which the correspondence relationship is obtained, and obtains a camera vector composed of the three-dimensional position coordinates and the three-dimensional rotation coordinates of the camera corresponding to each frame image from the three-dimensional position coordinates. And a camera vector calculation means having
The three-dimensional information unification processing means
The ground 3D information and the ground 3D image are unified based on the camera vector generated by the camera vector calculation means and the position coordinate information of the ground 3D information. The underground exploration device according to 1 or 2.