JP2017182612A - Image arrangement method and image arrangement computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image arrangement method that automatically arranges a plurality of inspection images so as to be easily analyzed.SOLUTION: An image arrangement method comprises the steps of: preparing GNSS information including latitude and longitude sequentially recorded while moving and receipt time therefor, and imaged images sequentially recorded and imaged time of each image, calculating position information of each image based on imaged time of each image, the GNSS information, and the receipt time, transforming each image to a front-facing image, calculating an imaged direction of each image based on each GNSS information recorded at time before and after the imaged time of each image, and arranging each image based on the position information and the imaged direction of each image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image arrangement method for automatically arranging a plurality of images and a computer program for image arrangement.

  A satellite positioning system (GNSS: Global Navigation Satellite System) is a general term for systems that receive radio waves from artificial satellites and measure the current position on the earth. In addition to typical GPS, GLONASS, QZSS etc. are included. These systems are conventionally used in a wide range of fields as a means for acquiring the current position.

  Conventionally, inspection information obtained from a plurality of types of sensors is stored in association with common position information. For example, a visible image and an infrared image obtained by photographing a road pavement surface are converted into position information and time information received by GPS. It is disclosed that the information is stored in association (for example, refer to Patent Document 1).

  There is also known an apparatus that records positional information in association with a thermal image and further selects subject information corresponding to the positional information (see, for example, Patent Document 2).

JP 2011-242293 A Special table 2015-534364 gazette

  Conventionally, when an object does not fit in a single image, an image obtained by dividing a plurality of images is analyzed for each image. For this reason, in the case of a large area such as an airport runway, it is necessary to manually process a large amount of inspection images, which is very inefficient and practically impossible to perform a full inspection.

  Although a plurality of images can be automatically arranged to some extent using commercially available image processing software, it is necessary to convert the thermal image into a format such as a bitmap that can be handled by the image processing software. When the thermal image is converted in this way, the temperature information disappears from the image, and there is a problem that the subsequent analysis cannot be performed.

  SUMMARY An advantage of some aspects of the invention is that it provides an image arrangement method that automatically arranges a plurality of inspection images so as to be easily analyzed.

The image arrangement method according to the present invention comprises preparing GNSS information including latitude and longitude recorded sequentially while moving, its reception time, sequentially recorded captured images, and captured time of each image;
Determining the position information of each image based on the shooting time of each image, the GNSS information and the reception time thereof;
Converting each image into a facing image;
Calculating the shooting direction of each image based on each GNSS information recorded at times before and after the shooting time of each image;
Arranging each of the images based on the positional information and the shooting direction of each of the images;
including.

The computer program for image arrangement according to the present invention prepares GNSS information including latitude and longitude recorded sequentially while moving, its reception time, captured images recorded sequentially, and shooting times of each image; ,
Determining the position information of each image based on the shooting time of each image, the GNSS information and the reception time thereof;
Converting each image into a facing image;
Calculating the shooting direction of each image based on each GNSS information recorded at times before and after the shooting time of each image;
Arranging each of the images based on the positional information and the shooting direction of each of the images;
Each step of the above is executed by the computer to arrange each image.

  According to the image arrangement method according to the present invention, a plurality of inspection images can be automatically arranged so as to be easily analyzed.

3 is a flowchart of an image arrangement method according to the first embodiment. (A) is a block diagram showing a functional configuration of the image arrangement apparatus according to Embodiment 1, and (b) is a block diagram showing a physical configuration of a computer which is an example of the image arrangement apparatus of (a). It is. It is a block diagram which shows an example of the GPS information and image acquisition vehicle corresponding to the GPS information and image acquisition part of Fig.2 (a). It is an example of the thermal image with which the image was image | photographed with the infrared camera. It is a timing chart at the time of acquisition of GPS information and an image. It is the schematic at the time of arrange | positioning an infrared image in linear form. It is the schematic at the time of arrange | positioning an infrared image along the positional information. It is the schematic at the time of arrange | positioning the infrared image of the runway of an airport along position information. It is the elements on larger scale of the runway of the airport of FIG.

The image arrangement method according to the first aspect includes the steps of preparing GNSS information including latitude and longitude recorded sequentially while moving, the reception time thereof, sequentially recorded captured images, and captured times of the respective images, and ,
Determining the position information of each image based on the shooting time of each image, the GNSS information and the reception time thereof;
Converting each image into a facing image;
Calculating the shooting direction of each image based on each GNSS information recorded at times before and after the shooting time of each image;
Arranging each of the images based on the positional information and the shooting direction of each of the images;
including.

  In the image arrangement method according to the second aspect, in the first aspect, the image may be a thermal image including temperature information.

  In the image arrangement method according to the third aspect, in the first or second aspect, the GNSS information sequentially recorded while moving and the reception time thereof, the captured image sequentially recorded, and the photographing time of each image are recorded. In the step of preparing the above, the travel distance at each time point recorded sequentially while moving and the recording time thereof may be further prepared.

In the image arrangement method according to a fourth aspect, in any one of the first to third aspects, in the step of determining the position information of each image,
A sub-step of acquiring each GNSS information recorded at times before and after the photographing time of each image;
A sub-step of calculating position information including the latitude and longitude of each image based on the shooting time of each image, each acquired GNSS information and its reception time;
May be included.

  In the image arrangement method according to the fifth aspect, in any one of the first to fourth aspects, in the step of arranging the images, the images may be arranged linearly.

  In the image arrangement method according to the sixth aspect, in any one of the first to fourth aspects, in the step of arranging the images, the images may be arranged on a plane.

The image arrangement computer program according to the seventh aspect acquires the GNSS information including the latitude and longitude recorded sequentially while moving, the reception time thereof, the sequentially recorded photographed images, and the photographing times of the respective images. Steps,
Calculating the position information of each image based on the shooting time of each image, the GNSS information and the reception time;
Converting each image into a facing image;
Calculating the shooting direction of each image based on each GNSS information recorded at times before and after the shooting time of each image;
Arranging each of the images based on the positional information and the shooting direction of each of the images;
Each step of the above is executed by the computer to arrange each image.

  A computer-readable recording medium storing an image layout creation computer program according to the eighth aspect stores the image layout computer program according to the seventh aspect.

  Hereinafter, an image arrangement method and an image arrangement computer program according to embodiments will be described with reference to the accompanying drawings. In the drawings, substantially the same members are denoted by the same reference numerals. Moreover, although this Embodiment demonstrates using GPS information as an example of GNSS information, you may use the information acquired from positioning systems other than GPS. Note that the arrangement of images means that a plurality of images are allocated (laid out) on the screen or printed drawing with the position and orientation adjusted.

(Embodiment 1)
<Image placement method>
FIG. 1 is a flowchart of the image arrangement method according to the first embodiment. The image arrangement method according to the first embodiment includes the following steps.
(A) GPS information including latitude and longitude sequentially recorded while moving, its reception time, sequentially recorded captured images, and capture times of the respective images are prepared (S01). In addition, you may prepare the travel distance and the recording time of each time point recorded sequentially while moving.
(B) The position of the latitude and longitude of each image based on the shooting time of each image, the two reception times prepared for GPS information before and after the shooting time, and the GPS information of the two reception times, respectively. Information is calculated (S02).
(C) The tilt correction of each image is performed and converted into a directly-facing image (S03). Note that the directly-facing image means an image having no inclination as viewed from the front with respect to the object. For example, if the object is a flat object such as a road surface, the image looks as if it was taken from a direction substantially perpendicular to the plane.
(D) The shooting direction of each image is calculated based on GPS information recorded at times before and after the shooting time of each image (S04). Specifically, the image was photographed in the direction of a straight line connecting the position coordinates of GPS information recorded immediately before the image capturing time and the position coordinates of GPS information recorded immediately after the image capturing time. The shooting direction may be used. When calculating with higher accuracy, interpolation may be performed using three or more points of GPS information as well as immediately before and after, or the direction may be adjusted in consideration of the shooting time and the GPS reception times before and after. Also good.
(E) Based on the positional information including the latitude and longitude of each image and the shooting direction, each image subjected to tilt correction and rotation correction is arranged (S05). In addition, by displaying CAD data, a map, an aerial photograph or the like as a background when placing an image, it is possible to make the image correspond to the CAD data or the like.

In this image arrangement method, the position information of the latitude and longitude of each image can be calculated based on the sequentially recorded captured images, the captured time of each image, and the GPS information acquired before and after the captured time. In addition, an image subjected to tilt correction and rotation correction can be arranged based on the calculated position information. Therefore, position information and images can be handled separately for a large number of images. That is, even when the image is a thermal image (FIG. 4), the position information can be associated with each thermal image, and the thermal information can be arranged based on the position information while maintaining the temperature information of each thermal image. it can.
Moreover, according to this image arrangement method, when there are a large number of images, it is possible to deal with, for example, an airport runway. In the case of a runway of a medium-scale airport of 2500 m × 60 m, about 50,000 images are generated. The target is not limited to an airport, and can be applied to a large area and planar structure such as a port, road, or building. According to the image arrangement method described above, even if the number of sheets is large, the image information can be arranged based on the position information while maintaining the temperature information of each thermal image.

<Calculation of latitude and longitude position information of each image>
FIG. 5 is a timing chart when GPS information and images are acquired. The calculation of the position information of the latitude and longitude of each image will be described with reference to FIG.
When acquiring GPS information and images, vehicle speed data of travel distance and recording time, thermal image data of thermal image and shooting time, and latitude and longitude and GPS data of reception time are sequentially acquired. ing. Note that these acquisition timings do not have to be linked in particular, but any two of them may be simultaneous. In addition, there may be kilopost (KP) information in the route. This kilopost information also includes latitude and longitude information.
As shown in FIG. 5, the photographing time t12 of the thermal image 1 does not coincide with the acquisition timing of GPS data. Therefore, the latitude and longitude of the shooting time t12 can be calculated using the latitude and longitude 1 of the reception time t1 and the latitude and longitude 2 of the reception time t2 when the GPS information before and after the shooting time t12 is acquired. That is, position information calculation is performed by interpolation calculation based on the time between each data.

In the above case, it is only necessary to move at a constant speed between the reception times t1 and t2 of the GPS data. However, when the speed or direction is changed, the interval between the reception times t1 and t2 of the GPS data is If it is vacant, the latitude and longitude at the photographing time t12 calculated may be significantly shifted.
On the other hand, the vehicle speed data of the travel distance and the recording time is recorded more densely. Therefore, the latitude and longitude information of the thermal image 1 can be obtained by the following two steps.
i) First, the latitude and longitude information of the recording times T1 and T2 of the vehicle speed data before and after the photographing time t12 of the thermal image 1 is calculated. For example, the latitude and longitude information at the recording time T1 is calculated based on the latitude and longitude information at the respective reception times when the GPS information was acquired before and after the recording time T1. Further, the latitude and longitude information of the recording time T2 is calculated based on the latitude and longitude information of the respective reception times when the GPS information is acquired before and after the recording time T2.
ii) Next, the latitude and longitude information of the thermal image 1 is calculated based on the calculated latitude and longitude information of the recording times T1 and T2.
In this case, the travel speed is included in the vehicle speed data, and the latitude and longitude information of the shooting time of the thermal image 1 can be obtained reflecting the change in speed and the like.
In addition, you may use kilopost (KP) information as needed. As described above, based on the vehicle speed data, thermal image data, and GPS data, the mileage at the shooting time of each image, the latitude, and the longitude are mutually converted, so that the latitude and Longitude information can be obtained.

<About image layout>
FIG. 6 is a schematic view when the infrared image of the image is linearly arranged. FIG. 7 is a schematic diagram when an infrared image is arranged along the position information. FIG. 8 is a schematic diagram when an infrared image of an airport runway is arranged along position information. FIG. 9 is a partially enlarged view of the airport runway of FIG.
For example, the images may be arranged in a straight line as shown in FIG. Alternatively, as illustrated in FIG. 7, each image may be arranged on an aerial photograph, CAD data, or a map, for example, according to the position information of each image. Since each image has position information and a shooting direction, for example, when a series of images shot while traveling on a highway is arranged, the images are arranged along the alignment of the actual road as shown in FIG. .
Further, as shown in FIGS. 8 and 9, each image may be arranged on a two-dimensional plane. In this case, the overlapping portion of each image may be simply deleted. Alternatively, the overlapping portions may be averaged and used. Note that the accuracy of GPS is several millimeters when the accuracy is high, and the relative error is small, so that each image can be arranged accurately.

<Image placement device>
FIG. 2A is a block diagram illustrating a functional configuration of the image arrangement apparatus 20 according to the first embodiment. FIG. 2B is a block diagram illustrating a physical configuration of the computer 10 which is an example of the image layout apparatus 20 in FIG.
From a functional viewpoint, the image arrangement device 20 includes GPS information and image acquisition unit 11, position information calculation unit 12, tilt correction unit 13, rotation correction unit 14, and image arrangement unit 15. Each component corresponds to each step of the image arrangement method. That is, the GPS information and image acquisition unit 11 acquires the GPS information including the received latitude and longitude sequentially recorded while moving, the reception time, the sequentially recorded captured image, and the captured time of each image ( prepare. Further, based on the shooting time of each image, the two reception times at which GPS information was acquired before and after the shooting time, and the GPS information at the two reception times, respectively, by the position information calculation unit 12. The position information of latitude and longitude is calculated. The tilt correction unit 13 corrects each image and converts the image into a directly-facing image. The rotation correction unit 14 calculates the shooting direction of each image based on GPS information before and after the shooting time of each image. Further, the image placement unit 15 places each image based on position information including the latitude and longitude of each image and the shooting direction.

  The image arrangement device 20 can cause a computer to function as an image arrangement device by operating a program on a general computer. In this case, the general computer 10 includes, for example, a CPU 1, a memory 2, an input / output unit 3, a storage unit 4, a display unit 5, and an interface 6.

<GPS information and image acquisition vehicle>
FIG. 3 is a block diagram illustrating an example of the GPS information and image acquisition vehicle 30 corresponding to the GPS information and image acquisition unit 11 of FIG.
The GPS information and image acquisition vehicle 30 includes a camera 21, a photographing unit 22, a recording unit 23, a GPS antenna 24, a GPS information receiving unit 25, and a control unit 26. The GPS information and the image acquisition vehicle 30 can sequentially record the captured image and the captured time of each image while moving, and sequentially record the received GPS information including the latitude and longitude and the received time.

<Thermal image>
FIG. 4 is a schematic diagram when the image is a thermal image. The thermal image is data holding a value corresponding to the temperature for each pixel. The temperature of the object to be photographed can be obtained by converting the value for each pixel into a temperature unit such as Celsius or Fahrenheit using a specific conversion formula. In the case of displaying as an image, the temperature information is displayed in color corresponding to the color information. In FIG. 4, for the sake of convenience, three types of temperature information are associated with shaded shades. The darkest hatch represents high temperature, the middle hatch represents medium temperature, and no hatch represents low temperature. Therefore, temperature information can be obtained visually by color display of a thermal image.

<Computer program for image placement>
The computer program for image arrangement causes the computer to execute the following steps to lay out each image.
(A) GPS information including latitude and longitude sequentially recorded while moving, its reception time, sequentially recorded captured images, and capture times of the respective images are prepared (S01).
(B) The position of the latitude and longitude of each image based on the shooting time of each image, the two reception times prepared for GPS information before and after the shooting time, and the GPS information of the two reception times, respectively. Information is calculated (S02).
(C) The tilt correction of each image is performed and converted into a directly-facing image (S03).
(D) The shooting direction of each image is calculated based on GPS information before and after the shooting time of each image (S04).
(E) Each image is arranged based on position information including the latitude and longitude of each image and the shooting direction (S05).

  Further, the image layout computer program may be stored in a computer-readable recording medium.

  It should be noted that the present disclosure includes appropriately combining any of the various embodiments and / or examples described above, and each of the embodiments and / or examples. The effect which an Example has can be show | played.

  According to the image arrangement method of the present invention, when an image is a thermal image, each thermal image can be arranged while maintaining the temperature information of the thermal image, which is particularly useful for the arrangement of the thermal image.

1 CPU
2 memory 3 input / output unit 4 storage unit 5 display unit 6 interface 10 computer 11 GPS information and image acquisition unit 12 position information calculation unit 13 tilt correction unit 14 rotation correction unit 15 image arrangement unit 20 image arrangement device 21 camera 22 photographing unit 23 Recording unit 24 GPS antenna 25 GPS information receiving unit 26 Control unit 30 GPS information and image acquisition vehicle

Claims (8)

Step of preparing GNSS information including latitude and longitude sequentially recorded while moving and its reception time, sequentially recorded photographed images and photographing times of the respective images, photographing times of the respective images, and the GNSS information And determining the position information of each image based on the reception time thereof;
Converting each image into a facing image;
Calculating the shooting direction of each image based on each GNSS information recorded at times before and after the shooting time of each image;
Arranging each of the images based on the positional information and the shooting direction of each of the images;
A method for arranging images.   The image arrangement method according to claim 1, wherein the image is a thermal image including temperature information.   In the step of preparing the GNSS information sequentially recorded while moving, the reception time thereof, the captured images sequentially recorded and the photographing times of the respective images, further, traveling at each time point recorded sequentially while moving The image arrangement method according to claim 1, wherein a distance and a recording time thereof are prepared. In the step of determining the position information of each image,
A sub-step of acquiring each GNSS information recorded at times before and after the photographing time of each image;
A sub-step of calculating position information including the latitude and longitude of each image based on the shooting time of each image, each acquired GNSS information and its reception time;
The image arrangement | positioning method as described in any one of Claim 1 to 3 containing these.   5. The image arrangement method according to claim 1, wherein, in the step of arranging the images, the images are arranged in a straight line.   5. The image arrangement method according to claim 1, wherein, in the step of arranging the images, the images are arranged on a plane. Step of preparing GNSS information including latitude and longitude sequentially recorded while moving and its reception time, sequentially recorded photographed images and photographing times of the respective images, photographing times of the respective images, and the GNSS information And determining the position information of each image based on the reception time thereof;
Converting each image into a facing image;
Calculating the shooting direction of each image based on each GNSS information recorded at times before and after the shooting time of each image;
Arranging each of the images based on the positional information and the shooting direction of each of the images;
The computer program for image arrangement which makes a computer perform each step of and arrange | positions each image.   A computer-readable recording medium storing the image arrangement computer program according to claim 7.

Images