JP6944560B2 - Tunnel image processing equipment and programs - Google Patents

Description

The present invention relates to a tunneled image processing apparatus and a program.

There is a technology to detect abnormalities based on the photographed data obtained by photographing the wall surface of a tunnel. Patent Document 1 discloses a technique for identifying the position of a noteworthy event in a captured image by polygonizing the wall surface of the tunnel based on the feature points of the captured image from a plurality of positions (viewpoints).

In addition, the position-specified shooting data can be displayed two-dimensionally as a developed view. Normally, the distance from the viewpoint to the tunnel wall surface is non-uniform, so simply connecting a plurality of captured images will increase the distortion. In Patent Document 1, the two-dimensional coordinates on the development view are obtained by being projected from the three-dimensional coordinates while considering the viewpoint position.

International Publication No. 2018/155590

However, if the wall shape of the tunnel is simply projected onto a two-dimensional surface with the shooting position as the viewpoint position, the line-of-sight direction changes according to the position of the curved tunnel, and the appearance of unevenness becomes a place. There is a problem that it is difficult to see differently depending on the situation.

An object of the present invention is to provide a tunneled image processing device and a program capable of obtaining a developed view that is easier to see.

To achieve the above objectives, this disclosure is:
Based on the position information of the tunnel wall surface obtained from the plurality of captured images of the tunnel wall surface, the wall surface is divided into a plurality of setting ranges independent of the photographing range of the plurality of captured images, and each of the setting ranges is divided. A tunnel comprising an image processing unit that generates a predetermined projected image by determining each viewpoint position according to the set range and generates a developed view of the wall surface by the projected images of the plurality of the set ranges. It is a captured image processing device.

According to the present disclosure, there is an effect that it is possible to obtain a more visible development view of the photographed image of the wall surface of the tunnel.

It is a block diagram which shows the functional structure of the processing apparatus which is a tunnel photograph image processing apparatus of this embodiment. It is a figure explaining the generation of the development view. It is a figure explaining the generation of the development view. It is a flowchart which shows the control procedure of the tunnel development drawing generation processing. It is a figure explaining the modification of the generation of the development view.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a functional configuration of a processing device 1 which is a tunnel-photographed image processing device (computer) of the present embodiment.

The processing device 1 may be an ordinary computer (PC). The processing device 1 includes a control unit 11 (image processing unit), a storage unit 12, an input / output interface 13 (I / F), an operation reception unit 14, a display unit 15, and the like.

The control unit 11 is a processor that performs various arithmetic processes, and includes, for example, a CPU (Central Processing Unit). The CPU performs various control processes by reading a program or the like from the storage unit 12 and executing the program.

The storage unit 12 is a memory for storing various data. The memory includes a RAM (Random Access Memory) and a non-volatile memory. The RAM provides the CPU with a working memory space and stores temporary data. The non-volatile memory stores and holds programs, setting data, generated tunnel development map data, and the like. The non-volatile memory may include a flash memory, or may include an HDD (Hard Disk Drive) or the like in place of or in addition to the flash memory. Further, the initial control program and the like may be stored in the ROM (Read Only Memory). The program includes a program 121 related to a tunnel development diagram generation process described later.

The input / output interface 13 includes a configuration for connecting to various devices having an external data storage unit 20 for storing various external data accessed by the processing device 1, here, a connection terminal 131, a communication unit 132, and the like. The connection terminal 131 may include, for example, one corresponding to various standards such as USB (Universal Serial Bus). Auxiliary storage devices such as large HDDs and peripheral devices such as optical readers 22 that read portable storage media such as optical disks such as CDROMs, DVDs and Blu-rays (registered trademarks) are connected to the connection terminal 131. Further, the portable storage medium may include a magnetic tape, and a reading device for reading the magnetic tape may be included in the peripheral device. The communication unit 132 controls communication with an external device via a network line, for example, based on a communication standard such as a LAN (Local Area Network). The external device may include another PC, a dedicated database device 21, and the like.

The data stored in the external data storage unit 20 may include captured image data 201, captured position data 202, tunnel wall surface image data 203, and the like.

The photographed image data 201 is image data of a wall surface in a tunnel photographed by a photographing device, here, an optical camera. The shooting position data 202 includes information indicating the shooting position related to the direction of the optical camera (elevation angle θ, see FIG. 2) and the extending direction of the tunnel. For example, prepare a plurality of optical cameras oriented in a direction perpendicular to the extending direction of the tunnel, arrange them at different elevation angles θ, and move the optical cameras in the extending direction of the tunnel to take pictures at predetermined intervals. To obtain captured image data of the entire wall surface of the tunnel. The shooting range of the optical cameras having the shooting ranges of adjacent elevation angles θ and the shooting range of the timing before and after by the same optical camera both have overlapping ranges. In this case, the shooting position data 202 may include information such as the shooting date and time. The information indicating the shooting position is three-dimensional coordinates, and the three-dimensional coordinates are, for example, values of latitude, longitude, and altitude, or values of the distance of the first component with the northward direction positive from a predetermined origin position. It is the value of the distance of the second component with the eastward direction positive from the predetermined origin position, and the value of the distance of the third component with the vertical upward direction positive from the predetermined origin position. The unit of the altitude value of the former and the distance value of each component in the latter case is, for example, "m".

Since the wall surface condition of the tunnel usually does not change in a short time, the left side and the right side of the tunnel may be photographed separately at different timings in the extending direction of the tunnel. For example, the photographed data of the wall surfaces on both sides may be acquired by reversing the direction of the traveling vehicle capable of photographing the left side in the traveling direction and reciprocating.

Further, the tunnel wall surface image data 203 includes the three-dimensional shape model data of the tunnel generated in the process described later based on the captured image data 201 and the captured position data 202, and the position data of each pixel associated with the three-dimensional shape. , Includes development view data.

The operation reception unit 14 receives an operation from the outside such as a user and outputs it as an input signal to the control unit 11. The operation reception unit 14 may include, for example, a keyboard, a pointing device, a touch panel, a push button switch, and the like, and is not limited thereto.

The display unit 15 causes the display screen to display the contents based on the control of the control unit 11. Examples of the display screen include a liquid crystal screen. Further, the display unit 15 may have an LED (Light Emitting Diode) lamp or the like indicating various states.

Next, the generation of the tunnel development diagram of the present embodiment will be described.
First, three-dimensional wall surface data (tunnel wall surface image data 203) is generated from the photographed image data (photographed image data 201) of a plurality of wall surfaces of the tunnel and the data of the position where the image was taken (photographed position data 202). The three-dimensional wall surface data is obtained by, for example, SfM (Structure from Motion). In SfM, a three-dimensional positional relationship is specified based on the features of overlapping portions in a plurality of shooting data, and three-dimensional coordinates of each feature point are determined. The three-dimensional wall surface data may be appropriately divided into a plurality of pieces in the extending direction of the tunnel. Then, a two-dimensional development view is obtained based on the three-dimensional wall surface data.

2 and 3 are diagrams for explaining the generation of the development view. FIG. 2 shows a cross section of the tunnel T. In FIG. 3, of the cross section of the tunnel T, the portion corresponding to two of the separated wall surface portions Ws is enlarged and shown.

As shown in FIG. 2, the wall surface W of the tunnel T is usually curved at least above (near the ceiling). The height-to-width ratio varies from tunnel to tunnel and can vary in shape depending on their size. The wall surface W of the tunnel T may have a recess for emergency evacuation and for arranging equipment in the tunnel. Further, lights, signs, cables and the like located along the wall surface W of the tunnel T may be regarded as protruding portions of the wall surface W.

As described above, the captured image data 201 of the wall surface W includes a plurality of optical cameras oriented in a direction perpendicular to the extending direction of the tunnel T, each having an elevation angle θ (angle from the horizontal plane, even if it is an angle from the bottom surface). It can be obtained by arranging the vehicles in different directions and taking pictures at appropriate intervals while traveling the vehicle in the extending direction of the tunnel T. In this case, the distance from the shooting position P to the wall surface W and the angle formed by the line-of-sight direction and the wall surface W change according to the shooting direction.

Therefore, in the present embodiment, the wall surface W of the tunnel T obtained by the above SfM is divided into a plurality of parts for each predetermined angle range a in the elevation angle θ direction. Next, as shown in FIG. 3, the line-of-sight direction according to each divided wall surface portion Ws (setting range), here, the position facing the plane (reference plane Wr, thick solid line) that defines each wall surface portion Ws. Two-dimensional image data (projection data) of the projected image seen from (projected on the opposite plane) is generated. Here, for example, data of a two-dimensional image projected in parallel (orthogonal projection) from a plane Wp that is parallel to the reference plane Wr is generated. For simplification of processing, the reference plane Wr is set so that the viewpoint position (plane Wp) is inside the tunnel T (positive position in the inward direction from the wall surface W of the tunnel T) with respect to all the points of the wall surface portion Ws. The distance between the surface Wp and the surface Wp may be set to a specific value. That is, as a result, for each wall surface portion Ws, the line-of-sight direction is changed from the shooting position to the direction in which the wall surface portion Ws is viewed, in particular, here, the viewpoint is changed to the direction perpendicular to the reference plane Wr, and the viewpoint is different from the shooting position. It becomes a projected image at the position. In addition, in the setting of the angle range a of the present embodiment, all the viewpoint positions are changed, but when the angle range a is originally determined by the division method including the portion whose shooting direction is perpendicular to the wall surface W, a part of the angle range a is set. As a result, the viewpoint position and the line-of-sight direction in the angle range a may not change. The process of determining the line-of-sight direction itself may be performed in the same manner as described above.

Then, the images of these projection data are connected to generate a development view. The angle range a that defines the wall surface portion Ws is defined within a range in which the deviation between the curved wall surface portion Ws and the reference plane Wr is sufficiently small, and here, for example, is 10 degrees. That is, the angle range a may be set independently of the number of optical cameras and the like. Alternatively, the number of wall surface portions Ws to be divided may be an integral multiple of the number of optical cameras. The angle range a may be changeable depending on the shape of the tunnel and the like.

By generating such a developed view, the appearance of the above-mentioned dents and protrusions is unified in the developed view.
As described above, in the wall surface portion Ws where the distance from the shooting position P to the wall surface W of the tunnel T is large, the area captured in the shooting area increases, so that the size of the surface Wp also increases. On the other hand, the resolution in the shooting area decreases relative to the size. Along with this, the resolution of the developed view also differs depending on the location. The image data may be adjusted by, for example, linear interpolation so that the actual area indicated by one pixel is the same.

The reference plane Wr may usually be a rectangular plane specified at the four corners of the divided wall surface portions Ws. Alternatively, when a part of the four corners corresponds to the above-mentioned protrusions or dents, the corresponding points may be excluded, or other surrounding points, the average position, or the like may be used.

FIG. 4 is a flowchart showing a control procedure by the control unit 11 of the tunnel development diagram generation process executed by the processing device 1. This tunnel development map generation process is started, for example, based on the designation of the photographed image data 201 and the photographed position data 202 of the wall surface W of the tunnel T and the input operation of the start command to the operation reception unit 14 by the user or the like.

The control unit 11 acquires the captured image data 201 and the captured position data 202 from the external data storage unit 20 (step S1). The control unit 11 uses the SfM model to three-dimensionally model the tunnel shape based on the feature points of the captured image. Further, the control unit 11 specifies the three-dimensional coordinates of each pixel of the shooting data (step S2). The three-dimensional model data may be stored in the external data storage unit 20 as the tunnel wall surface image data 203 via the input / output I / F 13.

The control unit 11 divides the wall surface W of the modeled tunnel T into predetermined angular ranges (step S3). The control unit 11 sets the reference plane Wr of the divided wall surface portion Ws (step S4). The control unit 11 determines the line-of-sight direction perpendicular to the reference plane Wr, and generates a projected image of the wall surface portion Ws by normal projection (parallel projection) along the line-of-sight direction (step S5).

The control unit 11 joins the projected images adjacent to each other in the elevation angle direction to generate a development view at a predetermined position in the extending direction of the tunnel (step S6). In addition, this development map is generated at each position in the extending direction of the tunnel and connected. The control unit 11 stores the obtained development drawing data in the external data storage unit 20 via the input / output I / F 13. Then, the control unit 11 ends the tunnel development diagram generation process.
Of the tunnel development map generation processes, at least the processes of steps S3 to S6 constitute the image processing means in the program of the present embodiment.

FIG. 5 is a diagram illustrating a modified example of generating a development view. FIG. 5 shows the same part as that of FIG.

In the above embodiment, the plan view of each wall surface portion Ws is generated by parallel projection (normal projection), but the plan view may be generated by perspective projection. In this case, the viewpoint position Pp passes through the center position of the reference plane Wr, that is, the center of gravity positions Pg of the points at the four corners of the rectangular reference plane Wr, and is on a line perpendicular to the reference plane Wr. The center of gravity position Pg may be obtained, for example, by the intersection of the diagonal lines at the four corners, or the middle line between the upper side and the lower side (for example, the line whose distances from the upper side and the lower side of the reference plane Wr1 in FIG. 5 are d1 respectively, and It may be obtained from the intersection of the middle line between the left side and the right side, etc.) and the line where the distance from the upper side and the lower side of the reference plane Wr2 is d2. Alternatively, the viewpoint position Pp has a predetermined angle formed by sandwiching the two opposing sides with the reference plane Wr (for example, in FIG. 5, the two angles b1 related to the reference plane Wr1 are the same. , The two angles b2 relating to the reference plane Wr2 are the same.) It may be obtained based on the intersection of two lines Lf (two surfaces including the line Lf).

The distance between the reference plane Wr and the viewpoint position Pp may be an appropriate uniform value, and similarly to the above, the distance is inside the tunnel T in the perpendicular direction of the reference plane Wr with respect to all the points of the wall surface portion Ws. May be determined as. For example, in particular, this distance may be set to a value of about half the width of the tunnel T at the bottom surface.

As described above, the processing device 1 of the present embodiment includes the control unit 11. As an image processing unit, the control unit 11 divides the wall surface W into each wall surface portion Ws based on the position information of the wall surface W of the tunnel T obtained from a plurality of captured images obtained by photographing the wall surface W of the tunnel T. A normal projection image is generated by determining a viewpoint position according to the wall surface portion Ws, and a developed view of the wall surface W of the tunnel T is generated by combining (joining) the projected images of the plurality of wall surface portions Ws.
The viewpoint position according to the wall surface portion Ws here means that it is determined due to the orientation of the wall surface portion Ws (reference plane Wr) and the like, and the viewpoint position such as the shooting position with respect to the wall surface W is predetermined. Doesn't mean that.
By such a process, the line-of-sight direction of the developed view of the curved wall surface W can be adjusted to be easily seen, and in particular, the developed view can be easily obtained. Further, since the same line-of-sight direction is set for each appropriate wall surface portion Ws, a developed view can be obtained by a simpler process than mapping each point individually to the projection surface according to the positional relationship with the shooting position. be able to.

Further, the line-of-sight direction from the determined viewpoint position to the wall surface W of the tunnel T is a direction perpendicular to the reference plane Wr that defines the wall surface portion Ws at the center of the developed view. That is, since the image projected from the position facing each wall surface portion Ws is obtained, the appearance of the uneven portion of the wall surface W can be made uniform. Therefore, it is easy to make a stable judgment by analysis and diagnosis using the development drawing obtained by the processing device 1.

Further, the control unit 11 generates a parallel projection image with the position facing the wall surface portion Ws as the viewpoint position for each wall surface portion Ws. By projecting not only the center position but the entire wall surface part Ws in parallel, it is possible to obtain a developed view with a more uniform appearance, so that the user of the developed view can see the wall surface state more easily, and it is uniformly abnormal. It is easy to judge.

Further, the viewpoint position is set to a positive position in the inward direction from the wall surface W of the tunnel T. By setting the viewpoint position so as not to be seen from the back side of the wall surface W at the time of projection, the projected image can be obtained by easier processing.

Further, by installing the program 121 of the present embodiment on a computer and causing the CPU of the control unit 11 to perform processing as an image processing means by software, a tunnel that is easier to see without preparing special hardware or the like can be obtained. A development view can be generated.

The present invention is not limited to the above embodiment, and various modifications can be made.
For example, in the above embodiment, the wall surface W of the tunnel T is divided for each angle range a, but it is not always necessary to divide the wall surface W for each angle range a. It may be separated based on the height (length), or the angle range a may not be a constant value.

Further, the program related to SfM that generates the three-dimensional wall surface data of the tunnel may not be included in the program 121. The three-dimensional wall surface data generated by the external program may be acquired, and only the process related to the generation of the development drawing based on the three-dimensional wall surface data may be performed by the program 121.

Further, the three-dimensional wall surface data may be obtained not only by SfM but also by another method as long as it can be appropriately obtained.

Further, when performing perspective projection, the viewpoint position Pp is not limited to the center position of the wall surface portion Ws. There may be some deviation from the center position. Further, in the case of parallel projection, if the line-of-sight directions are properly aligned, the line-of-sight direction is not always perpendicular to the reference plane Wr.

Further, the reference plane Wr does not have to be determined based on the points at the four corners. The reference plane Wr may be recursively determined from the points within the range surrounded by the four corner points. In this case, the points at the four corners do not have to be on the reference plane Wr.

Further, in the above embodiment, it has been described that the image is taken at the center in the width direction of the tunnel cross section, but the present invention is not limited to this. For example, the image may be taken at a position closer to the wall surface W of the tunnel T than the center.

Further, in the above embodiment, the viewpoint position is placed inside the tunnel T to obtain a projected image of the wall surface W, but if the processing for appropriately displaying the wall surface W is performed, the viewpoint position is not necessarily the tunnel. It does not have to be located inside the T.

Further, in the above embodiment, the processing is described as being executed entirely by the CPU of the control unit 11 based on the program 121, but a part of the processing may be executed by a dedicated hardware circuit or the like.

Further, in the above description, as a computer-readable medium for storing the program 121 related to the tunnel development diagram generation process, a storage unit 12 having a non-volatile memory including a flash memory and an HDD has been described as an example. Not limited to. As another computer-readable medium, a portable storage medium such as a CD-ROM or a DVD disc can be applied. A carrier wave is also applied to the present invention as a medium for providing data of a program according to the present invention via a communication line.
In addition, the specific configuration, the content and procedure of the processing operation, etc. shown in the above-described embodiment can be appropriately changed without departing from the spirit of the present invention. The scope of the present invention includes the scope of the invention described in the claims and the equivalent scope thereof.

1 Processing device 11 Control unit 12 Storage unit 121 Program 13 Input / output interface 131 Connection terminal 132 Communication unit 14 Operation reception unit 15 Display unit 20 External data storage unit 201 Photographed image data 202 Photographed position data 203 Tunnel wall surface image data 21 Database device 22 Optical reader P Imaging position Pp Viewpoint position T Tunnel W Wall surface Wr, Wr1, Wr2 Reference plane Ws Wall surface part a Angle range

Claims (5)

Based on the three-dimensional position information of the wall surface obtained from the plurality of captured images obtained by photographing the wall surface of the tunnel, the wall surface is divided into a plurality of setting ranges independent of the photographing range of the plurality of captured images, and the setting range is described. It is characterized by including an image processing unit that generates a predetermined projected image by determining each viewpoint position according to the set range for each and generates a developed view of the wall surface by the projected images of the plurality of the set ranges. Tunnel image processing equipment. The tunnel photographing image processing apparatus according to claim 1, wherein the line-of-sight direction from the determined viewpoint position to the wall surface is a direction perpendicular to a plane defining the setting range at the center of the developed view. The tunnel image processing apparatus according to claim 1 or 2, wherein the image processing unit generates a parallel projection image having a position facing the set range as the viewpoint position for each set range. The tunnel photographing image processing apparatus according to any one of claims 1 to 3, wherein the viewpoint position is set to a positive position with respect to the inward direction of the tunnel from the wall surface. Computer,
Based on the three-dimensional position information of the wall surface obtained from the plurality of captured images obtained by photographing the wall surface of the tunnel, the wall surface is divided into a plurality of setting ranges independent of the photographing range of the plurality of captured images, and the setting range is described. It is characterized in that a predetermined projection image is generated by determining each viewpoint position according to the setting range for each, and the projection images in the plurality of setting ranges are used as an image processing means for generating a developed view of the wall surface. Program to be.

Images