JP3415000B2 - Pipe development system - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for developing the inner surface of a sewer from a video data obtained by photographing the inner wall surface of the sewer. It is about. 2. Description of the Related Art Conventionally, inspection of a tunnel (sewer) wall surface has been performed.
The main work involved drawing visual changes while walking in a dark, long tunnel on foot, and creating a development map based on this. In such a method, since the inspection target is a large structure and there are many long and large tunnels, not only a lot of manpower and time are required, but also it is difficult to accurately detect all the deformations. [0003] Therefore, a method as shown in Figs. 7, 8 and 9 has been developed and reported as an apparatus for developing an inner surface of a sewer for continuous inspection. The device shown in FIG.
As shown in (a), four line sensor cameras 12 are located at substantially the center of a tunnel (sewer) 10 to be inspected.
a, 12b, 12c, and 12d are set. Each line sensor camera 12a, 12b, 12c, 12d
Are set in different directions and the projection angle θ
a, θb, θc, and θd, and the entire circumference of the wall surface 11 is divided into four parts for photographing. At this time, as shown in (b), the four wall images 13a, 13b, 13c, 13d
Are set so as to have a few overlapping portions 15 in the circumferential direction. Four line sensor cameras 12a, 12
After the entire circumference is photographed at b, 12c, and 12d, the entire circumference is similarly photographed while moving by a fixed distance in the length direction of the tunnel (sewer) 10. Also at this time, the image is taken so as to partially have the overlapping portion 15 in the length direction. From the wall images 13a, 13b, 13c, and 13d photographed in this manner, cracks 14 and other leaks, peeling, joint breaks, and the like are extracted and displayed as deformations of the wall 11 by the image processing device. The device shown in FIG. 8 is, as shown in FIG.
At the approximate center of the tunnel 10 to be inspected,
A rotary panoramic camera 17 is set on a carriage that moves on a rail 18. When the rail 18 is a single track, it is set so as to be located above the rail 18 and substantially at the center of the tunnel (sewer) 10. Is set above the middle of the tunnel and substantially at the center of the tunnel (sewer) 10. In such a configuration, first, as shown in the flowchart of (b), the rotary panoramic camera 17 uses a 360-degree rotation type, and divides the entire circumference by a predetermined angle in one photographing to form the wall surface 11. Take pictures continuously. After shooting once, move several meters and shoot in the same way. Next, a wall picture is recorded on the optical disk. Based on the recorded wall photos, photos developed by computer,
Create and manage development drawings. At this time, the developed picture and the developed view are merely connected, and no processing is performed. Next, a developed photograph, a developed view, attribute data, and the like are searched and output. From the developed photograph and the developed view obtained in this way, as the deformation of the wall surface 11, cracks 14 and others,
Extracts water leakage, peeling, joint breaks, etc. The device shown in FIG. 9 is, as shown in FIG.
A probe 22 is suspended by a winch unit so as to be movable up and down substantially at the center of a boring hole (drain) 21 to be inspected.
3. The cone mirror 24 and the internal control unit 25 are set. In such a configuration, light rays incident from the wall surface 11 of the borehole 21
At 4, the light is reflected and refracted and input to the CCD camera 23. This C
The image of the CD camera 23 works so as to converge toward the center axis of the torus as the probe 22 descends. A / D conversion of the entire circumference image and frame memory A2 as shown in (b)
9, the image is handled for each pixel. That is, the pixel group corresponding to the circumference R in the frame memory A29 is rearranged linearly in the frame memory B30 for each pixel as shown in FIG. , A pixel array of a minute section of the wall surface 11 corresponding to the circumference R is obtained. By the lowering of the probe 22, the above operation is repeated to obtain a continuous developed image in the frame memory B30. [0006] The apparatus shown in FIG. 7 requires four expensive wide-angle cameras, four cameras such as a camera position, a camera direction, and a camera moving speed. It is extremely troublesome to set all the shooting conditions of the camera to be constant, and if the distance from the wall surface 11 to the position of the camera is too short, a wide-angle lens is required, and the image is distorted and cannot be connected well, If the distance from the wall surface 11 to the position of the camera is increased, a wide-angle lens is not required, but the opposing camera interferes with the captured image,
Had such a problem. [0007] The apparatus shown in FIG.
A special and expensive camera that can continuously photograph the wall surface 11 by dividing the entire circumference by a predetermined angle in each shooting, requiring only a large number of photos without processing, There is a problem that it is difficult to know the exact position of the crack 14. The device shown in FIG. 9 is a mechanism for suspending a probe 22 at a substantially central part of a sewer 21 so as to be vertically movable by a winch unit. The method of use is limited. Has required special parts such as a CCD camera 23, a cone mirror 24, and an internal control unit 25, and had problems such as being expensive and cumbersome to operate. Further, in each case, developed images or developed drawings are simply connected or developed images are formed by using a special device, and there is a problem that the devices are complicated and expensive. An object of the present invention is to obtain a continuous development view from video data obtained by photographing the inside of a sewer with a general-purpose device. [0011] The present invention relates to a sewer wall 11.
And a video data input device 36 for taking in the image data and audio data of the image data, and developing processing of a development view for each frame based on the image data of the video data input device 36, and then synthesizing image data for a plurality of frames. A development processing unit 37, an audio processing unit 38 for processing the audio data of the video data input device 36 and sending the processed data to the development processing unit 37, and a development output unit 39 for outputting a combined development view. Things. The video data input device 36 captures image data of the culvert wall 11 on the video deck 40 and voice data of an operator corresponding to the image data. The voice processing device 38 stores the voice data of the video data input device 36, recognizes voice from the voice data, determines whether or not the culvert wall 11 is damaged, converts it into character information, and outputs it. The developed view processing device 37 includes a video data input device 36.
From the stored data, creates a development view for each frame, performs data matching for adjusting distortion between image data for each frame, and recognizes and develops characters from the image data. The character information to the figure is output, and the character information from the image data and the character information for the damage determination from the sound processing device 38 are added to create a development view by synthesizing the image data for a plurality of frames. DETAILED DESCRIPTION OF THE INVENTION Principle of the present invention (1) Actual position of wall surface 11 of tunnel (drain) 10,
Correspondence with the coordinate data of the developed view When the video camera 35 is set at the center of the inside of the tunnel (sewer) 10 and toward the center line S, the position of the wall surface 11 of the tunnel (sewer) 10 and the developed view are shown. The correspondence with the coordinate data is defined as shown in FIG. That is, the actual image between L1 of the wall surface 11 of the tunnel (drain) 10 in (a) is an image of one frame as shown by the video camera 35 as shown in (b), and this image is A development developed in the length direction of the culvert 10 is as shown in FIG. In the present invention, the coordinate point (n, n) on the developed view corresponding to the coordinate point (x, y) on the real image data shown in (b)
m) is obtained by a developing process, and a developed view is created from the obtained coordinate data. 1 (a) and 1 (b)
In (c), 14a indicates a crack in the actual tunnel (sewer) 10, 14b indicates a crack in an image projected by the video camera 35, and 14c indicates a crack in a developed view. (2) Image Data Expansion Processing The image data captured by the video camera 35 is 1
Expand every frame. The center point of the tunnel (drain) 10 is extracted by image processing, and the developed image data is created by the following equation. According to this method, image data after expansion can be easily created by inputting the following parameters shown in FIG. 2 in advance. Parameters • Conversion start point radius r • Conversion end point radius R • Distance between focal point and conversion start point L0 • Conversion distance (distance between conversion start point and conversion end point) L1 • Image data size after conversion M × N However, the circumference of the tunnel (sewer) 10 is divided into M equal parts and the vertical axis of the development diagram is used, and the distance L1 between the conversion start point and the conversion end point is N.
Divide equally and use it as the horizontal axis of the developed view. The focus of the video camera 35 is at the center of the tunnel (drain) 10, and the coordinate value on the image is (x0, y0). Under the above conditions, the coordinates (x, y) of the image data corresponding to the coordinates (n, m) of the developed view can be obtained by the following procedure. Formula: The image projection radius Rn of the n-th cross section is obtained. Since the distance between the n-th section and the focal point is L0 + L1 × n / N, the radius Rn of the circle projected on the image is obtained from the following equation. (L0 + L1) / R = L0 / r L1 = L0 × (R−r) / r (L0 + L1 × n / N) / R = L0 / Rn Rn = R × L0 / (L0 + L1 × n / N) = R / [ 1 + {(R−r) / r} × (n / N)] Expression: The coordinates (x, y) of the image data corresponding to the coordinates (n, m) of the developed view are obtained. Since the circumferential direction is divided into M equal parts, θ can be expressed by the following equation. θ = 2π × m / M The coordinates (x, y) of the image data are as follows: x = x0 + Rn × sin θ = x0 + R × sin θ / [1 + {(R−r) / r} × (n / N)] y = y0 + Rn × cos θ = y0 + R × cos θ / [1 + {(R−r) / r} × (n / N)] (3) Correction processing of image data As shown in FIG. The axis is
It is not necessarily parallel to the longitudinal direction of the conduit of the tunnel (drain) 10. Therefore, the image data on the image plane of the video camera 35 includes distortion. Therefore, distortion is removed by performing coordinate conversion on a virtual imaging plane perpendicular to the longitudinal direction of the pipeline and then performing expansion processing. This will be described with reference to FIG. Parameters • Angle of undulating rotation α • Angle of left / right rotation β • Coordinates on the image plane of video camera 35 (x ′, y ′) • Coordinates on virtual image plane (x, y) • f: Video camera 35 The focal length is set, and coordinate conversion is performed by the following equation. x ′ = f × tan (β + arctan x / f) y ′ = f × tan (α + arctan y / f) Next, an embodiment of a specific apparatus according to the present invention will be described with reference to FIG. The apparatus of the present invention includes a video data input device 36, a development processing device 37, and an audio processing device 3.
8. It is composed of four parts of the development view output device 39. The video data input device 36 comprises a video deck 40 and a video capture board 41. In addition to image data indicating the condition of the wall surface 11 of the tunnel (drain) 10, the video data input device 36 also displays the deterioration condition and other conditions of the wall surface 11. The voice data of the operator to be explained is taken in to supplement the image data. The development map processing device 37 stores image data from the video data input device 36, and recognizes characters from the image data in the image data storage device 42, and outputs character information to the development map. A character recognition processing unit 56, a correction processing unit 43 for correcting distortion of the stored data due to an optical axis shift of the video camera 35 of the video data input device 36, and 1 from the stored corrected data.
A development processing unit 44 for performing development processing of a development view for each frame;
A developed data storage section 45 for storing the developed view data, a data matching section 46 for performing data matching for distortion adjustment between image data of each frame, and character information from the character recognition processing section 56 and damage described later. The data matching unit 46 adds the character information from the determination unit 51 and
A development view creation unit 47 for creating a development view by combining image data of a plurality of frames after distortion adjustment from the development view, and a development view storage unit 4 for storing the development view created by the development view creation unit 47
Consists of eight. The audio processing unit 38 includes an audio data storage unit 4 for storing audio data of the video data input unit 36.
9. a voice recognition processing unit 50 for recognizing and processing voice from the voice data;
1 includes a damage determination unit 51 for determining the presence or absence of damage, and a voice recording character conversion unit 52 that converts voice information of the damage determination unit 51 into character information and outputs the character information. The development view output device 39
Comprises a development view output unit 53, a monitor 54, and a printer 55. FIG. 4 shows the sequence of creating a development view by the apparatus of the present invention.
This will be described with reference to the flowchart of FIG. (1) Start the process. (2) Video deck 40 of video data input device 36
At the same time, an image signal of the wall surface 11 of the tunnel (sewer) 10 is input for each frame, and an audio signal of an operator for compensating image data such as cracks, jumping out of an attached pipe, seam shift, and dirt is input each time. I do. At this time, a fixed number of the total number is input. For example, assuming that the total number of images to be taken is 100, 10 images are input as a fixed number. (3) The image data is stored in the image data storage unit 42 of the development processing unit 37, and the audio data is stored in the audio data storage unit 49 of the audio processing unit 38. (4) The voice recognition processing section 50 performs voice recognition processing based on the voice data in the voice data storage section 49. (5) The damage determination section 51 makes a damage determination. As a result, if there is an abnormality, record the damage information and move on to the next,
If there is no abnormality, go to the next step. (6) Based on the image data in the image data storage unit 42, the character recognition processing unit 56 performs recognition processing of character information on the screen such as the moving distance and the rotation angle of the video camera 35. (7) The image for the development view is determined. When using as an image for a development view (when yes), proceed to the next development view creation, and when not importing (when no)
Proceeds to the video data end determination described later. (8) When used as a development view image (when yes), image data development processing is performed based on the development processing method shown in FIGS. 1 and 2. If the optical axis of the video camera 35 is not parallel, the correction processing unit 43
The correction processing shown in FIGS. 3A and 3B is performed. Since the expansion processing of the image data is performed for each frame as shown in FIG. 5A, the depth of the image after the expansion processing becomes deeper in the pipeline, the information becomes insufficient and the image becomes distorted. In the developed view shown in FIG. 5A, 57 is a seam, 58 is a crack, 59 is a water surface, and 60 is a stain. The expanded image data for each frame is stored in the expanded data storage unit 45. (9) After the expansion processing, in order to remove a distorted portion, as shown in FIG. 5B, a synthesized image of a plurality of frames is synthesized. At the time of combining, since the center point of the pipeline may be different for each image, the data matching unit 46 performs data matching. (10) In the developed view creating section 47, as shown in FIG. Re-enter the obtained damage information. (11) The developed view information thus obtained is stored in the developed view storage unit 48. (12) The developed view information obtained from the developed view processing device 37 is output to the developed view output section 53 of the developed view output device 39.
Displayed on the monitor 54 via the
5 is printed out. When printing out,
Pay particular attention to the following points. 1) The length of the tunnel (ditch) 10 is reduced to, for example, 20 on A3 paper or roll paper at a scale of 1/1000 to 2/1000.
A developed image is output from the video data every m intervals, which makes it easy to grasp the overall situation in the pipe. This length can be set as appropriate, such as setting a short length when a fine image is required, and setting a long length when a fine image is not required. 2) On the same sheet, develop event information (such as cracks, protruding portions of mounting pipes, seam shifts, etc.) relating to a deterioration state obtained from image data and audio data. 3) For a part having a degree of deterioration recognized as rank A, one or a plurality of enlarged images are output and displayed on the same paper, so that important information in the sewer can be easily grasped. (13)
It is determined whether the video data is completed. When the total number of images to be taken does not reach 100 (in the case of no), the above (2)
And (2) to (12) are repeated. If the total number of images to be taken has reached 100 (yes), the process ends. In the above (7), when the image for the developed view is determined and is not used as the image for the developed view (when no)
Means that even if the end of video data is determined,
Since this is equivalent to the case where the number of sheets does not reach 00 (no),
Returning to (2), (2) to (12) are repeated. (1) All of the conventional apparatuses are mainly constituted by a special or expensive camera, and the apparatus is complicated.
Although it became expensive and the operation was extremely troublesome,
According to the present invention, a continuous development view can be obtained from video data obtained by photographing the inside of a sewer with a general-purpose inexpensive video device. Also, the operability is excellent. (2) Extraction processing section 4 from stored data
4, a development process is performed for each frame, and a distortion between each image data of the development diagram stored for each frame is subjected to data matching by the data matching unit 46.
Since the developed view is created by the developed view creating unit 47 by synthesizing the image data of a plurality of frames after the distortion adjustment, an accurate developed image with no shift is obtained. (3) Since character information is input to the developed view by the developed view creating unit 47 by the character recognition processing unit 56, the reliability of the image data is excellent, and it is convenient for organizing and managing. (4) Since the distortion due to the optical axis shift of the video camera 35 is corrected by the correction processing unit 43, it is possible to eliminate an error during the image data capturing. (5) Not only the character information from the character recognition processing unit 56 but also the character information obtained by converting the voice information of the operator from the damage judging unit 51 is added to the development view. It is possible to more clearly recognize the distinction between the event information on the deterioration state (cracks, mounting pipe jumping out, seam shift, wall surface contamination, etc.).

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a view for explaining the principle of a pipeline inner surface development and mapping apparatus according to the present invention, wherein (a) shows a tunnel (drain) 10;
(B) is a projection view of an image projected on the video camera 35, and (c) is a development view. FIGS. 2A and 2B are explanatory diagrams of a process of developing image data of a wall surface 11 of a tunnel 10; FIG.
0 is a cross-sectional view and an image data diagram before development, and (b) is an explanatory diagram of a relationship between coordinates before and after development. 3A and 3B are explanatory diagrams of a correction process of an optical axis shift of the video camera 35, wherein FIG. 3A is an explanatory diagram between a video camera 35 imaging plane and a virtual coupling plane, and FIG. FIG. FIG. 4 is a flowchart of developing a development view according to the present invention. FIG. 5 is a diagram showing an order of creating a development view according to the present invention;
(A) is a development view of one frame, (b) is a development view of a plurality of frames, and (c) is a development view after adding damage information. FIG. 6 is a block diagram showing an embodiment of a pipeline inner surface development and mapping device according to the present invention. FIGS. 7A and 7B show a first example of a conventional sewer inner surface development and mapping device, in which FIG. 7A is an explanatory diagram when the wall surface 11 is photographed, and FIG.
FIG. 8A and 8B show a second example of a conventional pipe inner surface development and mapping device, wherein FIG. 8A is an explanatory diagram when the wall surface 11 is photographed, and FIG.
It is a flowchart which shows the working process. 9A and 9B are views showing a third example of a conventional pipe inner surface development and mapping device, wherein FIG. 9A is an explanatory diagram of a borehole (drain) 21 taken at the time of photographing, FIG. (C) is a development view thereof. [Description of Signs] 10: tunnel (drain), 11: wall surface, 12a, 12
b, 12c, 12d ... line sensor camera, 13a, 1
3b, 13c, 13d: wall image, 14: crack, 1
4a: Crack in actual tunnel (drain) 10, 14b: Crack in image projected by video camera 35, 14c: Crack in developed view, 15: Overlapping section, 17 ... Panoramic camera, 18 ... Rail, 21 ... Boring Hole (drain), 2
2 ... probe, 23 ... CCD camera, 24 ... cone mirror, 25 ... internal control unit, 26 ... image processing device, 2
7 memory, 28 monitor, 29 frame memory A, 30 frame memory B, 35 video camera, 3
6 ... Video data input device, 37 ... Development view processing device, 3
Reference numeral 8: audio processing apparatus, 39: development view output apparatus, 40: video deck, 41: video capture board, 42: image data storage section, 43: correction processing section, 44: development processing section,
45: development data storage unit, 46: data matching unit,
47: development view creation section, 48: development view storage section, 49: voice data storage section, 50: voice recognition processing section, 51: damage determination section, 52: voice recording character conversion section, 53: development view output section,
54, monitor, 55, printer, 56, character recognition processing unit, 57, joint, 58, crack, 59, water surface, 60
... dirty.

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Takemi Wakikawa 2-6-1, Otemachi, Chiyoda-ku, Tokyo Inside Tokyo Sewerage Service Co., Ltd. (72) Inventor Shinji Numao 2304 Takasaki, Kakizaki-cho, Inashiki-gun, Ibaraki Japan Koei Co., Ltd. Central Research Laboratory (72) Inventor Zhang Jihang, 2304 Takasaki, Kusazaki-cho, Inashiki-gun, Ibaraki Japan Japan Koei Co., Ltd. (56) References JP-A-5-191902 (JP, A) JP-A-7-37065 (JP, A) (58) Fields studied (Int. Cl. ⁷ , DB name) G01L 15/00 G06T 1/00 280

Claims (1)

(57) [Claims 1] A video data input device 36 for taking in image data and audio data of the sewer wall 11, and a development view for each frame based on the image data of the video data input device 36. , A development processing unit 37 for synthesizing image data of a plurality of frames, and an audio processing unit 38 for processing the audio data of the video data input device 36 and sending the processed data to the development processing unit 37 And a development view output device 39 for outputting a synthesized development view. The video data input device 36 is a video deck 40 for capturing image data of the sewer wall surface 11 and voice data of an operator corresponding to the image data. The audio processing device 3
Reference numeral 8 denotes an audio data storage unit 49 for storing audio data of the video data input device 36, an audio recognition processing unit 50 for recognizing and processing audio from the audio data, and The development processing unit 37 includes a damage determination unit 51 that determines the presence or absence of damage, and a voice recording character conversion unit 52 that converts voice information of the damage determination unit 51 into character information and outputs the character information. Image data storage unit 42 for storing the image data of the device 36
A development processing unit 44 for performing development processing of a development view for each frame from the stored data; a development data storage unit 45 for storing the development view data; and a distortion adjustment between image data for each frame. A data matching unit 46 that performs data matching for the image data, a character recognition processing unit 56 that recognizes characters from the image data in the image data storage unit 42, and outputs character information to a development view, and a character recognition processing unit 56. And a development view creation section 47 for creating a development view by adding the character information from the damage determination section 51 and the character data from the damage determination section 51 and combining the image data for a plurality of frames after the distortion adjustment from the data matching section 46. An inner surface development and mapping device for a sewer characterized by comprising: