JP2014086826A - Road surface photographing system and tunnel wall surface photographing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a road surface photographing system capable of reducing processing load in generating a composite image of the road surface by synthesizing photographed images captured by on-vehicle cameras.SOLUTION: An image synthesizing device 112 includes: a first image division part 20 for dividing an image captured with a first on-vehicle camera into a number of first image blocks; a second image division part 21 for dividing an image captured by a second on-vehicle camera into a number of second image blocks so as to correspond to the first image blocks; an overlapping width calculation part 24 for obtaining an overlapping width W of a part of the first image blocks with the second image blocks, by comparing to corresponding second image blocks; an overlapping width interpolation calculation part 25 for obtaining an overlapping width W of the rest of the first image blocks with corresponding second image blocks by performing an interpolation calculation based on an overlapping width W obtained by the overlapping width calculation part 24; and a synthesized image generation part 26 for generating a synthesized image on the basis of an overlapping width W obtained by the overlapping width calculation part 24 and the overlapping width interpolation part 25.

Description

  The present invention relates to a road surface photographing system and a tunnel wall surface photographing system, and more particularly to an improvement of a road surface photographing system including first and second vehicle-mounted cameras that perform photographing while overlapping a part of photographing regions with each other.

  The road surface inspection system detects a crack or the like generated on the road surface by photographing the road surface with an in-vehicle camera while traveling on a paved road and analyzing the captured image. Usually, a camera with a large angle of view is more likely to distort the peripheral portion of the captured image than a camera with a small angle of view. For this reason, a plurality of in-vehicle cameras are used in a case where the road surface is photographed over a wide range in the vehicle width direction intersecting with the traveling direction of the vehicle. In this case, each in-vehicle camera is arranged so that a part of the shooting areas on the road surface overlap each other, and by connecting the shot images, a composite image having a wider field of view in the vehicle width direction than the original shot image Is generated. For road surface inspection, a road surface image composed of a band-shaped composite image whose field of view is expanded in the traveling direction by connecting a large number of vehicle width connecting images obtained by connecting captured images of in-vehicle cameras in the vehicle width direction. Is used.

  In such a road surface inspection system, if a vehicle for road surface photography passes through a step on the road surface, the entire vehicle largely shakes in the vertical direction, and the distance between the in-vehicle camera and the road surface changes greatly. If the distance between the in-vehicle camera and the road surface changes, the shooting area on the road surface expands or contracts in the vehicle width direction, and the size of the overlapping portion of the shooting area also changes between adjacent cameras. The conventional road surface inspection system does not take into account the phenomenon that the photographing area on the road surface and the overlapping portion change due to the vehicle passing through the road surface step, and the photographed images are connected between adjacent cameras. At this time, each captured image is synthesized by overlapping image areas of a certain size. For this reason, there has been a problem that missing or overlapping pixels occur in the composite image.

  In view of this, a road surface inspection system that generates a composite image by comparing captured images of adjacent cameras, obtaining overlapping regions on the captured images, and connecting the captured images has been proposed (for example, Patent Document 1). In the road surface inspection system described in Patent Document 1, an error due to the fact that the length of the overlapping region is not a constant value between the captured images due to the vibration of the vehicle is obtained by comparing the captured images to obtain the overlapping region. Has been removed.

  However, in the road surface inspection system disclosed in Patent Document 1, each time a captured image is acquired from an in-vehicle camera as the vehicle travels a certain distance, the captured image is compared between adjacent cameras to obtain an overlapping region. I had to. For this reason, there has been a problem that a processing load is large when generating a road surface image whose field of view is enlarged in the traveling direction of the vehicle from many vehicle width connection images.

JP 2011-90367 A

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a road surface photographing system capable of reducing a processing load when a composite image of a road surface is generated by combining captured images of an in-vehicle camera. And In particular, when generating a composite image in which the field of view is expanded in the traveling direction of the vehicle from a large number of vehicle-width-connected images while suppressing the occurrence of pixel omission or overlap in the composite image due to vehicle shake or the like. An object of the present invention is to provide a road surface photographing system capable of reducing the processing load.

  Further, the present invention provides a road surface photographing system capable of further reducing the processing load when generating a composite image or improving the resolution in the traveling direction in the composite image according to the length of the overlapping portion of the photographing region. The purpose is to provide. Further, the present invention provides a road surface photographing system that can further reduce the processing load when generating a composite image or improve the resolution in the traveling direction of the composite image according to the vibration period and amplitude of the vehicle. The purpose is to provide.

  It is another object of the present invention to provide a tunnel wall surface photographing system that can reduce a processing load when a composite image of a tunnel wall surface is generated by combining photographed images of a vehicle-mounted camera. In particular, when generating a composite image in which the field of view is expanded in the traveling direction of the vehicle from a large number of vehicle-width-connected images while suppressing the occurrence of pixel omission or overlap in the composite image due to vehicle shake or the like. An object of the present invention is to provide a tunnel wall surface photographing system capable of reducing the processing load.

  The road surface photographing system according to the first aspect of the present invention includes a line sensor in which a photographing region on a road surface intersects with a traveling direction of a vehicle, and a part of the photographing region is overlapped with each other and photographing is performed every predetermined traveling distance. A road surface photographing system including first and second vehicle-mounted cameras and an image composition device that generates a composite image based on images captured by the first and second vehicle-mounted cameras, wherein the image composition device includes: First image dividing means for dividing a photographed image of the first vehicle-mounted camera into a number of first image blocks composed of one or more lines, and photographing of the second vehicle-mounted camera so as to correspond to the first image block The second image dividing means for dividing the image into a plurality of second image blocks composed of one or two or more lines and a part of the first image blocks are compared with the corresponding second image blocks to obtain the second Picture bro The overlap width detecting means for obtaining the overlap width with the second image block and the remaining first image block are subjected to an interpolation operation based on the overlap width obtained by the overlap width detecting means to overlap with the corresponding second image block. An overlapping width interpolating means for obtaining a width, and a composite image generating means for generating the composite image based on the overlapping width obtained by the overlapping width detecting means and the overlapping width interpolating means.

  In the image synthesizing device of this road surface photographing system, only a part of the first image blocks is compared with the corresponding second image block to obtain the overlap width with the second image block, and the remaining first image blocks are determined. The overlap width with the corresponding second image block is obtained by interpolation calculation. By combining the first and second image blocks using the overlap width obtained in this way, the composite image has a large number of vehicle width concatenated images having a wide field of view in the vehicle width direction intersecting the vehicle traveling direction. It is obtained by acquiring and connecting these vehicle width connection images with respect to the traveling direction of the vehicle. For this reason, for all the image blocks, compared with the case where the image blocks are compared between the first and second vehicle-mounted cameras to obtain the overlap width, the composite in which the field of view is expanded in the traveling direction of the vehicle from a large number of vehicle width connected images. The processing load at the time of generating an image can be reduced. In addition, since the overlapped image is obtained by comparing the image blocks and the captured images are combined, it is possible to suppress the occurrence of missing or overlapping pixels in the combined image due to vehicle shake or the like.

  In the road surface photographing system according to the second aspect of the present invention, in addition to the above configuration, the overlap width detecting unit selects the part of the first image blocks with a predetermined number of blocks as an interval period, and the image composition device includes: It is comprised so that it may have an interval control means which determines the said interval period based on the said duplication width | variety.

  According to such a configuration, since the interval period is determined based on the overlap width obtained by the overlap width detection means or the overlap width interpolation means, a composite image is generated according to the length of the overlap portion of the imaging region. The processing load at the time can be further reduced, or the resolution in the running direction in the composite image can be improved.

  In the road surface photographing system according to the third aspect of the present invention, in addition to the above configuration, the overlapping width interpolation means linearly interpolates using the overlapping width of the partial first image blocks, thereby the remaining first image blocks. Is configured to determine the overlap width of.

  According to such a configuration, by using the overlap width obtained from a part of the first and second image blocks, the processing load when obtaining the overlap width of the remaining first and second image blocks by interpolation calculation is reduced. Can be reduced.

  A road surface photographing system according to a fourth aspect of the present invention includes, in addition to the above configuration, a vibration detection device that detects the vibration of the vehicle, wherein the overlap width detection means uses a predetermined number of blocks as an interval period. One image block is selected, and the image composition device is configured to have interval control means for determining the interval period based on the detection result of the vibration detection device.

  According to such a configuration, since the interval period is determined based on the detection result of the vibration of the vehicle, the processing load when generating the composite image is further reduced according to the vibration period and amplitude of the vehicle, or The resolution in the running direction in the composite image can be improved.

  A tunnel wall surface photographing system according to a fifth aspect of the present invention comprises a line sensor in which a photographing region on a tunnel wall surface intersects with the traveling direction of a vehicle, and a part of the photographing region is overlapped with each other to photograph each predetermined traveling distance. A tunnel wall surface photographing system comprising first and second in-vehicle cameras that perform the above and an image composition device that generates a composite image based on images captured by the first and second in-vehicle cameras. A first image dividing means for dividing the photographed image of the first in-vehicle camera into a plurality of first image blocks composed of one or more lines, and a second in-vehicle so as to correspond to the first image block; The ratio of the second image dividing means for dividing the photographed image of the camera into a plurality of second image blocks made up of one or more lines and the corresponding second image blocks for some of the first image blocks The overlap width detection means for calculating the overlap width with the second image block and the remaining first image block are subjected to an interpolation operation based on the overlap width determined by the overlap width detection means, and the corresponding Overlap width interpolation means for obtaining an overlap width with the second image block; and composite image generation means for generating the composite image based on the overlap width obtained by the overlap width detection means and the overlap width interpolation means. It is prepared for.

  In the image synthesizing apparatus of this tunnel wall surface photographing system, only a part of the first image blocks is compared with the corresponding second image block to obtain the overlap width with the second image block, and the remaining first image blocks For, the overlap width with the corresponding second image block is obtained by interpolation calculation. By combining the first and second image blocks using the overlap width obtained in this way, the composite image has a large number of vehicle width concatenated images having a wide field of view in the vehicle width direction intersecting the vehicle traveling direction. It is obtained by acquiring and connecting these vehicle width connection images with respect to the traveling direction of the vehicle. For this reason, for all the image blocks, compared with the case where the image blocks are compared between the first and second vehicle-mounted cameras to obtain the overlap width, the composite in which the field of view is expanded in the traveling direction of the vehicle from a large number of vehicle width connected images The processing load at the time of generating an image can be reduced. Further, since the composite image of the captured image is generated by obtaining the overlap width of the image blocks, it is possible to suppress the occurrence of pixel omission or overlap in the composite image due to vehicle shake or the like.

  In the road surface photographing system according to the present invention, it is possible to reduce the processing load when the composite image of the road surface is generated by combining the captured images of the in-vehicle camera. In particular, when generating a composite image in which the field of view is expanded in the traveling direction of the vehicle from a large number of vehicle-width-connected images while suppressing the occurrence of pixel omission or overlap in the composite image due to vehicle shake or the like. Processing load can be reduced.

  Further, it is possible to further reduce the processing load when generating the composite image, or to improve the resolution in the traveling direction in the composite image, according to the length of the overlapping portion of the imaging region. Furthermore, according to the vibration period and amplitude of the vehicle, it is possible to further reduce the processing load when generating the composite image, or to improve the resolution in the traveling direction in the composite image.

  Further, in the tunnel wall surface photographing system according to the present invention, it is possible to reduce the processing load when the composite image of the tunnel wall surface is generated by combining the captured images of the in-vehicle camera. In particular, when generating a composite image in which the field of view is expanded in the traveling direction of the vehicle from a large number of vehicle-width-connected images while suppressing the occurrence of pixel omission or overlap in the composite image due to vehicle shake or the like. Processing load can be reduced.

1 is a system diagram showing a configuration example of a road surface inspection system 100 including a road surface photographing system 110 according to an embodiment of the present invention. It is the external view which showed the structural example of the vehicle 111 of FIG. 1, and the mode that the left side surface of the vehicle 111 was seen from the vehicle width direction is shown. FIG. 2 is a perspective view illustrating a configuration example of the vehicle 111 in FIG. 1, in which a camera unit 1 and road illumination units 31, 32, and 4 attached to the rear surface of the vehicle 111 are illustrated. It is the external view which showed the structural example of the vehicle 111 of FIG. 1, and the mode that the rear surface of the vehicle 111 was seen from the running direction is shown. It is the block diagram which showed an example of the internal structure of the vehicle 111 of FIG. FIG. 2 is a block diagram illustrating a configuration example of an image composition device 112 in FIG. 1. It is explanatory drawing which showed typically the mode of the road surface imaging | photography using the vehicle-mounted cameras 1a-1c of FIG. It is explanatory drawing which showed the mode immediately after the vehicle 111 passed the level | step difference from the state of FIG. It is the figure which showed an example of the road surface image produced | generated by the conventional road surface imaging | photography system. It is the figure which showed an example of the road surface image produced | generated by the road surface imaging system 110 of FIG. It is an explanatory view schematically showing an example of a process for determining the overlap width W _1, W ₂ between check block CB ₁ and CB ₂ by the image synthesizing device 112 of FIG. 6. 3 is a flowchart showing an example of the operation of the road surface inspection system 100 of FIG. 1. It is the flowchart which showed the process sequence of the picked-up image composition process of FIG. It is the flowchart which showed the process sequence of the duplication width discrimination | determination process of FIG.

<Road surface inspection system 100>
FIG. 1 is a system diagram showing a configuration example of a road surface inspection system 100 including a road surface photographing system 110 according to an embodiment of the present invention. The road surface inspection system 100 includes a road surface photographing system 110 and a crack detection device 120. The road surface inspection system 100 is generated on the road surface by photographing the road surface with the in-vehicle cameras 1a to 1c of the vehicle 111 traveling on the paved road and analyzing the photographed image. Detect cracks.

  The road surface photographing system 110 includes a vehicle 111 provided with the on-vehicle cameras 1a to 1c and the vibration detection device 2, and an image composition device 112 that synthesizes images captured by the on-vehicle cameras 1a to 1c to generate a road surface image. Road surface photography is performed while traveling on the road surface at a predetermined speed. For example, road surface photography is performed while traveling at a vehicle speed of about 80 km / h to 120 km / h.

  Each of the in-vehicle cameras 1a to 1c is an imaging device including a line sensor in which an imaging region on the road surface intersects with the traveling direction of the vehicle 111, and performs imaging at every predetermined traveling distance. The vibration detection device 2 is a vibration detection sensor that detects vibration of the vehicle 111 that occurs as the road travels in order to adjust the resolution of the road surface image related to the traveling direction of the vehicle 111, and detects the period and amplitude of the vibration. . For example, the vibration detection device 2 includes an acceleration sensor.

  The image synthesizing device 112 has a field of view compared to the original captured image in the vehicle width direction intersecting with the traveling direction of the vehicle 111 by overlapping and connecting a part of the captured images acquired by the in-vehicle cameras 1a to 1c. A large number of wide-width connected images are generated. The road surface image is generated as a composite image in which the field of view is enlarged with respect to the traveling direction of the vehicle 111 by connecting the multiple vehicle width connection images. The crack detection device 120 detects where and what kind of crack exists on the road surface by analyzing such a road surface image.

<Vehicle 111 for road surface photography>
FIG. 2 is an external view showing a configuration example of the vehicle 111 of FIG. 1, and shows a state in which the left side surface of the vehicle 111 is viewed from the vehicle width direction. The vehicle 111 is provided with a camera unit 1 including vehicle-mounted cameras 1a to 1c and road surface illumination units 3 and 4, and road surface photographing is performed while traveling on the road surface 5 at a substantially constant speed.

  Each of the road surface illumination units 3 and 4 is a light projecting device that illuminates the road surface 5 for road surface photography, and is composed of a linear light source device in which the light projecting area on the road surface 5 intersects the traveling direction of the vehicle 111. The road surface illumination unit 3 is an illumination unit that illuminates the traveling line of the vehicle 111 with the left area and the right area as the projection area, and the road illumination unit 4 is an illumination unit that illuminates with the center area as the projection area. .

  The camera unit 1 receives reflected light from the road surface 5 of the irradiation light of the road surface illumination units 3 and 4 and generates a photographed image. The camera unit 1 and the road surface illumination units 3 and 4 are attached to the rear surface of the vehicle 111. The camera unit 1 is disposed at the upper part of the vehicle rear surface, and the road surface illumination units 3 and 4 are disposed near the road surface 5.

  FIG. 3 is a perspective view illustrating a configuration example of the vehicle 111 in FIG. 1, in which the camera unit 1 and the road surface illumination units 31, 32, and 4 attached to the rear surface of the vehicle 111 are illustrated. FIG. 4 is an external view showing a configuration example of the vehicle 111 in FIG. 1, and shows a state in which the rear surface of the vehicle 111 is viewed from the traveling direction. FIG. 4 shows a state where the light shielding cover of the camera unit 1 is removed.

  The camera unit 1 is provided with a light shielding cover for blocking outside light. When the light shielding cover is removed, as shown in FIG. 4, the three on-vehicle cameras 1a to 1c are exposed. Each of the in-vehicle cameras 1a to 1c is configured by a line sensor including an imaging element in which a large number of light receiving elements are arranged in a straight line, and performs imaging by overlapping a part of imaging areas on the road surface.

  By using a camera made of a line sensor as the in-vehicle cameras 1a to 1c, the road surface illumination device can be downsized. In addition, the aperture ratio can be increased as compared with the case of using a two-dimensional image sensor.

  The in-vehicle camera 1b is a central camera that performs imaging using the central area as an imaging region with respect to the traveling direction of the vehicle 111, and is disposed in the center of the rear surface of the vehicle. The in-vehicle camera 1b is arranged with the imaging axis facing directly below. On the other hand, the in-vehicle camera 1a is a left camera that takes an image with the left area as an imaging region with respect to the traveling direction of the vehicle 111, and is disposed on the left side of the vehicle rear surface. The in-vehicle camera 1c is a right camera that takes an image with the right area as an imaging region with respect to the traveling direction of the vehicle 111, and is disposed on the right side of the vehicle rear surface. The in-vehicle cameras 1a and 1c are arranged so as to be inclined so that the photographing axis faces the side.

  The road illumination unit 3 includes a road illumination unit 31 for illuminating the left area and a road illumination unit 32 for illuminating the right area with respect to the travel line of the vehicle 111. The road surface lighting units 31, 32, and 4 are respectively provided with linear light source devices 31a, 32a, and 4a, and a light shielding cover for preventing light leakage of irradiation light.

  Each light source device 31a, 32a, 4a is arrange | positioned so that a part of light projection area on the road surface 5 may mutually overlap. For example, each of the light source devices 31a, 32a, and 4a is configured by a number of white LEDs (light emitting diodes).

  FIG. 5 is a block diagram showing an example of the internal configuration of the vehicle 111 in FIG. The vehicle 111 includes an in-vehicle camera 1a to 1c, a vibration detection device 2, a vehicle speedometer 10, a vehicle speed pulse converter 11, a photographing control unit 12, a photographed image storage unit 13, a correction table storage unit 14, and a detection data storage unit 15. Is done.

  The vehicle speedometer 10 is a speed measuring unit for measuring the traveling speed of the vehicle 111, and generates a vehicle speed pulse indicating the traveling speed. The vehicle speed pulse converter 11 generates a shooting control pulse for defining shooting timing by converting the vehicle speed pulse, and outputs it to the shooting control unit 12.

  The imaging control unit 12 controls the in-vehicle cameras 1a to 1c based on the imaging control pulse, and acquires captured images for each predetermined travel distance. For example, a captured image is acquired every time the vehicle 111 travels a distance of 1 mm. The in-vehicle cameras 1 a to 1 c are photographed in synchronization with the photographing control pulse, and the obtained photographed images are sequentially stored in the photographed image storage unit 13.

  In addition, the imaging control unit 12 performs calibration imaging for geometric correction and shading correction of the captured image, generates a correction table from the acquired captured image, and stores the correction table in the correction table storage unit 14.

  The geometric correction is distortion correction for removing distortion of the lenses of the in-vehicle cameras 1a to 1c and distortion due to an attachment error when the in-vehicle cameras 1a to 1c are attached to the vehicle 111. The shading correction is a brightness correction for removing luminance unevenness due to uneven illumination. Calibration photography is performed by placing a calibration chart on the road surface and photographing this chart.

  The detection data storage unit 15 stores the period and amplitude of vibration detected by the vibration detection device 2 as detection data. For example, the photographed image storage unit 13, the correction table storage unit 14, and the detection data storage unit 15 are configured by an SSD (Solid State Drive), and a photographed image and a correction table are held in a detachable nonvolatile recording medium.

<Image Synthesizer 112>
FIG. 6 is a block diagram illustrating a configuration example of the image composition device 112 in FIG. The image composition device 112 includes a first image division unit 20, a second image division unit 21, a check block selection unit 22, a correction processing unit 23, an overlap width calculation unit 24, an overlap width interpolation unit 25, a composite image generation unit 26, The road surface image storage unit 27 and the interval control unit 28 are configured.

  The first image dividing unit 20 acquires a captured image of the central camera, that is, the in-vehicle camera 1b from the vehicle 111, and divides it into a number of first image blocks. Each first image block includes one or more lines, that is, one or more captured images. For example, each first image block is composed of a two-dimensional image obtained by integrating photographed images of about several hundred lines.

  The second image dividing unit 21 acquires the captured images of the left camera (vehicle-mounted camera 1a) and the right camera (vehicle-mounted camera 1c) from the vehicle 111, and divides each image into a plurality of second image blocks. The second image block is determined to correspond to the first image block. Each second image block consists of one or more lines.

  The check block selection unit 22 selects a part of first image blocks as a check block CB for obtaining the overlap width W by image comparison, with a predetermined number of blocks as an interval period T.

  The correction processing unit 23 acquires a correction table from the vehicle 111, and performs correction processing of the captured image based on the correction table. This correction processing includes geometric correction, image width correction, and shading correction. The image width correction is an image size correction for correcting distortion due to non-uniform length of the shooting area.

  The in-vehicle camera 1b is arranged with the photographing axis facing down, whereas the in-vehicle cameras 1a and 1c are arranged so as to be inclined so that the photographing axis faces the side. For this reason, if the length of the imaging area in the vehicle width direction is compared, the imaging areas of the in-vehicle cameras 1a and 1c are longer than the imaging area of the in-vehicle camera 1b. In the correction of the image width, such a non-uniformity of the shooting area is corrected by adjusting the size of the shot image.

  The overlap width calculation unit 24 compares the check block CB with the corresponding second image block to obtain the overlap width W with the second image block. The overlap width calculation unit 24 obtains the overlap width W based on the image block corrected by the correction processing unit 23. For example, the overlap width W is obtained by extracting a small area from the check block CB and detecting an image area having the closest luminance distribution in the second image block with respect to the small area. The check block selection unit 22 and the overlap width calculation unit 24 constitute an overlap width detection unit.

  The overlap width interpolation unit 25 performs an interpolation operation based on the overlap width W obtained by the overlap width calculation unit 24 for the remaining first image blocks to obtain the overlap width W with the corresponding second image block. The overlap width interpolation unit 25 obtains the overlap width W of the remaining first image blocks by performing linear interpolation using the overlap width W of the check block CB.

  The composite image generation unit 26 generates a composite image based on the overlap width W obtained by the overlap width calculation unit 24 and the overlap width interpolation unit 25. The composite image creates a plurality of vehicle width concatenated images by connecting the first image block and the second image block so as to overlap by the overlap width W, and these vehicle width concatenated images are related to the traveling direction of the vehicle 111. Generated by concatenation. This composite image is created based on the image block corrected by the correction processing unit 23. The road surface image storage unit 27 holds the composite image generated by the composite image generation unit 26 as a road surface image.

  The interval control unit 28 acquires the detection data of the vibration detection device 2 from the vehicle 111, and determines the interval period T based on the detection data. For example, if the period of vibration falls below a certain value, control to shorten the interval period T is performed. On the other hand, if the period of vibration exceeds a certain value, control for increasing the interval period T is performed.

  By configuring in this way, since the interval period T is determined based on the detection result of the vibration of the vehicle 111, the processing load when generating the road surface image is further reduced according to the vibration period of the vehicle 111, Or the resolution of the running direction in a road surface image can be improved.

FIG. 7 is an explanatory view schematically showing a state of road surface photographing using the in-vehicle cameras 1a to 1c of FIG. Vehicle camera 1a~1c on the vehicle 111 is a height _{H 1} from the road surface 5, requires photography was overlap a portion of the imaging area _A 1, _{B 1.}

For example, the left camera (vehicle-mounted camera 1a) performs photographing area of the left as the imaging region A ₁ than the running line of the vehicle 111. The central camera (vehicle-mounted camera 1b) performs photographing a central area of the running line of the vehicle 111 as the photographing region B _1. The imaging area _{A 1} and the photographing region _{B 1} represents, overlapping portion D1 has occurred.

  The imaging region of each of the in-vehicle cameras 1a to 1c is a linear area that is long in the vehicle width direction, and can cover the road surface 5 having a width of about 5 m. If the vehicle 111 passes the step on the road surface 5, the entire vehicle 111 shakes greatly in the vertical direction, and the distance between the in-vehicle cameras 1 a to 1 c and the road surface 5 changes greatly.

FIG. 8 is an explanatory diagram showing a state immediately after the vehicle 111 has passed the step from the state of FIG. By vehicle 111 from the state of FIG. 7 passes through the step, the height from the road surface 5 of the vehicle-mounted camera 1a~1c changes from H ₁ to H _2, which is ΔH as low. At this time, the imaging area A ₂ of the vehicle-mounted camera 1a, the imaging region B ₂ of the vehicle-mounted camera 1b are both the length of the vehicle width direction as compared to the prior passage of the step is short.

Therefore, overlapping portions _{D 2} between imaging regions _{A 2} and _{B 2} is shorter than the overlapping portion _{D 1} of the previous step pass. In this way, if the distance between the in-vehicle cameras 1a to 1c and the road surface 5 changes, the shooting area on the road surface 5 expands or contracts in the vehicle width direction, and the size of the overlapping portion of the shooting area also changes between adjacent cameras. To do.

  In the conventional road surface photographing system, it is not considered that the photographing region on the road surface 5 and the overlapping portion change due to the vehicle 111 passing through the step of the road surface 5, and the photographed image is taken between adjacent cameras. When connecting, each captured image is synthesized by overlapping image areas of a certain size. For this reason, missing or overlapping pixels occur in the road surface image.

  FIG. 9 is a diagram showing an example of a road surface image generated by a conventional road surface photographing system. In this figure, a road surface image obtained by photographing the vicinity of a manhole cover provided on the road surface 5 is shown. In this road surface image, missing portions of pixels are generated along the traveling direction of the vehicle 111.

  On the other hand, in the road surface photographing system 110 according to the present embodiment, the overlapped width W is obtained by comparing the image blocks, and the photographed images are synthesized. Therefore, the road surface image is missing or overlapped due to the shaking of the vehicle 111 or the like. Can be suppressed.

  FIG. 10 is a diagram showing an example of a road surface image generated by the road surface photographing system 110 of FIG. In this road surface image, a pixel missing portion as shown in FIG. 9 is not seen, and a high quality road surface image is obtained.

FIG. 11 is an explanatory view schematically showing an example of processing for obtaining the overlapping widths W ₁ and W ₂ between the check blocks CB ₁ and CB ₂ by the image composition device 112 of FIG. In this figure, the captured images of the central camera are divided into a number of first image blocks and arranged in time series, and the captured images of the left camera and the right camera are divided into a number of second image blocks and arranged in time series. ing.

The check block CB ₁ is automatically selected from the multiple first image blocks at an interval period T. Check block CB ₂ is in accordance with the check blocks CB _1, selected from among the plurality of second image blocks.

The overlapping width _W 1, _{W 2} between check block CB ₁ and CB ₂ are determined by matching these image blocks. Specifically, with respect to the small area P ₁ in the check block CB _1, by detecting the nearest image region luminance distribution from within check block CB _2, it is determined. For example, check block defines a small region P ₁ on the right end of CB _1, the position of the small area P ₁ and the traveling direction is the same, and a large matching area F ₁ than the length of the vehicle width direction is small regions P ₁ the searches by detecting the image region closest to the small regions P _1, the overlapping width W ₁ is determined.

Similarly, for the check blocks CB ₁ and CB ₂ having different shooting times by the interval period T, the overlapping width W ₂ is obtained by detecting the image area closest to the small area P ₁ from the matching area F ₂ . The overlap width W of the first and second image blocks other than the check blocks CB ₁ and CB ₂ is obtained by linear interpolation using W ₁ and W ₂ .

  Steps S101 to S105 in FIG. 12 are flowcharts showing an example of the operation of the road surface inspection system 100 in FIG. First, the imaging control unit 12 of the vehicle 111 performs calibration imaging by imaging the chart while traveling on the calibration chart, and performs a correction table based on the captured images acquired from the in-vehicle cameras 1a to 1c. Is created and stored in the correction table storage unit 14 (steps S101 and S102).

  Next, the imaging control unit 12 images the running road surface 5, acquires a large number of captured images from the in-vehicle cameras 1a to 1c, and stores them in the captured image storage unit 13 (step S103). Next, the image composition device 112 acquires the captured images and correction tables of the in-vehicle cameras 1a to 1c from the vehicle 111, performs captured image composition processing, and generates a road surface image (step S104). Next, the crack detection device 120 analyzes the road surface image and detects cracks on the road surface 5 (step S105).

  Steps S201 to S206 in FIG. 13 are flowcharts showing the processing procedure of the captured image composition process (step S104) in FIG. First, the image dividing units 20 and 21 divide the captured images of the in-vehicle cameras 1a to 1c into a large number of image blocks (step S201). Next, the correction processing unit 23 performs distortion correction of the captured image, correction of the image width, and shading correction based on the correction table (steps S202 to S204).

  Next, the overlap width calculation unit 24 and the overlap width interpolation unit 25 perform overlap width determination processing for obtaining the overlap width W between image blocks (step S205). Next, the composite image generation unit 26 combines the image blocks based on the overlap width W obtained by the overlap width calculation unit 24 and the overlap width interpolation unit 25, generates a road surface image, and stores the road image in the road surface image storage unit 27. (Step S206).

Steps S301 to S307 in FIG. 14 are flowcharts showing the processing procedure of the overlap width determination processing (step S205) in FIG. First, check block selection unit 22, from among the plurality of first and second image blocks, selects the check block CB1, CB ₂ in interval period T (step S301).

Next, the overlapping width calculating unit 24 compares the check block CB1, CB _2, calculates the overlap width W (step S302, S303). The overlap width interpolation unit 25 obtains the overlap width W of the remaining first and second image blocks by linear interpolation (step S304).

  If the overlap width W is obtained for the first and second image blocks for a certain time, the interval control unit 28 refers to the vibration cycle of the vehicle 111 and adjusts the interval cycle T according to the vibration cycle (step S305). ~ S307). The processing procedure from step S301 to step S307 is repeated each time an overlap width W is obtained for the first and second image blocks for a certain time.

  According to the present embodiment, as compared with the case where the overlap width W is obtained by comparing the image blocks between the vehicle-mounted cameras 1a to 1c for all the image blocks, the field of view is viewed in the traveling direction of the vehicle 111 from a large number of vehicle width connection images. It is possible to reduce the processing load when generating a road surface image with an enlarged scale. In addition, since the captured image is synthesized by obtaining the overlap width W by comparing the image blocks, it is possible to suppress the occurrence of missing or overlapping pixels in the road surface image due to the shaking of the vehicle 111 or the like.

  In this embodiment, the example in which the interval control unit 28 determines the interval period T based on the detection result of the vehicle vibration has been described. However, the present invention limits the method of determining the interval period T to this. Not what you want. For example, the configuration may be such that the interval period T is determined based on the overlap width W obtained by the overlap width calculation unit 24 or the overlap width interpolation unit 25.

  With this configuration, it is possible to further reduce the processing load when generating the road surface image according to the length of the overlapping portion of the imaging region, or to improve the resolution in the traveling direction on the road surface image.

  Moreover, in this Embodiment, in order to detect the crack etc. which arose on the road surface 5 of a paved road, the example in case the road surface 5 was image | photographed and a road surface image was produced | generated was demonstrated. However, the present invention can also be applied to a tunnel wall surface photographing system in which a tunnel wall surface is photographed and a photographed image is synthesized to generate a wall surface image in order to detect a crack or the like generated on the wall surface in the tunnel. In such a tunnel wall surface photographing system, the distance between the in-vehicle camera and the tunnel wall surface varies greatly depending on where the vehicle 111 is traveling in the tunnel in the vehicle width direction. Therefore, it is conceivable that a wall surface distance measuring device for measuring the distance to the tunnel wall surface is provided in the vehicle 111 and the captured image is corrected at a time interval sufficiently longer than the interval period T.

  In this embodiment, an example in which the image composition device 112 is provided outside the vehicle 111 has been described. However, a configuration in which the image composition device 112 is installed in the vehicle 1 may be used.

  In the present embodiment, an example in which linear interpolation is performed as an example of the interpolation calculation when obtaining the overlap width W of image blocks other than the check block CB has been described. However, the present invention limits the interpolation calculation to this example. Not what you want. For example, the configuration may be such that polynomial interpolation is performed using three or more overlap widths W obtained for the check block CB.

DESCRIPTION OF SYMBOLS 100 Road surface inspection system 110 Road surface imaging system 111 Vehicle 1a-1c for road surface imaging Car-mounted camera 2 Vibration detection apparatus 3, 31, 32, 4 Road surface illumination unit 5 Road surface 10 Vehicle speed meter 11 Vehicle speed pulse converter 12 Photographing control part 13 Photographed image Storage unit 14 Correction table storage unit 15 Detection data storage unit 112 Image composition device 20 First image division unit 21 Second image division unit 22 Check block selection unit 23 Correction processing unit 24 Overlap width calculation unit 25 Overlap width interpolation unit 26 Composite image Generation unit 27 Road surface image storage unit 28 Interval control unit 120 Crack detection device

Claims (5)

A first and a second in-vehicle camera that includes a line sensor in which a shooting area on a road surface intersects with a traveling direction of the vehicle, and that overlaps a part of the shooting area with each other and performs shooting at every predetermined travel distance;
In a road surface photographing system provided with an image composition device that generates a composite image based on images photographed by first and second vehicle-mounted cameras,
A first image dividing means for dividing the photographed image of the first in-vehicle camera into a plurality of first image blocks composed of one or more lines;
Second image dividing means for dividing the captured image of the second vehicle-mounted camera into a plurality of second image blocks composed of one or more lines so as to correspond to the first image block;
For some of the first image blocks, a comparison with a corresponding second image block is performed to obtain an overlap width with the second image block;
For the remaining first image blocks, an overlap width interpolating means for performing an interpolation operation based on the overlap width obtained by the overlap width detecting means to obtain an overlap width with the corresponding second image block;
A road surface photographing system comprising: a composite image generation unit configured to generate the composite image based on the overlap width obtained by the overlap width detection unit and the overlap width interpolation unit. The overlapping width detecting means selects the first image block of the part with a predetermined number of blocks as an interval period,
2. The road surface photographing system according to claim 1, wherein the image synthesizing apparatus includes interval control means for determining the interval period based on the overlap width.   The overlap width interpolation unit obtains the overlap width of the remaining first image blocks by performing linear interpolation using the overlap width of the partial first image blocks. The road surface photographing system described. Comprising a vibration detection device for detecting the vibration of the vehicle,
The overlapping width detecting means selects the first image block of the part with a predetermined number of blocks as an interval period,
2. The road surface photographing system according to claim 1, wherein the image composition device includes an interval control unit that determines the interval period based on a detection result of the vibration detection device. A first and a second in-vehicle camera that includes a line sensor in which a shooting area on the tunnel wall surface intersects with a traveling direction of the vehicle, and that overlaps a part of the shooting area to perform shooting at every predetermined travel distance;
In a tunnel wall surface photographing system including an image composition device that generates a composite image based on images captured by first and second vehicle-mounted cameras,
A first image dividing means for dividing the photographed image of the first in-vehicle camera into a plurality of first image blocks composed of one or more lines;
Second image dividing means for dividing the captured image of the second vehicle-mounted camera into a plurality of second image blocks composed of one or more lines so as to correspond to the first image block;
For some of the first image blocks, a comparison with a corresponding second image block is performed to obtain an overlap width with the second image block;
For the remaining first image blocks, an overlap width interpolating means for performing an interpolation operation based on the overlap width obtained by the overlap width detecting means to obtain an overlap width with the corresponding second image block;
A tunnel wall surface photographing system comprising: a composite image generation unit configured to generate the composite image based on the overlap width obtained by the overlap width detection unit and the overlap width interpolation unit.

Images