JP7119462B2 - Imaging system, imaging method, moving body equipped with imaging system, imaging device, and moving body equipped with imaging device - Google Patents

Description

TECHNICAL FIELD The present invention relates to an imaging system, an imaging method, a moving body equipped with the imaging system, an imaging device, and a moving body equipped with the imaging device.

On various roads, road surface properties such as crack rate, rut amount, flatness, etc., are regularly measured in order to ensure vehicle travel safety and comfort, and to identify areas requiring repair. Conventionally, when performing these measurements, it has been common practice to restrict vehicle traffic on the target route and manually perform measurements within the restricted area. In recent years, technology has been proposed for measuring road surface conditions without restricting traffic by acquiring information on the road surface while driving using vehicles equipped with cameras and sensors.

Japanese Patent Laid-Open No. 2002-200001 discloses a road correction surface investigation device that captures an image of a road surface using a plurality of illuminations to increase the detection rate of defects such as cracks in the road surface.

When detecting cracks or measuring ruts in the method of mounting a camera on a vehicle and capturing images of the road surface while driving, the image of the feature included in the captured image must have sufficient contrast. It is desirable to be However, there are cases where sufficient contrast cannot be obtained in the captured image due to road surface conditions, such as differences in pavement types and the presence or absence of shadows. In this case, it may become difficult to perform measurement based on the captured image.

In order to solve this problem, there is a method of acquiring an image with appropriate contrast by illuminating the imaging target area. However, this method requires a large amount of lighting and a large-capacity battery or generator in order to obtain the amount of light necessary for imaging. there was a point It is also conceivable to perform automatic exposure according to changes in the brightness of the road surface by general exposure control. However, in road surface imaging while a vehicle is running, there is a problem that there may be many frames in which sufficient contrast is not obtained due to the follow-up time lag of automatic exposure.

SUMMARY OF THE INVENTION The present invention has been made in view of the above, and it is an object of the present invention to make it possible to easily obtain an image with sufficient contrast suitable for measuring the road surface properties when imaging the road surface while driving a vehicle. and

In order to solve the above-described problems and achieve the object, the present invention provides a first imaging device mounted on a mobile body for imaging a road surface on which the mobile body moves, and a first imaging device mounted on the mobile body for imaging a road surface on which the mobile body moves. A second imaging device that captures an image of a road surface on the traveling direction side of the moving body with respect to the device, an acquisition unit that acquires current position information indicating the current position of the moving body, and a captured image captured by the second imaging device and an imaging control unit that determines an exposure parameter as an imaging condition of the first imaging device based on the current position information and the timing of setting the imaging condition to the first imaging device in advance. It is determined based on the determined distance from the moving object in the imaging range of the second imaging device .

ADVANTAGE OF THE INVENTION According to this invention, when imaging a road surface while driving by vehicle, it is effective in being able to acquire easily the image of sufficient contrast suitable for the measurement of a road surface property.

FIG. 1 is a diagram for explaining the measurement of the amount of rutting. FIG. 2 is a diagram for explaining flatness measurement. FIG. 3 is a diagram showing a configuration example of an imaging system according to the first embodiment. FIG. 4 is a diagram for explaining an imaging range in the direction of travel of a vehicle by a stereo camera, which is applicable to the first embodiment. FIG. 5 is a diagram for explaining an imaging range of a vehicle in the road width direction by a stereo camera, which is applicable to the first embodiment. FIG. 6 is a diagram showing overlapping of stereo imaging ranges of stereo cameras in the traveling direction of the vehicle, which is applicable to the first embodiment. FIG. 7 is a diagram illustrating an example of an imaging system including three stereo cameras according to the first embodiment; FIG. 8 is a block diagram showing an example of a schematic configuration of an imaging system according to the first embodiment; FIG. 9 is a block diagram showing an example of the hardware configuration of the imaging system according to the first embodiment; FIG. 10 is a block diagram showing an example configuration of a stereo camera according to the first embodiment. FIG. 11 is a block diagram showing an example configuration of an information processing apparatus applicable to the first embodiment. FIG. 12 is an example functional block diagram for explaining functions of an information processing apparatus applicable to the first embodiment. FIG. 13 is an exemplary flowchart illustrating flatness calculation processing according to the first embodiment. FIG. 14 is an exemplary flowchart illustrating depth map generation processing applicable to the first embodiment. FIG. 15 is a diagram for explaining trigonometry applicable to the first embodiment. FIG. 16 is a flowchart of an example of camera position and orientation estimation processing applicable to the first embodiment. FIG. 17 is a diagram for explaining camera position and orientation estimation processing applicable to the first embodiment. FIG. 18 is a diagram showing an example of a location on the road surface where there is a large luminance change. FIG. 19 is a flowchart of an example of processing related to the traveling direction camera according to the first embodiment. FIG. 20 is a flowchart of an example of processing related to a road surface imaging camera according to the first embodiment. FIG. 21 is a flowchart of an example of processing related to the traveling direction camera according to the first modification of the first embodiment. FIG. 22 is a diagram for explaining the relationship between the imaging range of the traveling direction camera and the stereo imaging range of the road surface imaging camera, which can be applied to the first modification of the first embodiment. 23A and 23B are diagrams showing examples of images captured by the road surface imaging camera and the traveling direction camera, which are applicable to the first modification of the first embodiment. FIG. 24 is a flowchart of an example of processing related to a road surface imaging camera according to the first modification of the first embodiment. FIG. 25 is a flowchart of an example of processing related to the traveling direction camera according to the second modification of the first embodiment. FIG. 26 is a flowchart of an example of imaging condition setting processing according to the second embodiment.

Embodiments of an image capturing system, an image capturing method, a mobile body equipped with the image capturing system, an image capturing device, and a mobile body equipped with the image capturing device will be described below in detail with reference to the accompanying drawings.

[Overview of existing road property inspection methods]
Before describing the embodiments, an existing method for inspecting road properties will be briefly described. As an index for evaluating road properties, a pavement maintenance control index (MCI: Maintenance Control Index) is defined. MCI quantitatively evaluates the usability of pavement using three types of road surface property values: "crack rate", "rutting amount" and "flatness".

Among these, the "crack rate" is obtained by calculating the crack area according to the number of cracks and the patching area in each area in which the road surface is divided into 50 cm meshes, as follows, and the calculation result is expressed in formula (1). Apply to determine the crack rate.

1 crack ⇒ 0.15 m 2 crack ² cracks ⇒ 0.25 m ² crack Patching area 0 to 25% ⇒ 0 m ² crack
Patching area 25-75% ⇒ Crack 0.125m ²
Patching area 75% or more ⇒ Crack 0.25m ²

The "rut amount" is measured by measuring the depths D1 and D2 of _two "ruts" generated _per lane, as illustrated in Figs. 1(a) and 1(b). The larger of the depths D ₁ and D ₂ is taken. Since there are several patterns in how "ruts" are dug, a measurement method suitable for each pattern is defined. In FIG. 1(a), the distance between two "ruts" is higher than both ends of the two "ruts." It shows an example of the measurement method when it is lower than both ends of the rut.

The flatness is measured by measuring the height from the reference plane at three points at intervals of 1.5 m along the traveling direction of the vehicle on the road surface. For example, as shown in FIG. 2, three measuring instruments are provided on the underside of the vehicle at intervals of 1.5 m to measure heights X ₁ , X ₂ and X ₃ . Based on the measured heights X ₁ , X ₂ and X ₃ , the amount of displacement d is obtained by equation (2). This measurement is performed multiple times while moving the vehicle. Equation (3) is calculated based on the plurality of displacement amounts d thus obtained to calculate the flatness σ.

The "crack rate", "rutting amount" and "flatness" described above are measured for each 100 m evaluation section. If the location of damage is known in advance, 100m may be divided into smaller units such as 40m+60m for measurement. MCI is calculated based on the results of measurements performed in units of 100m, and a report is created.

MCI is calculated using the four formulas shown in Table 1 based on the measured crack rate C [%], rutting amount D [mm], and flatness σ [mm] to obtain the values MCI, MCI1, MCI2 and Calculate MCI3. Then, among the calculated values MCI, MCI1, MCI2 and MCI3, the minimum value is adopted as MCI. Evaluation is performed according to the evaluation criteria shown in Table 2 based on the adopted MCI, and it is determined whether or not the road surface in the evaluation section needs repair.

[First embodiment]
Next, an imaging system according to the first embodiment will be described. In the first embodiment, a camera capable of stereo imaging (stereo camera) is attached to a vehicle to capture an image of the road surface. Depth information for the road surface is obtained from the imaging position based on the captured stereo images, the three-dimensional shape of the road surface is generated, and three-dimensional road surface data is created. By analyzing this three-dimensional road surface data, it is possible to obtain the "crack rate", "rutting amount" and "flatness" used to obtain the MCI.

More specific description will be given. A stereo camera is provided with two cameras separated by a predetermined length (referred to as a base line length), and outputs a pair of captured images (referred to as stereo captured images) captured by these two cameras. . By searching for a corresponding point between two captured images included in this stereo captured image, the depth distance can be restored for any point in the captured image. Data obtained by restoring the depth of the entire captured image and representing each pixel with depth information is called a depth map. That is, the depth map is three-dimensional point group information consisting of a set of points each having three-dimensional information.

This stereo camera is mounted downward at one location such as the rear of the vehicle so that it can take an image of the ground, and the vehicle is moved along the road to be measured. For the sake of explanation, it is assumed that the imaging range of the stereo camera mounted on the vehicle for measurement covers a specified length in the road width direction.

For the rutting amount D, the depth values of the striped portions cut in the road width direction in the depth map restored from the stereo image captured by the stereo camera are arranged. Based on this road width variation in depth, the rut depths D ₁ and D ₂ can be calculated.

The crack rate C is obtained by analyzing a captured image of the road surface to detect "cracks" and the like, and performing calculation using the above-described formula (1) based on the detection results.

Here, in the traveling direction of the vehicle, since the imaging range of one imaging is limited, for example, the crack rate C of a 100 m section cannot be calculated based on the imaging image obtained by one imaging. Therefore, while moving the vehicle along the road, images are sequentially captured for each movement corresponding to the length of the imaging range in the direction in which the vehicle travels. At this time, the imaging trigger is controlled so that the imaging range of the previous imaging and the imaging range of the current imaging overlap at a predetermined overlapping rate or more.

In this way, by controlling the imaging trigger according to the movement of the vehicle, the road surface to be measured can be imaged without omission. For this reason, captured images that are sequentially captured in accordance with the progress of the vehicle are joined using image processing called stitching, for example, to generate a single image including an image of the road surface in a 100 m section. By visually confirming or analyzing this image, the crack rate C on the road surface can be measured.

Flatness measurement uses a technique called Structure from Motion (SfM), which estimates the imaging position based on sufficiently overlapped images taken from images at different imaging points.

An outline of the SfM processing will be described. First, using images captured in overlapping imaging ranges, points obtained by imaging the same point in each image are detected as corresponding points. It is desirable to detect as many corresponding points as possible. Next, for example, simultaneous equations are established using the coordinates of the detected corresponding points for the movement of the camera from the imaging point of the first image to the imaging point of the second image, with rotation and translation as unknown parameters. , find the parameters that minimize the total error. In this way, the imaging position of the second image can be calculated.

As described above, the depth of an arbitrary point in the image can be restored as a depth map from the stereo-captured image at each imaging point. The depth map corresponding to the second stereo-captured image is obtained from the simultaneous equations described above with respect to the depth map corresponding to the first stereo-captured image, with the imaging position of the second camera as the origin. Coordinate transformation to map. As a result, the two depth maps can be unified into the coordinate system of the first depth map. In other words, one depth map can be generated by synthesizing two depth maps.

By performing this processing for, for example, depth maps based on all stereo imaged images captured in a 100 m section and synthesizing them, the road surface of the 100 m section is restored in one three-dimensional space. The amount of displacement d can be calculated by applying the depth value of the road surface thus restored to the above equation (2). Applying this displacement amount d to the above equation (3), the flatness σ is calculated.

In the first embodiment, images are captured by a stereo camera mounted on a vehicle, and image processing is performed on the captured stereo images. , the crack rate C can be collectively measured. In the first embodiment, this measurement can be performed using an imaging system that includes a stereo camera and an information processing device that performs image processing on stereo-captured images, and MCI can be obtained with a simple configuration. .

[Overview of the first embodiment]
In the first embodiment, when the road surface is imaged by a stereo camera for generating the three-dimensional shape of the road surface, an image of the road surface to be imaged by the stereo camera in front of the vehicle in which the imaging system is mounted is taken. Take an image of the range. Based on the captured image captured in the imaging range forward in the direction of travel, the imaging conditions including the amount of exposure when capturing the imaging range with the stereo camera are obtained, and the obtained imaging conditions are applied to the stereo camera. set. As a result, it is possible to appropriately control the imaging conditions when imaging the road surface with the stereo camera, and to generate a three-dimensional shape with higher accuracy based on the stereo-captured images.

[Camera Arrangement Applicable to First Embodiment]
Next, an example of camera arrangement applicable to the first embodiment will be described. FIG. 3 is a diagram showing a configuration example of an imaging system according to the first embodiment. FIG. 3A is a side view of the vehicle 1 showing how the imaging system according to the first embodiment is mounted on the vehicle 1. FIG. In FIG. 3( a ), the direction toward the left end of the drawing is the traveling direction of the vehicle 1 . That is, in FIG. 3( a ), the left end side of the vehicle 1 is the front portion of the vehicle 1 , and the right end side is the rear portion of the vehicle 1 . FIG. 3(b) is a diagram showing an example of the vehicle 1 viewed from the rear side.

In the imaging system according to the first embodiment, a mounting member 3 having a mounting portion 2 for mounting an imaging device is fixed to the rear portion of a vehicle body 1 as a mobile body on which the imaging system is mounted, and one or more Attach the stereo camera 6 of . Here, as exemplified in FIG. 3B, two stereo cameras 6L and 6R are attached to both end sides of the vehicle body of the vehicle 1 in the width direction. Each of the stereo cameras 6L and 6R is attached in an orientation to image the road surface 4 on which the vehicle 1 moves. Preferably, each stereo camera 6L and 6R is mounted so as to image the road surface 4 from the vertical direction.

Hereinafter, the stereo cameras 6L and 6R will be collectively described as the stereo camera 6 when there is no need to distinguish between the stereo cameras 6L and 6R.

The stereo camera 6 is controlled by, for example, a personal computer (PC) 5 installed inside the vehicle 1, for example. The operator operates the PC 5 and instructs the stereo camera 6 to start imaging. When instructed to start imaging, the PC 5 starts imaging with the stereo camera 6 . The imaging is controlled in timing according to the moving speed of the stereo camera 6, that is, the vehicle 1, and is repeatedly executed.

The operator operates the PC 5 and instructs the end of imaging, for example, according to the end of imaging of a required section. The PC 5 terminates the imaging by the stereo camera 6 in response to the imaging termination instruction.

The vehicle 1 is further provided with a camera 7 for capturing an image of the road surface 4 ahead of the vehicle 1 in the traveling direction. That is, the camera 7 images a predetermined area of the road surface 4 to be imaged by the stereo camera 6 prior to imaging of the predetermined area by the stereo camera 6 . A captured image captured by the camera 7 (referred to as a moving direction captured image) is sent to the PC 5 . The PC 5 detects the brightness of the predetermined area based on the traveling direction captured image sent from the camera 7, and sets the imaging conditions including the exposure amount of the stereo camera 6 based on the detection result.

The camera 7 is hereinafter referred to as a traveling direction camera 7 . Further, since the stereo cameras 6L and 6R are cameras that capture images of the road surface 4 from the vertical direction for MCI calculation, the stereo cameras 6L and 6R are described as the road surface imaging cameras 6L and 6R unless otherwise specified. , and the traveling direction camera 7 .

4 and 5 are diagrams showing examples of the imaging range of the road surface imaging camera 6 applicable to the first embodiment. In FIGS. 4 and 5, the same reference numerals are given to the parts common to FIG. 3 described above, and detailed description thereof will be omitted.

FIG. 4 is a diagram for explaining an imaging range in the traveling direction of the vehicle 1 (referred to as traveling direction visual field Vp) by the road surface imaging camera 6, which is applicable to the first embodiment. 4, as in FIG. 1 described above, the direction toward the left end of the drawing is the traveling direction of the vehicle 1. As shown in FIG. The traveling direction visual field Vp is determined according to the angle of view α of the road surface imaging camera 6 and the height h of the road surface imaging camera 6 with respect to the road surface 4, as shown in FIG.

FIG. 5 is a diagram for explaining an imaging range of the vehicle 1 in the road width direction by the road surface imaging camera 6, which is applicable to the first embodiment. FIG. 5 is a view of the vehicle 1 viewed from the rear side, and parts common to those in FIG.

In FIG. 5(a), the road surface imaging camera 6L has two imaging lenses 6L _L and 6L _R. In FIG. A line connecting the imaging lenses 6L _L and _6LR is called a base line, and its length is called a base line length. Similarly, the road surface imaging camera 6R also includes two imaging lenses 6R _L and 6R _R separated by the base line length, and arranged so that the base line is perpendicular to the traveling direction of the vehicle 1 .

FIG. 5(b) shows an example of the imaging range of the road surface imaging cameras 6L and 6R applicable to the first embodiment. In the road surface imaging camera 6L, the imaging ranges 60L _L and _60LR of the imaging lenses 6L _L and _6LR are overlapped with a shift according to the base line length and the height h. Similarly, with respect to the road surface imaging camera 6R, imaging ranges 60R _L and 60R _R respectively by the imaging lenses 6R _L and 6R _R are overlapped with a shift according to the base line length and height h.

Hereinafter, unless otherwise specified, these imaging ranges 60L _L and _60LR and imaging ranges 60R _L and 60R _R are collectively referred to as stereo imaging ranges 60L and 60R, respectively. As shown in FIG. 5(b), the road surface imaging cameras 6L and 6R have stereo imaging ranges 60L and 60R that extend from one end of the stereo imaging range 60L in the width direction of the vehicle 1 to the width direction of the stereo imaging range 60R. are arranged so as to overlap with the other end of the region 61 at a predetermined overlap rate.

FIG. 6 is a diagram showing overlap in the travel direction of the vehicle 1 of the stereo imaging range 60L by the road surface imaging camera 6L, which is applicable to the first embodiment. It is assumed that two images are taken by the road surface imaging camera 6L in the moving vehicle 1 . The road surface imaging camera 6L first captures the stereo imaging range 60L (imaging ranges 60L _L and _60LR ), and the second time, the vehicle 1 in the traveling direction of the vehicle 1 with respect to the stereo imaging range 60L. The stereo imaging range 60L' (imaging ranges 60L _L ' and 60L _R ') moved by a distance corresponding to the moving distance of .

At this time, the imaging timing of the road surface imaging camera 6L is such that the stereo imaging range 60L and the stereo imaging range 60L' overlap with the traveling direction visual field Vp at an overlap rate equal to or greater than the predetermined travel direction overlap rate Dr. to control. In this way, by sequentially imaging the stereo imaging ranges 60L and 60L' along the time axis, it is possible to facilitate the process of joining two stereo imaging images obtained by imaging the stereo imaging ranges 60L and 60L'. .

Although the imaging system according to the first embodiment uses the two road surface imaging cameras 6L and 6R in the above description, it is not limited to this example. For example, as shown in FIG. 7A, the imaging system according to the first embodiment further adds one stereo camera (road surface imaging camera) 6C to the road surface imaging cameras 6L and 6R. It may be configured using three road surface imaging cameras 6L, 6R and 6C.

In the example of FIG. 7A, the distance between the road surface imaging cameras 6L and 6R is wider than when only the road surface imaging cameras 6L and 6R are used, and the road surface imaging camera 6C is arranged in the center. there is As shown in FIG. 7B, imaging ranges 60C _L and 60C _R formed by imaging lenses 6C _L and 6C _R of the road surface imaging camera 6C constitute a stereo imaging range 60C. The road surface imaging cameras 6L, 6C and 6R are arranged so that the respective stereo imaging ranges 60L, 60C and 60R overlap in the width direction of the vehicle 1 at a predetermined overlap ratio.

In this way, by using the three road surface imaging cameras 6L, 6C and 6R in order to image one lane, it is possible to set imaging ranges on the right side, the center and the left side of the lane, respectively. A high-quality (high-resolution) stereo image can be captured by a small number of road surface imaging cameras. Here, especially the road width is generally defined as 3.5 m. Therefore, it is conceivable that both ends of the lane in the road width direction are imaged by the road surface imaging cameras 6L and 6R, and the central portion is imaged by the road surface imaging camera 6C, corresponding to the road width of 3.5 m.

The image pickup system may be configured using only one road surface image pickup camera having an angle of view capable of covering the road width (3.5 m) for which the image pickup range is defined.

[Configuration of Imaging System According to First Embodiment]
Next, the configuration of the imaging system according to the first embodiment will be described. In the following description, it is assumed that the imaging system has two road surface imaging cameras.

FIG. 8 is a block diagram showing an example of a schematic configuration of an imaging system according to the first embodiment; In FIG. 8, the imaging system 10 includes imaging units 100 ₁ and 100 ₂ , a position information acquisition unit 101 , an imaging control unit 102 , and a traveling direction imaging unit 103 .

The imaging units 100 ₁ and 100 ₂ correspond to the above-described road surface imaging cameras 6L and 6R, respectively. The position information acquisition unit 101 acquires the current position of the vehicle 1 . The travel direction imaging unit 103 corresponds to the travel direction camera 7 described above, and images the road surface 4 in front of the vehicle 1 in the travel direction to acquire a travel direction captured image.

The imaging control unit 102 controls imaging operations such as imaging timing, exposure, and shutter speed of the imaging units 100 ₁ and 100 ₂ . The imaging control unit 102 obtains the brightness of the area to be imaged by the imaging units 100 ₁ and 100 ₂ based on the traveling direction captured image acquired by the traveling direction imaging unit 103, and based on the obtained brightness, the imaging unit Imaging conditions including the amount of exposure are set for 100 ₁ and 100 ₂ . Also, the imaging control unit 102 determines the timing of setting imaging conditions for the imaging units 100 ₁ and 100 ₂ based on the current position of the vehicle 1 acquired by the position information acquiring unit 101 .

FIG. 9 is a block diagram showing an example of the hardware configuration of the imaging system 10 according to the first embodiment. 9, the imaging system 10 includes road surface imaging cameras 6L and 6R (stereo cameras 6L and 6R), a traveling direction camera 7, an information processing device 50 corresponding to the PC 5 in FIG. and including.

The information processing device 50 generates a trigger at a predetermined timing and sends the generated trigger to the road surface imaging cameras 6L and 6R. The road surface imaging cameras 6L and 6R perform imaging synchronously in response to this trigger. Each stereo captured image captured by the road surface capturing cameras 6L and 6R is supplied to the information processing device 50 . The information processing device 50 stores and accumulates the stereo captured images supplied from the road surface capturing cameras 6L and 6R in a storage or the like. The information processing device 50 performs image processing such as generating a depth map and connecting the generated depth maps based on the accumulated stereo captured images.

The information processing device 50 also generates a trigger for instructing the camera 7 for the direction of travel to take an image, and sends the trigger to the camera 7 for the direction of travel. This trigger does not have to be synchronized with the trigger sent to the road surface imaging cameras 6L and 6R described above. The traveling direction camera 7 takes an image in response to this trigger. The information processing device 50 obtains imaging conditions including the amount of exposure for the road surface imaging cameras 6L and 6R based on the traveling direction captured image supplied from the traveling direction camera 7, and uses the obtained imaging conditions as setting information for road surface imaging. They are supplied to cameras 6L and 6R, respectively. Here, the information processing device 50 can obtain imaging conditions for the road surface imaging camera 6L and the road surface imaging camera 6R. The information processing device 50 supplies each setting information including the imaging conditions determined for the road surface imaging cameras 6L and 6R to the road surface imaging cameras 6L and 6R, respectively. At this time, the information processing device 50 determines the timing of setting the setting information for the road surface imaging cameras 6L and 6R based on the current position of the vehicle 1 obtained by the position information obtaining section 51 .

FIG. 10 is a block diagram showing an example configuration of the road surface imaging camera 6L according to the first embodiment. Note that the road surface imaging camera 6R can be realized with the same configuration as the road surface imaging camera 6L, so description thereof will be omitted here.

In FIG. 10, the road surface imaging camera 6L includes imaging optical systems 600 _L and 600 _R , imaging elements 601 _L and 601 _R , driving units 602 _L and 602 _R , signal processing units 603 _L and 603 _R , and output and a portion 604 . Among these, the imaging optical system 600 _L , the imaging device 601 _L , the driving unit 602 _L , and the signal processing unit 603 _L are configured to correspond to the imaging lens 6L _L described above. Similarly, the imaging optical system 600 _R , the imaging element 601 _R , the drive unit 602 _R , and the signal processing unit 603 _R are configured to correspond to the imaging lens 6L _R described above.

The imaging optical system _600L is an optical system having an angle of view α and a focal length f, and projects light from a subject onto an imaging element _601L . The imaging element 601 _L is, for example, an optical sensor using CMOS (Complementary Metal Oxide Semiconductor), and outputs a signal corresponding to projected light. An optical sensor using a CCD (Charge Coupled Device) may be applied to the imaging device _601L . The drive unit _602L drives the image sensor _601L , performs predetermined processing such as noise removal and gain adjustment on the signal output from the image sensor _601L , and outputs the processed signal. The signal processing unit _603L performs A/D conversion on the signal output from the driving unit _602L to convert the signal into a digital image signal (captured image). The signal processing unit 603 _L performs predetermined image processing such as gamma correction on the converted image signal and outputs the processed image signal. The captured image output from the signal processing unit 603 _L is supplied to the output unit 604 .

The operations of the imaging optical system 600 _R , the imaging device 601 _R , the driving unit 602 _R , and the signal processing unit 603 _R are similar to those of the imaging optical system 600 _L , the imaging device 601 _L , the driving unit 602 _L , and the signal processing unit described above. Since it is the same as _603L , the explanation here is omitted.

The driving units 602 _L and 602 _R are supplied with triggers output from the information processing device 50, for example. This trigger is supplied in synchronization with drive units _602L and _602R included in the road surface imaging camera 6R. The driving units 602 _L and 602 _R take in signals from the imaging elements 601 _L and 601 _R according to this trigger and perform imaging.

Here, the drive units 602 _L and 602 _R perform exposure in the imaging elements 601 _L and 601 _R by a batch simultaneous exposure method. This method is called a global shutter. On the other hand, since the rolling shutter takes in light in order from the top of the pixel position (line order), each line in the frame does not exactly capture the subject at the same time. In the case of the rolling shutter method, if the camera or the object moves at high speed while one frame of imaging signal is captured, the image of the object is picked up shifted according to the line position. Therefore, in the road surface imaging cameras 6L and 6R according to the first embodiment, a global shutter is used so that the road shape is imaged correctly in terms of projection geometry.

Further, setting information including imaging conditions related to exposure output from the information processing device 50 is supplied to the driving units _602L and _602R and the signal processing units _603L and _603R , respectively. The driving units 602 _L and 602 _R set, for example, gain and exposure time according to this setting condition. In addition, the signal processing units 603 _L and 603 _R can set gamma coefficients and gradations for gamma correction according to these setting conditions.

The output unit 604 outputs the captured images of each frame supplied from the signal processing units 603 _L and 603 _R as a set of stereo captured images. The stereo captured images output from the output unit 604 are sent to the information processing device 50 and accumulated.

FIG. 11 is a block diagram showing an example configuration of an information processing apparatus 50 applicable to the first embodiment. 11, the information processing device 50 includes a CPU (Central Processing Unit) 5000, a ROM (Read Only Memory) 5001, a RAM (Random Access Memory) 5002, and a graphics I/F (interface) connected to a bus 5030. ) 5003 , a storage 5004 , an input device 5005 , a data I/F 5006 and a communication I/F 5007 . Further, the information processing apparatus 50 includes camera I/Fs 5010a and 5010b connected to the bus 5030, a speed acquisition section 5021, and a position information acquisition section 5022, respectively.

The storage 5004 is a storage medium that stores data in a nonvolatile manner, and can be a hard disk drive or flash memory. The storage 5004 stores programs and data for the CPU 5000 to operate.

The CPU 5000 uses the RAM 5002 as a work memory according to programs pre-stored in the ROM 5001 and the storage 5004 , for example, and controls the overall operation of the information processing apparatus 50 . The graphics I/F 5003 generates display signals that can be handled by the display 5020 based on display control signals generated by the CPU 5000 according to a program. A display 5020 displays a screen according to the display signal supplied from the graphics I/F 5003 .

The input device 5005 receives user operations and outputs control signals according to the received user operations. As the input device 5005, a pointing device such as a mouse or a tablet, or a keyboard can be applied. Further, the input device 5005 and the display 5020 may be integrally formed to form a so-called touch panel configuration.

A data I/F 5006 transmits and receives data to and from an external device. As the data I/F 5006, for example, a USB (Universal Serial Bus) is applicable. A communication I/F 5007 controls communication with an external network according to instructions from the CPU 5000 .

The camera I/F 5010a is an interface for the road surface imaging cameras 6L and 6R. Each stereo picked-up image output from each road surface imaging camera 6L and 6R is passed to CPU5000, for example via camera I/F5010a. In addition, the camera I/F 5010a generates the above-described triggers according to instructions from the CPU 5000, and sends the generated triggers to the road surface imaging cameras 6L and 6R. Furthermore, the camera I/F 5010a sends the setting information generated by the CPU 5000 to the road surface imaging cameras 6L and 6R.

A camera I/F 5010b is an interface for the traveling direction camera 7 . The traveling direction captured image output from the traveling direction camera 7 is passed to, for example, the CPU 5000 via the camera I/F 5010b. The camera I/F 5010b also sends a trigger, which is generated by the CPU 5000, to instruct the imaging timing, to the traveling direction camera 7. FIG.

The speed acquisition unit 5021 acquires speed information indicating the speed of the vehicle 1 . When the road surface imaging cameras 6L and 6R are attached to the vehicle 1, the speed information acquired by the speed acquisition unit 5021 indicates the speed of the road surface imaging cameras 6L and 6R with respect to the subject (road surface). The speed acquisition unit 5021 has, for example, a function of receiving a GNSS (Global Navigation Satellite System) signal, and acquires speed information indicating the speed of the vehicle 1 based on the Doppler effect of the received GNSS signal. The speed acquisition unit 5021 can also acquire the speed information directly from the vehicle 1 without being limited to this.

The position information acquisition unit 5022 acquires position information indicating the current position of the vehicle 1 . The position information acquisition unit 5022 acquires the position information using, for example, the GNSS signal reception function of the velocity acquisition unit 5021 .

FIG. 12 is an example functional block diagram for explaining functions of the information processing apparatus 50 applicable to the first embodiment. In FIG. 12 , the information processing apparatus 50 includes captured image acquisition units 500 a and 500 b, a UI unit 501 , a control unit 502 , a captured image storage unit 503 and an imaging control unit 504 . The information processing device 50 further includes a matching processing unit 510 , a 3D information generation unit 511 , a 3D information acquisition unit 520 , a state characteristic value calculation unit 521 , and a record creation unit 522 .

These captured image acquisition unit 500, UI unit 501, control unit 502, imaging control unit 504, matching processing unit 510, 3D information generation unit 511, 3D information acquisition unit 520, state characteristic value calculation unit 521, and record creation unit 522 are It is implemented by a program running on the CPU 5000 . The captured image acquisition unit 500, the UI unit 501, the control unit 502, the imaging control unit 504, the matching processing unit 510, the 3D information generation unit 511, the 3D information acquisition unit 520, the state characteristic value calculation unit 521, and the record A part or all of the creation unit 522 may be configured by hardware circuits that operate in cooperation with each other.

The captured image acquisition unit 500a acquires stereo captured images from the road surface imaging cameras 6L and 6R. The captured image acquisition unit 500b acquires a traveling direction captured image from the traveling direction camera 7 . The captured image storage unit 503 stores the stereo captured images and the traveling direction captured images acquired by the captured image acquisition units 500a and 500b in the storage 5004, for example. Also, the captured image storage unit 503 acquires the stored stereo captured image and traveling direction captured image from the storage 5004 .

The UI unit 501 implements a user interface by displaying on the input device 5005 and display 5020 . A control unit 502 controls the operation of the information processing apparatus 50 as a whole.

The imaging control unit 504 corresponds to the position information acquisition unit 101 and the imaging control unit 102 described above. That is, the imaging control unit 504 generates imaging conditions for the traveling direction camera 7 based on the traveling direction captured image acquired by the captured image acquiring unit 500b. Based on the position information indicating the current position of the vehicle 1 acquired by the position information acquisition unit 101, the imaging control unit 504 generates a trigger for instructing the traveling direction camera 7 to take an image.

In addition, the imaging control unit 504 acquires speed information indicating the speed of each of the road surface imaging cameras 6L and 6R with respect to the subject (road surface 4). A trigger for instructing synchronous imaging of the road surface imaging cameras 6L and 6R is generated based on the angle of view α and the height h of .

The matching processing unit 510 performs matching processing using the two captured images forming the stereo captured image acquired by the captured image acquisition unit 500 . The 3D information generation unit 511 performs processing related to 3D information. For example, the 3D information generation unit 511 obtains depth information by trigonometry or the like using the result of matching processing by the matching processing unit 510, and generates 3D point group information based on the obtained depth information.

The 3D information acquisition unit 520 acquires the 3D point cloud information obtained for each stereo image by the 3D information generation unit 511 . The state characteristic value calculation unit 521 uses each three-dimensional point cloud information acquired by the 3D information acquisition unit 520 and each stereo captured image acquired by the captured image acquisition unit 500 to determine the cracks for obtaining MCI. Each state characteristic value of the rate C, the amount of rutting D, and the flatness σ is calculated. The report creating unit 522 obtains MCI based on each state characteristic value calculated by the state characteristic value calculating unit 521 and creates a report.

A program for realizing each function according to the first embodiment in the information processing device 50 is a file in an installable format or an executable format, and can be stored on a CD (Compact Disk), a flexible disk (FD), or a DVD (Digital Versatile). Disc) or other computer-readable recording medium. Alternatively, the program may be provided by storing it on a computer connected to a network such as the Internet and downloading it via the network. Also, the program may be configured to be provided or distributed via a network such as the Internet.

The program includes captured image acquisition units 500a and 500b, UI unit 501, control unit 502, captured image storage unit 503, imaging control unit 504, matching processing unit 510, 3D information generation unit 511, 3D information acquisition unit 520, state property It has a module configuration including a value calculation unit 521 and a record creation unit 522 . As actual hardware, the CPU 5000 reads out the program from a storage medium such as the storage 5004 and executes it, so that the above-described units are loaded onto the main storage such as the RAM 5002, and the captured image acquisition units 500a and 500b, the UI A unit 501, a control unit 502, a captured image storage unit 503, an imaging control unit 504, a matching processing unit 510, a 3D information generation unit 511, a 3D information acquisition unit 520, a state characteristic value calculation unit 521, and a record creation unit 522 are main storage devices. generated above.

[Trigger Generation Method Applicable to First Embodiment]
Next, a method of generating triggers for instructing the road surface imaging cameras 6L and 6R to perform imaging, which is applicable to the first embodiment, will be described in more detail. In the first embodiment, the generation unit 103 determines the distance in the direction of the speed of the vehicle 1 in the stereo imaging range of the road surface imaging cameras 6L and 6R with respect to the object to be measured (road surface 4). Triggers are generated at time intervals of a predetermined time or less, which is shorter than the time during which the object moves at a high speed.

That is, the trigger needs to be generated so that the range of the road surface 4 photographed by the road surface imaging cameras 6L and 6R maintains a predetermined overlapping rate (advancing direction overlapping rate Dr) in the traveling direction of the vehicle 1 . As will be described later, the purpose of this is to detect sufficient corresponding points in order to stably and accurately calculate the camera position in the process of calculating the imaging position from each stereo imaged image. The lower limit of the traveling direction overlap rate Dr is experimentally determined, for example, as "60%".

In the first embodiment, the following three methods can be applied to the trigger generation method.
(1) Method of generating at regular time intervals (first generation method)
(2) A method of generating by detecting the movement speed of the camera (second generation method)
(3) A method of calculating and generating a movement distance using a captured image (third generation method)

(First generation method)
First, the first trigger generation method will be described. In the first generation method, the trigger time interval is determined from the maximum speed Speed of the vehicle 1 during imaging and the size of the imaging range (the length of the imaging range in the traveling direction of the vehicle 1). The speed acquisition unit 5021 acquires the maximum speed Speed of the vehicle 1 in advance based on system setting values in the vehicle 1 and user input to the information processing device 50 . The imaging control unit 504 acquires the maximum speed Speed from the speed acquisition unit 5021, and uses the acquired maximum speed Speed, the traveling direction visual field Vp, and the traveling direction overlapping rate Dr to obtain the trigger time interval using the following equation: Calculated by (4). Note that the above-described lower limit value is applied to the traveling direction overlapping rate Dr.

The number of triggers fps to be generated per second is calculated by the formula (4). The reciprocal of the number of triggers fps is the time interval until the next trigger to be generated.

As described with reference to FIG. 4, the traveling direction visual field Vp is schematically defined by the height h of the road surface imaging cameras 6L and 6R from the road surface 4 and the angle of view of the road surface imaging cameras 6L and 6R. can be set based on α. In practice, the angle of the road surface imaging cameras 6L and 6R with respect to the road surface 4 is also taken into consideration when setting the traveling direction visual field Vp.

Here, when the moving vehicle 1 curves to the right or left, the amount of movement of the outer camera of the road surface imaging cameras 6L and 6R increases. Therefore, it is more preferable to use the speed of rotational movement based on the position of the outer camera instead of using the maximum speed Speed of the vehicle 1 as it is.

The imaging control unit 504 stores and accumulates all the stereo imaged images imaged according to the generated trigger in the storage 5004 or the RAM 5002, for example.

(Second generation method)
Next, a second trigger generation method will be described. While the above-described first method is simple, when the vehicle 1 is stopped or moving at a speed lower than a predetermined speed, images are captured at excessively fine intervals with respect to the maximum speed Speed. As a result, the amount of stereo imaged images to be accumulated becomes large. In the second generation method, the imaging control unit 504 detects the moving speed of the camera and generates a trigger according to the detected moving speed.

The imaging control unit 504 uses the current speed of the vehicle 1 indicated by the speed information acquired by the speed acquisition unit 5021 as the maximum speed Speed in expression (4) to calculate the time interval until the next trigger, and performs imaging. I do. According to the second generation method, the slower the moving speed of the vehicle 1 is, the longer the time interval of trigger generation is, thereby suppressing wasteful imaging.

The imaging control unit 504 stores and accumulates all the stereo imaged images imaged according to the generated trigger in the storage 5004 or the RAM 5002, for example.

Note that this second generation method and the above-described first generation method can be implemented in combination.

(Third generation method)
Next, a third trigger generation method will be described. In the third generation method, as in the first generation method described above, triggers are generated at regular time intervals based on the maximum speed Speed of the vehicle 1 . Here, in the third generation method, not all the stereo images captured in response to the trigger are accumulated, but only when the traveling direction overlapping rate Dr is less than a preset value.

As will be described later, it is possible to calculate the moving distance of the camera (vehicle 1) using only the stereo captured images captured so as to overlap with the traveling direction of the vehicle 1 .

The imaging control unit 504 calculates the moving distance of the camera using the last accumulated stereo image and the immediately preceding stereo image. The imaging control unit 504 determines whether or not the calculated distance exceeds the moving distance corresponding to the lower limit of the traveling direction overlapping rate Dr. If the imaging control unit 504 determines that the threshold has been exceeded, the imaging control unit 504 accumulates the previously captured stereo image. On the other hand, if the image capturing control unit 504 determines that it does not exceed the threshold, it discards the stereo captured image captured immediately before.

As a result, even without using a sensor for measuring the current speed of the vehicle 1, wasteful image accumulation is not performed when the moving speed is low.

[Method for Calculating Road Condition Value Applicable to First Embodiment]
Next, a method of calculating road surface property values applicable to the first embodiment will be described. A method of measuring the flatness σ, the amount of rutting D, and the crack ratio C applicable to the first embodiment will be described below.

(flatness)
FIG. 13 is a flowchart of an example of flatness calculation processing applicable to the first embodiment. In step S100, the captured image acquisition unit 500 acquires stereo captured images captured by the road surface imaging cameras 6L and 6R and stored in the storage 5004, for example. When the stereo imaged images are acquired, the process moves to steps S101a and S101b, which can be processed in parallel.

In step S101a, a depth map is generated based on the stereo captured image acquired in step S100. Depth map generation processing applicable to the first embodiment will be described with reference to FIGS. 14 and 15 . FIG. 14 is an exemplary flowchart illustrating depth map generation processing applicable to the first embodiment.

In step S<b>120 , the matching processing unit 510 acquires stereo captured images from the captured image acquisition unit 500 . In the next step S121, the matching processing unit 510 performs matching processing based on the two captured images forming the acquired stereo captured image. In the next step S122, the 3D information generation unit 511 calculates depth information based on the matching processing result of step S121, and generates a depth map, which is three-dimensional point group information.

The processing of steps S121 and S122 will be described more specifically. In the first embodiment, depth information is calculated by a stereo method using two captured images that constitute a stereo captured image. The stereo method here uses two captured images captured from different viewpoints by two cameras, and for a certain pixel (reference pixel) in one captured image, a corresponding pixel (reference pixel) in the other captured image ( In this method, the corresponding pixel) is obtained, and the depth (depth value) is calculated by triangulation based on the reference pixel and the corresponding pixel.

In step S121, the matching processing unit 510 uses the two captured images constituting the stereo captured image, which are acquired from the captured image acquisition unit 500, and a predetermined size centering on the reference pixel in one captured image serving as a reference. A search is performed by moving a region corresponding to the region in the other captured image to be searched within the other captured image.

Various methods are known for searching for corresponding pixels, and for example, a block matching method can be applied. The block matching method acquires pixel values of a region cut out as a block of M pixels×N pixels centering on a reference pixel in one captured image. Also, in the other captured image, a pixel value of a region cut out as a block of M pixels×N pixels centering on the target pixel is acquired. Based on the pixel values, the degree of similarity between the area containing the reference pixel and the area containing the target pixel is calculated. The similarity is compared while moving blocks of M pixels×N pixels in the search target image, and the target pixel in the block at the position where the similarity is highest is taken as the corresponding pixel corresponding to the reference pixel.

Similarity can be calculated by various calculation methods. For example, the normalized cross-correlation (NCC) shown in Equation (5) is one of the cost functions, and the higher the value of the numerical value C _NCC indicating the cost, the higher the similarity. indicates In equation (5), the values M and N represent the size of the pixel block to search. Also, the value I(i, j) represents the pixel value of a pixel in a pixel block in one of the reference captured images, and the value T(i, j) represents the pixel block in the other captured image to be searched. represents the pixel value in

As described above, the matching processing unit 510 moves the pixel block in the other captured image corresponding to the pixel block of M pixels×N pixels in the other captured image, for example, in units of pixels. Calculate the numerical value C _NCC by performing the calculation of equation (5). In the other captured image, the center pixel of the pixel block with the largest numerical value C _NCC is set as the target pixel corresponding to the reference pixel.

Returning to the description of FIG. 14, in step S122, the 3D information generation unit 511 calculates a depth value (depth information) using trigonometry based on the reference pixels and corresponding pixels obtained by the matching processing in step S121. Then, three-dimensional point cloud information relating to one captured image and the other captured image that constitute the stereo captured image is generated.

FIG. 15 is a diagram for explaining trigonometry applicable to the first embodiment. Calculating the distance S to the target object 403 (for example, one point on the road surface 4) in the figure from the imaging position information in the image captured by each imaging element 402 (corresponding to the imaging elements 601 _L and 601 _R ). is the purpose of processing. That is, this distance S corresponds to the depth information of the target pixel. The distance S is calculated by Equation (6) below.

In equation (6), the value baseline represents the length of the baseline (baseline length) between the cameras 400a and 400b. In the example of FIG. 5, this corresponds to the baseline length by the imaging lenses 6L _L and 6L _R (in the case of the road surface imaging camera 6L). The value f represents the focal length of the lens 401 (corresponding to the imaging lenses 6L _L and 6L _R ). The value q represents parallax. The parallax q is a value obtained by multiplying the difference between the coordinate values of the reference pixel and the corresponding pixel by the pixel pitch of the imaging device. The coordinate values of the corresponding pixels are obtained based on the result of matching processing in step S121.

This formula (6) is a method of calculating the distance S when the two cameras 400a and 400b, that is, the imaging lenses 6L _L and 6L _R are used. This is to calculate the distance S from the captured images captured by the two cameras 400a and 400b, that is, the imaging lenses 6L _L and 6L _R , respectively. In the first embodiment, the calculation method based on this equation (6) is performed by the imaging lenses 6L _L and _{6LR of the road surface imaging camera 6L and the imaging lenses 6RL and 6R R of the} road surface imaging camera 6R. Applied to each captured image, the distance S is calculated for each pixel.

Returning to the description of FIG. 13, in step S101b, the 3D information generation unit 511 estimates the camera position and orientation. Here, the camera position indicates, for example, the coordinates of the center of the road surface imaging camera 6L when viewed as one unit. Details of the processing in step S101b will be described later.

When the processing of steps S101a and S101b ends, the processing moves to step S102. In step S102, the 3D information generating unit 511 converts two depth maps that are adjacent in time series to match the coordinate system of the depth map at the previous time based on the camera position and orientation estimated in step S101b. Coordinate transformation of the depth map of the next time after the time is performed.

Note that the previous time and the next time are the first time and the second time that the road surface imaging cameras 6L and 6R continuously perform the first and second imaging in time series in response to a trigger. The time at which the image was captured is the previous time, and the time at which the second imaging is performed is the next time. That is, the depth map at the previous time is a depth map generated based on the stereo image captured by the first imaging, and the depth map at the next time is a depth map generated based on the stereo image captured by the second imaging. be.

In the next step S103, the 3D information generation unit 511 integrates the depth map of the next time coordinate-transformed in step S102 into the depth map of the previous time. That is, the two depth maps of the previous time and the next time are arranged in a common coordinate system by the coordinate conversion in step S102, so that these two depth maps can be integrated. The processing of steps S102 and S103 is performed on stereo captured images captured at all times.

As shown in FIGS. 5 and 7, when a plurality of road surface imaging cameras 6L and 6R or road surface imaging cameras 6L, 6C and 6R are arranged side by side in the width direction of the road, road surface imaging cameras arranged in the road width direction are used. The camera positions and orientations of the road surface imaging cameras (for example, the road surface imaging cameras 6L and 6R) are estimated in the same manner, and depth map integration is then performed.

Further, the moving distance of the camera (vehicle 1) in the above-described third generation method can be calculated in the coordinate conversion in step S102 and the depth map integration processing in step S103.

In the next step S104, the 3D information acquisition unit 520 acquires the depth map integrated in step S103 from the 3D information generation unit 511. FIG. Then, the state characteristic value calculator 521 calculates the flatness σ based on the depth map acquired by the 3D information acquirer 520 . That is, when the depth maps are integrated, the road surface shape of the road in the measured section is restored as a point cloud in one three-dimensional space. When the point cloud is restored, for each sampling point on the road surface, take a coordinate system that connects the coordinates of the 1.5m points before and after according to the regulations, and calculate the distance to the 3D point cloud at the center, using the formula The amount of displacement d can be calculated in the same manner as in (2), and the flatness σ can be calculated from the amount of displacement d according to equation (3).

(rutting amount)
Regarding the rutting amount D, in step S104, the state characteristic value calculation unit 521 scans the depth map acquired by the 3D information acquisition unit 520 in the road width direction to obtain the values shown in FIGS. ), the depths D ₁ and D ₂ can be obtained. _The rutting amount D can be obtained based on the obtained depths D1 and D2 and the cross _- sectional information obtained by scanning the depth map.

(crack rate)
For example, the state characteristic value calculation unit 521 applies the stereo captured image acquired in step S100 to the depth map integrated in step S103. That is, the state characteristic value calculation unit 521 integrates each stereo captured image that is overlapped in the direction of travel of the vehicle 1 with the rate of overlap of the direction of travel Dr. The state characteristic value calculation unit 521 sets a mesh of 50 cm according to regulations for the integrated image, acquires information such as cracks and patching in each mesh by image analysis, and calculates the crack rate C according to regulations based on the acquired information. Calculate

(Processing for estimating camera position and orientation)
Next, the estimation processing of the camera position and direction in step S101b in the flowchart of FIG. 13 described above will be described in more detail. FIG. 16 is a flowchart of an example of camera position and orientation estimation processing applicable to the first embodiment. In a first embodiment, the camera position and orientation are estimated using the SfM described above.

In step S<b>130 , the 3D information generation unit 511 acquires stereo captured images from the captured image acquisition unit 500 . Here, the 3D information generation unit 511 acquires two stereo captured images (a stereo captured image at the previous time and a stereo captured image at the next time) continuously captured in time series. Next, in step S131, the 3D information generation unit 511 extracts feature points from each acquired stereo image. This feature point extraction process is a process of finding points that can be easily detected as corresponding points in each stereoscopic image. do.

In the next step S132, the 3D information generation unit 511 detects, from the stereo image of the next time, the point captured at the same place as the point extracted as the feature point in the stereo image of the previous time. For this detection processing, a technique called optical flow or a technique called feature point matching represented by SIFT (Scale-Invariant Feature Transform), SURF (Speed-Upped Robust Feature), and the like can be applied.

At the next step S133, the 3D information generation unit 511 estimates the initial position and orientation of the camera. The 3D information generating unit 511 fixes the coordinates of the corresponding points detected in each stereoscopic image in step S132, and solves the simultaneous equations using the position and orientation of the camera that captured the stereoscopic image at the next time as parameters. , the position of the camera is estimated, and the three-dimensional coordinates are calculated (step S134).

A method for estimating the initial position and orientation of the camera will be described with reference to FIG. Equation (7) expresses the relationship between the coordinates x _ij of the spatial point X _j projected onto the camera (viewpoint) P _i . In equation (7), the value n represents the number of points in the three-dimensional space, and the value m represents the number of cameras (captured images).

Each value P is a projection matrix that transforms the coordinates of a point in the three-dimensional space into the two-dimensional coordinates of the image for each camera, and is expressed by the following equation (8). Equation (8) consists of a transformation matrix of 2 rows and 2 columns of three-dimensional coordinates shown on the right side, and a projective transformation f _i from the three-dimensional coordinates to the two-dimensional coordinates.

In FIG. 17, the values X _j and P _i are calculated using simultaneous equations given only the coordinates x _ij captured by the camera. A linear least-squares method can be used to solve this simultaneous equation.

At the next step S135, the 3D information generation unit 511 optimizes the three-dimensional coordinates indicating the camera position calculated at step S134. The position and orientation of the camera calculated by solving the system of equations may not have sufficient accuracy. Therefore, the accuracy is improved by performing the optimization process using the calculated value as the initial value.

The two-dimensional coordinates x _ij (reprojection (called coordinates) and the difference is called the residual. For all corresponding points in all stereoscopic images used to calculate the position and orientation of the camera, the parameters are adjusted by an optimization operation so that the sum of residual errors is minimized. This is called bundle adjustment. Global optimization by bundle adjustment can improve the accuracy of camera position and orientation.

The above is the base processing of camera position and orientation estimation in normal SfM. In the first embodiment, since the road surface imaging cameras 6L and 6R with known base line lengths are used, corresponding points are detected in the stereo images captured by overlapping the imaging ranges according to the movement of the vehicle 1. In addition, it is easy to search for corresponding points in two captured images that constitute a stereo captured image. Therefore, the three-dimensional coordinates X _j in space are determined on an actual scale (actual size), and the position and orientation of the camera after movement can also be stably calculated.

As described above, in the imaging system 10 applicable to the first embodiment, the information processing device 50 generates triggers, and distributes the generated triggers to the road surface imaging cameras 6L and 6R. Thus, the road surface imaging cameras 6L and 6R are configured to perform imaging in synchronism. At this time, clock generators of the road surface imaging cameras 6L and 6R, trigger distribution components for distributing triggers in the information processing device 50, trigger wiring lengths in the information processing device 50 and the road surface imaging cameras 6L and 6R , there is a possibility that a slight deviation may occur in the timing at which the road surface imaging cameras 6L and 6R actually capture the trigger and execute imaging. It is preferable to suppress this imaging timing lag as much as possible.

Especially between the road surface imaging cameras 6L and 6R whose baseline lengths are known, it is desirable to minimize the deviation in imaging timing. For example, when wiring the trigger supply from the camera I/F 5010a to the road surface imaging cameras 6L and 6R, a repeater for ensuring trigger quality and a photocoupler for protection from electrostatic noise are used. Sometimes. In this case, it is desirable to share these repeaters and photocouplers between the road surface imaging cameras 6L and 6R.

[Processing for Determining Imaging Conditions According to First Embodiment]
Next, the imaging condition determination process according to the first embodiment will be described in more detail. In the first embodiment, even when the stereo imaging range captured by the road surface imaging cameras 6L and 6R includes a large luminance change, the road surface imaging cameras 6L and 6R can perform imaging under appropriate imaging conditions. It is possible.

FIG. 18 is a diagram showing an example of a location on the road surface 4 where there is a large luminance change. Note that FIGS. 18A and 18B show examples of images captured while traveling from the bottom to the top of the drawing.

FIG. 18(a) is an example of a portion that enters the shade from the sun. The lower half of the image is a sunny area 41 and the upper half is a shaded area 40. The shaded area 40 has lower road luminance than the sunny area 41. FIG. FIG. 18(b) is an example of a part where the road surface 4 is newly paved from the middle. The upper portion of the image is a newly paved area 42 that has been newly repaved at a location that has just been repaired. The newly paved area 42 has a lower reflectance than the other area 43 of the road surface 4, and has a lower road luminance.

In both FIGS. 18A and 18B, when a dark place (the shaded area 40 and the new pavement area 42) is imaged with the exposure set for the bright place (the sunny area 41 and the other area 43), it becomes dark. Images with sufficient contrast cannot be obtained in some areas. Also, even when the image is captured using automatic exposure, if the vehicle 1 is running and the image is captured, an image with insufficient contrast may be captured due to the time lag in following the automatic exposure.

Therefore, in the first embodiment, the traveling direction camera 7 captures an image of the stereo imaging range captured by the road surface imaging cameras 6L and 6R prior to imaging by the road surface imaging cameras 6L and 6R. Based on the traveling direction captured image thus obtained, the imaging conditions for the road surface imaging cameras 6L and 6R are determined.

19 and 20 are flowcharts of an example of imaging condition determination processing according to the first embodiment. Here, the traveling direction camera 7 and the road surface imaging cameras 6L and 6R are synchronized in imaging by a trigger supplied from the information processing device 50. FIG. In the flowcharts of FIGS. 19 and 20, when one frame is captured by the traveling direction camera 7, the imaging conditions of the road surface imaging cameras 6L and 6R are determined based on the traveling direction captured image obtained by the imaging. do.

First, the processing according to the flowchart of FIG. 19 will be described. The flowchart of FIG. 19 shows processing related to the traveling direction camera 7 according to the first embodiment. In step S<b>200 , the information processing apparatus 50 acquires position information indicating the current position from the position information acquisition unit 51 by the imaging control unit 504 . In the next step S201, the captured image acquisition unit 500b acquires a traveling direction captured image captured by the traveling direction camera 7. FIG. The acquired traveling direction captured image is stored in the memory (for example, the RAM 5002 ) of the information processing apparatus 50 by the captured image storage unit 503 .

In the next step S202, the information processing apparatus 50 causes the imaging control unit 504 to analyze the traveling direction captured image stored in step S201, and determines whether or not the imaging conditions need to be changed. The imaging control unit 504 calculates the brightness of the imaging range (imaging road surface) of the traveling direction camera 7 from the exposure setting of the traveling direction camera 7 and the image brightness in the attention area of the traveling direction captured image. Depending on the brightness, it is determined whether or not the imaging conditions need to be changed, and if so, what setting to use. Here, the imaging condition is, for example, exposure setting.

For example, when the change in road surface brightness exceeds a threshold value, the image capturing control unit 504 reduces the brightness so that the captured image becomes darker when the brightness changes in the brighter direction according to the amount of change. The exposure parameter is determined so that the captured image becomes brighter when the direction changes. The exposure parameter is at least one of exposure time and gain.

In step S203, when the imaging control unit 504 determines that the imaging conditions need to be changed by the determination processing in step S202 (step S203, "yes"), the process proceeds to step S204. In step S204, the imaging control unit 504 associates the changed imaging conditions with change position information indicating the position for changing the imaging conditions, and stores them in the RAM 5002, for example. This is because the traveling direction camera 7 and the road surface imaging cameras 6L and 6R have different imaging directions. This is because it is necessary to change the exposure at appropriate timing.

Note that the changed position information is, for example, the position information acquisition unit 5021 (information processing device 50 or GNSS receiver) of the imaging range imaged by the traveling direction camera 7 in addition to the position information acquired in step S200. can be applied by adding the distance from

When the storage of the setting information and the changed position information in step S204 is completed, the series of processing according to the flowchart of FIG. 19 is terminated. If it is determined in step S203 described above that there is no need to change the exposure setting (step S203, "No"), the series of processes according to the flowchart of FIG. 19 is terminated.

The processes of steps S200 to S204 described above are repeatedly executed at predetermined time intervals.

FIG. 20 is a flow chart showing an example of processing related to the road surface imaging cameras 6L and 6R according to the first embodiment. The information processing apparatus 50 acquires position information at high frequency at predetermined time intervals by the position information acquisition unit 5022 . In the information processing device 50, the imaging control unit 504 changes the current position from the changed position information stored in step S204 of FIG. is the position indicated by .

When the imaging control unit 504 determines that the current position is the position indicated by the changed position information (step S300, "Yes"), the process proceeds to step S301. In step S301, the imaging control unit 504 sets the setting information stored in association with the changed position information in step S204 of FIG. Change the 6R imaging conditions.

In the next step S302, the imaging control unit 504 generates a trigger for instructing imaging, sends it to the road surface imaging cameras 6L and 6R, and according to the trigger, each stereo imaged by the road surface imaging cameras 6L and 6R is captured. Acquire a captured image. The imaging control unit 504 stores each acquired stereoscopic image in, for example, the RAM 5002 and the storage 5004 .

On the other hand, if the imaging control unit 504 determines in step S300 that the current position is not the position indicated by the changed position information, the process proceeds to step S302, generates a trigger for instructing imaging, and performs road surface imaging. Then, according to the trigger, each stereo captured image captured by the road surface imaging cameras 6L and 6R is obtained.

The imaging system 10 according to the first embodiment repeatedly performs each process according to the flowcharts of FIGS.

According to the first embodiment, the traveling direction camera 7 acquires the brightness information of the stereo imaging range of the road surface imaging cameras 6L and 6R prior to imaging by the road surface imaging cameras 6L and 6R, and the imaging conditions are set. have decided. Therefore, the imaging of the stereo imaging range by the road surface imaging cameras 6L and 6R can be performed under appropriate imaging conditions. As a result, the crack rate C, the amount of rutting D, and the flatness σ can be calculated with higher accuracy based on the stereo images captured by the road surface imaging cameras 6L and 6R, and the reliability of evaluation by MCI using these can be calculated. It is possible to improve

[First Modification of First Embodiment]
Next, the 1st modification of 1st Embodiment is demonstrated. In a first modification of the first embodiment, for each of a plurality of road surface imaging cameras (for example, road surface imaging cameras 6L and 6R) for imaging the road surface 4 for MCI calculation, the imaging system 10 includes: The imaging conditions are set.

The imaging condition determination process according to the first modification of the first embodiment will be described with reference to FIGS. 21 to 24. FIG. In the same manner as described above, the traveling direction camera 7 and the road surface imaging cameras 6L and 6R are synchronized in imaging by a trigger supplied from the information processing device 50. FIG. In the first modification of the first embodiment, when one frame is imaged by the traveling direction camera 7, each of the road surface imaging cameras 6L and 6R is detected based on the traveling direction captured image obtained by the imaging. Determine imaging conditions.

FIG. 21 is a flowchart of an example of processing related to the traveling direction camera 7 according to the first modification of the first embodiment. In step S<b>210 , the information processing apparatus 50 acquires position information indicating the current position from the position information acquisition unit 51 by the imaging control unit 504 . In the next step S211, the captured image acquisition unit 500b acquires the traveling direction captured image captured by the traveling direction camera 7. FIG. The acquired traveling direction captured image is stored in the memory (for example, the RAM 5002 ) of the information processing apparatus 50 by the captured image storage unit 503 .

In the next step S212, the information processing apparatus 50 causes the imaging control unit 504 to analyze the traveling direction captured image stored in step S211, and determines whether or not the imaging conditions need to be changed. The imaging control unit 504 calculates the brightness of the imaging range (imaging road surface) of the traveling direction camera 7 from the exposure setting of the traveling direction camera 7 and the image brightness in the attention area of the traveling direction captured image. Depending on the brightness, it is determined whether or not the imaging conditions need to be changed, and if so, what setting to use. Here, the imaging condition is, for example, an exposure parameter.

For example, when the change in road surface brightness exceeds a threshold value, the image capturing control unit 504 reduces the brightness so that the captured image becomes darker when the brightness changes in the brighter direction according to the amount of change. The exposure parameter is determined so that the captured image becomes brighter when the direction changes. In the first modification of the first embodiment, the exposure parameters are individually determined for each of the road surface imaging cameras 6L and 6R.

FIG. 22 is a diagram for explaining the relationship between the imaging range of the traveling direction camera 7 and the stereo imaging range of the road surface imaging cameras 6L and 6R, which can be applied to the first modification of the first embodiment. is.

As shown in FIG. 22, the road surface imaging cameras 6L and 6R cover the rear of the vehicle 1. The imaging ranges 60L and 60R of the road surface imaging cameras 6L and 6R overlap each other, and the vehicle 1 is traveling. Imaging is performed so that the entire width direction of the lane on the road surface 4 in the middle falls within either of the imaging ranges 60L and 60R. For example, the road surface imaging cameras 6L and 6R are arranged so that the white lines 44 and 45 indicating both ends of the lane are included in either of the imaging ranges 60L and 60R. Further, the traveling direction camera 7 captures an image in front of the vehicle 1 in the traveling direction so that the entire width direction of the lane is included in the photographing range 70 .

FIG. 23 is a diagram showing an example of an image captured by the road surface imaging cameras 6L and 6R and the traveling direction camera 7 applicable to the first modification of the first embodiment. In addition, in FIG. 23, the same reference numerals are given to the same parts as in FIG. 23 described above, and detailed description thereof will be omitted. FIG. 23(a) corresponds to FIG. 23 described above, and FIG. 23(b) shows an example of a traveling direction captured image 71 in which the imaging range 70 is captured by the traveling direction camera 7. FIG. Also, FIG. 23(c) shows an example of stereo captured images 61L and 61R captured by the road surface imaging cameras 6L and 6R in the imaging ranges 60L and 60R.

In determining the imaging conditions (exposure parameters) of the road surface imaging cameras 6L and 6R based on the traveling direction captured image of the traveling direction camera 7, the evaluation area for evaluating the brightness of the traveling direction captured image 71 is: It is determined corresponding to each of the stereo imaged images 61L and 61R. As shown in FIG. 23(b), evaluation areas 710L and 710R for luminance evaluation corresponding to the respective stereo images 61L and 61R are located at the center of the imaging ranges 60L and 60R of the respective road surface imaging cameras 6L and 6R. It is set according to the neighborhood. The imaging conditions (exposure parameters) of the road surface imaging cameras 6L and 6R are individually determined according to the luminance information in the evaluation areas 710L and 710R.

In step S213, when the imaging control unit 504 determines that the imaging conditions need to be changed by the determination processing in step S212 (step S213, "yes"), the process proceeds to step S214. In step S204, the imaging control unit 504 associates the changed imaging conditions with changed position information indicating positions for changing the imaging conditions for each of the road surface imaging cameras 6L and 6R, and stores them in the RAM 5002, for example. .

When the storage of the setting information and the changed position information for each of the road surface imaging cameras 6L and 6R in step S214 is completed, the series of processing according to the flowchart of FIG. 21 is terminated. If it is determined in step S213 described above that there is no need to change the exposure setting (step S213, "NO"), the series of processes according to the flowchart of FIG. 21 is terminated.

The processes of steps S210 to S214 described above are repeatedly executed at predetermined time intervals.

FIG. 24 is a flow chart showing an example of processing related to the road surface imaging cameras 6L and 6R according to the first modification of the first embodiment. In the information processing device 50, the imaging control unit 504 changes the current position from the changed position information stored in step S214 of FIG. is the position indicated by .

When the imaging control unit 504 determines that the current position is the position indicated by the changed position information (step S310, "Yes"), the process proceeds to step S311. In step S301, the imaging control unit 504 sets each setting stored in association with the changed position information for each of the road surface imaging cameras 6L and 6R in step S214 of FIG. Information is set, and the imaging conditions of the road surface imaging cameras 6L and 6R are changed.

In the next step S312, the imaging control unit 504 generates a trigger for instructing imaging, sends it to the road surface imaging cameras 6L and 6R, and according to the trigger, each stereo imaged by the road surface imaging cameras 6L and 6R is captured. Acquire a captured image. The imaging control unit 504 stores each acquired stereoscopic image in, for example, the RAM 5002 and the storage 5004 .

On the other hand, if the imaging control unit 504 determines in step S310 that the current position is not the position indicated by the changed position information, the process proceeds to step S312, generates a trigger for instructing imaging, and performs road surface imaging. Then, according to the trigger, each stereo captured image captured by the road surface imaging cameras 6L and 6R is obtained.

The imaging system 10 according to the first embodiment repeatedly performs each process according to the flowcharts of FIGS.

According to the first modification of the first embodiment, the traveling direction camera 7 captures the brightness information of the stereo imaging ranges of the road surface imaging cameras 6L and 6R prior to imaging by the road surface imaging cameras 6L and 6R. , and the imaging conditions are determined individually for the road surface imaging cameras 6L and 6R. Therefore, the imaging of the stereo imaging range by each of the road surface imaging cameras 6L and 6R can be performed under appropriate imaging conditions. As a result, the crack rate C, the amount of rutting D, and the flatness σ can be calculated with higher accuracy based on the stereo images captured by the road surface imaging cameras 6L and 6R, and the reliability of evaluation by MCI using these can be calculated. It is possible to increase the reliability.

[Second Modification of First Embodiment]
Next, a second modification of the first embodiment will be described. The second modification of the first embodiment includes foreign matter detection processing, and is a more specific version of the processing according to the first modification of the first embodiment.

FIG. 25 is a flowchart of an example of processing related to the traveling direction camera 7 according to the second modification of the first embodiment. In step S<b>400 , the information processing apparatus 50 acquires position information indicating the current position from the position information acquisition unit 51 using the imaging control unit 504 . In the next step S401, the captured image acquisition unit 500b acquires a traveling direction captured image captured by the traveling direction camera 7. FIG. The acquired traveling direction imaging is stored in the memory (for example, the RAM 5002 ) of the information processing device 50 by the captured image storage unit 503 .

In the next step S402, the imaging control unit 504 analyzes the traveling direction captured image stored in step S401, and detects an image showing a foreign object from the traveling direction captured image. Here, foreign matter refers to things other than road surface pavement, such as white lines, road markings, manholes, fallen leaves, flower petals, earth and sand, snow, and garbage. As a foreign matter detection method, for example, white line detection by straight line detection by Hough transform and manhole (round) detection by circle detection by Hough transform can be applied. Further, as a foreign matter detection method, it is also possible to detect other foreign matter based on the learning results of deep learning.

In step S403, the imaging control unit 504 determines whether or not a foreign object is detected as a result of the processing in step S402. If the imaging control unit 504 determines that a foreign object has been detected (“Yes” in step S403), the imaging control unit 504 shifts the process to step S404, and sets areas that do not include an image of the foreign material as evaluation areas 710L and 710R. For example, when it is determined that the evaluation region 710L initially set corresponding to the stereo imaging range of the road surface imaging camera 6L includes an image of a foreign object, the imaging control unit 504 moves the position of the evaluation region 710L forward and backward or left and right. A shift process is performed so that the evaluation area 710L does not include the image of the foreign matter. Not limited to this, the evaluation area 710L may be reduced or deformed.

On the other hand, when the imaging control unit 504 determines that no foreign object is detected in step S403 (step S403, "none"), the process proceeds to step S405. In step S405, the imaging control unit 504 sets the original positions of the evaluation areas 710L and 710R in the traveling direction captured image 71, for example, the central area of the traveling direction captured image 71, to the evaluation areas 710L and 710R.

After the process of step S404 or step S405, the process proceeds to step S410. In step S410, the imaging control unit 504 obtains the luminance of the evaluation regions 710L and 710R based on the pixels included in the evaluation regions 710L and 710R determined in step S404 or S405, and the obtained luminance is a predetermined appropriate value. It is determined whether or not it is luminance. When the imaging processing unit 504 determines that the luminance is appropriate (step S410, “Yes”), the series of processing according to the flowchart of FIG. 25 is terminated.

If the imaging control unit 504 determines in step S410 that the luminance is not appropriate, the process proceeds to step S411. In step S411, the imaging control unit 504 determines whether or not the brightness of the evaluation areas 710L and 710R indicates a brighter value than a predetermined value. If the imaging control unit 504 determines that the brightness indicates a value brighter than the predetermined value, the imaging control unit 504 causes the process to proceed to step S412. In step S412, the imaging control unit 504 changes the imaging condition (exposure parameter) so as to darken the captured image.

On the other hand, if the imaging control unit 504 determines in step S411 that the luminance is darker than the predetermined value, the process proceeds to step S413. In step S413, the imaging control unit 504 changes the imaging condition (exposure parameter) so as to brighten the captured image.

After the process of step S412 or step S413, the imaging control unit 504 shifts the process to step S414. In step S414, the imaging control unit 504 associates the imaging conditions changed in step S412 or S413 with change position information indicating the position for changing the imaging conditions, and stores them in the RAM 5002, for example. When the storage of the setting information and the changed position information in step S414 is completed, a series of processing according to the flowchart of FIG. 25 is terminated.

According to the second modification of the first embodiment, the traveling direction camera 7 detects the luminance information of the stereo imaging range of the road surface imaging cameras 6L and 6R prior to imaging by the road surface imaging cameras 6L and 6R. It acquires and determines imaging conditions. At this time, if images of a foreign object are included in the evaluation areas 710L and 710R in the traveling direction captured image 71 captured by the traveling direction camera 7, the imaging control unit 504 avoids the image of the foreign matter and and 710R are set. Therefore, the imaging of the stereo imaging range by the road surface imaging cameras 6L and 6R can be performed under appropriate imaging conditions without being affected by the image of the foreign matter. As a result, the crack rate C, the amount of rutting D, and the flatness σ can be calculated with higher accuracy based on the stereo images captured by the road surface imaging cameras 6L and 6R. It is possible to improve

[Second embodiment]
Next, a second embodiment will be described. The second embodiment is an example in which an optical sensor for detecting light from an object is used instead of the traveling direction camera 7 described above. As the optical sensor, a photodiode or a phototransistor that simply detects light from an object may be used, or a laser scanner or the like may be used.

FIG. 26 is a flowchart of an example of imaging condition setting processing according to the second embodiment. In the flowchart of FIG. 26, the same reference numerals are assigned to the processes that are common to the flowchart of FIG. 19 described above, and detailed description thereof will be omitted. It should be noted that the output of the optical sensor is obtained by the captured image obtaining unit 500b in the same manner as described above. In step S<b>200 , the information processing apparatus 50 acquires position information indicating the current position from the position information acquisition unit 51 by the imaging control unit 504 . In the next step S221, the captured image acquisition unit 500b acquires the output of the optical sensor. The captured image storage unit 503 stores the acquired output values in the memory (for example, the RAM 5002 ) of the information processing apparatus 50 .

In the next step S222, the imaging control unit 504 analyzes the output value of the optical sensor stored in step S221 and determines whether or not the imaging conditions need to be changed. The imaging control unit 504 calculates the brightness of the detection range (imaging road surface) based on the output value of the optical sensor, and determines whether or not the imaging condition (exposure parameter) needs to be changed according to the calculated brightness. If so, decide what settings to use.

For example, when the change in road surface brightness exceeds a threshold value, the image capturing control unit 504 reduces the brightness so that the captured image becomes darker when the brightness changes in the brighter direction according to the amount of change. The exposure parameter is determined so that the captured image becomes brighter when the direction changes. The exposure parameter is at least one of exposure time and gain.

In step S203, when the imaging control unit 504 determines that the imaging conditions need to be changed by the determination processing in step S222 (step S203, "yes"), the process proceeds to step S204. In step S204, the imaging control unit 504 associates the changed imaging conditions with change position information indicating the position for changing the imaging conditions, and stores them in the RAM 5002, for example. This is because the directions of the detection range of the optical sensor and the stereo imaging range of the road surface imaging cameras 6L and 6R are different, so that the location where the luminance is detected by the optical sensor is captured by the road surface imaging cameras 6L and 6R. This is because there is a time lag and it is necessary to change the exposure at the appropriate timing.

Note that the changed position information is, for example, a value obtained by adding the distance from the position information acquisition unit 5021 to the known detection range of the detection target of the optical sensor to the position information acquired in step S200, in the same manner as described above. can be applied.

When the storage of the setting information and the change position information in step S504 is completed, the series of processing according to the flowchart of FIG. 26 is terminated. If it is determined in step S203 described above that there is no need to change the exposure setting (step S203, "No"), the series of processes according to the flowchart of FIG. 26 is terminated.

The processes of steps S200, S221, S222, S203 and S204 described above are repeatedly executed at predetermined time intervals.

Note that the processing relating to the road surface imaging cameras 6L and 6R is the same as the processing described with reference to FIG.

According to the second embodiment, in order to detect the brightness of the evaluation area in front of the vehicle 1 in the direction of travel, an optical sensor that can be implemented with a simpler configuration than the camera 7 for the direction of travel is used. This allows the imaging system to be configured on a smaller scale.

Although each of the above-described embodiments is a preferred embodiment of the present invention, it is not limited thereto, and various modifications can be made without departing from the scope of the present invention.

1 vehicle 4 road surface 5 PC
6, 6C, 6L, 6R stereo cameras 6L _L , 6L _R , 6R _L , 6R _R imaging lens 7 traveling direction camera 10 imaging system 50 information processing device 51, 5022 position information acquisition unit 60C, 60L, 60R stereo imaging range 60C _L , 60C _R , 60L _L , 60L _R , 60R _L , 60R _R Imaging ranges 100 ₁ , 100 ₂ Imaging units 102, 504 Imaging control unit 103 Generation units 500a, 500b Captured image acquisition unit 503 Captured image storage units 710L, 710R Evaluation Area 5021 Velocity acquisition unit

JP 2013-096740 A

Claims (8)

a first imaging device mounted on a moving body and capturing an image of a road surface on which the moving body moves;
a second imaging device that is mounted on the moving body and captures an image of the road surface on the traveling direction side of the moving body with respect to the first imaging device;
an acquisition unit that acquires current position information indicating the current position of the moving object;
an imaging control unit that determines an exposure parameter as an imaging condition of the first imaging device based on the captured image captured by the second imaging device;
with
The imaging control unit further
A timing for setting the imaging conditions in the first imaging device is determined based on the current position information and a predetermined distance from the moving body of the imaging range of the second imaging device.
imaging system. A plurality of the first imaging devices are mounted on the moving body,
The imaging control unit is
determining the imaging conditions for each of the plurality of first imaging devices based on images of respective regions corresponding to the imaging ranges of the plurality of first imaging devices in the imaging range of the second imaging device; The imaging system of Claim 1 . The imaging control unit is
Claim 1 or Claim 2 , wherein when the imaged image includes an image of a foreign object on the road surface, the imaging condition is determined using an image of an area of the imaged image excluding the image of the foreign object. The imaging system described. An image capturing method for an image capturing system including a first image capturing device and a second image capturing device each mounted on a moving body,
The first imaging device images a road surface on which the moving object moves,
The second imaging device images the road surface on the traveling direction side of the moving body with respect to the first imaging device,
an acquisition step of acquiring current position information indicating the current position of the moving object;
an imaging control step of determining an exposure parameter as an imaging condition of the first imaging device based on the captured image captured by the second imaging device ;
The imaging control step further includes
A timing for setting the imaging conditions in the first imaging device is determined based on the current position information and a predetermined distance from the moving body of the imaging range of the second imaging device.
Imaging method. A moving body equipped with the imaging system according to any one of claims 1 to 3 . An imaging device mounted on a mobile body,
an imaging unit that captures a first imaging range of the road surface on which the moving object moves ;
a location information acquisition unit that acquires current location information indicating the current location of the moving object;
an imaging control unit that determines an exposure parameter as an imaging condition of the imaging unit based on a captured image obtained by imaging a second imaging range of a road surface on the traveling direction side of the moving body, which is different from the first imaging range;
with
The imaging control unit further
determining the timing for setting the imaging conditions in the imaging unit based on the current position information and a predetermined distance from the moving body in the second imaging range;
Imaging device. 7. The imaging apparatus according to claim 6 , further comprising an acquisition section that acquires the captured image obtained by imaging the second imaging range by another imaging section different from the imaging section. A moving body equipped with the imaging device according to claim 6 or 7 .

Images