JP2023123625A - Detection device, detection method, detection program and recording medium - Google Patents

Abstract

To perform calibration corresponding to a change even if a distance between an imaging part and a road face is changed due to a change in the weight of a moving body and a change in the pneumatic pressure of wheels.SOLUTION: A moving amount detection part 321a detects a pixel moving amount on the basis of a captured imaged image which is captured by an imaging part 210 with respect to a stop line being a feature part on a road face having a well-known length along a traveling direction of a moving body MV. A specification part 322A specifies a white line region in the captured image. Then, when it is determined that the moving body MV linearly travels on the flat road face at a constant velocity, the calibration part 323A calibrates a distance on the road face corresponding to one pixel related to a distance between the imaging part 210 and the road face, on the basis of the well-known length along the traveling direction of the moving body MV on the stop line, in addition to the pixel moving amount and a specified result of the white line region in the captured image.SELECTED DRAWING: Figure 4

Description

The present invention relates to a detection device, a detection method, a detection program, and a recording medium on which the detection program is recorded.

The slip ratio λ, which is the normalized difference between the wheel speed of a moving body such as a vehicle and the moving body speed, and the friction coefficient μ, which is the normalized grip force between the wheel and the road surface, are shown in FIGS. relationship (hereinafter referred to as “μ-λ characteristic”). Here, FIG. 1 shows the .mu.-.lambda. characteristic during driving, and FIG. 2 shows the relationship with the .mu.-.lambda. characteristic during braking.

In FIGS. 1 and 2, the μ-λ characteristic on a dry road surface is indicated by a solid line, the μ-λ characteristic on a wet road surface is indicated by a dashed line, and the μ-λ characteristic on a frozen road surface is indicated by two points. indicated by dashed lines.

In the change in the friction coefficient μ accompanying the increase in the slip ratio λ during driving shown in FIG. 1, the state where the friction coefficient μ is smaller than the maximum slip ratio is the stable region in which the moving body can travel stably. there is On the other hand, when the coefficient of friction μ is greater than the maximum slip ratio, the gripping force is reduced, and in the worst case, an unstable region occurs, in which slipping or locking occurs.

In the change in the friction coefficient μ accompanying the increase in the slip ratio λ during braking shown in FIG. 2, the state where the friction coefficient μ is greater than the minimum slip ratio is the stable region. On the other hand, when the friction coefficient μ is smaller than the minimum slip ratio, the unstable region is established.

By controlling the slip ratio λ within a range below the absolute value of the slip ratio at which the absolute value of the friction coefficient μ becomes the maximum value, the moving body can maintain stable running. On the other hand, if the unstable state continues and the tires spin or lock, it becomes impossible to control the driving, braking, and steering of the vehicle.

Therefore, in order to avoid the danger of accidents, an ABS (Antilock Brake System) or the like, which mainly uses brake oil pressure control and engine control, is adopted in internal combustion engine automobiles. In such an ABS or the like, a slip state is determined, and the driving torque of the engine and the braking torque of the brake hydraulic pressure are controlled so as to approach a stable region. Also, for electric vehicles, an anti-slip control has been proposed in which the slip rate λ is estimated and the torque of the motor is appropriately controlled to maintain it within a stable region. Thus, detecting the slip ratio λ is very important in grasping the running state.

Now, since the slip ratio λ is calculated by the following equation (1), the wheel radius r, the rotational angular velocity ω, and the moving object velocity v are required.
λ=(r·ω−v)/Max(r·ω, v) (1)

Here, Max(r·ω, v) indicates the larger value of (r·ω) and v. Since (r·ω) is greater than v during driving, Max(r·ω, v)=r·ω. On the other hand, during braking, v is greater than (r·ω), so Max(r·ω, v)=v.

For example, when the moving object is an automobile, the wheel radius r can be considered constant if the air pressure in the wheels is sufficient. Also, the rotational angular velocity ω can be detected from the pulse output of an encoder mounted on a wheel, or from the signal output of a resolver connected to a motor in the case of an electric vehicle.

On the other hand, the following (a) to (c) are generally cited as methods for detecting the moving body speed v.
(a) Calculated from the rotational speed of the non-driven wheels (b) Calculated by integrating the acceleration value detected by the acceleration sensor (c) Calculated based on the detection result of the optical sensor

Here, in the method (a), the brakes are applied to all the wheels, so the speed v of the moving body during braking cannot be detected. Also, in the case of four-wheel drive, since there are no non-driving wheels, the moving body speed v cannot be detected.

In addition, since the method (b) integrates the output of the acceleration sensor, the offset existing in the output of the acceleration sensor is accumulated. As a result, the moving body speed v cannot be detected accurately.

The method (c) is expected, which in principle does not have the drawbacks of the methods (a) and (b). As a technique employing the method (c), a technique described in Patent Document 1 has been proposed (hereinafter referred to as "conventional example"). In this prior art technique, the running state or stopped state of the vehicle is determined from an image signal obtained by picking up a scene in the vicinity of the own vehicle. Also, the body speed of the own vehicle (that is, moving body speed) is detected based on the dimensions of the road markings in the image and the dimensions of the road markings on the road. In the conventional technology, the vehicle speed is detected only when the road marking is present in the image, and only the running/stop judgment is performed when the road marking is not in the image.

JP 2009-205642 A

The above-described conventional technique does not detect the vehicle body speed when there is no road marking in the captured image. As a result, even if the vehicle speed changes while the road marking does not exist in the captured image, the vehicle speed detected at the time when the road marking was present in the captured image is maintained as it is. Velocity will be estimated. For this reason, it is difficult to say that the vehicle speed at each point in time can be accurately detected.

Further, in the conventional technology, since the image pickup device is mounted on the vehicle body, the distance between the image pickup device and the road surface varies depending on the number of passengers, the amount of luggage, and the air pressure of the wheels. As a result, the imaging magnification changes, so the size of the road marking in the captured image changes.

For example, as shown in FIG. 3A, an imaging unit 210 includes an imaging lens system 211 (focal length: f) and a square imaging surface 212 (side length: D). Let “d” be the distance from the lens system 211 to the imaging surface 212 . In this case, when the road surface LD directly below is imaged, the distance from the imaging lens system 211 to the road surface LD (hereinafter also referred to as “the distance from the imaging unit 210 to the road surface LD”) is “h”. The optical magnification m of imaging by is represented by the following equation (2).
m=1/((h/f)−1) (2)
The pixel configuration on the imaging surface 212 is NU×NU (for example, 30×30).

As a result, the length H of one side of the square area to be imaged on the road surface LD is given by the following equation (3).
H=D/m=D·((h/f)−1) (3)

For example, if f=8 [mm], D=1.7 [mm], and h=495 [mm], then H=103.5 [mm].

Now, as shown in FIG. 3B, when the vehicle weight increases from the state of FIG ^. It changes from "m" in the state of FIG. 3A to "m ^* " represented by the following equation (4).
m ^* =1/((h ^* /f)-1) (4)

As a result, the length of one side of the square area to be imaged changes from "H" in the state of FIG. 3A to "H ^* " expressed by the following equation (5).
H ^* =D/m ^* =D.((h ^* /f)-1) (5)

For example, when h ^* =445 [mm], H ^* =92.9 [mm], which is about 10% change from H=103.5 [mm] in the state of FIG. 3A. As a result, the size of the area on the road surface corresponding to the size of one pixel also changes by about 10% from the state shown in FIG. 3A.

By the way, when the movement distance of the image is obtained by the correlation between the captured images for each unit time, and the vehicle body speed in the longitudinal direction and the lateral direction is calculated, the area on the road surface corresponding to the size of 1 pixel in the captured image When the magnitude of changes, the calculated vehicle body speed also changes. That is, if the size of the area on the road surface corresponding to the size of one pixel changes by about 10%, the calculated speed value will also change by about 10%, resulting in an error.

Dynamic errors due to road surface fluctuations can be less affected by errors through averaging. However, since changes in vehicle weight and changes in wheel air pressure are offset, the averaging process cannot suppress the effects of errors.

Therefore, there is a need for a technology capable of accurately detecting the vehicle body speed even when the vehicle weight changes or the air pressure of the wheels changes. Meeting such demands is one of the problems to be solved by the present invention.

The present invention has been made in view of the above, and can perform calibration according to the change even if the distance between the road surface and the imaging unit changes due to changes in vehicle weight or wheel air pressure. An object of the present invention is to provide a detection device and a detection method.

The invention according to claim 1 acquires a plurality of images including characteristic portions on the road surface having a known length, which are captured along the traveling direction at predetermined time intervals by an imaging unit mounted on a moving object. and; calculating the number of image regions of a predetermined width corresponding to the known length based on the plurality of images acquired by the acquisition unit, and based on the calculated number and the known length and an output unit for outputting distance information related to the distance on the road surface corresponding to the predetermined width.
The invention according to claim 4 is an acquisition unit that acquires a plurality of images including characteristic parts on the road surface that are captured along a traveling direction at predetermined time intervals by an imaging unit mounted on a moving body; Information indicating the distance on the road surface corresponding to the width of a predetermined image area in the images obtained based on the plurality of acquired images is calibrated based on the distance information output in the past, and the calibration result is newly updated. A detection device comprising: a first output unit for outputting as distance information; and a second output unit for outputting the speed of the moving body using the new distance information.

The invention according to claim 5 is a detection method used in a detection device comprising an acquisition unit and an output unit, wherein the acquisition unit is separated by a predetermined time from an imaging unit mounted on a moving object and is detected in the traveling direction. an acquiring step of acquiring a plurality of images including a road surface feature having a known length, imaged along a road surface; calculating the number of image regions of a predetermined width corresponding to the length of the road, and outputting distance information related to the distance on the road surface corresponding to the predetermined width based on the calculated number and the known length a distance information output step; and;

According to a sixth aspect of the present invention, there is provided a detection program that causes a computer of a detection device to execute the detection method according to the fifth aspect.

The invention according to claim 7 is a recording medium characterized in that the detection program according to claim 6 is recorded so as to be readable by a computer of the detection device.

FIG. 5 is a diagram showing the relationship between the slip ratio and the coefficient of friction during driving; FIG. 5 is a diagram showing the relationship between slip ratio and friction coefficient during braking; FIG. 4 is a diagram for explaining changes in speed calculation values due to changes in vehicle weight and the like; It is a figure showing composition of a detection device concerning a 1st embodiment of the present invention. FIG. 5 is a diagram for explaining the operation of the movement amount detection unit in FIG. 4; FIG. 4 is a diagram for explaining a calibration environment in the first embodiment; FIG. 5A and 5B are diagrams for explaining the operation of the specifying unit in FIG. 4; FIG. 5A and 5B are diagrams for explaining a calibration operation by the calibration unit of FIG. 4; FIG. It is a figure which shows the structure of the detection apparatus which concerns on 2nd Embodiment of this invention. FIG. 10 is a diagram for explaining the positional relationship between the two imaging units in FIG. 9; FIG. FIG. 12 is a diagram for explaining a calibration environment in the second embodiment; FIG. FIG. 10 is a diagram for explaining the operation of the specifying unit in FIG. 9; FIG. 10A and 10B are diagrams for explaining a calibration operation by the calibration unit of FIG. 9; FIG. It is a figure which shows the structure of the detection apparatus based on 1st Example of this invention. FIG. 15 is a flowchart for explaining a pixel movement amount detection process executed by the control unit of FIG. 14; FIG. FIG. 15 is a flowchart for explaining vehicle body speed output processing executed by the control unit of FIG. 14; FIG. FIG. 15 is a flowchart for explaining a white line area identification process executed by the control unit of FIG. 14; FIG. FIG. 15 is a flowchart for explaining calibration condition monitoring processing executed by the control unit of FIG. 14; FIG. FIG. 15 is a flowchart for explaining a pixel distance calibration process performed by the control unit of FIG. 14; FIG. FIG. 4 is a diagram showing the configuration of a detection device according to a second embodiment of the present invention; FIG. 21 is a flowchart for explaining a pixel movement amount detection process executed by the control unit of FIG. 20; FIG. FIG. 21 is a flow chart for explaining white line area identification processing executed by the control unit of FIG. 20; FIG. 21 is a flowchart for explaining a pixel distance calibration process performed by the control unit of FIG. 20;

Embodiments of the present invention will now be described with reference to FIGS. 4 to 13. FIG. In the following description and drawings, the same or equivalent elements are denoted by the same reference numerals, and overlapping descriptions are omitted.

[First embodiment]
First, a first embodiment of the invention will be described with reference to FIGS. 4 to 8. FIG.

<Configuration>
FIG. 4 shows a block diagram of the configuration of the detection device 300A according to the first embodiment. As shown in FIG. 4, the detection device 300A is connected to an imaging unit 210, a navigation device 220 and an ECU (Electronic Control Unit) 230. As shown in FIG. The detection device 300A, the imaging unit 210, the navigation device 220 and the ECU 230 are mounted on the mobile object MV.

The imaging unit 210 described above is mounted at a fixed position on the moving body MV, and as described above with reference to FIG. 3, images the road surface immediately below the fixed position. The image capturing unit 210 periodically captures images of the road surface at a cycle time TP. Data of the image captured in this manner (hereinafter referred to as “captured image data”) is sent to the detection device 300A.

It should be noted that the "periodic time TP" is such that even if the moving object MV is traveling at high speed, the two images captured with the periodal time TP separated from each other include a common road surface area. It is determined in advance based on experiments, simulations, or the like.

The navigation device 220 described above assists the user in driving the mobile object MV based on the map information and the current position information. This navigation device 220 notifies the detection device 300A when the mobile object MV is within a predetermined distance from an intersection in the traveling direction.

It should be noted that the "predetermined distance" is determined by experiments, experiments, It is determined in advance based on simulation, experience, or the like.

The ECU 230 described above controls the running of the mobile object MV and provides running information to the user based on sensor detection information acquired from various sensors such as a wheel rotation speed sensor, an acceleration sensor, a steering angle sensor, and an inclination sensor. provide. The ECU 230 sends acceleration information, steering angle information and tilt information to the detection device 300A.

The ECU 230 further acquires the moving body speed (running speed information) sent from the detection device 300A, and utilizes the acquired moving body speed for running control of the moving body MV.

The detection device 300A, as shown in FIG. 4, includes an acquisition section 310A and a first output section 320A. The detection device 300A also includes a second output section 330 .

The acquisition unit 310A described above receives captured image data sent from the imaging unit 210 . Then, the acquisition section 310A sends the captured image data to the first output section 320A.

320 A of said 1st output parts receive the captured image data sent from 310 A of acquisition parts. Then, the first output unit 320A detects the number of pixel movement amounts (hereinafter referred to as "pixel movement amounts") at the same road surface position between the two images captured with the period time TP between, and If the calibration condition is satisfied, the distance on the road corresponding to one pixel (hereinafter referred to as "pixel distance") is calibrated. The detected pixel movement and calibrated pixel distance are output to the second output 330 . The details of the configuration of the first output section 320A having such functions will be described later.

Each time the second output unit 330 receives the pixel movement amount sent from the first output unit 320A, based on the latest calibrated pixel distance and period time TP, the moving object MV as traveling information Calculate speed. Second output unit 330 then outputs the calculated moving body speed to ECU 230 .

<<Configuration of the first output unit 320A>>
Next, the configuration of the above-described first output section 320A will be described.

320 A of 1st output parts are provided with 321 A of movement amount detection parts, and 322 A of specific|specification parts, as FIG. 4 shows. The first output section 320A also includes a calibration section 323A.

321 A of said movement amount|distance detection parts receive the captured image data sent from 310 A of acquisition parts. Then, the movement amount detection unit 321A detects the pixel movement amount by a so-called displacement amount search method based on the captured image data. The pixel displacement thus detected is sent to the calibration section 323A and the second output section 330. FIG.

Details of the operation of the movement amount detection unit 321A will be described later.

322 A of said specific|specification parts receive the captured image data sent from 310 A of acquisition parts. Then, the identifying unit 322A identifies the white line area in the image obtained from the captured image data. Information on the white line area specified in this way (hereinafter referred to as "white line area information") is sent to the calibration unit 323A.

Details of the operation of the identification unit 322A will be described later.

The calibration unit 323A described above receives the pixel movement amount sent from the movement amount detection unit 321A and the white line area information sent from the identification unit 322A. Then, if the calibration unit 323A can determine that the moving object MV is traveling straight ahead on a flat road surface at a constant speed based on the acceleration information, the steering angle information, and the tilt information sent from the ECU 230, the calibration unit 323A In addition to the white line area information, the pixel distance is calibrated based on the fact that the length of the stop line drawn on the road in the traveling direction is approximately 45 [cm]. The pixel distances thus calibrated are sent to the second output 330 .

Details of the operation of the calibration unit 323A will be described later.

<Action>
Next, the operation of the detection device 300A calibrated as described above will be described, focusing on the processing performed by each element of the detection device 300A.

It is assumed that the imaging unit 210 has already started its operation and is sequentially sending the captured image data of the road surface image captured at the cycle time TP to the detection device 300A. It is also assumed that the navigation device 220 has already started operation, and when the mobile object MV is within a predetermined distance from an intersection in the traveling direction, it sends a notification to the detection device 300A. It is also assumed that the ECU 230 has already started operating and has sent acceleration information, steering angle information, and tilt information to the detection device 300A (see FIG. 4).

Furthermore, in the detection device 300A, the pixel distance has already been calibrated a plurality of times, and the calibrating unit 323A holds a calibration history including the calibration times for the latest predetermined number of times and the provisional pixel distance described later. It is assumed that there is It is also assumed that the second output unit 330 holds the latest calibrated pixel distance. Here, the average pixel distance is held in the second output section 330 until the first calibration of the pixel distance is performed.

Note that the "predetermined number of times" is based on experiments, simulations, experience, etc., from the viewpoint of performing averaging processing to suppress the influence of changes in the length of the stop line in the direction of travel, etc., on the calibration results of the pixel distance. pre-determined based on Also, the "average pixel distance" is determined in advance for the mobile object MV based on experiments, simulations, experience, and the like.

<<Acquisition processing of image data>>
In the detection device 300A, the acquisition unit 310A receives the captured image data sent from the imaging unit 210. FIG. Then, the acquisition unit 310A sends the captured image data to the movement amount detection unit 321A and the identification unit 322A of the first output unit 320A (see FIG. 4).

<<Processing to detect the amount of pixel movement>>
Next, detection processing of the pixel movement amount by the movement amount detection unit 321A will be described.

Upon receiving the captured image data sent from the acquisition unit 310A, the movement amount detection unit 321A detects the position of the common characteristic region in the current image obtained from the current captured image data and the previous image obtained from the previous captured image data. is detected as the pixel movement amount. Then, the movement amount detection section 321A sends the detected pixel movement amount to the calibration section 323A and the second output section 330 (see FIG. 4).

Note that when there are a plurality of common characteristic regions, the movement amount detection unit 321A adopts the average value of the respective displacement amounts of the plurality of characteristic areas as the pixel movement amount.

FIG. 5 shows an example of movement amount detection by the movement amount detection unit 321A. In the example of FIG. 5, the moving object MV is traveling straight along the X direction, the X direction position at time T _j is X _j , and the X direction position at time T _j+1 (=T _j +TP) is This is an example in the case of X _j+1 . Further, the example of FIG. 5 shows an example in which there are two types of characteristic regions, characteristic region A and characteristic region B, that are common to the captured image at time T _j and the captured image at time T _j+1 . In the example of FIG. 5, the amount of displacement of characteristic region A is "ΔX _jA ", and the amount of displacement of characteristic area B is "ΔX _{jB "} . Calculated.
ΔX _j =(ΔX _jA +ΔX _jB )/2 (6)

<<Output processing of moving body velocity v>>
Next, the process of outputting the moving object velocity v by the second output unit 330 will be described.

Upon receiving the pixel displacement sent from the displacement detection unit 321A, the second output unit 330 generates a to calculate the moving body velocity v. Then, the second output unit 330 outputs the calculated moving body speed v to the ECU 230 .

Here, the second output unit 330 calculates the moving object velocity v by the following equation (7), where "PN" is the pixel movement amount and "PD" is the pixel distance.
v=PN.PD/TP (7)

<Processing to specify the white line area>
Next, the identification processing of the white line area by the identification unit 322A will be described.

Such white line region identification processing is performed for pixel distance calibration by the calibration unit 323A, which will be described later. The environment for calibrating such pixel distances (hereinafter referred to as "calibration environment") is shown in FIG. As shown in FIG. 6, the pixel distance is calibrated when the moving body MV is running at a constant speed straight on a flat road surface and is a white line on the road surface with a length W (≈45 [cm]) in the traveling direction. It is performed when crossing the stop line SPL at a right angle to the longitudinal direction of the stop line SPL.

In the following description, it is assumed that the stop line SPL used to calibrate the new pixel distance is a white line area from the running direction position (that is, the X direction position) XR to the running direction position XP. .

Now, upon receiving the captured image data sent from the acquisition unit 310A, the specifying unit 322A specifies the white line region in the image obtained from the captured image data based on the brightness of each pixel in the image, and obtains the specified result. , to the calibration unit 323A as white line area information (see FIG. 4). The identification result of the white line area includes two types of white line, the front partial white line shown in FIG. 7(A) and the rear partial white line shown in FIG. 7(B).

If the identification result is a partial white line in the front, the identification unit 322A identifies the number of pixels a in the length of the white line region in the image in the X direction (hereinafter also referred to as "front length a"). Further, when the identification result is the rear partial white line, the identification unit 322A identifies the number of pixels b in the X-direction length of the non-white line region in the image (hereinafter also referred to as “front length b”).

Then, in the case of a partial front white line, the specifying unit 322A sends the white line area information [white line flag: ON, front length: a] shown in FIG. 7A to the calibration unit 323A (see FIG. 4). In addition, in the case of a partial rear white line, the identification unit 322A sends the white line area information [white line flag: ON, front length: b] shown in FIG. 7B to the calibration unit 323A (see FIG. 4).

<<Calibration process of pixel distance>>
Next, calibration processing by the calibration section 323A will be described.

In the calibration process, the calibration unit 323A determines whether or not it has received from the navigation device 220 that the moving object MV is within a predetermined distance from the intersection in the traveling direction. If the result of this intersection determination is affirmative, the calibration unit 323A can determine that the moving object MV is traveling straight ahead at a constant speed on a flat road surface, based on the acceleration information, steering angle information, and tilt information sent from the ECU 230. It is determined whether or not the vehicle is traveling straight at a constant speed.

When the result of the constant speed straight running determination is affirmative, the calibration unit 323A determines whether or not the white line area information sent from the specifying unit 322A is "partial front white line". If the result of this white line start determination is affirmative, the calibration unit 323A determines that the white line area information sent from the identification unit 322A is " The pixel movement amounts sent from the movement amount detection unit 321A are collected until the result of the white line end determination of whether or not the "rear part of the white line" is affirmative.

Then, when the result of the white line end determination is affirmative, the calibration unit 323A calculates the front length a included in the white line area information at the time when the "front partial white line" was obtained, the collected pixel movement amount ΔX ₁ , . Based on ΔX _M and the front length b included in the white line area information at the time when the "rear partial white line" is reached, the provisional pixel distance PT [cm] is calculated by the following equation (8).
PT=45/(a+ _ΔX1 +...+ _ΔXM -b)...(8)

Next, the calibration unit 323A calculates a weighted average of the current provisional pixel distance calculated and the provisional pixel distances calculated during the predetermined number of past calibrations held therein, and calculates the weighted average Compute the pixel distance. After that, the calibration unit 323A internally stores the current temporary pixel distance instead of the oldest one among the temporary pixel distances that have been internally stored.

In weighted averaging, the longer the elapsed time up to the present, the lower the weight.

Calibrator 323 A sends the calculated pixel distances to second output 330 . As a result, the second output unit 330 calculates the moving body velocity v using the new pixel distance.

Note that FIG. 8 shows an example in which a captured image of a front partial white line is obtained at time _T1 and a captured image of a rear partial white line is obtained at time _T5 . In the example of FIG. 8, the provisional pixel distance PT [cm] is calculated by the following equation (9).
PT=45/(a+ _ΔX1 + _ΔX2 + _ΔX3 + _ΔX4 -b) (9)

As described above, in the first embodiment, based on the image captured by the imaging unit 210 regarding the stop line, which is a characteristic portion on the road surface having a known length along the running direction of the moving body MV, the movement amount detection unit 321A detects the amount of pixel movement. Further, the identifying unit 322A identifies the white line area in the captured image. Then, the calibration unit 323A determines whether the mobile object MV is flat based on the known length of the stop line along the traveling direction of the mobile object MV, in addition to the pixel movement amount and the identification result of the white line region in the captured image. When it can be determined that the vehicle is traveling straight on the road at a constant speed, the distance on the road corresponding to one pixel, which is related to the distance between the imaging unit 210 and the road, is calibrated.

Therefore, according to the first embodiment, even if the distance between the road surface and the imaging unit 210 changes due to a change in the weight of the moving object or a change in the air pressure of the wheels, the distance on the road surface corresponding to one pixel can be adjusted according to the change. can be calibrated accordingly.

Also, in the first embodiment, the second output unit 330 calculates the velocity of the moving object based on the calibration result and the pixel movement amount. Therefore, it is possible to output the moving body speed with high accuracy as the traveling information.

In addition, in the first embodiment, a stop line is adopted as a feature on the road surface. Therefore, by checking the brightness of each pixel in the captured image, it is possible to easily identify the stop line area (that is, the white line area).

Further, in the first embodiment, information indicating that the vehicle is approaching an intersection is acquired, and it is presumed that the vehicle will pass through the stop line. Therefore, efficient calibration can be performed.

Further, in the first embodiment, a final calibration result is obtained by calculating a weighted average of the current temporary pixel distance and the temporary pixel distances calculated during a predetermined number of past calibrations.
This is because the estimated length may deviate from 45 [cm] due to missing stop lines or protruding paint. For this reason, it is possible to perform calibration that suppresses the influence of changes in the length of the stop line in the direction of travel, etc., on the calibration result of the pixel distance.

Further, in the first embodiment, calibration is performed when the vehicle is traveling straight ahead, traveling at a constant speed, and the road surface is not inclined. This is because when there is lateral movement, the passage of the stop line becomes oblique, and calibration is performed based on the length of 45 [cm] or more. In some cases, the expansion and contraction of the suspension or the like occurs, and this is due to the fact that the optical magnification is changing during imaging. Therefore, accurate pixel distance calibration can be performed.

[Second embodiment]
Next, a second embodiment of the present invention will be described mainly with reference to FIGS. 9 to 13. FIG.

<Configuration>
FIG. 9 shows a block diagram of the configuration of a detection device 300B according to the second embodiment. As shown in FIG. 9, the detection device 300B is connected to the imaging units 210 _F and 210 _R and the ECU 230 . The detection device 300B, the imaging units 210 _F and 210 _R , and the ECU 230 are mounted on the mobile object MV.

Each of the imaging units 210 _F and 210 _R described above is configured in the same manner as the imaging unit 210 described above. The imaging unit 210 _F is arranged on the front side of the mobile object MV, and the imaging unit 210 _R is arranged on the rear side of the mobile object MV at a distance D from the imaging unit 210 _F (see FIG. 10). Note that the imaging units 210 _F and 210 _R perform imaging at the same timing.

The data of the image captured by the imaging unit _210F (hereinafter referred to as "front side captured image") is sent to the detection device 300B. Data of an image captured by the imaging unit 210 _R (hereinafter referred to as a “rear side captured image”) is also sent to the detection device 300B.

Acceleration information, steering angle information, and tilt information are sent from the ECU 230 to the detection device 300B in the same manner as in the first embodiment described above. In addition, the ECU 230 further acquires the moving body speed sent from the detection device 300B, and utilizes the acquired moving body speed for running control of the moving body MV.

As shown in FIG. 9, the detection device 300B described above includes an acquisition section 310B and a first output section 320B. The detection device 300B also includes a second output section 330 .

The acquisition unit 310B described above receives the data of the front-side captured image sent from the imaging unit _210F and the data of the rear-side captured image sent from the imaging unit _210R . Then, the acquisition unit 310B sends the data of the front-side captured image and the data of the rear-side captured image to the first output unit 320B.

The first output unit 320B described above receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. Then, the first output unit 320B detects the pixel movement amount and calibrates the pixel distance when the same calibration conditions as in the first embodiment described above are satisfied. The detected pixel displacement and calibrated pixel distance are output to the second output 330 . The details of the configuration of the first output section 320B having such functions will be described later.

It should be noted that the second output section 330 described above outputs the latest calibrated pixel distance and cycle time, as in the first embodiment, each time it receives the pixel shift amount sent from the first output section 320B. Based on the TP, the moving body speed is calculated as the traveling information of the moving body MV. Then, the second output unit 330 outputs the calculated moving body speed of the moving body MV to the ECU 230 .

<<Configuration of First Output Unit 320B>>
Next, the configuration of the first output section 320B described above will be described.

The first output section 320B, as shown in FIG. 9, includes a movement amount detection section 321B and a specification section 322B. The first output section 320B also includes a calibration section 323B.

The movement amount detection unit 321B described above receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. Then, the movement amount detection unit 321B detects the pixel movement amount by a so-called displacement amount search method based on the data of the front side captured image and the data of the rear side captured image. The pixel shift amount thus detected is sent to the calibration section 323B and the second output section 330. FIG.

Details of the operation of the movement amount detection unit 321B will be described later.

The specifying unit 322B described above receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. Then, the specifying unit 322B specifies the white line region in the image obtained from the data of the front side captured image and the data of the rear side captured image. Information on the white line area specified in this way (hereinafter referred to as "white line area information") is sent to the calibration unit 323B.

Details of the operation of the identification unit 322B will be described later.

The calibration unit 323B described above receives the pixel movement amount sent from the movement amount detection unit 321B and the white line area information sent from the identification unit 322B. Then, if the calibration unit 323B can determine that the moving object MV is traveling straight ahead at a constant speed on a flat road surface based on the acceleration information, the steering angle information, and the tilt information sent from the ECU 230, the calibration unit 323B In addition to the white line area information, the pixel distance is calibrated based on the distance D between the imaging units 210 _F and 210 _R described above. The pixel distances thus calibrated are sent to the second output 330 .

Details of the operation of the calibration unit 323B will be described later.

<Action>
Next, the operation of the detection device 300B calibrated as described above will be described, focusing on the processing performed by each element of the detection device 300B.

Note that the imaging units 210 _F and 210 _R have already started operating, and sequentially send the data of the front side captured image and the data of the rear side captured image of the road surface image captured at the cycle time TP to the detection device 300B. It is assumed that there is It is also assumed that the ECU 230 has already started operating and has sent acceleration information, steering angle information, and tilt information to the detection device 300B (see FIG. 9).

Furthermore, in the detection device 300B, the pixel distance has already been calibrated a plurality of times, and the calibrating unit 323B holds a calibration history including the calibration times for a predetermined number of times and a provisional pixel distance described later. and the second output 330 holds the latest calibrated pixel distance. Here, until the pixel distance is calibrated for the first time, the average pixel distance is held in the second output unit 330 as in the first embodiment described above. ing.

<<Acquisition processing of image data>>
In the detection device 300B, the acquisition unit 310B receives the data of the front side captured image and the data of the rear side captured image sent from the imaging units _210F and _210R . Then, the acquisition unit 310B sends the data of the front-side captured image and the data of the rear-side captured image to the movement amount detection unit 321B and the identification unit 322B of the first output unit 320B (see FIG. 9).

<<Processing to detect the amount of pixel movement>>
Next, detection processing of the pixel movement amount by the movement amount detection unit 321B will be described.

When receiving the data of the front side captured image sent from the acquisition unit 310B, the movement amount detection unit 321B obtains from the data of the current front side captured image in the same manner as the detection of the pixel movement amount in the first embodiment described above. The amount of displacement of the position of the common characteristic region in the previous front side captured image obtained from the data of the current front side captured image and the previous front side captured image is detected as the front side pixel shift amount. Further, when receiving the data of the rear-side captured image sent from the acquisition unit 310B, the movement amount detection unit 321B detects the pixel movement amount from the data of the current rear-side captured image in the same manner as in the detection of the pixel movement amount in the first embodiment. The amount of displacement of the position of the common characteristic region in the currently obtained rear side captured image and the previous rear side captured image obtained from the data of the previous rear side captured image is detected as the rear side pixel shift amount.

Note that when there are a plurality of characteristic regions common to the current front-side captured image and the previous front-side captured image, the movement amount detection unit 321B calculates the average value of the displacement amounts of the plurality of characteristic areas as the front-side pixel movement amount. adopt. In addition, when there are a plurality of characteristic regions common to the current rear-side captured image and the previous rear-side captured image, the movement amount detection unit 321B calculates the average value of the displacement amount of each of the plurality of characteristic regions as a rear-side pixel. Adopted as the amount of movement.

Subsequently, the movement amount detection unit 321B calculates the average of the front side pixel movement amount and the rear side pixel movement amount as the pixel movement amount. Then, the movement amount detection section 321B sends the calculated pixel movement amount to the calibration section 323B and the second output section 330 (see FIG. 9).

<<Output processing of moving body speed v>>
Next, the process of outputting the moving object velocity v by the second output unit 330 will be described.

Upon receiving the pixel movement amount sent from the movement amount detection unit 321B, the second output unit 330 outputs the pixel movement amount, the retained pixel distance (that is, the latest calibrated The moving body velocity v is calculated based on the calculated pixel distance) and the period time TP. Then, the second output unit 330 outputs the calculated moving body speed v to the ECU 230 .

<Processing to specify the white line area>
Next, the process of specifying the white line area by the specifying unit 322B will be described.

Such white line area identification processing is performed for pixel distance calibration by the calibration unit 323B, which will be described later. Such a pixel distance calibration environment is shown in FIG. As shown in FIG. 11, pixel distance calibration is performed when the moving body MV crosses a white line area LCP, such as a pedestrian crossing sign, on a flat road surface at a constant speed straight line.

In the following description, it is assumed that the white line area LCP used for calibrating the new pixel distance is the white line area from the running direction position (that is, the X direction position) XR to the running direction position XP. .

Upon receiving the data of the front-side captured image and the data of the rear-side captured image sent from the acquisition unit 310B, the specifying unit 322B determines the white line region in the front-side captured image and the rear-side captured image by the brightness of each pixel in the image. Then, the identification result is sent to the calibrating section 323B as white line area information (see FIG. 9). The identification result of the white line area includes two types of "first rear partial white line" shown in FIG. 12(A) and "second rear partial white line" shown in FIG. 12(B). . Here, the "first rear partial white line" is the state of the rear partial white line in the front-side captured image obtained by the imaging unit _210F installed on the front side. Further, the “second rear partial white line” is the state of the rear partial white line in the rear side captured image obtained by imaging with the imaging unit 210 _R installed on the rear side.

Note that, in the second embodiment, the identification unit 322B continues to clearly identify the white line until it becomes the "first rear partial white line" when passing the white line in the front captured image. , to generate white line area information. Therefore, when the white line cannot be clearly specified due to tire marks or the like, the specifying unit 322B does not generate the white line area information. That is, in the second embodiment, when the white line cannot be clearly specified due to tire marks or the like, the pixel distance is not calibrated.

In the second embodiment, the white line region is specified by focusing on the first rear partial white line in the front side captured image and the second rear partial white line in the rear side captured image. This is also due to the fact that errors in measurement are likely to occur if there are multiple white lines on the line. By taking such measures, it is possible to avoid wasteful calibration, thereby eliminating wastefulness.

Note that when the identification result is the first rear partial white line in the front side captured image, the identification unit 322B determines the length a in the X direction of the non-white line region in the image (hereinafter also referred to as “front length a”). Identify. Further, when the identification result is the second rear partial white line in the rear side captured image, the identification unit 322B determines the length b in the X direction of the non-white line region in the image (hereinafter also referred to as “front length b”). identify.

Then, in the case of the first rear partial white line in the front side captured image, the specifying unit 322B sets the white line area information [first rear partial white line flag: ON, front length: a] shown in FIG. Send to calibration unit 323B (see FIG. 9). In the case of the second rear partial white line, the specifying unit 322B transmits the white line area information [second rear partial white line flag: ON, front length: b] shown in FIG. 12B to the calibration unit 323B. send (see Figure 9).

<<Calibration process of pixel distance>>
Next, the calibration processing by the calibration section 323B will be described.

In the calibration process, the calibration unit 323B determines whether or not the moving object MV is traveling straight ahead at a constant speed on a flat road surface based on the acceleration information, the steering angle information, and the inclination information sent from the ECU 230. Fast straight ahead judgment is performed.

If the result of the constant-velocity straight running determination is affirmative, the calibration unit 323B determines whether or not the white line area information in the front side captured image sent from the specifying unit 322B is the "first rear partial white line". Judge the end of the white line. When the result of the front side white line end determination is affirmative, the calibrating unit 323B is sent from the specifying unit 322B on condition that the result of the constant speed straight driving determination on a flat road surface is maintained affirmative. The pixel movement amount sent from the movement amount detection unit 321B is continued until the result of the rear-side white line end determination indicating whether or not the white line area information in the rear-side captured image is the "second rear partial white line" becomes affirmative. Collect ΔX ₁ , . . . , ΔX _N (see FIG. 13).

Then, when the result of the rear side white line end determination is affirmative, the calibration unit 323B calculates the front length a , the collected pixel _movement amounts ΔX ₁ , . , based on the distance D between the imaging unit 210 _F and the imaging unit 210 _R , the provisional pixel distance PT [cm] is calculated by the following equation (10).
PT=D/(a+ _ΔX1 +...+ _ΔXN -b)...(10)

Next, the calibrating unit 323B calculates a weighted average of the current temporary pixel distance calculated and the temporary pixel distances calculated during a predetermined number of past calibrations held internally, and calculates the current calibrated Compute the pixel distance. After that, the calibrating unit 323B internally stores the current temporary pixel distance instead of the oldest one among the temporary pixel distances that have been internally stored.

In the weighted averaging, as in the case of the first embodiment, the longer the elapsed time up to the present, the smaller the weight.

Calibrator 323 B sends the calculated pixel distances to second output 330 . As a result, the second output unit 330 calculates the moving body velocity v using the new pixel distance.

As described above, in the second embodiment, the image pickup units 210 _F and 210 _R arranged on the moving body MV at a known distance D along the running direction of the moving body MV capture images of the white line, which is a feature on the road surface. Based on the image, the movement amount detection unit 321B detects the pixel movement amount. Further, the identifying unit 322B identifies the white line area in the captured image. Then, the calibration unit 323B can determine that the mobile object MV is traveling straight ahead at a constant speed on a flat road surface based on the known distance D in addition to the pixel movement amount and the result of specifying the white line area in the captured image. In this case, calibrate the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging units 210 _F and 210 _R and the road surface.

Therefore, according to the second embodiment, regardless of whether the length of the white line, which is a feature on the road surface, along the traveling direction of the mobile body is known or unknown, the change in the weight of the mobile body and the air pressure of the wheels Even if the distance between the road surface and the imaging unit changes due to a change in , the distance on the road surface corresponding to one pixel can be calibrated according to the change.

Also, in the second embodiment, the second output unit calculates the velocity of the moving body based on the calibration result and the pixel movement amount. Therefore, it is possible to output the moving body speed with high accuracy as the traveling information.

In addition, in the second embodiment, a white line is adopted as a feature on the road surface. Therefore, it is possible to easily identify the white line area by checking the brightness of each pixel in the captured image.

Further, in the second embodiment, as in the first embodiment, a weighted average of the current provisional pixel distance and the provisional pixel distances calculated during a predetermined number of recent calibrations in the past is calculated, and the final to obtain accurate calibration results. For this reason, it is possible to perform calibration that suppresses the influence of changes in the length of the white line in the traveling direction, etc., on the calibration result of the pixel distance.

Further, in the second embodiment, as in the case of the first embodiment, calibration is performed when the vehicle is traveling straight ahead, traveling at a constant speed, and the road surface is not inclined. Therefore, accurate pixel distance calibration can be performed.

[Modification of Embodiment]
The present invention is not limited to the first and second embodiments described above, and various modifications are possible.

For example, in the above-described first embodiment, the stop line in front of the intersection is used as the characteristic portion on the road surface. types of road markings may be employed as road surface features.

In addition, in the above-described second embodiment, white lines such as pedestrian crossings are used as characteristic portions on the road surface, but road markings other than white lines may be used as characteristic portions on the road surface.

Further, in the above-described second embodiment, the average of the front side pixel movement amount obtained from the time change of the front side image and the rear side pixel movement amount obtained from the time change of the rear side image is detected as the pixel movement amount. did. On the other hand, either one of the front side pixel movement amount and the rear side pixel movement amount may be detected as the pixel movement amount.

Further, in the above-described second embodiment, based on the change in the captured image from the time when the front side captured image becomes the first rear partial white line to the time when the rear side captured image becomes the second rear partial white line, Now calibrate the pixel distance. On the other hand, the pixel distance is calibrated based on the change in the captured image from the time when the front side captured image becomes the first front partial white line to the time when the rear side captured image becomes the second front partial white line. can be

Further, in the first and second embodiments described above, the pixel movement amount is detected by the so-called displacement amount search method, but the pixel movement amount may be detected by the image correlation method or the spatial filter method.

In the first and second embodiments described above, the pixel distance, which is the distance on the road surface corresponding to one pixel, is calibrated as the distance-related information related to the distance between the imaging unit and the road surface. On the other hand, it is also possible to calibrate the image element distance, which is the distance on the road surface corresponding to the length in the running direction of a predetermined image element, which is a set of a plurality of pixels.

Further, in the first and second embodiments described above, the second output unit outputs the moving body speed (running speed) as the running information. may be used to output the traveled distance.

In the first and second embodiments described above, the pixel distance is calibrated by calculating the weighted average of the current temporary pixel distance and a predetermined number of previous temporary pixel distances. Alternatively, the current temporary pixel distance may be used as the calibrated pixel distance.

Further, in the above-described first and second embodiments, it is assumed that the imaging unit is provided outside the detecting device, but the detecting device may be provided with the imaging unit.

Further, in the above-described first and second embodiments, the detection device is mounted on the moving body. On the other hand, the detection device may be installed outside the moving body and connected to the vehicle-mounted imaging unit or ECU via a wireless communication network.

In addition, the detection device of the first and second embodiments is configured as a computer as a calculation means including a central processing unit (CPU: Central Processing Unit), etc., and a program prepared in advance is executed on the computer. Accordingly, part or all of the functions of the detection devices of the first and second embodiments may be executed. This program is recorded in a computer-readable recording medium such as a hard disk, CD-ROM, or DVD, and is loaded from the recording medium and executed by the computer. In addition, this program may be acquired in the form of being recorded on a portable recording medium such as a CD-ROM or DVD, or may be acquired in the form of distribution via a network such as the Internet. good too.

Next, embodiments of the present invention will be described mainly with reference to FIGS. 14 to 23. FIG. In addition, in the following description, including the embodiment described above, the same or equivalent elements are denoted by the same reference numerals, and overlapping descriptions are omitted as much as possible.

[First embodiment]
First, a first embodiment of the present invention will be described mainly with reference to FIGS. 14 to 19. FIG.

<Configuration>
FIG. 14 schematically shows the configuration of the detection device 100A according to the first embodiment. This detection device 100A is one aspect of the detection device 300A according to the first embodiment described above.

As shown in FIG. 14, the detection device 100A is connected to an imaging section 210, a navigation device 220 and an ECU 230. As shown in FIG. The detection device 100A, the imaging unit 210, the navigation device 220 and the ECU 230 are mounted on the vehicle CR.

The detection device 100A described above comprises a control unit 110A. The detection device 100A also includes a storage unit 120 .

The above control unit 110A is configured with a central processing unit (CPU) as a computing means. By executing a program, the control unit 110A functions as the detection device 300A in the above-described first embodiment, that is, functions as the acquisition unit 310A, the first output unit 320A, and the second output unit 330. It has become.

A program executed by the control unit 110A is stored in the storage unit 120, loaded from the recording unit and executed. This program may be obtained in a form recorded on a portable recording medium such as a CD-ROM or DVD, or may be obtained in a form of distribution via a network such as the Internet. .

Processing executed by the control unit 110A will be described later.

Various information data used by the control unit 110A are stored in the storage unit 120 described above. Such information data includes programs executed by the control unit 110A, calibration history including the most recent pixel displacements, most recent pixel distances, and the like. The storage unit 120 is accessible by the control unit 110A.

<Action>
Next, the detection operation by the detection device 100A configured as described above will be described, focusing on the processing by the control unit 110A.

It is assumed that the imaging unit 210 has already started operating and is sequentially sending the captured image data of the road surface image captured at the cycle time TP to the detection device 100A. It is also assumed that the navigation device 220 has already started its operation, and when the vehicle CR is within a predetermined distance from an intersection in the traveling direction, it sends a notification to the detection device 100A. It is also assumed that the ECU 230 has already started operating and has sent acceleration information, steering angle information, and tilt information to the detection device 100A (see FIG. 14).

Further, in the detection device 100A, the pixel distance has already been calibrated a plurality of times, and the storage unit 120 stores a predetermined number of recent calibration times, characteristics of the feature region extracted from the previous captured image, and It is assumed that the position in the image and the calibration history including the provisional pixel distance, which will be described later, are stored. Here, the average pixel distance is stored in the storage unit 120 until the pixel distance is calibrated for the first time.

<<Processing to detect the amount of pixel movement>>
First, the pixel movement amount detection processing by the control unit 110A will be described.

As shown in FIG. 15, the control unit 110A first determines whether or not the control unit 110A has received new captured image data in step S11 during the process of detecting the pixel movement amount. If the result of determination in step S11 is negative (step S11: N), the process of step S11 is repeated.

When new captured image data is received and the result of determination in step S11 becomes affirmative (step S11: Y), the process proceeds to step S12. In this step S12, the control unit 110A extracts the characteristic region in the captured image (currently captured image) obtained from the new captured image data, and specifies the characteristics and the position in the image of the characteristic region.

Next, in step S13, the control unit 110A detects the amount of pixel movement. When detecting such a pixel movement amount, the control unit 110A refers to the characteristic region and the position in the image extracted from the previous captured image in the storage unit 120, and compares the current captured image and the previous captured image. A displacement amount of the position of the common feature region in the image is detected as a new pixel displacement amount.

Note that when there are a plurality of common characteristic regions, the control unit 110A adopts the average value of the respective displacement amounts of the plurality of characteristic regions as the pixel movement amount.

Then, the control unit 110A registers, in the storage unit 120, the characteristics and the in-image position of the characteristic region in the current captured image as the characteristics and the in-image position of the characteristic region extracted from the previous captured image. Control unit 110A also registers the new pixel shift amount in storage unit 120 as the latest pixel shift amount.

When the process of step S13 ends in this way, the process returns to step S11. Thereafter, steps S11 to S13 are repeated, and the pixel shift amount is detected each time data of a new captured image is received.

<<Output processing of vehicle speed v>>
Next, the output processing of the vehicle body speed v by the control unit 110A will be described.

When outputting the vehicle body speed v, as shown in FIG. 16, first, in step S21, the control unit 110A determines whether or not a new pixel movement amount has been detected. If the result of determination in step S21 is negative (step S21: N), the process of step S21 is repeated.

When a new pixel shift amount is detected and the result of determination in step S21 is affirmative (step S21: Y), the process proceeds to step S22. In this step S22, based on the latest pixel movement amount (PN) and the latest pixel distance (PL) registered in the storage unit 120, the vehicle body speed v is calculated by the above equation (7). Then, control unit 110A outputs the calculated vehicle body speed v to ECU 230 .

When the process of step S22 ends in this way, the process returns to step S21. After that, the processing of steps S21 and S22 is repeated, and each time a new pixel movement amount is detected, that is, each time new imaging data is received, a new vehicle body speed v is calculated, and the calculated vehicle body speed v is output to the ECU 230 .

<Processing to specify the white line area>
Next, the process of identifying the white line area by the control unit 110A will be described. Note that the process of specifying the white line area is executed in parallel with the process of detecting the amount of pixel movement and the process of outputting the vehicle speed v described above.

Such white line region identification processing is performed for pixel distance calibration, which will be described later. The calibration environment for this pixel distance calibration is the same as the case shown in FIG. 6 described above.

As shown in FIG. 17, in the process of specifying the white line area, first, in step S31, the control unit 110A determines whether or not new captured image data has been received. If the result of determination in step S31 is negative (step S31: N), the process of step S31 is repeated.

When new captured image data is received and the result of determination in step S31 becomes affirmative (step S31: Y), the process proceeds to step S32. In this step S32, the control unit 110A identifies the white line area in the captured image obtained from the new captured image data based on the brightness of each pixel in the captured image.

Next, in step S33, the control unit 110A determines whether or not the specified result of the white line area is "partial front white line (see FIG. 7A)". If the result of determination in step S33 is negative (step S33: N), the process returns to step S31. Then, the processing of steps S31 to S33 is repeated until the determination result of step S33 becomes affirmative.

When the specified result of the white line area is "partial white line in front" and the result of determination in step S33 is affirmative (step S33: Y), the process proceeds to step S34. In this step S34, the control unit 110A calculates the front length a (see FIG. 7A). Then, the process proceeds to step S35.

Next, in step S35, the control unit 110A determines whether or not new captured image data has been received. If the result of determination in step S35 is negative (step S35: N), the process of step S35 is repeated.

When new captured image data is received and the result of determination in step S35 becomes affirmative (step S35: Y), the process proceeds to step S36. In this step S36, the control unit 110A identifies the white line area in the captured image obtained from the new captured image data based on the brightness of each pixel in the captured image.

Next, in step S37, the control unit 110A determines whether or not the specified result of the white line area is "rear partial white line (see FIG. 7B)". If the result of determination in step S37 is negative (step S37: N), the process returns to step S35. Then, the processing of steps S35 to S37 is repeated until the determination result of step S37 becomes affirmative.

When the specified result of the white line area is "rear partial white line" and the result of determination in step S37 is affirmative (step S37: Y), the process proceeds to step S38. In this step S38, the control unit 110A calculates the front length b (see FIG. 7B). Then, the process returns to step S31.

After that, steps S31 to S38 are repeated. As a result, the white line area is specified each time the vehicle CR crosses the white line.

<<Monitoring of calibration conditions>>
Next, the calibration condition monitoring process by the control unit 110A will be described. The calibration condition monitoring process is executed in parallel with the above-described pixel movement amount detection process, vehicle body speed v output process, and white line area specification process.

As shown in FIG. 18, in the calibration condition monitoring process, first, in step S41, it is determined whether or not the vehicle CR is traveling straight based on the latest steering angle information sent from the ECU 230 . If the result of determination in step S41 is negative (step S41: N), the process proceeds to step S44, which will be described later.

If the result of determination in step S41 is affirmative (step S41: Y), the process proceeds to step S42. In step S42, the control unit 110A determines whether the vehicle CR is accelerating or decelerating based on the latest acceleration information sent from the ECU 230. If the result of determination in step S42 is affirmative (step S42: Y), the process proceeds to step S44.

If the result of determination in step S42 is negative (step S42: N), the process proceeds to step S43. In this step S43, the control unit 110A determines whether or not there is a body tilt of the vehicle CR based on the latest tilt information sent from the ECU 230. FIG.

If the result of determination in step S43 is affirmative (step S43: Y), the process proceeds to step S44. In this step S44, the control unit 110A sets the calibration condition flag to "OFF". Then, the process returns to step S41.

On the other hand, if the result of determination in step S43 is negative (step S43: N), the process proceeds to step S45. In this step S45, the control unit 110A sets the calibration condition flag to "ON". Then, the process returns to step S41.

After that, the processing of steps S41 to S44 is repeated. As a result, when it is determined that the calibration condition that the vehicle CR is traveling straight ahead at a constant speed on a flat road surface is satisfied, the calibration condition flag is turned "ON". On the other hand, when it is determined that the calibration condition is not satisfied, the calibration condition flag is set to "OFF".

<<Calibration process of pixel distance>>
Next, the pixel distance calibration process by the control unit 110A will be described. The pixel distance calibration process is executed in parallel with the above-described pixel movement amount detection process, vehicle body speed v output process, white line area specification process, and calibration condition monitor process.

In the pixel distance calibration process, as shown in FIG. 19, first, in step S50, whether or not the control unit 110A receives from the navigation device 220 an indication that an intersection existing in the traveling direction is within a predetermined distance. By determining , it is determined whether or not an intersection is approached. If the result of determination in step S50 is negative (step S50: N), the process of step S50 is repeated.

When the vehicle approaches the intersection and the result of determination in step S50 becomes affirmative (step S50: Y), the process proceeds to step S51. In this step S51, the control unit 110A determines whether or not the calibration condition flag is "ON", thereby determining whether or not the vehicle CR is traveling straight ahead at a constant speed on a flat road surface.

If the result of determination in step S51 is negative (step S51: N), the process returns to step S50. Then, the processing of steps S50 and S51 is repeated.

If the result of determination in step S51 is affirmative (step S51: Y), the process proceeds to step S52. In this step S52, the control unit 110A determines whether or not the above-described white line area identifying process has identified that the "front part of the white line" has been identified, and whether or not the front length a has been obtained.

If the result of determination in step S51 is negative (step S52: N), the process returns to step S51. Then, the processes of steps S51 and S52 are repeated on condition that the result of determination in step S51 is not negative.

If the result of determination in step S52 is affirmative (step S52: Y), the process proceeds to step S53. In this step S53, the distance parameter L is set to the forward length a.

Next, in step S54, the control unit 110A determines whether or not the calibration condition flag is "ON", thereby determining whether or not the vehicle CR is traveling straight ahead at a constant speed on a flat road surface. If the result of determination in step S54 is negative (step S54: N), the process returns to step S50.

If the result of determination in step S54 is affirmative (step S54: Y), the process proceeds to step S55. In this step S55, the control unit 110A determines whether or not a new pixel movement amount (ΔX _j ) has been detected by the pixel movement amount detection process described above.

If the result of determination in step S55 is negative (step S55: N), the process returns to step S54. Then, the processes of steps S54 and S55 are repeated on condition that the result of determination in step S54 is not negative.

If the result of determination in step S55 is affirmative (step S55: Y), the process proceeds to step S56. In this step S56, the control unit 110A updates the value of the distance parameter L by adding the new pixel movement amount (ΔX _j ) to the value of the distance parameter L up to that point.

Next, in step S57, the control unit 110A determines whether or not the "rear part of the white line" is specified by the white line area specifying process described above, and the front length b is obtained. If the result of determination in step S57 is negative (step S57: N), the process returns to step S54. Then, on condition that the result of determination in step S54 is not negative, the processing of steps S54 to S57 is repeated.

If the result of determination in step S57 is affirmative (step S57: Y), the process proceeds to step S58. In this step S58, the control unit 110A updates the value of the distance parameter L by subtracting the forward length b from the value of the distance parameter L up to that point.

Next, in step S59, the control unit 110A calculates the provisional pixel distance PT [cm] by the following equation (11) based on the value of the distance parameter L at that time.
PT=45/L (11)
Note that the formula (11) is equivalent to the formula (8) described above.

Subsequently, the control unit 110A calculates a weighted average of the current calculated interim pixel distance and the interim pixel distances calculated during the predetermined number of past calibrations stored in the storage unit 120, and newly Calculate the calibrated pixel distance. After this, control unit 110A updates the latest pixel distance in storage unit 120 to the newly calibrated pixel distance and the oldest interim pixel distance previously stored in storage unit 120. Instead, the current temporary pixel distance is stored in the storage unit 120 together with the current time. As a result, the control unit 110A will calculate the vehicle body speed v using the new pixel distance in the process of outputting the vehicle body speed v described above.

When the process of step S59 ends in this way, the process returns to step S50. Then, the processing of steps S50 to S59 is repeated. As a result, the pixel distance is calibrated each time the vehicle CR runs straight on a flat road surface at a constant speed and crosses the stop line (white line) before the intersection.

As described above, in the first embodiment, the control unit 110A detects the pixel movement amount based on the image captured by the imaging unit 210 of the vehicle CR. Also, the control unit 110A identifies the white line area in the captured image. Then, the control unit 110A determines whether the vehicle CR is traveling on a flat road surface based on the known length of the stop line along the running direction of the vehicle CR, in addition to the pixel movement amount and the result of specifying the white line area in the captured image. When it can be determined that the vehicle is traveling straight at a constant speed, the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging unit 210 and the road surface, is calibrated.

Therefore, according to the first embodiment, even if the distance between the road surface and the imaging unit changes due to changes in the vehicle weight or the air pressure of the wheels, the distance on the road surface corresponding to one pixel can be adjusted according to the change. can be calibrated.

Also, in the first embodiment, the control unit 110A calculates the vehicle body speed based on the calibration result and the pixel movement amount. Therefore, it is possible to output the accurate vehicle body speed as the traveling information.

In addition, in the first embodiment, the stop line is adopted as the feature on the road surface. Therefore, by checking the brightness of each pixel in the captured image, it is possible to easily identify the stop line area (that is, the white line area).

Further, in the first embodiment, information indicating that the vehicle is approaching an intersection is acquired, and it is presumed that the vehicle will pass through the stop line. Therefore, efficient calibration can be performed.

In addition, in the first embodiment, a final calibration result is obtained by calculating a weighted average of the current temporary pixel distance and the temporary pixel distances calculated during a predetermined number of past calibrations. For this reason, it is possible to perform calibration that suppresses the influence of changes in the length of the stop line in the direction of travel, etc., on the calibration result of the pixel distance.

Further, in the first embodiment, the calibration is performed when the vehicle is traveling straight ahead, traveling at a constant speed, and the road surface is not inclined. Therefore, accurate pixel distance calibration can be performed.

[Second embodiment]
Next, a second embodiment of the present invention will be described mainly with reference to FIGS. 20-23.

<Configuration>
FIG. 20 schematically shows the configuration of the detection device 100B according to the first example. This detection device 100B is one aspect of the detection device 300B according to the second embodiment described above.

As shown in FIG. 20, the detection device 100B is connected to imaging units 210 _F and 210 _R and the ECU 230 . The detection device 100B, the imaging units _210F and _210R , and the ECU 230 are mounted on the vehicle CR.

The detection device 100B described above comprises a control unit 110B. The detection device 100B also includes a storage unit 120 .

The above control unit 110B is configured with a central processing unit (CPU) as a calculation means. By executing a program, the control unit 110B functions as the detection device 300B in the above-described second embodiment, that is, functions as the acquisition unit 310B, the first output unit 320B, and the second output unit 330. It has become.

A program executed by the control unit 110B is stored in the storage unit 120, loaded from the recording unit and executed. This program may be obtained in a form recorded on a portable recording medium such as a CD-ROM or DVD, or may be obtained in a form of distribution via a network such as the Internet. .

Processing executed by the control unit 110B will be described later.

Various information data used by the control unit 110B are stored in the storage unit 120 described above. Such information data includes the program executed by control unit 110B, the most recent pixel displacement, calibration history including the most recent pixel distance, and the like. The storage unit 120 is accessible by the control unit 110B.

<Action>
Next, the detection operation by the detection device 100B configured as described above will be described, focusing on the processing by the control unit 110B.

Note that the imaging units 210 _F and 210 _R have already started operating, and sequentially send the data of the front side captured image and the data of the rear side captured image of the road surface image captured at the cycle time TP to the detection device 100B. It is assumed that there is It is also assumed that the ECU 230 has already started operating and has sent acceleration information, steering angle information, and tilt information to the detection device 100B (see FIG. 20).

Furthermore, in the detection device 100B, the pixel distance has already been calibrated a plurality of times, and the storage unit 120 stores a predetermined number of recent calibration times, characteristics of the feature region extracted from the previous captured image, and It is assumed that the position in the image and the calibration history including the provisional pixel distance, which will be described later, are stored. Here, the average pixel distance is stored in the storage unit 120 until the pixel distance is calibrated for the first time.

<<Processing to detect the amount of pixel movement>>
First, the pixel movement amount detection processing by the control unit 110B will be described.

As shown in FIG. 21, in the process of detecting the amount of pixel movement, first, in step S61, the control unit 110B determines whether or not new front-side captured image data and new rear-side captured image data have been received. judge. If the result of determination in step S61 is negative (step S61: N), the process of step S61 is repeated.

When the data of the new front side captured image is received and the result of determination in step S61 becomes affirmative (step S61: Y), the process proceeds to step S62. In this step S62, the control unit 110B extracts a characteristic region in the front-side captured image obtained from the data of the new front-side captured image, and specifies the characteristic and the position in the image of the characteristic region.

Next, in step S63, the control unit 110B detects the front side pixel movement amount. When detecting the front side pixel movement amount, the control unit 110B refers to the characteristic region and the position in the image extracted from the previous front side captured image in the storage unit 120, and compares the current front side captured image and the previous front side image. The amount of displacement of the in-image position of the characteristic region common to the captured image is detected as a new amount of front-side pixel movement.

Note that when there are a plurality of common characteristic regions, the control unit 110B adopts the average value of the respective displacement amounts of the plurality of characteristic regions as the new front side pixel movement amount.

Then, the control unit 110B registers, in the storage unit 120, the characteristics and the intra-image position of the characteristic region in the current captured image as the characteristics and the intra-image position of the characteristic region extracted from the previous captured image. Control unit 110B also registers the new pixel shift amount in storage unit 120 as the latest pixel shift amount.

Next, in step S64, the control unit 110B extracts a characteristic region in the rear-side captured image obtained from the data of the new rear-side captured image, and specifies the characteristic and the position in the image of the characteristic region.

Next, in step S65, the control unit 110B detects the rear pixel movement amount. When detecting the rear-side pixel movement amount, the control unit 110B refers to the characteristic region and the position in the image extracted from the previous rear-side captured image in the storage unit 120, and compares it with the current rear-side captured image. The amount of displacement of the in-image position of the feature region common to the previous captured image of the rear side is detected as a new amount of movement of pixels on the rear side.

Note that if there are a plurality of common feature regions, the control unit 110B calculates the average value of the respective displacement amounts of the plurality of feature regions as a new rear pixel shift, as in the case of detecting the front pixel shift amount. adopted as quantity.

Next, in step S66, control unit 110B detects a new pixel shift amount. In detecting such pixel movement amount, the control unit 110B detects a new pixel movement amount by calculating the average of the front side pixel movement amount and the rear side pixel movement amount.

Then, the control unit 110B converts the characteristics and in-image positions of the characteristic regions in the current front-side captured image and the characteristics and in-image positions of the characteristic regions in the current rear-side captured image into the characteristics and in-image positions of the characteristic regions in the previous front-side captured image. They are registered in the storage unit 120 as the characteristics and the intra-image position, and the characteristics and the intra-image position of the characteristic region in the previous rear-side captured image. Control unit 110B also registers the new pixel shift amount in storage unit 120 as the latest pixel shift amount.

When the process of step S66 is completed in this way, the process returns to step S61. After that, steps S61 to S66 are repeated, and the pixel movement amount is detected each time new front-side captured image data and new rear-side captured image data are received.

<<Output processing of vehicle speed v>>
The output processing of the vehicle body speed v by the control unit 110B is the same as the above-described output processing of the vehicle body speed v by the control unit 110A (see FIG. 16).

<Processing to specify the white line area>
Next, the processing for specifying the white line area by the control unit 110B will be described. Note that the process of specifying the white line area is executed in parallel with the process of detecting the amount of pixel movement and the process of outputting the vehicle speed v described above.

Such white line region identification processing is performed for pixel distance calibration, which will be described later. The calibration environment for this pixel distance calibration is the same as the case shown in FIG. 11 described above.

As shown in FIG. 22, in the process of identifying the white line area, first, in step S71, the control unit 110B determines whether or not new front-side captured image data has been received. If the result of determination in step S71 is negative (step S71: N), the process of step S71 is repeated.

When new front side captured image data is received and the result of determination in step S71 becomes affirmative (step S71: Y), the process proceeds to step S72. In this step S72, the control unit 110B identifies the white line region in the front side captured image obtained from the data of the new front side captured image based on the brightness of each pixel in the front side captured image.

Next, in step S73, the control unit 110B determines whether or not the specified result of the white line area is the "first rear partial white line (see FIG. 12A)". If the result of determination in step S73 is negative (step S73: N), the process returns to step S71. Then, the processing of steps S71 to S73 is repeated until the determination result of step S73 becomes affirmative.

If the white line region specified in the front side captured image is "first rear partial white line" and the determination result in step S73 is affirmative (step S73: Y), the process proceeds to step S74. In this step S74, the control unit 110B obtains the front length a (see FIG. 12A).

Note that, in the second embodiment, the control unit 110B performs the “first rear partial white line” on the condition that the white line continues to be clearly identified until it becomes the “first rear partial white line” in the front side captured image. It is designed to identify the "white line".

Next, in step S75, the control unit 110B determines whether or not new rear-side captured image data has been received. If the result of determination in step S75 is negative (step S75: N), the process of step S75 is repeated.

When the new rear side captured image data is received and the determination result in step S75 is affirmative (step S75: Y), the process proceeds to step S76. In this step S76, the control unit 110B identifies the white line region in the rear side captured image obtained from the data of the new rear side captured image based on the brightness of each pixel in the captured image.

Next, in step S77, the control unit 110B determines whether or not the specified result of the white line area is the "second rear partial white line (see FIG. 12B)". If the result of determination in step S77 is negative (step S77: N), the process returns to step S75. Then, the processing of steps S75 to S77 is repeated until the determination result of step S77 becomes affirmative.

When the specified result of the white line area in the rear side captured image is "second rear partial white line" and the result of determination in step S77 is affirmative (step S77: Y), the process proceeds to step S78. In this step S78, the control unit 110B obtains the front length b (see FIG. 12B). Then, the process returns to step S71.

After that, steps S71 to S78 are repeated. As a result, the white line area is specified each time the vehicle CR crosses the white line.

<<Monitoring of calibration conditions>>
The calibration condition monitoring process by the control unit 110B is the same as the calibration condition monitoring process by the control unit 110A described above (see FIG. 18).

<<Calibration process of pixel distance>>
Next, the pixel distance calibration process by control unit 110B will be described. The pixel distance calibration process is executed in parallel with the above-described pixel movement amount detection process, vehicle body speed v output process, white line area specification process, and calibration condition monitor process.

During the pixel distance calibration process, as shown in FIG. 23, first, in step S81, the control unit 110B determines whether or not the calibration condition flag is "ON" to ensure that the vehicle CR is flat. It is determined whether or not the vehicle is traveling straight on the road at a constant speed. If the result of determination in step S81 is negative (step S81: N), the process of step S81 is repeated.

If the result of determination in step S81 is affirmative (step S81: Y), the process proceeds to step S82. In this step S82, the control unit 110B determines whether or not the front side captured image has been identified as the "first rear partial white line" by the white line region identification processing described above, and the front length a has been obtained.

If the result of determination in step S82 is negative (step S82: N), the process returns to step S81. Then, the processes of steps S81 and S82 are repeated.

If the result of determination in step S82 is affirmative (step S82: Y), the process proceeds to step S83. In this step S83, the distance parameter L is set to the forward length a.

Next, in step S84, the control unit 110B determines whether or not the calibration condition flag is "ON", thereby determining whether or not the vehicle CR is traveling straight ahead at a constant speed on a flat road surface. If the result of determination in step S84 is negative (step S84: N), the process returns to step S81.

If the result of determination in step S84 is affirmative (step S84: Y), the process proceeds to step S85. In this step S85, the control unit 110B determines whether or not a new pixel movement amount (ΔX _j ) has been detected by the pixel movement amount detection processing described above.

If the result of determination in step S85 is negative (step S85: N), the process returns to step S84. Then, the processes of steps S84 and S85 are repeated on condition that the result of determination in step S84 is not negative.

If the result of determination in step S85 is affirmative (step S85: Y), the process proceeds to step S86. In this step S86, the control unit 110B updates the value of the distance parameter L by adding the new pixel movement amount (ΔX _j ) to the value of the distance parameter L up to that point.

Next, in step S87, the control unit 110B determines whether the above-described white line area specifying process has specified that the rear captured image is the "second rear partial white line" and whether the front length b has been obtained. judge. If the result of determination in step S87 is negative (step S87: N), the process returns to step S84. Then, on condition that the result of determination in step S84 is not negative, the processing of steps S84 to S87 is repeated.

If the result of determination in step S87 is affirmative (step S87: Y), the process proceeds to step S88. In this step S88, the control unit 110B updates the value of the distance parameter L by subtracting the forward length b from the value of the distance parameter L up to that point.

Next, in step S89, the control unit 110B calculates the provisional pixel distance PT [cm] by the following equation (12) based on the value of the distance parameter L at that time.
PT=D/L (12)
Note that the expression (12) is equivalent to the expression (10) described above.

Subsequently, the control unit 110B calculates a weighted average of the current calculated interim pixel distance and the interim pixel distances calculated during the predetermined number of past calibrations stored in the storage unit 120, and newly Calculate the calibrated pixel distance. After this, control unit 110B updates the latest pixel distance in storage unit 120 to the newly calibrated pixel distance and the oldest interim pixel distance previously stored in storage unit 120. Instead, the current temporary pixel distance is stored in the storage unit 120 together with the current time. As a result, the control unit 110B will calculate the vehicle body speed v using the new pixel distance in the process of outputting the vehicle body speed v described above.

When the process of step S89 is completed in this way, the process returns to step S81. Then, the processing of steps S81 to S89 is repeated. As a result, the pixel distance is calibrated each time the vehicle CR runs straight on a flat road surface at a constant speed and crosses the white line before the intersection.

As _described above, in the second embodiment, the control unit _110B performs the Detect pixel movement. Also, the control unit 110B identifies the white line area in the captured image. Then, when the control unit 110B can determine that the vehicle CR is traveling straight ahead on a flat road surface at a constant speed based on the known distance D in addition to the pixel movement amount and the result of specifying the white line area in the captured image. First, calibrate the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging units 210 _F and 210 _R and the road surface.

Therefore, according to the second embodiment, regardless of whether the length of the white line along the traveling direction of the moving body, which is a feature on the road surface, is known or unknown, the change in the weight of the moving body and the movement of the wheels Even if the distance between the road surface and the imaging unit changes due to changes in air pressure, the distance on the road surface corresponding to one pixel can be calibrated according to the change.

Also, in the second embodiment, the control unit 110B outputs the vehicle body speed based on the calibration result and the pixel movement amount. Therefore, it is possible to output the accurate vehicle body speed as the traveling information.

In addition, in the second embodiment, a white line is adopted as a feature on the road surface. Therefore, it is possible to easily identify the white line area by checking the brightness of each pixel in the captured image.

In addition, in the second embodiment, as in the first embodiment, a weighted average of the current temporary pixel distance and the temporary pixel distances calculated during calibration a predetermined number of times in the past is calculated, and the final Get calibration results. For this reason, it is possible to perform calibration that suppresses the influence of changes in the length of the white line in the traveling direction, etc., on the calibration result of the pixel distance.

Further, in the second embodiment, as in the case of the first embodiment, calibration is performed when the vehicle is traveling straight ahead, traveling at a constant speed, and the road surface is not inclined. Therefore, accurate pixel distance calibration can be performed.

[Modification of Embodiment]
The present invention is not limited to the first and second embodiments described above, and various modifications are possible.

For example, the first embodiment can be modified in the same manner as the first embodiment described above. Also, the second example can be modified in the same manner as the above-described modification of the second embodiment.

100A, 100B ... detection device 110A, 110B ... control unit (acquisition unit, first output unit, second output unit)
300A, 300B... Detector 310A, 310B... Acquisition unit 320A, 320B... First output unit 321A, 321B... Movement amount detection unit 322A, 322B... Identification unit 323A, 323B... Calibration unit 330... Second output unit

Claims (1)

an acquisition unit that acquires an image of the characteristic part on the road surface captured by the imaging unit mounted on the moving body;
an output unit that outputs distance-related information related to the distance between the mounting position of the imaging unit and the road surface based on the image of the characteristic part on the road surface acquired by the acquisition unit;
The image capturing unit captures an image of the road surface immediately below a fixed position where the image capturing unit is fixed to the moving body.
A detection device characterized by: