JP2020074136A - Detection device, detection method, detection program and recording medium - Google Patents

Abstract

To perform calibration corresponding to a change even if a distance between an imaging part and a road face is changed due to a change in the weight of a moving body and a change in the pneumatic pressure of wheels.SOLUTION: A moving amount detection part 321a detects a pixel moving amount on the basis of a captured imaged image which is captured by an imaging part 210 with respect to a stop line being a feature part on a road face having a well-known length along a traveling direction of a moving body MV. A specification part 322A specifies a white line region in the captured image. Then, when it is determined that the moving body MV linearly travels on the flat road face at a constant velocity, the calibration part 323A calibrates a distance on the road face corresponding to one pixel related to a distance between the imaging part 210 and the road face, on the basis of the well-known length along the traveling direction of the moving body MV on the stop line, in addition to the pixel moving amount and a specified result of the white line region in the captured image.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a detection device, a detection method and a detection program, and a recording medium having the detection program recorded therein.

  The slip ratio λ that normalizes the difference between the wheel speed of a moving body such as a vehicle and the moving body speed, and the friction coefficient μ that normalizes the grip force between the wheel and the road surface are as shown in FIGS. 1 and 2. (Hereinafter referred to as “μ-λ characteristic”). Here, FIG. 1 shows the μ-λ characteristic during driving, and FIG. 2 shows the relationship with the μ-λ characteristic during braking.

  1 and 2, the μ-λ characteristic on a dry road surface is shown by a solid line, the μ-λ characteristic on a wet road surface is shown by a chain line, and the μ-λ characteristic on a frozen road surface is shown by two points. It is shown by the dashed line.

  In the change of the friction coefficient μ accompanying the increase of the slip ratio λ during driving shown in FIG. 1, a state where the friction coefficient μ is smaller than the maximum slip ratio is a stable region in which the moving body can stably travel. There is. On the other hand, in a state where the friction coefficient μ is larger than the maximum slip ratio, the grip force is reduced, and in the worst case, an idling or locking phenomenon occurs, which is an unstable region.

  In the variation of the friction coefficient μ with the increase of the slip ratio λ during braking shown in FIG. 2, the stable region is a state in which the friction coefficient μ is larger than the minimum slip ratio. On the other hand, in the state where the friction coefficient μ is smaller than the minimum slip ratio, the region is unstable.

  Then, if the slip ratio λ is controlled within a range equal to or smaller than the absolute value of the slip ratio at which the absolute value of the friction coefficient μ becomes the maximum value, the moving body can maintain stable traveling. On the other hand, if the unstable state continues and the tire slips or locks, control of driving, braking, and steering of the vehicle becomes impossible.

  Therefore, in order to avoid the risk of accidents, an ABS (Antilock Brake System) mainly using brake hydraulic pressure control and engine control is adopted in an internal combustion engine vehicle. In such an ABS or the like, the slip state is determined, and the drive torque of the engine and the braking torque of the brake hydraulic pressure are controlled to bring the control close to the stable region. Also in electric vehicles, anti-slip control has been proposed in which the slip ratio λ is estimated and the torque of the motor is appropriately controlled to maintain the slip ratio in a stable region. Thus, detecting the slip ratio λ is very important for understanding the running state.

Since the slip ratio λ is calculated by the following equation (1), the wheel radius r, the rotational angular velocity ω, and the moving body velocity v are required.
λ = (r · ω−v) / Max (r · ω, v) (1)

  Here, Max (r · ω, v) represents the larger one of (r · ω) and v. At the time of driving, since (r · ω) is larger than v, Max (r · ω, v) = r · ω. On the other hand, during braking, v is larger than (r · ω), and thus Max (r · ω, v) = v.

  For example, when the moving body is an automobile, the wheel radius r is considered to be constant if the air pressure of the wheels is sufficient. Further, the rotational angular velocity ω can be detected from a pulse output of an encoder mounted on a wheel or, in the case of an electric vehicle, a signal output of a resolver connected to a motor.

On the other hand, the following (a) to (c) are generally mentioned as a method for detecting the moving body speed v.
(A) Calculated from the rotational speed of the non-driving wheel (b) Calculated by integrating the acceleration value detected by the acceleration sensor (c) Calculated based on the detection result by the optical sensor

  Here, in the method of (a), since the brake is applied to all the wheels, the moving body speed v at the time of braking cannot be detected. Further, in the case of four-wheel drive, since there are no non-driving wheels, the moving body speed v cannot be detected.

  Further, in the method of (b), since the output of the acceleration sensor is integrated, the offset existing in the output of the acceleration sensor is accumulated. As a result, the moving body speed v cannot be detected accurately.

  The method (c) is expected in which the drawbacks of the methods (a) and (b) do not exist in principle. As a technique adopting the method (c), the technique described in Patent Document 1 has been proposed (hereinafter, referred to as "conventional example"). In the technique of this conventional example, the running state or the stopped state of the vehicle is determined from an image signal obtained by capturing an image of a scene near the vehicle. Further, the vehicle body speed of the host vehicle (that is, the moving body speed) is detected based on the size of the road marking on the image and the size of the road marking on the road. In the conventional technology, the vehicle speed is detected only when the road marking is present in the image, and only the running / stopping determination is made when the road marking is not in the image.

JP, 2009-205642, A

  In the technique of the conventional example described above, the vehicle speed is not detected when the road marking does not exist in the captured image. As a result, even if the vehicle speed changes while the road marking is not present in the captured image, the vehicle speed detected at the time when the road marking was last present in the captured image remains as it is. It will be estimated as speed. For this reason, it was difficult to accurately detect the vehicle speed at each time point.

  Further, in the technique of the conventional example, since the image pickup device is mounted on the vehicle body, the distance between the image pickup device and the road surface changes depending on the number of passengers, the load of luggage, and the air pressure of the wheels. As a result, since the imaging magnification changes, the size of the road marking in the captured image changes.

For example, as illustrated in FIG. 3A, the imaging unit 210 includes an imaging lens system 211 (focal length: f) and a square imaging surface 212 (length of one side: D) to form an image. The distance from the lens system 211 to the imaging surface 212 is “d”. In this case, when the image of the road surface LD directly below is taken, if the distance from the imaging lens system 211 to the road surface LD (hereinafter, also referred to as “the distance from the image pickup unit 210 to the road surface LD”) is “h”, the image pickup unit 210. The optical magnification m of the image pickup by is expressed by the following equation (2).
m = 1 / ((h / f) -1) (2)
The pixel configuration on the imaging surface 212 is NU × NU (for example, 30 × 30).

As a result, the length H of one side of the square area to be imaged on the road surface LD is expressed by the following equation (3).
H = D / m = D · ((h / f) −1) (3)

  For example, if f = 8 [mm], D = 1.7 [mm], and h = 495 [mm], then H = 103.5 [mm].

Now, as shown in FIG. 3 (B), when the vehicle weight becomes heavier and the distance h becomes shorter and becomes the distance h ^* from the state of FIG. 3 (A), the optical magnification of the image pickup by the image pickup unit 210 becomes In the state of FIG. 3A, “m” is changed to “m ^* ” represented by the following expression (4).
m ^* = 1 / ((h ^* / f) -1) (4)

As a result, the length of one side of the square area to be imaged changes from “H” in the state of FIG. 3A to “H ^* ” represented by the following equation (5).
H ^* = D / m ^* = D ^* ((h ^* / f) -1) (5)

For example, when h ^* = 445 [mm], H ^* = 92.9 [mm], which is about 10% change from H = 103.5 [mm] in the state of FIG. As a result, the size of the area on the road surface corresponding to the size of 1 pixel also changes by about 10% from the state of FIG. 3A described above.

  By the way, when the moving distance of the image is obtained by the correlation between the captured images for each unit time and the vehicle speed in the front-rear direction and the lateral direction is calculated, an area on the road surface corresponding to the size of one pixel in the captured image. If the magnitude of the changes, the calculated vehicle speed also changes. That is, when the size of the area on the road surface corresponding to the size of one pixel changes by about 10%, the calculated speed value also changes by about 10%, which causes an error.

  The dynamic error with respect to the road surface fluctuation can be less affected by the error by the averaging process. However, since the change in vehicle weight and the change in air pressure of the wheels are offset, the influence of error cannot be suppressed by the averaging process.

  Therefore, there is a demand for a technique capable of accurately detecting the vehicle body speed even when the vehicle weight changes or the wheel air pressure changes. Responding to such a request is one of the problems to be solved by the present invention.

  The present invention has been made in view of the above, and even if the distance between the road surface and the imaging unit changes due to a change in vehicle weight or a change in wheel air pressure, calibration can be performed according to the change. An object is to provide a detection device and a detection method.

The invention according to claim 1 obtains a plurality of images including a road surface characteristic portion having a known length, which are imaged along an advancing direction at predetermined time intervals by an imaging unit mounted on a moving body. And a number of image areas of a predetermined width corresponding to the known length based on the plurality of images acquired by the acquisition unit, and based on the calculated number and the known length. And an output unit that outputs distance information related to the distance on the road surface corresponding to the predetermined width.
According to a fourth aspect of the present invention, an acquisition unit that acquires a plurality of images including a road surface feature imaged by the imaging unit mounted on the moving body along the traveling direction at predetermined time intervals; The information indicating the distance on the road surface corresponding to the width of the predetermined image area in the image obtained based on the plurality of acquired images is calibrated based on the distance information output in the past, and the calibration result is updated. A detection device, comprising: a first output unit that outputs distance information; and a second output unit that outputs the speed of the moving body using the new distance information.

  According to a fifth aspect of the present invention, there is provided a detection method used in a detection device including an acquisition unit and an output unit, wherein the acquisition unit is moved at a predetermined time interval by an imaging unit mounted on a moving body. An acquisition step of acquiring a plurality of images including a road surface characteristic portion having a known length, which are imaged along the; and the output section, based on the plurality of images acquired in the acquisition step, the known step. Calculates the number of image areas of a predetermined width corresponding to the length of, and outputs distance information related to the distance on the road surface corresponding to the predetermined width based on the calculated number and the known length. And a distance information output step of:

  A sixth aspect of the present invention is a detection program that causes a computer included in the detection device to execute the detection method according to the fifth aspect.

  The invention according to claim 7 is a recording medium characterized in that the detection program according to claim 6 is recorded so that it can be read by a computer included in the detection device.

It is a figure which shows the relationship between the slip ratio at the time of driving, and a friction coefficient. It is a figure which shows the relationship between the slip ratio at the time of braking, and a friction coefficient. FIG. 6 is a diagram for explaining a change in a speed calculation value due to a change in vehicle weight or the like. It is a figure which shows the structure of the detection apparatus which concerns on 1st Embodiment of this invention. 5 is a diagram for explaining the operation of the movement amount detection unit of FIG. 4. FIG. It is a figure for demonstrating the calibration environment in 1st Embodiment. FIG. 5 is a diagram for explaining the operation of the identifying unit in FIG. 4. FIG. 5 is a diagram for explaining a calibration operation by the calibration unit of FIG. 4. It is a figure which shows the structure of the detection apparatus which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating the relationship of the arrangement position of two imaging parts of FIG. It is a figure for demonstrating the calibration environment in 2nd Embodiment. It is a figure for demonstrating operation | movement of the specific part of FIG. FIG. 10 is a diagram for explaining a calibration operation by the calibration unit in FIG. 9. It is a figure which shows the structure of the detection apparatus which concerns on 1st Example of this invention. 15 is a flowchart for explaining a pixel movement amount detection process executed by the control unit of FIG. 14. 16 is a flowchart for explaining a vehicle body speed output process executed by the control unit of FIG. 14. 15 is a flowchart for explaining white line area specifying processing executed by the control unit of FIG. 14. 15 is a flowchart for explaining a calibration condition monitoring process executed by the control unit of FIG. 14. 15 is a flowchart for explaining a pixel distance calibration process executed by the control unit of FIG. 14. It is a figure which shows the structure of the detection apparatus which concerns on 2nd Example of this invention. 21 is a flow chart for explaining a pixel movement amount detection process executed by the control unit of FIG. 20. 21 is a flowchart for explaining white line area specifying processing executed by the control unit of FIG. 20. 21 is a flowchart illustrating a pixel distance calibration process executed by the control unit of FIG. 20.

  Hereinafter, an embodiment of the present invention will be described with reference to FIGS. In the following description and drawings, the same or equivalent elements will be denoted by the same reference symbols, without redundant description.

[First Embodiment]
First, a first embodiment of the present invention will be described with reference to FIGS.

<Structure>
FIG. 4 is a block diagram showing the configuration of the detection device 300A according to the first embodiment. As shown in FIG. 4, the detection device 300A is connected to the imaging unit 210, the navigation device 220, and an ECU (Electronic Control Unit) 230. The detection device 300A, the imaging unit 210, the navigation device 220, and the ECU 230 are mounted on the moving body MV.

  The image capturing unit 210 is mounted at a fixed position of the moving body MV and captures an image of the road surface immediately below the fixed position, as described with reference to FIG. The image capturing unit 210 periodically captures an image of the road surface at a cycle time TP. The data of the image thus captured (hereinafter referred to as “captured image data”) is sent to the detection device 300A.

  It should be noted that “cycle time TP” means that, even if the moving body MV is traveling at high speed, two images captured at the cycle time TP include a common road surface area, It is determined in advance based on experiments, simulations and the like.

  The navigation device 220 described above provides the user with driving operation support for the moving body MV based on the map information and the current position information. When the moving body MV is within a predetermined distance from an intersection existing in the traveling direction, the navigation device 220 sends a message to that effect to the detection device 300A.

  Note that the “predetermined distance” is, from the viewpoint of appropriate prior notification for imaging from the beginning to the end along the traveling direction regarding the stop line before the intersection adopted as the road surface characteristic portion in the first embodiment, from the viewpoint of the experiment, It is determined in advance based on simulation, experience, etc.

  The ECU 230 controls the traveling of the moving body MV or sends traveling information to the user based on sensor detection information acquired from various sensors such as a wheel rotation speed sensor, an acceleration sensor, a steering angle sensor, and an inclination sensor. To provide. The ECU 230 sends acceleration information, steering angle information, and tilt information to the detection device 300A.

  The ECU 230 further acquires the moving body speed (running speed information) sent from the detection device 300A, and uses the acquired moving body speed for traveling control of the moving body MV.

  As shown in FIG. 4, the detection device 300A includes an acquisition unit 310A and a first output unit 320A. Further, the detection device 300A includes a second output section 330.

  The acquisition unit 310A described above receives the captured image data sent from the imaging unit 210. Then, the acquisition unit 310A sends the captured image data to the first output unit 320A.

  The first output unit 320A receives the captured image data sent from the acquisition unit 310A. Then, the first output unit 320A detects the number of movement amounts of pixels at the same road surface position between two images captured at the cycle time TP (hereinafter, referred to as “pixel movement amount”), and will be described later. If the calibration condition is satisfied, the distance on the road surface corresponding to one pixel (hereinafter, referred to as “pixel distance”) is calibrated. The detected pixel movement amount and the calibrated pixel distance are output to the second output unit 330. Details of the configuration of the first output unit 320A having such a function will be described later.

  Each time the second output unit 330 receives the pixel movement amount sent from the first output unit 320A, the second output unit 330 uses the latest calibrated pixel distance and the cycle time TP as the traveling information of the moving body MV. Calculate speed. Then, the second output unit 330 outputs the calculated moving body speed to the ECU 230.

<< Configuration of the first output unit 320A >>
Next, the configuration of the above-described first output unit 320A will be described.

  As shown in FIG. 4, the first output section 320A includes a movement amount detection section 321A and a specification section 322A. In addition, the first output unit 320A includes a calibration unit 323A.

  The movement amount detection unit 321A receives the captured image data sent from the acquisition unit 310A. Then, the movement amount detection unit 321A detects the pixel movement amount by a so-called displacement amount search method based on the captured image data. The pixel movement amount thus detected is sent to the calibration unit 323A and the second output unit 330.

  The details of the operation of the movement amount detection unit 321A will be described later.

  The specifying unit 322A receives the captured image data sent from the acquisition unit 310A. Then, the identifying unit 322A identifies the white line region in the image obtained from the captured image data. Information on the white line area thus specified (hereinafter referred to as “white line area information”) is sent to the calibration unit 323A.

  The details of the operation of the identifying unit 322A will be described later.

  The calibration unit 323A receives the pixel movement amount sent from the movement amount detection unit 321A and the white line area information sent from the identification unit 322A. Then, the calibration unit 323A determines, based on the acceleration information, the steering angle information, and the tilt information sent from the ECU 230, that the pixel movement amount and the pixel movement amount when the moving body MV is traveling straight on a flat road surface at a constant speed. In addition to the white line region information, the pixel distance is calibrated based on the fact that the length of the stop line drawn on the road in the traveling direction is approximately 45 [cm]. The pixel distance thus calibrated is sent to the second output section 330.

  The details of the operation of the calibration unit 323A will be described later.

<Operation>
Next, the operation of the detection apparatus 300A calibrated as described above will be described focusing on the processing performed by each element of the detection apparatus 300A.

  It is assumed that the imaging unit 210 has already started its operation and sequentially sends the captured image data of the road surface image captured at the cycle time TP to the detection device 300A. Further, it is assumed that the navigation device 220 has already started its operation, and when the moving body MV is within a predetermined distance from the intersection existing in the traveling direction, the fact is sent to the detection device 300A. It is also assumed that the ECU 230 has already started its operation and has sent the acceleration information, the steering angle information, and the tilt information to the detection device 300A (see FIG. 4).

  Further, the detection device 300A has already calibrated the pixel distance a plurality of times, and the calibration unit 323A holds the calibration history including the latest predetermined number of times of calibration and a temporary pixel distance described later. Be present. Further, it is assumed that the second output unit 330 holds the latest calibrated pixel distance. Here, in the period until the first calibration of the pixel distance is performed, the second output unit 330 holds the average pixel distance.

  Note that the "predetermined number of times" refers to experiments, simulations, experience, etc. from the viewpoint of performing averaging processing to suppress the influence on the calibration result of the pixel distance such as the variation in the length of the stop line in the traveling direction. It is predetermined based on Further, the “average pixel distance” is predetermined corresponding to the moving body MV based on experiments, simulations, experiences, and the like.

<Image data acquisition processing>
In the detection device 300A, the acquisition unit 310A receives the captured image data sent from the imaging unit 210. Then, the acquisition unit 310A sends the captured image data to the movement amount detection unit 321A and the identification unit 322A of the first output unit 320A (see FIG. 4).

<< Pixel movement amount detection processing >>
Next, a process of detecting the pixel movement amount by the movement amount detection unit 321A will be described.

  Upon receiving the captured image data sent from the acquisition unit 310A, the movement amount detection unit 321A causes the position of the common characteristic region in the current image obtained from the captured image data of this time and the previous image obtained from the previous captured image data. The displacement amount of is detected as the pixel movement amount. Then, the movement amount detection unit 321A sends the detected pixel movement amount to the calibration unit 323A and the second output unit 330 (see FIG. 4).

  If there are a plurality of common characteristic regions, the movement amount detection unit 321A adopts the average value of the displacement amounts of the plurality of characteristic regions as the pixel movement amount.

FIG. 5 shows an example of movement amount detection by the movement amount detection unit 321A. In the example of FIG. 5, the moving body MV is traveling straight along the X direction, the X direction position at time T _j is X _j , and the X direction position at time T _{j + 1} (= T _j + TP) is. This is an example of the case of X _{j + 1} . In addition, the example of FIG. 5, a common characteristic region in the captured image in the captured image and the time T _{j + 1} at time T _j is two and is an example of a characteristic region A and the characteristic region B are shown. In the example of FIG. 5, since the displacement amount of the characteristic region A is “ΔX _jA ” and the displacement amount of the characteristic region B is “ΔX _jB ”, the pixel movement amount ΔX _j is _calculated by the following equation (6). Is calculated.
ΔX _j = (ΔX _jA + ΔX _jB ) / 2 (6)

<< Output process of moving body speed v >>
Next, the output process of the moving body speed v by the second output unit 330 will be described.

  Upon receiving the pixel movement amount sent from the movement amount detection unit 321A, the second output unit 330 may determine the pixel movement amount, the held pixel distance (ie, the latest calibrated pixel distance), and the cycle time TP. Then, the moving body speed v is calculated. Then, the second output unit 330 outputs the calculated moving body speed v to the ECU 230.

Here, when the pixel movement amount is “PN” and the pixel distance is “PD”, the second output unit 330 calculates the moving body velocity v by the following equation (7).
v = PN PD / TP (7)

<< White line area specification processing >>
Next, the white line area specifying process by the specifying unit 322A will be described.

  The white line area specifying process is performed for the pixel distance calibration by the calibration unit 323A described later. An environment for calibrating the pixel distance (hereinafter, referred to as “calibration environment”) is shown in FIG. As shown in FIG. 6, in the calibration of the pixel distance, the mobile body MV is a white line on the road surface having a length W (≈45 [cm]) in the traveling direction when the mobile body MV travels at a constant speed on a flat road surface. This is performed when the stop line SPL is crossed at right angles to the longitudinal direction of the stop line SPL.

  In the following description, the stop line SPL used for the calibration of the new pixel distance is assumed to be a white line region from the traveling direction position (that is, the X direction position) XR to the traveling direction position XP. ..

  Now, when receiving the captured image data sent from the acquisition unit 310A, the identification unit 322A identifies the white line region in the image obtained from the captured image data based on the brightness of each pixel in the image, and determines the identification result. , And sends it to the calibration unit 323A as white line area information (see FIG. 4). The white line area identification result includes two types, a front partial white line shown in FIG. 7A and a rear partial white line shown in FIG. 7B.

  When the identification result is a front partial white line, the identification unit 322A identifies the number of pixels a in the X-direction length of the white line area in the image (hereinafter, also referred to as “front length a”). Further, when the specification result is a partial back white line, the specifying unit 322A specifies the number of pixels b in the length of the non-white line region in the image in the X direction (hereinafter, also referred to as “front length b”).

  Then, in the case of a partial front white line, the specifying unit 322A sends the white line area information [white line flag: ON, front length: a] shown in FIG. 7A to the calibration unit 323A (see FIG. 4). In the case of a partial white line behind, the specifying unit 322A sends the white line area information [white line flag: ON, front length: b] shown in FIG. 7B to the calibration unit 323A (see FIG. 4).

]
<< Pixel distance calibration process >>
Next, the calibration process by the calibration unit 323A will be described.

  In the calibration process, the calibration unit 323A determines whether or not the navigation device 220 has received from the navigation device 220 that the moving body MV is within a predetermined distance from the intersection existing in the traveling direction. If the result of this intersection determination is affirmative, the calibration unit 323A can determine that the moving body MV is traveling straight at a constant speed on the flat road surface based on the acceleration information, the steering angle information, and the tilt information sent from the ECU 230. Whether or not to go straight at a constant speed is determined.

  If the result of the straight-ahead straight velocity determination is affirmative, the calibration unit 323A determines whether or not the white line region information sent from the identification unit 322A has become a "front partial white line". If the result of the white line start determination is affirmative, the white line area information sent from the identifying unit 322A is “the condition that the result of the constant speed straight ahead determination is positive” is maintained. The pixel movement amount sent from the movement amount detection unit 321A is collected until the result of the white line end determination of whether or not it becomes "a rear partial white line" is affirmative.

Then, when the result of the white line end determination is affirmative, the calibration unit 323A causes the front length a included in the white line region information at the time when the “front partial white line” is set, the collected pixel movement amount ΔX ₁ , ..., The provisional pixel distance PT [cm] is calculated by the following equation (8) based on ΔX _M and the front length b included in the white line area information at the time when the “rear partial white line” is reached.
PT = 45 / (a + ΔX ₁ + ... + ΔX _M −b) (8)

  Next, the calibration unit 323A calculates the weighted average of the calculated provisional pixel distance of this time and the provisional pixel distance that is internally stored and is calculated at a predetermined number of times in the past, and is calibrated this time. Calculate pixel distance. After that, the calibration unit 323A internally retains the provisional pixel distance of this time in place of the oldest one of the provisional pixel distances retained therein.

  In the weighted average, the weight is decreased as the elapsed time to the present is longer.

  The calibration unit 323A sends the calculated pixel distance to the second output unit 330. As a result, the second output unit 330 comes to calculate the moving body speed v using the new pixel distance.

Note that FIG. 8 shows an example in which a captured image of the front partial white line is obtained at time T ₁ and a captured image of the rear partial white line is obtained at time T ₅ . In the example of FIG. 8, the provisional pixel distance PT [cm] is calculated by the following equation (9).
PT = 45 / (a + ΔX ₁ + ΔX ₂ + ΔX ₃ + ΔX ₄ −b) (9)

  As described above, in the first embodiment, the movement amount detecting unit is based on the image captured by the image capturing unit 210 regarding the stop line which is the road surface characteristic portion having the known length along the traveling direction of the moving body MV. 321A detects the pixel movement amount. Further, the identifying unit 322A identifies the white line area in the captured image. Then, the calibration unit 323A flattens the moving body MV based on the known length of the stop line along the traveling direction of the moving body MV in addition to the pixel movement amount and the result of identifying the white line region in the captured image. When it can be determined that the vehicle is traveling straight at a constant speed on the road surface, the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging unit 210 and the road surface, is calibrated.

  Therefore, according to the first embodiment, even if the distance between the road surface and the image capturing unit 210 changes due to a change in the weight of the moving body or a change in the air pressure of the wheels, the distance on the road surface corresponding to one pixel is changed. It can be calibrated accordingly.

  In addition, in the first embodiment, the second output unit 330 calculates the moving body speed based on the calibration result and the pixel movement amount. Therefore, it is possible to accurately output the moving body speed as the traveling information.

  Further, in the first embodiment, a stop line is adopted as the road surface characteristic portion. Therefore, by checking the brightness of each pixel of the captured image, the stop line area (that is, the white line area) can be easily specified.

  In addition, in the first embodiment, information indicating that an intersection is approached is acquired, and it is presumed that the vehicle will pass the stop line. Therefore, efficient calibration can be performed.

Further, in the first embodiment, a final calibration result is obtained by calculating a weighted average of the provisional pixel distance of this time and the provisional pixel distance calculated during a predetermined number of times of calibration in the past.
This is because there is a case where the estimated length is deviated from 45 [cm] due to a lack of a stop line or a protruding coating. Therefore, it is possible to perform the calibration while suppressing the influence of the pixel distance on the calibration result such as the variation of the length of the stop line in the traveling direction.

  Further, in the first embodiment, the calibration is performed when the vehicle is traveling straight ahead, traveling at a constant speed, and when the road surface is not inclined. This is because the passage of the stop line becomes slanted when there is a lateral movement, and calibration is performed based on a length of 45 [cm] or more, and when the acceleration or deceleration is large or the road inclination is large. In some cases, expansion and contraction of the suspension and the like occur, and this is because the optical magnification at the time of imaging is being changed. Therefore, it is possible to perform accurate pixel distance calibration.

[Second Embodiment]
Next, a second embodiment of the present invention will be described mainly with reference to FIGS. 9 to 13.

<Structure>
FIG. 9 is a block diagram showing the configuration of the detection device 300B according to the second embodiment. As shown in FIG. 9, the detection device 300B is connected to the imaging units 210 _F and 210 _R and the ECU 230. The detection device 300B, the imaging units 210 _F and 210 _R, and the ECU 230 are mounted on the moving body MV.

Each of the image capturing units 210 _F and 210 _R described above is configured similarly to the image capturing unit 210 described above. Then, the imaging unit 210 _F is arranged on the front side of the moving body MV, and the imaging unit 210 _R is arranged on the rear side of the moving body MV separated by a distance D from the imaging unit 210 _F (see FIG. 10). The image capturing units 210 _F and 210 _R capture images at the same timing.

The data of the image captured by the image capturing unit 210 _F (hereinafter, referred to as “front side captured image”) is sent to the detection device 300B. Further, the data of the image captured by the image capturing unit 210 _R (hereinafter, referred to as “rear side captured image”) is also sent to the detection device 300B.

  Note that, similarly to the case of the above-described first embodiment, the ECU 230 sends the acceleration information, the steering angle information, and the tilt information to the detection device 300B. In addition, the ECU 230 further acquires the moving body speed sent from the detection device 300B, and uses the acquired moving body speed for traveling control of the moving body MV.

  The detection device 300B includes an acquisition unit 310B and a first output unit 320B, as shown in FIG. 9. Further, the detection device 300B includes a second output section 330.

Additional acquisition unit 310B, the data of the front captured image sent from the imaging unit 210 _F, and receives the data of the side captured image after being sent from the imaging unit 210 _R. Then, the acquisition unit 310B sends the data of the front side captured image and the data of the rear side captured image to the first output unit 320B.

  The first output unit 320B described above receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. Then, the first output unit 320B detects the pixel movement amount and calibrates the pixel distance when the same calibration condition as in the case of the first embodiment described above is satisfied. The detected pixel movement amount and the calibrated pixel distance are output to the second output unit 330. Details of the configuration of the first output unit 320B having such a function will be described later.

  It should be noted that the second output section 330 receives the latest calibrated pixel distance and cycle time each time it receives the pixel movement amount sent from the first output section 320B, as in the case of the first embodiment described above. Based on TP, the moving body speed is calculated as the traveling information of the moving body MV. Then, the second output unit 330 outputs the calculated moving body speed of the moving body MV to the ECU 230.

<< Configuration of the first output unit 320B >>
Next, the configuration of the above-described first output unit 320B will be described.

  As shown in FIG. 9, the first output unit 320B includes a movement amount detection unit 321B and a specifying unit 322B. The first output unit 320B also includes a calibration unit 323B.

  The movement amount detection unit 321B described above receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. Then, the movement amount detection unit 321B detects the pixel movement amount by a so-called displacement amount search method based on the data of the front side captured image and the data of the rear side captured image. The pixel movement amount thus detected is sent to the calibration unit 323B and the second output unit 330.

  The details of the operation of the movement amount detection unit 321B will be described later.

  The specifying unit 322B receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. Then, the identifying unit 322B identifies the white line region in the image obtained from the data of the front captured image and the data of the rear captured image. Information on the white line area thus specified (hereinafter referred to as “white line area information”) is sent to the calibration unit 323B.

  The details of the operation of the identifying unit 322B will be described later.

The calibration unit 323B receives the pixel movement amount sent from the movement amount detection unit 321B and the white line area information sent from the identification unit 322B. Then, the calibration unit 323B determines, based on the acceleration information, the steering angle information, and the inclination information sent from the ECU 230, that the pixel movement amount and the pixel movement amount when the moving body MV is traveling straight on a flat road surface at a constant speed. In addition to the white line area information, the pixel distance is calibrated based on the distance D between the image capturing unit 210 _F and the image capturing unit 210 _R described above. The pixel distance thus calibrated is sent to the second output section 330.

  The details of the operation of the calibration unit 323B will be described later.

<Operation>
Next, the operation of the detection device 300B calibrated as described above will be described focusing on the processing performed by each element of the detection device 300B.

Note that the imaging units 210 _F and 210 _R have already started to operate, and sequentially send the front-side captured image data and the rear-side captured image data of the road surface image captured at the cycle time TP to the detection device 300B. Be present. It is also assumed that the ECU 230 has already started its operation and has sent the acceleration information, the steering angle information, and the tilt information to the detection device 300B (see FIG. 9).

  Furthermore, the detection device 300B has already calibrated the pixel distance a plurality of times, and the calibration unit 323B holds the calibration history including the latest predetermined number of times of calibration and a temporary pixel distance described later. In addition, it is assumed that the second output section 330 holds the latest calibrated pixel distance. Here, in the period until the first calibration of the pixel distance is performed, as in the case of the above-described first embodiment, the second output unit 330 holds the average pixel distance. ing.

<Image data acquisition processing>
In the detection device 300B, the acquisition unit 310B receives the data of the front side captured image and the data of the rear side captured image sent from the image capturing units 210 _F and 210 _R. Then, the acquisition unit 310B sends the data of the front side captured image and the data of the rear side captured image to the movement amount detection unit 321B and the identification unit 322B of the first output unit 320B (see FIG. 9).

<< Pixel movement amount detection processing >>
Next, a process of detecting the pixel movement amount by the movement amount detection unit 321B will be described.

  Upon receiving the data of the front side captured image sent from the acquisition unit 310B, the movement amount detection unit 321B obtains the data of the front side captured image this time in the same manner as the detection of the pixel movement amount of the first embodiment described above. The displacement amount of the position of the common characteristic region in the previous front side captured image obtained from the data of the front side captured image this time and the previous front side captured image is detected as the front side pixel movement amount. Further, when receiving the data of the rear side captured image sent from the acquisition unit 310B, the movement amount detection unit 321B determines from the data of the rear side captured image of this time in the same manner as the detection of the pixel movement amount of the first embodiment. The displacement amount of the position of the common characteristic region in the obtained current rear side captured image and the previous rear side captured image obtained from the data of the previous rear side captured image is detected as the rear side pixel movement amount.

  If there are a plurality of characteristic regions common to the front side captured image this time and the previous front side captured image, the movement amount detection unit 321B determines the average value of the displacement amounts of the plurality of characteristic regions as the front side pixel movement amount. adopt. In addition, when there are a plurality of characteristic regions common to the rear-side captured image this time and the previous rear-side captured image, the movement amount detection unit 321B calculates the average value of the displacement amounts of the plurality of characteristic regions as the rear-side pixels. Used as the amount of movement.

  Subsequently, the movement amount detection unit 321B calculates the average of the front side pixel movement amount and the rear side pixel movement amount as the pixel movement amount. Then, the movement amount detection unit 321B sends the calculated pixel movement amount to the calibration unit 323B and the second output unit 330 (see FIG. 9).

<< Output process of moving body speed v >>
Next, the output process of the moving body speed v by the second output unit 330 will be described.

  Upon receiving the pixel movement amount sent from the movement amount detection unit 321B, the second output unit 330, similarly to the case of the first embodiment, the pixel movement amount, the held pixel distance (that is, the latest calibration). The moving body speed v is calculated based on the calculated pixel distance) and the cycle time TP. Then, the second output unit 330 outputs the calculated moving body speed v to the ECU 230.

<< White line area specification processing >>
Next, the white line area specifying process by the specifying unit 322B will be described.

  The white line area specifying process is performed for calibration of the pixel distance by the calibration unit 323B described later. Such a pixel distance calibration environment is shown in FIG. As shown in FIG. 11, the pixel distance is calibrated when the moving body MV crosses a white line region LCP on a flat road surface, such as a pedestrian crossing sign, in a straight line at a constant speed.

  In the following description, the white line area LCP used for calibration of a new pixel distance is a white line area from the traveling direction position (that is, the X direction position) XR to the traveling direction position XP. ..

Now, when receiving the data of the front side captured image and the data of the rear side captured image sent from the acquisition section 310B, the identification section 322B sets the white line areas in the front side captured image and the rear side captured image to the brightness of each pixel in the image. Based on this, the specification result is sent to the calibration unit 323B as white line area information (see FIG. 9). The result of specifying the white line area includes two types of “first rear partial white line” shown in FIG. 12A and “second rear partial white line” shown in FIG. 12B. .. Here, the “first rear partial white line” is a state of a rear partial white line in the front side captured image obtained by the image capturing by the image capturing unit 210 _F installed on the front side. Further, the “second rear partial white line” is a state of a rear partial white line in the rear side captured image obtained by the image pickup by the image pickup unit 210 _R installed on the rear side.

  Note that in the second embodiment, the specifying unit 322B is conditioned on the condition that the white line until the "first rear partial white line" can be clearly specified when passing through the white line in the front side captured image is continued. , White line area information is generated. Therefore, if the white line cannot be clearly identified due to tire marks or the like, the identifying unit 322B does not generate the white line area information. That is, in the second embodiment, the pixel distance is not calibrated when the white line cannot be clearly identified due to tire marks or the like.

  Further, in the second embodiment, the white line region is specified by focusing on the first rear partial white line in the front side captured image and the second rear partial white line in the rear side captured image. This is also because it is easy to make an error in the measurement when there are multiple white lines in. By carrying out such a device, it is possible to avoid wasteful calibration, so that waste can be eliminated.

  When the identification result is the first rear partial white line in the front side captured image, the identifying unit 322B determines the length a in the X direction of the non-white line region in the image (hereinafter, also referred to as “front length a”). Identify. Further, when the identification result is the second rear partial white line in the back side captured image, the identifying unit 322B has a length b in the X direction of the non-white line region in the image (hereinafter, also referred to as “front length b”). Specify.

  Then, in the case of the first rear partial white line in the front side captured image, the specifying unit 322B sets the white line area information [first rear partial white line flag: ON, front length: a] shown in FIG. It is sent to the calibration unit 323B (see FIG. 9). Further, in the case of the second rear partial white line, the specifying unit 322B sends the white line area information [second rear partial white line flag: ON, front length: b] shown in FIG. 12B to the calibration unit 323B. Send (see FIG. 9).

<< Pixel distance calibration process >>
Next, the calibration process by the calibration unit 323B will be described.

  In the calibration process, the calibration unit 323B determines whether the moving body MV is traveling straight at a constant speed on a flat road surface based on the acceleration information, the steering angle information, and the tilt information sent from the ECU 230. Judgment of straight ahead.

If the result of the straight-ahead straight velocity determination is affirmative, the calibration unit 323B determines whether or not the white line area information in the front side captured image sent from the specifying unit 322B is “first rear partial white line”. White line end judgment is performed. When the result of this front side white line end determination is affirmative, the calibration unit 323B is sent from the specifying unit 322B on condition that the result of the constant velocity straight ahead determination on a flat road surface is maintained affirmative. The amount of pixel movement sent from the movement amount detection unit 321B until the result of the rear white line end determination whether the white line region information in the rear side captured image is “second rear partial white line” is affirmative. ΔX ₁ , ..., ΔX _N are collected (see FIG. 13).

Then, when the result of the rear white line end determination is affirmative, the calibration unit 323B causes the front length a included in the white line region information at the time when the white line region information in the front side captured image becomes the “first rear partial white line”. , The collected pixel movement amounts ΔX ₁ , ..., ΔX _N , and the front length b included in the white line area information at the time when the white line area information in the back side captured image becomes the “second rear partial white line”, and , The provisional pixel distance PT [cm] is calculated by the following equation (10) based on the distance D between the imaging unit 210 _F and the imaging unit 210 _R.
PT = D / (a + ΔX ₁ + ... + ΔX _N −b) (10)

  Next, the calibration unit 323B calculates the weighted average of the calculated provisional pixel distance of this time and the provisional pixel distance that is internally stored and is calculated a predetermined number of times in the past, and is calibrated this time. Calculate pixel distance. After that, the calibration unit 323B internally retains the provisional pixel distance of this time in place of the oldest provisional pixel distance retained up to that point.

  In the weighted average, the weight is reduced as the elapsed time to the present is longer, as in the case of the first embodiment.

  The calibration unit 323B sends the calculated pixel distance to the second output unit 330. As a result, the second output unit 330 comes to calculate the moving body speed v using the new pixel distance.

As described above, in the second embodiment, the imaging of the white line which is the road surface feature by the imaging units 210 _F and 210 _R arranged on the moving body MV at the known distance D along the traveling direction of the moving body MV. The movement amount detector 321B detects the pixel movement amount based on the image. Further, the identifying unit 322B identifies the white line area in the captured image. Then, the calibration unit 323B can determine that the moving body MV is traveling straight at a constant speed on the flat road surface based on the known distance D in addition to the pixel movement amount and the result of specifying the white line region in the captured image. In this case, the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging units 210 _F and 210 _R and the road surface, is calibrated.

  Therefore, according to the second embodiment, regardless of whether the length of the white line on the road surface along the traveling direction of the moving body is known or unknown, the change in the weight of the moving body and the air pressure of the wheels are reduced. Even if the distance between the road surface and the image capturing unit changes due to the change of, the distance on the road surface corresponding to one pixel can be calibrated according to the change.

  In the second embodiment, the second output unit calculates the moving body speed based on the calibration result and the pixel movement amount. Therefore, it is possible to accurately output the moving body speed as the traveling information.

  Further, in the second embodiment, a white line is adopted as the road surface characteristic portion. Therefore, the white line region can be easily specified by checking the brightness of each pixel of the captured image.

  Further, in the second embodiment, as in the case of the first embodiment, a weighted average of the provisional pixel distances of this time and the provisional pixel distances calculated at a predetermined number of times of calibration in the past is calculated, and the final average is calculated. To obtain the correct calibration result. Therefore, it is possible to perform the calibration while suppressing the influence of the pixel distance on the calibration result such as the variation of the length of the white line in the traveling direction.

  In addition, in the second embodiment, as in the case of the first embodiment, the calibration is performed when the vehicle is traveling straight ahead, traveling at a constant speed, and when there is no road surface inclination. Therefore, it is possible to perform accurate pixel distance calibration.

[Modification of Embodiment]
The present invention is not limited to the first and second embodiments described above, and various modifications are possible.

  For example, in the above-described first embodiment, the stop line before the intersection is adopted as the road surface characteristic portion, but if the road marking has a known length along the traveling direction of the moving body, other The types of road markings may be adopted as the road surface characteristic portion.

  Further, in the above-described second embodiment, the white line such as a pedestrian crossing is adopted as the road surface characteristic part, but road markings other than the white line may be adopted as the road surface characteristic part.

  In the second embodiment, the average of the front pixel movement amount obtained from the time change of the front image and the rear pixel movement amount obtained from the time change of the rear image is detected as the pixel movement amount. did. On the other hand, either the front side pixel movement amount or the rear side pixel movement amount may be detected as the pixel movement amount.

  In the second embodiment described above, based on the change in the captured image from the time when the front captured image becomes the first rear partial white line to the time when the rear captured image becomes the second rear partial white line, Pixel distance was calibrated. On the other hand, the pixel distance is calibrated based on the change in the captured image from the time when the front captured image becomes the first front partial white line to the time when the rear captured image becomes the second front partial white line. You can

  Further, in the first and second embodiments described above, the pixel movement amount is detected by the so-called displacement amount search method, but the pixel movement amount may be detected by the image correlation method or the spatial filter method.

  In addition, in the above-described first and second embodiments, the pixel distance, which is the distance on the road surface corresponding to one pixel, is calibrated as the distance-related information related to the distance between the imaging unit and the road surface. On the other hand, the image element distance, which is the distance on the road surface corresponding to the length in the traveling direction of a predetermined image element that is a set of a plurality of pixels, may be calibrated.

  Further, in the above-described first and second embodiments, the second output unit outputs the moving body speed (running speed) as the traveling information, but instead of the moving body speed or in addition to the moving body speed. Then, the traveling distance may be output.

  In the first and second embodiments described above, the pixel distance is calibrated by calculating the weighted average of the provisional pixel distance of this time and the predetermined number of past provisional pixel distances. On the other hand, the provisional pixel distance of this time may be used as it is as the calibrated pixel distance.

  Further, in the above-described first and second embodiments, it is assumed that the imaging unit is provided outside the detection device, but the detection device may include the imaging unit.

  In addition, in the above-described first and second embodiments, the detection device is mounted on the moving body. On the other hand, the detection device may be installed outside the moving body and connected to the vehicle-mounted image pickup unit or ECU via a wireless communication network.

  The detection devices of the above-described first and second embodiments are configured as a computer as an arithmetic unit having a central processing unit (CPU) and the like, and a program prepared in advance is executed by the computer. By doing so, some or all of the functions of the detection devices of the first and second embodiments described above may be executed. This program is recorded on a computer-readable recording medium such as a hard disk, a CD-ROM, or a DVD, and is loaded from the recording medium by the computer and executed. Further, this program may be acquired in a form recorded in a portable recording medium such as a CD-ROM or a DVD, or may be acquired in a distributed form via a network such as the Internet. Good.

  Next, an embodiment of the present invention will be described mainly with reference to FIGS. In the following description, the same or equivalent elements, including those in the above-described embodiment, will be denoted by the same reference symbols, and redundant description will be omitted as much as possible.

[First embodiment]
First, a first embodiment of the present invention will be described mainly with reference to FIGS.

<Structure>
FIG. 14 schematically shows the configuration of the detection device 100A according to the first embodiment. The detection device 100A is an aspect of the detection device 300A according to the above-described first embodiment.

  As shown in FIG. 14, the detection device 100A is connected to the imaging unit 210, the navigation device 220, and the ECU 230. The detection device 100A, the imaging unit 210, the navigation device 220, and the ECU 230 are mounted on the vehicle CR.

  The detection device 100A includes a control unit 110A. In addition, the detection device 100A includes a storage unit 120.

  The control unit 110A is configured to include a central processing unit (CPU) as a calculation means. By executing the program, the control unit 110A functions as the detection device 300A in the above-described first embodiment, that is, functions as the acquisition unit 310A, the first output unit 320A, and the second output unit 330. It has become.

  The program executed by the control unit 110A is stored in the storage unit 120, loaded from the recording unit and executed. This program may be acquired in a form recorded on a portable recording medium such as a CD-ROM or a DVD, or may be acquired in a form distributed via a network such as the Internet. ..

  The processing executed by the control unit 110A will be described later.

  The storage unit 120 stores various information data used by the control unit 110A. Such information data includes the program executed by the control unit 110A, the latest pixel movement amount, the calibration history including the latest pixel distance, and the like. The control unit 110A can access the storage unit 120.

<Operation>
Next, the detection operation performed by the detection device 100A configured as described above will be described focusing on the processing performed by the control unit 110A.

  It is assumed that the image capturing unit 210 has already started its operation and sequentially transmits the captured image data of the road surface image captured at the cycle time TP to the detection device 100A. Further, it is assumed that the navigation device 220 has already started its operation, and when the vehicle CR is within a predetermined distance from an intersection existing in the traveling direction, the fact is sent to the detection device 100A. It is also assumed that the ECU 230 has already started its operation and has sent the acceleration information, the steering angle information, and the tilt information to the detection device 100A (see FIG. 14).

  Further, in the detection apparatus 100A, the pixel distance has already been calibrated a plurality of times, and the storage unit 120 stores the calibration times for the latest predetermined times, the characteristics of the characteristic region extracted from the previous captured image, and It is assumed that the position in the image, a calibration history including a temporary pixel distance described later, and the like are stored. Here, the average pixel distance is stored in the storage unit 120 until the first calibration of the pixel distance is performed.

<< Pixel movement amount detection processing >>
First, the process of detecting the pixel movement amount by the control unit 110A will be described.

  In the process of detecting the pixel movement amount, as shown in FIG. 15, first, in step S11, the control unit 110A determines whether or not new captured image data has been received. When the result of the determination in step S11 is negative (step S11: N), the process of step S11 is repeated.

  When new captured image data is received and the result of the determination in step S11 is affirmative (step S11: Y), the process proceeds to step S12. In step S12, the control unit 110A extracts the characteristic region in the captured image (current captured image) obtained from the new captured image data, and specifies the characteristic of the characteristic region and the position in the image.

  Next, in step S13, the control unit 110A detects the pixel movement amount. In detecting the pixel movement amount, the control unit 110A refers to the characteristic of the characteristic region and the position in the image extracted from the previous captured image in the storage unit 120, and refers to the current captured image and the previous captured image. The amount of displacement of the common feature region in the image is detected as a new pixel movement amount.

  When there are a plurality of common characteristic regions, the control unit 110A adopts the average value of the displacement amounts of the plurality of characteristic regions as the pixel movement amount.

  Then, the control unit 110A registers, in the storage unit 120, the characteristic and the in-image position of the characteristic region in the current captured image as the characteristic and the in-image position of the characteristic region extracted from the previously captured image. Further, the control unit 110A registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

  When the process of step S13 is completed in this way, the process returns to step S11. After that, steps S11 to S13 are repeated, and the pixel movement amount is detected every time data of a new captured image is received.

<< Processing to output vehicle speed v >>
Next, the output processing of the vehicle body speed v by the control unit 110A will be described.

  In the output processing of the vehicle body speed v, as shown in FIG. 16, first, in step S21, the control unit 110A determines whether or not a new pixel movement amount is detected. If the result of the determination in step S21 is negative (step S21: N), the process of step S21 is repeated.

  When a new pixel movement amount is detected and the result of the determination in step S21 is affirmative (step S21: Y), the process proceeds to step S22. In step S22, the vehicle body speed v is calculated by the above-described equation (7) based on the latest pixel movement amount (PN) and the latest pixel distance (PL) registered in the storage unit 120. Then, control unit 110A outputs the calculated vehicle body speed v to ECU 230.

  When the process of step S22 ends in this way, the process returns to step S21. Thereafter, the processes of steps S21 and S22 are repeated, and a new vehicle body speed v is calculated each time a new pixel movement amount is detected, that is, each time new imaging data is received, and the calculated vehicle body speed v Is output to the ECU 230.

<< White line area specification processing >>
Next, the white line area specifying process by the control unit 110A will be described. The white line area specifying process is executed in parallel with the pixel movement amount detecting process and the vehicle body speed v output process described above.

  The white line area specifying process is performed for the purpose of calibrating the pixel distance, which will be described later. The calibration environment for this pixel distance calibration is the same as that shown in FIG. 6 described above.

  In the white line area specifying process, as shown in FIG. 17, first, in step S31, the control unit 110A determines whether or not new captured image data has been received. When the result of the determination in step S31 is negative (step S31: N), the process of step S31 is repeated.

  When new captured image data is received and the result of the determination in step S31 is affirmative (step S31: Y), the process proceeds to step S32. In this step S32, the control unit 110A specifies the white line area in the captured image obtained from the new captured image data based on the brightness of each pixel in the captured image.

  Next, in step S33, the control unit 110A determines whether or not the identification result of the white line area is “a front partial white line (see FIG. 7A)”. If the result of the determination in step S33 is negative (step S33: N), the process returns to step S31. Then, the processes of steps S31 to S33 are repeated until the result of the determination of step S33 becomes affirmative.

  If the white line area identification result is “front partial white line” and the determination result in step S33 is affirmative (step S33: Y), the process proceeds to step S34. In step S34, the control unit 110A calculates the front length a (see FIG. 7A). Then, the process proceeds to step S35.

  Next, in step S35, the control unit 110A determines whether or not new captured image data has been received. If the result of the determination in step S35 is negative (step S35: N), the process of step S35 is repeated.

  When new captured image data is received and the result of the determination in step S35 is affirmative (step S35: Y), the process proceeds to step S36. In step S36, the control unit 110A identifies the white line area in the captured image obtained from the new captured image data based on the brightness of each pixel in the captured image.

  Next, in step S37, the control unit 110A determines whether or not the identification result of the white line area is “rear partial white line (see FIG. 7B)”. If the result of the determination in step S37 is negative (step S37: N), the process returns to step S35. Then, the processes of steps S35 to S37 are repeated until the result of the determination of step S37 becomes affirmative.

  If the white line area identification result is “rear partial white line” and the determination result in step S37 is affirmative (step S37: Y), the process proceeds to step S38. In step S38, the control unit 110A calculates the front length b (see FIG. 7B). Then, the process returns to step S31.

  After that, steps S31 to S38 are repeated. As a result, the white line area is specified every time the vehicle CR crosses the white line.

<< Monitoring of calibration conditions >>
Next, the process of monitoring the calibration conditions by the control unit 110A will be described. The calibration condition monitoring process is executed in parallel with the above-described pixel movement amount detection process, vehicle body speed v output process, and white line region identification process.

  In the calibration condition monitoring process, as shown in FIG. 18, first, in step S41, it is determined whether or not the vehicle CR is traveling straight ahead based on the latest steering angle information sent from the ECU 230. When the result of the determination in step S41 is negative (step S41: N), the process proceeds to step S44 described later.

  If the result of the determination in step S41 is affirmative (step S41: Y), the process proceeds to step S42. In this step S42, control unit 110A determines whether or not vehicle CR is accelerating or decelerating based on the latest acceleration information sent from ECU 230. When the result of the determination in step S42 is affirmative (step S42: Y), the process proceeds to step S44.

  When the result of the determination in step S42 is negative (step S42: N), the process proceeds to step S43. In step S43, control unit 110A determines whether or not there is a vehicle body inclination of vehicle CR based on the latest inclination information sent from ECU 230.

  If the result of the determination in step S43 is affirmative (step S43: Y), the process proceeds to step S44. In step S44, the control unit 110A sets the calibration condition flag to "OFF". Then, the process returns to step S41.

  On the other hand, if the result of the determination in step S43 is negative (step S43: N), the process proceeds to step S45. In step S45, the control unit 110A sets the calibration condition flag to "ON". Then, the process returns to step S41.

  After that, the processes of steps S41 to S44 are repeated. As a result, when it is determined that the calibration condition that the vehicle CR travels straight on a flat road surface at a constant speed is satisfied, the calibration condition flag is set to "ON". On the other hand, when it is determined that the calibration condition is not satisfied, the calibration condition flag is “OFF”.

<< Pixel distance calibration process >>
Next, the pixel distance calibration process by the control unit 110A will be described. The pixel distance calibration process is performed in parallel with the above-described pixel movement amount detection process, vehicle body speed v output process, white line region identification process, and calibration condition monitor process.

  In the pixel distance calibration process, as shown in FIG. 19, first, in step S50, it is determined whether or not the control unit 110A receives from the navigation device 220 that it is within a predetermined distance from an intersection existing in the traveling direction. By determining, it is determined whether or not the intersection is approached. If the result of the determination in step S50 is negative (step S50: N), the process of step S50 is repeated.

  When the vehicle approaches the intersection and the result of the determination in step S50 is affirmative (step S50: Y), the process proceeds to step S51. In step S51, the control unit 110A determines whether the calibration condition flag is "ON" to determine whether the vehicle CR is traveling straight on a flat road surface at a constant speed.

  If the result of the determination in step S51 is negative (step S51: N), the process returns to step S50. Then, the processes of steps S50 and S51 are repeated.

  When the result of the determination in step S51 is affirmative (step S51: Y), the process proceeds to step S52. In step S52, the control unit 110A determines that the front part white line is identified by the white line area specifying process described above, and determines whether the front length a is obtained.

  When the result of the determination in step S51 is negative (step S52: N), the process returns to step S51. Then, the processing in steps S51 and S52 is repeated, provided that the result of the determination in step S51 is not negative.

  When the result of the determination in step S52 is affirmative (step S52: Y), the process proceeds to step S53. In this step S53, the distance parameter L is set to the front length a.

  Next, in step S54, the control unit 110A determines whether or not the calibration condition flag is "ON", thereby determining whether or not the vehicle CR is traveling straight at a constant speed on a flat road surface. When the result of the determination in step S54 is negative (step S54: N), the process returns to step S50.

When the result of the determination in step S54 is affirmative (step S54: Y), the process proceeds to step S55. In step S55, the control unit 110A determines whether or not a new pixel movement amount (ΔX _j ) is detected by the above-described pixel movement amount detection processing.

  If the result of the determination in step S55 is negative (step S55: N), the process returns to step S54. Then, the processing in steps S54 and S55 is repeated, provided that the result of the determination in step S54 is not negative.

When the result of the determination in step S55 is affirmative (step S55: Y), the process proceeds to step S56. In this step S56, the control unit 110A updates the value of the distance parameter L by adding the new pixel movement amount (ΔX _j ) to the value of the distance parameter L up to that point.

  Next, in step S57, the control unit 110A determines that the "rear partial white line" is identified by the white line area identification processing described above, and determines whether the front length b is obtained. If the result of the determination in step S57 is negative (step S57: N), the process returns to step S54. Then, the processing in steps S54 to S57 is repeated, provided that the result of the determination in step S54 is not negative.

  When the result of the determination in step S57 is affirmative (step S57: Y), the process proceeds to step S58. In this step S58, the control unit 110A updates the value of the distance parameter L by subtracting the front length b from the value of the distance parameter L up to that point.

Next, in step S59, the control unit 110A calculates the provisional pixel distance PT [cm] by the following equation (11) based on the value of the distance parameter L at that time.
PT = 45 / L (11)
The equation (11) is an equation equivalent to the above equation (8).

  Subsequently, the control unit 110A calculates a weighted average of the calculated current provisional pixel distance and the provisional pixel distance stored in the storage unit 120 during a predetermined number of calibrations in the past, and newly calculates the weighted average. Calculate the calibrated pixel distance. Thereafter, the control unit 110A updates the latest pixel distance in the storage unit 120 to the newly calibrated pixel distance, and the oldest pixel distance previously stored in the storage unit 120. Instead of this, the provisional pixel distance of this time is stored in the storage unit 120 together with the current time. As a result, during the above-described output processing of the vehicle body speed v, the control unit 110A comes to calculate the vehicle body speed v using the new pixel distance.

  When the process of step S59 is completed in this way, the process returns to step S50. Then, the processing of steps S50 to S59 is repeated. As a result, the pixel distance is calibrated each time the vehicle CR travels straight and straight on a flat road surface and crosses the stop line (white line) before the intersection.

  As described above, in the first embodiment, the control unit 110A detects the pixel movement amount based on the image captured by the image capturing section 210 of the vehicle CR. In addition, the control unit 110A identifies the white line area in the captured image. Then, the control unit 110A determines the flat surface of the vehicle CR based on the known length of the stop line along the traveling direction of the vehicle CR in addition to the pixel movement amount and the result of identifying the white line area in the captured image. When it can be determined that the vehicle is traveling straight at a constant speed, the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging unit 210 and the road surface, is calibrated.

  Therefore, according to the first embodiment, even if the distance between the road surface and the imaging unit changes due to a change in vehicle weight or a change in wheel air pressure, the distance on the road surface corresponding to one pixel is changed according to the change. Can be calibrated.

  Further, in the first embodiment, the control unit 110A calculates the vehicle body speed based on the calibration result and the pixel movement amount. Therefore, it is possible to output the vehicle speed with high accuracy as the traveling information.

  Further, in the first embodiment, a stop line is adopted as the road surface characteristic portion. Therefore, by checking the brightness of each pixel of the captured image, the stop line area (that is, the white line area) can be easily specified.

  In addition, in the first embodiment, information indicating that an intersection is approached is acquired and it is presumed that the vehicle will pass the stop line. Therefore, efficient calibration can be performed.

  Further, in the first embodiment, the final calibration result is obtained by calculating the weighted average of the provisional pixel distance of this time and the provisional pixel distance calculated during the predetermined number of times of calibration in the past. Therefore, it is possible to perform the calibration while suppressing the influence of the pixel distance on the calibration result such as the variation of the length of the stop line in the traveling direction.

  Further, in the first embodiment, the calibration is carried out when the vehicle is traveling straight ahead, traveling at a constant speed, and when there is no road surface inclination. Therefore, it is possible to perform accurate pixel distance calibration.

[Second Embodiment]
Next, a second embodiment of the present invention will be described mainly with reference to FIGS.

<Structure>
FIG. 20 schematically shows the configuration of the detection device 100B according to the first embodiment. The detection device 100B is an aspect of the detection device 300B according to the second embodiment described above.

As shown in FIG. 20, the detection device 100B is connected to the imaging units 210 _F and 210 _R and the ECU 230. The detection apparatus 100B, the imaging unit 210 _F, 210 _R and ECU230 is mounted on the vehicle CR.

  The detection device 100B described above includes a control unit 110B. The detection device 100B also includes a storage unit 120.

  The control unit 110B is configured to include a central processing unit (CPU) as a calculation means. By executing the program, the control unit 110B functions as the detection device 300B in the second embodiment described above, that is, functions as the acquisition unit 310B, the first output unit 320B, and the second output unit 330. It has become.

  The program executed by the control unit 110B is stored in the storage unit 120, loaded from the recording unit and executed. This program may be acquired in a form recorded on a portable recording medium such as a CD-ROM or a DVD, or may be acquired in a form distributed via a network such as the Internet. ..

  The processing executed by the control unit 110B will be described later.

  The storage unit 120 stores various information data used by the control unit 110B. Such information data includes the program executed by the control unit 110B, the latest pixel movement amount, the calibration history including the latest pixel distance, and the like. The control unit 110B can access the storage unit 120.

<Operation>
Next, the detection operation by the detection device 100B configured as described above will be described focusing on the processing by the control unit 110B.

Note that the imaging units 210 _F and 210 _R have already started to operate, and sequentially send the front-side captured image data and the rear-side captured image data of the road surface image captured at the cycle time TP to the detection device 100B. Be present. It is also assumed that the ECU 230 has already started its operation and has sent the acceleration information, the steering angle information, and the tilt information to the detection device 100B (see FIG. 20).

  Further, in the detection apparatus 100B, the pixel distance has already been calibrated a plurality of times, and the storage unit 120 stores the calibration times for a predetermined number of recent times, the characteristics of the characteristic region extracted from the previous captured image, and It is assumed that the position in the image, a calibration history including a temporary pixel distance described later, and the like are stored. Here, the average pixel distance is stored in the storage unit 120 until the first calibration of the pixel distance is performed.

<< Pixel movement amount detection processing >>
First, a process of detecting the pixel movement amount by the control unit 110B will be described.

  In the pixel movement amount detection processing, as shown in FIG. 21, first, in step S61, it is determined whether the control unit 110B has received new front side captured image data and new rear side captured image data. judge. When the result of the determination in step S61 is negative (step S61: N), the process of step S61 is repeated.

  When the data of the new front side captured image is received and the result of the determination in step S61 is affirmative (step S61: Y), the process proceeds to step S62. In step S62, the control unit 110B extracts the characteristic region in the front side captured image obtained from the data of the new front side captured image, and specifies the characteristic of the characteristic region and the position in the image.

  Next, in step S63, the control unit 110B detects the front pixel movement amount. When detecting the front side pixel movement amount, the control unit 110B refers to the characteristic and the image position of the characteristic region extracted from the previous front side captured image in the storage unit 120 to refer to the front side captured image of this time and the front side of the previous time. The displacement amount of the position within the image of the common characteristic region in the captured image is detected as a new front pixel movement amount.

  When there are a plurality of common characteristic regions, the control unit 110B adopts the average value of the displacement amounts of the plurality of characteristic regions as the new front pixel movement amount.

  Then, the control unit 110B registers, in the storage unit 120, the characteristics and the in-image position of the characteristic area in the captured image of this time as the characteristics and the in-image position of the characteristic area extracted from the previously captured image. Further, the control unit 110B registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

  Next, in step S64, the control unit 110B extracts the characteristic region in the rear side captured image obtained from the data of the new rear side captured image, and specifies the characteristic of the characteristic region and the position in the image.

  Next, in step S65, the control unit 110B detects the rear pixel movement amount. When detecting the rear side pixel movement amount, the control unit 110B refers to the characteristics and the image position of the characteristic region extracted from the previous rear side captured image in the storage unit 120, The amount of displacement of the in-image position of the common characteristic region in the previous captured image on the rear side is detected as a new amount of rear pixel movement.

  Note that when there are a plurality of common feature regions, the control unit 110B determines the average value of the displacement amounts of the plurality of feature regions as new rear pixel movements, as in the case of detecting the front pixel movement amount. Adopt as quantity.

  Next, in step S66, the control unit 110B detects a new pixel movement amount. When detecting the pixel movement amount, the control unit 110B detects a new pixel movement amount by calculating the average of the front side pixel movement amount and the rear side pixel movement amount.

  Then, the control unit 110B determines the characteristics and intra-image position of the characteristic area in the front side captured image of this time, and the characteristics and intra-image position of the characteristic area in the rear side captured image of this time as the characteristics area of the previous front side captured image. It is registered in the storage unit 120 as the characteristic and the position in the image and the characteristic and the position in the image of the characteristic region in the last captured image on the rear side. Further, the control unit 110B registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

  When the process of step S66 is completed in this way, the process returns to step S61. After that, steps S61 to S66 are repeated, and the pixel movement amount is detected each time the data of the new front side captured image and the data of the new rear side captured image are received.

<< Processing to output vehicle speed v >>
The output processing of the vehicle body speed v by the control unit 110B is similar to the output processing of the vehicle body speed v by the control unit 110A described above (see FIG. 16).

<< White line area specification processing >>
Next, the white line area specifying process by the control unit 110B will be described. The white line area specifying process is executed in parallel with the pixel movement amount detecting process and the vehicle body speed v output process described above.

  The white line area specifying process is performed for the purpose of calibrating the pixel distance, which will be described later. The calibration environment for this pixel distance calibration is the same as that shown in FIG. 11 described above.

  Now, in the white line area specifying process, as shown in FIG. 22, first, in step S71, the control unit 110B determines whether or not new front side captured image data has been received. When the result of the determination in step S71 is negative (step S71: N), the process of step S71 is repeated.

  When new front side captured image data is received and the result of the determination in step S71 is affirmative (step S71: Y), the process proceeds to step S72. In step S72, the control unit 110B identifies the white line region in the front side captured image obtained from the data of the new front side captured image based on the brightness of each pixel in the front side captured image.

  Next, in step S73, the control unit 110B determines whether or not the result of identifying the white line area is “first rear partial white line (see FIG. 12A)”. If the result of the determination in step S73 is negative (step S73: N), the process returns to step S71. Then, the processes of steps S71 to S73 are repeated until the result of the determination of step S73 is affirmative.

  If the result of specifying the white line area in the front captured image is “first rear partial white line” and the result of the determination in step S73 is affirmative (step S73: Y), the process proceeds to step S74. In step S74, the control unit 110B determines the front length a (see FIG. 12A).

  In addition, in the second embodiment, the control unit 110B, on the condition that the white line until the "first rear partial white line" in the front side captured image can be clearly specified continues, "the first rear partial white line". The "white line" is specified.

  Next, in step S75, the control unit 110B determines whether or not the data of the new rear side captured image has been received. If the result of the determination in step S75 is negative (step S75: N), the process of step S75 is repeated.

  When the new rear-side captured image data is received and the result of the determination in step S75 is affirmative (step S75: Y), the process proceeds to step S76. In step S76, the control unit 110B identifies the white line region in the rear side captured image obtained from the data of the new rear side captured image based on the brightness of each pixel in the captured image.

  Next, in step S77, the control unit 110B determines whether or not the white line area identification result is the "second rear partial white line (see FIG. 12B)". If the result of the determination in step S77 is negative (step S77: N), the process returns to step S75. Then, the processes of steps S75 to S77 are repeated until the result of the determination of step S77 becomes affirmative.

  If the white line region identification result in the rear side captured image is “second rear partial white line” and the determination result in step S77 is affirmative (step S77: Y), the process proceeds to step S78. In step S78, the control unit 110B obtains the front length b (see FIG. 12B). Then, the process returns to step S71.

  After that, steps S71 to S78 are repeated. As a result, the white line area is specified every time the vehicle CR crosses the white line.

<< Monitoring of calibration conditions >>
The calibration condition monitoring process by the control unit 110B is similar to the calibration condition monitoring process by the control unit 110A described above (see FIG. 18).

<< Pixel distance calibration process >>
Next, the pixel distance calibration process by the control unit 110B will be described. The pixel distance calibration process is performed in parallel with the above-described pixel movement amount detection process, vehicle body speed v output process, white line region identification process, and calibration condition monitor process.

  In the pixel distance calibration process, as shown in FIG. 23, first, in step S81, the control unit 110B determines whether or not the calibration condition flag is “ON”, so that the vehicle CR is flat. It is determined whether or not the vehicle is traveling straight at a constant speed on the road surface. When the result of the determination in step S81 is negative (step S81: N), the process of step S81 is repeated.

  When the result of the determination in step S81 is affirmative (step S81: Y), the process proceeds to step S82. In step S82, the control unit 110B specifies that the front side captured image is the "first rear partial white line" by the white line area specifying process described above, and determines whether the front length a is obtained.

  When the result of the determination in step S82 is negative (step S82: N), the process returns to step S81. Then, the processes of steps S81 and S82 are repeated.

  When the result of the determination in step S82 is affirmative (step S82: Y), the process proceeds to step S83. In this step S83, the distance parameter L is set to the front length a.

  Next, in step S84, the control unit 110B determines whether or not the calibration condition flag is “ON”, thereby determining whether or not the vehicle CR is traveling straight at a constant speed on a flat road surface. When the result of the determination in step S84 is negative (step S84: N), the process returns to step S81.

When the result of the determination in step S84 is affirmative (step S84: Y), the process proceeds to step S85. In step S85, the control unit 110B determines whether or not a new pixel movement amount (ΔX _j ) has been detected by the above-described pixel movement amount detection processing.

  When the result of the determination in step S85 is negative (step S85: N), the process returns to step S84. Then, the processing in steps S84 and S85 is repeated, provided that the result of the determination in step S84 is not negative.

When the result of the determination in step S85 is affirmative (step S85: Y), the process proceeds to step S86. In this step S86, the control unit 110B updates the value of the distance parameter L by adding the new pixel movement amount (ΔX _j ) to the value of the distance parameter L up to that point.

  Next, in step S87, the control unit 110B specifies that the back side captured image is the "second rear partial white line" by the white line area specifying process described above, and determines whether the front length b is obtained. judge. If the result of the determination in step S87 is negative (step S87: N), the process returns to step S84. Then, the processing in steps S84 to S87 is repeated, provided that the result of the determination in step S84 is not negative.

  When the result of the determination in step S87 is affirmative (step S87: Y), the process proceeds to step S88. In step S88, the control unit 110B updates the value of the distance parameter L by subtracting the front length b from the value of the distance parameter L up to that point.

Next, in step S89, the control unit 110B calculates the provisional pixel distance PT [cm] by the following equation (12) based on the value of the distance parameter L at that time.
PT = D / L (12)
The equation (12) is an equation equivalent to the above equation (10).

  Subsequently, the control unit 110B calculates a weighted average of the calculated current provisional pixel distance and the provisional pixel distance stored in the storage unit 120 during a predetermined number of calibrations in the past, and newly calculates the weighted average. Calculate the calibrated pixel distance. Thereafter, the control unit 110B updates the latest pixel distance in the storage unit 120 to the newly calibrated pixel distance, and the oldest provisional pixel distance previously stored in the storage unit 120. Instead of this, the provisional pixel distance of this time is stored in the storage unit 120 together with the current time. As a result, during the above-described output processing of the vehicle body speed v, the control unit 110B comes to calculate the vehicle body speed v using the new pixel distance.

  When the process of step S89 ends in this way, the process returns to step S81. Then, the processes of steps S81 to S89 are repeated. As a result, the pixel distance is calibrated each time the vehicle CR travels straight and straight on a flat road surface and crosses the white line before the intersection.

As described above, in the second embodiment, the control unit 110B, based on the images captured by the image capturing units 210 _F and 210 _R arranged on the vehicle CR at the known distance D along the traveling direction of the vehicle CR, Detect the amount of pixel movement. Further, the control unit 110B identifies the white line area in the captured image. When the control unit 110B can determine that the vehicle CR is traveling straight at a constant speed on the flat road surface based on the known distance D, in addition to the pixel movement amount and the result of identifying the white line area in the captured image. First, the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging units 210 _F and 210 _R and the road surface, is calibrated.

  Therefore, according to the second embodiment, regardless of whether or not the length of the white line that is the road surface characteristic portion along the traveling direction of the moving body is known or unknown, the change in the moving body weight or the wheel Even if the distance between the road surface and the imaging unit changes due to the change in air pressure, the distance on the road surface corresponding to one pixel can be calibrated according to the change.

  Further, in the second embodiment, the control unit 110B outputs the vehicle body speed based on the calibration result and the pixel movement amount. Therefore, it is possible to output the vehicle speed with high accuracy as the traveling information.

  In addition, in the second embodiment, a white line is adopted as the road surface characteristic portion. Therefore, the white line region can be easily specified by checking the brightness of each pixel of the captured image.

  Further, in the second embodiment, as in the case of the first embodiment, a final weighted average of the provisional pixel distance of this time and the provisional pixel distance calculated at the time of the predetermined number of calibrations in the past is calculated, and the final weighted average is calculated. Get calibration results. Therefore, it is possible to perform the calibration while suppressing the influence of the pixel distance on the calibration result such as the variation of the length of the white line in the traveling direction.

  Further, in the second embodiment, as in the case of the first embodiment, the calibration is carried out when the vehicle is traveling straight ahead, traveling at a constant speed, and when there is no road surface inclination. Therefore, it is possible to perform accurate pixel distance calibration.

[Modification of Example]
The present invention is not limited to the above-mentioned first and second embodiments, and various modifications can be made.

  For example, the same modification as that of the above-described first embodiment can be performed on the first example. Further, the same modification as that of the above-described second embodiment can be applied to the second example.

100A, 100B ... Detection device 110A, 110B ... Control unit (acquisition part, 1st output part, 2nd output part)
300A, 300B ... Detecting device 310A, 310B ... Acquisition part 320A, 320B ... 1st output part 321A, 321B ... Moving amount detection part 322A, 322B ... Specific part 323A, 323B ... Calibration part 330 ... 2nd output part

Claims (1)

An acquisition unit for acquiring an image of the road surface characteristic part imaged by the imaging unit mounted on the moving body;
An output unit that outputs distance-related information related to the distance between the mounting position of the imaging unit and the road surface, based on the image of the road surface characteristic portion acquired by the acquisition unit;
The image capturing unit captures an image of a road surface directly below a fixed position where the image capturing unit is fixed to the moving body,
A detection device characterized by the above.

Images