JP2018113065A - Detection device, detection method, detection program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform calibration corresponding to a change even if a distance between an imaging part and a road face is changed due to a change in the weight of a moving body and a change in the pneumatic pressure of wheels.SOLUTION: A moving amount detection part 321a detects a pixel moving amount on the basis of a captured imaged image which is captured by an imaging part 210 with respect to a stop line being a feature part on a road face having a well-known length along a traveling direction of a moving body MV. A specification part 322A specifies a white line region in the captured image. Then, when it is determined that the moving body MV linearly travels on the flat road face at a constant velocity, the calibration part 323A calibrates a distance on the road face corresponding to one pixel related to a distance between the imaging part 210 and the road face, on the basis of the well-known length along the traveling direction of the moving body MV on the stop line, in addition to the pixel moving amount and a specified result of the white line region in the captured image.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a detection device, a detection method, a detection program, and a recording medium on which the detection program is recorded.

  The slip ratio λ obtained by normalizing the difference between the wheel speed of the moving body such as a vehicle and the moving body speed, and the friction coefficient μ obtained by normalizing the grip force between the wheel and the road surface are as shown in FIGS. (Hereinafter referred to as “μ-λ characteristics”). Here, FIG. 1 shows the μ-λ characteristic during driving, and FIG. 2 shows the relationship with the μ-λ characteristic during braking.

  1 and 2, the μ-λ characteristic on the dry road surface is indicated by a solid line, the μ-λ characteristic on the wet road surface is indicated by a one-dot chain line, and the μ-λ characteristic on the frozen road surface is indicated by two points. It is indicated by a chain line.

  In the change of the friction coefficient μ accompanying the increase of the slip ratio λ during driving shown in FIG. 1, a state where the friction coefficient μ is smaller than the slip ratio at which the friction coefficient μ is maximum is a stable region where the moving body can travel stably. Yes. On the other hand, in a state where the friction coefficient μ is larger than the slip ratio at which the friction coefficient μ is maximum, the gripping force is lowered, and in the worst case, an unstable region in which idling or a lock phenomenon occurs.

  In the change of the friction coefficient μ accompanying the increase of the slip ratio λ during braking shown in FIG. 2, the state where the friction coefficient μ is larger than the slip ratio at which the friction coefficient μ is minimum is the stable region. On the other hand, in a state where the friction coefficient μ is smaller than the slip ratio at which the friction coefficient μ is minimum, the region is unstable.

  If the slip ratio λ is controlled within a range equal to or less than the absolute value of the slip ratio at which the absolute value of the friction coefficient μ is the maximum value, the moving body can maintain stable running. On the other hand, if the unstable state continues and the tire slips or locks, control of driving, braking and steering of the vehicle becomes impossible.

  For this reason, in order to avoid the danger of accidents, an ABS (Antilock Brake System) using mainly brake hydraulic pressure control and engine control is adopted in internal combustion engine vehicles. In such ABS or the like, the slip state is determined, and the engine driving torque and the braking torque of the brake hydraulic pressure are controlled so as to approach the stable region. Also in an electric vehicle, an anti-slip control is proposed in which the slip ratio λ is estimated and the motor torque is appropriately controlled so as to be maintained in a stable region. As described above, detecting the slip ratio λ is very important for grasping the traveling state.

Since the slip ratio λ is calculated by the following equation (1), the wheel radius r, the rotational angular velocity ω, and the moving body velocity v are required.
λ = (r · ω−v) / Max (r · ω, v) (1)

  Here, Max (r · ω, v) indicates the larger value of (r · ω) and v. Since (r · ω) is larger than v at the time of driving, Max (r · ω, v) = r · ω. On the other hand, at the time of braking, since v is larger than (r · ω), Max (r · ω, v) = v.

  For example, when the moving body is an automobile, the wheel radius r is considered to be constant if the wheel air pressure is sufficient. Further, the rotational angular velocity ω can be detected from the pulse output of an encoder mounted on a wheel or the signal output of a resolver connected to a motor in the case of an electric vehicle.

On the other hand, the following methods (a) to (c) are generally cited as methods for detecting the moving body speed v.
(A) Calculation based on rotation speed of non-driving wheels (b) Calculation by integrating acceleration values detected by acceleration sensor (c) Calculation based on detection result by optical sensor

  Here, in the method (a), since the brake is applied to all the wheels, the moving body speed v at the time of braking cannot be detected. Further, in the case of four-wheel drive, there is no non-drive wheel, so the moving body speed v cannot be detected.

  In the method (b), since the output of the acceleration sensor is integrated, an offset existing in the output of the acceleration sensor is accumulated. As a result, the moving body speed v cannot be detected accurately.

  The method (c) in which the drawbacks of the methods (a) and (b) do not exist in principle is expected. As a technique employing the method (c), a technique described in Patent Document 1 has been proposed (hereinafter referred to as “conventional example”). In the conventional technique, the running state or the stopped state of the vehicle is determined from an image signal obtained by capturing a scene in the vicinity of the host vehicle. Further, based on the dimensions of the road markings in the image and the dimensions of the road markings on the road, the vehicle body speed (that is, the moving body speed) of the host vehicle is detected. In the conventional technique, the vehicle body speed is detected only when the road marking is present in the image, and only the running / stop determination is performed when the road marking is not present in the image.

JP 2009-205642 A

  In the conventional technique described above, the vehicle body speed is not detected when there is no road marking in the captured image. As a result, even if the vehicle speed changes while the road marking does not exist in the captured image, the vehicle speed detected at the time when the road marking was last present in the captured image remains unchanged. It will be estimated as speed. For this reason, it is difficult to accurately detect the vehicle speed at each time point.

  In the prior art, since the imaging device is mounted on the vehicle body, the distance between the imaging device and the road surface changes depending on the number of passengers, the load, and the wheel air pressure. As a result, since the imaging magnification changes, the size of the road marking in the captured image changes.

For example, as shown in FIG. 3A, the imaging unit 210 includes an imaging lens system 211 (focal length: f) and a square imaging surface 212 (length of one side: D), and forms an image. The distance from the lens system 211 to the imaging surface 212 is “d”. In this case, when imaging the road surface LD immediately below, if the distance from the imaging lens system 211 to the road surface LD (hereinafter also referred to as “distance from the imaging unit 210 to the road surface LD”) is “h”, the imaging unit 210. The optical magnification m of imaging by is expressed by the following equation (2).
m = 1 / ((h / f) -1) (2)
The pixel configuration on the imaging surface 212 is NU × NU (for example, 30 × 30).

As a result, the length H of one side of the square area to be imaged on the road surface LD is expressed by the following equation (3).
H = D / m = D · ((h / f) −1) (3)

  For example, when f = 8 [mm], D = 1.7 [mm], and h = 495 [mm], H = 103.5 [mm].

Now, as shown in FIG. 3 (B), when the vehicle weight increases from the state of FIG. 3 (A) and the distance h becomes shorter and becomes the distance h ^* , the optical magnification of imaging by the imaging unit 210 becomes It changes from “m” in the state of FIG. 3A to “m ^* ” expressed by the following equation (4).
m ^* = 1 / ((h ^* / f) -1) (4)

As a result, the length of one side of the square area to be imaged changes from “H” in the state of FIG. 3A to “H ^* ” expressed by the following equation (5).
H ^* = D / m ^* = D · ((h ^* / f) −1) (5)

For example, when h ^* = 445 [mm], H ^* = 92.9 [mm], which changes by about 10% from H = 103.5 [mm] in the state of FIG. As a result, the size of the area on the road surface corresponding to the size of one pixel also changes by about 10% from the state shown in FIG.

  By the way, when the movement distance of an image is calculated | required by the correlation between the captured images for every unit time, and the vehicle body speed of the front-back direction and a horizontal direction is calculated, the area | region on the road surface corresponding to the magnitude | size of 1 pixel in a captured image When the size of the vehicle changes, the calculated vehicle speed also changes. That is, if the size of the area on the road surface corresponding to the size of one pixel changes by about 10%, the speed calculation value also changes by about 10%, resulting in an error.

  The dynamic error with respect to the road surface fluctuation can be less affected by the error by the averaging process. However, since the change in the vehicle weight and the change in the wheel air pressure are offset, the averaging process cannot suppress the influence of the error.

  For this reason, there is a need for a technique that can accurately detect the vehicle body speed even when the vehicle weight changes or the wheel air pressure changes. Meeting this requirement is one of the problems to be solved by the present invention.

  The present invention has been made in view of the above, and even when the distance between the road surface and the imaging unit changes due to a change in vehicle weight or a change in wheel air pressure, calibration according to the change can be performed. An object is to provide a detection device and a detection method.

The invention according to claim 1 acquires a plurality of images including road surface features having a known length, which are imaged along a traveling direction at predetermined time intervals by an imaging unit mounted on a moving body. A number of image regions having a predetermined width corresponding to the known length based on the plurality of images acquired by the acquisition unit, and based on the calculated number and the known length And an output unit that outputs distance information related to the distance on the road surface corresponding to the predetermined width.
According to a fourth aspect of the present invention, there is provided an acquisition unit that acquires a plurality of images including a feature on the road surface captured along a traveling direction at predetermined time intervals by an imaging unit mounted on a moving body; Information indicating the distance on the road surface corresponding to the width of the predetermined image region in the image obtained based on the plurality of acquired images is calibrated based on the distance information output in the past, and the calibration result is newly determined. A detection apparatus comprising: a first output unit that outputs as distance information; and a second output unit that outputs the speed of the moving object using the new distance information.

  The invention according to claim 5 is a detection method used in a detection apparatus including an acquisition unit and an output unit, wherein the acquisition unit is traveled at predetermined time intervals by an imaging unit mounted on a moving body. An acquisition step of acquiring a plurality of images including a feature on the road surface having a known length taken along the line; and the output unit based on the plurality of images acquired in the acquisition step And calculating distance information related to the distance on the road surface corresponding to the predetermined width based on the calculated number and the known length. And a distance information output step. A detection method comprising:

  A sixth aspect of the present invention is a detection program that causes a computer included in the detection apparatus to execute the detection method according to the fifth aspect.

  The invention described in claim 7 is a recording medium in which the detection program according to claim 6 is recorded so as to be readable by a computer included in the detection apparatus.

It is a figure which shows the relationship between the slip ratio at the time of a drive, and a friction coefficient. It is a figure which shows the relationship between the slip ratio at the time of braking, and a friction coefficient. It is a figure for demonstrating the change of the speed calculation value by changes, such as a vehicle weight. It is a figure which shows the structure of the detection apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating operation | movement of the movement amount detection part of FIG. It is a figure for demonstrating the calibration environment in 1st Embodiment. It is a figure for demonstrating operation | movement of the specific part of FIG. It is a figure for demonstrating the calibration operation | movement by the calibration part of FIG. It is a figure which shows the structure of the detection apparatus which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating the relationship of the arrangement position of two imaging parts of FIG. It is a figure for demonstrating the calibration environment in 2nd Embodiment. It is a figure for demonstrating operation | movement of the specific part of FIG. It is a figure for demonstrating the calibration operation | movement by the calibration part of FIG. It is a figure which shows the structure of the detection apparatus which concerns on 1st Example of this invention. It is a flowchart for demonstrating the detection process of the pixel moving amount performed by the control unit of FIG. FIG. 15 is a flowchart for explaining vehicle speed output processing executed by the control unit of FIG. 14. FIG. It is a flowchart for demonstrating the identification process of the white line area | region performed by the control unit of FIG. It is a flowchart for demonstrating the monitoring process of the calibration conditions performed by the control unit of FIG. It is a flowchart for demonstrating the calibration process of the pixel distance performed by the control unit of FIG. It is a figure which shows the structure of the detection apparatus which concerns on 2nd Example of this invention. FIG. 21 is a flowchart for explaining pixel movement amount detection processing executed by the control unit of FIG. 20. FIG. It is a flowchart for demonstrating the specific process of the white line area | region performed by the control unit of FIG. FIG. 21 is a flowchart for explaining pixel distance calibration processing executed by the control unit of FIG. 20. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS. In the following description and drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description is omitted.

[First Embodiment]
First, a first embodiment of the present invention will be described with reference to FIGS.

<Configuration>
FIG. 4 is a block diagram showing the configuration of the detection apparatus 300A according to the first embodiment. As illustrated in FIG. 4, the detection device 300 </ b> A is connected to an imaging unit 210, a navigation device 220, and an ECU (Electronic Control Unit) 230. The detection device 300A, the imaging unit 210, the navigation device 220, and the ECU 230 are mounted on the moving body MV.

  The imaging unit 210 is mounted at a fixed position of the moving body MV, and images the road surface immediately below the fixed position as described with reference to FIG. The imaging unit 210 periodically performs imaging of the road surface with a period time TP. Data of the image thus captured (hereinafter referred to as “captured image data”) is sent to the detection apparatus 300A.

  In addition, from the viewpoint that the “cycle time TP” includes a common area on the road surface in the two images captured with the cycle time TP separated even when the moving body MV is traveling at a high speed. It is determined in advance based on experiments, simulations, and the like.

  The navigation device 220 described above performs driving operation support of the moving body MV for the user based on the map information and the current position information. When the moving body MV is within a predetermined distance from the intersection existing in the traveling direction, the navigation device 220 sends a message to that effect to the detection device 300A.

  Note that the “predetermined distance” is an experiment from the viewpoint of appropriate prior notice for imaging from the beginning to the end along the traveling direction regarding the stop line before the intersection adopted as the road surface feature in the first embodiment, It is determined in advance based on simulation, experience, and the like.

  The ECU 230 controls traveling of the moving body MV based on sensor detection information acquired from various sensors such as a wheel rotation speed sensor, an acceleration sensor, a steering angle sensor, and a tilt sensor, and provides traveling information to the user. Or provide. The ECU 230 sends acceleration information, steering angle information, and tilt information to the detection device 300A.

  The ECU 230 further acquires the moving body speed (traveling speed information) sent from the detection device 300A, and uses the acquired moving body speed for traveling control of the moving body MV.

  As shown in FIG. 4, the detection apparatus 300A includes an acquisition unit 310A and a first output unit 320A. The detection apparatus 300 </ b> A includes a second output unit 330.

  The acquisition unit 310A receives the captured image data transmitted from the imaging unit 210. Then, the acquisition unit 310A sends the captured image data to the first output unit 320A.

  The first output unit 320A receives the captured image data sent from the acquisition unit 310A. The first output unit 320A detects the number of movement amounts of pixels at the same road surface position (hereinafter referred to as “pixel movement amount”) between two images imaged at intervals of the period time TP. When the calibration condition is satisfied, the distance on the road surface corresponding to one pixel (hereinafter referred to as “pixel distance”) is calibrated. The detected pixel movement amount and the calibrated pixel distance are output to the second output unit 330. Details of the configuration of the first output unit 320A having such a function will be described later.

  Whenever the second output unit 330 receives the pixel movement amount sent from the first output unit 320A, the second output unit 330 uses the latest calibrated pixel distance and cycle time TP as the travel information of the mobile unit MV. Calculate the speed. Then, the second output unit 330 outputs the calculated moving body speed to the ECU 230.

<< Configuration of First Output Unit 320A >>
Next, the configuration of the first output unit 320A described above will be described.

  As shown in FIG. 4, the first output unit 320A includes a movement amount detection unit 321A and a specifying unit 322A. The first output unit 320A includes a calibration unit 323A.

  The movement amount detection unit 321A receives the captured image data sent from the acquisition unit 310A. Then, the movement amount detection unit 321A detects the pixel movement amount by a so-called displacement amount search method based on the captured image data. The pixel movement amount thus detected is sent to the calibration unit 323A and the second output unit 330.

  The details of the operation of the movement amount detection unit 321A will be described later.

  The specifying unit 322A receives the captured image data sent from the acquisition unit 310A. Then, the specifying unit 322A specifies a white line region in an image obtained from captured image data. Information on the white line area thus identified (hereinafter referred to as “white line area information”) is sent to the calibration unit 323A.

  Details of the operation of the specifying unit 322A will be described later.

  The calibration unit 323A receives the pixel movement amount sent from the movement amount detection unit 321A and the white line area information sent from the specifying unit 322A. When the calibration unit 323A can determine that the moving body MV is traveling straight at a constant speed on a flat road surface based on the acceleration information, the steering angle information, and the tilt information transmitted from the ECU 230, the calibration unit 323A In addition to the white line area information, the pixel distance is calibrated based on the fact that the length of the stop line drawn on the road is approximately 45 [cm]. The pixel distance thus calibrated is sent to the second output unit 330.

  The details of the operation of the calibration unit 323A will be described later.

<Operation>
Next, the operation of the detection apparatus 300A calibrated as described above will be described by focusing on the processing performed by each element of the detection apparatus 300A.

  It is assumed that the imaging unit 210 has already started the operation and sequentially sends the captured image data of the road surface image captured at the cycle time TP to the detection device 300A. In addition, the navigation device 220 has already started to operate, and when the moving body MV is within a predetermined distance from the intersection existing in the traveling direction, it is assumed that this is sent to the detection device 300A. Further, it is assumed that the ECU 230 has already started operation and has sent acceleration information, steering angle information, and tilt information to the detection device 300A (see FIG. 4).

  Furthermore, in the detection apparatus 300A, the calibration of the pixel distance is already performed a plurality of times, and the calibration unit 323A holds the calibration history including the latest predetermined calibration time and the provisional pixel distance described later. It shall be. In addition, it is assumed that the latest calibrated pixel distance is held in the second output unit 330. Here, in the period until the first calibration of the pixel distance is performed, the average pixel distance is held in the second output unit 330.

  Note that the “predetermined number of times” is based on experiments, simulations, experiences, etc. from the viewpoint of performing an averaging process to suppress the influence of the pixel distance on the calibration result, such as fluctuations in the length of the stop line in the traveling direction. It is predetermined based on. The “average pixel distance” is determined in advance corresponding to the moving object MV based on experiments, simulations, experiences, and the like.

<< Image data acquisition process >>
In detection device 300A, acquisition unit 310A receives the captured image data sent from imaging unit 210. Then, the acquiring unit 310A sends the captured image data to the movement amount detecting unit 321A and the specifying unit 322A of the first output unit 320A (see FIG. 4).

<< Pixel movement detection process >>
Next, pixel movement amount detection processing by the movement amount detection unit 321A will be described.

  Upon receiving the captured image data sent from the acquisition unit 310A, the movement amount detection unit 321A positions the common feature region in the current image obtained from the current captured image data and the previous image obtained from the previous captured image data. Is detected as a pixel movement amount. Then, the movement amount detection unit 321A sends the detected pixel movement amount to the calibration unit 323A and the second output unit 330 (see FIG. 4).

  When there are a plurality of common feature areas, the movement amount detection unit 321A employs the average value of the displacement amounts of the plurality of feature areas as the pixel movement amount.

FIG. 5 shows an example of movement amount detection by the movement amount detection unit 321A. In the example of FIG. 5, the moving body MV travels straight along the X direction, the X direction position at time T _j is X _j , and the X direction position at time T _{j + 1} (= T _j + TP) is This is an example in the case of X _{j + 1} . In addition, the example of FIG. 5, a common characteristic region in the captured image in the captured image and the time T _{j + 1} at time T _j is two and is an example of a characteristic region A and the characteristic region B are shown. In the example of FIG. 5, the displacement amount of the feature region A is “ΔX _jA ” and the displacement amount of the feature region B is “ΔX _jB ”. Therefore, the pixel movement amount ΔX _j is expressed by the following equation (6). Calculated.
ΔX _j = (ΔX _jA + ΔX _jB ) / 2 (6)

<< Output processing of moving body speed v >>
Next, the output process of the moving body speed v by the second output unit 330 will be described.

  Upon receiving the pixel movement amount sent from the movement amount detection unit 321A, the second output unit 330 is based on the pixel movement amount, the held pixel distance (that is, the latest calibrated pixel distance), and the period time TP. To calculate the moving body speed v. Then, the second output unit 330 outputs the calculated moving body speed v to the ECU 230.

Here, when the pixel movement amount is “PN” and the pixel distance is “PD”, the second output unit 330 calculates the moving body velocity v by the following equation (7).
v = PN / PD / TP (7)

《White line area identification processing》
Next, the white line area specifying process by the specifying unit 322A will be described.

  The white line region specifying process is performed for the calibration of the pixel distance by the calibration unit 323A described later. An environment for the calibration of such pixel distance (hereinafter referred to as “calibration environment”) is shown in FIG. As shown in FIG. 6, the calibration of the pixel distance is a white line on the road surface where the moving body MV is traveling at a constant speed on a flat road surface and has a length W (≈45 [cm]) in the traveling direction. This is performed when the stop line SPL is crossed at right angles to the longitudinal direction of the stop line SPL.

  In the following description, it is assumed that the stop line SPL used for calibration of the new pixel distance is a white line region from the travel direction position (that is, the X direction position) XR to the travel direction position XP. .

  Now, when the captured image data sent from the acquisition unit 310A is received, the identifying unit 322A identifies the white line region in the image obtained from the captured image data based on the brightness of each pixel in the image, and identifies the identified result. The white line area information is sent to the calibration unit 323A (see FIG. 4). The white line region identification results include two types, a front partial white line shown in FIG. 7A and a rear partial white line shown in FIG. 7B.

  Note that the specifying unit 322A specifies the number of pixels a in the length of the white line region in the image in the X direction a (hereinafter also referred to as “front length a”) when the specifying result is a partial front white line. Further, when the identification result is a rear partial white line, the identification unit 322A identifies the number of pixels b in the length in the X direction of the non-white line region in the image (hereinafter also referred to as “front length b”).

  Then, in the case of a partial front white line, the specifying unit 322A sends white line region information [white line flag: ON, front length: a] shown in FIG. 7A to the calibration unit 323A (see FIG. 4). Further, in the case of a partial rear white line, the specifying unit 322A sends white line area information [white line flag: ON, front length: b] shown in FIG. 7B to the calibration unit 323A (see FIG. 4).

]
<< Pixel distance calibration process >>
Next, calibration processing by the calibration unit 323A will be described.

  In the calibration process, the calibration unit 323A performs an intersection determination as to whether or not the navigation apparatus 220 has received that the moving body MV is within a predetermined distance from the intersection existing in the traveling direction. If the result of the intersection determination is affirmative, the calibration unit 323A can determine that the moving body MV is traveling straight at a constant speed on a flat road surface based on the acceleration information, the steering angle information, and the inclination information transmitted from the ECU 230. It is determined whether or not the vehicle travels straight at a constant speed.

  When the result of the constant speed straight-ahead determination is affirmative, the calibration unit 323A performs white line start determination as to whether or not the white line region information sent from the specifying unit 322A is a “front partial white line”. If the result of the white line start determination is affirmative, the calibration unit 323A indicates that the white line region information sent from the specifying unit 322A is “on the condition that the result of the constant-velocity straight-ahead determination is positive is maintained. The pixel movement amount sent from the movement amount detection unit 321A is collected until the result of the white line end determination regarding whether or not “backward partial white line” is affirmative.

When the result of the white line end determination is affirmative, the calibration unit 323A determines that the forward length a included in the white line region information at the time when the “front partial white line” is obtained, the collected pixel movement amount ΔX ₁ ,. Based on ΔX _M and the front length b included in the white line region information at the time when the “backward partial white line” is reached, the provisional pixel distance PT [cm] is calculated by the following equation (8).
PT = 45 / (a + ΔX ₁ +... + ΔX _M −b) (8)

  Next, the calibration unit 323A calculates the weighted average of the calculated current provisional pixel distance and the provisional pixel distance calculated during the predetermined number of calibrations held in the past, and is calibrated this time. Calculate the pixel distance. Thereafter, the calibration unit 323A retains the current provisional pixel distance inside instead of the oldest provisional pixel distance retained therein.

  In the weighted average, the longer the elapsed time up to the present time, the smaller the weight.

  The calibration unit 323A sends the calculated pixel distance to the second output unit 330. As a result, the second output unit 330 calculates the moving body speed v using the new pixel distance.

FIG. 8 shows an example in which a captured image of the front partial white line is obtained at time T ₁ and a captured image of the rear partial white line is obtained at time T ₅ . In the example of FIG. 8, the provisional pixel distance PT [cm] is calculated by the following equation (9).
PT = 45 / (a + ΔX ₁ + ΔX ₂ + ΔX ₃ + ΔX ₄ −b) (9)

  As described above, in the first embodiment, the movement amount detection unit is based on the captured image by the imaging unit 210 regarding the stop line that is a road surface characteristic unit having a known length along the traveling direction of the moving body MV. 321A detects the amount of pixel movement. Further, the specifying unit 322A specifies a white line region in the captured image. Then, the calibration unit 323A determines that the moving body MV is flat based on the known length of the stop line along the traveling direction of the moving body MV in addition to the pixel movement amount and the result of specifying the white line area in the captured image. When it can be determined that the vehicle is traveling straight at a constant speed on the road surface, the distance on the road surface corresponding to one pixel related to the distance between the imaging unit 210 and the road surface is calibrated.

  Therefore, according to the first embodiment, even if the distance between the road surface and the imaging unit 210 changes due to a change in the weight of the moving body or a change in the air pressure of the wheels, the distance on the road surface corresponding to one pixel is changed to the change. Can be calibrated accordingly.

  In the first embodiment, the second output unit 330 calculates the moving body speed based on the calibration result and the pixel movement amount. For this reason, it is possible to output a moving body speed with high accuracy as travel information.

  Moreover, in 1st Embodiment, a stop line is employ | adopted as a road surface characteristic part. For this reason, the stop line region (that is, the white line region) can be easily specified by examining the brightness of each pixel of the captured image.

  In the first embodiment, information indicating that the vehicle is approaching an intersection is acquired, and it is estimated in advance that the vehicle will pass a stop line. For this reason, efficient calibration can be performed.

In the first embodiment, a weighted average of the current provisional pixel distance and the provisional pixel distance calculated during a predetermined number of calibrations in the past is calculated to obtain a final calibration result.
This is because the assumed length may deviate from 45 [cm] due to missing stop lines or protruding paint. For this reason, the calibration which suppressed the influence on the calibration result of pixel distance, such as a fluctuation | variation of the length of the advancing direction of a stop line, can be performed.

  In the first embodiment, calibration is performed when the vehicle is traveling straight, traveling at a constant speed, and having no road surface inclination. This means that if there is a lateral movement, the stop line passes diagonally and calibration is performed based on a length of 45 [cm] or more, and when the acceleration or deceleration is large or the road surface is inclined. In some cases, the expansion and contraction of the suspension or the like occurs, so that the optical magnification at the time of imaging is changing. For this reason, accurate pixel distance calibration can be performed.

[Second Embodiment]
Next, a second embodiment of the present invention will be described mainly with reference to FIGS.

<Configuration>
FIG. 9 is a block diagram showing the configuration of the detection apparatus 300B according to the second embodiment. As shown in FIG. 9, the detection device 300 </ b> B is connected to the imaging units 210 _F and 210 _R and the ECU 230. The detection device 300B, the imaging units 210 _F and 210 _R, and the ECU 230 are mounted on the moving body MV.

Each of the imaging units 210 _F and 210 _R is configured similarly to the imaging unit 210 described above. The imaging unit 210 _F is arranged on the front side of the moving body MV, and the imaging unit 210 _R is arranged on the rear side of the moving body MV that is separated from the imaging unit 210 _F by a distance D (see FIG. 10). Note that the imaging units 210 _F and 210 _R perform imaging at the same timing.

Image captured by the imaging unit 210 _F (hereinafter, referred to as "front side captured image") data is transmitted to the detector 300B. Data of an image captured by the imaging unit 210 _R (hereinafter referred to as “rear captured image”) is also sent to the detection apparatus 300B.

  Note that the ECU 230 sends acceleration information, steering angle information, and tilt information to the detection device 300B, as in the case of the first embodiment described above. In addition, the ECU 230 further acquires the moving body speed sent from the detection device 300B, and uses the acquired moving body speed for traveling control of the moving body MV.

  As shown in FIG. 9, the detection device 300B includes an acquisition unit 310B and a first output unit 320B. In addition, the detection device 300 </ b> B includes a second output unit 330.

Additional acquisition unit 310B, the data of the front captured image sent from the imaging unit 210 _F, and receives the data of the side captured image after being sent from the imaging unit 210 _R. Then, the acquisition unit 310B sends the data of the front side captured image and the data of the rear side captured image to the first output unit 320B.

  The first output unit 320B receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. The first output unit 320B detects the pixel movement amount and calibrates the pixel distance when the same calibration condition as that in the first embodiment described above is satisfied. The detected pixel movement amount and the calibrated pixel distance are output to the second output unit 330. Details of the configuration of the first output unit 320B having such a function will be described later.

  Whenever the second output unit 330 receives the pixel movement amount sent from the first output unit 320B, the latest calibrated pixel distance and period time are the same as in the case of the first embodiment described above. Based on TP, the moving body speed is calculated as the traveling information of the moving body MV. Then, the second output unit 330 outputs the calculated moving body speed of the moving body MV to the ECU 230.

<< Configuration of First Output Unit 320B >>
Next, the configuration of the first output unit 320B described above will be described.

  As shown in FIG. 9, the first output unit 320B includes a movement amount detection unit 321B and a specifying unit 322B. The first output unit 320B includes a calibration unit 323B.

  The movement amount detection unit 321B receives the data of the front captured image and the data of the rear captured image sent from the acquisition unit 310B. Then, the movement amount detection unit 321B detects the pixel movement amount by a so-called displacement amount search method based on the data of the front side captured image and the data of the rear side captured image. The pixel movement amount thus detected is sent to the calibration unit 323B and the second output unit 330.

  Details of the operation of the movement amount detection unit 321B will be described later.

  The specifying unit 322B receives the data of the front captured image and the data of the rear captured image sent from the acquisition unit 310B. Then, the specifying unit 322B specifies a white line region in an image obtained from the data of the front captured image and the data of the rear captured image. Information on the white line area thus identified (hereinafter referred to as “white line area information”) is sent to the calibration unit 323B.

  Details of the operation of the specifying unit 322B will be described later.

The calibration unit 323B receives the pixel movement amount sent from the movement amount detection unit 321B and the white line area information sent from the specifying unit 322B. When the calibration unit 323B can determine that the moving body MV is traveling straight on a flat road surface at a constant speed based on the acceleration information, the steering angle information, and the inclination information transmitted from the ECU 230, the calibration unit 323B in addition to the white line area information, based on the distance D between the imaging unit 210 _F and the imaging unit 210 _R described above, to calibrate the pixel distance. The pixel distance thus calibrated is sent to the second output unit 330.

  The details of the operation of the calibration unit 323B will be described later.

<Operation>
Next, the operation of the detection device 300B calibrated as described above will be described by paying attention to processing performed by each element of the detection device 300B.

Note that the imaging units 210 _F and 210 _R have already started operation, and sequentially transmit the data of the front side captured image and the data of the rear side captured image of the road surface image captured at the period time TP to the detection device 300B. It shall be. Further, it is assumed that the ECU 230 has already started operation and has sent acceleration information, steering angle information, and tilt information to the detection device 300B (see FIG. 9).

  Furthermore, the detection apparatus 300B has already performed a plurality of times of pixel distance calibration, and the calibration unit 323B holds a calibration history including a recent predetermined number of calibration times and a provisional pixel distance described later. In addition, it is assumed that the latest calibrated pixel distance is held in the second output unit 330. Here, in the period until the initial calibration of the pixel distance, the average pixel distance is held in the second output unit 330 as in the case of the first embodiment described above. ing.

<< Image data acquisition process >>
In the detection apparatus 300B, the acquisition unit 310B receives the data of the front side captured image and the data of the rear side captured image sent from the imaging units 210 _F and 210 _R. Then, the acquisition unit 310B sends the data of the front side captured image and the data of the rear side captured image to the movement amount detection unit 321B and the specifying unit 322B of the first output unit 320B (see FIG. 9).

<< Pixel movement detection process >>
Next, pixel movement amount detection processing by the movement amount detection unit 321B will be described.

  When receiving the data of the front side captured image sent from the acquisition unit 310B, the movement amount detection unit 321B is obtained from the data of the current front side captured image in the same manner as the detection of the pixel movement amount of the first embodiment described above. A displacement amount of the position of the common feature region in the previous front side captured image obtained from the data of the current front side captured image and the previous front side captured image is detected as a front pixel movement amount. In addition, when receiving the rear captured image data sent from the acquisition unit 310B, the movement amount detection unit 321B uses the current rear captured image data in the same manner as the detection of the pixel movement amount of the first embodiment. A displacement amount of the position of the common feature region in the current rear captured image obtained from the current rear captured image obtained from the previous rear captured image data is detected as a rear pixel movement amount.

  When there are a plurality of feature areas common to the current front captured image and the previous front captured image, the movement amount detection unit 321B uses the average value of the displacement amounts of the plurality of feature areas as the front pixel movement amount. adopt. In addition, when there are a plurality of feature areas common to the current rear side captured image and the previous rear side captured image, the movement amount detection unit 321B determines the average value of the displacement amounts of the plurality of feature areas as the rear pixel. Adopted as the amount of movement.

  Subsequently, the movement amount detection unit 321B calculates the average of the front pixel movement amount and the rear pixel movement amount as the pixel movement amount. Then, the movement amount detection unit 321B sends the calculated pixel movement amount to the calibration unit 323B and the second output unit 330 (see FIG. 9).

<< Output processing of moving body speed v >>
Next, the output process of the moving body speed v by the second output unit 330 will be described.

  Upon receiving the pixel movement amount sent from the movement amount detection unit 321B, the second output unit 330 performs the pixel movement amount and the held pixel distance (that is, the latest calibration) in the same manner as in the first embodiment. The moving body speed v is calculated based on the pixel distance) and the period time TP. Then, the second output unit 330 outputs the calculated moving body speed v to the ECU 230.

《White line area identification processing》
Next, the white line area specifying process by the specifying unit 322B will be described.

  The white line region specifying process is performed for the calibration of the pixel distance by the calibration unit 323B described later. Such a pixel distance calibration environment is shown in FIG. As shown in FIG. 11, the calibration of the pixel distance is performed when the moving body MV crosses a white line region LCP such as a pedestrian crossing sign on a flat road surface by a constant speed straight line traveling.

  In the following description, it is assumed that the white line region LCP used for calibration of a new pixel distance is a white line region from the traveling direction position (that is, the X direction position) XR to the traveling direction position XP. .

When receiving the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B, the specifying unit 322B determines the white line area in the front side captured image and the rear side captured image as the brightness of each pixel in the image. The identification result is sent to the calibration unit 323B as white line area information (see FIG. 9). Such white line region identification results include two types: a “first rear partial white line” shown in FIG. 12A and a “second rear partial white line” shown in FIG. . Here, "first rear part white line" is a state of the rear part white line in front captured image obtained by the imaging by the imaging section 210 _F of the front installation. Further, "second rear part white line", according to the imaging unit 210 _R of the rear installation is the state of the rear part white line on the side captured image after being obtained by the imaging.

  In the second embodiment, the specifying unit 322B is configured on the condition that a state in which the white line until the “first rear partial white line” can be clearly specified when passing the white line in the front captured image has continued. White line area information is generated. For this reason, when the white line cannot be clearly specified due to the tire marks or the like, the specifying unit 322B does not generate the white line region information. That is, in the second embodiment, the pixel distance is not calibrated when the white line cannot be clearly specified due to tire marks or the like.

  In the second embodiment, the white line region is specified by focusing on the first rear partial white line in the front captured image and the second rear partial white line in the rear captured image. It also depends on the fact that multiple white lines are likely to cause errors in measurement. By carrying out such a device, it is not necessary to perform useless calibration, so that uselessness can be eliminated.

  Note that when the identification result is the first rear partial white line in the front-side captured image, the identification unit 322B determines the length a in the X direction of the non-white line region in the image (hereinafter also referred to as “front length a”). Identify. Further, when the specifying result is the second rear partial white line in the rear captured image, the specifying unit 322B determines the length b in the X direction of the non-white line region in the image (hereinafter also referred to as “front length b”). Is identified.

  Then, in the case of the first rear partial white line in the front-side captured image, the specifying unit 322B displays the white line area information [first rear partial white line flag: ON, front length: a] illustrated in FIG. The data is sent to the calibration unit 323B (see FIG. 9). Further, in the case of the second rear partial white line, the specifying unit 322B sends the white line area information [second rear partial white line flag: ON, front length: b] illustrated in FIG. 12B to the calibration unit 323B. Send (see FIG. 9).

<< Pixel distance calibration process >>
Next, calibration processing by the calibration unit 323B will be described.

  In the calibration process, whether or not the calibration unit 323B can determine that the moving body MV is traveling straight at a constant speed on a flat road surface based on the acceleration information, the steering angle information, and the tilt information transmitted from the ECU 230, and the like. Performs a straight-ahead determination.

When the result of the constant-velocity straight-ahead determination is affirmative, the calibration unit 323B determines whether the white line region information in the front-side captured image sent from the specifying unit 322B is “first rear partial white line”. The white line end determination is performed. If the result of the front white line end determination is affirmative, the calibration unit 323B is sent from the specifying unit 322B on the condition that the result of the constant speed straight traveling determination on a flat road surface is maintained positive. The pixel movement amount sent from the movement amount detection unit 321B until the result of the determination of the end of the rear white line whether or not the white line region information in the rear captured image becomes “second rear partial white line” is affirmative. ΔX ₁ ,..., ΔX _N are collected (see FIG. 13).

When the result of the determination of the end of the rear white line becomes affirmative, the calibration unit 323B determines that the front length a included in the white line area information at the time when the white line area information in the front captured image becomes the “first rear partial white line”. , collected pixel shift amount [Delta] X _1, ..., [Delta] X _N, and the front length is included in the white line area information at the time the white line area information becomes "second rear part white line" in the rear captured image b, and , based on the distance D between the imaging unit 210 _F and the imaging unit 210 _R, the interim pixel distance PT [cm], it is calculated by the following equation (10).
PT = D / (a + ΔX ₁ +... + ΔX _N −b) (10)

  Next, the calibration unit 323B calculates the weighted average of the calculated current provisional pixel distance and the provisional pixel distance calculated during the predetermined number of calibrations held in the past, and is calibrated this time. Calculate the pixel distance. Thereafter, the calibration unit 323B retains the current provisional pixel distance inside instead of the oldest provisional pixel distance retained therein.

  In the weighted average, as in the case of the first embodiment, the weight is decreased as the elapsed time up to the present is longer.

  The calibration unit 323B sends the calculated pixel distance to the second output unit 330. As a result, the second output unit 330 calculates the moving body speed v using the new pixel distance.

As described above, in the second embodiment, imaging of a white line that is a feature on the road surface by the imaging units 210 _F and 210 _R arranged on the moving body MV at a known distance D along the traveling direction of the moving body MV. Based on the image, the movement amount detection unit 321B detects the pixel movement amount. Further, the specifying unit 322B specifies a white line region in the captured image. Then, the calibration unit 323B can determine that the moving body MV is traveling straight at a constant speed on a flat road surface based on the known distance D in addition to the pixel movement amount and the identification result of the white line region in the captured image. In this case, the distance on the road surface corresponding to one pixel related to the distance between the imaging units 210 _F and 210 _R and the road surface is calibrated.

  Therefore, according to the second embodiment, regardless of whether the length along the traveling direction of the moving body is known or unknown, the white line that is the road surface characteristic portion is changed in the moving body weight or the air pressure of the wheel. Even if the distance between the road surface and the imaging unit changes due to the change, the distance on the road surface corresponding to one pixel can be calibrated according to the change.

  In the second embodiment, the second output unit calculates the moving body speed based on the calibration result and the pixel movement amount. For this reason, it is possible to output a moving body speed with high accuracy as travel information.

  Moreover, in 2nd Embodiment, a white line is employ | adopted as a road surface characteristic part. For this reason, the white line region can be easily specified by examining the brightness of each pixel of the captured image.

  In the second embodiment, as in the case of the first embodiment, a weighted average of the current provisional pixel distance and the provisional pixel distance calculated during a predetermined number of recent calibrations in the past is calculated, and the final A typical calibration result. For this reason, the calibration which suppressed the influence on the calibration result of pixel distance, such as the fluctuation | variation of the length of the advancing direction of a white line, can be performed.

  In the second embodiment, as in the case of the first embodiment, calibration is performed when the vehicle is traveling straight, traveling at a constant speed, and having no road surface inclination. For this reason, accurate pixel distance calibration can be performed.

[Modification of Embodiment]
The present invention is not limited to the first and second embodiments described above, and various modifications are possible.

  For example, in the first embodiment described above, the stop line before the intersection is adopted as the road surface characteristic portion. However, if the road marking has a known length along the traveling direction of the moving body, These types of road markings may be employed as road surface features.

  In the second embodiment, a white line such as a pedestrian crossing is used as the road surface feature. However, a road marking other than the white line may be used as the road surface feature.

  In the second embodiment, the average of the front pixel movement amount obtained from the temporal change of the front image and the rear pixel movement amount obtained from the temporal change of the rear image is detected as the pixel movement amount. did. On the other hand, one of the front pixel movement amount and the rear pixel movement amount may be detected as the pixel movement amount.

  Moreover, in said 2nd Embodiment, based on the change of the captured image from the time of a front side captured image becoming a 1st back partial white line until the time of a back side captured image becoming a 2nd back partial white line, The pixel distance was calibrated. On the other hand, the pixel distance is calibrated based on the change in the captured image from the time when the front captured image becomes the first front partial white line to the time when the rear captured image becomes the second front partial white line. It may be.

  In the first and second embodiments, the pixel movement amount is detected by a so-called displacement amount search method. However, the pixel movement amount may be detected by an image correlation method or a spatial filter method.

  In the first and second embodiments described above, the pixel distance, which is the distance on the road surface corresponding to one pixel, is calibrated as distance related information related to the distance between the imaging unit and the road surface. On the other hand, the image element distance which is the distance on the road surface corresponding to the length in the traveling direction of the predetermined image element which is a set of a plurality of pixels may be calibrated.

  In the first and second embodiments described above, the second output unit outputs the moving body speed (traveling speed) as the traveling information. However, instead of the moving body speed or in addition to the moving body speed. Thus, the travel distance may be output.

  In the first and second embodiments, the pixel distance is calibrated by calculating a weighted average of the current provisional pixel distance and a predetermined number of provisional pixel distances in the past. On the other hand, the temporary pixel distance of this time may be used as it is as the calibrated pixel distance.

  In the first and second embodiments described above, it is assumed that the imaging unit is prepared outside the detection device. However, the detection device may include the imaging unit.

  In the first and second embodiments, the detection device is mounted on the moving body. On the other hand, the detection device may be installed outside the moving body and connected to an imaging unit or ECU mounted on the vehicle via a wireless communication network.

  In addition, the detection apparatus of said 1st and 2nd embodiment is comprised as a computer as a calculation means provided with the central processing unit (CPU: Central Processing Unit) etc., and the program prepared beforehand is run with the said computer Accordingly, some or all of the functions of the detection devices of the first and second embodiments described above may be executed. This program is recorded on a computer-readable recording medium such as a hard disk, CD-ROM, or DVD, and is loaded from the recording medium and executed by the computer. The program may be acquired in a form recorded on a portable recording medium such as a CD-ROM or DVD, or may be acquired in a form distributed via a network such as the Internet. Also good.

  Next, an embodiment of the present invention will be described with reference mainly to FIGS. In the following description, including the above-described embodiment, the same or equivalent elements are denoted by the same reference numerals, and redundant description is omitted as much as possible.

[First embodiment]
First, a first embodiment of the present invention will be described with reference mainly to FIGS.

<Configuration>
FIG. 14 schematically shows the configuration of the detection apparatus 100A according to the first embodiment. This detection apparatus 100A is an aspect of the detection apparatus 300A according to the first embodiment described above.

  As illustrated in FIG. 14, the detection device 100 </ b> A is connected to the imaging unit 210, the navigation device 220, and the ECU 230. The detection device 100A, the imaging unit 210, the navigation device 220, and the ECU 230 are mounted on the vehicle CR.

  The detection device 100A includes a control unit 110A. Further, the detection apparatus 100A includes a storage unit 120.

  The control unit 110A includes a central processing unit (CPU) as a calculation unit. The control unit 110A executes a program so as to perform the function as the detection device 300A in the first embodiment, that is, the function as the acquisition unit 310A, the first output unit 320A, and the second output unit 330. It has become.

  The program executed by the control unit 110A is stored in the storage unit 120, loaded from the recording unit, and executed. This program may be acquired in the form recorded on a portable recording medium such as a CD-ROM or DVD, or may be acquired in the form of distribution via a network such as the Internet. .

  The processing executed by the control unit 110A will be described later.

  The storage unit 120 stores various information data used by the control unit 110A. Such information data includes a program executed by the control unit 110A, the latest pixel movement amount, a calibration history including the latest pixel distance, and the like. The storage unit 120 can be accessed by the control unit 110A.

<Operation>
Next, the detection operation performed by the detection apparatus 100A configured as described above will be described focusing on the processing performed by the control unit 110A.

  It is assumed that the imaging unit 210 has already started the operation, and sequentially sends the captured image data of the road surface image captured at the cycle time TP to the detection device 100A. Further, it is assumed that the navigation device 220 has already started to operate, and when the vehicle CR is within a predetermined distance from an intersection existing in the traveling direction, the navigation device 220 is configured to send a message to that effect to the detection device 100A. Further, it is assumed that the ECU 230 has already started operation and has sent acceleration information, steering angle information, and tilt information to the detection device 100A (see FIG. 14).

  Furthermore, in the detection apparatus 100A, the calibration of the pixel distance is already performed a plurality of times, and the storage unit 120 stores the calibration time for the most recent predetermined number of times, the characteristics of the feature region extracted from the previous captured image, and It is assumed that a position in the image and a calibration history including a provisional pixel distance described later are stored. Here, in the period until the initial calibration of the pixel distance is performed, the average pixel distance is stored in the storage unit 120.

<< Pixel movement detection process >>
First, the pixel movement amount detection process by the control unit 110A will be described.

  In the pixel movement amount detection process, as shown in FIG. 15, first, in step S11, the control unit 110A determines whether or not new captured image data has been received. If the result of the determination in step S11 is negative (step S11: N), the process of step S11 is repeated.

  When new captured image data is received and the result of determination in step S11 is affirmative (step S11: Y), the process proceeds to step S12. In step S12, the control unit 110A extracts a feature region in a captured image (current captured image) obtained from new captured image data, and specifies the characteristics of the feature region and the position in the image.

  Next, in step S13, the control unit 110A detects the pixel movement amount. When detecting the amount of pixel movement, the control unit 110A refers to the characteristics of the feature region and the position in the image extracted from the previous captured image in the storage unit 120, and determines the current captured image and the previous captured image. A displacement amount of the position in the image of the common feature region is detected as a new pixel movement amount.

  When there are a plurality of common feature areas, the control unit 110A employs an average value of the displacement amounts of the plurality of feature areas as the pixel movement amount.

  Then, the control unit 110A registers the characteristics of the feature region and the position in the image in the current captured image in the storage unit 120 as the characteristics of the feature region and the position in the image extracted from the previous captured image. In addition, the control unit 110A registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

  When the process of step S13 is thus completed, the process returns to step S11. Thereafter, steps S11 to S13 are repeated, and each time a new captured image data is received, the pixel movement amount is detected.

<< Output processing of body speed v >>
Next, the output process of the vehicle body speed v by the control unit 110A will be described.

  In the output processing of the vehicle body speed v, as shown in FIG. 16, first, in step S21, the control unit 110A determines whether or not a new pixel movement amount has been detected. If the result of the determination in step S21 is negative (step S21: N), the process of step S21 is repeated.

  When a new pixel movement amount is detected and the result of determination in step S21 is affirmative (step S21: Y), the process proceeds to step S22. In this step S22, the vehicle body speed v is calculated by the above-described equation (7) based on the latest pixel movement amount (PN) and the latest pixel distance (PL) registered in the storage unit 120. Then, control unit 110 </ b> A outputs calculated vehicle body speed v to ECU 230.

  When the process of step S22 is thus completed, the process returns to step S21. Thereafter, the processes of steps S21 and S22 are repeated, and each time a new pixel movement amount is detected, that is, every time new imaging data is received, a new vehicle speed v is calculated, and the calculated vehicle speed v Is output to the ECU 230.

《White line area identification processing》
Next, the white line area specifying process by the control unit 110A will be described. The white line region specifying process is executed in parallel with the pixel movement amount detection process and the vehicle body speed v output process described above.

  The white line area specifying process is performed for the pixel distance calibration described later. The calibration environment for the calibration of the pixel distance is the same as that shown in FIG.

  In the white line region specifying process, as shown in FIG. 17, first, in step S31, the control unit 110A determines whether or not new captured image data has been received. If the result of the determination in step S31 is negative (step S31: N), the process of step S31 is repeated.

  When new captured image data is received and the result of determination in step S31 is affirmative (step S31: Y), the process proceeds to step S32. In step S32, the control unit 110A specifies the white line region in the captured image obtained from the new captured image data based on the brightness of each pixel in the captured image.

  Next, in step S33, the control unit 110A determines whether or not the result of specifying the white line area is “front partial white line (see FIG. 7A)”. If the determination result of step S33 is negative (step S33: N), the process returns to step S31. And the process of step S31-S33 is repeated until the result of determination of step S33 becomes affirmative.

  When the white line region specifying result is “front partial white line” and the result of determination in step S33 is affirmative (step S33: Y), the process proceeds to step S34. In step S34, the control unit 110A calculates the front length a (see FIG. 7A). Then, the process proceeds to step S35.

  Next, in step S35, the control unit 110A determines whether or not new captured image data has been received. If the result of the determination in step S35 is negative (step S35: N), the process of step S35 is repeated.

  When new captured image data is received and the determination result in step S35 is affirmative (step S35: Y), the process proceeds to step S36. In step S36, the control unit 110A specifies the white line region in the captured image obtained from the new captured image data based on the brightness of each pixel in the captured image.

  Next, in step S <b> 37, the control unit 110 </ b> A determines whether or not the white line region specifying result is “rear part white line (see FIG. 7B)”. If the result of the determination in step S37 is negative (step S37: N), the process returns to step S35. And the process of step S35-S37 is repeated until the result of determination of step S37 becomes affirmative.

  When the white line region specifying result is “backward partial white line” and the result of determination in step S37 is affirmative (step S37: Y), the process proceeds to step S38. In step S38, the control unit 110A calculates the front length b (see FIG. 7B). Then, the process returns to step S31.

  Thereafter, steps S31 to S38 are repeated. As a result, each time the vehicle CR crosses the white line, the white line region is specified.

<< Monitoring process for calibration conditions >>
Next, calibration condition monitoring processing by the control unit 110A will be described. The monitor process of the calibration condition is executed in parallel with the pixel movement amount detection process, the vehicle body speed v output process, and the white line area specifying process described above.

  In the calibration condition monitoring process, as shown in FIG. 18, first, in step S41, it is determined based on the latest steering angle information sent from the ECU 230 whether or not the vehicle CR is traveling straight ahead. If the result of the determination in step S41 is negative (step S41: N), the process proceeds to step S44 described later.

  If the result of the determination in step S41 is affirmative (step S41: Y), the process proceeds to step S42. In step S42, the control unit 110A determines whether the vehicle CR is accelerating or decelerating based on the latest acceleration information sent from the ECU 230. If the determination result in step S42 is affirmative (step S42: Y), the process proceeds to step S44.

  If the result of the determination in step S42 is negative (step S42: N), the process proceeds to step S43. In step S43, the control unit 110A determines whether or not there is a vehicle body inclination of the vehicle CR based on the latest inclination information sent from the ECU 230.

  If the result of the determination in step S43 is affirmative (step S43: Y), the process proceeds to step S44. In step S44, the control unit 110A sets the calibration condition flag to “OFF”. Then, the process returns to step S41.

  On the other hand, when the result of the determination in step S43 is negative (step S43: N), the process proceeds to step S45. In step S45, the control unit 110A sets the calibration condition flag to “ON”. Then, the process returns to step S41.

  Thereafter, the processes of steps S41 to S44 are repeated. As a result, when it is determined that the calibration condition that the vehicle CR is traveling straight at a constant speed on a flat road surface is satisfied, the calibration condition flag is set to “ON”. On the other hand, when it is determined that the calibration condition is not satisfied, the calibration condition flag is “OFF”.

<< Pixel distance calibration process >>
Next, a pixel distance calibration process by the control unit 110A will be described. The pixel distance calibration process is executed in parallel with the pixel movement amount detection process, the vehicle body speed v output process, the white line region specifying process, and the calibration condition monitoring process.

  In the pixel distance calibration process, as shown in FIG. 19, first, in step S50, whether or not the control unit 110A receives from the navigation device 220 that it is within a predetermined distance from an intersection existing in the traveling direction. It is determined whether or not the intersection is approached. If the result of the determination in step S50 is negative (step S50: N), the process of step S50 is repeated.

  When the vehicle approaches the intersection and the determination result in step S50 is affirmative (step S50: Y), the process proceeds to step S51. In step S51, the control unit 110A determines whether or not the vehicle CR is traveling straight at a constant speed on a flat road surface by determining whether or not the calibration condition flag is “ON”.

  If the result of the determination in step S51 is negative (step S51: N), the process returns to step S50. Then, the processes of steps S50 and S51 are repeated.

  If the result of the determination in step S51 is affirmative (step S51: Y), the process proceeds to step S52. In this step S52, the control unit 110A determines whether or not the “front partial white line” has been specified by the white line region specifying process described above and the front length a has been obtained.

  If the result of the determination in step S51 is negative (step S52: N), the process returns to step S51. And the process of step S51, S52 is repeated on the condition that the result of determination in step S51 is not negative.

  If the result of the determination in step S52 is affirmative (step S52: Y), the process proceeds to step S53. In this step S53, the distance parameter L is set to the front length a.

  Next, in step S54, the control unit 110A determines whether or not the vehicle CR is traveling straight ahead at a constant speed on a flat road surface by determining whether or not the calibration condition flag is “ON”. If the result of the determination in step S54 is negative (step S54: N), the process returns to step S50.

If the result of the determination in step S54 is affirmative (step S54: Y), the process proceeds to step S55. In step S55, the control unit 110A determines whether or not a new pixel movement amount (ΔX _j ) has been detected by the pixel movement amount detection process described above.

  If the result of the determination in step S55 is negative (step S55: N), the process returns to step S54. And the process of step S54, S55 is repeated on condition that the result of determination in step S54 is not negative.

If the result of determination in step S55 is affirmative (step S55: Y), the process proceeds to step S56. In step S56, the control unit 110A adds the new pixel movement amount (ΔX _j ) to the value of the distance parameter L so far, and updates the value of the distance parameter L.

  Next, in step S57, the control unit 110A determines whether or not the “rear part white line” has been specified by the white line region specifying process described above, and the front length b is obtained. If the result of the determination in step S57 is negative (step S57: N), the process returns to step S54. And the process of step S54-S57 is repeated on condition that the result of determination in step S54 is not negative.

  If the result of determination in step S57 is affirmative (step S57: Y), the process proceeds to step S58. In this step S58, the control unit 110A updates the value of the distance parameter L by subtracting the forward length b from the value of the distance parameter L so far.

Next, in step S59, the control unit 110A calculates the provisional pixel distance PT [cm] by the following equation (11) based on the value of the distance parameter L at that time.
PT = 45 / L (11)
Note that equation (11) is equivalent to equation (8) described above.

  Subsequently, the control unit 110A calculates a weighted average between the calculated current provisional pixel distance and the provisional pixel distance calculated during a predetermined number of calibrations stored in the storage unit 120 in the past, and newly Calculate the calibrated pixel distance. Thereafter, the control unit 110A updates the latest pixel distance in the storage unit 120 to the newly calibrated pixel distance, and the oldest provisional pixel distance stored in the storage unit 120 until then. Instead of this, the current provisional pixel distance is stored in the storage unit 120 together with the current time. As a result, in the above-described output processing of the vehicle body speed v, the control unit 110A calculates the vehicle body speed v using the new pixel distance.

  When the process in step S59 ends in this way, the process returns to step S50. Then, the processes in steps S50 to S59 are repeated. As a result, each time the vehicle CR travels straight on a flat road at a constant speed and crosses a stop line (white line) before the intersection, the pixel distance is calibrated.

  As described above, in the first embodiment, the control unit 110A detects the pixel movement amount based on the image captured by the imaging unit 210 of the vehicle CR. In addition, the control unit 110A specifies a white line region in the captured image. Then, in addition to the pixel movement amount and the result of specifying the white line area in the captured image, the control unit 110A determines that the vehicle CR has a flat road surface based on the known length of the stop line along the traveling direction of the vehicle CR. When it can be determined that the vehicle is traveling straight at a constant speed, the distance on the road surface corresponding to one pixel related to the distance between the imaging unit 210 and the road surface is calibrated.

  Therefore, according to the first embodiment, even if the distance between the road surface and the imaging unit changes due to a change in vehicle weight or a change in wheel air pressure, the distance on the road surface corresponding to one pixel is changed according to the change. Can be calibrated.

  In the first embodiment, the control unit 110A calculates the vehicle body speed based on the calibration result and the pixel movement amount. For this reason, it is possible to output an accurate vehicle body speed as travel information.

  In the first embodiment, a stop line is adopted as the road surface feature. For this reason, the stop line region (that is, the white line region) can be easily specified by examining the brightness of each pixel of the captured image.

  In the first embodiment, information indicating that the vehicle is approaching the intersection is acquired, and it is estimated in advance that the vehicle will pass the stop line. For this reason, efficient calibration can be performed.

  In the first embodiment, a final calibration result is obtained by calculating a weighted average between the current provisional pixel distance and the provisional pixel distance calculated during a predetermined number of calibrations in the past. For this reason, the calibration which suppressed the influence on the calibration result of pixel distance, such as a fluctuation | variation of the length of the advancing direction of a stop line, can be performed.

  Further, in the first embodiment, calibration is performed when the vehicle is traveling straight, traveling at a constant speed, and having no road surface inclination. For this reason, accurate pixel distance calibration can be performed.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference mainly to FIGS.

<Configuration>
FIG. 20 schematically shows the configuration of the detection apparatus 100B according to the first example. This detection device 100B is an aspect of the detection device 300B according to the second embodiment described above.

As shown in FIG. 20, the detection device 100 </ b> B is connected to the imaging units 210 _F and 210 _R and the ECU 230. The detection device 100B, the imaging units 210 _F and 210 _R, and the ECU 230 are mounted on the vehicle CR.

  The detection device 100B includes a control unit 110B. In addition, the detection apparatus 100B includes a storage unit 120.

  The control unit 110B includes a central processing unit (CPU) as a calculation unit. The control unit 110B executes a function of the detection device 300B in the second embodiment described above, that is, functions as the acquisition unit 310B, the first output unit 320B, and the second output unit 330 by executing a program. It has become.

  The program executed by the control unit 110B is stored in the storage unit 120, loaded from the recording unit, and executed. This program may be acquired in the form recorded on a portable recording medium such as a CD-ROM or DVD, or may be acquired in the form of distribution via a network such as the Internet. .

  The processing executed by the control unit 110B will be described later.

  The storage unit 120 stores various information data used by the control unit 110B. Such information data includes a program executed by the control unit 110B, the latest pixel movement amount, a calibration history including the latest pixel distance, and the like. The storage unit 120 can be accessed by the control unit 110B.

<Operation>
Next, the detection operation by the detection apparatus 100B configured as described above will be described focusing on the processing by the control unit 110B.

Note that the imaging units 210 _F and 210 _R have already started operation, and sequentially transmit the data of the front side captured image and the data of the rear side captured image of the road surface image captured at the period time TP to the detection apparatus 100B. It shall be. Further, it is assumed that the ECU 230 has already started operation and has sent acceleration information, steering angle information, and tilt information to the detection apparatus 100B (see FIG. 20).

  Further, in the detection apparatus 100B, the calibration of the pixel distance is already performed a plurality of times, and the storage unit 120 stores the calibration time for the most recent predetermined number of times, the characteristics of the feature region extracted from the previous captured image, and It is assumed that a position in the image and a calibration history including a provisional pixel distance described later are stored. Here, in the period until the initial calibration of the pixel distance is performed, the average pixel distance is stored in the storage unit 120.

<< Pixel movement detection process >>
First, the pixel movement amount detection process by the control unit 110B will be described.

  In the pixel movement amount detection process, as shown in FIG. 21, first, in step S61, it is determined whether or not the control unit 110B has received new front-side captured image data and new rear-side captured image data. judge. If the result of the determination in step S61 is negative (step S61: N), the process of step S61 is repeated.

  When new front-side captured image data is received and the result of determination in step S61 is affirmative (step S61: Y), the process proceeds to step S62. In step S62, the control unit 110B extracts a feature region in the front captured image obtained from the data of the new front captured image, and specifies the characteristics of the feature region and the position in the image.

  Next, in step S63, the control unit 110B detects the front pixel movement amount. When detecting the front pixel movement amount, the control unit 110B refers to the characteristics of the feature region and the position in the image extracted from the previous front side captured image in the storage unit 120, and the current front side captured image and the previous front side A displacement amount of the position in the image of the common feature region in the captured image is detected as a new front pixel movement amount.

  When there are a plurality of common feature areas, the control unit 110B employs the average value of the displacement amounts of the plurality of feature areas as a new front pixel movement amount.

  Then, the control unit 110B registers the characteristics of the feature region and the position in the image in the current captured image in the storage unit 120 as the characteristics of the feature region and the position in the image extracted from the previous captured image. Also, the control unit 110B registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

  Next, in step S64, the control unit 110B extracts the feature region in the rear captured image obtained from the data of the new rear captured image, and specifies the characteristics of the feature region and the position in the image.

  Next, in step S65, the control unit 110B detects the rear pixel movement amount. When detecting the rear pixel movement amount, the control unit 110B refers to the characteristics of the feature region and the position in the image extracted from the previous rear captured image in the storage unit 120, and determines the current rear captured image and The displacement amount of the position in the image of the common feature area in the previous rear captured image is detected as a new rear pixel movement amount.

  Note that when there are a plurality of common feature areas, the control unit 110B determines the average value of the displacement amounts of the plurality of feature areas as a new rear pixel movement, as in the case of detecting the front pixel movement amount. Adopt as a quantity.

  Next, in step S66, the control unit 110B detects a new pixel movement amount. In detecting the pixel movement amount, the control unit 110B detects a new pixel movement amount by calculating an average of the front pixel movement amount and the rear pixel movement amount.

  Then, the control unit 110B determines the characteristics of the feature region and the position in the image of the current front-side captured image, and the characteristics and characteristics of the feature region in the current rear-side captured image of the feature region in the previous front-side captured image. The characteristics and the position in the image, and the characteristics of the characteristic region and the position in the image in the previous rear side captured image are registered in the storage unit 120. Also, the control unit 110B registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

  When the process of step S66 is thus completed, the process returns to step S61. Thereafter, steps S61 to S66 are repeated, and the pixel movement amount is detected each time new front-side captured image data and new rear-side captured image data are received.

<< Output processing of body speed v >>
The output process of the vehicle body speed v by the control unit 110B is the same as the output process of the vehicle body speed v by the control unit 110A described above (see FIG. 16).

《White line area identification processing》
Next, the white line area specifying process by the control unit 110B will be described. The white line region specifying process is executed in parallel with the pixel movement amount detection process and the vehicle body speed v output process described above.

  The white line area specifying process is performed for the pixel distance calibration described later. The calibration environment for the calibration of the pixel distance is the same as that shown in FIG.

  In the white line region specifying process, as shown in FIG. 22, first, in step S71, the control unit 110B determines whether or not new front-side captured image data has been received. When the result of the determination in step S71 is negative (step S71: N), the process of step S71 is repeated.

  When new front-side captured image data is received and the result of determination in step S71 is affirmative (step S71: Y), the process proceeds to step S72. In step S72, the control unit 110B specifies the white line region in the front captured image obtained from the data of the new front captured image based on the brightness of each pixel in the front captured image.

  Next, in step S <b> 73, the control unit 110 </ b> B determines whether or not the white line region specifying result is “first rear partial white line (see FIG. 12A)”. If the result of the determination in step S73 is negative (step S73: N), the process returns to step S71. And the process of step S71-S73 is repeated until the result of determination of step S73 becomes affirmative.

  If the result of specifying the white line area in the front captured image is “first rear partial white line” and the result of determination in step S73 is affirmative (step S73: Y), the process proceeds to step S74. In step S74, the control unit 110B calculates the front length a (see FIG. 12A).

  In the second embodiment, the control unit 110B performs the “first rear part” on the condition that the white line until the “first rear part white line” can be clearly identified in the front captured image has continued. "White line" is specified.

  Next, in step S75, the control unit 110B determines whether or not new rear captured image data has been received. If the result of the determination in step S75 is negative (step S75: N), the process of step S75 is repeated.

  If new rear captured image data is received and the result of the determination in step S75 is affirmative (step S75: Y), the process proceeds to step S76. In step S76, the control unit 110B specifies the white line region in the rear captured image obtained from the data of the new rear captured image based on the brightness of each pixel in the captured image.

  Next, in step S77, the control unit 110B determines whether or not the white line region specifying result is “second rear partial white line (see FIG. 12B)”. If the result of the determination in step S77 is negative (step S77: N), the process returns to step S75. And the process of step S75-S77 is repeated until the result of determination of step S77 becomes affirmative.

  If the result of specifying the white line region in the rear captured image is “second rear partial white line” and the result of determination in step S77 is affirmative (step S77: Y), the process proceeds to step S78. In step S78, the control unit 110B obtains the front length b (see FIG. 12B). Then, the process returns to step S71.

  Thereafter, steps S71 to S78 are repeated. As a result, each time the vehicle CR crosses the white line, the white line region is specified.

<< Monitoring process for calibration conditions >>
The calibration condition monitoring process by the control unit 110B is similar to the calibration condition monitoring process by the control unit 110A described above (see FIG. 18).

<< Pixel distance calibration process >>
Next, a pixel distance calibration process by the control unit 110B will be described. The pixel distance calibration process is executed in parallel with the pixel movement amount detection process, the vehicle body speed v output process, the white line region specifying process, and the calibration condition monitoring process.

  In the calibration process of the pixel distance, as shown in FIG. 23, first, in step S81, the control unit 110B determines whether or not the calibration condition flag is “ON”, so that the vehicle CR is flat. It is determined whether or not the vehicle is traveling straight ahead at a constant speed. If the result of the determination in step S81 is negative (step S81: N), the process of step S81 is repeated.

  If the result of the determination in step S81 is affirmative (step S81: Y), the process proceeds to step S82. In step S82, the control unit 110B determines whether or not the front-side captured image is “first rear partial white line” by the white line region specifying process described above, and the front length a is obtained.

  If the result of the determination in step S82 is negative (step S82: N), the process returns to step S81. Then, the processes of steps S81 and S82 are repeated.

  If the result of the determination in step S82 is affirmative (step S82: Y), the process proceeds to step S83. In this step S83, the distance parameter L is set to the front length a.

  Next, in step S84, the control unit 110B determines whether or not the vehicle CR is traveling straight at a constant speed on a flat road surface by determining whether or not the calibration condition flag is “ON”. If the result of the determination in step S84 is negative (step S84: N), the process returns to step S81.

If the result of the determination in step S84 is affirmative (step S84: Y), the process proceeds to step S85. In step S85, the control unit 110B determines whether or not a new pixel movement amount (ΔX _j ) has been detected by the pixel movement amount detection process described above.

  If the result of the determination in step S85 is negative (step S85: N), the process returns to step S84. And the process of step S84, S85 is repeated on the condition that the result of determination in step S84 is not negative.

If the result of the determination in step S85 is affirmative (step S85: Y), the process proceeds to step S86. In this step S86, the control unit 110B adds the new pixel movement amount (ΔX _j ) to the value of the distance parameter L so far, and updates the value of the distance parameter L.

  Next, in step S87, the control unit 110B determines whether the rear side captured image is “second rear partial white line” by the white line region specifying process described above, and whether the front length b is obtained. judge. If the result of the determination in step S87 is negative (step S87: N), the process returns to step S84. And the process of step S84-S87 is repeated on the condition that the result of determination in step S84 is not negative.

  If the result of the determination in step S87 is affirmative (step S87: Y), the process proceeds to step S88. In step S88, the control unit 110B updates the value of the distance parameter L by subtracting the forward length b from the value of the distance parameter L so far.

Next, in step S89, the control unit 110B calculates a provisional pixel distance PT [cm] by the following equation (12) based on the value of the distance parameter L at that time.
PT = D / L (12)
Note that equation (12) is equivalent to equation (10) described above.

  Subsequently, the control unit 110B calculates a weighted average between the calculated current provisional pixel distance and the provisional pixel distance calculated during the predetermined number of calibrations stored in the storage unit 120 in the past. Calculate the calibrated pixel distance. Thereafter, the control unit 110B updates the latest pixel distance in the storage unit 120 to the newly calibrated pixel distance, and the oldest provisional pixel distance stored in the storage unit 120 until then. Instead of this, the current provisional pixel distance is stored in the storage unit 120 together with the current time. As a result, in the above-described output process of the vehicle body speed v, the control unit 110B calculates the vehicle body speed v using the new pixel distance.

  When the process in step S89 ends in this way, the process returns to step S81. And the process of step S81-S89 is repeated. As a result, each time the vehicle CR travels straight on a flat road at a constant speed and crosses the white line before the intersection, the pixel distance is calibrated.

As described above, in the second embodiment, the control unit 110B is based on the captured images by the imaging units 210 _F and 210 _R arranged in the vehicle CR at a known distance D along the traveling direction of the vehicle CR. Detect the amount of pixel movement. In addition, the control unit 110B specifies a white line region in the captured image. Then, when the control unit 110B can determine that the vehicle CR is traveling straight at a constant speed on a flat road surface based on the known distance D in addition to the pixel movement amount and the white line region identification result in the captured image. In addition, the distance on the road surface corresponding to one pixel related to the distance between the imaging units 210 _F and 210 _R and the road surface is calibrated.

  Therefore, according to the second embodiment, regardless of whether or not the length along the traveling direction of the moving body of the white line that is the feature on the road surface is known and unknown, the change in the weight of the moving body and the wheel Even if the distance between the road surface and the imaging unit changes due to the change in air pressure, the distance on the road surface corresponding to one pixel can be calibrated according to the change.

  In the second embodiment, the control unit 110B outputs the vehicle speed based on the calibration result and the pixel movement amount. For this reason, it is possible to output an accurate vehicle body speed as travel information.

  In the second embodiment, a white line is used as the road surface feature. For this reason, the white line region can be easily specified by examining the brightness of each pixel of the captured image.

  In the second embodiment, as in the case of the first embodiment, a weighted average of the current provisional pixel distance and the provisional pixel distance calculated during a predetermined number of calibrations in the past is calculated to obtain a final result. Get the calibration result. For this reason, the calibration which suppressed the influence on the calibration result of pixel distance, such as the fluctuation | variation of the length of the advancing direction of a white line, can be performed.

  In the second embodiment, as in the first embodiment, calibration is performed when the vehicle is traveling straight, traveling at a constant speed, and having no road surface inclination. For this reason, accurate pixel distance calibration can be performed.

[Modification of Example]
The present invention is not limited to the first and second embodiments described above, and various modifications are possible.

  For example, the same modification as the modification of the first embodiment described above can be performed on the first example. Further, the second example can be modified in the same manner as the modification of the second embodiment described above.

100A, 100B ... Detection device 110A, 110B ... Control unit (acquisition unit, first output unit, second output unit)
300A, 300B ... Detection device 310A, 310B ... Acquisition unit 320A, 320B ... First output unit 321A, 321B ... Movement amount detection unit 322A, 322B ... Identification unit 323A, 323B ... Calibration unit 330 ... Second output unit

Claims (1)

An acquisition unit that acquires an image of a road surface feature imaged by an imaging unit mounted on a moving body;
An output unit that outputs distance related information related to the distance between the mounting position of the imaging unit and the road surface based on the image of the on-road feature acquired by the acquiring unit;
The imaging unit images a road surface immediately below a fixed position where the imaging unit is fixed to the moving body.
A detection device characterized by that.

Images