JP2010210486A - Image processor, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To rapidly measure the distance to a body with desired accuracy by a mobile stereo system. <P>SOLUTION: An obstacle detector 123 measures the distance to an obstacle by a mobile stereo system by using images taken at different positions by an imaging part 111. Based on the distance to the obstacle measured by the obstacle detector 123, a necessary moving amount setter 141 sets a necessary moving amount which is the distance from an imaging position of an image taken later of the two images used for measurement to an imaging position of an image used for next measurement. This image processor can be applied to an on-vehicle obstacle detector for example. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method, and a program, and more particularly, to an image processing apparatus and method suitable for use in measuring a distance to an object by a moving stereo method, and a program.

  Conventionally, using an image captured by a camera attached to the front or rear of a vehicle such as an automobile, an obstacle existing in the front or rear of the vehicle is detected, and a warning or the like is given to the driver according to the detection result. A system has been developed.

  In such a system, the three-dimensional position of an object shown in the image is detected based on the principle of triangulation using two images taken at different positions at different times by a single camera installed in the vehicle. A moving stereo system is often used.

  Here, referring to FIG. 1, an image I1 captured at time t by a single camera installed in the vehicle and an image I2 captured at time t + 1 after a predetermined time Δt from time t. Consider a case where three-dimensional measurement is performed by a moving stereo method using two images. When the vehicle moves between time t and time t + 1 and the optical center of the camera when the image I1 is captured and the optical center of the camera when the image I2 is captured are separated on the image I1 Based on the position P1 (xl, yl) of the object Q and the position P2 (xu, yu) of the object Q on the image I2, using the same triangulation principle as in the stereo camera system, the three-dimensional of the object Q Position (X, Y, Z) can be detected. Further, the coordinate in the Z-axis direction obtained here is the distance from the camera (its optical center) to the object Q.

  More specifically, in the moving stereo system, the distance Z to the object is obtained by the following equation (1).

  Here, f is the focal length of the camera. B is the baseline length. In the example of FIG. 1, B is the distance between the optical center for the image I1 and the optical center for the image I2, and is substantially equal to the amount of camera movement between time t and time t + 1. d is the parallax. In the example of FIG. 1, d is a distance between the position P1 (xl, yl) on the image and the position P2 (xu, yu).

  Further, the measurement error ΔZ of the distance Z is obtained by the following equation (2).

  Note that the same equations (1) and (2) can also be used when measuring the distance to an object by stereo viewing (stereo camera method) using two cameras.

  From equation (2), it can be seen that the greater the camera movement amount B and the smaller the distance Z to the object, the smaller the measurement error and the greater the measurement accuracy.

  Therefore, when Δt is constant and the distance Z to the object is measured by a moving stereo method at a constant time interval, the amount of movement B varies depending on the speed of the vehicle. Accuracy will be reduced. Therefore, when the vehicle approaches the object and decelerates, the measurement accuracy decreases, and the possibility of erroneously detecting the distance Z to the object increases.

  On the other hand, when the distance B to the object is measured by the moving stereo method every time the vehicle travels a certain distance with the movement amount B kept constant, the measurement accuracy decreases as the distance Z to the object increases. Further, when the movement amount B is set to be long in order to increase the measurement accuracy, when the vehicle approaches the object and the distance Z between the vehicle and the object becomes shorter than the movement amount B, the object may be approached thereafter. The distance Z to the object is not measured. Therefore, for example, even when the vehicle approaches the object, the driver is not warned and the risk of collision with the object increases. On the other hand, when the movement amount B is set short in order to prevent this, the measurement accuracy is lowered.

  Therefore, when a feature point (hereinafter referred to as a reference feature point) of an object is detected in an image (hereinafter referred to as a reference image) of a certain frame (hereinafter referred to as a reference frame), each frame after the reference frame is detected. Are searched for a feature point corresponding to the reference feature point (hereinafter referred to as a search feature point), and a distance Lm (parallax XM) between the reference feature point and the search feature point is obtained. This process is repeated until the measurement error | ΔL | of the distance to the object calculated using the distance Lm is equal to or less than a predetermined threshold, and the image of the frame in which the measurement error | ΔL | It has been proposed to measure the distance to an object using a reference image (see, for example, Patent Document 1). Thereby, the measurement error can be kept substantially constant.

JP 2007-322357 A

  However, in the invention described in Patent Document 1, it is necessary to search for a search feature point in each frame until a frame used for measuring the distance of the object is found, and the processing time is increased accordingly.

  The present invention has been made in view of such a situation, and makes it possible to quickly measure the distance to an object by a moving stereo method with a desired accuracy.

  An image processing apparatus according to one aspect of the present invention is an image processing apparatus that performs processing for measuring a distance to an object using an image captured by an imaging device provided on a moving body. Based on the measurement means for measuring the distance to the object using the moving stereo method and the distance to the object measured by the measurement means, the image captured later of the two images used for the measurement is used. Setting means for setting a necessary movement amount that is a distance from the imaging position to the imaging position of the image used for the next measurement.

  In the image processing apparatus according to one aspect of the present invention, the distance to the object is measured by the moving stereo method using two images captured at different positions, and the measurement is performed based on the measured distance to the object. A necessary movement amount that is a distance from an imaging position of an image captured later of the two used images to an imaging position of an image used for the next measurement is set.

  Therefore, it is possible to quickly measure the distance to the object by the moving stereo method with desired accuracy.

  This moving body is configured by a vehicle such as an automobile, a motorbike, a bicycle, a ship, and a train. The measuring means and setting means are constituted by, for example, a CPU (Central Processing Unit).

  When the measuring unit measures the distance to the object using the first image and the second image captured after the first image, the setting unit includes a distance to the object measured by the measuring unit. The distance from the imaging position of the second image to the imaging position of the image used for the next measurement can be set as the required movement amount based on the The distance to the object can be measured using the third image captured when the necessary amount of movement is moved.

  As a result, the distance to the object can be measured quickly and with a desired accuracy by the moving stereo method.

  In the setting means, the distance to the target when measuring the distance to the target by stereo vision, the measurement error of the distance to the target, and the relational expression representing the relationship between the base line lengths are calculated by the measuring means as the distance to the target. By substituting the distance to the measured object and substituting a desired value for the measurement error, it is possible to set the base line length obtained as the required movement amount.

  As a result, the distance to the object can be measured quickly and with a desired accuracy by the moving stereo method.

  In the setting means, when there are a plurality of objects in the image, the object having the closest distance to the moving object, or the object having the closest distance to the moving object among the objects existing in the traveling direction of the moving object Alternatively, the necessary movement amount can be set based on the distance between any of the objects existing in the region where the moving body can travel and any of the objects closest to the moving body.

  Thereby, the distance to an object with high risk of colliding with a moving body can be quickly measured with a moving stereo method with desired accuracy.

  According to an image processing method of one aspect of the present invention, two image processing apparatuses that perform a process of measuring a distance to an object using an image captured by an imaging apparatus provided on a moving body are captured at different positions. Using the image, the distance to the object is measured by the moving stereo method, and based on the measured distance to the object, the next measurement is performed from the imaging position of the image captured later of the two images used for the measurement. Including a step of setting a necessary movement amount that is a distance to an image pickup position of the image used for.

  A program according to one aspect of the present invention uses two images captured at different positions on a computer that performs a process of measuring a distance to an object using an image captured by an imaging device provided on a moving body. The distance to the object is measured by the moving stereo method, and based on the measured distance to the object, the image used for the next measurement is determined from the imaging position of the image captured later of the two images used for the measurement. A process including a step of setting a necessary movement amount that is a distance to the imaging position is executed.

  In the image processing method according to one aspect of the present invention or the computer that executes the program according to one aspect of the present invention, the distance to an object is measured by a moving stereo method using two images captured at different positions. Based on the measured distance to the object, a necessary movement amount that is the distance from the imaging position of the image captured later to the imaging position of the image used for the next measurement is set out of the two images used for the measurement. Is done.

  Therefore, it is possible to quickly measure the distance to the object by the moving stereo method with desired accuracy.

  The moving body is configured by a vehicle such as an automobile, a motorbike, a bicycle, a ship, and a train. This image processing apparatus is constituted by, for example, a CPU (Central Processing Unit).

  According to one aspect of the present invention, the amount of movement of the moving body until the next measurement can be set according to the measured distance to the object. In particular, according to one aspect of the present invention, it is possible to quickly measure the distance to an object with a moving stereo method with desired accuracy.

It is a figure for demonstrating the principle of the three-dimensional measurement by a moving stereo system. It is a block diagram which shows 1st Embodiment of the obstruction detection system to which this invention is applied. It is a figure which shows the example of the attachment position of an imaging part. It is a flowchart for demonstrating 1st Embodiment of an obstruction detection process. It is a flowchart for demonstrating 1st Embodiment of a three-dimensional measurement process. It is a figure for demonstrating the detection method of an obstruction. It is a figure for demonstrating the example of the setting method of the danger rank of an obstruction. It is a figure for demonstrating the example of the setting method of the danger rank of an obstruction. It is a figure for demonstrating the example of the setting method of the danger rank of an obstruction. It is a figure for demonstrating the example of the setting method of the danger rank of an obstruction. It is a figure for demonstrating the example of the setting method of the danger rank of an obstruction. It is a figure which shows the example of a display of the detection result of an obstruction. It is a figure which shows the other example of a display of the detection result of an obstruction. It is a figure for demonstrating the timing which acquires an image. It is a figure for demonstrating the timing which measures three-dimensional information. It is a figure for demonstrating the effect of this invention. It is a figure for demonstrating the effect of this invention. It is a block diagram which shows 2nd Embodiment of the obstruction detection system to which this invention is applied. It is a flowchart for demonstrating 2nd Embodiment of an obstruction detection process. It is a flowchart for demonstrating an obstruction attribute determination process. It is a figure which shows the classification | category of the area | region used for determination of the attribute of an obstruction. It is a figure which shows the example of the table used for determination of the attribute of an obstruction. It is a figure which shows the example of the attribute of an obstruction. It is a block diagram which shows 3rd Embodiment of the obstacle detection system to which this invention is applied. It is a flowchart for demonstrating 3rd Embodiment of an obstacle detection process. It is a flowchart for demonstrating a moving object detection process. It is a figure for demonstrating the detection method of a moving object. It is a block diagram which shows 4th Embodiment of the obstruction detection system to which this invention is applied. It is a flowchart for demonstrating 4th Embodiment of an obstruction detection process. It is a flowchart for demonstrating 2nd Embodiment of an obstruction attribute determination process. It is a figure which shows the other example of the table used for determination of the attribute of an obstruction. It is a figure which shows the other example of the table used for determination of the attribute of an obstruction. It is a figure which shows the other example of the attribute of an obstruction. And FIG. 11 is a block diagram illustrating an example of a configuration of a computer.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, a first embodiment of the present invention will be described with reference to FIGS.

  FIG. 2 is a block diagram showing a first embodiment of an obstacle detection system to which the present invention is applied. The obstacle detection system 101 of FIG. 2 is a system that is provided in a vehicle such as an automobile, detects an obstacle around the vehicle, and notifies the driver. The obstacle detection system 101 is configured to include an imaging unit 111, an image processing unit 112, and a result output unit 113. The image processing unit 112 is configured to include an image acquisition unit 121, a storage unit 122, an obstacle detection unit 123, a danger rank setting unit 124, a movement amount detection unit 125, and an image acquisition control unit 126. Further, the image acquisition control unit 126 is configured to include a necessary movement amount setting unit 141 and an acquisition request output unit 142.

  The imaging unit 111 is configured by a camera, for example, and is installed at a position where the front or rear of the host vehicle can be imaged.

  In the following description, as shown in FIG. 3, the obstacle detection system 101 is installed in the vehicle 201, and imaging is performed so that the direction of translational movement of the vehicle 201 and the optical axis of the imaging unit 111 are substantially parallel. Assume that the unit 111 is installed behind the vehicle 201.

  Here, with respect to a coordinate system based on the imaging unit 111 (hereinafter referred to as a camera coordinate system), the horizontal direction of the image obtained as a result of imaging is the X axis, the vertical direction is the Y axis, and the direction perpendicular to the image plane is Z. Axis. Further, with respect to a coordinate system based on the road surface (hereinafter referred to as a road surface coordinate system), the direction of translational movement of the vehicle 201 is the Zg axis, the directions perpendicular to the Zg axis and perpendicular to the road surface are the Yg axis, Zg axis, and Yg axis. The direction perpendicular to is the Xg axis. Furthermore, regarding the coordinate system (hereinafter referred to as an image coordinate system) of an image captured by the imaging unit 111, the horizontal direction is the x axis and the vertical direction is the y axis.

  The image processing unit 112 performs an obstacle detection process behind the vehicle 201 based on the image captured by the imaging unit 111 and notifies the result output unit 113 of the detection result.

  Specifically, the image acquisition unit 121 of the image processing unit 112 acquires an image from the imaging unit 111 when an acquisition of an image is requested from the acquisition request output unit 142 of the image acquisition control unit 126, and acquires the acquired image. And stored in the storage unit 122 constituted by a memory. In addition, the image acquisition unit 121 notifies the obstacle detection unit 123 that an image has been acquired from the imaging unit 111.

  The obstacle detection unit 123 performs an obstacle detection process behind the vehicle 201 by the moving stereo method using the image stored in the storage unit 122, and displays obstacle information indicating the detection result as a danger rank setting unit. 124.

  The danger rank setting unit 124 sets a danger rank indicating the rank of the danger level of the obstacle detected by the obstacle detection unit 123. The danger rank setting unit 124 adds the danger rank of each set obstacle to the obstacle information acquired from the obstacle detection unit 123, and the result output unit 113 and the necessary movement amount setting unit of the image acquisition control unit 126. 141.

  The movement amount detection unit 125 detects the movement amount of the vehicle 201 (= movement amount of the imaging unit 111) at a predetermined timing based on information supplied from the outside, and performs image acquisition control on the detected movement amount of the vehicle 201. The acquisition request output unit 142 of the unit 126 is notified.

  The method by which the movement amount detection unit 125 detects the movement amount of the vehicle 201 is not limited to a specific method, and any method can be adopted.

  For example, a wheel speed pulse output from a wheel speed sensor provided on a wheel of the vehicle 201 may be acquired, and the movement amount of the vehicle 201 may be detected based on the acquired number of wheel speed pulses.

  For example, “Shuhei Takimoto, Takaaki Ito,“ Development of a monocular ranging verification system using an in-vehicle camera ”, Technical Papers SEI Technical Review No.169, Sumitomo Electric Industries, Ltd., July 2006, p.82 The amount of movement of the vehicle 201 may be detected using an image obtained by imaging the periphery of the vehicle 201 using the method described in “-87”.

  Furthermore, for example, the movement amount of the vehicle 201 may be detected based on position information acquired using a GPS (Global Positioning System) receiver or the like.

  The image acquisition control unit 126 controls the timing at which the image acquisition unit 121 acquires an image from the imaging unit 111.

  Specifically, the necessary movement amount setting unit 141 of the image acquisition control unit 126 determines whether the image captured later is one of the two images used for the obstacle detection by the moving stereo method based on the obstacle detection result. A distance from the imaging position to the imaging position of the image used for the next measurement (hereinafter referred to as necessary movement amount B) is set. The necessary movement amount setting unit 141 notifies the acquisition request output unit 142 of the set necessary movement amount B.

  The acquisition request output unit 142 is set in the necessary movement amount setting unit 141 after the image acquisition unit 121 acquires an image from the imaging unit 111 based on the detection result of the movement amount of the vehicle 201 by the movement amount detection unit 125. The image acquisition unit 121 is requested to acquire an image at the timing when the vehicle 201 has moved by the necessary movement amount B. In addition, the acquisition request output unit 142 notifies the obstacle detection unit 123 of the amount of movement of the vehicle 201 from the time when the acquisition of the previous image is requested until the acquisition of the current image is requested.

  The result output unit 113 is configured by a display device such as a display of a car navigation system, for example. The result output unit 113 acquires, for example, an image obtained by imaging the rear of the vehicle 201 stored in the storage unit 122, and displays the information indicating the detection result of the obstacle superimposed on the acquired image.

  Next, the obstacle detection process executed by the obstacle detection system 101 will be described with reference to the flowchart of FIG. This process is started when, for example, the accessory (ACC) power supply of the vehicle 201 is turned on, and is ended when the accessory power supply is turned off.

  In step S <b> 1, the acquisition request output unit 142 requests the image acquisition unit 121 to acquire an image.

  In step S2, the obstacle detection system 101 acquires and stores an image. Specifically, for example, the image acquisition unit 121 commands the imaging unit 111 to capture an image, and the imaging unit 111 that receives the command captures the rear of the vehicle 201. The imaging unit 111 supplies an image obtained as a result of imaging to the image acquisition unit 121. The image acquisition unit 121 stores the acquired image in the storage unit 122.

  Alternatively, for example, the imaging unit 111 always captures a moving image at a predetermined frame rate, and the image acquisition unit 121 receives an image from the imaging unit 111 at a timing when acquisition of an image is requested from the acquisition request output unit 142. get. Then, the image acquisition unit 121 causes the storage unit 122 to store the acquired image.

  In step S <b> 3, the necessary movement amount setting unit 141 sets a specified value for the necessary movement amount B. The necessary movement amount setting unit 141 notifies the acquisition request output unit 142 of the set necessary movement amount B.

  The prescribed value of the required movement amount B is set to a movement amount that can measure a distance to an object separated by a predetermined distance (for example, 50 m) with a predetermined measurement error ΔZ.

In step S4, the acquisition request output unit 142, based on information from the moving amount detecting unit 125 obtains a movement amount t _z of the vehicle 201 after acquiring the previous image.

In step S5, the acquisition request output unit 132 determines whether or not the movement amount t _z ≧ the required movement amount B. If it is determined that the movement amount t _z <the necessary movement amount B, the process returns to step S4, and in step S5, until it is determined that the movement amount t _z ≧ the necessary movement amount B, the processing in steps S4 and S5 is performed. The process is executed repeatedly.

On the other hand, if it is determined in step S5 that the movement amount t _z ≧ the required movement amount B, the process proceeds to step S6.

In step S <b> 6, the acquisition request output unit 142 requests the image acquisition unit 121 to acquire an image, and the amount of movement t _z of the vehicle 201 from the previous image acquisition to the current image acquisition (more accurately, The obstacle detection unit 123 is notified of the movement amount t _z ) of the vehicle 201 from when the previous image acquisition is requested until the current image acquisition is requested.

  In step S7, the obstacle detection system 101 acquires and stores an image, and deletes the previous two images. Specifically, the image acquisition unit 121 acquires an image from the imaging unit 111 by the same processing as in step S1. Then, the image acquisition unit 121 deletes the image captured two times before from the storage unit 122 and causes the storage unit 122 to store the acquired image. Thus, the latest two images are always stored in the storage unit 122. In the process of the first step S7, since the second previous image is not stored in the storage unit 122, only the acquired image is stored without being erased.

  Hereinafter, of the two images stored in the storage unit 122 and used to detect an obstacle described later, an image acquired earlier is referred to as a past image, and an image acquired later is referred to as a latest image.

  In step S8, the obstacle detection unit 123 executes an obstacle detection process. Here, the details of the obstacle detection processing will be described with reference to the flowchart of FIG.

  In step S51, the obstacle detection unit 123 acquires two images, a past image and a latest image, stored in the storage unit 122.

  In step S52, the obstacle detection unit 123 extracts feature points from the past image. For example, the obstacle detection unit 123 extracts points (locations) such as edges and corners of an object shown in the past image as feature points. Note that the feature point extraction method is not limited to a specific method, and is described in, for example, “Xu Tsuyoshi, Saburo Tsuji,“ 3D Vision ”, Kyoritsu Publishing, April 1998, p.25-27”. It is possible to employ any method such as

  In step S53, the obstacle detection unit 123 searches for a corresponding point that is a point in the latest image corresponding to the feature point extracted in the past image. Note that the corresponding point search method is not limited to a specific method, and is described in, for example, “Xu Goh, Saburo Tsubaki,“ 3D Vision ”, Kyoritsu Publishing, April 1998, p.112-115. It is possible to employ any method such as

  In step S54, the obstacle detection unit 123 calculates an inter-camera parameter. Here, the inter-camera parameters are the rotation angle (pitch angle) θ around the X axis and the rotation angle around the Y axis (pitch angle) θ in the camera coordinate system of the imaging unit 111 from when the past image is captured until the latest image is captured. Yaw angle) ψ and rotation angle (roll angle) φ around the Z axis.

  Here, the motion vector of the object detected from the image and the inter-camera parameter can be defined as a relationship such as the following equation.

f × x × θ + f × y × ψ− (x ² + y ² ) × φ−v _x × x + v _y × y = 0 (3)

In Expression (3), f represents the focal length of the image capturing unit 111 and is a value inherent to the image capturing unit 111, and thus is substantially a constant. Further, v _x and v _y represent components in the x-axis direction and the y-axis direction in the image coordinate system of the detected object motion vector (optical flow) Vs, respectively. That is, the motion vector of the object Vs = (v _x , v _y ). X and y represent the coordinates in the x-axis direction and the y-axis direction in the image coordinate system of the feature points corresponding to the motion vectors, respectively. That is, the feature point position Pf = (x, y).

  In the expression (3), when the focal length f, the motion vector Vs, and the position Pf of the feature point are known, the variables are in the linear form of the pitch angle θ, the yaw angle ψ, and the roll angle φ. Therefore, the pitch angle θ, the yaw angle ψ, and the roll angle φ can be obtained by solving the equation (3) as a linear optimization problem.

  In step S55, the obstacle detection unit 123 calculates the three-dimensional information of the detected object.

  For example, when the detected object is a stationary object, the motion vector Vs of the object can be expressed by the following equation (4).

In the equation _{(4), t x, t} y, t z is X-axis direction of each imaging unit 111, represents the translation of Y-axis and Z-axis directions, X, Y, Z, the camera coordinate system, respectively Represents the coordinates of the object in the X-axis direction, the Y-axis direction, and the Z-axis direction.

  When Expression (4) is transformed so that each of X, Y, and Z is the left side, the following Expressions (5) to (7) are obtained.

  The position (X, Y, Z) of the camera coordinate system of the object can be converted to the position (Xg, Yg, Zg) of the road surface coordinate system by the following equation (8).

  In Expression (8), α, β, and γ represent the installation roll angle, the installation pitch angle, and the installation yaw angle of the imaging unit 111, and H represents the height of the imaging unit 111 from the road surface.

  As described above, the three-dimensional information of the detected stationary object, that is, the position (Xg, Yg, Zg) of the road surface coordinate system can be calculated, and even when the detected object is moving, The position (Xg, Yg, Zg) of the road surface coordinate system of the object can be calculated. Then, the obstacle detection unit 123 obtains three-dimensional information for all objects detected from the image.

  In step S56, the obstacle detection unit 123 detects an obstacle from the detected objects. For example, as shown in FIG. 6, the obstacle detection unit 123 sets a detection range of 2 m × 2 m (Xg direction × Zg direction) in the road surface coordinate system, and moves the detection range in units of 25 cm in a 50 cm × 50 cm window. Scan while. The obstacle detection unit 123 detects the object as an obstacle when there are five or more points having a height of 30 cm or more in the window. Note that the detection range, window size, and scanning unit values shown here are just examples, and other values may be adopted.

  Then, the obstacle detection unit 123 supplies obstacle information including presence / absence of the obstacle and three-dimensional information (that is, position information) of the detected obstacle to the danger rank setting unit 124. Thereafter, the obstacle detection process ends.

  Returning to FIG. 4, in step S <b> 9, the danger rank setting unit 124 sets the danger rank of the detected obstacle. Here, with reference to FIG. 7 to FIG. 11, an example of a method for setting an obstacle risk ranking will be described. 7 to 11, the downward direction in the figure is the Zg axis direction of the road surface coordinate system, that is, the rear of the vehicle 201.

  For example, it is conceivable to set the hazard ranking of the obstacle based only on the distance to the vehicle 201. That is, an obstacle with a shorter distance from the vehicle 201 has a higher risk ranking, and an obstacle with a longer distance from the vehicle 201 has a lower risk ranking. In this case, for example, as shown in FIG. 7, when the distance Da between the vehicle 201 and the obstacle 221 </ b> A <the distance Db between the vehicle 201 and the obstacle 211 </ b> B, the obstacle 221 </ b> A is the obstacle. The risk ranking is set higher than 221B.

  Hereinafter, the distance between the vehicle 201 and the obstacle is defined as the shortest distance between the contour point of the region of the vehicle 201 and the contour point of the region of the obstacle. In this case, for example, as shown in FIG. 8, the distance between the vehicle 201 and the convex obstacle 221 </ b> C is the distance Dc between the portion of the obstacle 221 </ b> C that protrudes most toward the vehicle 201 and the vehicle 201. It becomes. The definition of the distance between the vehicle 201 and the obstacle is not limited to this, and may be defined as, for example, the distance between a predetermined portion of the vehicle 201 and the obstacle. .

  In addition, for example, the traveling direction of the vehicle 201 may be predicted, and the danger rank of the obstacle may be set based on the distance from the vehicle 201 only for an area expected to travel.

  For example, as shown in FIG. 9, when the obstacle 221D is located almost directly behind the vehicle 201 and the obstacle 221E is located immediately behind the vehicle 201, the distance De between the obstacle 221E and the vehicle 201 is shown. Is shorter than the distance Dd between the obstacle 221D and the vehicle 201. However, when the vehicle 201 is expected to move straight back in the direction of the arrow A1, the obstacle 221D is included in the predicted travel region R1 of the vehicle 201, but the obstacle 221E is included in the travel region R1. Absent. Therefore, it is determined that the possibility that the vehicle 201 collides with the obstacle 221E is low, and the obstacle rank 221D is set to have a higher risk ranking than the obstacle 221E.

  For example, as shown in FIG. 10, the obstacles 221F and 221G are located at substantially the same position in the Zg axis direction, the obstacle 221F is located on the right rear side of the vehicle 201, and the obstacle 221G is located behind the vehicle 201. The distance Dg between the obstacle 221G and the vehicle 201 is shorter than the distance Df between the obstacle 221F and the vehicle 201. However, when the vehicle 201 is expected to move backward while turning in the direction of the arrow A2, the obstacle 221F is included in the predicted traveling region R2 of the vehicle 201, but the obstacle 221G is included in the traveling region R2. Not included. Therefore, it is determined that the possibility that the vehicle 201 collides with the obstacle 221G is low, and the obstacle rank is set higher for the obstacle 221F than for the obstacle 221G.

  Note that the method of predicting the traveling direction of the vehicle 201 is not limited to a specific method. For example, the prediction is based on the traveling direction of the vehicle 201 up to the present, or the steering angle provided in the vehicle 201 Based on the detection results of the sensor, “Serizawa, Hino,“ Fundamental considerations of automatic parking system using RF-ID ”, Automobile Engineering Society Proceedings Vol.37 No.2, March 2006, p. Any method such as the method described in “.185-190” can be adopted.

  Furthermore, for example, regardless of the traveling direction of the vehicle 201, the danger rank of the obstacle may be set based on the distance to the vehicle 201 in an area where the vehicle 201 can travel.

  For example, a region R3 in FIG. 11 indicates a region that can travel when the vehicle 201 moves backward. That is, the region R3 includes from a region where the vehicle 201 moves backward with the leftward steering fully opened to a region where the vehicle 201 moves backward with the rightward steering fully opened. In FIG. 11, an obstacle 221H located at the left rear of the vehicle 201 and an obstacle 221I located immediately behind the vehicle 201 are both included in the region R3. Accordingly, since the distance Dh between the vehicle 201 and the obstacle 221H is shorter than the distance between the vehicle 201 and the obstacle 221I, the obstacle 221H has a higher risk ranking than the obstacle 221I. .

  In addition, since it is rare that the vehicle 201 turns backward with the steering wheel fully opened to the left or right, the left and right regions of the region R3 may be narrowed.

  Then, the danger rank setting unit 124 adds the danger rank of each set obstacle to the obstacle information acquired from the obstacle detection unit 123, and supplies the result to the result output unit 113 and the necessary movement amount setting unit 141.

  In step S10, the result output unit 113 outputs the detection result. Specifically, the result output unit 113 acquires the latest image stored in the storage unit 122. And the result output part 113 displays the image which gave the effect which emphasizes the detected obstruction on the newest image on display apparatuses, such as a display of a car navigation system.

  For example, as shown in FIG. 12, when obstacles 231A and 231B are detected in the latest image, rectangular frames Fa1 and Fb1 surrounding the obstacles 231A and 231B are displayed superimposed on the latest image. As a result, the positions of the obstacles 231A and 231B can be recognized at a glance.

  Furthermore, as shown in FIG. 13, the color of the rectangular frame and the thickness of the line may be changed based on the danger rank of the obstacle. That is, in the example of FIG. 13, the rectangular frame Fa2 that surrounds the obstacle 231A that is closer to the vehicle 201 and has a higher risk ranking is farther away from the vehicle 201 and that surrounds the obstacle 231B that has a lower risk ranking. It is thicker than the line of Fb2. As a result, obstacles with a higher risk ranking are highlighted and displayed, and it is possible to quickly recognize a place where the driver should be careful.

  Returning to FIG. 4, in step S <b> 11, the necessary movement amount setting unit 141 determines whether an obstacle has been detected based on the obstacle information acquired from the danger rank setting unit 124. If it is determined that an obstacle has been detected, the process proceeds to step S12.

  In step S <b> 12, the necessary movement amount setting unit 141 selects an obstacle for which the necessary movement amount B is to be calculated. Specifically, the necessary movement amount setting unit 141 selects an obstacle with the highest risk ranking as an obstacle for which the necessary movement amount B is to be calculated.

  In step S <b> 13, the necessary movement amount setting unit 141 calculates a necessary movement amount B. Specifically, when the equation (2) for calculating the measurement error ΔZ of the distance Z in the moving stereo method is solved for B, the following equation (9) is obtained.

  Here, when the focal length of the imaging unit 111 is set as a constant in f, and the parallax error of the imaging unit 111 obtained as an empirical value is set as a constant in Δd, the equation (9) is expressed as the distance Z to the object, This is a relational expression representing the relation between the measurement error ΔZ of the distance Z and the base line length B. Substituting the distance to an obstacle (hereinafter also referred to as a calculation target obstacle) for calculating the required movement amount B into Z and substituting a desired measurement error into ΔZ, the calculation target obstacle is obtained by the moving stereo method. In order to measure the distance to the object with the measurement error ΔZ, it is possible to obtain the necessary distance (that is, the necessary movement amount B) between the positions where the two images used for measurement are captured. That is, by determining the desired measurement error ΔZ, it is possible to determine the necessary movement amount B of the imaging unit 111 (vehicle 201) necessary for measuring the distance Z to the calculation target obstacle with the measurement error ΔZ. .

  As described above, the necessary movement amount setting unit 141 includes the distance Z to the obstacle to be calculated, the desired measurement error ΔZ preset by the driver, the focal distance f of the imaging unit 111, and the parallax error of the imaging unit 111. Substituting Δd into equation (9), the required movement amount B is calculated. The necessary movement amount setting unit 141 notifies the acquisition request output unit 142 of the calculated necessary movement amount B. Thereafter, the process returns to step S4, and the processes after step S4 are executed.

  On the other hand, if it is determined in step S11 that no obstacle has been detected, the process proceeds to step S14. In step S14, a prescribed value is set for the required movement amount B, similar to the process in step S3. Thereafter, the process returns to step S4, and the processes after step S4 are executed.

  Accordingly, for example, as illustrated in FIG. 14, at time t, the obstacle 241 having the highest danger rank exists behind the vehicle 201, and the image 1 captured by the imaging unit 111 is acquired and the obstacle 241 is acquired. Is obtained, the required movement amount B is set based on the distance Z to the obstacle 241 at that time. Then, at time t + 1 when the vehicle 201 moves backward by the necessary movement amount B, the image 2 captured by the imaging unit 111 is acquired, and the three-dimensional information of the obstacle 241 is obtained.

  Therefore, the required movement amount B is set shorter based on the above-described equation (9) as the vehicle 201 approaches the obstacle 241 and the distance Z between the vehicle 201 and the obstacle 241 becomes shorter. That is, as shown in FIG. 15, if the required movement amounts B1 to B3 are set from time t to time t + 2, the required movement amount B1> the required movement amount B2> the required movement amount B3. Therefore, the closer the vehicle 201 is to the obstacle 241, the three-dimensional measurement of the obstacle 241 can be performed at short distance intervals, and a warning can be given to the driver. As a result, the collision of the vehicle 201 with the obstacle 241 can be avoided more reliably.

  Further, as shown in FIGS. 16 and 17, consider a case where the vehicle 201 moves backward while turning right and approaches the obstacle 251 in an attempt to park. In this case, when the driver is aware of the presence of the obstacle 251, a position close to the obstacle 251 as shown in FIG. 17 from a position far from the obstacle 251 as shown in FIG. 16. As the vehicle 201 moves quickly, the vehicle 201 is decelerated.

  At this time, even when the vehicle 201 is far from the obstacle 251 and the vehicle speed is high, or when the vehicle 201 approaches the obstacle 251 and decelerates, the vehicle 201 is always between the obstacle 251 with a constant accuracy regardless of the vehicle speed. The distance is measured. That is, the measurement accuracy does not decrease as the vehicle decelerates, as in the case of measuring the distance to the obstacle 251 at predetermined time intervals.

  Further, as the obstacle 251 is approached, the distance to the obstacle 251 is measured, and the distance of the distance notified to the driver is shortened. Accordingly, as in the case of measuring the distance to the obstacle 251 at predetermined time intervals, the distance to the obstacle 251 is frequently measured and notified at a stage far from the obstacle 251, and driving is performed. It is possible to prevent a person from feeling annoyed. Further, when the distance to the obstacle 251 is measured at a predetermined distance interval, the distance to the obstacle 251 is not measured and the driver is notified when the distance to the obstacle 251 is approached. It is possible to prevent a situation that is not performed.

  Further, as described above, in order to detect an obstacle, it is only necessary to store two images of the past image and the latest image in the storage unit 122, and a required storage capacity is reduced.

  Further, the distance to the obstacle can be measured quickly and with a desired accuracy by a moving stereo method by simply adding the calculation of the required movement amount B.

  Next, a second embodiment of the present invention will be described with reference to FIGS. Note that the second embodiment of the present invention has the function of determining the attribute of an obstacle and changing an alarm to the driver according to the determined attribute of the obstacle in the first embodiment of the present invention. It is added.

  FIG. 18 is a block diagram showing a second embodiment of an obstacle detection system to which the present invention is applied. The obstacle detection system 301 in FIG. 18 is configured to include an imaging unit 111, an image processing unit 311, and a result output unit 312. The image processing unit 311 includes an image acquisition unit 121, a storage unit 122, an obstacle detection unit 123, a danger ranking setting unit 124, a movement amount detection unit 125, an image acquisition control unit 126, and an obstacle attribute determination unit 321. Configured to include. Further, the image acquisition control unit 126 is configured to include a necessary movement amount setting unit 141 and an acquisition request output unit 142.

  In the figure, portions corresponding to those in FIG. 2 are denoted by the same reference numerals, and description of portions having the same processing will be omitted because it will be repeated. Hereinafter, it is assumed that the obstacle detection system 301 is installed in the vehicle 201 and the imaging unit 111 is installed at the position shown in FIG. 3 described above.

  The obstacle detection system 301 is different from the obstacle detection system 101 in FIG. 2 in that the image processing unit 112 is replaced with an image processing unit 311 and the result output unit 113 is replaced with a result output unit 312. Further, the image processing unit 311 of the obstacle detection system 301 is different from the image processing unit 112 of the obstacle detection system 101 in that an obstacle attribute determination unit 321 is added.

  The obstacle attribute determination unit 321 acquires obstacle information from the danger rank setting unit 124, and determines the attribute of the obstacle detected by the obstacle detection unit 123 based on the obstacle information. The obstacle attribute determination unit 321 adds an obstacle attribute determination result to the obstacle information acquired from the danger rank setting unit 124, supplies the result to the result output unit 312, and stores the result in the storage unit 122.

  The result output unit 312 includes, for example, a display device such as a display of a car navigation system, and an audio output device such as a speaker, a buzzer, or a siren. Similar to the result output unit 113 of the obstacle detection system 101, the result output unit 312 acquires an image of the rear of the vehicle 201 stored in the storage unit 122, and the obstacle detection result is obtained in the acquired image. The information indicating is superimposed and displayed. Further, the result output unit 312 outputs an alarm sound based on the result of the obstacle attribute determination.

  Next, the obstacle detection process executed by the obstacle detection system 301 will be described with reference to the flowchart of FIG. This process is started when, for example, the accessory (ACC) power supply of the vehicle 201 is turned on, and is ended when the accessory power supply is turned off.

  The processing in steps S101 to S108 is the same as the processing in steps S1 to S8 in FIG.

  In step S109, the danger ranking setting unit 124 performs the same process as in step S9 in FIG. 4, and sets the danger ranking of the detected obstacle. The danger rank setting unit 124 adds the danger rank of each set obstacle to the obstacle information acquired from the obstacle detection unit 123 and supplies the added obstacle rank to the obstacle attribute determination unit 321 and the necessary movement amount setting unit 141.

  In step S110, the obstacle attribute determination unit 321 executes an obstacle attribute determination process. Here, the details of the obstacle attribute determination processing will be described with reference to the flowchart of FIG.

  First, with reference to FIG. 21, the definition of the area | region used for determination of the attribute of an obstruction is demonstrated. For the attribute determination of the obstacle, four types of areas, a measurement area Rin, a dangerous area Rd, a safety area Rs, and a non-measurement area Rout are used. A measurement area Rin indicated by diagonal lines in the figure is an area that appears in an image captured by the imaging unit 111 and includes a dangerous area Rd indicated by shading. The dangerous area Rd is an area within a predetermined distance from the vehicle 201 in the measurement area Rin. Further, the area other than the dangerous area Rd in the measurement area Rin is the safety area Rs. Furthermore, a region other than the measurement region Rin that is not captured in an image captured by the imaging unit 111 is a non-measurement region Rout.

  Returning to FIG. 20, in step S151, the obstacle attribute determination unit 321 determines whether there is an obstacle in the measurement region Rin based on the obstacle information acquired from the danger ranking setting unit 124, in other words, the current measurement. It is determined whether an obstacle is detected in the region Rin.

  In step S152, the obstacle attribute determining unit 321 determines whether there is an obstacle in the dangerous area Rd, in other words, whether an obstacle has been detected in the current dangerous area Rd.

  In step S153, the obstacle attribute determination unit 321 acquires the previous obstacle detection result. In other words, the obstacle attribute determination unit 321 indicates the previous obstacle detection result, and acquires obstacle information including the three-dimensional information and danger rank of the obstacle detected last time from the storage unit 122.

  In step S154, the obstacle attribute determination unit 321 associates the obstacle detected this time with the obstacle detected last time. For example, the obstacle attribute determination unit 321 associates the previously detected obstacle and the obstacle detected this time as the same obstacles. Alternatively, the obstacle attribute determination unit 321 obtains the past image and the latest image from the storage unit 122, calculates the similarity between the previously detected obstacle image and the currently detected obstacle image, and obtains the highest similarity. Corresponding objects with high scores as the same obstacle.

  Note that the method of associating the obstacle is not limited to the method described above, and any method can be employed.

  In step S155, the obstacle attribute determination unit 321 determines whether the obstacle with the highest risk ranking has changed based on the result of association between the obstacle detected this time and the obstacle detected last time.

  In step S156, the obstacle attribute determination unit 321 determines that the obstacle with the highest risk ranking is between the areas based on the obstacle information and the result of association between the obstacle detected this time and the obstacle detected last time. It is determined whether or not. Specifically, the obstacle attribute determination unit 321 determines that the area where the obstacle with the highest danger rank exists is the danger area Rd last time and the safety area Rs this time, or in the measurement area Rin. If an obstacle was detected last time and not detected this time, it is determined that the obstacle with the highest danger rank has moved to the safe side. In addition, the obstacle attribute determination unit 321 determines that the area where the obstacle with the highest danger rank exists is the safety area Rs last time, or is the danger area Rd this time, or in the measurement area Rin, the previous time If no obstacle is detected, but this time it is detected, it is determined that the obstacle with the highest danger rank has moved to the danger side. Further, the obstacle attribute determination unit 321 has detected no obstacle in the measurement area Rin when the area where the obstacle having the highest danger rank exists does not change between the previous time and this time, or in the previous time and this time. In this case, it is determined that there is no movement between areas of the obstacle having the highest risk ranking.

  In step S157, the obstacle attribute determination unit 321 determines the attribute of the obstacle. Here, an example of obstacle attribute determination will be described with reference to FIGS. 22 and 23. 22 shows an example of a table used for obstacle attribute determination, and FIG. 23 shows an example of obstacle attributes.

  As shown in FIG. 22, according to combinations of determination conditions A to D, the states of the obstacles are classified into 11 types of states 1 to 11, and obstacle attributes are associated with the respective states.

  For the determination condition A, the determination result obtained in step S151 is used to determine whether an obstacle exists in the measurement region Rin. As the determination condition B, the determination result obtained in step S152 is used as to whether there is an obstacle in the dangerous area Rd. As the determination condition C, the determination result obtained in step S155 is used as to whether or not the obstacle having the highest danger rank has changed. As the determination condition D, the determination result obtained in step S156 is used to determine whether or not the obstacle having the highest danger rank has moved between the areas.

  In state 1, there is no obstacle in the measurement area Rin, no obstacle in the danger area Rd, the obstacle with the highest danger rank does not change, and the danger rank is the highest. There is no movement between areas of high obstacles. That is, no obstacle has been detected in the measurement region Rin both in the previous time and this time. In this case, since the non-dangerous state continues, no obstacle attribute is set.

  In state 2, there is no obstacle in the measurement area Rin, no obstacle in the danger area Rd, the obstacle with the highest danger rank changes, and the danger rank is the highest. The obstacle has moved to the safe side. That is, in the measurement region Rin, the obstacle was detected last time and the obstacle was not detected this time. In this case, the obstacle attribute is set to D1. The obstacle attribute D1 indicates that the obstacle with the highest danger rank has moved to the safe side, as shown in FIG.

  In state 3, there is an obstacle in the measurement area Rin, no obstacle in the danger area Rd, the obstacle with the highest danger rank does not change, and the danger rank is the highest. There is no movement between the obstacle areas. That is, the obstacle with the highest risk ranking is the same in the previous time and the current time, and the obstacle exists in the safe area Rs both in the previous time and this time. In this case, since the non-dangerous state continues, no obstacle attribute is set.

  In state 4, there is an obstacle in the measurement area Rin, no obstacle in the danger area Rd, the obstacle with the highest danger rank does not change, and the danger rank is the highest. This is a state in which the obstacle has moved to the safe side, that is, a state in which the obstacle having the highest risk ranking in the previous time and the current time is the same, and the obstacle has moved from the danger area Rd to the safety area Rs. In this case, as in the case of the state 1, the obstacle attribute is set to D1.

  State 5 is an obstacle in which an obstacle exists in the measurement area Rin, no obstacle exists in the danger area Rd, the obstacle with the highest danger rank changes, and the danger rank is highest. This is a state in which an object has moved to the danger side, that is, an obstacle has been detected in the safety region Rs this time, although no obstacle was previously detected. In this case, the obstacle attribute is set to C + D2. As shown in FIG. 23, the obstacle attribute C indicates that the obstacle having the highest danger rank, that is, the obstacle for which the necessary movement amount B is calculated has changed. The obstacle attribute D2 indicates that the obstacle having the highest danger rank has moved to the danger side.

  State 6 is an obstacle in which an obstacle is present in the measurement region Rin, no obstacle is present in the danger region Rd, an obstacle having the highest danger rank is changed, and the danger rank is highest. The state where there is no movement between the object areas, that is, the state where the obstacle with the highest danger rank is different between the previous time and the current time, and the area where the obstacle with the highest danger order exists is not changed from the safety area Rs. It is. In this case, the obstacle attribute is set to C.

  State 7 is an obstacle in which there is an obstacle in the measurement area Rin, no obstacle in the danger area Rd, the obstacle with the highest danger rank is changed, and the danger rank is highest. A state in which an object has moved to the safe side, that is, a state in which the obstacle having the highest risk ranking is different from the previous time and this time, and the region where the obstacle having the highest risk ranking is present has changed from the danger region Rd to the safety region Rs. It is. In this case, the obstacle attribute is set to C + D1.

  State 8 is an obstacle in which an obstacle is present in the measurement region Rin, an obstacle is present in the danger region Rd, the obstacle having the highest risk ranking is not changed, and the risk ranking is highest. This is a state in which an object has moved to the danger side, that is, a state in which the obstacle having the highest danger rank is the same in the previous time and the current time, and the obstacle has moved from the safety area Rs to the danger area Rd. In this case, the obstacle attribute is set to B + D2. The obstacle attribute B indicates that there is an obstacle in the dangerous area Rd as shown in FIG.

  State 9 is an obstacle in which an obstacle exists in the measurement region Rin, an obstacle exists in the danger region Rd, the obstacle with the highest risk ranking does not change, and the risk ranking is the highest. This is a state in which there is no movement between the object areas, that is, a state in which the obstacle having the highest danger rank is the same between the previous time and the current time, and the obstacle remains in the danger area Rd. In this case, the obstacle attribute is set to B.

  In state 10, there is an obstacle in the measurement area Rin, an obstacle in the danger area Rd, an obstacle having the highest danger rank, and an obstacle having the highest danger rank. Is moving to the danger side, that is, the state where the obstacle having the highest danger rank is different between the previous time and the current time, and the area where the obstacle having the highest danger rank exists is changed from the safety area Rs to the danger area Rd, Alternatively, an obstacle is not detected in the previous measurement area Rin, but an obstacle is detected in the current dangerous area Rd. In this case, the obstacle attribute is set to B + C + D2.

  In state 11, there are obstacles in the measurement area Rin, obstacles in the danger area Rd, obstacles having the highest danger rank, and obstacles having the highest danger rank. In other words, there is no movement between the areas, that is, the obstacle with the highest risk ranking is different between the previous time and the current time, and the area where the obstacle with the highest risk ranking exists is the dangerous area Rd. In this case, the obstacle attribute is set to B + C.

  The obstacle attribute determination unit 321 adds the obstacle attribute determination result to the obstacle information acquired from the danger rank setting unit 124 after determining the obstacle attribute, and supplies the result to the result output unit 312. .

  In step S158, the image processing unit 311 stores the current obstacle detection result. That is, the obstacle attribute determination unit 321 deletes the obstacle information indicating the previous obstacle detection result from the storage unit 122 and causes the storage unit 122 to store the obstacle information indicating the current obstacle detection result. . Thereafter, the obstacle attribute determination process ends.

  Returning to FIG. 19, in step S <b> 111, the result output unit 312 outputs the detection result. Specifically, the result output unit 312 displays the latest image on which the effect of emphasizing the detected obstacle is applied, as in the process of step S10 of FIG. Further, the result output unit 312 outputs an alarm sound corresponding to the obstacle attribute determined by the obstacle attribute determination unit 321. The timbre, ringing interval, volume, etc. of this alarm sound can be changed according to the obstacle attribute.

  Here, a specific example of the alarm sound will be described with reference to FIG.

  When the obstacle attribute is D1, the type of alarm sound is set to sound 1aL, and the volume is set to be small. The sound 1aL is output, for example, with a tone color “ringing” and a ringing interval (interval between sounds).

  When the obstacle attribute is C + D2, the type of alarm sound is set to sound 1bL, and the volume is set to be large. The sound 1bL is output, for example, with a tone color “ringing” and a ringing interval. That is, the timbre of the alarm sound changes depending on whether the obstacle having the highest danger level has changed or not.

  When the obstacle attribute is C, the type of alarm sound is set to sound 1bL, and the volume is set to medium.

  When the obstacle attribute is C + D1, the type of alarm sound is set to sound 1bL, and the volume is set to be small.

  When the obstacle attribute is B + D2, the type of alarm sound is set to sound 1aS, and the volume is set to be large. For example, the sound 1aS is output with a tone color and a ringing interval of “pipipi”, and the ringing interval is shorter than that of the sound 1aL with the same tone color as the sound 1aL.

  When the obstacle attribute is B, the type of alarm sound is set to sound 1aS, and the volume is set to medium.

  When the obstacle attribute is B + C + D2, the type of alarm sound is set to sound 1bS, and the volume is set to be large. Note that the sound 1bS is output with, for example, a tone color and a ringing interval of “Puppu”, and the ringing interval is shorter than that of the sound 1bL with the same tone as the sound 1bL.

  When the obstacle attribute is B + C, the type of alarm sound is set to sound 1bS, and the volume is set to medium.

  If no obstacle attribute is set, no alarm sound is output.

  As described above, when an obstacle is present in the dangerous area Rd, an alarm sound is generated so that the driver can be alerted more than when no obstacle is present in the dangerous area Rd. The interval is shortened. In addition, the timbre of the alarm sound changes depending on whether the obstacle having the highest danger rank has changed or not. Furthermore, when the obstacle with the highest danger rank moves to the dangerous side, when there is no movement between areas, it moves to the safe side in order of warning sound so that the driver can be alerted more. The volume of increases.

  The processing in steps S112 to S115 is the same as the processing in steps S11 to S14 in FIG.

  As described above, according to the attribute of the obstacle, the type of alarm for the driver can be changed, and the degree of alerting the driver can be changed.

  Next, a third embodiment of the present invention will be described with reference to FIGS. Note that the third embodiment of the present invention improves the measurement accuracy of the three-dimensional information of the moving obstacle as compared with the first embodiment of the present invention.

  FIG. 24 is a block diagram showing a third embodiment of an obstacle detection system to which the present invention is applied. The obstacle detection system 401 of FIG. 24 is configured to include an imaging unit 111, an image processing unit 411, and a result output unit 113. The image processing unit 411 includes an image acquisition unit 121, a storage unit 122, an obstacle detection unit 123, a danger rank setting unit 124, a movement amount detection unit 125, an image acquisition control unit 126, a moving object detection unit 421, and a danger. The order setting unit 422 is included.

  In the figure, portions corresponding to those in FIG. 2 are denoted by the same reference numerals, and description of portions having the same processing will be omitted because it will be repeated. Hereinafter, it is assumed that the obstacle detection system 401 is installed in the vehicle 201 and the imaging unit 111 is installed at the position shown in FIG. 3 described above.

  The obstacle detection system 401 is different from the obstacle detection system 101 in FIG. 2 in that the image processing unit 112 is replaced with an image processing unit 411. In addition, the image processing unit 411 of the obstacle detection system 401 is added with a moving object detection unit 421 and the danger rank setting unit 124 is added to the risk rank setting unit 422 as compared with the image processing unit 112 of the obstacle detection system 101. It is different in that it has been replaced.

  The moving object detection unit 421 acquires, from the acquisition request output unit 142, the amount of movement of the vehicle 201 from when the previous image acquisition is requested until the current image acquisition is requested. In addition, the moving object detection unit 421 uses the image stored in the storage unit 122 to perform detection processing of the moving object behind the vehicle 201 based on the motion vector, and sets the moving object information indicating the detection result as the risk ranking. To the unit 422.

  The danger rank setting unit 422 sets the danger rank of the obstacle based on the obstacle information acquired from the obstacle detection unit 123 and the moving object information acquired from the moving object detection unit 421. In addition, the danger rank setting unit 422 includes the three-dimensional information of the obstacle detected as the moving object by the moving object detection unit 421 among the three-dimensional information of the obstacle included in the obstacle information acquired from the obstacle detection unit 123. Is replaced with the three-dimensional information calculated by the moving object detection unit 421. Further, the danger rank setting unit 422 adds the set danger rank of each obstacle to the obstacle information and supplies the result to the result output unit 113 and the necessary movement amount setting unit 141.

  Next, the obstacle detection process executed by the obstacle detection system 401 will be described with reference to the flowchart of FIG. This process is started when, for example, the accessory (ACC) power supply of the vehicle 201 is turned on, and is ended when the accessory power supply is turned off.

  The processing in steps S201 to S205 is the same as the processing in steps S1 to S5 in FIG.

In step S206, the acquisition request output unit 142 requests the image acquisition unit 121 to acquire an image, and the movement amount t _z of the vehicle 201 from the acquisition of the previous image to the acquisition of the current image (more precisely, Then, the obstacle detection unit 123 and the moving object detection unit 421 are notified of the movement amount t _z of the vehicle 201 from when the previous image acquisition is requested until the current image acquisition is requested.

  In step S207, as in the process of step S7 in FIG. 4, a new image is acquired and stored in the storage unit 122, and the previous two images are deleted from the storage unit 122.

  In step S208, as in the process of step S8 of FIG. 4, the obstacle detection process is executed, and obstacle information is supplied from the obstacle detection unit 123 to the risk ranking setting unit 422.

  In step S209, the moving object detection unit 421 executes a moving object detection process. Here, the details of the moving object detection process will be described with reference to FIG.

  In step S <b> 251, the moving object detection unit 421 acquires two images of the past image and the latest image stored in the storage unit 122.

  In step S252, the moving object detection unit 421 extracts feature points of the past image in the same manner as the processing by the obstacle detection unit 123 in step S52 of FIG.

  In step S253, the moving object detection unit 421 searches for corresponding points that are points in the latest image corresponding to the feature points extracted in the past image, similarly to the processing by the obstacle detection unit 123 in step S53 of FIG. .

  In step S254, the moving object detection unit 421 calculates an inter-camera parameter in the same manner as the processing by the obstacle detection unit 123 in step S54 of FIG.

  In step S255, the moving object detection unit 421 calculates a motion vector in which the inter-camera parameter is corrected. Specifically, the moving object detection unit 421 calculates a motion vector that connects each feature point of the past image and the corresponding point of the latest image corresponding to each feature point. Furthermore, the moving object detection unit 421 calculates a motion vector in which the movement of the pitch angle, yaw angle, and roll angle of the imaging unit 111 calculated in step S254 is corrected.

  In step S256, the moving object detection unit 421 determines whether each feature point is moving. Specifically, the moving object detection unit 421 calculates an epipolar line for each feature point using the inter-camera parameter. The moving object detection unit 421 calculates the angle between the corrected motion vector and the epipolar line for each feature point. The moving object detection unit 421 determines that the feature point is moving when the calculated angle is larger than a predetermined threshold, and determines that the feature point is not moving when the calculated angle is equal to or less than the predetermined threshold. .

  In step S257, the moving object detection unit 421 detects the moving object. For example, as shown in FIG. 27, the moving object detection unit 421 scans the latest image while moving a window 10 pixels in the horizontal direction × 10 pixels in the vertical direction in units of 5 pixels. Then, when there are 10 or more feature points determined to be moving in the window, the moving object detection unit 421 regards the area as an area where the moving object exists. Note that the window size and scan unit values shown here are examples, and other values may be adopted.

  In step S257, the moving object detection unit 421 calculates three-dimensional information of the moving object. Assuming that the height of the moving object is T, the coordinates (x, y) of the moving object in the image coordinate system are used to calculate the coordinates of the moving object in the camera coordinate system ( X, Y, Z) can be calculated.

  In Expression (12), α, β, and γ represent the installation roll angle, the installation pitch angle, and the installation yaw angle of the imaging unit 111, and H represents the height of the imaging unit 111 from the road surface.

  Then, the position (X, Y, Z) of the camera coordinate system of the moving object can be converted to the position (Xg, Yg, Zg) of the road surface coordinate system by the following equation (13).

  As described above, the three-dimensional information of the detected moving object, that is, the position (Xg, Yg, Zg) of the road surface coordinate system can be calculated. Then, the obstacle detection unit 123 obtains three-dimensional information for all moving objects detected from the image.

  At this time, for example, the three-dimensional information of the moving object is calculated by setting the height T of the moving object T = 0, that is, considering the point on the road surface.

  The moving object detection unit 421 supplies moving object information including presence / absence of the moving object and three-dimensional information (that is, position information) of the detected moving object to the risk ranking setting unit 422.

  Returning to FIG. 25, in step S210, the danger ranking setting unit 422 sets the danger ranking of the detected obstacle by the same process as in step S9 of FIG. However, among the obstacles detected by the obstacle detection unit 123, the danger rank setting unit 422 calculates the obstacles detected by the moving object detection unit 421 as moving objects 3 calculated by the moving object detection unit 421. For the other obstacles using the dimension information, the risk ranking of the obstacle is set using the three-dimensional information calculated by the obstacle detection unit 123.

  In addition, the danger rank setting unit 422 includes the three-dimensional information of the obstacle detected as the moving object by the moving object detection unit 421 among the three-dimensional information of the obstacle included in the obstacle information acquired from the obstacle detection unit 123. Is replaced with the three-dimensional information calculated by the moving object detection unit 421. Further, the danger rank setting unit 422 adds the set danger rank of each obstacle to the obstacle information and supplies the result to the result output unit 113 and the necessary movement amount setting unit 141.

  The processing in steps S211 to S215 is the same as the processing in steps S11 to S14 in FIG.

  In this manner, by adding the moving object detection process, the distance between the vehicle 201 and the moving object can be measured more accurately, and more accurate information can be notified to the driver. It becomes possible.

  Note that the processing in steps S208 and S209 can be performed in parallel or the processing order can be reversed. Further, since the processing of steps S252 to S254 executed by the moving object detection unit 421 is the same as the processing of steps S52 to S54 of FIG. 5 executed by the obstacle detection unit 123, either one is executed, and the information May be shared.

  Furthermore, when the obstacle to be calculated is detected as a moving object by the moving object detection unit 421 and is approaching the vehicle 201, the following distance is not the distance between the current vehicle 201 and the obstacle to be calculated. The required movement amount B may be calculated based on the predicted distance between the vehicle 201 and the calculation target obstacle during the obstacle detection process.

  For example, the distance between the calculation target obstacle that is a moving object and the vehicle 201 is detected as 1.5 m in the previous obstacle detection process, and the calculation target obstacle and the vehicle 201 are detected in the current obstacle detection process. When the distance is detected as 1.3 m, the distance between the calculation target obstacle and the vehicle 201 in the next obstacle detection process is predicted to be about 1.1 m. In this case, the required movement amount B may be calculated using 1.1 m which is the predicted value of the next distance instead of 1.3 m which is the current distance. Thereby, it can prevent more reliably that the vehicle 201 collides with a calculation object obstacle by the next obstacle detection process.

  Next, a fourth embodiment of the present invention will be described with reference to FIGS. Note that the fourth embodiment of the present invention combines the second embodiment of the present invention and the third embodiment of the present invention, determines the attributes of the obstacle in more detail, and determines the determined obstacle. The warning to the driver is changed according to the attribute of the object.

  FIG. 28 is a block diagram showing a third embodiment of an obstacle detection system to which the present invention is applied. The obstacle detection system 501 of FIG. 28 is configured to include an imaging unit 111, an image processing unit 511, and a result output unit 512. In addition, the image processing unit 511 includes an image acquisition unit 121, a storage unit 122, an obstacle detection unit 123, a risk ranking setting unit 124, a movement amount detection unit 125, an image acquisition control unit 126, a moving object detection unit 421, and a risk ranking setting. A unit 422 and an obstacle attribute determination unit 521 are included.

  In the figure, portions corresponding to those in FIG. 24 are denoted by the same reference numerals, and description of portions having the same processing will be omitted because it will be repeated. Hereinafter, it is assumed that the obstacle detection system 501 is installed in the vehicle 201 and the imaging unit 111 is installed at the position shown in FIG. 3 described above.

  The obstacle detection system 501 is different from the obstacle detection system 401 in FIG. 24 in that the image processing unit 411 is replaced with an image processing unit 511 and the result output unit 113 is replaced with a result output unit 512. Also, the image processing unit 511 of the obstacle detection system 501 is different from the image processing unit 411 of the obstacle detection system 401 in that an obstacle attribute determination unit 521 is added.

  The obstacle attribute determination unit 521 acquires the obstacle information from the danger rank setting unit 422, and determines the attribute of the obstacle detected by the obstacle detection unit 123 based on the obstacle information. The obstacle attribute determination unit 521 adds the obstacle attribute determination result to the obstacle information acquired from the danger ranking setting unit 422, supplies the result to the result output unit 512, and stores the result in the storage unit 122.

  The result output unit 512 includes, for example, a display device such as a display of a car navigation system, and an audio output device such as a speaker, a buzzer, or a siren. Similar to the result output unit 113 of the obstacle detection system 101, the result output unit 512 acquires an image obtained by imaging the rear of the vehicle 201 stored in the storage unit 122, and the obstacle detection result is acquired in the acquired image. The information indicating is superimposed and displayed. Further, the result output unit 512 outputs an alarm sound based on the result of the obstacle attribute determination.

  Next, the obstacle detection process executed by the obstacle detection system 501 will be described with reference to the flowchart of FIG. This process is started when, for example, the accessory (ACC) power supply of the vehicle 201 is turned on, and is ended when the accessory power supply is turned off.

  The processing in steps S301 through S309 is the same as the processing in steps S201 through S209 in FIG.

  In step S310, the danger ranking setting unit 422 sets the danger ranking of the detected obstacle, as in the process of step S210 of FIG. In addition, the danger rank setting unit 422 includes the three-dimensional information of the obstacle detected as the moving object by the moving object detection unit 421 among the three-dimensional information of the obstacle included in the obstacle information acquired from the obstacle detection unit 123. Is replaced with the three-dimensional information calculated by the moving object detection unit 421. Further, the danger rank setting unit 422 adds the set danger rank of each obstacle to the obstacle information, and supplies the obstacle information to the obstacle attribute determination unit 521 and the necessary movement amount setting unit 141.

  In step S311, the obstacle attribute determination unit 521 executes obstacle attribute determination processing. Here, details of the obstacle attribute determination processing will be described with reference to the flowchart of FIG.

  The processing in steps S351 through S356 is the same as the processing in steps S151 through S156 in FIG. 20, and the description thereof will be omitted because it will be repeated.

  In step S357, the obstacle attribute determination unit 521 calculates the movement amount of each obstacle. Specifically, the obstacle attribute determination unit 521 calculates, for each obstacle, a value obtained by subtracting the distance between the current vehicle 201 and the obstacle from the previous distance between the vehicle 201 and the obstacle. Is calculated as a movement amount M. Therefore, the movement amount M is a positive value when the distance between the vehicle 201 and the obstacle is approaching, and is a negative value when the distance is moving away.

  In step S358, the obstacle attribute determination unit 521 determines whether there is an obstacle that has approached more than a predetermined threshold based on the result of the process in step S357. That is, the obstacle attribute determination unit 521 determines whether the movement amount M is greater than or equal to a predetermined threshold for each obstacle.

  The threshold is set based on, for example, the amount of movement of the vehicle 201 from the previous obstacle detection process to the current obstacle detection process or the speed of the vehicle 201.

  Further, even with the same amount of movement, obstacles that are closer to the vehicle 201 are more dangerous, and distance obstacles generally have a larger distance measurement error, and the amount of movement is less reliable. Considering this point, a different threshold may be set according to the distance to each obstacle.

  In step S359, the obstacle attribute determination unit 521 determines the attribute of the obstacle. Here, an example of obstacle attribute determination will be described with reference to FIGS. 31 and 33. 31 and 32 show an example of a table used for obstacle attribute determination, and FIG. 33 shows an example of an obstacle attribute.

  In FIG. 31 and FIG. 32, as compared with FIG. 22, a determination condition E is added, and the state of the obstacle is subdivided accordingly. Also, in FIG. 33, an obstacle attribute E is added as compared to FIG. The obstacle attribute E indicates that there is an obstacle that has approached the vehicle 201 by a predetermined threshold value or more.

  State 1 is the same as state 1 in FIG. In this case, the obstacle attribute is not particularly set as in the case of the state 1 in FIG.

  State 2 is the same as state 2 in FIG. In this case, the obstacle attribute is set to D1 as in the case of the state 2 in FIG.

  The state 3A is a state in which there is no obstacle approaching the predetermined threshold or more in the state 3 of FIG. In this case, the obstacle attribute is not particularly set as in the case of the state 3 in FIG.

  State 3B is a state in which there is an obstacle approaching a predetermined threshold or more in state 3 of FIG. In this case, the obstacle attribute is set to E.

  State 4A is a state in which there is no obstacle approaching a predetermined threshold or more in state 4 of FIG. In this case, the obstacle attribute is set to D1 as in the case of the state 4 in FIG.

  State 4B is a state in which there is an obstacle approaching a predetermined threshold or more in state 4 of FIG. In this case, the obstacle attribute is set to D1 + E.

  State 5A is a state in which there is no obstacle approaching a predetermined threshold or more in state 5 of FIG. In this case, as in the case of the state 5 in FIG. 22, the obstacle attribute is set to C + D2.

  State 5B is a state in which there is an obstacle approaching a predetermined threshold or more in state 5 of FIG. In this case, the obstacle attribute is set to C + D2 + E.

  The state 6A is a state in which there is no obstacle approaching the predetermined threshold or more in the state 6 of FIG. In this case, the obstacle attribute is set to C as in the case of the state 6 in FIG.

  State 6B is a state in which there is an obstacle approaching a predetermined threshold or more in state 6 of FIG. In this case, the obstacle attribute is set to C + E.

  The state 7A is a state in which there is no obstacle approaching the predetermined threshold or more in the state 7 of FIG. In this case, as in the case of the state 7 in FIG. 22, the obstacle attribute is set to C + D1.

  State 7B is a state in which there is an obstacle approaching a predetermined threshold or more in state 7 of FIG. In this case, the obstacle attribute is set to C + D1 + E.

  The state 8A is a state in which there is no obstacle approaching the predetermined threshold or more in the state 8 of FIG. In this case, as in the case of the state 8 in FIG. 22, the obstacle attribute is set to B + D2.

  State 8B is a state in which there is an obstacle approaching a predetermined threshold or more in state 8 of FIG. In this case, the obstacle attribute is set to B + D2 + E.

  The state 9A is a state in which there is no obstacle approaching the predetermined threshold or more in the state 9 of FIG. In this case, the obstacle attribute is set to B as in the case of the state 9 in FIG.

  State 9B is a state in which there is an obstacle approaching a predetermined threshold or more in state 9 of FIG. In this case, the obstacle attribute is set to B + E.

  State 10A is a state in which there is no obstacle approaching a predetermined threshold or more in state 10 of FIG. In this case, the obstacle attribute is set to B + C + D2 as in the case of the state 10 in FIG.

  State 10B is a state in which there is an obstacle approaching a predetermined threshold or more in state 10 of FIG. In this case, the obstacle attribute is set to B + C + D2 + E.

  The state 11A is a state in which there is no obstacle approaching the predetermined threshold or more in the state 11 of FIG. In this case, the obstacle attribute is set to B + C as in the case of the state 11 in FIG.

  State 11B is a state in which there is an obstacle approaching a predetermined threshold or more in state 11 of FIG. In this case, the obstacle attribute is set to B + C + E.

  Then, after determining the obstacle attribute, the obstacle attribute determination unit 521 adds the determination result of the obstacle attribute to the obstacle information acquired from the danger rank setting unit 422 and supplies the result to the result output unit 512. .

  In step S360, the obstacle information indicating the current obstacle detection result is stored in the storage unit 122, as in the process of step S158 of FIG. Thereafter, the obstacle attribute determination process ends.

  Returning to FIG. 29, in step S312, the result output unit 512 displays the latest image on which the effect of enhancing the detected obstacle is displayed, in the same manner as the process of step S111 of FIG. An alarm sound corresponding to the obstacle attribute determined by the object attribute determination unit 521 is output.

  Here, a specific example of the alarm sound will be described with reference to FIGS. 31 and 32.

  31 and FIG. 32 are compared with FIG. 22, when the obstacle attribute E is set, that is, when there is an obstacle approaching a predetermined threshold or more, a warning sound of sound 2 is output. Is different. That is, in step S312, in addition to the alarm sound output in step S111 of FIG. 19, when there is an obstacle approaching a predetermined threshold or more, an alarm sound of sound 2 is output. The sound 2 is set to a tone color (for example, “boo”) different from the above-described sound 1aL, sound 1aS, sound 1bL, and sound 1bS so that it can be easily understood that an obstacle is approaching.

  The processing in steps S313 to S316 is the same as the processing in steps S11 to S14 in FIG.

  In this way, when an obstacle approaches, a warning can be given to the driver.

  Hereinafter, modifications of the embodiment of the present invention will be described.

  The present invention is applicable not only to a system for detecting obstacles behind the vehicle, but also to a system for detecting obstacles in front of the vehicle, or a system for detecting obstacles both in front and rear of the vehicle. Is possible.

  Further, an upper limit value may be provided for the required movement amount B. As a result, after setting the required travel distance B for a distant obstacle, the vehicle 201 changes its direction of travel and approaches or collides with other nearby obstacles before moving by the required travel distance B. It can prevent more reliably.

  Furthermore, a lower limit value may be provided for the required movement amount B. This lower limit value is set to, for example, the minimum value that can accurately detect the movement amount of the vehicle. For example, consider a case in which the vehicle speed is detected using a vehicle speed detected by a vehicle speed sensor, and the reliability of the vehicle speed sensor decreases when the vehicle speed is smaller than 2 km / h (0.55 m / s). Note that the frame rate of the imaging unit 111 is 30 fps (0.033 s / frame). In this case, the movement amount of the vehicle that can be accurately detected in units of one frame is 0.018 m / frame (= 0.55 m / s × 0.033 s / frame), and the lower limit value of the necessary movement amount B is, for example, 0.018 m. Is set.

  Further, the required movement amount B is changed in a staircase pattern instead of simply proportional to the square of the distance between the vehicle and the calculation target obstacle based on the above-described equation (9). It may be. This is effective when the detected value of the moving amount of the vehicle becomes a discrete value, for example, when the moving amount of the vehicle is obtained using a wheel speed sensor.

  Furthermore, a plurality of calculation target obstacles that are targets for calculating the required movement amount B may be set. In this case, different required movement amounts B are set for the respective obstacles to be calculated. Then, according to each required movement amount B, acquisition of an image is individually requested, and accordingly, an image is acquired from the imaging unit 111, and three-dimensional information of each calculation target obstacle is obtained. In this case, it is sufficient that the storage unit 122 has a capacity for storing two images of the latest image and the past image for each obstacle to be calculated.

  Moreover, although the example which applies this invention to a vehicle was shown in the above description, this invention can also be applied to other moving bodies, such as vehicles, such as a motorbike, a bicycle, a ship, and a train.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.

  FIG. 34 is a block diagram illustrating a hardware configuration example of a computer that executes the above-described series of processing by a program.

  In a computer, a CPU (Central Processing Unit) 701, a ROM (Read Only Memory) 702, and a RAM (Random Access Memory) 703 are connected to each other by a bus 704.

  An input / output interface 705 is further connected to the bus 704. An input unit 706, an output unit 707, a storage unit 708, a communication unit 709, and a drive 710 are connected to the input / output interface 705.

  The input unit 706 includes a keyboard, a mouse, a microphone, and the like. The output unit 707 includes a display, a speaker, and the like. The storage unit 708 includes a hard disk, a nonvolatile memory, and the like. The communication unit 709 includes a network interface. The drive 710 drives a removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer configured as described above, the CPU 701 loads the program stored in the storage unit 708 to the RAM 703 via the input / output interface 705 and the bus 704 and executes the program, for example. Is performed.

  The program executed by the computer (CPU 701) can be provided by being recorded on a removable medium 711 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer, the program can be installed in the storage unit 708 via the input / output interface 705 by attaching the removable medium 711 to the drive 710. Further, the program can be received by the communication unit 709 via a wired or wireless transmission medium and installed in the storage unit 708. In addition, the program can be installed in advance in the ROM 702 or the storage unit 708.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  Further, in the present specification, the term “system” means an overall apparatus composed of a plurality of apparatuses and means.

  Furthermore, the embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 101 Obstacle detection system 111 Image pick-up part 112 Image processing part 113 Result output part 121 Image acquisition part 122 Storage part 123 Obstacle detection part 124 Risk order setting part 125 Movement amount detection part 126 Image acquisition control part 141 Necessary movement amount setting part 142 Acquisition request output unit 201 Vehicle 301 Obstacle detection system 311 Image processing unit 312 Result output unit 321 Obstacle attribute determination unit 401 Obstacle detection system 411 Image processing unit 421 Moving object detection unit 422 Risk ranking setting unit 501 Obstacle detection system 511 Image processing unit 512 Result output unit 521 Obstacle attribute determination unit

Claims (6)

In an image processing apparatus that performs a process of measuring a distance to an object using an image captured by an imaging apparatus provided on a moving body,
Measuring means for measuring the distance to the object by a moving stereo method using two images taken at different positions;
Based on the distance to the object measured by the measuring means, it is necessary to be the distance from the imaging position of the image captured later of the two images used for measurement to the imaging position of the image used for the next measurement An image processing apparatus comprising: setting means for setting a movement amount. When the measurement unit measures the distance to the object using the first image and the second image captured after the first image,
The setting means sets the distance from the imaging position of the second image to the imaging position of the image used for the next measurement as the necessary movement amount based on the distance to the object measured by the measuring means. ,
The measurement unit measures a distance to the object using a third image captured when the movable body moves the necessary movement amount from an imaging position of the second image. The image processing apparatus described. When the distance to the target is measured by stereo vision, the setting means includes a relational expression representing a relationship between the distance to the target, a measurement error of the distance to the target, and a baseline length as a distance to the target. The image processing apparatus according to claim 1, wherein the distance to the object measured by the measurement unit is substituted, and the baseline length obtained by substituting a desired value for the measurement error is set as the necessary movement amount. . When a plurality of objects are present in the image, the setting means is configured such that an object having the closest distance to the moving body is a distance between the moving body and an object existing in a traveling direction of the moving body. The required amount of movement is calculated based on the distance between the closest object or one of the objects existing in the region where the moving object can travel and the closest object to the moving object. The image processing apparatus according to claim 1, which is set. An image processing apparatus that performs a process of measuring a distance to an object using an image captured by an imaging apparatus provided on a moving body,
Using two images taken at different positions, measure the distance to the object by moving stereo method,
Based on the measured distance to the object, a necessary movement amount that is the distance from the imaging position of the image captured later of the two images used for measurement to the imaging position of the image used for the next measurement is set. An image processing method including a step. In a computer that performs processing to measure the distance to an object using an image captured by an imaging device provided on a moving body,
Using two images taken at different positions, measure the distance to the object by moving stereo method,
Based on the measured distance to the object, a necessary movement amount that is the distance from the imaging position of the image captured later of the two images used for measurement to the imaging position of the image used for the next measurement is set. A program that executes processing including steps.

Images