JP2005235017A - Device for recognizing vehicle traveling state - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for recognizing vehicle traveling state, capable of generating accurate photographed image information which will not be influenced by the initial mounting state or the like of an imaging means. <P>SOLUTION: The vehicle traveling state recognizing device 10 comprises the imaging means 12 mounted on a vehicle body to image the periphery of the vehicle so that the predetermined portion of the vehicle body is included in a part of the imaging area; an image processing means 14 executing image recognition processing to the taken image obtained from the imaging means, to recognize the predetermined portion of the vehicle body and a predetermined recognition object; and a recognition information generating means 16 relatively derive information for the recognition object in the taken image, based on information for the predetermined portion of the vehicle body in the same taken image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a vehicle travel state recognition device that recognizes a travel state of a vehicle using captured image information.

2. Description of the Related Art Conventionally, a technique for recognizing a lane on a road using captured image information from two left and right imaging means (CCD cameras) is known (see, for example, Patent Document 1). In this prior art, two left and right CCD cameras are mounted on the vehicle body so as to capture a predetermined area on the left and right sides of the vehicle, and each captured image information (coordinate values of feature points) is processed in association with each other, thereby detecting lanes. Trying to improve accuracy.
Japanese Patent Laid-Open No. 10-11580

  However, since the imaging area captured by the imaging unit changes depending on the initial mounting state of the imaging unit and the subsequent change in the mounting state over time, the coordinate meaning of the captured image information obtained by the imaging unit In other words, there is a problem that the position of the actual coordinate corresponding to the pixel position of the captured image is not always accurate.

  Therefore, an object of the present invention is to provide a vehicle traveling state recognition device that can generate highly accurate captured image information that is not affected by an initial mounting state or the like of an imaging unit.

In order to solve the above-described problem, according to one aspect of the present invention, an imaging unit that is attached to a vehicle body so as to image the periphery of the vehicle and includes a predetermined part of the vehicle body in a part of the imaging area;
An image processing unit that performs an image recognition process on a captured image obtained from the imaging unit and recognizes a predetermined part of the vehicle body and a predetermined recognition target;
Recognition information generation means for relatively deriving information on the recognition object in the captured image on the basis of information on the predetermined part of the vehicle body in the captured image based on the image recognition result by the image processing means. A vehicle travel situation recognition device is provided.

  In the present embodiment, the imaging means may be provided in two symmetrically with respect to the vehicle longitudinal axis, and the predetermined recognition object may be a lane on a road on the side of the vehicle. Further, the information related to the recognition target generated by the recognition information generating means may be information related to the relative position and / or direction of the recognition target with respect to a predetermined part of the vehicle body. Information on these recognition objects can be effectively used for vehicle control.

  According to the present invention, it is possible to obtain high-accuracy captured image information that is not affected by the initial mounting state or the like of the imaging unit.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of a vehicle travel situation recognition device 10 according to the present invention. The vehicle travel state recognition device 10 of this embodiment includes an image pickup means 12 such as a CCD (stereo) camera. As shown in FIG. 1, the imaging means 12 is provided on each of the left and right sides of the vehicle. For example, the imaging means 12 may be mounted in the vicinity of a vehicle door mirror symmetrically with respect to the vehicle longitudinal axis. The imaging means 12 is fixed so that the imaging area partially includes the vehicle body. For example, the image pickup means 12 shown in FIG. 1 is attached so as to pick up an image of the left and right side regions of the vehicle obliquely forward and downward, and is attached so that the fender portion of the vehicle body is included in the image pickup area.

  The image data obtained by the imaging means 12 is supplied to an image processing unit 14 that may be configured with an FPGA (Field Programmable Gate Array) or the like. The image processing unit 14 recognizes a predetermined part of the vehicle body included in the imaging area together with a predetermined recognition object using an appropriate image recognition processing technique. For example, the image processing unit 14 sets the fender part (fender line) of the vehicle body as the recognition target together with the lane (white line) as the predetermined recognition target object, and executes image recognition processing based on each given feature point. Good.

  The result of the image recognition process performed by the image processing unit 14 is supplied to the recognition information generation unit 16. The recognition information generation unit 16 according to the present embodiment uses the relative positional relationship on the captured image between a predetermined recognition object other than the vehicle body and a predetermined part of the vehicle body, and the predetermined recognition object and the vehicle. Is recognized, and based on the relative positional relationship, various types of information relating to the predetermined recognition object are generated and stored.

  By the way, generally, the position (pixel position) in the imaging area (captured image) and the corresponding position in the real space are the imaging characteristics of the imaging means 12 (such as the lens characteristics of the wide-angle lens) and the imaging means 12. Using a predetermined conversion formula based on the mounting position, angle, and the like, one-to-one correspondence is made. However, this coordinate conversion formula is based on the premise that the imaging unit 12 is mounted as designed, so that there is a deviation in the initial mounting state of the imaging unit 12, the mounting state has changed over time, etc. In some cases, the imaging area changes, an error occurs in the position information after coordinate conversion, and the accuracy of the image recognition information decreases.

  On the other hand, in the present embodiment, as described above, a predetermined part of the vehicle body is included in the imaging area of the imaging unit 12, and position information regarding the recognition target in the captured image is the position of the vehicle body in the captured image. It is relatively derived with reference to position information relating to the predetermined part. As a result, in this embodiment, it is not affected by a change in the mounting state of the imaging means 12 (including a change with time) (that is, a coordinate conversion error that may occur due to a change in the imaging area is compensated ( Therefore, highly reliable image recognition information can be obtained. In other words, by calculating the position of the recognition object in the captured image using the predetermined part of the vehicle body in the captured image as a reference point, the reference point and the recognition object Since the positional relationship with the position does not substantially change, it is possible to obtain highly accurate position information that is not affected by the mounting state of the imaging unit 12.

  In the present embodiment, the position information related to the recognition target in the captured image may be recognized in various manners according to the vehicle control device that uses the information. For example, the position information of the recognition target may be recognized by a two-dimensional coordinate system in a horizontal plane with a predetermined point of a predetermined part of the vehicle body as the origin (reference point), and the three-dimensional position of the predetermined point (for example, GPS If it can be detected (based on the positioning result), it may be recognized in a three-dimensional absolute coordinate system based on the three-dimensional position of the predetermined point.

  In the present embodiment, for a recognition object having a direction such as a white line, for example, information regarding the direction may be derived based on a part of the vehicle body. That is, the direction information regarding the recognition target object in the captured image may be relatively derived with reference to the direction information regarding the predetermined part of the vehicle body in the captured image. For example, the directional relationship between the extending direction of the white line as the recognition object and the traveling direction of the vehicle is a predetermined part of the vehicle body in the captured image (for example, predetermined with respect to the front and rear of the vehicle). Can be derived on the basis of the extending direction of a part having a directivity of (such as a line). Even in this case, similarly, the coordinate conversion error caused by the change in the mounting state of the imaging unit 12 (including the change with time) is compensated (cancelled), so that highly accurate image recognition information (information on the direction) is obtained. be able to.

  As described above, the information related to the recognition target obtained from the imaging result of the imaging unit 12 is information relative to the vehicle, and can be used as information for recognizing the traveling state of the vehicle itself. For example, it can be used effectively for control parameter correction and adaptive control in various vehicle control devices. Hereinafter, the usage example will be described.

  FIG. 1 shows a configuration of a vehicle control device 30 that cooperates with the vehicle travel state recognition device 10. The state information calculation unit 32 of the vehicle control device 30 is connected to the recognition information generation unit 16 of the vehicle travel state recognition device 10, and information relating to the recognition object obtained as described above (in the example of FIG. Left and right distances Xl and Xr) between the vehicle and the vehicle body. Various sensors such as a vehicle speed sensor 40 and a rudder angle sensor 42 are connected to the state quantity calculation unit 32, and information regarding the running state of the vehicle is input.

  The control parameter correction determination unit 34 of the vehicle control device 30 determines the validity of the control parameter used for vehicle control based on the calculation result of the state quantity by the state quantity calculation unit 32 (that is, the vehicle is controlled as intended). If necessary, request correction of the control parameter. The control unit 36 of the vehicle control device 30 corrects / updates the control parameter in response to a request from the control parameter correction determination unit 34, and executes predetermined vehicle control based on the updated control parameter.

  FIG. 2 is a flowchart of roll angle and pitch angle measurement and correction control processing realized by one embodiment of the vehicle control device 30 described above. FIG. 3 is an explanatory diagram of this measurement and correction control process. The vehicle control device 30 according to the present embodiment is a device that actively controls a vehicle suspension system based on predetermined control parameters. The process shown in FIG. 2 may be executed every predetermined cycle (for example, every information generation cycle by the recognition information generation unit 16) while the vehicle control device 30 actively controls the suspension system.

  First, in step 110, the state quantity calculation unit 32 of the vehicle control device 30 receives information from the recognition information generation unit 16 (the distances X1 and Xr on the left and right between the white line and a predetermined part of the vehicle body, the outermost edge of the imaging area). Left and right distances Yl and Yr and widths Wl and Wr of the left and right white lines between the vehicle and a predetermined part of the vehicle body are taken in (see FIG. 3). The state quantity calculation unit 32 uses, for example, a predetermined map to calculate the roll angle θ1 and the pitch angle θ2 (see FIG. 3) corresponding to the information (steps 120 and 130). The pitch angle θ2 may be derived from the captured images of the imaging means 12 attached at positions separated in the vehicle front-rear direction on the same principle as the roll angle θ1.

  Here, the predetermined map that may be used in steps 120 and 130 may be experimentally derived as a simple method. For example, based on the amount of change in Xr + Xl and / or Yr + Yl and the amount of change in Wl and Wr obtained from the image processing result for the captured images of the left and right imaging means 12 when the vehicle is intentionally rolled by a known amount Thus, mapping data indicating the relationship between the amount of change and the roll angle may be created. This is because the value of Xr + Xl changes according to the roll angle, and the values of the white line widths Wl and Wr change according to the roll angle (because the imaging means 12 approaches or separates from the ground). Based on that. A map relating to the pitch angle may be created experimentally as well.

  Next, in step 140, the control parameter correction determination unit 34 compares the roll angle θ1 and the pitch angle θ2 calculated in steps 120 and 130 with respective target values, and in step 150, the comparison result is obtained. Based on this, it is determined whether or not the control parameter (for example, gain) should be corrected. If it is determined that correction of the control parameter is not necessary (for example, when the target value is within a predetermined allowable range), it can be determined that the control is being executed normally, and thus the current processing routine ends. Is done.

  On the other hand, when it is determined in step 150 that the control parameter should be corrected, the control parameter correction determination unit 34 determines whether the required correction of the control parameter is within the appropriate range (step 160). As a result, if it can be determined that the value is within the appropriate range, the control parameter is required to be corrected. In response to this, the control unit 36 of the vehicle control device 30 corrects and updates the control parameter (step 170), and thereafter executes vehicle control based on the updated control parameter. On the other hand, if it is determined that the necessary correction of the control parameter is outside the proper range, the control system may determine that the control system is abnormal, and the control parameter correction determination unit 34 outputs a warning to the control unit 36 or stops the control. Request. In response to this, the control unit 36 outputs a warning or stops control (step 180).

  FIG. 4 is a flowchart of the lane maintenance control executed by the embodiment of the vehicle control device 30 described above. FIG. 5 is an explanatory diagram of this lane maintenance control. The vehicle control apparatus 30 according to the present embodiment sets the steering system in a predetermined manner so that the vehicle travels along the target track based on the detection result of a camera or the like (a camera different from the imaging unit 12) that images the front of the vehicle. It is a device that actively controls using the control parameters. The process shown in FIG. 4 may be started when the vehicle control device 30 starts active control of the steering system.

  First, in step 210, in order to confirm the responsiveness of the vehicle to the command from the vehicle control device 30, information from the recognition information generation unit 16 at the start of control (distances X10 on the left and right between the white line and a predetermined part of the vehicle body). , Xr0). The command at the start of control by the vehicle control device 30 is, for example, as shown in FIG. 5 from the left and right white lines with respect to a vehicle that is close to one white line side with respect to the center line (indicated by a dotted line). This may be a command for traveling a predetermined distance away (that is, traveling while maintaining the center line).

  Next, a convergence determination is executed using predetermined determination conditions for Xl and Xr (step 220) captured every predetermined period (step 230). For example, the determination condition may be satisfied when a state where both Xl and Xr are equal to or less than a predetermined value continues for a predetermined period or more. When the determination condition is satisfied, the convergence time T1 required for convergence (that is, the time from the control start time to the convergence time) is calculated based on the time at that time (step 240). An average vehicle speed V1 of the vehicle is calculated (step 250).

  Next, the control parameter correction determination unit 34 applies predetermined determination conditions to the initial values X10, Xr0, the convergence time T1, and the average vehicle speed V1 acquired and calculated in the above-described steps 210 to 230, thereby stabilizing the convergence. Then, it is determined whether or not the control parameter (for example, gain) should be corrected (step 260). For example, the convergence stability may be determined by comparing the convergence time T1 with a target convergence time set in advance according to X10, Xr0, and V1. If it is determined that the control parameter need not be corrected, it can be determined that the control is normally performed with convergence stability, and the process proceeds to step 291.

  On the other hand, if it is determined in step 260 that the control parameter should be corrected, the control parameter correction determination unit 34 determines whether the required correction of the control parameter is within the appropriate range (step 270), and the appropriate range. If it can be determined that it is within the range, the control parameter is required to be corrected. In response to this, the control parameter is corrected and updated by the control unit 36 of the vehicle control device 30 (step 280), and the process proceeds to step 291. On the other hand, if it is determined that the necessary correction of the control parameter is outside the proper range, the control system may determine that the control system is abnormal, and the control parameter correction determination unit 34 outputs a warning to the control unit 36 or stops the control. Request. In response to this, the control unit 36 outputs a warning or stops control (step 290).

  In step 291, the degree of deviation (deviation) δ with respect to the target values of the data of Xl and Xr over a predetermined time after convergence stabilization is calculated in order to evaluate the lane maintenance accuracy after convergence stabilization. Next, in step 292, it is determined whether or not the variation degree δ calculated in step 291 is within a predetermined range. If the degree of variation δ is within the predetermined range, the control is normally executed. The determination can be made and the process is terminated. On the other hand, if it is determined that the variation degree δ is outside the predetermined range, the process returns to step 270 to determine whether or not the control parameter can be corrected. Depending on the result, warning processing or the like (step 290) or A control parameter correction process (step 280) is executed.

  FIG. 6 is a flowchart of the lane departure avoidance control executed by the embodiment of the vehicle control device 30 described above. FIG. 7 is an explanatory diagram of this lane departure avoidance control. The vehicle control device 30 of the present embodiment is a device that actively controls the steering system of the vehicle using predetermined control parameters so as to avoid the departure when the vehicle is about to depart from the lane.

First, in step 310, in order to confirm the controllability by the vehicle control device 30, information from the recognition information generation unit 16 at the start of control (distances X10 and Xr0 on the left and right between the white line and a predetermined part of the vehicle body) are obtained. It is captured. Next, Xl and Xr are taken at predetermined intervals until the vehicle approaches the white line on one side and departure avoidance control is activated (step 320). When the vehicle is closest to the white line on one side and the departure avoidance control is activated in step 330, the distance Xl _min or Xr _min (see FIG. 7) at the time of closest approach is calculated (step 340), and the vehicle at that time The lane direction component V1 of the velocity vector and the departure direction component (deviation velocity) V2 perpendicular thereto are calculated (step 350). V1 and V2 may be average values from the control start time to the departure avoidance control start time.

Then, the control parameter correction determining unit 34 obtains, calculated initial values in step 310, 340, 350 XL0, xr0, speed V1, V2, and, given determined for the distance of closest approach _{Xl min} or _{Xr min} The condition is applied to determine the departure avoidance ability, and it is determined whether or not the control parameter (for example, gain) should be corrected (step 360). For example, the deviation avoidance ability may be determined by comparing the closest approach distance Xl _min or Xr _min with a target closest approach distance set in advance according to X10, Xr0 and the speeds V1, V2. If it is determined that the control parameter need not be corrected, it can be determined that the control is being executed normally, and the process is terminated.

  On the other hand, if it is determined in step 360 that the control parameter should be corrected, the control parameter correction determination unit 34 determines whether the required correction of the control parameter is within an appropriate range (step 370). As a result, if it can be determined that the value is within the appropriate range, the control parameter is required to be corrected. In response to this, the control unit 36 of the vehicle control device 30 corrects / updates the control parameter (step 380), and thereafter executes vehicle control based on the updated control parameter. On the other hand, if it is determined that the necessary correction of the control parameter is outside the proper range, the control system may determine that the control system is abnormal, and the control parameter correction determination unit 34 outputs a warning to the control unit 36 or stops the control. Request. In response to this, the control unit 36 outputs a warning or stops control (step 390).

  As is clear from the above description, according to each of the above-described embodiments, it is possible to quantitatively measure the influence on the vehicle when the gain is changed by a certain amount in the control system of each vehicle. It is also possible to optimize the change reference of the adjustment gain amount, that is, the increase amount when changing the gain. Further, each of the above-described embodiments can be used not only at the time of actual travel after completion of the vehicle, but also at a vehicle shipment inspection, a development stage, and the like.

  Further, in each of the above-described embodiments, information from the recognition information generation unit 16 (left and right distances X1, Xr, etc. between the white line and a predetermined part of the vehicle body) is used for feedback control. It can also be used.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

It is a block diagram which shows one Example of the vehicle travel condition recognition apparatus 10 by this invention. 4 is a flowchart of roll angle and pitch angle measurement and correction control processing realized by an embodiment of the vehicle control device 30. It is explanatory drawing of the process of the flowchart shown in FIG. 4 is a flowchart of lane maintenance control executed by an embodiment of the vehicle control device 30. It is explanatory drawing of the process of the flowchart shown in FIG. 4 is a flowchart of lane departure avoidance control executed by an embodiment of the vehicle control device 30. It is explanatory drawing of the process of the flowchart shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Vehicle traveling condition recognition apparatus 12 Imaging means 14 Image processing part 16 Recognition information generation part 30 Vehicle control apparatus 32 State quantity calculation part 34 Control parameter correction determination part 36 Control part 40 Vehicle speed sensor 42 Rudder angle sensor

Claims (4)

An imaging means attached to the vehicle body so as to image the periphery of the vehicle and including a predetermined part of the vehicle body in a part of the imaging area;
Image processing means for executing an image recognition process on a captured image of the imaging means, and recognizing a predetermined part of the vehicle body and a predetermined recognition object in the captured image;
Recognition information generation means for relatively deriving information on the recognition object in the captured image on the basis of information on the predetermined part of the vehicle body in the captured image based on the image recognition result by the image processing means. A vehicle travel situation recognition device comprising:   2. The vehicle travel state recognition device according to claim 1, wherein the imaging means is provided in two symmetrically with respect to a vehicle longitudinal axis, and the predetermined recognition object is a lane on a road on a side of the vehicle.   The vehicle travel state recognition according to claim 1, wherein the information related to the recognition object generated by the recognition information generating means is information related to a relative position and / or direction of the recognition object with respect to a predetermined part of the vehicle body. apparatus. A vehicle control device that performs vehicle control using information on the recognition object generated by the recognition information generation means of the vehicle travel situation recognition device according to any one of claims 1 to 3.

Images