JP2012176656A - Parking support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the cost of a harness and the wiring work in sensor connection by making the sensor, for detecting a steering angle in prediction of an advancing passage in parking, needless.SOLUTION: A front wheel is detected from a binarized image (S3) by performing pretreatment such as binarization acquiring the image from a camera (S1, S2). Then, the shape or the outline of the front wheel in the image is collated with a pattern of the front wheel previously stored in a system (S4) and a steering angle is calculated based on the pattern collation result (S5). Next, the advancing route of an own vehicle is estimated based on the steering angle (S6) and a guideline of the estimated advancing track is formed to be displayed on a display (S7). As a result, the steering angle is recognized without using signal from a steering angle sensor to estimate the advancing route of the own vehicle and driving operation of a driver in parking is supported while reducing the cost required for the harness and the wiring work in the connection of the steering angle sensor.

Description

  The present invention relates to a parking assistance device that predicts a traveling route of a vehicle and supports a parking operation.

  2. Description of the Related Art In recent years, parking assistance devices that support a parking operation by predicting and presenting to a driver a traveling route of a vehicle when the vehicle is parked have been developed. For example, Patent Document 1 discloses a parking assist that predicts and calculates a route based on a switching operation in a process in which a vehicle moves from an initial parking position to a parking target position, and draws and displays the predicted result on an overhead image around the vehicle. An apparatus is disclosed.

JP 2010-89716 A

  In the conventional technique disclosed in Patent Document 1, the steering angle of the wheel necessary for predicting the traveling path of the vehicle is acquired by inputting a signal from a steering angle sensor provided in the steering. ing. For this reason, not only a relatively long harness is required to connect the parking assist device and the steering angle sensor, but also a complicated wiring work is required, which causes an increase in cost.

  The present invention has been made in view of the above circumstances, and does not require a sensor for detecting a steering angle in predicting a traveling path during parking, and can reduce the cost of a harness and wiring work due to sensor connection. The object is to provide a device.

  A parking assist device according to the present invention is a parking assist device that assists a driver's driving operation when parking a vehicle. An image capturing unit that captures an image around a vehicle including at least wheels, and an image captured by the image capturing unit. A steering angle recognition unit that recognizes the steering angle of the photographed wheel based on the relationship between the wheel and the steering angle on the image that is extracted by the steering operation and changes the steering operation, and the steering that is recognized by the steering angle recognition unit A course estimation unit that estimates and calculates the travel route of the vehicle based on the corner, and an information presentation unit that presents support information including the travel route estimated by the course estimation unit to the driver.

  According to the present invention, a sensor for detecting a steering angle is not required for predicting a traveling path at the time of parking, and the driver's driving operation at the time of parking is supported while reducing the cost of harness and wiring work by sensor connection. be able to.

The block diagram of the parking assistance apparatus according to the first embodiment of the present invention. As above, an explanatory view showing a side camera for front wheel photography As above, an explanatory diagram showing the estimation of the front wheel position in the image Same as above, explanatory diagram showing the recognition frame in the image Same as above, explanatory diagram when recognizing only steering direction As above, an explanatory diagram showing a display example of a predicted course trajectory Same as above, flowchart of parking support control processing Explanatory drawing of the look-up table which concerns on 2nd Embodiment of this invention and shows the relationship between a wheel area and a steering angle. Same as above, flowchart of parking support control processing

Embodiments of the present invention will be described below with reference to the drawings.
When parking a vehicle such as an automobile, the parking assist device of the present invention captures an image around the vehicle including at least wheels with an in-vehicle camera without requiring a signal from a steering angle sensor, and the captured image The vehicle travel route is predicted from the vehicle, and driving assistance is provided by presenting the predicted travel route to the driver. In the present embodiment, the steering angle is recognized from the shape of the steered wheel (front wheel) that changes as the steering is moved, and the traveling path of the vehicle is predicted based on the recognized steering angle.

  Specifically, as shown in FIG. 1, the parking assist device 1 in the present embodiment is photographed with a camera 2 that constitutes an imaging unit that photographs the periphery of the front wheels, an in-vehicle sensor 3 that detects a vehicle state, and the camera 2. The controller 10 that estimates the steering angle of the front wheels from the image and predicts the traveling path of the host vehicle based on the estimated steering angle and the vehicle information from the vehicle-mounted sensor 3, and the guide information related to the predicted traveling path with images and sounds A guide information presentation device 20 as an information presentation unit presented to the driver is mainly configured.

  In the present embodiment, as the camera 2 constituting the imaging unit, as shown in FIGS. 2 (a) and 2 (b), the camera 2 is integrally installed below the side mirror 4a on the left side in the vehicle forward direction. Two cameras are used, that is, a side camera 2a and a side camera 2b that is integrally installed below the side mirror 4b on the right side in the vehicle forward direction. These left and right side cameras 2a and 2b are both wide-angle cameras, and can capture a peripheral image including a part of the vehicle body on the side of the vehicle and the front wheels.

  Note that the camera 2 is not limited to a side mirror installation type, and any other camera installed for confirming safety around the vehicle can be used as long as the surroundings of the front wheels can be photographed. .

  The in-vehicle sensor 3 is a sensor that can easily acquire a signal with a simple wiring, such as a vehicle speed sensor that detects a vehicle speed and a shift position sensor that detects a shift position (gear position) of a transmission. That is, the parking assistance apparatus 1 recognizes the steering angle from the image of the wheel without using the steering angle sensor that detects the steering angle of the steering wheel.

  On the other hand, the controller 10 is a unit mainly composed of a microcomputer, and includes various circuits for image processing, a memory, peripheral circuits for input / output, and the like. The controller 10 mainly composed of such a microcomputer has a functional configuration based on the steering angle recognition unit 11 that recognizes the steering angle of the front wheels by processing the image captured by the camera 2 and the recognized steering angle. The route estimation unit 12 for estimating the travel route of the host vehicle, and the guide information generation unit 13 that generates guide information such as an image and sound for displaying the estimated travel route and the position of the host vehicle and outputs the guide information to the guide information presentation device 20 It has.

  The guide information presentation device 20 may be a dedicated device including a display and a speaker, but may be configured as a part of a multifunction display of an instrument panel, for example, and includes a navigation device. Alternatively, a display or a speaker of the navigation device may be used.

  The steering angle recognizing unit 11 detects the front wheels from the images photographed by the left and right side cameras 2a and 2b, and recognizes the steering angle from the shape of the front wheels that changes by moving the steering. The position of the front wheel in the image is estimated based on the characteristic part of the vehicle body shown in the image. This is because when the steering angle is small, the shape of the front wheel in the image is small, and the position of the front wheel may not be detected depending on the position and posture of the camera 2. By estimating the position of the front wheel, the front wheel can be reliably detected.

  The characteristic parts of the vehicle body include, for example, an underline of the vehicle body, a door joint, a fender, a door knob, a blinker lamp, and the like. The position of the front wheel is estimated from the positional relationship with these characteristic parts. FIG. 3 shows an example in which the underline L of the vehicle body B is used. The underline L is recognized by edge detection or the like of the image, and the position W1 of the front wheel 5 is estimated from the positional relationship with the underline L and steered. Recognize corners.

  In this case, if the vehicle body is slightly distorted even if it hits the object against any object, etc., there is a large error in the estimation of the front wheel position, and the steering angle is recognized. There is a risk of adversely affecting processing. For this reason, a range in which the position of the front wheel is predicted from the image is set in advance, and the recognition process is performed in this range. For example, as shown in FIG. 4, a recognition frame W2 having a coordinate P (x1, y1) as a base point is set in advance in the image, and the front wheel 5 is detected within the recognition frame W2 to estimate the steering angle. . The coordinates and the size of the recognition frame are set in advance in consideration of camera position / posture errors and the like.

  In the present embodiment, the estimation of the steering angle from the front wheel image is performed using a known pattern matching technique. Specifically, the shape or contour pattern of the front wheels corresponding to the steering angle is stored in advance in the controller 10, the front wheels reflected in the image are compared with the stored pattern, and the matching pattern is Find the steering angle.

  For simplicity, in order to reduce the amount of calculation required for the steering angle recognition process, it may be possible to recognize only the steering direction, that is, the direction of the front wheel in the image, that is, the left or right direction. Specifically, as shown in FIG. 5 (a), when both the tire 5a and the wheel 5b are reflected in the image, the steering direction is the right direction, as shown in FIG. 5 (b). When only the tire 5a is shown in the image and the wheel 5b is not shown, it can be determined that the steering direction is from straight ahead to the left.

  Further, in the present embodiment, the above steering angle recognition is performed on the left and right images by the left and right side cameras 2a and 2b. However, if at least two cameras are used, there is a problem with one of the cameras. Even if this occurs, if one or more cameras operate normally, the steering angle can be recognized from the captured image of the normal camera.

  The course estimation unit 12 uses the steering angle recognized by the steering angle recognition unit 11 to use a route with a recognized steering angle constant (a tire turning angle is constant) or a curve with a tangent to the recognized steering angle (change). A path drawn by a curve having one curve point is predicted and output to the guide information generation unit 13. At this time, since the course direction changes depending on whether the host vehicle moves forward or backward, the course direction is determined based on the shift position from the in-vehicle sensor 3 (shift position sensor).

  The guide information generation unit 13 sets lines with a predetermined interval based on the width of the host vehicle with the predicted route as a center as a guideline for the predicted route trajectory, and generates an image signal for displaying the guideline. The image signal of the guideline is output to the guide information presentation device 20, and the guideline GL that estimates the traveling route of the host vehicle C is drawn on the display, for example, as illustrated in FIG. At this time, a voice guide via a speaker may be output as appropriate.

  In addition, when a plurality of cameras for photographing the surroundings of the own vehicle are mounted, an overhead view image looking down from a virtual viewpoint above the own vehicle is generated using images from each camera, and the own vehicle is You may make it superimpose the self-vehicle figure which shows a position, the parking target position used as the parking target of the own vehicle, and the guideline of a prediction course locus. Moreover, it is good also as only displaying the arrow (refer FIG. 6) etc. which display which goes to right or left, without drawing the guideline of a predicted course locus.

  Next, the parking assistance control program process executed by the controller 10 will be described with reference to the flowchart of FIG. The parking assistance control program is activated, for example, when the vehicle speed is equal to or lower than a set value and the shift position is set to the reverse position.

  First, images are acquired from the left and right cameras 2a and 2b in the first step S1, and preprocessing such as binarization of the images is performed in step S2. Next, proceeding to step S3, a front wheel is detected from the binarized image, and at step S4, the shape (or contour) of the front wheel in the image is collated with the pattern of the front wheel stored in the system in advance. Process. If there is no matching pattern in this pattern matching, the process returns to step S1 to acquire the image of the next frame, and the same processing is performed. When the patterns match, the process proceeds from step S4 to step S5, and the steering angle is calculated based on the pattern matching result.

  Thereafter, the process proceeds to step S6, and when the traveling path of the host vehicle is estimated based on the steering angle, a guideline for the predicted path trajectory is generated in step S7, and an image of the guideline is displayed on the display of the guide information presentation device 20. In step S8, it is checked whether or not parking is completed from a parking brake signal or the like. When parking is not completed, the process returns to step S1 to detect the steering angle from the front wheel image and estimate the traveling route. This control is finished when the parking is completed repeatedly.

  As described above, in the present embodiment, the steering angle is recognized from the image obtained by photographing the front wheels without using the signal from the steering angle sensor, and the traveling path of the host vehicle is estimated based on the recognized steering angle. Thereby, the driving operation of the driver at the time of parking can be supported while reducing the cost required for the wiring work and the harness for connecting the rudder angle sensor to the parking assist device 1.

  Next, a second embodiment of the present invention will be described.

  In the second mode, the wheels in the image are distinguished from tires and wheels, and the steering angle is recognized from the wheel information. That is, when detecting a wheel reflected in an image, it is possible to recognize the steering angle more accurately by using a wheel whose edge is easier to catch as compared to a tire.

  Specifically, as described in (a) and (b) below, information such as the coordinates and area of the wheel corners in the image is acquired, and the steering angle is recognized based on the acquired information.

(A) Steering angle recognition based on wheel corner coordinate information The captured image is binarized, and the wheel angle is detected from the binarized image to obtain coordinate information. Since the wheel corner position changes depending on the position and orientation of the camera, the wheel corner coordinates are detected based on the coordinates of the characteristic part of the vehicle body, and the steering angle is obtained from the coordinate information at the time of detection.

(B) Recognition of Steering Angle Based on Wheel Area The captured image is binarized to distinguish the tire from the wheel, the number of pixels in the wheel portion surrounded by the tire is counted, and this number of pixels is used as the wheel area. In the controller 10, a look-up table in which the relationship between the wheel area and the steering angle is tabulated in advance is stored, and the look-up table is referred to by the wheel area (number of pixels) acquired from the image. Find the steering angle.

  The look-up table is created by taking into account various parameters of the camera and the positional relationship with the vehicle body, and previously obtaining a steering angle corresponding to the wheel area (number of pixels) in the image by actual measurement or simulation. FIG. 8 shows a characteristic example of such a lookup table, and representative data points are shown discretely. From the characteristics of this lookup table, it can be seen that the steering angle increases with a predetermined relationship as the wheel area (number of pixels) increases.

  Since the look-up table depends on the size of the tire to be mounted, it is desirable to have a look-up table for each allowable tire size.

  Next, the parking assist control program process using the steering angle based on the wheel area will be described with reference to the flowchart of FIG.

  In this program processing, first, in step S11, images are acquired from the left and right cameras 2a and 2b, and preprocessing such as binarization of the images is performed in step S2. Next, the process proceeds to step S13 to check whether or not the front wheel can be detected from the binarized image.

  As a result, when the steering angle is small and the wheel is not reflected in the image, the process returns from step S13 to step S11 to acquire the image of the next frame, and the same processing is performed. If the wheel can be detected from the binarized image, the process proceeds from step S13 to step S14, and the number of pixels in the area detected as the wheel is counted and calculated as the wheel area.

  Next, it progresses to step S15 and the lookup table memorize | stored in the system is collated with the wheel area calculated | required by step S14. If the lookup table collation is unsuccessful, the process returns to the first step S11. If the collation is successful, the process proceeds from step S15 to step S16, and the steering angle is calculated from the collation result.

  Thereafter, the process proceeds to step S17, where the traveling path of the host vehicle is estimated based on the steering angle. In step S18, a guideline for the predicted path locus is generated, and the generated guideline image is displayed on the display of the guide information presentation device 20. Then, in step S19, it is checked whether or not parking is completed from the parking brake signal or the like. If parking is not completed, the process returns to step S11 to detect the steering angle from the wheel image of the front wheel and estimate the traveling route. The process is repeated, and when the parking is completed, this control is finished.

  Also in the second mode, as in the first mode, it is possible to support the driving operation of the driver at the time of parking while reducing the cost required for the wiring work and the harness for connecting the steering angle sensor to the parking assist device 1. However, in the second embodiment, since a wheel with a clear edge is a detection target, the steering angle can be recognized more accurately.

DESCRIPTION OF SYMBOLS 1 Parking assistance apparatus 2 Camera 5 Front wheel 5a Tire 5b Wheel 10 Controller 11 Steering angle recognition part 12 Course estimation part 13 Guide information generation part 20 Guide information presentation apparatus

Claims (9)

In the parking assistance device that supports the driving operation of the driver when parking the vehicle,
An imaging unit that captures an image around the vehicle including at least the wheels;
A steering angle recognition unit that extracts a wheel from an image captured by the imaging unit and recognizes a steering angle of the captured wheel based on a relationship between a wheel and a steering angle on an image that is changed by a steering operation;
A course estimation unit that estimates and calculates a travel path of the vehicle based on the steering angle recognized by the steering angle recognition unit;
A parking assistance device comprising: an information presentation unit that presents assistance information including a travel route estimated by the route estimation unit to a driver.   The parking assist device according to claim 1, wherein the steering angle recognizing unit recognizes the steering angle by estimating the position of the wheel with reference to a specific part of the vehicle body in the image.   The parking assist device according to claim 1, wherein the steering angle recognizing unit detects a front wheel within a recognition frame set in advance in an image and recognizes the steering angle.   The parking assist device according to claim 1, wherein the steering angle recognizing unit recognizes a steering angle from a shape of a wheel in an image that changes in accordance with a steering operation.   The parking assist device according to claim 1, wherein the steering angle recognizing unit distinguishes between a wheel tire and a wheel in the image, and recognizes the steering angle based on the distinguished wheel image information.   The parking assist device according to claim 5, wherein the steering angle recognizing unit recognizes a steering angle from an area of a wheel in an image that changes in accordance with a steering operation.   The parking assist device according to claim 5, wherein the steering angle recognition unit recognizes a steering angle from a coordinate value of a wheel angle in an image that changes in accordance with a steering operation.   The parking assist device according to claim 1, wherein the steering angle recognizing unit recognizes only the steering direction from the shape of the wheel in the image.   The steering angle is recognized from an image captured by a normally operating camera when the imaging unit is configured by at least two cameras and a malfunction occurs in any of the cameras. Parking assistance device.

Images