JP5924053B2 - Parking assistance device - Google Patents

Description

  The present invention relates to a parking assistance device that supports a driver's parking operation.

  Conventionally, in a parking assist device, the target parking lot is imaged by the right side camera and the left side camera, and the surroundings of the vehicle are imaged by the front camera and the rear camera. In some cases, a video in a range where the “” is missing is cut out from the video of the target parking lot, and the video around the vehicle is complemented and displayed (see Patent Document 1).

  In this case, by displaying an image around the vehicle with few image missing portions due to the blind spot of the camera, the white line of the parking frame that divides the parking range can be clearly displayed. For this reason, the operation for parking by the driver can be favorably supported.

JP 2002-29314 A

  In the above-mentioned in-vehicle image processing device, the image of the range that becomes the blind spot of the vehicle body in the image around the vehicle is cut out from the image of the target parking lot, and the image around the vehicle is complemented and displayed to classify the parking range Although the parking frame can be clearly displayed, it is necessary to store the image of the target parking lot in the memory in addition to the image around the vehicle. For this reason, the use capacity of the memory is increased.

  The present invention has been made in view of the above points, and it is an object of the present invention to suppress an increase in memory usage capacity in a parking assistance device that assists a driver's parking operation.

In order to achieve the above object, according to the first aspect of the present invention, when the vehicle travels in the vicinity of the parking frame, the parking frame is extracted from the captured image of the camera (10) for repeatedly capturing the scenery on the parking frame side. A parking assist device that detects and assists the driver's parking operation based on the detected parking frame,
First determination means (S110) for determining whether or not the vehicle has arrived in the vicinity of the parking frame;
The vicinity of the parking space second determination means for determining whether the vehicle has passed the (S190),
Acquisition means (S130) for acquiring position information of the own vehicle every time the camera (10) takes a picture;
When the vehicle is moving forward, since the vehicle near the parking space it is determined that the first determination means has reached, when the vicinity of the parking frame and the vehicle has passed the second determination means determines Until now, the coordinates of the edge point in the photographed image are converted for each photographing so that the coordinates of the edge point in the photographed image are associated with the positional information for each photographing based on the positional information for each photographing of the camera. Voting means (S160, S170, S180) for voting edge points on the coordinated coordinates to a common Hough space and storing the voting history in the memory (20) for each voting;
Third determination means (S200) for determining whether or not the vehicle has started to reverse;
It is characterized by comprising detection means (S210) for detecting a straight line indicating a parking frame from the history of each vote stored in the memory when the third determination means determines that the host vehicle has started to move backward.

  According to the first aspect of the present invention described above, the voting means votes the edge points on the coordinates to a common Hough space every time when the vehicle travels in the vicinity of the parking frame. The voting history is stored in the memory for each vote. Thus, the history of voting for each vote is stored in the memory in order to detect a straight line indicating the parking frame. For this reason, compared with the case where a picked-up image is repeatedly memorize | stored in memory, the use capacity of memory can be reduced. As described above, an increase in the use capacity of the memory in the parking assistance device can be suppressed.

  An edge point is a point that shows a change in luminance in an image taken by a camera.

  The Hough space is a three-dimensional space having first and second parameters representing a straight line and the number of votes of edge points as coordinate axes.

In the invention according to claim 2, when the vehicle travels in the vicinity of the parking frame, the parking frame is detected from the captured image of the camera (10) for repeatedly photographing the scenery on the parking frame side, and this is detected. A parking assistance device for assisting a driver's parking operation based on the parking frame,
Acquisition means (S130) for acquiring the position information of the vehicle every time the camera takes a picture;
Distance determining means (S220) for repeatedly determining whether or not the vehicle has traveled a certain distance or more;
Setting means for setting a Hough space for the imaging region of the camera corresponding to the position of the own vehicle each time the distance determining means determines whether the own vehicle has traveled a certain distance or more when the own vehicle is moving forward (S230),
When the number of determinations determined by the distance determination means that the traveling distance of the vehicle is equal to or greater than a certain distance is N−1, the coordinates of the edge points in the captured image are determined based on the position information for each shooting of the camera. Voting means (S160, voting means) for converting the coordinates of the edge points in the captured image for each photographing so as to be associated with the positional information for each photographing, and voting the edge points converted for each conversion to at least the Nth Hough space S170A, S170B),
Control means (S180, S250) for storing a history of voting in the memory (20) for each vote of the voting means;
Third determination means (S200) for determining whether or not the vehicle has started to reverse;
Detecting means (S210, S260) for detecting a straight line including a parking frame from a history of voting for each Hough space stored in the memory when the third determining means determines that the host vehicle has started to reverse;
Capacity determination means (S240) for determining whether or not the capacity for storing the history of each mapping in the memory is greater than or equal to a predetermined value,
When the capacity determination means determines that the capacity for storing the history for each mapping in the memory is greater than or equal to a predetermined value, the control means determines the Hough space voted at the oldest time among the history for each vote stored in the memory. Instead of this history, the history of the Hough space voted at the latest timing is stored in the memory.

  In the invention described in claim 2, the capacity for storing the history for each mapping in the memory is limited by the control means. For this reason, compared with the case where a picked-up image is repeatedly memorize | stored in memory, the use capacity of memory can be reduced. As described above, similarly to the first aspect of the invention, it is possible to suppress an increase in the use capacity of the memory in the parking assistance device. N is an integer of 1 or more.

  According to a third aspect of the present invention, the detection means integrates the voting history for each Hough space stored in the memory into the voting history of one Hough space, and the integrated one Hough space. The straight line including the parking frame is detected from the history of voting.

  Here, the Nth Hough space is the Nth Hough space set by the setting means. The (N + 1) th Hough space is the H + 1 space set by the setting unit. The (N + 2) th Hough space is a Hough space set to the (N + 2) th by the setting unit.

  According to a fourth aspect of the present invention, when the number of times that the distance determining means determines that the traveling distance of the host vehicle is equal to or greater than a certain distance is N-1 times, the voting means In addition, when the edge point on the coordinates is voted for the Nth Hough space and the N + 1th Hough space, and the number of determinations is N, the voting means The edge point on the coordinates is voted for the (N + 1) th Hough space and the (N + 2) th Hough space.

  According to the fourth aspect of the present invention, the voting means performs the (N + 1) th time every time the camera shoots both when the number of determinations is N-1 and when the number of determinations is N. Vote for the edge point on the coordinates in the Hough space. Therefore, when the number of determinations is N-1 times, the voting means votes only for the Nth Hough space, and when the number of determinations is N times, the voting means votes only for the N + 1th Hough space. The number of votes voted for the (N + 1) th Hough space can be increased. Therefore, in the (N + 1) th Hough space, the first parameter and the second parameter representing a straight line are clearly represented by the number of votes. Thereby, a straight line can be easily detected.

  The invention according to claim 5 is characterized in that two adjacent Hough spaces among the plurality of Hough spaces set by the setting means are set so that the shooting areas of the camera overlap each other. To do.

  According to the fifth aspect of the present invention, it is possible to avoid the occurrence of an imaging region that is excluded from the target of the Hough space between two adjacent Hough spaces.

  In the invention according to claim 6, the voting means (S150, S160, S170) extracts an edge point from the photographed image every time the camera is photographed, and the extracted edge point is detected every time the camera is photographed. The coordinates are converted into edge points on the bird's eye coordinates viewed from above, and the coordinates of the edge points that are coordinate-converted based on the position information for each photographing of the camera are converted into the position information for each photographing. The coordinates are transformed so as to be associated with each other, and the edge points on the bird's-eye coordinates obtained by the coordinate transformation are voted on the Hough space every time the camera is photographed.

  In addition, the code | symbol in the bracket | parenthesis of each means described in this column and the claim shows the correspondence with the specific means as described in embodiment mentioned later.

It is a figure which shows schematic structure of the parking assistance apparatus in 1st Embodiment of this invention. It is a figure for demonstrating the specific example of 1st Embodiment. It is a figure which shows the picked-up image of a CCD camera, and its bird's-eye view image in order to demonstrate the specific example of 1st Embodiment. It is a flowchart which shows the parking frame detection process of 1st Embodiment. It is a figure which shows the parking frame detected by execution of a parking frame detection process. It is a flowchart which shows the parking frame detection process of 2nd Embodiment of this invention. It is a figure which shows the several Hough space used in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, parts that are the same or equivalent to each other are given the same reference numerals in the drawings in order to simplify the description.

(First embodiment)
FIG. 1 shows a schematic configuration of a parking assistance apparatus 1 of the present invention. The parking assist device 1 includes a CCD camera 10, a memory 20, and a microcomputer 30.

  The CCD camera 10 is arranged behind the host vehicle and photographs the scenery behind the host vehicle. The own vehicle is a vehicle on which the parking assistance device 1 is mounted. The memory 20 stores a computer program of the microcomputer 30 and functions as a buffer for storing data generated when the microcomputer 30 executes the computer program.

  The microcomputer 30 uses the Hough transform from the captured image of the CCD camera 10 using the output signals of the distance measuring sensor 40, the steering angle sensor 41, the vehicle speed sensor 42, the parking start switch 43, and the automatic transmission 44. The parking frame detection process for detecting the parking frame is executed.

  The distance measuring sensor 40 is a sensor for measuring a distance between the own vehicle and an obstacle around the vehicle. For example, an ultrasonic sensor is used as the distance measuring sensor 40 of the present embodiment. The ultrasonic sensor measures the propagation time ts required to receive the reflected wave that has been reflected and returned from the obstacle surface after transmitting the ultrasonic signal, and converts this propagation time ts into a distance. Then, a distance L {= (ts / 2) × Vs (sound speed)} between the ultrasonic sensor and the obstacle is measured.

  The steering angle sensor 41 is a sensor that detects the steering angle of the front wheel of the host vehicle. The vehicle speed sensor 42 is a sensor that detects the speed of the host vehicle. The parking start switch 43 is a switch operated by an occupant when instructing the start of the parking frame detection process. The automatic transmission 44 switches the shift range according to the vehicle speed, engine rotation speed, driver's operation, and the like. The automatic transmission 44 outputs an output signal indicating the set shift range.

  Next, as a specific example of the operation of the present embodiment, when the own vehicle 40 in FIG. 2 moves forward in the vicinity of the parking lot 50 from the point A to the point B, the parking frame 51 on the road from the captured image of the CCD camera 10. An example of detecting the will be described. The parking frame 51 is composed of two straight white lines arranged in parallel on the road, and indicates a parking area of the vehicle. FIG. 2 shows an example in which the host vehicle 40 moves in the order of point A → B point → C point → D point and parks in the parking frame 51 between the automobiles 41 and 42.

  3 (a), (c), (e), and (g) are images captured by the CCD camera 10, and FIGS. 3 (b), (d), (f), and (h) are bird's-eye conversions of the captured images. It is a bird's-eye view image. 3A is a captured image of point A, FIG. 3B is a bird's-eye image of the captured image of point A, FIG. 3C is a captured image of point B, and FIG. 3D is a captured image of point B. 3E is a captured image of point C, FIG. 3F is a bird's-eye image of the captured image of point C, FIG. 3G is a captured image of point D, and FIG. It is a bird's-eye view image of a pointed image.

  The microcomputer 30 executes the parking frame detection process according to the flowchart of FIG. The execution of the parking frame detection process is started when the parking start switch 43 is turned on.

  First, in step S100, the operation of the distance measuring sensor 40 is started. Thereby, the distance measuring sensor 40 repeatedly measures the distance L between the obstacle.

  For example, when the host vehicle 40 is located in the vicinity of the automobile 41, the ultrasonic signal from the distance measuring sensor 40 is reflected by the automobile 41 as an obstacle, so that the measurement distance L of the distance measuring sensor 40 is less than a predetermined value. Become.

  Next, whether or not the end point 41a on the parking frame 51 side of the vehicle (specifically, the automobile 41) adjacent to the parking frame 51 is detected based on the measurement distance L of the distance measuring sensor 40 in step S110. Determine.

  For example, while the measurement distance L calculated by the distance measuring sensor 40 based on the reflected wave reflected by the automobile 41 is less than a predetermined value, when the host vehicle 40 reaches the vicinity of the parking frame 51, the distance from the distance measuring sensor 40 is Since the ultrasonic signal is reflected by an obstacle located far from the parking frame 51, the measurement distance L becomes a predetermined value or more.

  Therefore, when the previous measurement distance L and the current measurement distance L of the distance measuring sensor 40 are each less than a predetermined value, the distance L to the automobile 41 is measured to detect the end point 41a on the parking frame 51 side of the automobile 41. If it is not, NO is determined in step S110. For this reason, the determination of step S110 is repeated.

  On the other hand, when the previous measurement distance L of the distance measuring sensor 40 is less than the predetermined value and the current measurement distance L of the distance measuring sensor 40 is greater than or equal to the predetermined value, the previous measurement distance L is set as the vehicle 41. It is determined that the distance between the parking frame 51 side end point 41a and the distance measuring sensor 40 has been measured. That is, it is determined that the vehicle 40 has advanced from the point A and has reached the vicinity of the parking frame 51 in step S110.

  Next, in step S120, the CCD camera 10 is controlled to photograph the rear scenery. At this time, as shown in FIG. 4A, the CCD camera 10 captures a rear image including the scenery directly behind the vehicle 40, the right rear scenery, and the left rear scenery.

  Next, in step S130, based on the output signal of the steering angle sensor 41 and the output signal of the vehicle speed sensor 42, the position information of the own vehicle 40 (that is, the photographing position of the CCD camera 10) is calculated.

  Next, in step S140, edge points are extracted from the captured image of the CCD camera 10. The edge point in the present embodiment is a point indicating a change in luminance in the captured image of the CCD camera 10. Thus, top view conversion is performed on the plurality of edge points extracted from the photographed image (step S150). That is, coordinate conversion is performed on a plurality of edge points to a plurality of edge points on the bird's-eye view coordinates viewed from above the host vehicle 40. That is, for each captured image, a plurality of edge points are coordinate-converted to a plurality of edge points on the bird's-eye view coordinates.

  Here, as will be described later, a plurality of captured images are used for detecting the parking frame. The shooting positions of the plurality of shot images are different for each shot image. In addition to this, the bird's-eye coordinates for each photographing are set independently. For this reason, the positional relationship of bird's-eye coordinates for each captured image is not fixed.

  Therefore, in the next step S160, the bird's-eye coordinates are taken every time the position information of the own vehicle 41 (that is, the shooting position of the CCD camera 10) is used so that the bird's-eye coordinates of the step S150 are associated with the shooting position of the CCD camera 10. Convert coordinates. That is, a plurality of edge points on the bird's-eye coordinates in step S150 are mapped onto the common bird's-eye coordinates for each captured image. Thereby, the positional relationship of the bird's-eye coordinates for each captured image can be set using the positional information for each captured image.

  A plurality of edge points on the bird's eye coordinate coordinate-transformed in step S160 are voted for a common Hough space (parameter space) (step S170). That is, a plurality of edge points on the bird's-eye coordinates after the coordinate conversion are mapped to a common Hough space. The Hough space is a three-dimensional coordinate for representing a straight line. Specifically, it is a space where the X axis is ρ, the Y axis is θ, and the Z axis is the number of votes of edge points.

  Here, ρ and θ are parameters used to express the straight line A passing through the point (xi, yi) on the two-dimensional XY coordinates as ρ = xi · cos θ + yi · sin θ in the polar coordinate expression. ρ is a first parameter indicating the length of a perpendicular line drawn from the origin to the straight line A. θ is a second parameter indicating the angle formed by the X axis and the perpendicular. The Hough space of the present embodiment is targeted for the shooting area including the parking frame 51. The shooting area is an area shot by the CCD camera 10. In this way, a plurality of edge points in the captured image are voted for a common Hough space. As a result, a feature amount for detecting a straight line from the captured image is calculated. Next, the voting history voted for the common Hough space as described above is stored in the memory 20 (step S180).

  Next, in step S190, it is determined whether or not the vehicle 40 has passed near the parking frame 51 (that is, the parking space).

  For example, when the measurement distance L of the distance measuring sensor 40 is greater than or equal to a predetermined value, it is determined that the host vehicle 40 is moving forward in the vicinity of the parking frame 51, and NO is determined in step S190.

  On the other hand, when the measurement distance L of the distance measuring sensor 40 is less than a predetermined value, it is determined that the distance between the end point 42a on the parking frame 51 side of the automobile 42 and the distance measuring sensor 40 is measured as the measurement distance L. That is, it is determined YES in step S190 that the host vehicle 40 has passed the parking frame 51 and has reached the vicinity of the vehicle 42.

  For example, when it is determined NO in step S190, the process returns to step S120. For this reason, until passing the vicinity of the parking frame 51 of the own vehicle 40, the camera photographing process of step S120, the position calculation process of step S130, the edge extraction process of step S140, the top view conversion process of step S150, the edge coordinates of step S160 The conversion process, the Hough space voting process in step S170, and the memory saving process in step S180 are repeated.

  As described above, a plurality of edge points are extracted from the captured image every time the CCD camera 10 captures the image (step S140), top view conversion is performed on the extracted plurality of edge points (step S150), and the top view is converted. The bird's-eye coordinates are converted for each image so that the bird's-eye coordinates of the plurality of edge points correspond to the shooting positions of the CCD camera 10 (step S160), and the plurality of edge points on the bird's-eye coordinates after the coordinate conversion are shared. Vote in the Hough space (step S170), and save this voting history in the memory 20 for each vote (step S180).

  Thereafter, when the measurement distance L of the distance measuring sensor 40 becomes less than a predetermined value, it is determined that the distance between the end point 42a on the parking frame 51 side of the automobile 42 and the distance measuring sensor 40 is measured as the measurement distance L. That is, it is determined YES in step S190 because the own vehicle 40 has passed the parking frame 51 and has reached the vicinity of the automobile 42. Accordingly, in the next step S200, whether or not the transmission 44 is set to the reverse range is determined based on the output signal of the transmission 44. That is, it is determined whether or not the host vehicle 40 has reached point B and is starting to move backward.

  At this time, when the transmission 44 is set to a range other than the reverse range, NO is determined in step S200, and the determination in step S200 is repeated.

  Thereafter, when the transmission 44 is set to the reverse range, YES is determined in step S200, and a straight line indicating the parking frame 51 is determined based on the voting history for each vote stored in the memory 20 in the next step S210. Detection is performed as shown in FIG. FIG. 5 shows the parking frame 51 detected in step S210 along with the movement locus on the XY coordinates. A dotted line in FIG. 5 indicates a trajectory in which the host vehicle 40 moves forward from the point A to the point B, and a thick solid line indicates a trajectory in which the host vehicle 40 moves backward from the point B to the point D through the point C. Since the detection of the straight line is well known, its description is omitted.

  The parking frame 51 thus detected assists the parking operation by being used as a target object for automatic parking control. The automatic parking control is control that automatically controls the steering angle to park the vehicle 40 in the parking frame 51 as a target object. In addition to this, as shown in FIGS. 3C to 3H, the parking frame 51 is superimposed on the photographed image of the CCD camera 10 and displayed to the driver by the display in the passenger compartment, thereby driving the vehicle. Support the parking operation.

  For example, FIG. 3C and FIG. 3D are display images on the display when the host vehicle 40 starts moving backward at point B. FIG. 3E and FIG. 3F are display images on the display when the vehicle 40 passes through the point C. FIGS. 3G and 3H are display images on the display when the host vehicle 40 arrives at point D. FIG.

  In addition, in the above-mentioned step S210, when the straight line indicating the parking frame 51 cannot be detected, the straight line indicating the parking frame 51 is estimated using various vehicle signals.

  According to the present embodiment described above, the edge point in the captured image is determined every time the CCD camera 10 captures until the vehicle 40 passes through the parking frame 51 after the vehicle 40 reaches the vicinity of the parking frame 51. The coordinates are converted into edge points on the bird's-eye coordinates, and the coordinates of the edge points that have been coordinate-converted are correlated with the position information for each photographing based on the positional information for each photographing of the CCD camera 10 for each coordinate transformation. In addition, the edge points on the bird's-eye coordinates that have been subjected to the coordinate conversion are voted for each photographing of the CCD camera 10 in a common Hough space. The voting history is stored in the memory 20 for each vote. Thus, the voting history for each vote is stored in the memory 20 in order to detect a straight line indicating the parking frame 51. For this reason, compared with the case where a picked-up image is repeatedly memorize | stored in the memory 20, the use capacity of the memory 20 can be reduced. As described above, an increase in the used capacity of the memory 20 in the parking assistance device 1 can be suppressed.

  In the present embodiment, when the parking frame 51 is detected, a plurality of edge points in the captured image are bird-eye-transformed, and a vote is given to a common Hough space using the plurality of edge points that have been bird-eye-transformed.

  Here, when the distance between the parking frame 51 and the automobile 41 (or the automobile 42) adjacent to the parking frame 51 is very small, a plurality of edge points in the captured image are converted to a plurality of points without performing bird's-eye conversion. If the edge point is voted on the Hough space, the parking frame 51 becomes a shadow of the automobile 41 (or the automobile 42), and the edge point of the parking frame 51 that becomes the shadow cannot be extracted. For this reason, it becomes impossible to vote a plurality of edge points which show parking frame 51 to Hough space, and parking frame 51 may not be detected favorably.

  On the other hand, in the present embodiment, as described above, a plurality of edge points in the photographed image are subjected to bird's-eye conversion, and the plurality of edge points subjected to bird's-eye conversion are voted on a common Hough space. For this reason, it becomes difficult for the edge point of the parking frame 51 to become a shadow of the automobile 41 (or the automobile 42). Thereby, even when the distance between the parking frame 51 and the automobile 41 (or the automobile 42) adjacent thereto is very small, the edge point of the parking frame 51 can vote for the Hough space. It can be detected well.

  In this embodiment, in step S140, edge points are extracted from the captured image of the CCD camera 10, and in step S150, top view conversion is performed on the extracted edge points. Therefore, it is possible to reduce the amount of image processing compared to a case where the top view conversion is performed on the captured image of the CCD camera 10 and an edge point is extracted from the top view converted captured image.

(Second Embodiment)
In the first embodiment described above, the use of the memory 20 is reduced by voting to the Hough space only when the vehicle 40 travels in the vicinity of the parking frame 51 using the measurement distance L of the distance measuring sensor 40. The example of reducing the used capacity of the memory 20 when the measurement distance L of the distance measuring sensor 40 is not used will be described.

  FIG. 6 shows a flowchart of the parking frame detection process of the present embodiment. The microcomputer 30 executes the parking frame detection process according to the flowchart of FIG. 6 instead of the flowchart of FIG.

  First, prior to the detailed description of the parking frame detection process of the present embodiment, the Hough space used in the parking frame detection process of the present embodiment will be described.

  In the parking frame detection process of the present embodiment, a plurality of hough spaces are set instead of a common hough space.

  Specifically, every time the host vehicle 40 travels a predetermined distance, a Hough space is set for the imaging region corresponding to the position information of the host vehicle 40. The shooting area is an area shot by the CCD camera 10. As a result, the imaging areas of the plurality of Hough spaces are arranged at predetermined distances in the traveling direction of the host vehicle 40. The imaging regions of the plurality of Hough spaces are each set in a rectangular shape (strip shape) extending from the position of the host vehicle 40 in a direction orthogonal to the traveling direction of the host vehicle 40.

  Here, the predetermined distance is a reference value of the travel distance for setting the Hough space, and is, for example, 1 meter. As the predetermined distance becomes shorter, the number of Hough spaces that are sequentially set increases, so it is necessary to increase the capacity of the memory 20 that stores the voting history for the Hough space. For this reason, the predetermined distance is set according to the capacity of the memory 20 to be used.

  Further, in the present embodiment, the imaging regions of two adjacent Hough spaces among the plurality of Hough spaces are set to overlap each other.

  FIG. 7 shows an example in which the Hough space N, the Hough space N + 1, the Hough space N + 2, and the Hough space N + 3 are sequentially set when the host vehicle 40 moves forward from the A point toward the B point. In the Hough space N and the Hough space N + 1, the shooting areas overlap each other. In the Hough space N + 1 and the Hough space N + 2, the shooting areas overlap each other. In the Hough space N + 2 and the Hough space N + 3, the shooting areas overlap each other.

  Here, N is an integer of 1 or more. The Hough space N indicates the Nth set Hough space, the Hough space N + 1 indicates the N + 1th set Hough space, the Hough space N + 2 indicates the N + 2 set Hough space, and the Hough space. N + 3 indicates a Hough space set as the (N + 3) th.

  Next, as a specific example of the parking frame detection process of this embodiment, an example in which the host vehicle 40 moves forward from the point A to the point B as shown in FIG. 7 will be described. 6, the same reference numerals as those in FIG. 4 denote the same steps, and the description thereof will be simplified. The execution of the parking frame detection process is started when the parking start switch 43 is turned on.

  First, in step S120, the rear scene is photographed by the CCD camera 10. Next, in step S130, based on the output signal of the steering angle sensor 41 and the output signal of the vehicle speed sensor 42, position information of the host vehicle 40 is calculated. Next, in step S140, edge points are extracted from the captured image of the CCD camera 10. Next, top view conversion is performed on the plurality of edge points extracted from the captured image in this way (step S150).

  Thereafter, in step S160, the bird's-eye coordinates are coordinate-converted for each photographing so that the bird's-eye coordinates in step S150 are associated with the photographing position of the CCD camera 10 using the position information of the own vehicle 41. In this way, the plurality of edge points on the bird's-eye coordinates transformed in step S160 are voted to the Hough space 1 and the Hough space 2, respectively (Steps S170A and S170B). The Hough space 1 and the Hough space 2 are preset as Hough spaces. The Hough space 1 is the first set Hough space. The Hough space 2 is a Hough space set second. Thus, the voting history voted for the Hough space 1 and the Hough space 2 is stored in the memory 20 (step S180).

  Next, in step S220, after the parking start switch 43 is turned on, it is determined whether or not the host vehicle 40 has traveled more than a predetermined distance (for example, 1 meter).

  Specifically, the position information of the own vehicle 40 is obtained based on the output signal of the steering angle sensor 41 and the output signal of the vehicle speed sensor 42, and the own vehicle 40 obtains the travel distance from this position information. When the travel distance is less than the predetermined distance, it is determined as NO and the process returns to step S120.

  Thereafter, until the traveling distance becomes equal to or greater than the predetermined distance, the CCD camera 10 photographing process in step S120, the position information calculation process in step S130, the edge point extraction process in step S140, the top view conversion process in step S150, and the step S160 coordinate conversion process The voting process in steps S170A and S170B, the voting history saving process in step S180, and the NO determination process in step S220 are repeated.

  Therefore, a plurality of edge points are extracted from the captured image every time the CCD camera 10 captures (step S140), top view conversion is performed on the extracted plurality of edge points (step S150), and the top view is converted. Then, the bird's-eye view coordinates of the plurality of edge points are coordinate-converted for each photographing so as to correspond to the photographing positions of the CCD camera 10 (step S160), and the plurality of edge points on the coordinate-converted bird's-eye coordinates are photographed. Next, the voting space 1 and the Hough space 2 are voted (steps S170A and S170B), and the voting history is stored in the memory 20 for each voting (S180).

  Thereafter, when the travel distance is equal to or greater than the predetermined distance, YES is determined in step S220. Accordingly, the process proceeds to step S230, and the number of the Hough space to be voted is incremented in steps S170A and S170B. That is, the Hough space 3 corresponding to the position of the own vehicle 40 is newly set. Therefore, the Hough space to be voted in step S170A is referred to as Hough space 2, and the Hough space to be voted in Step S170B is referred to as Hough space 3. At this time, the position information of the vehicle 40 corresponding to the Hough space 3 is stored in the memory 20.

  Thereafter, in step S240, it is determined whether or not the storage capacity of the memory 20 in which the voting history is stored is greater than or equal to the specified capacity. At this time, when the storage capacity of the memory 20 in which the voting history is stored is less than the specified capacity, NO is determined in step S240, and the process proceeds to the next step S200. At this time, whether or not the transmission 44 is set to the reverse range is determined based on the output signal of the transmission 44. At this time, when the transmission 44 is set to a range other than the reverse range, NO is determined in step S200, and the process returns to step S120.

  After that, each process of steps S120, S130, S140, S150, S160, S170A, S170B, and S180 is executed, and then the process proceeds to step S220. In this step S220, it is determined whether or not the vehicle has traveled more than a predetermined distance (for example, 1 meter) since it was previously determined as YES in step S220. If the travel distance traveled by the vehicle 40 is less than the predetermined distance since the previous determination in step S220 as YES, the determination in step S220 is NO and the process returns to step S120.

  After that, as long as the travel distance traveled by the vehicle 40 is less than the predetermined distance from the previous determination in step S220 as YES, steps S120, S130, S140, S150, until the next determination in step S220 is YES. Each process of S160, S170A, S170B, and S180 and the NO determination process of step S220 are repeated.

  For this reason, a plurality of edge points on the bird's eye coordinate coordinate-transformed in step S160 are voted for the Hough space 2 and Hough space 3 every time the CCD camera 10 is photographed (Steps S170A and S170B). Thus, the voting history voted for the Hough space 2 and the Hough space 3 is stored in the memory 20 every time the CCD camera 10 is photographed (step S180).

  Thereafter, if the travel distance traveled by the vehicle 40 after a determination of YES in step S220 last time becomes equal to or greater than a predetermined distance, it is determined in step S220 to be YES.

  In step S230, the number of the Hough space to be voted is incremented in steps S170A and S170B. That is, the Hough space 4 corresponding to the position of the own vehicle 40 is newly set. For this reason, the Hough space to be voted in step S170A is referred to as Hough space 3, and the Hough space to be voted in Step S170B is referred to as Hough space 4. At this time, the position information of the vehicle 40 corresponding to the Hough space 4 is stored in the memory 20.

  Thus, when the number of times YES is determined in step S220 (hereinafter simply referred to as the number of times of determination) is voted for the Hough space 3 and Hough space 4 for each photographing by the CCD camera 10, this vote Every time, the voting history is stored in the memory 20 (step S180).

  Thereafter, in step S240, it is determined whether or not the storage capacity of the memory 20 in which the voting history is stored is greater than or equal to the specified capacity. At this time, when the storage capacity of the memory 20 in which the voting history is stored is less than the prescribed capacity, NO is determined in step S240, and the process proceeds to the next step S200. At this time, when the transmission 44 is set to a range other than the reverse range, it is determined as NO, and the process returns to step S120.

  For this reason, the processes in steps S120, S130, S140, S150, S160, S170A, S170B, and S180 and the NO determination process in step S220 are repeated until it is determined as YES in step S220. For this reason, a plurality of edge points on the bird's eye coordinate coordinate-converted in step S160 are voted for the Hough space 3 and the Hough space 4 every time the CCD camera 10 is photographed (Steps S170A and S170B). Thus, the voting history voted for the Hough space 2 and the Hough space 3 is stored in the memory 20 every time the CCD camera 10 is photographed (step S180).

  Thereafter, when YES is determined in step S220 for the third time, in the next step S230, the Hough space to be voted is determined to be Hough space 4 in Step S170A, and the Hough space to be voted is determined to be Hough space 5 in Step S170B. To do. At this time, the position information of the vehicle 40 corresponding to the Hough space 5 is stored in the memory 20.

  Thereafter, in step S240, it is determined whether or not the storage capacity of the memory 20 in which the voting history is stored is greater than or equal to the specified capacity. At this time, if the storage capacity of the memory 20 in which the voting history is stored is equal to or greater than the specified capacity, YES is determined in step S240, and the process proceeds to step S250. Accordingly, the history of the Hough space voted at the oldest timing among the history of the plurality of Hough spaces stored in the memory 20 is initialized. For example, when the histories of the Hough space 1, the Hough space 2, the Hough space 3, and the Hough space 4 are stored in the memory 20, the history of the Hough space 1 voted at the oldest timing is deleted. .

  In the next step S200, when the transmission 44 is set to a range other than the reverse range, NO is determined and the process returns to step S120. After that, through the processes of steps S120, S130, S140, S150, and S160, a plurality of edge points on the bird's eye coordinate coordinate-converted in step S160 are voted for the Hough space 4 and the Hough space 5 (Step S170A, S170B). Thus, the voting history voted for the Hough space 4 and the Hough space 5 is stored in the memory 20 every time the CCD camera 10 captures the image (Step S180). At this time, the voting history of the Hough space 5 is stored in the memory 20 instead of the voting history of the Hough space 1. Then, a NO determination process (or YES determination process) in step S220, a NO determination process in step S230, a step S240 (or YES determination process in step S240 and step S250), and a NO determination in step S200 are performed.

  Thereafter, as long as the transmission 44 is set to a range other than the reverse range, the NO determination process (or YES determination process) of steps S120, S130, S140, S150, S160, S170A, S170B, S180, and step S220, step S230, NO determination processing in step S240 (or YES determination processing in step S240 and step S250), and NO determination in step S200 are repeated.

  For example, when the number of determinations is N−1, a plurality of edge points on the bird's-eye coordinate coordinate-converted in step S160 are voted for the Hough space N and the Hough space N + 1 every time the CCD camera 10 captures ( Steps S170A and 170B).

  On the other hand, when the number of determinations is N, the plurality of edge points on the bird's eye coordinate coordinate-converted in step S160 are voted for the Hough space N + 1 and the Hough space N + 2 every time the CCD camera 10 captures the image (Step S170A). 170B). The voting history of the Hough space voted in this way is stored in the memory 20 (step S180).

  For example, when the voting history of each of the Hough space N−1, Hough space N, Hough space N + 1, and Hough space N + 2 is stored in the memory 20, the process proceeds to Step S230 after determining YES in Step S220. In step S170A, the Hough space to be voted is set as Hough space N + 2, and in Step S170B, the Hough space to be voted is set as Hough space N + 3. At this time, the position information of the host vehicle 40 corresponding to the Hough space N + 3 is stored in the memory 20. That is, every time the Hough space is set, the position information of the vehicle 40 corresponding to the set Hough space is stored in the memory 20.

  Thereafter, when the process proceeds to step S240, the voting history is initialized in the memory 20 as follows. That is, if the memory 20 stores the voting history in the memory 20 and the determination is YES in step S240, the memory 20 stores the Hough space N−1, the Hough space N, the Hough space N + 1, and the Hough space. The history of the Hough space N-1 voted at the oldest timing among N + 2 is deleted (step S200).

  After that, when it is determined as NO in step S200, a plurality of edge points on the bird's eye coordinate converted in step S160 are processed through steps S120, S130, S140, S150, and S160, and the Hough space N + 2 and the Hough space Vote for N + 3 (steps S170A, S170B). Thus, the voting history voted for the Hough space N + 2 and the Hough space N + 3 is stored in the memory 20 every time the CCD camera 10 is photographed (step S180). At this time, the voting history of the Hough space N + 3 is stored in the memory 20 instead of the voting history of the Hough space N-1. Then, the NO determination process (or YES determination process) in step S220, the NO determination process in step S230 and step S240 (or the YES determination process in step S240 and step S250) are performed.

  Thereafter, when the transmission 44 is set to the reverse range, YES is determined in the step S200, and the process proceeds to the next step S260. For example, when the voting history of the Hough space N, Hough space N + 1, Hough space N + 2, and Hough space N + 3 is stored in the memory 20, the Hough space N, Hough space N + 1, and Hough space N + 2 stored in the memory 20, respectively. And the voting history of Hough space N, Hough space N + 1, Hough space N + 2, and Hough space N + 3 is mapped to a common Hough space using position information of own vehicle 40 corresponding to Hough space N + 3.

  Here, the shooting area of the common Hough space is an area that covers all of the shooting areas of Hough space N, Hough space N + 1, Hough space N + 2, and Hough space N + 3.

  By mapping to the common Hough space, the Hough space N, Hough space N + 1, Hough space N + 2, and Hough space N + 3 are integrated. Thereafter, in step S210, a straight line indicating the parking frame 51 is detected from the voting history mapped to the common hough space.

  According to this embodiment described above, instead of the voting history of the Hough space voted at the oldest timing among the voting histories of the plurality of Hough spaces stored in the memory 20, the Hough space voted at the latest timing is used. The voting history is stored. This limits the storage capacity used when storing the voting history of a plurality of Hough spaces in the memory 20. When the shift range of the automatic transmission is set to the reverse range, the voting histories of the plurality of Hough spaces stored in the memory 20 are mapped to a common Hough space, and the parking frame 51 is determined from the mapped voting history. A straight line indicating is detected. For this reason, compared with the case where a picked-up image is repeatedly memorize | stored in memory, the use capacity of memory can be reduced. By the above, like the said 1st Embodiment, the increase in the usage capacity of the memory 20 in the parking assistance apparatus 1 can be suppressed.

  In this embodiment, when the number of determinations is N−1 times and when the number of determinations is N times, the N + 1-th Hough space is displayed on the bird's-eye coordinate system every time the CCD camera 10 captures images. Vote for edge points. For this reason, when the number of determinations is N−1, only the Hough space N is voted, and when the number of determinations is N, the voting means votes only for the Hough space N + 1. The number of votes cast in the Hough space can be increased.

  Here, in the Hough space, ρ and θ at points with a large number of votes represent straight lines on the captured image. Therefore, as described above, by increasing the number of votes voted for the (N + 1) th Hough space, ρ and θ representing a straight line are clearly represented by the number of votes in the (N + 1) th Hough space. Thereby, a straight line can be easily detected.

  In the present embodiment, two adjacent Hough spaces among a plurality of Hough spaces arranged in the traveling direction of the host vehicle 40 overlap each other. For this reason, it can avoid that the area | region which remove | deviates from the object of several Hough space among the imaging | photography area | regions of CCD camera 10 arises.

(Other embodiments)
In the first and second embodiments described above, the first parameter for representing a straight line is ρ, the second parameter for representing a straight line is θ, the X axis is ρ, the Y axis is θ, and the Z axis is an edge point. However, instead of this, the first parameter for representing a straight line is defined as a slope a, the second parameter for representing a straight line is defined as an intercept b, and the X axis May be a Hough space in which the slope is a, the Y axis is the intercept b, and the Z axis is the number of votes of edge points. Here, the inclination a and the intercept b are parameters used to express the straight line A passing through the point (xi, yi) on the two-dimensional XY coordinates as yi = a × xi + b.

  In the first and second embodiments described above, the example in which the position information of the host vehicle 40 is calculated based on the output signal of the steering angle sensor 41 and the output signal of the vehicle speed sensor 42 has been described. The position information of the host vehicle 40 may be calculated using

  In the first and second embodiments described above, an example has been described in which the edge point in the photographed image is bird's-eye converted and the straight line indicating the parking frame 51 is detected using the edge point on the bird's-eye coordinate that has been bird's-eye converted. Alternatively, a straight line indicating the parking frame 51 may be detected using the edge points in the captured image without performing bird's-eye conversion on the edge points in the captured image.

  For example, in the first embodiment, a plurality of edge points are extracted from the captured image every time the CCD camera 10 captures images (step S140), and the coordinates of the extracted edge points are associated with the capturing positions of the CCD camera 10. The coordinates are converted for each shooting (step S160), a plurality of edge points on the coordinate-converted coordinates are voted for a common Hough space for each shooting, and the voting history is stored in the memory 20 for each vote. .

  For example, in the second embodiment, a plurality of edge points are extracted from the captured image every time the CCD camera 10 captures images (Step S140), and the coordinates of the extracted plurality of edge points are associated with the capturing positions of the CCD camera 10. The coordinates are converted for each shooting (step S160), and a plurality of edge points on the coordinate-converted coordinates are voted for the Hough space N and the Hough space N + 1 for each shooting, and the voting history is recorded for each vote. Save in the memory 20.

  In the second embodiment described above, when the number of determinations is N−1, a plurality of edge points are voted for the Hough space N and the Hough space N + 1 every time the CCD camera 10 captures, and the number of determinations is N times. Although an example has been described in which a plurality of edge points are voted for the Hough space N + 1 and the Hough space N + 2 every time the CCD camera 10 captures images, the following may be used instead.

  That is, when the number of determinations is N−1, a plurality of edge points are voted only on the Hough space N for each photographing of the CCD camera 10, and when the number of determinations is N, each photographing of the CCD camera 10 is performed. In addition, a plurality of edge points may be voted only on the Hough space N + 1.

  In the first and second embodiments described above, the example in which the CCD camera 10 disposed behind the host vehicle is used to photograph the scenery on the parking frame side as the surrounding scenery has been described. If it is possible to photograph the frame-side scenery, the CCD camera 10 arranged at a location other than the back of the host vehicle may be used.

  In the first and second embodiments described above, an example in which edge points are extracted from the captured image of the CCD camera 10 and top view conversion is performed on the extracted plurality of edge points has been described. Instead of this, top view conversion may be performed on the captured image of the CCD camera 10, and edge points may be extracted from the top view converted captured image.

1 Parking Assist Device 10 CCD Camera 20 Memory 30 Microcomputer

Claims (6)

When the vehicle travels in the vicinity of the parking frame, the parking frame is detected from the captured image of the camera (10) for repeatedly capturing the scenery on the parking frame side, and driving is performed based on the detected parking frame. A parking assistance device for assisting a person's parking operation,
First determination means (S110) for determining whether or not the vehicle has reached the vicinity of the parking frame;
Wherein said near the parking space second determination means for determining whether the vehicle has passed (S190),
Acquisition means (S130) for acquiring the position information of the own vehicle every time the camera (10) takes an image;
When the host vehicle is moving forward, the first determining means determines that the host vehicle has reached the vicinity of the parking frame. 2 Until the determination means determines, the coordinates of the edge points in the captured image are associated with the position information for each of the captured images based on the positional information for each captured image of the camera. Voting means for converting coordinates for each photographing, voting edge points on the converted coordinates for each conversion to a common Hough space, and storing the voting history in the memory (20) for each vote (S160, S170, S180),
Third determination means (S200) for determining whether or not the vehicle has started to reverse;
Detecting means (S210) for detecting the straight line indicating the parking frame from the history for each vote stored in the memory when the third determining means determines that the host vehicle has started to move backward; An in-vehicle image processing apparatus characterized by the above. When the vehicle travels in the vicinity of the parking frame, the parking frame is detected from the captured image of the camera (10) for repeatedly capturing the scenery on the parking frame side, and driving is performed based on the detected parking frame. A parking assistance device for assisting a person's parking operation,
Acquisition means (S130) for acquiring position information of the own vehicle every time the camera takes a picture;
Distance determining means (S220) for repeatedly determining whether or not the vehicle has traveled a certain distance or more;
When the vehicle is moving forward, each time the distance determination unit determines whether the vehicle has traveled more than a certain distance, a Hough space that targets the shooting area of the camera corresponding to the position of the vehicle Setting means (S230) for setting
When the number of determinations determined by the distance determination means that the traveling distance of the host vehicle is equal to or greater than a certain distance is N-1 times, the distance in the captured image is determined based on the position information for each image captured by the camera. The coordinates of the edge point in the photographed image are converted for each photographing so that the coordinates of the edge point are associated with the position information for each photographing, and the converted edge point is converted into at least the Nth Hough for each conversion. Voting means (S160, S170A, S170B) for voting against space;
Control means (S180, S250) for storing the voting history in the memory (20) for each vote of the voting means;
Third determination means (S200) for determining whether or not the vehicle has started to reverse;
Detection means (S210, S260) that detects the straight line including the parking frame from the voting history for each of the Hough spaces stored in the memory when the third determination means determines that the host vehicle has started to move backward. )When,
Capacity determining means (S240) for determining whether or not a capacity for storing a history for each mapping in the memory is a predetermined value or more,
When the capacity determination unit determines that the capacity for storing the history for each mapping in the memory is equal to or greater than a predetermined value, the control unit determines the oldest timing among the history for each vote stored in the memory. An in-vehicle image processing apparatus, wherein a history of the Hough space voted at the latest timing is stored in the memory instead of the history of the Hough space voted in (1).   The detection means integrates the voting history for each Hough space stored in the memory into a voting history for one Hough space, and determines the parking frame from the integrated voting history for one Hough space. The in-vehicle image processing apparatus according to claim 2, wherein the straight line is detected. When the number of times of determination that the distance determination means determines that the traveling distance of the host vehicle is equal to or greater than a certain distance is N-1 times, the voting means determines an edge point on the coordinates every time the camera takes a picture. Vote for the Nth Hough space and the N + 1th Hough space,
When the number of times of determination is N, the voting means votes the edge points on the coordinates for the N + 1th Hough space and the N + 2th Hough space every time the camera shoots. The in-vehicle image processing apparatus according to claim 2 or 3, wherein   5. The adjacent two Hough spaces among the plurality of Hough spaces set by the setting means are set so that the shooting areas of the camera overlap each other. The vehicle-mounted image processing apparatus as described in one.   The voting means (S150, S160, S170) extracts an edge point from the photographed image every time the camera is photographed, and an edge on the bird's eye view of the extracted edge point viewed from the upper side of the vehicle every time the camera is photographed. Coordinate-transformed into points, and coordinate-transformed so that the coordinates of the edge points that have been coordinate-transformed are associated with the position information for each photographing based on the positional information for each photographing with the camera, and 6. The in-vehicle image processing apparatus according to claim 1, wherein an edge point on the bird's eye coordinates after coordinate conversion is voted in the Hough space every time the camera is photographed.

Images