JP2016016681A - Parking frame recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To surely detect a parking possible space existing in the advancing direction of its own vehicle from a very long distance as much as possible.SOLUTION: An image of including a road surface of the lateral side of a vehicle 10 is imaged by a side imaging part 12 installed in the vehicle 10 (its own vehicle), and a side parking frame line detection part 23 detects a parking frame line wi on the side of the vehicle 10, and a parking frame line characteristic calculation part 24 calculates a characteristic of the parking frame line wi. An advancing direction parking frame line detection parameter setting part 26 sets a parameter for detecting the parking frame line wi in the advancing direction of the vehicle 10 on the basis of the characteristic of the parking frame line wi calculated in this way, and an advancing direction parking possible area detection part 28 detects a parking possible area of the vehicle 10 from among the image of including the road surface in the advancing direction of the vehicle 10 imaged by an advancing direction imaging part 15.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a parking frame recognition device that detects a parking space in the traveling direction of a host vehicle.

  Recently, when a vehicle is parked in a parking lot, a system that detects a parking space by using a camera or a distance measuring sensor installed in the vehicle and performs parking control (for example, automatic parking) with the detected parking space as a target. Proposed.

  As a specific system, one that executes parking control by setting a parking frame detected with the highest accuracy among parking frames captured by a camera as a parking position (for example, Patent Document 1), and distance measurement A sensor that detects a space between parked vehicles using a sensor (for example, Patent Document 2) has been proposed.

JP 2012-136206 A JP 2013-52754 A

  However, in the parking control system described in Patent Document 1, since the parking frame recognized most reliably from the image is determined as the parking destination, the travel route from the current position to the parking destination is not taken into consideration. . Therefore, there are events that make smooth parking support difficult, such as an increase in the distance traveled and the number of turnovers until parking is completed, and sudden deceleration or sudden steering is required because the parking destination point is too close to the vehicle. There was a problem that there was a risk of doing.

  Moreover, in the parking assistance apparatus described in Patent Document 2, the parking type is determined and the parking area is detected based on the distance between the parked vehicles measured by the distance measuring sensor. Even if it is detected that there is a space three cars ahead, there is no guarantee that the space can be parked, so it is difficult to detect a space that can be parked based only on the distance to the parked vehicle. There was a problem.

  In such a parking assistance device, after detecting a parking area, information on a parking area is provided to the driver, or a parking route for performing automatic parking by controlling the vehicle is formulated. Therefore, it is desirable to detect a parking possible region from as far away as possible so that parking assistance can be performed at an early stage. However, since the far parking frame line is thinner than the neighboring parking frame line and the interval between the adjacent parking frame lines is narrow, it is difficult to stably detect the image from the image. Therefore, for example, simply combining the methods described in Patent Document 1 and Patent Document 2 does not guarantee that a distant parking frame line can be reliably detected.

  This invention is made | formed in view of the said subject, and it aims at detecting the space which can be parked in the advancing direction of the own vehicle from as far as possible.

  The parking frame recognition apparatus according to the present invention reliably detects a space (parking frame) where the host vehicle can be parked from as far as possible from an image captured by an imaging unit attached to the host vehicle.

  That is, the parking frame recognizing device according to the present invention is attached to the host vehicle, and is provided with a side imaging unit that captures an image including a road surface on the left and right side of the host vehicle, and the host vehicle is attached to A traveling direction imaging unit that captures an image including a road surface in a direction, a side parking frame line detection unit that detects a parking frame line on the side of the host vehicle from the images captured by the side imaging unit, A parking frame line feature calculating unit that calculates the characteristics of the parking frame line on the side of the vehicle, and a parking frame line in the traveling direction of the host vehicle from the images captured by the traveling direction imaging unit based on the features. Based on the setting result of the advancing direction parking frame line detection parameter setting unit for setting parameters for detection and the advancing direction parking frame line detection parameter setting unit, from the images captured by the advancing direction imaging unit, The process of detecting a parking frame where the vehicle can park It characterized by having a a direction parking area detection unit.

  According to the parking frame recognition device according to the present invention configured as described above, an image including a road surface on the left and right sides of the host vehicle is captured by the side imaging unit attached to the host vehicle, and the side parking frame line is captured. The detection unit detects the parking frame line on the side of the host vehicle, and the parking frame line feature calculation unit calculates the feature of the parking frame line. Based on the characteristics of the parking frame line calculated in this way, the traveling direction parking frame line detection parameter setting unit sets parameters for detecting the parking frame line in the traveling direction of the host vehicle, and the traveling direction imaging unit captures an image. From the image including the road surface in the traveling direction of the host vehicle, the parking area detection unit for the traveling direction detects the parking frame in which the host vehicle can be parked. Can be reliably detected.

  According to the parking assistance apparatus according to the present invention, a parking space in the traveling direction of the host vehicle can be reliably detected from as far away as possible.

It is a functional block diagram which shows schematic structure of the parking frame recognition apparatus which concerns on Example 1 which is 1 embodiment of this invention. It is a figure explaining operation | movement of a side imaging part, (a) is a figure explaining the imaging range of the side imaging part mounted in the own vehicle and the own vehicle in a parking lot. (B) is a figure which shows the example of the image imaged with the right side camera. (C) is a figure which shows the example of the image imaged with the left camera. It is a figure explaining operation | movement of a advancing direction imaging part, (a) is a figure explaining the imaging range of the front camera mounted in the own vehicle and the own vehicle in a parking lot. (B) is a figure showing an example of an image picturized with a front camera. It is a figure explaining the detection method of a parking frame line, (a) is a figure explaining the content of the process which detects a parking frame line from the image imaged with the right side camera. (B) is a figure which shows the result of having projected the parking frame line candidate point detected by the process shown to Fig.4 (a) on the vehicle coordinate system (X, Y). It is a figure explaining the characteristic which a parking frame line has. It is a figure explaining the shape feature of a parking frame line and a parking frame, (a) is a figure showing the example of the shape of a parallel parking frame. (B) is a figure which shows the example of a shape of a diagonal parking frame. (C) is a figure which shows the example of a shape of a parallel parking frame. It is a figure explaining the method of recognizing that the parking classification in a parking lot is parallel parking. It is a figure explaining the method of recognizing that the parking classification in a parking lot is diagonal parking. It is a figure explaining the method of recognizing that the parking classification in a parking lot is parallel parking. It is a figure explaining the content of the process which detects a parking possible area | region in the parking lot parked in parallel, (a) is a figure explaining the positional relationship of the parking lot parked in parallel and the own vehicle. (B) is a figure explaining the result of having detected the parking possible area. It is a figure explaining the content of the process which detects a parking possible area | region in the parking lot parked diagonally, (a) is a figure explaining the positional relationship of the parking lot diagonally parked and the own vehicle. (B) is a figure explaining the result of having detected the parking possible area. It is a figure explaining the content of the process which detects a parking possible area | region in the parking lot parked in parallel, (a) is a figure explaining the positional relationship of the parking lot parked in parallel and the own vehicle. (B) is a figure explaining the result of having detected the parking possible area. It is a figure which shows an example of the result of having output the detected parking possible area | region. It is a flowchart which shows the flow of the whole process performed in Example 1 which is 1 embodiment of this invention. It is a flowchart which shows the flow of the process which calculates the characteristic of a parking frame line. It is a flowchart which shows the flow of the process which detects a parking frame line. It is a flowchart which shows the flow of the process which performs the parameter setting for detecting the parking frame line of the advancing direction. It is a flowchart which shows the flow of the process which detects the advancing direction parking possible area | region. It is a flowchart which shows the flow of the process which outputs the detected parking possible area | region. It is a flowchart which shows the flow of the process which determines the priority of the several parkable area | region detected. FIG. 6 is a diagram illustrating an arrangement form of image pickup units in a modification example of Example 1.

  Hereinafter, an embodiment of a parking assistance device according to the present invention will be described with reference to the drawings.

In this embodiment, a parking frame recognition device according to the present invention is mounted on a vehicle, and a parking assist system for detecting a parkable area in the traveling direction and transmitting it to a driver of the vehicle is configured.
(Description of the configuration of the embodiment)

  First, the configuration of the apparatus will be described with reference to FIG. As shown in FIG. 1, a parking frame recognition apparatus 100 according to the present embodiment is installed in a vehicle 10 (own vehicle) and captures an image of a region including a road surface on the left and right sides of the vehicle 10. 12, a traveling direction imaging unit 15 that captures an image of an area including a road surface before and after the vehicle 10, input of images captured by the side imaging unit 12 and the traveling direction imaging unit 15, and necessary image processing are performed. An electronic control unit (ECU) 20, a vehicle state acquisition unit 40 that acquires a vehicle state such as a vehicle speed, a steering angle, and a shift position of the vehicle 10, and a parking frame recognition result that is provided on the dashboard of the vehicle 10 is displayed. And a monitor 50.

  The side imaging unit 12 is further installed in the vicinity of the right door mirror of the vehicle 10, and is installed in the vicinity of the right camera 12 a that captures an image of an area including the road surface on the right side of the vehicle 10 and the left door mirror of the vehicle 10. And a left camera 12b that captures an image of a region including the road surface on the left side of the vehicle 10. The right camera 12a and the left camera 12b are preferably color cameras. However, the present invention is not limited to a color camera, and a monochrome camera may be used. In that case, as will be described later, the types of features of the detectable parking frame line are limited.

  The traveling direction imaging unit 15 is further installed at the front end of the vehicle 10, and is installed at the front camera 15 a that captures an image of an area including the road surface in front of the vehicle 10, and at the rear end of the vehicle 10. And a rear camera 15b that captures an image of an area including the rear road surface. The front camera 15a and the rear camera 15b are preferably color cameras. However, the present invention is not limited to a color camera, and a monochrome camera may be used. In that case, as will be described later, the types of features of the detectable parking frame line are limited.

  It is assumed that the right camera 12a, the left camera 12b, the front camera 15a, and the rear camera 15b are installed at predetermined positions and facing predetermined directions, respectively.

  The electronic control unit (ECU) 20 is composed of an image processing processor, an arithmetic unit made up of a microcomputer, and a storage device made up of a memory, and more specifically, an image input unit 22, a side parking frame line detection unit 23, a parking device The frame line feature calculation unit 24, a traveling direction parking frame line detection parameter setting unit 26, a traveling direction parking possible area detection unit 28, a parking frame priority order determination unit 30, and a parking frame information output unit 32 are included.

  The image input unit 22 performs A / D conversion on images captured by the side imaging unit 12 and the traveling direction imaging unit 15 and inputs the images to the electronic control unit (ECU) 20 as digital images.

  The side parking frame line detection unit 23 is a parking frame that indicates the boundary position of the parking frame drawn on the road surface from the left and right side images of the input vehicle 10 captured by the side imaging unit 12. Detect lines.

  Based on the detection result of the left and right side parking frame lines of the vehicle 10, the parking frame line feature calculation unit 24 determines whether or not there is a parking frame line, the interval between the parking frame lines, the shape of the parking frame line, the color of the parking frame line, An angle formed by the traveling direction of the vehicle 10 and the parking frame line is calculated and stored as a feature of the parking frame line. Furthermore, based on these characteristics, the parking classification showing the parking form with respect to a parking frame is determined. The type of feature calculated here differs depending on the specifications of the right camera 12a, left camera 12b, front camera 15a, and rear camera 15b to be used. For example, when a monochrome camera is used as the right camera 12a, the left camera 12b, the front camera 15a, and the rear camera 15b, the color of the parking frame line cannot be detected. Among the features, features other than the color of the parking frame line are calculated, and the subsequent processing is performed.

  The traveling direction parking frame line detection parameter setting unit 26 calculates the image captured by the traveling direction imaging unit 15 based on the characteristics of the parking frame line on the left and right sides of the vehicle 10 calculated by the parking frame line feature calculation unit 24. From the inside, parameters necessary for detecting a parking frame line existing in the traveling direction of the vehicle 10 are set.

  The traveling direction parking possible area detection unit 28 further includes a parking frame line length calculation unit 28a and a wheel detection unit 28b.

  The traveling direction parking area detection unit 28 exists in the traveling direction of the vehicle 10 using the parameters set by the traveling direction parking frame line detection parameter setting unit 26 from the images captured by the traveling direction imaging unit 15. Detect parking borders. Then, the vehicle 10 can be parked based on the length of the parking frame line calculated by the parking frame line length calculation unit 28a and the appearance of the wheel of the parked vehicle (other vehicle) detected by the wheel detection unit 28b. Parking areas R1, R2,... (Parking frame) (see FIG. 13) are detected.

  The parking frame priority order determination unit 30 determines the priority order in the order of easy parking when a plurality of parking possible areas (R1, R2,...) (Parking frames) are detected in the traveling direction of the vehicle 10.

The parking frame information output unit 32 converts the information of the detected parking area Ri (= R1, R2,...) (Parking frame) into a form that is easy for the driver of the vehicle 10 to understand.
(Explanation of operation outline of the embodiment)

Next, an outline of the operation of the present embodiment will be described with reference to FIGS. 2 (a) to 2 (c), FIGS. 3 (a), and 3 (b). 2A is a diagram illustrating the imaging range of the side imaging unit 12 (12a, 12b), and FIG. 2B is an image I _R (x, x) actually captured by the right camera 12a. An example of y) is shown. FIG. 2C shows an example of an image I _L (x, y) actually captured by the left camera 12b. Further, FIG. 3A is a diagram for explaining that the front camera 15a captures an image in the imaging range 16a, and FIG. 3B is an image _IF (x) actually captured by the front camera 15a. , Y).

When the vehicle 10 (own vehicle) enters the parking lot 200, first, the left and right side images are captured by the side imaging unit 12 (12a, 12b). That is, as illustrated in FIG. 2A, the image I _R (x, y) in the right imaging range 13 a and the image I _L (x in the left imaging range 13 b with respect to the traveling direction F of the vehicle 10. , Y) are respectively imaged.

  In addition, it can be judged based on the information of the car navigation system (not shown) with which the vehicle 10 was equipped, for example that the vehicle 10 entered the parking lot 200. Further, when the vehicle 10 exists inside the parking lot 200, the electronic control unit (ECU) 20 determines that the side imaging unit 12 (12a, 12b) when the vehicle speed v of the vehicle 10 falls below a predetermined threshold value vth. Shall be activated.

A line segment that is considered to represent a parking frame line is detected from the image I _R (x, y) and the image I _L (x, y) thus captured. A specific method for detecting the parking frame line will be described later.

Further, based on the parking frame line detected from the image I _R (x, y) and the image I _L (x, y), the feature of the parking frame line is calculated. This feature includes, for example, the presence / absence of parking frame lines, the interval between parking frame lines, the shape of the parking frame lines, the color of the parking frame lines, and the angle between the traveling direction of the vehicle 10 and the parking frame lines. The details of each feature and the calculation method will be described later in detail.

Next, based on the characteristics of the parking frame line thus calculated, a parking type representing a parking mode for the parking frame in the parking lot 200 is determined. Specifically, parallel parking is a form in which the vehicle is parked in a parking frame that is aligned side by side in a direction orthogonal to the road side end, and the vehicle is parked in a parking frame that is aligned obliquely to the road side end. It is determined whether the parking is oblique or parallel parking in which the vehicle is parked in a vertical state in a parking frame aligned vertically along the road side end. For example, in the case of FIGS. 2B and 2C, it is determined that parallel parking is performed. A specific parking type determination method will be described later.
(Description of how to set the parking frame search area in the vehicle traveling direction)

Next, the presence position of the parking frame line that is predicted to be observed by the front camera 15a that captures the traveling direction of the vehicle 10 is estimated. This prediction is estimated based on the fact that the parking type is parallel parking, and the actual mounting position and mounting direction of the right camera 12a, the left camera 12b, and the front camera 15a. By this estimation, it is estimated that the parking frame line wi in the image I _F (x, y) captured by the front camera 15a exists, for example, inside the search areas 18a and 18b shown in FIG. Is done.

Then, the parking frame line wi shown in FIG. 3A is detected from the image _IF (x, y) shown in FIG. 3B actually captured by the front camera 15a. At this time, the interval between the parking frame lines wi, the characteristics of the parking frame line wi calculated earlier, the shape of the parking frame line wi, the color of the parking frame line wi, the traveling direction F of the vehicle 10 and the parking frame line wi Information on the angle θ formed can be used.

In this manner, the parking frame line wi is detected from the image _IF (x, y) without any clues, but the right camera is limited to the search areas 18a and 18b. By using the feature of the parking frame line wi actually detected from the images captured by the camera 12a and the left camera 12b, the parking frame line wi in the front (traveling direction) of the vehicle 10 can be detected more reliably. it can.
(Description of a method for detecting a parking frame line from an image captured by the side imaging unit)

Next, a specific method for detecting the parking frame line will be described with reference to FIGS. FIG. 4A is a diagram for explaining how a parking frame line is detected from the image I _R (x, y) (FIG. 2B) captured by the right camera 12a (FIG. 1). is there.

As shown in FIG. 4A, the image I _R (x, y) has an image coordinate system (x, y) where the upper left corner of the image is the origin, the horizontal right direction is the x axis, and the vertical downward direction is the y axis. ) Is set. At this time, the image _I R (x, y) a plurality of longitudinal positions yi of (= y1, y2, ...) to set the horizontal line, on each horizontal line, the image _I R (x, y) to the left While changing from right to left, the change (value of luminance difference) stored in each pixel is calculated between adjacent pixels.

  Then, a point where the value (luminance) stored in each pixel changes from dark to light (plus edge point) and a point where the value changes from dark to light (minus edge point) are detected.

Since the thickness of the parking frame line that appears in the image I _R (x, y) is known in advance, the distance between the detected positive edge point and the negative edge point is calculated to determine the expected parking frame line thickness. All pairs of plus edge point and minus edge point that are substantially equal are detected.

  When the position of the midpoint of the pair of the positive edge point and the negative edge point detected in this way is obtained, as shown in FIG. 4A, parking frame line candidate points E11 to E51 are detected at the vertical position y1. Thus, parking frame line candidate points E12 to E52 are detected at the vertical position y2. Although FIG. 4A shows only the results of the vertical positions y1 and y2, in practice, the same processing is executed at more vertical positions yi.

In addition, there is a possibility that a parking frame line extending in a direction orthogonal to the parking frame line shown in FIG. Therefore, in addition to the parking frame line candidate points described above, vertical lines are set at a plurality of lateral positions xi (= x1, x2,...) Of the image I _R (x, y), and on the vertical lines. While changing the image I _R (x, y) from the top to the bottom, the change of the value (luminance) stored in each pixel is calculated between the adjacent pixels, and the parking frame line as before. Candidate points may be detected.

Next, as shown in FIG. 4B, the detected parking frame line candidate points E11 to E51 and the parking frame line candidate points E12 to E52 are respectively obtained from the coordinate system of the image I _R (x, y). Projection is performed on the vehicle coordinate system (X, Y) where the vehicle 10 exists.

  For example, as shown in FIG. 4B, the vehicle coordinate system (X, Y) is a traveling direction of the vehicle with the installation position of the front camera 15a (FIG. 1) of the vehicle 10 as the origin O (0, 0). Is a coordinate system set on the road surface with the X axis as the X axis and the direction orthogonal to the X axis as the Y axis.

  As described above, the right camera 12a, the left camera 12b, the front camera 15a, and the rear camera 15b are installed at predetermined positions in a predetermined direction, respectively. The position on x, y) and the position on the vehicle coordinate system (X, Y) can be uniquely associated. Therefore, the projection processing described above can be performed by simple numerical calculation. This projection process can also be performed using a table created in advance.

  After executing the projection processing, the detected parking frame line candidate points (E11 to E51) are detected based on the distribution state of the plurality of parking frame line candidate points (E11 to E51, E12 to E52) shown in FIG. , E12 to E52), it is determined that consecutive parking frame line candidate points, that is, parking frame line candidate points considered to form a line segment, constitute one parking frame line. For example, when the result of the projection process shown in FIG. 4B is obtained, it is determined that there are five parking frame lines w1, w2, w3, w4, and w5.

Note that the parking frame line wi can be similarly detected for the image I _L (x, y) captured by the left camera 12b.
(Explanation of parking frame feature calculation method)

Next, contents of features of the parking frame line detected from the images (I _R (x, y), I _L (x, y)) captured by the side imaging unit 12 (12a, 12b). This will be described with reference to FIGS. 5 and 6A to 6C.

(1) When the parking frame line w i is not detected from the parking frame line presence / absence image (I _R (x, y), I _L (x, y)), the current position is the parking frame with the parking frame drawn. Judge that it is not a parking lot and stop the subsequent processing. On the other hand, when the parking frame line wi is detected from the images (I _R (x, y), I _L (x, y)), it is determined that the current position is the parking lot where the parking frame is drawn. Then, the subsequent processing is executed.

(2) The interval between the adjacent parking frame lines wi and the parking frame line wi + 1 among the detected parking frame lines wi is calculated. And when the space | interval Jw of the adjacent parking frame line wi and parking frame line wi + 1 is around 2.5 m, it determines that the parking classification which is the parking form to a parking frame is parallel parking or diagonal parking. Further, when the interval Jw between the adjacent parking frame line wi and the parking frame line wi + 1 is about 5 m, it is determined that the parking type is parallel parking.

More specifically, when the parking frame line interval threshold W _PL is set to, for example, W _PL = 4 m, and the interval Jw between the adjacent parking frame line wi and the parking frame line wi + 1 is Jw <W _PL Determines that the parking type is parallel parking or diagonal parking. In addition, when it is Jw ≧ W _PL determines that the parking type is a parallel parking.

(3) The shape of the parking frame line The parking frame line wi may be composed of a single line, that is, a single line, or may be composed of two different lines, that is, a double line. Among these, the parking frame line wi is constituted by a single line, for example, among the parking frame lines wi detected by the method described in FIGS. 4A and 4B, The interval Jw is determined by being substantially equal between all detected parking frame lines wi and parking frame line wi + 1.

  6A to 6C show an example of a parking frame line and a shape of the parking frame that are generally present, and a parking example of the vehicle V. FIG. 6A shows an example of the shape of the parallel parking frame, FIG. 6B shows an example of the shape of the oblique parking frame, and FIG. 6C shows an example of the shape of the parallel parking frame.

  6A to 6C, as an example in which the parking frame line wi is formed of a single line, for example, the parking frames Pa and Pc in FIG. 6A and the parking frame Pf in FIG. Examples of Pg and parking frames Ph to Pl in FIG. 6C can be given.

  On the other hand, as an example in which the parking frame line wi is composed of double lines, for example, the parking frames Pb, Pd, and Pe in FIG. The fact that the parking frame line wi is composed of double lines in this way is, for example, that the parking frame line wi is adjacent to the parking frame line wi detected by the method described in FIGS. 4 (a) and 4 (b). And the parking frame line wi + 1 are determined by appearing alternately as a large value and a small value.

In particular, as with the parking frames Pd and Pe in FIG. 6A, the parking frame line wi in which one end point of the double line is connected will be described with reference to FIGS. 4A and 4B. After the parking frame line wi is detected by the above-described method, whether or not the end points of the adjacent parking frame line wi and the parking frame line wi + 1 are connected is determined again by the image I _R (x, y) or the image I _L. By referring to the luminance distribution of (x, y), it may be determined that the parking frame line wi is composed of a double line.

(4) The shape of the parking frame There are various variations in the shape of the parking frame composed of a plurality of parking frame lines wi, but the left and right parallel lines, the front and rear parallel lines, the U-shape, the rectangular shape shown in FIG. The shape is a typical shape. And the specific shape of each parking frame is as having shown to drawing corresponding to FIG. 6 (a)-(c), as described in FIG. Which of the shapes of the parking frame formed by the detected parking frame line wi can be determined by analyzing the connection relationship between the detected different parking frame lines wi.

  That is, the shape of the parking frame includes the presence / absence of the parking frame line wi detected by the method described in FIGS. 4A and 4B and the connection relationship between the end points of the detected different parking frame lines wi. It can be specified by seeking.

(5) Color of parking frame line When a color camera is used as the right camera 12a, left camera 12b, front camera 15a, and rear camera 15b, the color of the parking frame line can be detected. The color of the parking frame line wi is typically white and orange. The color of the parking frame line wi reads an image _{(I R (x, y)} , I L (x, y)) color information (e.g. RGB values) of the pixels corresponding to the parking frame line candidate points in the Can be detected.

(6) Direction of Parking Frame Line The angle θ formed by the traveling direction F of the vehicle 10 and the parking frame line wi is represented by the angle θ shown in FIGS. This angle θ can be calculated, for example, by obtaining an angle formed by the X axis indicating the direction of the detected parking frame line wi and the traveling direction of the vehicle 10 in FIG.

In general, a plurality of parking frame lines wi are detected from the image I _R (x, y) and the image I _L (x, y), respectively, but the detected parking frame lines wi are parallel. Therefore, the angle θ is obtained for each of the plurality of parking frame lines wi, and the calculated plurality of angles θ may be averaged to obtain an angle θ formed by the traveling direction of the vehicle and the parking frame line wi.
(Description of parking type judgment method)

  Next, based on the detection result of the detected parking frame line wi, a parking type determination method representing a parking mode for the parking frame will be described with reference to FIGS. 5 and 7 to 9.

  The determination of the parking type can be performed using the angle θ formed by the traveling direction F of the vehicle 10 and the parking frame line wi, and the interval Jw between adjacent parking frame lines.

  That is, as shown in FIG. 5, when the angle θ is 90 ° ± 10 °, it is determined that the parking type for the detected parking frame is parallel parking or parallel parking. Further, when the angle θ is 45 ° ± 15 °, it is determined that the parking type for the detected parking frame is oblique parking.

  Moreover, as shown in FIG. 5, when the interval Jw between the adjacent parking frame line wi and the parking frame line wi + 1 is about 2.5 m, the parking type for the detected parking frame is parallel parking or diagonal parking. To be judged. Moreover, when the space | interval Jw is around 5 m, it is judged that the parking classification with respect to the detected parking frame is parallel parking.

  Therefore, when the angle θ is 90 ° ± 10 ° and the interval Jw is about 2.5 m, it is determined that the parking type for the parking frame constituted by the detected parking frame line is parallel parking. . FIG. 7 is a diagram illustrating a state of the angle θ and the parking frame line interval Jw when it is determined that the parking type is parallel parking.

  Since the detection of the parking frame line wi described in FIGS. 4A and 4B detects the parking frame line wi based on the distance between the plus edge point and the minus edge point, as shown in FIG. Even if the parked vehicle 250 (another vehicle) is parked in the parking frame, it can be executed without being affected by the presence of the parked vehicle 250.

  In addition, when the angle θ is 45 ° ± 15 ° and the interval Jw is about 2.5 m, the parking type for the parking frame constituted by the detected parking frame line is determined to be oblique parking. . FIG. 8 is a diagram illustrating a state of the angle θ and the interval Jw between the parking frame lines when it is determined that the parking type is oblique parking.

When the angle θ is 90 ° ± 10 ° and the interval Jw is around 5 m, it is determined that the parking type for the parking frame constituted by the detected parking frame line is parallel parking. FIG. 9 is a diagram illustrating the state of the angle θ and the interval Jw between the parking frame lines when it is determined that the parking type is parallel parking.
(Description of how to set the travel direction parking frame detection parameter)

Next, on the basis of the calculated feature of the parking frame line wi, the parking frame line wi ahead of the vehicle 10 (host vehicle) in the traveling direction from the image _IF (x, y) captured by the front camera 15a. A method for setting a parameter for detecting the error will be described.

  The parameters set here are the distance between the parking frame lines wi, the shape of the parking frame line w i, the color of the parking frame line w i, the traveling direction F of the vehicle 10 and the parking, which are calculated by the parking frame line feature calculation unit 24 previously. The angle θ formed by the frame line wi and the parking type. Features that cannot be detected are excluded depending on the specifications of the right camera 12a, left camera 12b, front camera 15a, and rear camera 15b used.

Here, first, based on the determined parking type, a search area for the parking frame line wi is set in the image _IF (x, y) captured by the front camera 15a. Specifically, based on the alignment state of the parking frames detected from the images captured by the right camera 12a and the left camera 12b, the relative relationship between the right camera 12a and the left camera 12b is set at a known position. The position of the parking frame predicted to be captured in the image captured by the forward camera 15a is predicted, and the search area for the parking frame line wi is set.

  Further, the interval between the parking frame lines wi previously calculated by the parking frame line feature calculating unit 24, the shape of the parking frame line wi, the color of the parking frame line wi, the angle θ between the traveling direction F of the vehicle 10 and the parking frame line wi. Based on, the detection conditions for the parking frame line wi are limited.

That is, as described above, the parking frame line wi is detected as an area where parking frame line candidate points detected as a pair of plus edge point and minus edge point are continuous. The color of the region between the plus edge point and the minus edge point can be predicted, and the extending direction of the parking frame line candidate point is predicted from the angle θ formed by the traveling direction F of the vehicle 10 and the parking frame line wi. be able to. Furthermore, when determining which of the detected plurality of parking frame lines wi constitutes the parking frame, the interval between the parking frame lines wi and the shape of the parking frame line wi can be used.
(Description of parking frame line detection method from front camera image)

The detection of the parking frame line wi from the image I _F (x, y) captured by the front camera 15a is performed by using the parking frame line wi from the image I _R (x, y) captured by the right camera 12a as described above. It can be performed in the same manner as the detection method.

Specifically, the inside of the previously set search area is scanned in a direction parallel to the traveling direction F of the vehicle 10 to calculate the luminance distribution of the image I _F (x, y). A parking frame line candidate point is detected based on the position of the minus edge point and the color between the plus edge point and the minus edge point. Then, among the detected parking frame line candidate points, continuous parking frame line candidate points extending in a direction near the angle θ formed by the traveling direction F of the vehicle 10 and the parking frame line wi are calculated. Detect as line wi.

Next, based on the parking frame line wi detected from the image _IF (x, y) captured by the front camera 15a, the parking area (parking frame) of the vehicle 10 is determined using FIGS. A detection method will be described.
(Explanation of the parking direction detection method for the direction of travel (Parallel parking))

  FIGS. 10A and 10B are diagrams for explaining the contents of a process for detecting a parking area in a parking lot 200 that is parked in parallel, and FIG. It is a figure explaining the positional relationship of the own vehicle. (B) is a figure explaining the result of having detected the parking possible area.

  As shown to Fig.10 (a), the parking frame lines w1-w7 were detected by the detection process of the parking frame line wi demonstrated previously, and the parking frames C1-C6 were detected based on the result. To do. And among the detected parking frames C1-C6, the parked vehicle 250 (other vehicles) shall stop other than the parking frame C3.

At this time, first, the length of the portion of the parking frame line wi that is shown in the image _IF (x, y) shown in FIG. 10B is obtained. Here, of the parking frame line wi, the length of the portion shown in the image I _F (x, y) is represented by the parking frame line length L (wi) (FIG. 10A).

  That is, when in the state of FIG. 10A, the parking frame line lengths L (w1) to L (w7) are respectively calculated. For example, as shown in FIG. 10A, the parking frame line length L (w1) extends from the position of the front camera 15a and is the end of the parked vehicle 250 (another vehicle) parked in the parking frame C1. The distance between the intersection of the half line Q1 in contact with the part and the parking frame line w1 and the end point on the right side of the parking frame line w1 is shown. Similarly, for the parking frame line lengths L (w2) to L (w7), the parking frame line lengths L (w2) to L are based on the positions of the intersections of the half lines Q2 to Q7 and the parking frame lines w2 to w7. (W7) is calculated respectively.

  Next, among the adjacent parking frame lines, the parking frame line length L (wi + 1) on the side close to the vehicle 10 (own vehicle) is compared with the parking frame line length L (wi) on the far side.

  When the parked vehicle 250 (other vehicle) is parked in parallel with the parking frame Ci, generally, the parking frame line wi + 1 on the side close to the vehicle 10 (own vehicle) among the parking frame lines wi constituting the parking frame Ci is: It is observed longer than the parking frame line wi far from the vehicle 10 (own vehicle). On the other hand, when the parked vehicle 250 is not parked in parallel with the parking frame Ci, generally, the parking frame line wi + 1 on the side closer to the vehicle 10 (the host vehicle) among the parking frame lines wi constituting the parking frame Ci is the vehicle 10. Observed shorter than the parking frame line wi on the far side from the host vehicle.

  Accordingly, the parking frame line lengths L (wi) and L (wi + 1) are calculated for the adjacent parking frame lines wi and wi + 1, respectively. When L (wi)> L (wi + 1), the parking frame Ci is It is determined that the parking frame can be parked in parallel. On the other hand, when L (wi) ≦ L (wi + 1), it is determined that the parking frame Ci cannot be parked in parallel.

  In the case of FIG. 10A, it is determined that the parking frame C3 is a parking frame that can be parked in parallel, and it is determined that parking cannot be performed in parallel to other parking frames. In the example of FIG. 10 (a), only the parking frame C3 is detected as a parking area (parking frame) that can be parked in parallel. When is not stopped, a plurality of parking frames are detected as parking frames that can be parked in parallel.

  In addition to the parking frame line lengths L (wi) and L (wi + 1), a parking area where parallel parking is possible can be detected with reference to how the wheel H of the parked vehicle 250 (other vehicle) looks. . That is, among the adjacent parking frame lines wi and wi + 1, when the wheel H of another parked vehicle 250 is detected in the vicinity of the far side of the far side parking frame line wi as viewed from the vehicle 10, the adjacent parking frame line It can be determined that the space between wi and wi + 1 is a parking frame that can be parked in parallel. The method based on the appearance of the wheel H of the parked vehicle 250 can be applied only when the parked vehicle 250 is parked in the adjacent parking frame on the far side of the parking area.

  The detection of the wheel of the parked vehicle 250 (another vehicle) can be performed using Hough conversion, for example. Specifically, a parameter space for estimating the center position of the wheel H is prepared, and among the adjacent parking frame lines wi and wi + 1, the wheel H is further located near the far side of the far side parking frame line wi. Is set, and an edge composing point having a large luminance difference between adjacent pixels is detected within the small area, and the normal direction of the edge at the pixel is obtained. Then, in the prepared parameter space, the vote value at a position corresponding to the predicted wheel radius is incremented from the position of the edge composing point toward the obtained normal direction.

  When the same processing is performed on the pixels inside the set small area, when the wheel H exists in the set small area, the voting that protrudes at the position corresponding to the center of the wheel in the parameter space. A value is obtained. By detecting this protruding vote value, it can be determined that the wheel H is in the set small area.

Since the wheel radius observed in the image _IF (x, y) can be predicted in advance according to the vertical position in the image, if the Hough transform is executed based on the predicted value, Good. That is, in the image I _F (x, y), when the small region is set upward, the parameter space is voted with a small radius value, and conversely, when the small region is set downward, it is large. Vote on the parameter space with the radius value.

In the case of the image _IF (x, y) in FIG. 10B, the parking frame C3 satisfies the condition of L (w3)> L (w4) as shown in FIG. Although it is determined that the parking frame is possible, it is further determined that the parking frame C3 is a parking frame that can be parked in parallel by detecting the wheel H1 in the far adjacent parking frame of the parking frame line w3. be able to.
(Description of how to detect the direction of parking in the direction of travel (diagonal parking))

  11 (a) and 11 (b) are diagrams for explaining the contents of processing for detecting a parking area in the parking lot 200 parked obliquely. FIG. 11 (a) shows the parking lot and the vehicle 10 ( It is a figure explaining the positional relationship of the own vehicle. (B) is a figure explaining the result of having detected the parking possible area.

  As shown to Fig.11 (a), parking frame line w10-w15 is detected by the detection process of parking frame line wi demonstrated previously, and parking frame C7-C11 is detected based on the result. To do. And among the detected parking frames C7 to C11, it is assumed that the parked vehicle 250 (other vehicle) is stopped except for the parking frames C9 and C10.

At this time, first, the length of the portion of the parking frame line wi that appears in the image _IF (x, y) shown in FIG.

  That is, when it is in the state of Fig.11 (a), parking frame line length L (w10) -L (w15) is each calculated. For example, as shown in FIG. 11A, the parking frame line length L (w10) extends from the position of the front camera 15a and is the end of the parked vehicle 250 (another vehicle) parked in the parking frame C7. This represents the distance between the intersection of the half line Q6 in contact with the part and the parking frame line w10 and the left end point of the parking frame line w10. Similarly, for the parking frame line lengths L (w11) to L (w15), the parking frame line lengths L (w11) to L are based on the positions of the intersections of the half lines Q7 to Q9 and the parking frame lines w11 to w15. (W15) is calculated respectively.

  Next, it is determined whether or not the adjacent parking frame line lengths L (wi) and L (wi + 1) are longer than a predetermined parking frame line length L0 set in advance.

  When the parked vehicle 250 (other vehicle) is parked diagonally in the parking frame Ci, generally, the parking frame line wi + 1 on the side closer to the vehicle 10 (own vehicle) and the vehicle out of the parking frame lines wi constituting the parking frame Ci. Both of the parking frame lines wi on the side far from 10 (own vehicle) are observed short. On the other hand, when the parked vehicle 250 is not diagonally parked in the parking frame Ci, generally, the parking frame line wi + 1 and the vehicle 10 (on the side closer to the vehicle 10 (own vehicle)) out of the parking frame lines wi constituting the parking frame Ci. Both of the parking frame lines wi far from the host vehicle are observed for a long time.

  Therefore, the parking frame line lengths L (wi) and L (wi + 1) are calculated for the adjacent parking frame lines wi and wi + 1, respectively, and both L (wi) and L (wi + 1) are predetermined parking frame lines. When longer than the length L0, it is determined that the parking frame Ci is a parking frame that can be parked obliquely. On the other hand, when both L (wi) and L (wi + 1) are shorter than the predetermined parking frame line length L0, it is determined that the parking frame Ci cannot be parked obliquely.

  In the case of FIG. 11A, it is determined that the parking frames C8, C9, C10, and C11 are parking frames that can be diagonally parked, and it is determined that the other parking frames cannot be diagonally parked.

  Next, referring to how the wheel H of the parked vehicle 250 (other vehicle) is seen, it is confirmed whether or not the parking frames C8, C9, C10, C11 are certainly parking frames that can be obliquely parked. That is, when the wheel H of another parked vehicle 250 is not detected in the area between the adjacent parking frame lines wi and wi + 1, the space between the adjacent parking frame lines wi and wi + 1 is a parking frame that can be parked obliquely. It can be judged.

  As described above, the wheel of the parked vehicle 250 (other vehicle) can be detected using the Hough transform. The specific method is the same as that described above except that a small region for detecting the wheel H is set between the adjacent parking frame lines wi and wi + 1, and thus the description thereof is omitted.

And in the image of FIG.11 (b), wheel H2 is detected inside parking frame C8 among parking frames C8, C9, C10, C11 detected as parking frames which can be parked in parallel, and parking frame C11. The wheel H3 is detected in the interior. Therefore, it can be determined that only the parking frames C9 and C10 are parking frames that can be parked obliquely.
(Description of how to detect the direction of parking in the direction of travel (Parallel parking))

  12 (a) and 12 (b) are diagrams illustrating the contents of a process for detecting a parking area in a parking lot 200 that is parked in parallel, and FIG. It is a figure explaining the positional relationship of the own vehicle. (B) is a figure explaining the result of having detected the parking possible area.

  As shown to Fig.12 (a), the parking frame lines w16-w20 are detected by the detection process of the parking frame line wi demonstrated previously, and the parking frames C12 and C13 were detected based on the result. To do. And among the detected parking frames C12 and C13, the parked vehicle 250 (another vehicle) shall stop other than parking frame C12.

At this time, first, the length of the portion of the parking frame line wi shown in the image I _F (x, y) shown in FIG.

  That is, when it is in the state of Fig.11 (a), parking frame line length L (w16) -L (w20) is each calculated. For example, as shown in FIG. 12A, the parking frame line length L (w17) extends from the position of the front camera 15a and is the end of the parked vehicle 250 (another vehicle) parked in the parking frame C13. This represents the distance between the intersection of the half line Q10 in contact with the part and the parking frame line w17 and the end point on the right side of the parking frame line w17. Similarly, the parking frame line lengths L (w16) and L (w18) to L (w20) are calculated for the parking frame line lengths L (w16) and L (w18) to L (w20), respectively.

  Next, among the adjacent parking frame lines, the parking frame line length L (wi + 1) on the side close to the vehicle 10 (own vehicle) is compared with the parking frame line length L (wi) on the far side.

  When the parked vehicle 250 (other vehicle) is parked in parallel in the parking frame Ci, generally, the parking frame line wi + 1 on the side close to the vehicle 10 (own vehicle) among the parking frame lines wi constituting the parking frame Ci is: It is observed longer than the parking frame line wi on the side far from the vehicle 10. On the other hand, when the parked vehicle 250 is not parked in parallel in the parking frame Ci, the parking frame line wi + 1 on the side closer to the vehicle 10 among the parking frame lines wi constituting the parking frame Ci is generally on the side far from the vehicle 10. Observed shorter than the parking frame wi.

  Accordingly, the parking frame line lengths L (wi) and L (wi + 1) are calculated for the adjacent parking frame lines wi and wi + 1, respectively. When L (wi)> L (wi + 1), the parking frame Ci is It is determined that the parking frame can be parked in parallel. On the other hand, when L (wi) ≦ L (wi + 1), it is determined that parallel parking is not possible in the parking frame Ci.

In the case of FIG. 12A, it is determined that the parking frame C12 is a parking frame that can be parked in parallel, and it is determined that the other parking frames cannot be parked in parallel. In the example of FIG. 12 (a), only the parking frame C12 is detected as a parking area where tandem parking is possible, but of course, the parked vehicle 250 (other vehicles) does not stop in a plurality of parking frames. Sometimes, a plurality of parking frames are detected as parking frames that can be parked in parallel.
(Description of parking frame priority determination method)

  When a plurality of parking areas Ri (parking frames) are detected, the parking frame priority order determination unit 30 gives priority to the detected plurality of parking areas Ri. Specifically, based on the current speed of the vehicle 10 (own vehicle) and the distance between the detected parking area Ri and the vehicle 10, the priority order is determined in the order in which parking is easy.

  Specifically, first, a distance between the vehicle 10 (own vehicle) and each detected parking area Ri (parking frame) is calculated.

  Next, when the vehicle 10 is moved to each parking area Ri based on the current vehicle speed of the vehicle 10 (own vehicle) and the distance to each parking area Ri, without sudden braking. It is determined whether or not each parking area Ri can be stopped, that is, whether or not there is a deceleration allowance. For example, when the vehicle 10 approaches the parking area Ri with the current vehicle speed, and then gives a preset allowable maximum deceleration acceleration to the vehicle 10, the vehicle 10 It is determined based on whether or not the vehicle can stop in the parking area Ri, and when it is determined that the vehicle 10 can stop, it is determined that there is a deceleration allowance.

  Further, when the vehicle 10 is moved under the above conditions, the time required to move the vehicle 10 to each parking area Ri is calculated.

  Based on the calculated deceleration allowance and the time required to move the vehicle 10 to each parking area Ri, up to the parking area Ri among the parking areas Ri determined to have a deceleration margin. It is determined that the order of priority is higher in the order of shorter time required for movement.

When the priority order is determined, the steering angle generated in the vehicle 10 is also taken into consideration, and when the vehicle is moved to the parking area Ri without performing a predetermined steering or higher, a higher priority order is given. You may make it do.
(Description of parking frame information output form)

  The parking frame information output unit 32 converts information of the detected parking area Ri (parking frame) into a form of an image that can be easily understood by the driver of the vehicle 10 (own vehicle), and displays the image on the monitor 50.

  Specifically, as shown in FIG. 13, the coordinates of the images captured by the front camera 15a, the rear camera 15b, the right camera 12a, and the left camera 12b are converted into a form in which the vehicle 10 is viewed from diagonally above the rear. Then, the coordinate-converted images are combined into a single image, and the detected parking area Ri (R1, R2 in the example of FIG. 13) is superimposed and displayed on the monitor 50.

  At this time, information indicating the determined priority order is given to the overlapping parking area Ri.

Further, since the vehicle 10 (own vehicle) is not imaged by the front camera 15a, the rear camera 15b, the right camera 12a, and the left camera 12b, an image of the vehicle 10 created by computer graphics is formed in an area where the vehicle 10 exists. Is superimposed.
(Explanation of processing flow)

  Next, the flow of processing performed in the first embodiment will be described with reference to the flowchart of FIG.

  (Step S1) In the electronic control unit (ECU) 20, it is determined whether or not the vehicle speed v of the vehicle 10 (host vehicle) is smaller than the threshold value vth. If v <vth, the process proceeds to step S10, and otherwise, step S1 is repeated.

  (Step S10) The parking frame line feature calculating unit 24 calculates the characteristics of the parking frame line. More detailed processing contents will be described later.

  (Step S50) In the traveling direction parking frame line detection parameter setting unit 26, a parking frame line detection parameter in the traveling direction of the vehicle 10 (own vehicle) is set. More detailed processing contents will be described later.

  (Step S60) In the traveling direction parking area detection unit 28, a parking area in the traveling direction of the vehicle 10 (own vehicle) is detected. More detailed processing contents will be described later.

(Step S90) The parking frame information output unit 32 outputs information on the detected parking area. More detailed processing contents will be described later.
(Description of parking frame feature calculation processing flow)

  Next, the flow of the parking frame line feature calculation process (step S10 in FIG. 14) will be described using the flowchart in FIG.

  (Step S12) Image input is performed from the left camera 12b and the right camera 12a, respectively.

  (Step S15) A parking frame line is detected from the captured image, and its feature is calculated. More detailed processing contents will be described later.

  (Step S40) It is determined whether or not the detected parking frame line is a double line. If it is a double line, the process proceeds to step S42; otherwise, the process proceeds to step S41.

  (Step S41) It is determined whether the interval Jw of the parking frame line is smaller than 4 m. When the interval Jw is smaller than 4 m, the process proceeds to step S42, and otherwise, the process proceeds to step S46.

  (Step S42) It is determined whether or not an angle θ (see FIG. 7) formed by the traveling direction F of the vehicle 10 and the parking frame line wi is within a range of 80 ° to 100 °. When the angle θ is within the range of 80 ° to 100 °, the process proceeds to step S43, and otherwise, the process proceeds to step S44.

  (Step S43) It is determined that the parking type is parallel parking.

  (Step S44) It is determined whether or not an angle θ (see FIG. 8) formed by the traveling direction F of the vehicle 10 and the parking frame line wi is within a range of 30 ° to 60 °. When the angle θ is in the range of 30 ° to 60 °, the process proceeds to step S45, and otherwise, it is determined that there is no parking type, and the process returns to the main routine (FIG. 14). When it is determined that there is no parking type, it is determined that there is no parking area, and the process is terminated.

  (Step S45) It is determined that the parking type is diagonal parking.

  (Step S46) It is determined that the parking type is parallel parking.

(Step S47) The calculated characteristics of the parking frame line (including the parking type) are stored, and the process returns to the main routine (FIG. 14).
(Description of parking frame line detection process)

  Next, the flow of the parking frame line detection process (step S15 in FIG. 15) will be described using the flowchart in FIG.

  (Step S20) A positive edge point and a negative edge point in the horizontal direction are detected from the images captured by the left camera 12b and the right camera 12a, respectively.

  (Step S22) Threshold processing is performed on the interval between the plus edge point and the minus edge point, and a pair of plus edge point and minus edge point having an interval close to the expected thickness of the parking frame line is extracted.

  (Step S24) The position of the midpoint of the pair of the extracted plus edge point and minus edge point is calculated.

  (Step S26) The calculated position of the midpoint of the pair of plus edge point and minus edge point is projected on the XY plane constituting the vehicle coordinate system (X, Y).

(Step S28) Based on the projection result onto the XY plane, a parking frame line is determined, and its characteristics are calculated. Thereafter, returning to the flowchart of FIG. 15, the process proceeds to step S40.
(Description of the flow of the direction-of-travel parking frame detection parameter setting process)

  Next, the flow of the travel direction parking frame line detection parameter setting process (step S50 in FIG. 14) will be described with reference to the flowchart in FIG.

  (Step S52) The feature of the stored parking frame line is read out.

  (Step S54) The search range of the traveling direction (front) parking frame line is set.

(Step S56) The parameter for detecting the advancing direction (front) parking frame line is set. Thereafter, the process returns to the main routine (FIG. 14).
(Explanation of the flow of the direction-of-travel parking area detection process)

  Next, the flow of the traveling direction parking possible area detection process (step S60 in FIG. 14) will be described with reference to the flowchart in FIG.

  (Step S62) An image is input from the front camera 15a.

  (Step S64) A parking frame line is detected from the captured image. The details of the processing are as shown in FIG.

  (Step S66) Based on the detection result of the parking frame line, it is determined whether there is a parking frame line pair. When a parking frame line pair is found, the process proceeds to step S68. Otherwise, the process proceeds to step S84.

  (Step S68) It is determined whether the parking type is parallel parking. When it is parallel parking, it progresses to step S70, and when that is not right, it progresses to step S74.

  (Step S70) It is determined whether or not the detected parking frame line length satisfies L (wi)> L (wi + 1). When the condition is satisfied, the process proceeds to step S82, and otherwise, the process proceeds to step S72.

  (Step S72) It is determined whether or not there is a wheel of another vehicle near the far side of the parking frame line wi. When there is a wheel of another vehicle, the process proceeds to step S82, and otherwise, the process proceeds to step S84.

  (Step S74) It is determined whether the parking type is diagonal parking. When it is diagonal parking, it progresses to step S76, and when that is not right, it progresses to step S80.

  (Step S76) It is determined whether L (wi)> L0 and L (wi + 1)> L0. If the condition is satisfied, the process proceeds to step S78. Otherwise, the process proceeds to step S84.

  (Step S78) It is determined whether or not there is a wheel of another vehicle between adjacent parking frame lines. If there is no wheel of another vehicle, the process proceeds to step S82, and otherwise, the process proceeds to step S84.

  (Step S80) It is determined whether or not the detected parking frame line length satisfies L (wi)> L (wi + 1). When the condition is satisfied, the process proceeds to step S82, and otherwise, the process proceeds to step S84.

  (Step S82) It is determined that there is a parking area. Thereafter, the process returns to the main routine (FIG. 14).

(Step S84) It is determined that there is no parking area. Thereafter, the process returns to the main routine (FIG. 14) to end the process.
(Description of the flow of parking area output processing)

  Next, the flow of the parking area output process (step S90 in FIG. 14) will be described with reference to the flowchart in FIG.

  (Step S92) It is determined whether a parking area is found. If a parking area is found, the process proceeds to step S93. Otherwise, the process returns to the main routine (FIG. 14) and ends.

  (Step S93) When a plurality of parking areas are found, the priority order of the parking frames constituting the parking area is determined. More detailed processing contents will be described later.

(Step S99) Image information displaying the detected parking area is generated and output. Thereafter, the process returns to the main routine (FIG. 14) to end the process.
(Description of parking frame priority order decision process)

  Next, the flow of the parking frame priority determination process (step S93 in FIG. 19) will be described using the flowchart in FIG.

  (Step S94) The distance from the vehicle 10 (own vehicle) to the detected parking area Ri is calculated.

  (Step S95) Assuming that the vehicle 10 (own vehicle) has been moved to each parking area Ri, it is determined whether or not there is a deceleration allowance.

  (Step S96) The time required to reach each detected parking area Ri is calculated.

(Step S97) The order of priority of each detected parking area Ri is determined in the order of the shortest arrival time with the allowance for deceleration. Thereafter, returning to the flowchart of FIG. 19, the process proceeds to step S99.
(Description of Modified Example of Example 1)

  The front camera 15a described in the first embodiment may be a camera that captures an area farther and narrower than the side imaging unit 12 (the right camera 12a and the left camera 12b). That is, as shown in FIG. 21, a front narrow-angle camera 17a that captures an area narrower than the right camera 12a and the left camera 12b is installed in the front (traveling direction) of the vehicle 10 (own vehicle). An image of the imaging range 16b may be captured using an image captured by the front narrow-angle camera 17a to detect a parking area in front of the vehicle 10 (traveling direction).

  Note that the front narrow-angle camera 17a shown in FIG. 21 is installed in the vicinity of the room mirror in the room of the vehicle 10 so as to capture the front of the vehicle 10, and the position of the front lane marker during the traveling of the vehicle 10 This is an example in which a camera used for a device that detects the lane departure of the vehicle 10 and detects the lane departure of the vehicle 10 is used for parking frame detection. In this way, by using the front narrow-angle camera 17a that captures an area narrower than the right camera 12a and the left camera 12b, a space (parking frame) in which the host vehicle can be parked is surely detected from farther away. can do.

  As described above, according to the parking frame recognition device 100 according to the first embodiment of the present invention configured as described above, the side imaging unit 12 (right camera 12a, attached to the vehicle 10 (own vehicle)). The left camera 12b) captures an image including the road surface on the left and right sides of the vehicle 10, and the side parking frame line detection unit 23 detects the parking frame line wi on the side of the vehicle 10, and the parking frame line feature is detected. The calculation unit 24 calculates the characteristics of the parking frame line wi. Based on the characteristics of the parking frame line wi calculated in this way, the traveling direction parking frame line detection parameter setting unit 26 sets parameters for detecting the parking frame line wi in the traveling direction of the vehicle 10, and the traveling direction imaging unit 15 (front camera 15a), the traveling direction parking available area detection unit 28 detects the parking frame in which the vehicle 10 can be parked from the image including the road surface in the traveling direction of the vehicle 10, and the vehicle 10 travels. The parking space in the direction can be reliably detected from as far away as possible.

  Moreover, according to the parking frame recognition apparatus 100 according to the first embodiment of the present invention configured as described above, the parking frame line feature calculation unit 24 uses the parking frame line w i as a feature of the parking frame line w i, While calculating any one of the interval Jw between the frame lines wi, the shape of the parking frame line wi, the color of the parking frame line wi, the traveling direction F of the vehicle 10 and the parking frame line wi, and based on the calculated result And since the parking classification showing the parking form of the vehicle 10 with respect to the parking frame line wi is determined, a parking classification can be specified reliably by simple processing.

  Moreover, according to the parking frame recognition apparatus 100 according to the first embodiment of the present invention configured as described above, the traveling direction parking frame line detection parameter setting unit 26 calculates the parking frame line calculated by the parking frame line feature calculation unit 24. Based on the characteristics of wi and the parking type, as parameters representing the characteristics of the parking frame line wi, the search area of the parking frame line wi in the traveling direction F of the vehicle 10 and the parking frame line of the traveling direction F of the vehicle 10 In order to set the detection threshold value for detecting the feature of wi, the feature of the parking frame line wi detected from the image captured by the side imaging unit 12 (the right camera 12a and the left camera 12b) is included. The feature of the parking frame line wi that appears in the image captured by the traveling direction imaging unit 15 (the front camera 15a) can be predicted, and the parking frame line wi can be predicted from the image captured by the traveling direction imaging unit 15. Can be reliably detected Kill.

  Moreover, according to the parking frame recognition device 100 according to the first embodiment of the present invention configured as described above, the traveling direction parking possible area detection unit 28 uses the setting result of the traveling direction parking frame line detection parameter setting unit 26 as a result of the setting. Based on the detected parking frame line, an adjacent parking frame is detected based on the detected parking frame line from the image captured by the traveling direction imaging unit 15 (front camera 15a). A parking frame line length calculation unit 28a that calculates the length of each line, and a wheel that detects the wheel of the parked vehicle 250 (another vehicle) from images captured by the traveling direction imaging unit 15 (front camera 15a). In order to detect the parking frame in which the vehicle 10 can be parked based on the parking type, the calculation result of the parking frame line length calculation unit 28a, and the detection result of the wheel detection unit 28b. Parking frame line wi only If you can not determine the appearance of the wheel of the parked vehicle 250 (another vehicle) may be utilized, which makes it possible to reliably detect the available parking parking frame.

  Moreover, according to the parking frame recognition device 100 according to the first embodiment of the present invention configured as described above, the parking type calculated by the parking frame line feature calculation unit 24 is parallel parking, and the adjacent parking frame Among the lengths of the lines, when the parking frame line wi on the far side is longer than the parking frame line wi on the side closer to the vehicle 10 (the host vehicle), or when the wheel detection unit 28b is adjacent to the parking frame line wi, When the wheel of the parked vehicle 250 (another vehicle) is detected in the vicinity of the far side of the far side parking frame line wi among the wi + 1, the traveling direction parking possible area detecting unit 28 detects the adjacent parking frame lines wi, wi + 1. Since it is determined that the vehicle 10 (own vehicle) is a parking frame that can be parked during the period, a parking frame that can be parked can be reliably detected in a parking lot that performs parallel parking.

  Moreover, according to the parking frame recognition device 100 according to the first embodiment of the present invention configured as described above, the parking type calculated by the parking frame line feature calculation unit 24 is diagonal parking, and an adjacent parking frame When the lengths of the lines wi and wi + 1 are both equal to or greater than a predetermined value and the wheel detection unit 28b does not detect the wheel of the parked vehicle 250 (other vehicle) between the adjacent parking frame lines, the traveling direction Since the parking area detection unit 28 determines that the vehicle 10 (own vehicle) is a parking frame that can be parked between adjacent parking frame lines wi and wi + 1, parking is possible in a parking lot that performs oblique parking. The frame can be reliably detected.

  Moreover, according to the parking frame recognition apparatus 100 according to the first embodiment of the present invention configured as described above, the parking type calculated by the parking frame line feature calculation unit 24 is parallel parking, and the adjacent parking frame When each of the lengths of the line wi is longer than the parking frame line wi farther than the parking frame line wi closer to the vehicle 10 (the host vehicle), the advancing direction parking possible region detection unit 28 performs the adjacent parking Since it is determined that the vehicle 10 (own vehicle) is a parking frame that can be parked between the frame lines wi, a parking frame that can be parked can be reliably detected in a parking lot that performs parallel parking.

  Moreover, according to the parking frame recognition apparatus 100 which concerns on Example 1 of this invention comprised in this way, the parking frame which the vehicle 10 (own vehicle) can park which the advancing direction parking possible area | region detection part 28 detected is possible. Since the parking frame information output unit 32 is displayed as image information, the occupant of the vehicle 10 can immediately recognize the location of the parking frame that can be parked.

  Moreover, according to the parking frame recognition device 100 according to the first embodiment of the present invention configured as described above, when the traveling direction parking possible area detection unit 28 detects a plurality of parking frames, a plurality of detected parking spaces are detected. Since each of the frames has a parking frame priority determining unit 30 that determines the priority in the order in which the vehicle 10 (own vehicle) is easy to park, when parking on a parking frame that can be parked, The vehicle 10 can be parked without causing a sudden change in vehicle behavior such as steering.

  Moreover, according to the parking frame recognition apparatus 100 which concerns on Example 1 of this invention comprised in this way, the parking frame priority order determination part 30 detects the speed of the vehicle 10 (own vehicle), and the advancing direction parking possible area | region. Since the priority order is determined based on the distance between each parking frame detected by the unit 28 and the vehicle 10, parking from the position far from the parking frame to the parking area Ri (parking frame) It is possible to reliably determine the ease with which it is applied.

  Furthermore, according to the parking frame recognition device 100 according to the modified example of the first embodiment of the present invention configured as described above, the traveling direction imaging unit 15 (front narrow-angle camera 17a) includes the side imaging unit 12 (rightward). Since the imaging range 16b that is farther and narrower than the camera 12a and the left camera 12b) is imaged, the parking possible region in the traveling direction of the vehicle can be reliably detected from a further distance.

In the first embodiment, the parking frame line wi in the traveling direction of the vehicle 10 (own vehicle) is detected from the image _IF (x, y) captured by the front camera 15a. In the parking environment, the rear side of the vehicle 10 is in the traveling direction, so the parking frame line wi is detected from the image captured by the rear camera 15b. Since the traveling direction of the vehicle 10 can be determined based on the shift position, switching between the front camera 15a and the rear camera 15b may be performed based on the detection result of the shift position.

  As mentioned above, although the Example of this invention was explained in full detail with drawing, since an Example is only an illustration of this invention, this invention is not limited only to the structure of an Example. Of course, changes in design and the like within a range not departing from the gist are included in the present invention.

DESCRIPTION OF SYMBOLS 10 Own vehicle 12 Side imaging part 12a Right camera 12b Left camera 15 Travel direction imaging part 15a Front camera 15b Rear camera 20 Electronic control unit (ECU)
22 Image input unit 23 Side parking frame line detection unit 24 Parking frame line feature calculation unit 26 Travel direction parking frame line detection parameter setting unit 28 Travel direction parking possible area detection unit 28a Parking frame line length calculation unit 28b Wheel detection unit 30 Parking frame priority determination unit 32 Parking frame information output unit 40 Vehicle state acquisition unit 50 Monitor 100 Parking frame recognition device

Claims (11)

A side imaging unit that is attached to the host vehicle and captures an image including a road surface on the left and right sides of the host vehicle;
A traveling direction imaging unit that is attached to the host vehicle and captures an image including a road surface in the traveling direction of the host vehicle;
From the image captured by the side imaging unit, a side parking frame line detection unit that detects a parking frame line on the side of the host vehicle, and
A parking frame line feature calculation unit for calculating the characteristics of the parking frame line on the side of the host vehicle;
Based on the characteristics, a traveling direction parking frame detection parameter setting unit that sets a parameter for detecting a parking frame line in the traveling direction of the host vehicle from the image captured by the traveling direction imaging unit;
Based on the setting result of the advancing direction parking frame line detection parameter setting unit, an advancing direction parking area detection unit that detects a parking frame in which the host vehicle can be parked from images captured by the advancing direction imaging unit; And a parking frame recognition device.   The parking frame line feature calculation unit, as the characteristics of the parking frame line, the presence or absence of the parking frame line, the interval between the parking frame lines, the shape of the parking frame line, the color of the parking frame line, the traveling direction of the host vehicle and the parking frame line The parking frame recognition device according to claim 1, wherein a parking type representing a parking mode of the vehicle with respect to the parking frame line is determined based on the calculated result while calculating any one of the angles formed.   The advancing direction parking frame line detection parameter setting unit, based on the characteristics and the parking type, includes, as the parameters, a search area for a parking frame line in the advancing direction of the own vehicle, and parking in the advancing direction of the own vehicle The parking frame recognition device according to claim 2, wherein a detection threshold value for detecting the feature of the frame line is set. Based on the setting result of the traveling direction parking frame line detection parameter setting unit, the traveling direction parking area detection unit detects a parking frame line in the traveling direction of the host vehicle from the images captured by the traveling direction imaging unit. A parking frame line length calculating unit that detects and calculates the length of each adjacent parking frame line based on the detected parking frame line;
A wheel detection unit that detects a wheel of another vehicle from the image captured by the traveling direction imaging unit, the parking type, the calculation result of the parking frame line length calculation unit, and the wheel detection unit The parking frame recognition apparatus according to any one of claims 1 to 3, wherein a parking frame in which the host vehicle can be parked is detected based on the detection result.   The parking type calculated by the parking frame line feature calculation unit is parallel parking, and among the lengths of the adjacent parking frame lines, the parking frame farther than the parking frame line closer to the host vehicle When the line is long, or when the wheel detector detects a wheel of another vehicle in the vicinity of the far side parking frame line among the adjacent parking frame lines, the parking in the traveling direction is possible. 5. The parking frame recognition device according to claim 4, wherein the area detection unit determines that the vehicle is a parking frame that can be parked between the adjacent parking frame lines.   The parking type calculated by the parking frame line feature calculation unit is diagonal parking, and the lengths of the adjacent parking frame lines are both equal to or greater than a predetermined value, and the wheel detection unit is When the wheel of the other vehicle is not detected between the parking frame lines to be performed, the traveling direction parking area detection unit determines that the vehicle is a parking frame in which the own vehicle can be parked between the adjacent parking frame lines. The parking frame recognition device according to claim 4.   The parking frame type calculated by the parking frame line feature calculation unit is parallel parking, and among the lengths of the adjacent parking frame lines, the parking frame farther than the parking frame line closer to the host vehicle 5. The parking according to claim 4, wherein when the line is long, the parking area detection unit in the traveling direction determines that the own vehicle is a parking frame that can be parked between the adjacent parking frame lines. Frame recognition device.   8. A parking frame information output unit that displays, as image information, a parking frame in which the host vehicle detected by the traveling direction parking area detection unit can be parked. 8. The parking frame recognition device described in 1.   When the traveling direction parking possible area detection unit detects a plurality of parking frames, a parking frame priority determination is performed for determining the priority order in which the host vehicle is easy to park for each of the detected parking frames. The parking frame recognition device according to any one of claims 1 to 8, further comprising a portion.   The parking frame priority determining unit determines the priority based on the speed of the host vehicle and the distance between each parking frame detected by the traveling direction parking possible region detecting unit and the host vehicle. The parking frame recognition device according to claim 9.   The parking frame recognition device according to any one of claims 1 to 10, wherein the traveling direction imaging unit captures an area farther and narrower than the side imaging unit.