WO2015186294A1

WO2015186294A1 - Vehicle-mounted image-processing device

Info

Publication number: WO2015186294A1
Application number: PCT/JP2015/002349
Authority: WO
Inventors: 下村　修; 直輝川嵜; 博彦柳川
Original assignee: 株式会社デンソー
Priority date: 2014-06-02
Filing date: 2015-05-08
Publication date: 2015-12-10
Also published as: JP6459226B2; DE112015002593T5; JP2015227139A

Abstract

　An ECU (10) for detecting a parking frame constituting a parking area using an image acquired from a rear camera (21) for imaging the surroundings of the host vehicle. The ECU (10) is provided with an image acquisition unit (110) for acquiring an image from the rear camera (21). The ECU (10) is provided with a straight-line detection unit (111) for detecting straight lines from the image acquired by the image acquisition unit (110). The ECU (10) is also provided with a parking frame candidate detector (112) for detecting, if a set of straight lines comprising two straight lines selected from a plurality of straight lines detected by the straight-line detection unit (111) satisfies a predetermined condition, the set of straight lines as a parking frame candidate. The ECU (10) is also provided with a parking frame detector (113) for detecting, as a parking frame, a parking frame candidate in which the orthogonality of the angle between a straight line constituting the parking frame candidate and the travel path of the host vehicle is within a predetermined range.

Description

In-vehicle image processing device

Cross-reference of related applications

This application is based on Japanese Application No. 2014-114205 filed on June 2, 2014, the contents of which are incorporated herein by reference.

This disclosure relates to an in-vehicle image processing device.

Conventionally, an in-vehicle image processing device that detects a straight line by acquiring an image from an imaging device mounted on a vehicle and performing image processing on the image is known. And a vehicle-mounted image processing apparatus detects the parking frame which comprises a parking area from the detected straight line.

However, in the on-vehicle image processing apparatus shown above, there is no description regarding a method for detecting a parking frame that specifically constitutes a parking section. Therefore, the inventors tried a method of combining arbitrary two straight lines from the detected straight lines and detecting whether the set of the two straight lines is a parking frame. As a result, the present inventor has newly found that an arrow or a pedestrian crossing drawn on a road may be erroneously detected as a parking frame by a method that only detects a pair of two straight lines. .

JP 2013-193545 A

The present disclosure has been made in view of the above points, and an object thereof is to provide an in-vehicle image processing device that can detect a parking frame more accurately.

The inventors of the present invention have newly created the following disclosure, paying attention to the fact that the parking section is constituted by a line that is substantially orthogonal to the traveling route of the host vehicle.

An in-vehicle image processing device that detects a parking frame that forms a parking section using an image acquired from an imaging device that captures the periphery of the host vehicle according to an aspect of the present disclosure includes an image acquisition unit that acquires an image from the imaging device; When a set of straight lines configured by a straight line detection unit that detects a plurality of straight lines from an image acquired by the image acquisition unit and two straight lines selected from the plurality of straight lines detected by the straight line detection unit satisfies a predetermined condition A parking frame candidate detection unit that detects the set of straight lines as a parking frame candidate, and a parking frame candidate in which the degree of orthogonality between the straight line forming the parking frame candidate and the travel route of the host vehicle is within a predetermined range. And a parking frame detector for detecting as a parking frame.

According to the above apparatus, it is possible to reduce erroneous detection of a road arrow or a pedestrian crossing as a parking frame, which is unlikely to be substantially orthogonal to the traveling route of the host vehicle. On the other hand, it is possible to more accurately detect a parking frame that forms a parking section that is highly likely to be substantially orthogonal to the travel route of the host vehicle.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. The drawing
FIG. 1 is a diagram illustrating a state in which an imaging device is attached to the host vehicle in the first embodiment of the present disclosure. FIG. 2 is a diagram for explaining the configuration of the in-vehicle image processing device in the first embodiment. FIG. 3 is a diagram for explaining how the in-vehicle image processing device detects the travel route of the host vehicle in the first embodiment. FIG. 4 is a diagram illustrating a state where the in-vehicle image processing device detects a parking frame in the first embodiment. FIG. 5 is a flowchart for explaining the flow of processing performed by the in-vehicle image processing device in the first embodiment. FIG. 6 is a flowchart for explaining a flow in which the vehicle-mounted image processing device detects parking frame candidates in the first embodiment. FIG. 7 is a flowchart for explaining a flow in which the vehicle-mounted image processing device detects a parking frame in the first embodiment. FIG. 8 is a flowchart for explaining a flow in which the detected parking frame is notified to the user in the first embodiment. FIG. 9 is a diagram illustrating a state in which the vehicle-mounted image processing device detects a parking frame in the second embodiment of the present disclosure. FIG. 10 is a diagram illustrating a surrounding environment of the host vehicle when the in-vehicle image processing device detects a parking frame at the current time in the third embodiment of the present disclosure. FIG. 11 is a diagram illustrating a state in which the vehicle-mounted image processing device detects a parking frame at a past time in the third embodiment. FIG. 12 is a diagram illustrating a state where the in-vehicle image processing device detects a parking frame at the current time in the third embodiment. FIG. 13 is a diagram illustrating a relationship between the width between straight lines and the probability of a parking frame in the fourth embodiment of the present disclosure; FIG. 14 is a diagram showing the relationship between the degree of parallelism between straight lines and the probability of parking frame in the fourth embodiment. FIG. 15 is a diagram illustrating a relationship between the degree of orthogonality and the probability of a parking frame in the fourth embodiment.

Hereinafter, a plurality of modes for carrying out the present disclosure will be described with reference to the drawings. In each embodiment, portions corresponding to the matters described in the preceding embodiment may be denoted by the same reference numerals and redundant description may be omitted. In each embodiment, when only a part of the configuration is described, the other configurations described above can be applied to other portions of the configuration.

(First embodiment)
As shown in FIG. 1, a rear camera 21 that is an imaging device in the present embodiment is installed at the rear of the host vehicle 1. Moreover, the rear camera 21 is installed in the center part of the vehicle left-right direction.

As shown in FIG. 2, the in-vehicle image processing apparatus in the present embodiment is an ECU 10.

The ECU 10 is connected to the rear camera 21, the vehicle speed sensor 23, the rudder angle sensor 24, and the display device 30.

A CCD camera is used as the rear camera 21 to photograph the scenery behind the host vehicle 1.

The vehicle speed sensor 23 is a sensor that detects the vehicle speed of the host vehicle 1. The steering angle sensor 24 is a sensor that detects the steering angle of the front wheels of the host vehicle 1.

The display device 30 has a display such as a liquid crystal display and a control unit. Therefore, the control unit receives an image signal from the ECU 10 and displays an image corresponding to the received image signal on the display.

The ECU 10 includes a microcomputer 11 and a memory 12. The ECU 10 acquires various types of information from the rear camera 21, the vehicle speed sensor 23, and the rudder angle sensor 24. And ECU10 performs arithmetic processing based on the acquired information. Details of the processing will be described later.

Further, the ECU 10 outputs an image signal to the display device 30. Then, the display device 30 displays an image based on the image signal received from the ECU 10.

The memory 12 is connected to the microcomputer 11. The memory 12 is composed of a non-volatile memory or the like, and can store a result of processing calculated by the microcomputer 11. The parking frame storage unit in the present embodiment corresponds to the memory 12.

The microcomputer 11 includes a CPU, a RAM, a ROM, and the like. The CPU performs various arithmetic processes based on programs stored in the RAM and ROM.

In the present embodiment, the microcomputer 11 serves as the image acquisition unit 110, the straight line detection unit 111, the parking frame candidate detection unit 112, and the parking frame detection unit 113.

The image acquisition unit 110 acquires images from the rear camera 21 at predetermined time intervals. The predetermined time interval can be set as appropriate. Further, the image acquisition unit 110 converts the acquired image into bird's-eye coordinates viewed from the bird's viewpoint, and generates a bird's-eye image.

The straight line detection unit 111 performs straight line detection by performing a Hough transform on the bird's-eye view image generated by the image acquisition unit 110. Specifically, the straight line detection unit 111 first performs edge detection on the bird's-eye view image generated by the image acquisition unit 110. The edge detection is to detect an edge point that is a point where the luminance changes sharply in the image. The straight line detection unit 111 satisfies the Hough transform equation ρ = x · cos θ + y · sin θ (ρ, θ) for the position coordinates (x, y) in the image for each of the detected plurality of edge points. Extract a pair. Here, the value θ of the Hough transform equation corresponds to the angle formed by the straight line passing through the position coordinates (x, y) with respect to the x axis, and the value ρ is from the origin to the straight line passing through the position coordinates (x, y). Corresponds to the length of the perpendicular. A straight line is detected based on (ρ, θ).

The parking frame candidate detection unit 112 selects any two straight lines from the plurality of straight lines acquired by the straight line detection unit 111. When the set of straight lines constituted by the two straight lines satisfies a predetermined condition, the parking frame candidate detection unit 112 detects the set of straight lines as a parking frame candidate. The predetermined condition in the present embodiment is whether or not the degree of parallelism between the angles formed by the two straight lines in the set of straight lines belongs to a predetermined range. A specific value in a predetermined range of the degree of parallelism is an angle of 0 ° to 45 ° or 135 ° to 180 °. The predetermined condition is whether or not the width between the straight lines in the set of straight lines is within a predetermined range. In the present embodiment, the specific width range is from 1.8 m to 3.0 m. And the parking frame candidate detection part 112 determines whether the said predetermined conditions are satisfy | filled about all the combinations of the some straight line which the straight line detection part 111 detected, and detects a parking frame candidate.

The parking frame detection unit 113 detects a parking frame from the parking frame candidates detected by the parking frame candidate detection unit 112. Specifically, the parking frame detection unit 113 selects an arbitrary parking frame candidate from the parking frame candidates detected by the parking frame candidate detection unit 112. And it is detected whether the orthogonality of the angle which the straight line which comprises the parking frame candidate detection part 112 and the driving | running route of the own vehicle 1 belongs in the predetermined range. When the parking frame detection unit 113 determines that the angle formed above belongs to a predetermined range, the parking frame detection unit 113 detects the parking frame candidate as a parking frame. In the present embodiment, a specific value in a predetermined range of the orthogonality is an angle of 45 ° to 135 °.

The parking area for parking the vehicle is often located on the side of the travel route of the host vehicle 1. And the parking area for one vehicle in the parking area is often divided as a parking frame by two straight lines that are substantially orthogonal to the traveling route of the vehicle. The ECU 10 in the present embodiment detects the parking frame more accurately in order to determine whether or not the orthogonality of the angle formed by the travel route of the host vehicle 1 and the straight line of the parking frame constituting the parking section belongs to a predetermined range. I can do it.

A method for determining the travel route of the host vehicle 1 in the present embodiment will be described. The parking frame detection unit 113 detects the vehicle speed of the host vehicle 1 at a certain timing T from the vehicle speed sensor 23. In addition, the parking frame detection unit 113 detects the steering angle of the host vehicle 1 at the time T from the steering angle sensor 24. The time T-1 that is the timing when the parking frame detection unit 113 performed the previous parking frame is already known. Therefore, the parking frame detection unit 113 can detect a travel route including the traveling direction and the amount of movement of the host vehicle 1 between a certain time T-1 and a certain time T. Then, what has a predetermined width in a direction orthogonal to the travel route becomes the travel route of the host vehicle 1 at the interval ΔT. This runway can be superimposed on an image acquired by the image acquisition unit 110 at time T, and an example thereof is shown in FIG.

FIG. 3 shows an image acquired by the image acquisition unit 110 at a certain time T.

The runway 70 indicates the runway of the host vehicle 1 between time T-1 and time T. A runway 71 indicates a runway of the host vehicle 1 between time T-2 and time T-1. A runway 72 indicates the runway of the host vehicle 1 between time T-3 and time T-2. A travel route 701 indicates the traveling direction and the amount of movement of the host vehicle 1 between time T-1 and time T. The predetermined width 702 is orthogonal to the travel route 701 and has a predetermined width.

The travel route 711 indicates the traveling direction and the amount of movement of the host vehicle 1 between time T-2 and time T-1. The predetermined width 712 is orthogonal to the travel route 711 and has a predetermined width.

The travel route 721 indicates the traveling direction and the amount of movement of the host vehicle 1 between time T-3 and time T-2. The predetermined width 722 is orthogonal to the travel route 721 and has a predetermined width.

Next, a specific state in which the ECU 10 detects the parking frame will be described with reference to FIG.

FIG. 4 shows a bird's-eye view image generated by the image acquisition unit 110 using an image taken by the rear camera 21 at time T, which is a certain timing. First, an arrow 40 exists in the bird's-eye view image. In addition, a crosswalk 50 exists in the bird's-eye view image. The bird's eye image includes a parking section 60 and a parking section 61. From these, the ECU 10 detects the parking frames constituting the parking section 60 and the parking section 61.

In addition, the bird's-eye view image includes information on the travel path 70 and the travel path 701 of the host vehicle 1 between time T-1 and time T. In addition, information on the travel path 71 and the travel path 711 of the host vehicle 1 between the time T-2 and the time T-1 which are past timings is included. In addition, information on the traveling path 72 and the traveling path 721 of the host vehicle 1 between the time T-3 and the time T-2, which are past timings, is included.

First, the straight line detection unit 111 detects a straight line from the generated bird's-eye view image using the method described above. Then, the straight line detection unit 111 detects a straight line 40 a and a straight line 40 b that are straight lines constituting the arrow 40. Further, the straight line detection unit 111 detects a straight line 50 a and a straight line 50 b that are straight lines constituting the pedestrian crossing 50. Further, the straight line detection unit 111 detects a straight line 60 a and a straight line 60 b that are straight lines constituting the parking section 60. Further, the straight line detection unit 111 detects the

straight lines

61 a and 61 b constituting the parking section 61. The straight line detection unit 111 also detects other straight lines from the image, but is omitted for simplification of description. Therefore, the straight line detection unit 111 detects eight straight lines from the image.

Note that the width between the

straight lines

60a and 60b is 2.5 m. The width between the

straight lines

61a and 61b is 2.5 m. The width between the

straight lines

50a and 50b is 2.0 m. The width between the straight line 40a and the straight line 40b is 2.0 m.

Next, how the parking frame candidate detection unit 112 detects parking frame candidates will be described.

First, the manner in which the parking frame candidate detection unit 112 detects a parking frame candidate will be described as an example in which the straight line 60a and the straight line 60b are a set of straight lines. At this time, the parking frame candidate detection unit 112 determines the degree of parallelism between the straight line 60a and the straight line 60b. Then, the parking frame candidate detection unit 112 determines whether or not the degree of parallelism of the angles formed by the straight line 60a and the straight line 60b belongs to a predetermined range. Then, the degree of parallelism of the angle formed by the straight line 60a and the straight line 60b belongs to a predetermined range. In addition, the parking frame candidate detection unit 112 determines whether or not the width between the straight line 60a and the straight line 60b belongs to a predetermined range. Then, since this width is 2.5 m, the parking frame candidate detection unit 112 determines that the width between the

straight lines

60a and 60b satisfies a predetermined range. As described above, since the predetermined condition is satisfied, the parking frame candidate detection unit 112 detects a straight line set of the straight line 60a and the straight line 60b as a parking frame candidate.

Next, how the parking frame candidate detection unit 112 is not detected as a parking frame candidate will be described as an example in which the straight line 40a and the straight line 60a are a set of straight lines. At this time, the parking frame candidate detection unit 112 determines the degree of parallelism between the straight line 40a and the straight line 60a. Then, the angle formed by the straight line 40a and the straight line 60a is substantially orthogonal, and the degree of parallelism does not belong within a predetermined range. Therefore, the set of the straight line 40a and the straight line 60a is not detected as a parking frame candidate.

Similarly, as a result of performing the above processing on other straight line combinations, four straight line groups are detected as parking frame candidates this time. The first is a set of

straight lines

40a and 40b. The second is a set of

straight lines

60a and 60b. The third is a set of

straight lines

61a and 61b. The fourth is a set of

straight lines

50a and 50b.

Next, a description will be given of how the ECU 10 in the present embodiment detects a parking frame constituting a parking section by describing some examples.

As a first example, a state is shown in which the parking frame detection unit 113 determines whether a parking frame candidate configured by a straight line 60a and a straight line 60b is a parking frame that constitutes a parking section. The parking frame detection unit 113 detects an angle formed by the travel route 701 of the host vehicle 1 and the straight line 60a. At this time, the formed angle is approximately 90 °, and the orthogonality of the formed angle belongs to a predetermined range of 45 ° to 135 °. Therefore, the parking frame detection part 113 detects the parking frame candidate comprised by the straight line 60a and the straight line 60b as a parking frame.

Further, the parking frame detection unit 113 detects whether the parking frame candidate constituted by the straight line 40a and the straight line 40b is a parking frame. The parking frame detection unit 113 detects an angle formed by the travel route 701 and the straight line 40a. Then, the angle formed is almost 0 ° and does not satisfy the predetermined range of 45 ° to 135 °. Therefore, the parking frame detection unit 113 does not detect a parking frame candidate constituted by the straight line 40a and the straight line 40b as a parking frame.

In addition, the parking frame detection unit 113 detects whether the parking frame candidate configured by the straight line 50a and the straight line 50b is a parking frame. The parking frame detection unit 113 detects an angle formed by the travel route 701 and the straight line 50a. Then, the angle formed is almost 0 ° and does not satisfy the predetermined range of 45 ° to 135 °. Therefore, the parking frame detection unit 113 does not detect a parking frame candidate constituted by the straight line 50a and the straight line 50b as a parking frame.

As described above, the ECU 10 according to the present embodiment determines whether or not the degree of orthogonality between the straight line constituting the detected parking frame candidate and the travel route 701 of the host vehicle 1 is within a predetermined range. . Therefore, it is possible to prevent erroneous detection of the arrow 40 or the pedestrian crossing 50 as a parking frame.

Note that the detection of the parking frame based on the travel route 711 and the travel route 721 has already been performed at a past timing. That is, the parking frame on the bird's-eye view image generated by the image acquisition unit 110 at time T-1 is detected based on whether or not the orthogonality of the angle formed by the travel route 711 and the parking frame candidate is within a predetermined range. The The parking frame on the bird's-eye view image generated by the image acquisition unit 110 at time T-2 is detected based on whether or not the orthogonality of the angle formed by the travel route 721 and the parking frame candidate is within a predetermined range. The

Next, the flow of processing performed by the ECU 10 in this embodiment will be described with reference to FIGS.

As shown in FIG. 5, in step S10, the ECU 10 detects a parking frame candidate. In step S11, the ECU 10 detects a parking frame from the detected parking frame candidates. In step S12, the ECU 10 notifies the user of the detected parking frame.

FIG. 6 shows a specific processing flow in which the ECU 10 detects parking frame candidates. First, in step S <b> 101, the image acquisition unit 110 acquires an image from the rear camera 21. In addition, the image acquisition unit 110 generates a bird's-eye image that is an image obtained by viewing the acquired image from the viewpoint of the bird. Thereafter, the process proceeds to step S102.

In step S102, the straight line detection unit 111 detects a straight line from the generated bird's-eye view image by the method described above. The detected straight line is stored in the memory 12. This step is performed until all straight lines are detected on the bird's-eye view image. Thereafter, the process proceeds to step S103.

In step S103, the parking frame candidate detection unit 112 selects any two straight lines from the straight lines detected by the straight line detection unit 111 and sets them as a set of straight lines. Thereafter, the process proceeds to step S104.

In step S104, the parking frame candidate detection unit 112 detects whether the set of straight lines satisfies a predetermined condition. Specifically, the parking frame candidate detection unit 112 determines whether or not the distance between the straight lines in the set belongs to a range of 1.8 m to 3.0 m. In addition, the parking frame candidate detection unit 112 determines whether the degree of parallelism between the straight lines of the set of straight lines belongs to a predetermined range of 0 ° to 45 ° or 135 ° to 180 °. If the set of straight lines satisfies a predetermined condition, the process proceeds to step S105, and if not, the process proceeds to step S106.

In step S105, the parking frame candidate detection unit 112 writes the selected straight line set in the memory 12 as a parking frame candidate. Then, it progresses to step S106. In step S106, the parking frame candidate detection part 112 detects whether it checked whether it was a parking frame candidate about all the combinations of the straight line memorize | stored in the memory 12. FIG. If all combinations have been determined, this process ends, otherwise the process returns to step S103.

FIG. 7 shows a specific processing flow in which the ECU 10 detects a parking frame from the detected parking frame candidates. In step S111, the parking frame detection unit 113 detects the travel route 701 of the host vehicle 1 by the method described above. Thereafter, the process proceeds to step S112.

In step S <b> 112, the parking frame detection unit 113 selects an arbitrary parking frame candidate stored in the memory 12. Thereafter, the process proceeds to step S113. In step S113, the parking frame detection unit 113 determines that the orthogonality between the angles formed by at least one of the straight lines constituting the parking frame candidate and the travel route 701 of the host vehicle 1 is within a predetermined range. Determine if it belongs. If it belongs to the predetermined range, the process proceeds to step S114. Otherwise, the process proceeds to step S115.

In step S114, the parking frame detection unit 113 writes the selected parking frame candidate in the memory 12 as a parking frame. Further, the parking frame detection unit 113 writes the time when the parking frame is detected in the memory 12. Thereafter, the process proceeds to step S115.

In step S115, the parking frame detection unit 113 determines whether or not the parking frame detection processing has been performed on all the parking frame candidates stored in the memory 12. A process is complete | finished when a process is performed with respect to all the parking frame candidates, and when that is not right, it returns to step S112.

FIG. 8 is a flowchart showing how the parking frame detected by the parking frame detection unit 113 is notified to the user.

In step S121, the ECU 10 acquires a parking frame from the memory 12. And the image which shows the surrounding image of the own vehicle 1 is produced | generated in the state which highlighted the acquired parking frame. Specifically, the parking frame is surrounded by colored markers. Then, the ECU 10 transmits the image to the display device 30. Then, the display device 30 displays the image. Thereafter, the process proceeds to step S122.

In step S122, the user selects one parking frame among the parking frames highlighted in the image. At that time, the ECU 10 receives information on the parking frame selected by the user from the display device 30. Thereafter, the process proceeds to step S123.

In step S123, the ECU 10 generates an image in which only the parking frame selected by the user is highlighted, and transmits the image to the display device 30. Then, the display device 30 displays an image in which only the parking frame selected by the user is highlighted. Thereafter, the process ends.

Hereinafter, the effect of the ECU 10 in the first embodiment will be described.

The ECU 10 in the present embodiment is an ECU 10 that detects a parking frame that forms a parking section using an image acquired from a rear camera 21 that captures the periphery of the host vehicle 1. The ECU 10 includes an image acquisition unit 110 that acquires an image from the rear camera 21. In addition, the ECU 10 includes a straight line detection unit 111 that detects a straight line from the image acquired by the image acquisition unit 110. In addition, when a set of straight lines configured by two straight lines selected from a plurality of straight lines detected by the straight line detection unit 111 satisfies a predetermined condition, the ECU 10 detects the set of straight lines as a parking frame candidate. A candidate detection unit 112 is provided. Further, the ECU 10 includes a parking frame detection unit 113 that detects, as a parking frame, a parking frame candidate in which the degree of orthogonality between the straight line forming the parking frame candidate and the travel route 701 of the host vehicle 1 is within a predetermined range. I have.

For this reason, it is possible to reduce erroneous detection of a road arrow 40 or a pedestrian crossing 50 as a parking frame, which is unlikely to be orthogonal to the travel route 701 of the host vehicle 1. On the other hand, the parking frame which comprises the parking area with high possibility of orthogonally crossing with respect to the traveling route 701 of the own vehicle 1 can be detected more accurately.

(Second Embodiment)
In 1st Embodiment, the parking frame detection part 113 is detecting the parking frame based on whether the orthogonality of the angle | corner which the driving | running route of the own vehicle 1 and the parking frame candidate belong in the predetermined range. It was.

The parking frame detection unit 113 according to the second embodiment is positioned on the past or current traveling path of the host vehicle 1 even if the parking frame candidate has an orthogonality with respect to the traveling route of the host vehicle 1 within a predetermined range. A parking frame candidate that is present is prohibited from being detected as a parking frame.

In addition, since the structure of ECU10 in this embodiment is the same as that of 1st Embodiment, description is abbreviate | omitted.

FIG. 9 shows a bird's-eye view image generated by the image acquisition unit 110 according to the present embodiment at a certain timing time T.

The difference from the first embodiment is that a straight line 501 and a straight line 502 exist in the longitudinal direction of the pedestrian crossing 50. The width of the straight lines 501 and 502 is 2.5 m.

In this situation, the ECU 10 in the present embodiment performs the following processing.

First, the straight line detection unit 111 detects a straight line 501 and a straight line 502. And the parking frame candidate detection part 112 detects the group of the straight line comprised by the straight line 501 and the straight line 502 as a parking frame candidate.

Next, the parking frame detection unit 113 determines whether or not the orthogonality of the angle formed by the parking frame candidate configured by the straight line 501 and the straight line 502 and the travel route 701 of the host vehicle 1 belongs to a predetermined range. Determine.

Specifically, the parking frame detection unit 113 determines whether or not the orthogonality of the angle formed by the straight line 501 and the travel route 701 of the host vehicle 1 belongs to 45 ° to 135 °. Then, since the formed angle is 90 °, it belongs to a predetermined range. Therefore, at this time, the parking frame detection unit 113 may detect a parking frame candidate constituted by the straight line 501 and the straight line 502 as a parking frame.

The parking frame detection unit 113 of the present embodiment further detects whether or not the straight line 501 is located on the past or current runway of the host vehicle 1. That is, the parking frame detection unit 113 determines whether the straight line 501 is located on the runway 70, the runway 71, or the runway 72. When the straight line 501 is located on any one of the roads, the parking frame detection unit 113 prohibits detection of a parking frame candidate configured by the straight line 501 and the straight line 502 as a parking frame. In this case, the straight line 501 is located on the runway 72. Therefore, the parking frame detection unit 113 prohibits detection of a parking frame candidate configured by the straight line 501 and the straight line 502 as a parking frame.

As described above, the parking frame detection unit 113 in the present embodiment prohibits detection of a parking frame candidate positioned on the traveling path of the host vehicle 1 as a parking frame.

Like the pedestrian crossing 50, parking frame candidates that are substantially orthogonal to the travel route of the host vehicle 1 may appear on the travel route of the host vehicle 1. However, it is almost impossible that the parking frame which comprises a parking area will appear on the runway of the own vehicle 1.

Even if a parking frame candidate orthogonal to the traveling route of the host vehicle 1 is detected on the traveling path of the host vehicle 1, the ECU 10 of the present embodiment erroneously detects such a parking frame candidate as a parking frame. Can be prevented.

Hereinafter, effects of the ECU 10 in the second embodiment will be described.

Parking frame detection unit 113 prohibits detection of parking frame candidates located on a running road having a predetermined width with respect to the travel route as parking frames among the parking frame candidates.

The parking section rarely appears on the track of the vehicle 1. Even if a parking frame candidate formed by an angle formed with the traveling route of the host vehicle 1 is detected on the traveling path of the host vehicle 1, the ECU 10 according to the present embodiment uses the parking frame candidate as a parking frame due to the above characteristic configuration. It is possible to prevent erroneous detection.

(Third embodiment)
When detecting the parking frame at a certain timing, the ECU 10 according to the third embodiment refers to the parking frame detected at a timing earlier than the certain timing. And when the parking frame candidate currently existing on the image and the parking frame detected in the past acquired from the memory 12 correspond, the parking frame candidate is detected as a parking frame.

Whether or not the current parking frame candidate corresponds to the parking frame acquired from the memory 12 is detected by performing the following process.

First, the parking frame detection unit 113 acquires the parking frame from the memory 12 and the position coordinates on the image when detected in the past. And the driving | running route which the own vehicle 1 drive | worked is detected between the present timing and the past timing when the parking frame was detected. And the parking frame detection part 113 can pinpoint the position coordinate where the parking frame should exist in the present image based on the past position coordinate of the parking frame acquired from the memory 12, and the said driving | running route. .

If there is a parking frame candidate at the specified position coordinates, the parking frame detection unit 113 detects the parking frame candidate as a parking frame.

Specific examples will be described with reference to FIG. 10, FIG. 11, and FIG.

The own vehicle 1 indicated by the broken line in FIG. 10 indicates the own vehicle 1 at time T-1. A solid vehicle 1 represents the vehicle 1 at time T. The own vehicle 1 travels like a travel route 701 between time T-1 and time T.

FIG. 11 shows a bird's-eye view image generated by the image acquisition unit 110 at time T-1. FIG. 12 shows a bird's-eye view image generated by the image acquisition unit 110 at time T.

First, as shown in FIG. 11, at time T-1, the parking frame detection unit 113 detects a pair of a straight line 61a and a straight line 61b as a parking frame. The parking frame is stored in the memory 12.

Next, as shown in FIG. 12, at time T, the parking frame candidate detection unit 112 detects a pair of the straight line 61a and the straight line 61b as a parking frame candidate. And the parking frame detection part 113 determines whether the parking frame candidate is a parking frame. Then, since the orthogonality between the parking frame candidate and the current travel route 701 of the host vehicle 1 does not belong to a predetermined range, the parking frame detection unit 113 does not detect the parking frame candidate as a parking frame.

The parking frame detection unit 113 of the present embodiment further performs the following processing. First, the parking frame detection unit 113 detects a parking frame stored in the memory 12. That is, the parking frame detection unit 113 detects a pair of the straight line 61a and the straight line 61b. Next, the parking frame detection unit 113 detects a travel route from the time T-1 when the pair of the straight line 61a and the straight line 61b is detected as a parking frame to the current time T. That is, in this case, the travel route 701. Then, the parking frame detection unit 113 specifies the position coordinates where the current parking frame should exist based on the position coordinates where the parking frame was detected and the travel route 701 at the past time T-1.

And the parking frame detection part 113 detects the parking frame candidate which corresponds to the position coordinate where the parking frame specified by said method should exist among the parking frame candidates detected at the present time T as a parking frame. To do. That is, the pair of

straight lines

61a and 61b detected as a parking frame at time T-1 is also detected as a parking frame at time T.

As described above, the parking frame detection unit 113 according to the present embodiment does not satisfy the predetermined range of the orthogonality between the parking frame candidate and the current travel route for the parking frame candidate once detected as a parking frame in the past. However, it can be detected as a parking frame.

Hereinafter, effects of the ECU 10 in the third embodiment will be described.

The travel route is a travel route on which the host vehicle has traveled from the timing when the parking frame detection unit 113 previously detected the parking frame to the present, and the ECU 10 detects the parking frame detected by the parking frame detection unit 113 in the past. A memory 12 for storing is provided. The parking frame detection unit 113 detects a parking frame candidate corresponding to the parking frame stored in the memory 12 as a parking frame in addition to the parking frame candidate whose orthogonality is within a predetermined range.

When the traveling direction of the host vehicle 1 changes, the travel route of the host vehicle 1 changes. Therefore, there is a possibility that a parking frame candidate detected as a parking frame at a certain timing may not be detected as a parking frame because the orthogonality of the angle formed with the travel route of the host vehicle 1 does not belong within a predetermined range at the next timing. There is. The ECU 10 in this embodiment can detect a parking frame candidate once detected as a parking frame as a parking frame even if the orthogonality of the angle formed with respect to the travel route does not belong within a predetermined range thereafter.

(Fourth embodiment)
In the above-described embodiment, a straight line set that satisfies a predetermined width and whose parallel degree falls within a predetermined range is detected as a parking frame candidate. And among the parking frame candidates, all the ones in which the orthogonality between at least one straight line and the running road belong within a predetermined range were treated as the same parking frame.

In this embodiment, in order to determine whether or not the possibility of a parking frame is higher among the detected parking frames, a probability of a parking frame is set in the detected parking frame.

Specifically, as shown in FIGS. 13 to 15, probabilities are set for each parameter.

First, FIG. 13 shows the relationship between the width of straight lines of the above-mentioned straight line set and the probability of parking frame. When the width between the straight lines is in the range of 1.8 m to 3.0 m, the probability of the parking frame is set to 1. And the probability of a parking frame is set to 0 for 1.5 m or less and 3.5 m or more. When the frame width is x1 and the probability of parking frame likelihood is A, the probability A within the range of 1.5 m to 1.8 m is a predetermined formula, for example, A = (1 / 0.3) × ( x1-1.5). Then, the probability A in the range of 3.0 m to 3.5 m is obtained by a predetermined expression, for example, A = 2 × (x1−3.0).

FIG. 14 shows the relationship between the degree of parallelism between the straight lines of the set of straight lines and the probability of parking frame. When the parallel degree is x2 and the probability of the parking frame is B, the probability B in the parallel degree range of 0 ° to 45 ° is a predetermined formula, for example, B = (1/45) × (45−x2) Is required. And in the range of the parallel degree 45 ° to 135 °, the probability B of the parking frame is 0. Then, the probability B of the parking frame in the parallel degree range of 135 ° to 180 ° is obtained by a predetermined formula, for example, B = (1/45) × (x2−135).

FIG. 15 shows the relationship between the degree of orthogonality between at least one straight line of the set of straight lines and the travel route 701 of the host vehicle 1 and the probability of a parking frame. If the orthogonality is x3 and the probability of the parking frame is C, the probability of the parking frame C = 0 within the range of the orthogonality from 0 ° to 45 °. Then, within the range of the orthogonality from 45 ° to 90 °, the probability C of the parking frame likelihood is obtained by a predetermined formula, for example, C = (1/45) × (x3-45). Within the range of the orthogonality from 90 ° to 135 °, the probability C of the parking frame likelihood is obtained by a predetermined formula, for example, C = (1/45) × (135−x3).

Then, the probabilities A, B, and C are integrated based on the following calculation formula, and the final parking frame-like integration probability E is detected.

The probability integration formula that integrates the two probabilities is shown below. First, assuming that the integration probability between the probabilities A and B is D, the integration probability D is a predetermined formula, for example, D = (A × B) / (A × B + (1−A) × (1−B). )). Then, when the probability integration arithmetic expression is similarly applied to the integration probability D and the probability C of orthogonality, and the integration probability of D and C is E, the total probability E is a predetermined expression, for example, E = (D × C) / (D × C + (1-D) × (1-C)). This integration probability E is the integration probability of A, B, and C.

By using the probability integration calculation formula, the parking frame detection unit 113 can determine which of the parking frame candidates is more likely to be a parking frame, and use the calculated probability of parking frame, It can be determined whether the probability of likelihood exceeds a threshold value. For example, the parking frame detection unit 113 can detect a parking frame candidate that is more likely to be a parking frame among those having a parking frame likelihood of 50% or more.

If a simple probability multiplication is performed, the parking frame detection unit 113 reduces the probability of the final parking frame as the parking frame probability is multiplied. That is, it can be determined only whether the parking frame candidates are more likely to be parking frames.

A specific example is shown. The frame width between the straight lines is 2.3 m, the degree of parallelism is 0 °, and the probability of each of the straight line sets in which the orthogonality between the straight lines and the runway is 90 ° is A = 1, B = 1 , C = 1. Therefore, the integration probability E = 1.

When the frame width between the straight lines is 1.6 m and the degree of parallelism is 30 °, and the orthogonality between the straight lines and the runway is 70 °, the probability of each of the straight line sets is A = 1 / 3, B = 3/7, C = 5/9, and the integration probability E = 15/47. And among the detected parking frames, the one with the higher integration probability E is more likely to be a parking frame.

The effects of the ECU 10 in the fourth embodiment will be described below.

The predetermined condition is that the length of the width between the straight lines belongs to a predetermined range, and the degree of parallelism of the angles formed by the straight lines belongs to the predetermined range. And the parking frame detection part 113 is set based on the probability of the parking frame set based on the length of the width, the probability of the parking frame set based on the degree of parallelism, and the orthogonality. A parking frame is detected based on the probability that the probability of the parking frame is integrated.

Thus, the ECU 10 of the present embodiment sets a probability indicating the possibility of the parking frame for the parking frame to be detected. For this reason, ECU10 in this embodiment can detect the parking frame with higher possibility of being the parking frame which comprises a parking area.

(Other embodiments)
The preferred embodiments of the disclosure have been described above, but the disclosure is not limited to the above-described embodiments, and various modifications can be made as illustrated below. Not only combinations of parts that clearly show that combinations are possible in each embodiment, but also combinations of the embodiments even if they are not explicitly stated unless there is a problem with the combination. Is also possible.

In the first embodiment, the determination of the degree of orthogonality with the parking frame candidate is performed only for the current travel route, but a comparison with a past travel route may also be performed. Specifically, in the first embodiment, it is determined whether or not the orthogonality of the angle between the parking frame candidate and the travel route 711 in the image acquired by the image acquisition unit 110 at time T belongs to a predetermined range. It may be. Furthermore, you may make it determine whether the orthogonality of the angle | corner which the parking frame candidate and the travel route 721 in the image which the image acquisition part 110 acquired at the time T belongs to a predetermined range.

In the third embodiment, the ECU 10 detects the parking frame based on the orthogonality between the traveling route and the parking frame candidate in both the straight traveling route and the traveling route that changes the rudder angle. It was. The present disclosure is not limited to this, and the parking frame is detected based on the orthogonality between the travel route and the parking frame candidate when the host vehicle 1 is traveling straight, and the host vehicle 1 is not traveling straight. The parking frame may not be detected based on the travel route. Most of the situations where the parking frame needs to be searched are when the host vehicle 1 is traveling straight. On the other hand, in a state where the steering angle of the host vehicle 1 is turned off, there is almost no situation where it is necessary to search for a parking frame. Therefore, the straight line of the parking frame candidate whose orthogonality satisfies a predetermined range with respect to the travel route in a state where the steering angle of the host vehicle 1 is turned off is highly likely not to be a parking frame forming a parking section. . Therefore, with the above configuration, it is possible to prevent the ECU 10 from erroneously detecting the parking frame when the steering angle of the host vehicle 1 is turned off.

In the above embodiment, the degree of parallelism between the straight lines and the width between the straight lines are used as the predetermined conditions when the parking frame candidate is detected. However, the present invention is not limited to this.

For example, only one of the degree of parallelism between the straight lines or the width between the straight lines may be adopted to detect the parking frame candidate.

For example, it may be detected whether another straight line exists in the vicinity of one of the two straight lines.

Also, the degree of coincidence between two straight lines may be compared. Specifically, parameters relating to the degree of coincidence between straight lines include color, brightness, length, thickness, shape, and contrast. At least one of the above parameters may be adopted. Furthermore, for each parameter, the parking frame-likeness as shown in the fourth embodiment is set, and in addition to the parking frame-likeness of the width, parallelism and orthogonality between the straight lines, the parking of each parameter is set. The final likelihood of parking frame may be calculated based on the likelihood of frame.

That is, the predetermined condition is that the brightness of the straight lines is the same, the contrast is the same, the length is the same, the thickness is the same, the colors are the same, and the widths of the straight lines are in a predetermined range. Satisfy at least one of the continuous appearance of a set of straight lines belonging to the inside.

The parking frame detection unit is a parking frame set based on a probability of a parking frame set based on the degree of coincidence of brightness, a probability of a parking frame set based on a degree of coincidence of contrast, and a degree of coincidence of length. Probability of frameness, probability of parking frame that is set based on matching degree of thickness, probability of parking frame that is set based on matching degree of color, and the length of the width between straight lines are within a predetermined range A parking frame may be detected based on a probability obtained by integrating at least one probability of a parking frame among the probability of a parking frame set based on the degree to which a set of straight lines belonging to the inside is continuous.

Further, the parking frame may be detected in consideration of whether a plurality of straight line sets having the same width and color between the straight lines appear continuously in a predetermined direction. Furthermore, it may be possible to set the likelihood of a parking frame and use it for the final calculation of the likelihood of a parking frame as to whether or not straight lines having the same width and color are continuous.

In addition, the parking frame detection unit 113 performs a process of determining whether the parking frame candidate is a pedestrian crossing based on the width or parallelism of the straight lines, and the parking frame candidate determined to be a pedestrian crossing is a parking frame. You may make it not detect.

Or the parking frame detection part 113 performs the process which determines whether a parking frame candidate is a character based on the width | variety of the said straight lines, parallelism, etc., and the parking frame candidate determined to be a character is not detected as a parking frame. You may do it.

Further, the method for detecting the travel route of the host vehicle 1 is not limited to the above embodiment. For example, the position of the host vehicle 1 at each timing may be specified based on GPS, and the travel route may be obtained using the difference. Further, the travel route of the host vehicle 1 may be calculated based on the coordinate change amount of an arbitrary edge point between each timing.

In the above embodiment, the predetermined range of the orthogonality is 45 ° to 135 °. However, the present invention is not limited to this and can be set as appropriate. For example, the angle may be 80 ° to 120 °. Similarly, a predetermined range of the degree of parallelism and the length of the width can be set as appropriate.

In the above embodiment, the travel route includes both the movement direction and the movement amount, but may include information only on the movement direction.

The parking frame detection unit 113 in the above embodiment detects the parking frame every time the image acquisition unit 110 generates a bird's-eye view image. However, after the image acquisition unit 110 generates the bird's-eye image a predetermined number of times, parking is performed. A frame may be detected.

In the above embodiment, the parking frame storage unit is the memory 12 provided in the ECU 10, but may be a storage medium of a device different from the ECU 10.

In the above embodiment, the imaging device is the rear camera 21, but a plurality of cameras may be attached to the host vehicle 1, and the image acquisition unit 110 may generate a bird's-eye view image from the plurality of cameras.

In the above embodiment, the straight line detection unit 111 detects a straight line for the bird's-eye view image generated by the image acquisition unit 110. However, the present invention is not limited to this, and the straight line detection unit 111 may perform straight line detection on an image obtained by performing distortion processing on the image acquired by the image acquisition unit 110 from the rear camera 21.

The flowchart described in the present disclosure or the processing of the flowchart is configured by a plurality of units (or referred to as steps), and each unit is expressed as S10, for example. Furthermore, each part can be divided into a plurality of sub-parts, while the plurality of parts can be combined into one part. Furthermore, each part configured in this manner can be referred to as a circuit, a device, a module, and a means.

Each of the above-mentioned plurality of parts or a combination thereof is not only (i) a software part combined with a hardware unit (for example, a computer), but also (ii) hardware (for example, an integrated circuit, As a part of the (wiring logic circuit), it can be realized with or without including the functions of related devices. Furthermore, the hardware unit can be configured inside a microcomputer.

Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments and structures. The present disclosure includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more or less, are within the scope and spirit of the present disclosure.

Claims

An in-vehicle image processing device that detects a parking frame that constitutes a parking section using an image acquired from an imaging device that captures the periphery of the host vehicle,
An image acquisition unit (110) for acquiring an image from the imaging device;
A line detection unit (111) for detecting a plurality of lines from the image acquired by the image acquisition unit;
A parking frame candidate detection unit that detects a set of straight lines as a parking frame candidate when the set of straight lines configured by two straight lines selected from the plurality of straight lines detected by the straight line detection unit satisfies a predetermined condition. (112),
A parking frame detection unit (113) for detecting, as the parking frame, the parking frame candidate in which an orthogonality between angles formed by the straight line constituting the parking frame candidate and the travel route of the host vehicle is within a predetermined range; A vehicle-mounted image processing apparatus comprising:
The said parking frame detection part prohibits detecting the said parking frame candidate located on the runway which has a predetermined width with respect to the said driving | running route among the said parking frame candidates as the said parking frame. The on-vehicle image processing apparatus described.
The travel route is a travel route traveled by the host vehicle from the timing when the parking frame detection unit previously detected the parking frame to the present time,
A parking frame storage unit (12) for storing the parking frame detected in the past by the parking frame detection unit;
The parking frame detection unit uses the parking frame candidate corresponding to the parking frame stored in the parking frame storage unit as the parking frame in addition to the parking frame candidate whose orthogonality is within a predetermined range. The in-vehicle image processing apparatus according to claim 1, wherein the on-vehicle image processing apparatus is detected.
The predetermined condition is that the length of the width between the straight lines belongs to a predetermined range and that the degree of parallelism of the angles formed by the straight lines belongs to the predetermined range,
The parking frame detection unit is set based on the probability of the parking frame set based on the length of the width, the probability of the parking frame set based on the degree of parallelism, and the orthogonality. The in-vehicle image processing device according to any one of claims 1 to 3, wherein the parking frame is detected based on a probability obtained by integrating a probability of a parking frame.
The predetermined conditions further include: the brightness of the straight lines matches, the contrast matches, the length matches, the thickness matches, the colors match, and the widths of the lines match each other. Satisfying at least one of a set of straight lines belonging to the range of
The parking frame detection unit further determines the probability of parking frame set based on the degree of coincidence of luminance, the probability of parking frame set based on the degree of coincidence of contrast, and the degree of coincidence of length. The probability of the parking frame set based on the probability of the parking frame set based on the matching degree of the thickness, the probability of the parking frame set based on the matching degree of the color, and the straight lines Based on the probability of integrating at least one probability of parking frame out of the probability of parking frame that is set based on the degree to which a set of straight lines belonging to a predetermined range of the length of the width is continuous, The in-vehicle image processing apparatus according to claim 4, wherein the parking frame is detected.