WO2018088263A1

WO2018088263A1 - Parking frame recognition device

Info

Publication number: WO2018088263A1
Application number: PCT/JP2017/039141
Authority: WO
Inventors: 大輔杉浦; 博彦柳川
Original assignee: 株式会社デンソー
Priority date: 2016-11-10
Filing date: 2017-10-30
Publication date: 2018-05-17
Also published as: JP6677142B2; JP2018078479A

Abstract

This parking frame recognition device (20) is provided with: an image acquisition unit (S110); a line extraction unit (S120, S150); and a possible frame recognition unit (S210, S220). The image acquisition unit is configured so as to acquire a plurality of captured images respectively captured at different positions by image capturing devices provided to a host vehicle. The line extraction unit is configured so as to extract a parking frame from the plurality of captured images, and extract a line which defines an end of the parking frame at the side furthest from the host vehicle. The possible frame recognition unit is configured so as to determine whether the shape of the line is the same for the same parking frame extracted from the plurality of captured images, and recognize, as a frame in which parking is possible, a parking frame for which the shape of the line has been determined to be the same.

Description

Parking frame recognition device

Cross-reference of related applications

This international application claims priority based on Japanese Patent Application No. 2016-219646 filed with the Japan Patent Office on November 10, 2016, and is based on Japanese Patent Application No. 2016-219646. The entire contents are incorporated herein by reference.

This disclosure relates to a technique for recognizing a parking frame that represents a parking frame in which the host vehicle can park.

In the following Patent Document 1, as a technique for recognizing a parking frame, a white line farther to the host vehicle than a white line closer to the host vehicle out of the white lines constituting the parking frame that can be observed from the host vehicle. A technique has been proposed for determining that the frame is a parking frame when it is long.

JP 2016-016681 A

As a result of the detailed examination by the inventor, in the technique of Patent Document 1 described above, when a vehicle with a short vehicle length such as a light vehicle is parked in the parking frame, it is farther from the own vehicle than the white line on the side closer to the own vehicle. Since the white line on the side may be observed to be long, a problem has been found that it is easy to misrecognize that a parking frame in which another vehicle is parked is a parking frame.

One aspect of the present disclosure is to provide a technique that allows a parking frame recognition device that recognizes a parking frame representing a parking frame in which the host vehicle can be parked to recognize the parking frame more accurately.

The parking frame recognition device according to one aspect of the present disclosure includes an image acquisition unit, a line extraction unit, and a possible frame recognition unit. The image acquisition unit is configured to acquire a plurality of captured images representing respective captured images captured at different positions by an imaging device arranged in the host vehicle.

The line extraction unit is configured to extract a parking frame from a plurality of captured images and extract a target line representing a line that defines an end of the parking frame farther from the vehicle. The possible frame recognizing unit determines whether or not the shape of the target line is the same for the same parking frame extracted from a plurality of captured images, and can park the parking frame determined to have the same shape of the target line It is configured to be recognized as a frame.

That is, in the parking frame recognition device according to the present disclosure, when a parked vehicle is present in the parking frame, the shape of the same target line captured at different positions is different because the target line is hidden by the parked vehicle. Take advantage of the property of being observed. The shape of the target line can be determined by how much the comparison parameters such as the number of line corners, line length, aspect ratio, and edge curvature match. The degree of matching can be determined, for example, by comparing the comparison parameter ratio with a threshold value.

According to such a parking frame recognizing device, since it is determined whether or not it is a parkingable frame depending on whether or not the shape of the same target line imaged at different positions is the same, the parkingable frame is accurately determined. Can be recognized.

In addition, the code | symbol in the parenthesis described in the claim shows the correspondence with the specific means as described in embodiment mentioned later as one aspect, Comprising: It does not limit the technical scope of this indication. Absent.

It is a block diagram which shows the structure of an imaging system. It is a flowchart which shows a parking possible frame recognition process. It is an image figure which shows an example of the captured image with the front camera in the present location. It is an image figure which shows an example of the captured image from the front camera in the near side. It is a bird's-eye view which shows an example of how the target line looks at the near side. It is a bird's-eye view which shows an example of how the target line looks at the present value.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
[1. Embodiment]
[1-1. Constitution]
As shown in FIG. 1, the imaging system 1 of this embodiment is a system mounted on a vehicle such as a passenger car, and includes a control unit 10. Moreover, you may provide the front camera 4F, the rear camera 4B, the display part 30, the vehicle control part 32, etc. A vehicle on which the imaging system 1 is mounted is also referred to as a host vehicle.

The front camera 4F and the rear camera 4B are for imaging roads ahead and behind the host vehicle, respectively, and are attached to the front and rear of the host vehicle.
The control unit 10 generates a bird's-eye image obtained by bird's-eye view of the road around the vehicle from the images captured by the

cameras

4F and 4B. Then, the generated bird's-eye view image is displayed on the display unit 30 configured with a liquid crystal display or the like and disposed in the vehicle interior.

Also, the control unit 10 recognizes a parking frame from images captured by the

cameras

4F and 4B. A parking frame means an object having at least one line having a width within a preset range and a side or line parallel to the line at a position separated from the line by a distance corresponding to the width of the vehicle. Represents the area between.

Also, the parking frame means a parking frame where the own vehicle can be parked. In addition, the term “parallel” includes substantially parallel. Further, the “object having a side or line parallel to the line” referred to here includes, for example, a curbstone, another line, a wall, a tree, a guardrail, and the like.

The control unit 10 includes an imaging signal input unit 12, a detection signal input unit 14, a memory 16, a display control unit 18, and an image processing unit 20.
The imaging signal input unit 12 has a function of capturing imaging signals from the front camera 4F and the rear camera 4B and inputting them to the image processing unit 20 as captured image data.

The detection signal input unit 14 takes in detection signals from the wheel speed sensor 6 that detects the rotational speed of each wheel of the host vehicle and the steering angle sensor 8 that detects the steering angle of the steering, respectively, and wheel speed data and steering angle data. And a function of inputting to the image processing unit 20.

The image processing unit 20 is mainly configured by a known microcomputer having a CPU 18 and a semiconductor memory (hereinafter, memory 16) such as a RAM, a ROM, and a flash memory. Various functions of the image processing unit 20 are realized by the CPU 18 executing a program stored in a non-transitional physical recording medium. In this example, the memory 16 corresponds to a non-transitional tangible recording medium that stores a program.

Also, when this program is executed, a method corresponding to the program is executed. Note that the non-transitional tangible recording medium means that the electromagnetic waves in the recording medium are excluded. Further, the number of microcomputers constituting the image processing unit 20 may be one or plural.

As shown in FIG. 1, the image processing unit 20 includes a line detection unit 22, a position correspondence unit 24, a frame estimation unit 26, and a tracking unit 28 as functional configurations realized by the CPU 18 executing a program. And comprising. The method of realizing these elements constituting the image processing unit 20 is not limited to software, and some or all of the elements may be realized using one or a plurality of hardware. For example, when the above function is realized by an electronic circuit that is hardware, the electronic circuit may be realized by a digital circuit including a large number of logic circuits, an analog circuit, or a combination thereof.

The function as the line detection unit 22 performs a process of detecting a line such as a white line or a yellow line by performing a known image process such as a Hough transform on each captured image.
In the function as the position correspondence unit 24, it is determined for each detected line whether or not the lines existing in the imaging range common to other cameras are the same line, and the process of associating the same line is performed. Note that correspondence information in which an imaging range common to other cameras and corresponding coordinates are set for each camera is stored in the memory 16, and it is determined whether or not the lines are the same using the correspondence information.

The function as the frame estimation unit 26 performs a process of estimating one or a plurality of parking frames present in the captured image and estimating a parking available frame from these parking frames.
The function as the tracking unit 28 performs processing for tracking an object in the captured image by associating the amount of movement of the host vehicle with the amount of movement of the object in the captured image. In particular, in this embodiment, the parking frame is tracked and the position of the parking frame is recognized. Information on the position of the parking frame is sent to the vehicle control unit 32.

Also, the function as the tracking unit 28 generates an image showing a parking frame while taking into account the amount of movement of the host vehicle, and this image is output to the display control unit 18. The display control unit 18 converts the image sent from the image processing unit into a video signal that can be displayed on the display unit 30, and sends the video signal to the display unit 30.

The vehicle control unit 32 receives the position of the parking frame, generates a track of the host vehicle for parking in the parking frame, and moves the host vehicle according to the track. Control the rudder angle, etc.

[1-2. processing]
Next, the parkable frame recognition process executed by the image processing unit 20 will be described with reference to the flowchart of FIG. The parking-permitted frame recognition process is a process that is started, for example, when the vehicle is turned on and then repeatedly executed. The parkingable frame recognition process may be activated every imaging cycle of the camera.

In the parking frame recognition process, as shown in FIG. 2, first, in S110, an image captured by the camera is acquired. In this process, each captured image captured at a different position by the front camera 4F arranged in the host vehicle is acquired.

For example, a captured image captured by the front camera 4F at the current position of the host vehicle and a captured image captured in front of a preset distance are acquired. As a captured image captured in front, for example, an image captured in front of the current location about several meters may be acquired. For example, a captured image as illustrated in FIG. 3 is obtained as the captured image captured in the foreground, and a captured image as illustrated in FIG. 4 is acquired as the captured image captured at the current position.

Further, if the host vehicle is moving, an image captured before a preset time from the present time may be acquired as a captured image captured in front. In this case, the preset time may be set to about several seconds, for example.

Subsequently, in S120, a line such as a white line or a yellow line is detected for each acquired captured image. In this process, the edges that are the boundaries of brightness and color are detected in a large number of pixels that make up the captured image, and processing such as well-known Hough transform is performed on the edges, so that all existing in the captured image are detected. Detect the line. The line here has a width and includes, for example, paint on the road. The detected line is called a detection line. In the example shown in FIGS. 3 and 4, all the

lines

41, 42, and 43 among the white lines 40 indicating all the parking frames are detection lines.

Subsequently, in S130, it is determined whether or not there are a plurality of detection lines in each captured image. In this process, since an area sandwiched between two lines is recognized as a parking frame, it is recognized that there is no parking frame when two or more lines are not detected. Therefore, in S130, it can be said that it is determined whether there is a possibility that a parking frame exists.

If there are not a plurality of detection lines, the fact that there is no usable parking frame is recorded in the memory 16 in S240, and the parking frame recognition process is terminated. If there are a plurality of detection lines, the entire captured image obtained in S140 or at least the detection lines of the captured image is converted into a planar coordinate system. This process may be realized by a well-known process for generating a bird's-eye image obtained by looking down on a captured image from the vertical direction.

Subsequently, the parking frame is estimated at S150. In this process, a parking frame is extracted from a plurality of captured images. The parking frame is an area that is sandwiched between at least two lines, and an area in which the interval between the two lines is within a preset range that is set based on the width of the vehicle is extracted. At this time, all the parking frames present in the captured image are extracted. In the example shown in FIGS. 3 and 4, the area between the

white lines

41 and 42 and the area between the

white lines

42 and 43 are extracted as parking frames. Although the area between the

white lines

41 and 43 can also be extracted as a parking frame, in this process, only the area where the distance between the white lines is within a preset range is recognized as a parking frame. The area is excluded from the parking frame.

In addition, when extracting a parking frame, you may extract only a specific parking frame, such as the parking frame nearest to the own vehicle and the parking frame located only on the right side or the left side of the own vehicle. Further, in this process, a target line representing a line that defines the end of the parking frame farther from the host vehicle is extracted. In the example shown in FIGS. 3 and 4, the white line 42 is the target line in the parking frame between the

white lines

41 and 42, and the white line 43 is the target line in the parking frame between the

white lines

42 and 43.

Subsequently, in S160, it is determined whether there is a parking frame candidate. Here, the candidate parking frame includes a parking frame, but is not limited to a parking frame, and indicates a general parking frame including a parking frame in which another vehicle has already been parked.

If there is no parking frame candidate, the process proceeds to S240 described above. If there is a parking frame candidate, it is determined in S210 whether or not the shape of the target line is the same in an imaging region common to a plurality of captured images.

In this process, first, the same parking frame observed from different positions is extracted from a plurality of captured images, and the shapes of the same target lines constituting the parking frame are compared. The same parking frame specifies a common area in which the same object appears in the plurality of captured images according to the conditions under which a plurality of captured images are obtained, for example, the moving distance of the front camera 4F, and is the same as the parking frame in a certain captured image. It is specified by extracting a parking frame within the area.

Further, when comparing the shapes of the target lines, for example, the lengths of the target lines are compared. Here, the length of the line represents the length of the line in the longitudinal direction. The longitudinal direction is a direction orthogonal to the line width direction. When comparing the lengths of the target lines, for example, as shown in FIGS. 3 and 4, the lengths L ₁ and L ₂ of the same white line 42 are compared. The length of the white line is compared with the length when coordinate conversion is performed in the plane coordinate system.

As shown in FIG. 5 and FIG. 6, when a vehicle is parked in the parking frame, a part of the white line 42 is hidden by the vehicle, and a part of the white line 42 is hidden by the difference in the imaging position of the camera. Is different. For this reason, a difference occurs in the length of the white line 42 observed by the camera.

On the other hand, in the case of a parking frame in which no vehicle is parked in the parking frame, for example, the target line such as the white line 43 is observed in a relatively wide range without being hidden by the vehicle. In this case, the difference in the length of the target line is reduced.

Therefore, in the present embodiment, if the difference in length of the target line is within a preset threshold value, the shapes are recognized as being the same. If the shape of the target line is the same in S210, the parking frame determined to have the same shape of the target line in S220 is recognized as a parking available frame. Further, if the shape of the target line is not the same, the parking frame determined in S230 that the shape of the target line is not the same is recognized as not being a parkingable frame.

When such a process ends, the parking frame recognition process ends.
[1-3. effect]
According to the embodiment described in detail above, the following effects can be obtained.

(1a) In the imaging system 1 described above, the image processing unit 20 is configured to acquire a plurality of captured images representing respective captured images captured at different positions by an imaging device disposed in the host vehicle. Then, the image processing unit 20 extracts a parking frame from a plurality of captured images, and extracts a target line representing a line that defines an end of the parking frame farther from the host vehicle. Further, the image processing unit 20 determines whether or not the shape of the target line is the same for the same parking frame extracted from the plurality of captured images, and determines the parking frame that is determined to have the same shape as the target line. Recognize as a parking frame.

According to such an imaging system 1, since it is determined whether or not it is a parking-capable frame depending on whether or not the shape of the same target line captured at different positions is the same, the parking-capable frame is accurately recognized. it can.

(1b) In the imaging system 1 described above, the image processing unit 20 determines whether or not the length of the target line is the same as the shape of the target line.
According to such an imaging system 1, since the shape of the target line is determined by comparing the lengths of the target lines, the parking frame can be configured with a simple configuration as compared with the configuration that is determined by the shape of the entire line. Can be recognized.

(1c) In the imaging system 1 described above, the image processing unit 20 extracts a region sandwiched between at least two lines as a parking frame.
According to such an imaging system 1, since an area sandwiched by at least two lines is extracted as a parking frame, the parking frame can be extracted using a known white line recognition technique. Therefore, the parking frame can be recognized with a simpler configuration.

(1d) In the imaging system 1 described above, the image processing unit 20 converts each captured image into a planar coordinate system, and applies the subject line to the same parking frame extracted from each captured image converted into the planar coordinate system. It is determined whether or not the shape of the target line is the same, and the parking frame in which the shape of the target line is determined to be the same is recognized as a parkingable frame.

According to such an imaging system 1, since it is determined whether or not the shape of the target line is the same in the planar coordinate system, it is good even if the target line is deformed due to different positions where the captured image is obtained. The shape can be determined.

[2. Other Embodiments]
As mentioned above, although embodiment of this indication was described, this indication is not limited to the above-mentioned embodiment, and can carry out various modifications.

(2a) In the above embodiment, whether or not the shape of the target line is the same is determined based on the length of the target line, but the present invention is not limited to this. For example, as the shape of the target line, it may be determined whether or not the overall shape of the target line, the shape of the end portion on the side far from the own vehicle in the target line, and the like are the same.

Specifically, when determining whether or not the shape of the end portion on the side far from the host vehicle in the target line is the same, as shown in FIG. 3 and FIG. The shapes, that is, the shapes of the white lines 42 in the region [A] and the region [B] are compared. When there is a vehicle in the parking frame, the end of the white line 42 far from the vehicle is hidden by the vehicle, and the part of the vehicle where the white line 42 is hidden changes depending on the position of the camera. The shape of the end far from the vehicle changes. On the other hand, when the white line 42 is not hidden, the shape of the end of the white line 42 far from the vehicle does not change. Therefore, it can be determined whether or not the parking frame is a parkingable frame based on whether or not the shape of the end portion on the side far from the host vehicle is the same in the target line.

According to such an imaging system, since the shape of the target line is determined by comparing the shape of the end of the target line that is far from the host vehicle, it is simpler than the configuration that is determined by the shape of the entire line. It is possible to recognize the parking available frame with a simple configuration.

(2b) In the imaging system 1, when determining whether or not the parking frame is a parking frame, it may be determined using only the parking frame recognition process described above. You may determine by combining a process and a well-known process. For example, a parkable frame may be temporarily extracted by a known process, and a parkable frame recognition process may be used for final determination. Moreover, you may determine with a parking frame being a parking possible frame, when it determines with it being a parking possible frame by at least one of a well-known process and a parking possible frame recognition process, or a majority of processes.

(2c) A plurality of functions of one constituent element in the embodiment may be realized by a plurality of constituent elements, or a single function of one constituent element may be realized by a plurality of constituent elements. . Further, a plurality of functions possessed by a plurality of constituent elements may be realized by one constituent element, or one function realized by a plurality of constituent elements may be realized by one constituent element. Moreover, you may abbreviate | omit a part of structure of the said embodiment. Further, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified from the wording described in the claims are embodiments of the present disclosure.

(2d) In addition to the imaging system 1 described above, various devices such as a parking frame recognition device as a component of the imaging system 1, a program for causing a computer to function as the imaging system 1, a semiconductor memory storing the program, and the like The present disclosure can also be realized in various forms such as a non-transition actual recording medium and a parking frame recognition method.

[3. Correspondence between Configuration of Embodiment and Configuration of Present Disclosure]
In the above-described embodiment, the image processing unit 20 of the imaging system 1 corresponds to a parking frame recognition device in the present disclosure. In addition, the process of S110 among the processes executed by the image processing unit 20 corresponds to the image acquisition unit referred to in the present disclosure, and the processes of S120 and S150 correspond to the line extraction unit referred to in the present disclosure in the above embodiment. In the above embodiment, the process of S140 corresponds to the coordinate conversion unit referred to in the present disclosure, and the processes of S210 and S220 in the above embodiment correspond to the possible frame recognition unit referred to in the present disclosure.

Claims

A parking frame recognition device (20),
An image acquisition unit (S110) configured to acquire a plurality of captured images representing respective captured images captured at different positions by an imaging device disposed in the host vehicle;
A line extraction unit (S120, S150) configured to extract a parking frame from the plurality of captured images and extract a target line representing a line defining an end of the parking frame farther from the vehicle; ,
For the same parking frame extracted from the plurality of captured images, it is determined whether or not the shape of the target line is the same, and the host vehicle parks the parking frame determined to have the same shape of the target line. A possible frame recognition unit (S210, S220) configured to recognize as a possible parking frame representing a possible parking frame;
A parking frame recognition device.
The parking frame recognition device according to claim 1,
The parking frame recognition device configured to determine whether or not the length of the target line is the same as the shape of the target line.
The parking frame recognition device according to claim 1 or 2,
The said possible frame recognition part is the parking frame recognition apparatus comprised so that it might determine whether the shape of the edge part far from the own vehicle in the said target line is the same as the shape of the said target line.
The parking frame recognition device according to any one of claims 1 to 3,
The said line extraction part is the parking frame recognition apparatus comprised so that the area | region pinched | interposed by the at least 2 line as the said parking frame may be extracted.
The parking frame recognition device according to any one of claims 1 to 4,
A coordinate conversion unit (S140) configured to convert each captured image into a planar coordinate system;
Further comprising
The possible frame recognizing unit determines whether or not the shape of the target line is the same for the same parking frame extracted from each captured image converted into the plane coordinate system, and the shape of the target line is The parking frame recognition apparatus comprised so that the parking frame determined to be the same may be recognized as the said parking possible frame.