JP2022101749A - Vehicle support device - Google Patents

Abstract

To provide a vehicle support device that can detect a parking space regardless of a presence or absence of a parking frame.SOLUTION: A vehicle support device 100 comprises: a side camera 110 that captures a side of a vehicle; a road surface/three-dimensional object recognition unit 114 that recognizes a road surface and a three-dimensional object included in an imaging range based on the captured side image; a bird's-eye view image generation unit 116 that converts an image recognized as the three-dimensional object by the road surface/three-dimensional object recognition unit 114 into a bird's-eye view image; a vehicle coordinate detection unit 122 that detects coordinates of a three-dimensional object corresponding to the bird's-eye view image; and a parking space detection unit 124 that detects a parking space adjacent to the three-dimensional object based on the coordinates of the three-dimensional object detected by the vehicle coordinate detection unit 122.SELECTED DRAWING: Figure 1

Description

The present invention relates to a vehicle support device that detects the position of another parked vehicle.

Conventionally, by converting an image obtained by capturing the surroundings of the vehicle with a camera mounted on the own vehicle into a bird's-eye view image (bird's-eye view), a parking frame drawn by a white line or the like can be specified and the own vehicle can be parked. A detection device (for example, see Patent Document 1) and a parking position detection system (for example, see Patent Document 2) for detecting a parking position are known.

Japanese Unexamined Patent Publication No. 2019-137182 Japanese Unexamined Patent Publication No. 2017-147629

By the way, in the detection devices and the like disclosed in Patent Documents 1 and 2 described above, since the parking frame is specified based on the image captured by the camera, when two vehicles are parked with a gap between them. There was a problem that if there was a parking space in the meantime but there was no parking frame, it would not be recognized as a parking space.

The present invention has been created in view of these respects, and an object of the present invention is to provide a vehicle support device capable of detecting a parking space regardless of the presence or absence of a parking frame.

In order to solve the above-mentioned problems, the vehicle support device of the present invention includes an image pickup means for imaging the side of the vehicle, and a road surface and a three-dimensional object included in the image pickup range based on the side image captured by the image pickup means. A three-dimensional object recognition means for recognizing a three-dimensional object, a bird's-eye view image generation means for converting an image recognized as a three-dimensional object by the road surface / three-dimensional object recognition means into a bird's-eye view image, and a three-dimensional object for detecting the coordinates of the three-dimensional object corresponding to the bird's-eye view image. It includes an object coordinate detecting means and a parking space detecting means for detecting a parking space adjacent to the three-dimensional object based on the coordinates of the three-dimensional object detected by the three-dimensional object coordinate detecting means. Since the three-dimensional object on the road surface is recognized and the coordinates of the three-dimensional object are detected based on the bird's-eye view image, it is possible to detect the parking space adjacent to the three-dimensional object regardless of the presence or absence of the parking frame.

Further, it is desirable that the above-mentioned three-dimensional object coordinate detecting means detect the coordinates of the side surface by the position of the image pickup means and the angle with respect to the side surface when the image pickup means is present on the plane including the side surface of the three-dimensional object. Specifically, the image pickup operation by the above-mentioned imaging means, the recognition operation by the road surface / three-dimensional object recognition means, and the bird's-eye view image generation operation by the bird's-eye view image generation means are repeatedly performed, and the three-dimensional object coordinate detection means is generated. It is desirable to detect the coordinates of the side surface by finding the bird's-eye view image in which the imaging means exists on the plane including the side surface of the three-dimensional object from the plurality of bird's-eye view images.

When a three-dimensional object such as a vehicle included in the bird's-eye view image is floating from the road surface, the position of the three-dimensional object on the bird's-eye view image deviates from the actual position. If it exists, the position (coordinates) of this side surface does not deviate. Therefore, by specifying the position of the imaging means that captured the bird's-eye view image at such timing, the exact position (coordinates) of the side surface of the three-dimensional object is specified. Can be detected.

Further, the above-mentioned three-dimensional object coordinate detecting means is based on the bird's-eye view image obtained at the imaging timing before the bird's-eye view image used for the side surface coordinate detection and the detected side surface coordinates, and the lower contour of the side surface. It is desirable to determine the surface height of the surface and detect the coordinates of the front surface of the three-dimensional object included in the bird's-eye view image using this surface height. By assuming that the ground surface height of the lower side surface of the three-dimensional object is the same as the ground surface height of the lower front surface, it is possible to detect the accurate position (coordinates) of the front surface floating from the road surface.

Further, it is desirable that the parking space detecting means described above detect a parking space adjacent to the three-dimensional object based on the coordinates of the side surface and the front surface of the three-dimensional object detected by the three-dimensional object coordinate detecting means. By detecting the accurate coordinates of the side surface and the front surface of the three-dimensional object, it is possible to accurately detect the parking space as an empty space on the road surface adjacent to the three-dimensional object without using a parking frame or the like.

It is a figure which shows the structure of the vehicle support device of one Embodiment. It is a figure which shows the situation of the image pickup by the side camera. It is a flow chart which shows the operation procedure which detects the vehicle coordinates by the vehicle coordinates detection part using the generated bird's-eye view image. It is a figure which shows the principle of the side surface detection of a vehicle. It is a figure which shows the specific example of the side surface detection of a vehicle. It is a figure which shows the specific example of the count result. It is explanatory drawing of the ground surface height determination. It is explanatory drawing of the coordinate calculation of the front surface of a vehicle body.

Hereinafter, a vehicle support device according to an embodiment to which the present invention is applied will be described with reference to the drawings.

FIG. 1 is a diagram showing a configuration of a vehicle support device according to an embodiment. As shown in FIG. 1, the vehicle support device 100 of the present embodiment includes a side camera 110, a captured image storage unit 112, a road surface / three-dimensional object recognition unit 114, a bird's-eye view image generation unit 116, a bird's-eye view image storage unit 118, and a position sensor 120. , The vehicle coordinate detection unit 122 and the parking space detection unit 124 are provided.

The side camera 110 is attached to a predetermined position on the side of the vehicle (for example, the lower part of the door mirror), and photographs the side of the vehicle including the road surface through a fisheye lens or a wide-angle lens. The image (side image) obtained by the image taken by the side camera 110 is stored in the image image storage unit 112. Further, the side camera 110 repeatedly images the side of the vehicle at predetermined intervals (for example, at regular time intervals or at regular mileage intervals), and the side images for a plurality of times of imaging are sequentially stored in the captured image storage unit 112. Stored.

The road surface / three-dimensional object recognition unit 114 recognizes the road surface included in the imaging range and the vehicle as a three-dimensional object based on the side image captured by the side camera 110. For example, it may be possible to recognize the road surface and other (three-dimensional objects) by determining whether or not each pixel value of the side image is similar to the road surface. The road surface / three-dimensional object recognition unit 114 creates a binary image as a recognition result.

The bird's-eye view image generation unit 116 generates a bird's-eye view image using a binary image created by the road surface / three-dimensional object recognition unit 114. The bird's-eye view image is an image when the road surface is viewed from the sky, and can be generated by performing viewpoint conversion after considering the distortion parameter and the mounting position parameter of the lens of the side camera 110. The bird's-eye view image generated by the bird's-eye view image generation unit 116 is stored in the bird's-eye view image storage unit 118.

The position sensor 120 detects the position of the own vehicle on which the vehicle support device 100 is mounted. For example, it can be realized by using a GPS, a mileage sensor, an acceleration sensor, or the like.

The vehicle coordinate detection unit 122 detects the coordinates of the vehicle as a three-dimensional object included in the bird's-eye view image generated by the bird's-eye view image generation unit 116. The details of coordinate detection will be described later.

The parking space detection unit 124 detects the parking space between adjacent vehicles based on the vehicle coordinates detected by the vehicle coordinate detection unit 122. Information on the detected parking space is sent to the automatic parking control unit 200. This information includes the coordinates of the side surface and the front surface of the vehicle adjacent to the parking space, and the automatic parking control unit 200 automatically drives and parks the own vehicle in the parking space based on this information. Since the operation itself of controlling the steering angle and the like to perform automatic driving and parking in the parking space can be performed by using the existing technique, the details thereof will be omitted.

The side camera 110 described above is used as an image pickup means, the road surface / three-dimensional object recognition unit 114 is used as a road surface / three-dimensional object recognition means, the bird's-eye view image generation unit 116 is used as a bird's-eye view image generation means, and the vehicle coordinate detection unit 122 is used as a three-dimensional object coordinate detection means. , The parking space detection unit 124 corresponds to the parking space detection means, respectively.

The vehicle support device 100 of the present embodiment has such a configuration, and next, an operation of detecting a parking space will be described.

The user (vehicle driver) operates the operation unit (not shown) to instruct the start of detection of the parking space, and then travels at a low speed in the parking lot including the parking space to be detected. During this low-speed driving, the image pickup operation by the side camera 110, the recognition operation by the road surface / three-dimensional object recognition unit 114, and the bird's-eye view image generation operation by the bird's-eye view image generation unit 116 are repeatedly performed, and the plurality of bird's-eye view images generated are each. It is stored in the bird's-eye view image storage unit 118 together with the image pickup position (position represented by the world coordinate system of the own vehicle) corresponding to. It should be noted that the above-mentioned traveling operation of the own vehicle may be performed by automatic driving instead of being performed by the driver himself.

FIG. 2 is a diagram showing a situation of imaging by the side camera 110. In FIG. 2, G indicates the own vehicle, and T1 to T4 indicate other vehicles in the parking lot. Further, P is a parking space to be detected, and there may or may not be a white line that divides the parking space on the road surface. The own vehicle G moves the traveling position while looking sideways at the vehicles T1 to T4 and the parking space P (while capturing these images with the side camera 110). In the example shown in FIG. 2, an example in which the side camera 110 is provided on the left side of the vehicle is shown, but in reality, the side camera 110 is also provided on the right side of the vehicle, and the image can be captured by the two side cameras 110. It is done in parallel.

FIG. 3 is a flow chart showing an operation procedure for detecting vehicle coordinates by the vehicle coordinate detection unit 122 using the generated bird's-eye view image.

First, the vehicle coordinate detection unit 122 detects the side surfaces of other vehicles (vehicles T1 to T4 shown in FIG. 2) (step 100).

FIG. 4 is a diagram showing the principle of side surface detection of a vehicle. FIG. 4 shows how the relative positions of the side camera 110 and the vehicle T3 change with the passage of time. Further, in these figures, the frame shown by the dotted line t3 indicates the exact position of the vehicle T3 on the road surface. The three-dimensional object (vehicle) included in the generated bird's-eye view image has the following features.
(1) Since a tall object is large and an object floating above the ground surface is reflected in the distance, the coordinates on the bird's-eye view image cannot be used as they are as the coordinates of the vehicle.
(2) When the vertical surface (side surface of the vehicle body) of a three-dimensional object and the side camera 110 are on the same plane, the vertical surface looks linear on the bird's-eye view image (FIG. 4 (B)). The slope and intercept of this straight line match the real-world coordinates of the vertical plane.

From such a feature, if the bird's-eye view image at the timing when the vertical plane is on the same plane as the side camera 110 can be found from the time-series bird's-eye view images, the coordinates of the side surface of the vehicle body can be known.

FIG. 5 is a diagram showing a specific example of side surface detection of a vehicle. In the specific example shown in FIG. 5, the imaging range of the side camera 110 is divided into N at the center of the field of view within a range of ± 90 ° to the left and right to define a scan line for the side surface of the vehicle body. In FIG. 5, for the sake of clarity, the three angles θ1, θ2, and θ3 are shown, but in reality, they are divided into 90 at 1 ° intervals, for example.

The vehicle coordinate detection unit 122 counts the number of pixels of a three-dimensional object displayed on the scan line for the bird's-eye view image at each time. The coordinates of the side surface of the vehicle body are determined corresponding to the angle and position (position of the side camera 110 of the own vehicle) in which the count of the number of pixels sharply increases (or decreases) on the scan line.

FIG. 6 is a diagram showing a specific example of the count result. In this example, since the number of pixels at the angle θ2 increases sharply at time 5 and decreases sharply at time 10, the side surface of the vehicle body can be detected correspondingly.

After the side surface of the vehicle body is detected in this way, the vehicle coordinate detection unit 122 determines the ground surface height of the vehicle body (step 102). Since the coordinates of the side surface of the vehicle body have been obtained so far, the surface height can be determined by the method of triangulation.

FIG. 7 is an explanatory diagram of the surface height determination. The vehicle coordinate detection unit 122 detects the contour line (L1 in FIG. 7A) of the lower part of the vehicle body by using the bird's-eye view image at the time before the timing when the outer surface of the vehicle is detected. Further, the vehicle coordinate detection unit 122 calculates the ground surface height H of the vehicle body by triangulation using the coordinates of the side surface of the vehicle body and the coordinates of the lower contour line L1 (FIG. 7 (B)).

After the ground surface height H of the vehicle body is obtained in this way, the vehicle coordinate detection unit 122 detects the coordinates of the front surface of the vehicle body (step 104). Since the ground surface height of the vehicle body has been obtained up to this point, the coordinates of the front surface of the vehicle body can be calculated by the method of triangulation.

FIG. 8 is an explanatory diagram of coordinate calculation on the front surface of the vehicle body. The vehicle coordinate detection unit 122 detects the contour line (L2 in FIG. 8A) of the lower part of the front surface of the vehicle body by using the bird's-eye view image. Further, the vehicle coordinate detection unit 122 calculates the coordinates of the front surface of the vehicle body by triangulation using the coordinates of the ground surface height H of the vehicle body and the lower contour line L2 (FIG. 8 (B)). In this way, the exact coordinates of the side surface and the front surface of the vehicle body (the exact position of the frame shown by the dotted line t3 in FIG. 4), that is, the accurate parking position detection operation is completed.

When the position of the parked vehicle is detected in this way, the parking space detection unit 124 detects the parking space using the obtained parking position. For example, in the example shown in FIG. 2, when the coordinates of the side surface s2 and the front surface f2 of the vehicle T2 and the coordinates of the side surface s3 and the front surface f3 of the vehicle T3 are detected, these side surfaces s2, s3 and the front surface f2, f3 It becomes possible to detect the divided parking space p.

As described above, the vehicle support device 100 of the present embodiment recognizes the three-dimensional object (vehicle) on the road surface and detects the coordinates of the three-dimensional object based on the bird's-eye view image, so that the parking space adjacent to the three-dimensional object is detected. Can be detected regardless of the presence or absence of a parking frame.

In addition, when a three-dimensional object such as a vehicle included in the bird's-eye view image is floating from the road surface, the position of the three-dimensional object on the bird's-eye view image deviates from the actual position, but the side is on the plane including the side surface of the three-dimensional object. When the camera 110 is present, the position (coordinates) of this side surface does not deviate. Therefore, by specifying the position of the side camera 110 that captured the bird's-eye view image at such a timing, the side surface of the three-dimensional object can be accurately determined. It is possible to detect the position (coordinates).

Further, by assuming that the ground surface height of the lower side surface of the three-dimensional object is the same as the ground surface height of the lower front surface, it is possible to detect the accurate position (coordinates) of the front surface floating from the road surface.

In particular, by detecting the accurate coordinates of the side surface and the front surface of the three-dimensional object in this way, the parking space as an empty space on the road surface adjacent to the three-dimensional object can be accurately detected without using a parking frame or the like. Is possible.

The present invention is not limited to the above embodiment, and various modifications can be made within the scope of the gist of the present invention. For example, in the above-described embodiment, the coordinates of the front surface of the vehicle body are calculated using the bird's-eye view image generated based on the image captured by the side camera 110, but a distance measuring sensor such as a radar or sonar is used. The distance to the front surface or the side surface of the vehicle body may be detected, and these may be used together for calculating the distance. In this case, if a relatively accurate distance can be detected, such as the front of a three-dimensional object, the detection result by the distance measuring sensor is preferentially used, and the detection accuracy by the distance measuring sensor, such as the side surface of a three-dimensional object, is used. If the value decreases, a method such as using the detection result by the distance measuring sensor as an auxiliary can be considered.

As described above, according to the present invention, since the three-dimensional object on the road surface is recognized and the coordinates of the three-dimensional object are detected based on the bird's-eye view image, the parking space adjacent to the three-dimensional object has a parking frame. It is possible to detect regardless of.

100 Vehicle support device 110 Side camera 112 Captured image storage unit 114 Road surface / three-dimensional object recognition unit 116 Bird's-eye view image generation unit 118 Bird's-eye view image storage unit 120 Position sensor 122 Vehicle coordinate detection unit 124 Parking space detection unit

Claims (5)

An imaging means that captures the side of the vehicle and
A road surface / three-dimensional object recognition means for recognizing a road surface and a three-dimensional object included in the imaging range based on a side image captured by the image pickup means.
A bird's-eye view image generation means for converting an image recognized as the three-dimensional object by the road surface / three-dimensional object recognition means into a bird's-eye view image,
A three-dimensional object coordinate detecting means for detecting the coordinates of the three-dimensional object corresponding to the bird's-eye view image,
A parking space detecting means for detecting a parking space adjacent to a three-dimensional object based on the coordinates of the three-dimensional object detected by the three-dimensional object coordinate detecting means, and a parking space detecting means.
A vehicle support device characterized by being equipped with. The three-dimensional object coordinate detecting means is characterized in that when the image pickup means is present on a plane including the side surface of the three-dimensional object, the coordinates of the side surface are detected by the position of the image pickup means and the angle with respect to the side surface. The vehicle support device according to claim 1. The image pickup operation by the image pickup means, the recognition operation by the road surface / three-dimensional object recognition means, and the bird's-eye view image generation operation by the bird's-eye view image generation means are repeatedly performed.
The three-dimensional object coordinate detecting means detects the coordinates of the side surface by finding a bird's-eye view image in which the imaging means exists on a plane including the side surface of the three-dimensional object from a plurality of generated bird's-eye views images. The vehicle support device according to claim 2, wherein the vehicle support device is characterized by the above. The three-dimensional object coordinate detecting means of the side surface is based on the bird's-eye view image obtained at an imaging timing before the bird's-eye view image used for the coordinate detection of the side surface and the detected coordinates of the side surface. The vehicle support device according to claim 3, wherein the surface height of the lower contour is determined, and the coordinates of the front surface of the three-dimensional object included in the bird's-eye view image are detected using the surface height. 4. The parking space detecting means is characterized in that it detects the parking space adjacent to the three-dimensional object based on the coordinates of the side surface and the front surface of the three-dimensional object detected by the three-dimensional object coordinate detecting means. The vehicle support device described in.

Images