JP6549511B2 - Parking assistance apparatus and parking assistance method - Google Patents

Description

  The present invention relates to a parking assist apparatus for displaying on a display unit a captured image captured by an imaging unit that captures the periphery of a vehicle, and a parking assist method for the parking assist apparatus.

  DESCRIPTION OF RELATED ART When parking a vehicle, the parking assistance apparatus which assists a driver's parking operation is known by displaying the picked-up image (rear view image) of the vehicle back on the display apparatus mounted in a vehicle. Moreover, in recent years, a parking assistance device capable of switching between such a photographed image and a bird's-eye image (top view image) looking down on the vehicle from above is realized. Thus, first, the surroundings of the vehicle such as the position of the obstacle and the parking frame are confirmed by the photographed image at the rear of the vehicle, and then the vehicle position is checked with respect to the wheel stopper and the parking frame at the rear by the overhead image be able to.

  And in such a parking assistance device, the art about change control of a display picture is indicated (for example, refer to patent documents 1). The parking assistance device disclosed in Patent Document 1 calculates the deflection angle between the vehicle and the parking area (the actual angle of the vehicle relative to the direction of the vehicle when the vehicle is properly accommodated in the parking area) in parking accompanied by turning. As a result of the calculation, when the deflection angle becomes equal to or less than the reference deflection angle, the photographed image such as the rear view image is switched to the changed image such as the top view image.

JP, 2008-296697, A

  However, in the parking assistance device of Patent Document 1 described above, when the vehicle is parked, it is considered that the parking operation of the driver is at the end of the display image when the deflection angle becomes equal to or less than the reference deflection angle. In order to switch, depending on the actual parking situation, switching may not be performed at an appropriate position. For example, in the case where the entering angle of the vehicle into the parking space is shallow (when the deflection angle at the start of parking is close to the reference deflection angle), the deflection angle becomes less than the reference deflection angle immediately after the start of the parking operation. . In such a case, since the display image is switched immediately after the start of parking, there is a possibility that the situation around the vehicle can not be sufficiently confirmed by the photographed image. On the other hand, when the entry angle of the vehicle into the parking area is deep (when the deflection angle at the start of parking is far from the reference deflection angle), the deflection angle is the reference deflection angle even if the driving operation for parking is late. It may not be the case below. In such a case, even at the end of the parking operation, the user can not park while checking the parking position with the overhead image, and a situation may occur where convenience is poor.

  The present invention has been made in view of such circumstances, and is a parking assistance apparatus capable of favorably supporting a parking operation by switching between a photographed image and a bird's-eye view around the vehicle at appropriate positions in various parking situations. And provide a parking assistance method.

  A parking assistance apparatus according to the present invention is a parking assistance apparatus for displaying on a display means a photographed image photographed by a photographing means for photographing the surroundings of a vehicle, and generates a bird's-eye view image looking down the vehicle from above Generating means, parking area setting means for setting a parking area based on the photographed image, overlap degree calculation means for calculating an overlap degree between the vehicle and the parking area, and the overlap degree is equal to or more than a predetermined value And the display control means for displaying the overhead view image on the display means.

  With such a configuration, the generation unit generates a bird's-eye view image looking down at the vehicle from above from the photographed image photographed by the photographing unit photographing the surroundings of the vehicle, and the photographed area photographed by the photographing unit by the parking area setting unit The parking area is set based on the above, and the overlap degree of the vehicle and the parking area is computed by the computing means. Then, when the calculated degree of overlap is equal to or greater than a predetermined value by the display control means, the image displayed on the display means is located above the vehicle from the photographed image photographed by the photographing means for photographing the surroundings of the vehicle It is switched to the eyebrow image which looks down from. Therefore, since the image displayed on the display means can be switched according to the degree of overlap between the vehicle and the parking area, it is possible to prevent the display image from being switched from the photographed image to the overhead image immediately after the start of parking. At the end of the parking operation, the display image can be reliably switched from the photographed image to the overhead image. That is, as described above, the images displayed on the display means can be switched at appropriate positions in various parking situations.

  The parking assistance apparatus according to the present invention further comprises storage means for storing a vehicle area set based on at least the vehicle width and the vehicle length of the vehicle, and the overlap degree calculation means sets the area of the vehicle area to "S". When the area where the vehicle area and the target parking area of the vehicle overlap is "A", the overlapping degree (R) can be obtained by the following equation, R = A / S.

  With such a configuration, using the vehicle area set based on at least the vehicle width and the vehicle length of the vehicle stored by the storage means, the area of the vehicle area is “S”, the vehicle area and the target parking of the vehicle When the area overlapping with the area is “A”, the overlapping degree (R) is obtained by the following equation, R = A / S. Thus, the overlapping degree can be easily obtained using the vehicle area based on the vehicle width and the vehicle length of the vehicle.

  The parking assistance device according to the present invention further comprises storage means for storing a vehicle area set based on at least the vehicle width and the vehicle length of the vehicle, and the overlap degree calculation means comprises the vehicle area and the target parking of the vehicle. The overlapping area with the area is "A", the overlapping area with the vehicle area and the parking area other than the target parking area is "B", and the overlapping area with the vehicle area and the non-parking area is "C". In this case, the degree of overlap (R) can be obtained by the following equation: R = (A−B) / (A + B + C).

  With such a configuration, in addition to the fact that the overlap degree can be easily obtained using the vehicle area based on the vehicle width and the vehicle length of the vehicle as in the case described above, the vehicle area and the Since the area “B” where the vehicle area and the parking area other than the target parking area overlap is subtracted from the area “A” where the target parking area of the vehicle overlaps, the display image is thus displayed even with parking accompanied by turning. In the case where the switching position of the vehicle is straightly approached to the target parking area and parked, it can be brought close to a substantially equivalent position. Further, when the parking is performed largely deviated from the target parking area and the area in the parking area other than the target parking area becomes larger than the predetermined value, the overlapping degree does not become the predetermined value or more. Therefore, the display image is not switched, that is, the state in which the photographed image is displayed is maintained. As described above, the degree of deviation at the time of parking can be determined by the degree of overlap (R).

  The parking assist apparatus according to the present invention may further include a notification unit that notifies the driver of the degree of overlap.

  With such a configuration, since the notification means notifies the user of the overlap degree calculated by the overlap degree calculation means, the user can know the progress of parking and is displayed on the display means It is possible to know the switching timing of the display image.

  A parking assistance method according to the present invention is a parking assistance method of a parking assistance apparatus for displaying on a display means a photographed image taken by a photographing means mounted on a vehicle, the image generation means comprising the vehicle from the photographed image Step of generating a bird's-eye view image looking down from above, step of setting a parking area based on the photographed image, step of setting a parking area based on the photographed image, and overlapping degree computing means computing an overlapping degree between the vehicle and the parking area The display control means may include the step of displaying the overhead view image on the display means when the overlap degree is equal to or more than a predetermined value.

  According to the parking assist apparatus and the parking assist method according to the present invention, the captured image and the overhead image around the vehicle can be switched at appropriate positions in various parking situations, so that the parking operation can be favorably assisted. .

It is a block diagram showing an example of composition of an in-vehicle device concerning an embodiment of the invention. It is a flowchart which shows an example of the flow of a process in the vehicle-mounted apparatus shown in FIG. It is a figure which shows an example of the state at the time of a parking start. It is a figure which shows an example of the display screen at the time of a parking start. It is a figure which shows an example of the state at the time of the parking late stage. It is a figure which shows an example of the display screen at the time of the parking late stage. It is a figure which shows an example of the setting of the predetermined value of the overlap degree to which a display screen switches. It is a figure which shows an example (the 1) of a parking condition. It is a figure which shows an example (the 2) of a parking condition.

  Hereinafter, embodiments of the present invention will be described based on the drawings.

  An on-vehicle apparatus 100 (a parking assistance apparatus according to an embodiment of the present invention) shown in FIG. 1 is mounted on a vehicle and assists the parking operation when the driver moves the vehicle backward for parking. It is a thing. As illustrated, the on-vehicle apparatus 100 includes a processing unit 11, a camera unit 12, a sensor group 13, a storage unit 14, a display unit 15, an operation unit 16, a speaker 17, an output circuit 18, an image conversion unit 19, and parking frame information generation. A unit 20 and an HMI image generation unit 21 are provided.

  The processing unit 11 is configured by a computer unit (for example, including a CPU). A camera unit 12 is connected to the processing unit 11 and is configured to be able to acquire a photographed image output from the camera unit 12.

  The camera unit 12 (photographing means according to the present embodiment) has a digital camera having an imaging device such as a CCD (Charge Coupled Device), and is used in a vehicle (hereinafter referred to as a vehicle) on which the in-vehicle device 100 is mounted. The surroundings are continuously photographed on a time series (for example, 30 fps), and the photographed image is output. Specifically, the camera unit 12 is configured of a camera device for capturing the rear of the vehicle. The camera unit 12 is not limited to this, and is configured of four camera devices that respectively capture four directions of the front, rear, left, and right of the vehicle, and may be configured to capture all 360 ° around the vehicle. Good. In order to photograph the surroundings of the own vehicle in a wide range, as a camera device which the camera unit 12 has, one provided with a wide-angle lens, a fisheye lens and the like may be used.

  Further, a sensor group 13 is connected to the processing unit 11 so that various sensor information output from the sensor group 13 can be acquired. The sensor group 13 is various sensor groups that detect various states of the vehicle, and specifically, a shift position sensor that detects a shift position, a vehicle speed sensor that detects a vehicle speed, and a steering angle sensor that detects a steering angle, The engine speed sensor is configured to detect an engine speed.

  Furthermore, in the processing unit 11, a display configured by a storage unit 14 (storage means according to the present embodiment) such as a hard disk drive or the like in which various information used for processing in the processing unit 11 and programs are stored An operation unit 16 including a touch panel on the display unit 15 and used to input an instruction for processing and various information is connected. The display unit 15 (display unit according to the present embodiment) displays various information such as an image for supporting the parking operation based on the processing in the processing unit 11 under the control of the processing unit 11. In addition, an output circuit 18 connected to the speaker 17 is connected to the processing unit 11, and an audio signal based on processing in the processing unit 11 is provided to the speaker 17 via the output circuit 18. The voice based on the voice signal is uttered.

  Further, an image conversion unit 19 is connected to the processing unit 11 so that a bird's-eye view image output from the image conversion unit 19 can be obtained. The image conversion unit 19 (generation unit according to the present embodiment) performs known image conversion processing for performing viewpoint conversion on a captured image output from the camera unit 12 under the control of the processing unit 11. The camera generates and outputs a bird's-eye view image (an image looking down on the ground surface in the vertical direction) looking down on the vehicle from above. The processing unit 11 acquires vehicle frame information (to be described in detail later) representing the vehicle frame of the vehicle stored in advance in the storage unit 14 and uses a known method to determine the position of the vehicle in the overhead image. The frame image is synthesized at a location corresponding to (the position at the time when each image was taken). For example, when a part of the vehicle is included in the captured image, the position of the vehicle can be specified based on the part of the vehicle. If a part of the vehicle is not included, the camera position can be identified from the camera position and the direction of the optical axis, and the position of the vehicle can be identified from the identified camera position. The movement of the vehicle may be specified using various sensor information output from the sensor group 13, and the position of the vehicle may be specified using the specified movement.

  In addition, the parking frame information generation unit 20 is connected to the processing unit 11, and the parking frame information output from the parking frame information generation unit 20 can be acquired. Under control of the processing unit 11, the parking frame information generation unit 20 parks behind the vehicle using known image processing methods such as edge detection and Hough conversion based on the photographed image output from the camera unit 12. A frame is detected, and parking frame information indicating the shape and size of the parking frame is generated and output based on the detection result. In addition, the processing unit 11 combines the parking frame image based on the parking frame information with the corresponding location of the overhead image by a known method, as in the case of the above-described vehicle frame image. Then, the processing unit 11 sets the area represented by the parking frame in the parking area by a known method.

  Furthermore, an HMI image generation unit 21 is connected to the processing unit 11, and is configured to be able to acquire an HMI (Human Machine Interface) image output from the HMI image generation unit 21. The HMI image generation unit 21 is based on the photographed image output from the camera unit 12 under the control of the processing unit 11, the parking frame information output from the parking frame information generation unit 20, the above-described combined overhead image, etc. It generates and outputs HMI images such as guidelines and images showing parking conditions.

  The on-vehicle apparatus 100 configured as described above performs the process described below. First, after the switch operated when starting the vehicle, for example, the accessory switch (ACC) is turned on, in the on-vehicle apparatus 100, a function for assisting the driver's parking operation is activated by the operation of the operation unit 16 or the like. Then, the processing unit 11 executes the process according to the procedure shown in FIG.

  First, the processing unit 11 determines whether the vehicle has started parking based on various sensor information and the like input from the sensor group 13 (S11). For example, the start of parking can be detected by detecting the switching of the gear to reverse (reverse) based on sensor information from the shift state sensor. Here, when it is determined that the parking is not started (NO in S11), the waiting state is started until the parking is started. On the other hand, when it is determined that parking has been started (YES in S11), the processing unit 11 controls the camera unit 12 to capture the surroundings of the vehicle and acquires the captured image (S12). Next, the processing unit 11 controls the image conversion unit 19 to generate and output a bird's-eye view image to acquire a bird's-eye view image (S13). At this time, the processing unit 11 composites the above-described vehicle frame image with the acquired overhead image.

  Then, the processing unit 11 controls the parking frame information generation unit 20 to generate and output parking frame information, and sets a parking area based on the parking frame information. Specifically, as described above, the processing unit 11 synthesizes the parking frame image with the overhead image, and sets a parking area (S14: parking area setting means). Here, the processing unit 11 determines the presence or absence of a parking frame based on the acquired parking frame information (S15). If it is determined that there is no parking space (NO in S15), the processing unit 11 returns the process to S14. On the other hand, when it is determined that there is a parking frame (YES in S15), the processing unit 11 uses, for example, a bird's-eye view image (a composite of a frame image and a parking frame image) The degree of overlap is calculated (S17: degree of overlap calculation means), and it is determined whether the degree of overlap obtained by the calculation is equal to or more than a predetermined value (details will be described later) (S17). At this time, when it is determined that it is not the predetermined value or more (NO in S17), the photographed image of the vehicle rear is displayed on the display unit 15 (S18), and when it is determined that it is the predetermined value or more (S17) In YES), the bird's-eye view image is displayed on the display unit 15 (S19: display control means).

  Then, the processing unit 11 causes the HMI image generation unit 21 to generate and output HMI image information to acquire an HMI image, and causes the HMI image information to be superimposed and displayed on the display image displayed on the display unit 15. , And notify the degree of overlap (S20: notification means). Then, the processing unit 11 determines whether an operation to end the parking operation or an end condition such that the overlapping degree is 100% and parking is completed is satisfied (S21), and it is determined that the condition is not satisfied. If it is determined (NO at S21), the process returns to S16, and if it is determined that the termination condition is satisfied (YES at S21), the process is terminated. The above processing is performed each time at a predetermined cycle, for example, at a frame acquisition interval of a photographed image.

  For example, as shown in FIG. 3, when the vehicle C1 is to be parked in the target parking area AP where the vehicle C1 is to be parked, the vehicle area AC represented by the frame FC of the vehicle C1 overlaps the target parking area AP. Since the degree is 0%, the display unit 15 displays a photographed image behind the own vehicle C1 as illustrated in FIG. 4. Here, the vehicle frame FC is a frame having a size that completely encloses the host vehicle C1, and is, for example, a rectangle as shown in the drawing, and each adjacent side of the rectangle is approximately the entire length and width of the host vehicle C1. It has the same shape (specifically, the same or a slightly larger shape with a margin). That is, the vehicle area AC is set based on the vehicle width and the vehicle length of the own vehicle C1. Here, the vehicle frame FC may have the same shape as the outline of the vehicle C1 when looking down from above. In addition, in the case of the example shown in FIG. 4, the parking rate bar BP which represented the overlap degree with the bar graph, and the parking rate description PP represented with the numerical value are displayed as an HMI image which shows a parking situation. This allows the user to be notified how much certainty the parking will be possible.

  When parking is started from the state shown in FIG. 3 and the vehicle C1 is moved backward, the numerical value of the degree of overlap will rise. Then, as shown in FIG. 5, when the own vehicle C1 is at an ideal switching position (for example, when looking at the overhead image, it is possible to confirm the degree of parallel to the parking frame FP behind the own vehicle C1 and the wheel stopper etc. appear The overhead view image as illustrated in FIG. 6 is displayed on the display unit 15 when moving to the ideal position PI, which is the position), and the overlapping degree reaches a predetermined value (70% (entry%) in the illustrated example). That is, the display image of the display unit 15 is switched from the photographed image on the rear side of the vehicle C1 to the overhead view image, whereby the image can be switched at the ideal position PI without depending on the approach route. In addition, when there is a high possibility of being able to park in the target parking area AP, it switches to the bird's-eye view image, which can improve the convenience of the user In the example shown in FIG. Down the border line L Region AD (rear of own vehicle C1) is composed of a photographed image (including one to which image processing such as distortion correction has been performed), and upper region AU thereof is composed of a virtual image, but is limited thereto The processing unit 11 zooms the overhead image displayed on the display unit 15 when detecting that the target parking area AP is extremely large (long) compared to the vehicle area AC. By going out, it is possible to display up to the frame line of the rear parking frame FP.

  Here, the predetermined value of the overlapping degree in the above-described determination is appropriately set in accordance with the parking characteristics of the vehicle, specifically, the length and width of the vehicle. For example, the user designates this setting using the operation unit 16. For example, in the case of a large vehicle, as shown in FIG. 7, the predetermined value is set to 60%. As described above, by setting the predetermined value in accordance with the size of the vehicle, the predetermined value can be set according to the parking characteristic according to the size of the vehicle, and a wide range of vehicle types can be supported. For example, a small vehicle may set the predetermined value to a large value (eg, 80%) as compared to a normal vehicle, and a large vehicle may set the predetermined value to a small value (eg, 60%) as compared to a normal vehicle. Preferred for parking characteristics.

  In the on-vehicle apparatus 100, as shown in FIG. 8, when the vehicle is parked, the deflection angle λ between the vehicle area AC and the target parking area AP (with respect to the direction of the own vehicle when correctly accommodated in the target parking area AP) An ideal position PI where the overlapping degree is equal to or greater than a predetermined value even when the deflection angle is large even in the final stage of the parking operation, for example, when entering the target parking area AP in a parking situation where the actual vehicle angle is large). The display image of the display unit 15 can be switched (including the proximity position). The degree of overlap (R) may be calculated simply by, for example, R = A / S, but in the present embodiment, it is determined by calculating R = (A−B) / (A + B + C). . Here, the code S is the area of the vehicle area AC, the code A is the area of a portion where the vehicle area AC and the target parking area AP overlap, and the code B is not a parking target for the vehicle area AC The area of the portion where the parking area NP other than the target parking area is overlapped, and the code C is the area of the area where the vehicle area AC and the non-parkable area which is an area where parking can not be performed overlap. By calculating the overlap degree (R) in this manner, the area B is subtracted from the area A in the numerator of the division formula, so the switching position of the display image can be set to the target parking area AP even in parking with turning. It can be approached to a position substantially the same as when it entered straight and parked.

  Here, for example, as shown in FIG. 9, when the parking is performed with a large deviation from the target parking area AP (when the parking area NP is adjacent to the target parking area AP), the above-described calculation is performed. In this case, since the degree of overlap does not reach the predetermined value or more, the display image of the display unit 15 is not switched, that is, the state in which the photographed image is displayed is maintained. As described above, the degree of deviation at the time of parking can be determined by the degree of overlap (R).

  According to the on-vehicle apparatus 100 (parking support apparatus) as described above, the image displayed on the display unit 15 can be switched according to the degree of overlap between the vehicle C1 and the target parking area AP. It is possible to prevent the display image from being switched from the captured image to the overhead image, and to switch the displayed image from the captured image to the overhead image reliably at the end of the parking operation. That is, as described above, the images displayed on the display unit 15 can be switched at appropriate positions in various parking situations.

  Further, as described above, when calculating the overlap degree, the overlap degree is calculated using the vehicle area AC based on the vehicle width and the vehicle length of the vehicle stored in advance in the storage unit 14, so the overlap degree can be easily made Further, since the area B is subtracted from the area A on the numerator side of the division formula at that time, the switching position of the display image is determined only by the area A where the numerator side is in the target parking area AP. Compared with the case where it carried out, it can approach close to a position substantially equivalent to the case where it approached into the target parking area AP straightly, and was parked. Furthermore, when the parking is performed largely deviated from the target parking area AP, and the area of the portion overlapping the parking area NP in the vehicle area AC becomes larger than a predetermined value, the overlapping degree (R) is the above Since the value does not exceed the predetermined value, switching of the display image is not performed. As described above, the degree of deviation at the time of parking can be determined by the degree of overlap (R).

  Furthermore, since the overlapping degree is notified to the user via the display unit 15, the user can know the progress of parking in real time, and also know the switching timing of the display image displayed on the display unit 15. be able to.

  As described above, the parking support device and the parking support method according to the present invention can appropriately switch the captured image and the overhead image around the vehicle at appropriate positions in various parking situations, so that the parking operation can be favorably supported. The present invention is useful as a parking assist device for displaying on a display means a captured image captured by a capturing means for capturing the surroundings of a vehicle, and a parking assist method for the parking assist device.

11 processing unit 12 camera unit 13 sensor group 14 storage unit 15 display unit 16 operation unit 17 speaker 18 output circuit 19 image conversion unit 20 parking frame information generation unit 21 HMI image generation unit 100 vehicle-mounted device

Claims (5)

A parking assist apparatus for displaying on a display unit a photographed image photographed by a photographing unit for photographing the surroundings of a vehicle, comprising:
Generation means for generating a bird's-eye view image that looks down the vehicle from above from the captured image;
Parking area setting means for setting a parking area based on the photographed image;
Overlap degree calculation means for calculating an overlap degree between the vehicle and the parking area;
And a display control unit configured to display the overhead view image on the display unit when the overlapping degree is equal to or more than a predetermined value. It further comprises storage means for storing a vehicle area set based on at least the width and the length of the vehicle.
The overlapping degree computing means sets the overlapping degree (R) to be the following, when the area of the vehicle area is “S” and the area where the vehicle area and the target parking area of the vehicle overlap is “A”: formula,
R = A / S
The parking assistance device according to claim 1, which is obtained by It further comprises storage means for storing a vehicle area set based on at least the width and the length of the vehicle.
The overlap degree calculation means sets an area where the vehicle area and the target parking area of the vehicle overlap to be “A”, and sets an area where the vehicle area and a parking area other than the target parking area to overlap is “B” Assuming that the area where the vehicle area and the non-parking area overlap is "C", the overlapping degree (R) is expressed by the following equation:
R = (A-B) / (A + B + C)
The parking assistance device according to claim 1, which is obtained by   The parking assistance device according to any one of claims 1 to 3, further comprising a notification unit that notifies the driver of the degree of overlap. A parking assistance method of a parking assistance apparatus, which displays a photographed image photographed by a photographing unit mounted on a vehicle on a display unit,
Generating a bird's-eye view of the vehicle from above the image from the photographed image;
Parking area setting means sets a parking area based on the photographed image;
Calculating the degree of overlap between the vehicle and the parking area;
And displaying the bird's-eye view image on the display means when the display control means has the overlap degree equal to or greater than a predetermined value.

Images