JP2017027396A - Vehicular driving support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicular driving support device capable of improving obstacle collision avoidance performance of an unmanned flying body in supporting driving.SOLUTION: A vehicular driving support device comprises: an unmanned flying body 1 with an imaging device 7 capable of flying a prescribed position in front of a travel direction of a self vehicle 30; a presentation unit 37 for presenting an image, captured by the imaging device, of the front of the travel direction of the vehicle to a crew of the vehicle; and control units 13 and 41 for controlling the unmanned flying body to fly the prescribed position in front of the vehicle. The unmanned flying body includes a flying body side detection unit 7 for detecting an obstacle therearound, the vehicle includes a vehicle side detection unit 49 for detecting an obstacle in front of the travel direction of the vehicle, and the control units include an obstacle avoidance indication unit 50 for indicating flying of the unmanned flying body to a direction for avoiding an obstacle on the basis of the obstacle detected by the flying body side detection unit and the obstacle detected by the vehicle side detection unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a driving support apparatus for a vehicle that supports driving of the host vehicle.

  In automobiles (vehicles), recently, driving support devices that support driving of the own vehicle using an unmanned air vehicle, for example, a drone (unmanned air vehicle) equipped with an imaging camera (imaging device) have been proposed. This driving support device flies a drone in front of the host vehicle, presents a captured image ahead of the traveling direction of the host vehicle with an imaging camera to the driver (driver) of the host vehicle, and informs the driver of the host vehicle the current obstacle. To inform the driver of driving and driving according to the front situation (see, for example, Patent Document 1).

JP 2010-250478 A

The drone follows the host vehicle if the host vehicle turns to the left and flies ahead of the host vehicle. If the host vehicle turns to the right, the drone follows the host vehicle and follows the host vehicle. To fly. At this time, a collision sensor is mounted on the drone so that the drone does not collide with an obstacle so that the flight can be continued while avoiding the obstacle.
By the way, when the own vehicle turns right or left, for example, the drone makes a turn to the left and right from the original position to catch up with the front position of the own vehicle, and flies to a predetermined position in front of the own vehicle that has turned right and left. . At this time, the traveling direction of the drone changes.

Usually, drone obstacle detection range is limited to the limited area around the drone, most of which is limited to the front of the flight direction. May fly.
However, if there is an obstacle above the obstacle detection range of the drone, for example, if a branch of a tree overhangs the sky, or if a bird such as a sparrow or a spider is flying in the sky, the drone itself detects it. The drone in flight may collide with obstacles.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a vehicle driving assistance device capable of improving the obstacle collision avoidance performance of an unmanned air vehicle that is driving assistance.

  Aspects of the present invention present an unmanned aerial vehicle with an imaging device capable of flying at a predetermined position ahead in the traveling direction of the host vehicle, and an image of the host vehicle in the traveling direction captured by the imaging device to an occupant of the host vehicle. A vehicle driving support device comprising: a presenting unit that controls the flight of the unmanned air vehicle so that the unmanned air vehicle continues to fly at a predetermined position in front of the host vehicle. A vehicle-side detection unit that detects obstacles around the body, the host vehicle has a vehicle-side detection unit that detects obstacles ahead of the traveling direction of the host vehicle, and the control unit is Based on the detected obstacle and the obstacle detected by the vehicle side detection unit, an obstacle avoidance instruction unit that instructs the flight of the unmanned air vehicle in a direction to avoid the obstacle is provided.

According to the present invention, even if it is difficult for the unmanned air vehicle itself to detect an obstacle, the host vehicle catches the obstacle and instructs the flight direction of the unmanned air vehicle to avoid the obstacle. For this reason, the obstacle detection function that the unmanned aerial vehicle lacks is compensated for by the host vehicle, so that the unmanned aerial vehicle will not collide with the obstacle, regardless of the attitude. Fly while avoiding.
Therefore, it is possible to improve the collision avoidance performance for the obstacle of the unmanned air vehicle that is driving assistance by cooperative control of the unmanned air vehicle and the host vehicle.

The perspective view which shows the driving assistance device of the vehicle which concerns on the 1st Embodiment of this invention with a control system. The flowchart explaining the control of the collision avoidance in the drone (unmanned aerial vehicle) and an obstacle which a control system performs. The figure explaining the condition of the control. The perspective view explaining that the position of an obstruction is calculated | required three-dimensionally. The flowchart explaining the control used as the principal part of the 2nd Embodiment of this invention.

Hereinafter, the present invention will be described based on a first embodiment shown in FIGS.
FIG. 1 shows an entire vehicle driving support apparatus and a control system of the apparatus. In FIG. 1, reference numeral 1 denotes, for example, a drone that is an unmanned air vehicle, and reference numeral 30 denotes a host vehicle.
As shown in FIG. 1, for example, the drone 1 is installed on the main body 3, a plurality of, for example, four motor-driven propellers 5 a (propulsion unit) installed radially on the side of the main body 3, and the main body 3. Cameras, for example, a stereo camera 7 having the functions of both an imaging device that images the front of the drone (around the drone) and an obstacle sensor that detects obstacles in front of the drone (around the drone), and a battery housed in the main body 3 (Not shown) and a multicopter having a control unit 9. The cover for housing the propeller driving motor 5b also serves as a landing / landing leg. In FIG. 1, reference numeral 7 a indicates a pair of left and right imaging units (only one side is illustrated) of the stereo camera 7.

  As shown in the block diagram of FIG. 1, the control unit 9 of the drone 1 houses a control unit 13, a posture sensor 15 for posture control, a GPS 19, and a transmission / reception unit 21. And by the command of the control part 13, the drive of the motor 5b which used the battery as an electric power source, the rotation control of the propeller 5a, etc. are performed, and it can fly by desired altitude and desired direction. Of course, the drone 1 flies while imaging the front of the drone with the stereo camera 7 under the control of the control unit 13. Then, an obstacle around the drone 1 in flight, in this case, an obstacle ahead of the drone 1 is detected from the image captured by the stereo camera 7, and the flight avoiding the obstacle is performed (details will be described later).

  The host vehicle 30 has, for example, a drone heliport 35 (takeoff and landing portion) on the roof 33a of the vehicle body 33. Further, for example, the instrument panel unit 30 a includes a display 37 (corresponding to the presenting unit of the present application) and a control unit 39 that can be viewed by a driver (occupant) of the host vehicle 30. Then, the drone 1 is mounted on the heliport 35 so that the drone 1 can be taken off and landing at the upper part of the vehicle body. The drone 1 on the heliport 35 is detachably held by a hook mechanism (not shown).

  As shown in the block diagram of FIG. 1, the control unit 39 houses a control unit 41, a GPS 43, and a transmission / reception unit 45 that performs communication with the transmission / reception unit 21 on the drone 1 side. Then, the position of the host vehicle 30 detected by the GPS 43 is displayed on a map displayed on the display 37, or a captured image in front of the drone captured by the stereo camera 7 of the drone 1 is displayed on the display 37. . On the display 37, a scene captured by the imaging unit 7a on one side of the stereo camera 7 is displayed. Incidentally, when there are signals from both the GPS 43 and the imaging unit 7a, for example, the image from the drone 1 is displayed on one of the left and right halves of the screen of the display 37, and the host vehicle position is displayed on the other side.

  The drone 1 flies over the front of the host vehicle 30 in response to a command from the host vehicle 30. Then, in order to continue to capture the scene ahead of the traveling direction of the own vehicle 30 with the stereo camera 7 of the drone 1, the control unit 41 of the own vehicle 30 is set with a function for setting the flight target position of the drone 1. The For example, on the basis of the position of the host vehicle 30 detected by the GPS 43, a certain forward position, a certain sky height position, etc. are set as the flight position. Then, the flight target position is transmitted from the transmission / reception unit 45 of the host vehicle 30 to the transmission / reception unit 21 (both communication units) of the drone 1.

  The control unit 13 of the drone 1 is set with a control program for controlling the flight of the drone 1 according to the flight target position received. As a result, the drone 1 can continue to fly at a fixed position ahead of the traveling direction of the host vehicle 30 according to the set flight target position. Then, a front image of the host vehicle 30 (an image captured on one side of the stereo camera 7) is presented to the driver (driver) of the host vehicle 30 through the display 37, and driving support of the host vehicle 30 is performed.

Further, the driving support device is provided with means for detecting obstacles in a wide area in cooperation with the drone 1 side and the own vehicle 30 side, and making it possible to avoid collision even for obstacles that cannot be detected by the drone 1 itself. .
For example, in this embodiment, the drone 1 is equipped with a stereo camera 7 (corresponding to the flying object side detection unit of the present application) instead of a normal imaging camera as described above. Equipped with a stereo camera 49 (corresponding to the vehicle-side detection unit of the present application) that detects an obstacle ahead of the vehicle 30, an obstacle in a wide area around the drone 1 including the front of the drone 1 and the rear as the blind spot of the drone 1 The object can be detected. That is, the stereo camera 7 of the drone 1 captures an obstacle located ahead in the traveling direction of the drone 1 with a pair of imaging units 7a arranged on both the left and right sides at a predetermined distance. The stereo camera 49 of the host vehicle 30 captures an obstacle located forward in the traveling direction of the host vehicle 30 with a pair of imaging units 49a disposed on both the left and right sides of the room mirror 30b. Of course, the pair of imaging units 49 a also captures the drone 1 positioned in front of the host vehicle 30.

  In addition, the control units 13 and 41 are provided with indexing units 51 and 55 so that the size and position of all obstacles captured by the stereo cameras 7 and 49 can be detected. That is, the indexing unit 51 provided in the control unit 13 performs image processing and further parallax processing on the image in front of the drone 1 acquired from the pair of imaging units 7a and 7a (stereo camera 7), and converts the image information into image information. The three-dimensional position of the obstacle is determined based on the size of the included obstacle and the position of the drone 1. Information on the detected obstacle is transmitted from the transmission / reception unit 21 of the drone 1 to the transmission / reception unit 45 of the host vehicle 30.

The indexing unit 55 provided in the control unit 41 performs image processing and further parallax processing on an image in front of the host vehicle 30 acquired from the pair of imaging units 49a and 49a (stereo camera 49), and includes them in the image information. The three-dimensional position of the obstacle based on the size of the obstacle to be detected and the position of the host vehicle 30 is determined. Of course, the position of the drone 1 in front of the host vehicle 30 is also determined.
Further, the control unit 41 of the host vehicle 30 compares the size and three-dimensional position of the obstacle determined on the drone 1 side with the size and three-dimensional position of the obstacle determined on the host vehicle 30 side. A wide-area matching unit 57 is provided. By the collation in the wide area collation unit 57, the positions of all obstacles around the drone 1 including the drone 1 are recognized with the own vehicle 30 as a reference. Further, the wide-area collation unit 57 can collide with the drone 1 and the obstacle when the obstacle approaches the drone 1 or when the course of the drone 1 in flight changes (such as when the host vehicle 30 turns right or left). A drone that focuses on obstacles subject to collision based on the function of judging whether there is a possibility of collision or the position of an obstacle when it is judged that there is a possibility of collision, and also considers other obstacles. 1 has a function of instructing the direction of flight (flight position) in a direction that does not collide with an obstacle, that is, a direction (flight position) that avoids collision. Accordingly, when there is a possibility that the drone 1 and the obstacle collide, the drone 1 sends a flight instruction signal to the target flight direction (target flight position) while moving away from the obstacle to be collided. Is given (send). That is, the obstacle avoidance instruction unit 50 that avoids the drone 1 from the obstacle is configured by a combination of the wide area collation unit 57 and the indexing units 51 and 55.

Next, the operation of the driving support apparatus configured as described above will be described with reference to the flowchart shown in FIG. 2 and the traveling state of the host vehicle 30 shown in FIG.
For example, it is assumed that the driving support device receives help in order to perform driving in consideration of traffic conditions while the host vehicle 30 is traveling. At this time, the driver operates a take-off and landing operation unit (not shown) provided in the host vehicle 30. Then, the hold (hook mechanism) of the drone 1 on the heliport 35 is released, and each motor 5b is further driven.

  As a result, as shown in FIGS. 1 and 3, the drone 1 is driven by the control signal transmitted from the host vehicle 30 and is taken off from the heliport 35 by the propulsive force provided by the propeller 5a. Fly over a predetermined position in front of the host vehicle 30. Then, a front image of the host vehicle 30 captured by the stereo camera 7 mounted on the drone 1 (an image captured by the imaging unit 7a on one side) is presented to the driver of the host vehicle 30 through the display 37, and the driver is in a forward situation. Encourage appropriate driving.

During such driving assistance, the drone 1 and the host vehicle 30 acquire information on obstacles around the drone 1.
That is, in the drone 1, for example, as shown in step S1, through the stereo camera 7, information on obstacles around the drone 1, for example, on both sides of the road α as shown in FIGS. 3A and 3B, for example. Image information including image information of various obstacles such as the planted trees X1 to X4 and the small bird X5 flying in front of the drone 1 is acquired.

  This image information is subjected to processing such as image processing and parallax processing performed by the indexing unit 51, and the sizes of the trees X1 to X4 and the small bird X5 included in the image are determined three-dimensionally. Similarly, as shown in step S3, the positions of the trees X1 to X4 and the small bird X5 are also determined by the three-dimensional position by image processing, parallax processing, and the like. For example, as shown in FIG. 4, the positions of the trees X1 to X4 and the small bird X5 are divided by the coordinate positions represented by the left and right (x), front and rear (y), and height (z) of the three-dimensional coordinates centered on the drone 1. Is issued. In FIGS. 3A and 3B, e1 indicates the imaging range (obstacle detection range) of the stereo camera 7.

  In the host vehicle 30, for example, as shown in step S <b> 7, the information around the drone 1 is provided through the stereo camera 49, for example, immediately before or behind the drone 1 as shown in FIGS. Image information including various obstacles such as the trees X3 and X4 and the small bird X6 flying in the lower right rear of the drone 1 is acquired. The bird X6 is located at a position that cannot be detected from the drone 1 in flight, that is, at the blind spot of the drone 1.

  This image information is subjected to processing such as image processing and parallax processing in the indexing unit 55, and the sizes of the trees X3 and X4 and the small bird X6 included in the image are determined three-dimensionally. Similarly, as shown in step S9, the positions of the trees X3 and X4, the small bird X6, and the drone 1 in flight are also determined by the three-dimensional position by image processing, parallax processing, and the like. For example, as shown in FIG. 4, the positions of the trees X3, X4, the small bird X6, and the drone 1 are represented by the left and right (x), the front and rear (y), and the height (z) of the three-dimensional coordinates around the host vehicle 30. Calculated by coordinate position. In FIGS. 3A and 3B, e2 indicates an imaging range (obstacle detection range) of the stereo camera 49.

  In the following step S13, the size and three-dimensional position of the obstacle (trees X1 to X4, small bird X5) determined from the image of the stereo camera 7 (drone 1) and the image of the stereo camera 49 (own vehicle 30) are determined. The size of the obstacle (tree X3, X4, small bird X6) and the three-dimensional position are collated based on the position of the drone 1 and the position of the host vehicle 30. By this collation, as shown in FIG. 4, the positions of all obstacles (X1 to X6) are clarified around the host vehicle 30 (recognition).

  At this time, it is assumed that the host vehicle 30 receiving driving assistance makes a right turn in the arrow direction a in FIG. Then, the flight position of the drone 1 is changed in accordance with the own vehicle 30 that makes a right turn in response to the course change of the own vehicle 30. That is, the drone 1 changes the flight direction (flight position) to the right rear so as to catch up with a predetermined position in front of the host vehicle 30 turned right. When this course changes, the drone 1 and the obstacle may collide.

  That is, when an obstacle is detected only by image information from the drone 1 as in the prior art, obstacles X1 to X5 around the drone 1 are detected, but outside the obstacle detection range of the drone 1, that is, in the blind spot. The obstacle (small bird X6) in the lower right of the drone 1 is not detected. For this reason, if the flight direction of the drone 1 changes to the right rear, the drone 1 may collide with an obstacle (small bird X6).

  Therefore, the present embodiment detects an obstacle not only from the drone 1 but also from the host vehicle 30. That is, the positions of all the obstacles around the drone 1 including the blind spot of the drone 1 are covered by the detection by the stereo camera 7 of the drone 1 and the stereo camera 49 of the host vehicle 30. For this reason, when the flight direction of the drone 1 changes to the right rear as in step S15 and there is a possibility that the drone 1 and the obstacle collide, it is impossible with the drone 1 alone as in the subsequent step S17. Drone 1 instruction for avoiding obstacles of drone 1, for example, avoiding collision with obstacles (X1 to X5) within the detection range of drone 1 and obstacles (small bird X6) outside the detection range of drone 1 It is possible to set the flight direction (flight position).

  Here, for example, as shown by the arrow direction b in FIG. 3, the obstacle (small bird X6) is sandwiched in a direction toward the direction opposite to the drone 1, specifically, the obstacle (small bird X6) is wound in the far direction to the right rear. The direction of flight (flight position) is set. And the control signal of the flight direction (flight position) is transmitted from the own vehicle 30, and instruct | indicates the flight of the drone 1 like step S19. Thereby, even if the obstacle (bird X6) is located in the blind spot of the drone 1, the drone 1 follows the own vehicle 30 that has turned right without colliding with the obstacle (bird X6). Return to a predetermined position in front of the host vehicle 30 (return). And continue driving assistance again.

  As described above, the driving support device instructs the flight of the drone 1 in the direction of avoiding the obstacle by using not only the obstacle information obtained from the drone 1 but also the obstacle information obtained from the own vehicle 30. Therefore, even if it is difficult to detect an obstacle such as an obstacle located in the blind spot of the drone 1, the drone 1 can be caused to fly in a direction to avoid the obstacle (obstacle avoiding direction). For this reason, no matter what posture the drone 1 flies, the collision between the drone 1 and the obstacle can be avoided.

Therefore, the collision avoidance performance with respect to the obstacle of the drone during driving support can be improved by cooperative control by the drone 1 and the host vehicle 30.
In particular, the collision avoidance instruction for the drone 1 is performed after the positions of the obstacles captured by the drone 1 and the positions of the obstacles captured by the host vehicle 30 are collated to recognize the positions of all obstacles. Is expensive.

FIG. 5 shows a second embodiment of the present invention.
This is a parallel technique as in the first embodiment, in which the drone 1 is not instructed to avoid the collision, but the serial technique is instructed to avoid the collision. It is. FIG. 5 shows a flowchart of the control that is the main part.
That is, in the present embodiment, the position of the obstacle on the front side of the drone 1 is captured by the stereo camera 7 of the drone 1, and the obstacle at the blind spot of the drone 1 on the rear side of the drone 1 is detected by the stereo camera 49 of the host vehicle 30. An obstacle is caught and all obstacles are monitored (steps S1 to S11). When there is a possibility that the drone 1 collides with an obstacle, such as a change in the traveling direction of the host vehicle 30 (step S21), first, the image information captured by the stereo camera 49 (host vehicle 30) for the drone 1 is used. Based on the determined obstacle position information, a flight direction (flight position) for avoiding a collision with an obstacle located in the blind spot (outside the detection range) of the drone 1 is indicated. Thereafter, based on the position information of the obstacle determined from the image information captured by the stereo camera 7 (drone 1), the flight direction (flight position) is avoided so as to avoid a collision with an obstacle located within the detection range of the drone 1. ).

  During driving assistance, if the direction of the drone 1 changes (when there is a possibility of collision with an obstacle), many of them will behave toward the rear of the drone 1 first. After instructing to avoid the obstacle captured by the vehicle 30), the drone 1 can effectively avoid the obstacle by instructing to avoid the obstacle captured by the stereo camera 7 (drone 1). .

Even if it does in this way, there exists an effect similar to 1st Embodiment. However, in FIG. 5, the same parts as those of the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.
In addition, each structure, combination, etc. in embodiment mentioned above are examples, and it cannot be overemphasized that addition, omission, substitution, and other change of a structure are possible within the range which does not deviate from the meaning of the present invention. . Further, the present invention is not limited by the above-described embodiment, and it is needless to say that the present invention is limited only by the “claims”. For example, in the above-described embodiment, the position of an obstacle is detected using a stereo camera. However, the present invention is not limited to this, and the position of an obstacle is detected using, for example, a sonar that uses ultrasonic waves as a detection medium. You may make it.

1 drone (unmanned aerial vehicle)
7 Stereo camera (aircraft side detector)
30 Vehicle 37 Display (presentation part)
13, 41 Control unit 49 Stereo camera (vehicle side detection unit)
50 Obstacle avoidance instruction unit 51 Index unit (aircraft side position detection unit)
55 Indexing part (vehicle position detection part)
57 Wide-area verification unit

Claims (3)

An unmanned air vehicle with an imaging device capable of flying at a predetermined position ahead of the traveling direction of the host vehicle;
A presentation unit that presents an image ahead of the traveling direction of the host vehicle captured by the imaging device to an occupant of the host vehicle;
A vehicle driving support device comprising: a control unit that controls the unmanned air vehicle so that the unmanned air vehicle continues to fly at a predetermined position in front of the host vehicle;
The unmanned air vehicle has an air vehicle side detection unit that detects obstacles around the unmanned air vehicle,
The host vehicle has a vehicle-side detection unit that detects an obstacle ahead of the host vehicle in the traveling direction;
The controller is
Based on the obstacle detected by the vehicle-side detection unit and the obstacle detected by the vehicle-side detection unit, the vehicle has an obstacle avoidance instruction unit that instructs the flight of the unmanned air vehicle in a direction to avoid the obstacle. A vehicle driving support device characterized by the above. The obstacle avoidance instruction unit
A vehicle-side position detector that determines the position of an obstacle detected by the aircraft-side detector;
A vehicle side position detection unit for determining the position of the obstacle detected by the vehicle side detection unit;
A wide-area collation unit that collates the position of the obstacle determined by the vehicle-side position detection unit and the position of the obstacle calculated by the vehicle-side position detection unit and recognizes the position of all obstacles;
The vehicle driving support according to claim 1, wherein the unmanned air vehicle is instructed to fly in a flight direction avoiding the obstacle based on the positions of all obstacles recognized by the collation. apparatus. The obstacle avoidance instruction unit
A vehicle-side position detector that determines the position of an obstacle detected by the aircraft-side detector;
A vehicle-side position detection unit for determining the position of the obstacle detected by the vehicle-side detection unit;
After instructing the flight of the unmanned air vehicle to avoid the obstacle determined by the vehicle-side position detection unit, the unmanned air vehicle is arranged to avoid the obstacle determined by the vehicle-side position detection unit. The vehicle driving support apparatus according to claim 1, wherein the vehicle driving instruction is performed.

Images