JP2014000921A - Vehicle periphery monitoring device, vehicle periphery monitoring method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a vehicle periphery monitoring device for grasping presence/absence of an object distant upward from the road surface as an example.SOLUTION: A vehicle periphery monitoring device of an embodiment includes: an information acquisition part which acquires imaging data output from an imaging part which is provided to the vehicle to pick up the periphery of the vehicle, and a detection result output from a ranging part which is provided to the vehicle, emits waves to the imaging area of the imaging part to detect reflected waves of the waves; and a determination part which determines whether or not there is an object distant upward from the road surface in the imaging area by using the imaging data and the detection result.

Description

  Embodiments described herein relate generally to a vehicle periphery monitoring device, a vehicle periphery monitoring method, and a program.

  2. Description of the Related Art Conventionally, a technique for displaying a rear image of a vehicle imaged by a camera on a display device arranged in a vehicle interior is known.

JP 2002-29345 A

  However, with this type of technology, as an example, it is difficult to grasp the sense of distance from the vehicle in an image with respect to an object (aerial object) separated upward from the road surface. In this type of technology, as an example, it is desired that the presence or absence of an object separated upward from the road surface can be grasped.

  The vehicle periphery monitoring device according to the embodiment emits a wave to an imaging area of the imaging unit provided in the vehicle and imaging data output from an imaging unit provided in the vehicle and imaging the periphery of the vehicle. Using the detection result output from the distance measurement unit that detects the reflected wave, the information acquisition unit that acquires the imaging data, and the detection result, there is an object that is separated from the road surface in the imaging region And a determination unit for determining whether or not. Therefore, according to the vehicle periphery monitoring apparatus, as an example, it is possible to grasp the presence or absence of an object that is separated upward from the road surface.

  In the vehicle periphery monitoring device, when the determination unit determines that there is an object separated upward from the road surface, a notification unit provided in the vehicle that there is an object separated upward from the road surface You may provide the alerting | reporting control part to alert | report. According to this configuration, as an example, it is possible to notify the driver that there is an object away from the road surface, and thus it is possible to assist the driver in driving.

  In the vehicle periphery monitoring device, the determination unit determines whether there is an object grounded on a road surface in the imaging region, and the notification control unit is an object on which the determination unit is grounded on the road surface. Different notifications may be made to the notification unit when it is determined that there is an object and when the determination unit determines that there is an object away from the road surface. According to this configuration, as an example, notification when the determination unit determines that there is an object grounded on the road surface, and notification when the determination unit determines that there is an object separated upward from the road surface. And the driver can distinguish and recognize them well.

  In the vehicle periphery monitoring device, the notification unit may be a display device, and when the notification control unit determines that there is an object grounded on the road surface, When the display unit displays a panel display superimposed on an object grounded on the road surface in the image using the imaging data, and the determination unit determines that there is an object separated upward from the road surface, the display device, The panel display is overlapped and displayed on an object away from the road surface in the image using the imaging data, and an alarm indicating that there is an object away from the road surface is displayed in the image. Good. According to the said structure, the driver | operator can recognize favorably by the said warning that there exists an object left | separated upwards from the said road surface as an example.

  In the vehicle periphery monitoring device, the notification unit may be a display device, and when the notification control unit determines that there is an object grounded on the road surface, If the first panel display is superimposed on the object grounded on the road surface in the image using the imaging data, and the determination unit determines that there is an object away from the road surface, the display The apparatus may cause a second panel display different from the first panel display to be displayed in an overlapping manner on an object separated upward from the road surface in the image using the imaging data. According to this configuration, as an example, the driver can well recognize from the second panel display that there is an object separated upward from the road surface.

  In the vehicle periphery monitoring device, the determination unit obtains a first distance using the detection result, detects a determination target object from the imaging data, and determines the determination target object from the vehicle. When the second distance is acquired using the imaging data and there is a difference between the first distance and the second distance, it may be determined that there is an object away from the road surface. According to this configuration, as an example, it is possible to determine whether there is an object away from the road surface using the first distance and the second distance.

  In the vehicle periphery monitoring device, the determination unit may acquire the second distance on the assumption that a feature point of the object to be determined is located on the road surface. According to this configuration, as an example, the second distance can be acquired even when the feature point of the object to be determined is actually separated upward from the road surface.

  In the vehicle periphery monitoring device, the determination unit may determine that there is an object away from the road surface when the difference between the first distance and the second distance is equal to or greater than a threshold value. . According to this configuration, as an example, when determining whether there is an object that is away from the road surface, errors in the first distance and the second distance can be absorbed.

  In the vehicle periphery monitoring device, when the determination unit determines that there is an object separated upward from the road surface, the determination unit uses the first distance and the second distance to move upward from the road surface. The distance of the object far away from the road surface may be estimated. According to the said structure, the estimated distance from the said road surface of the object which left | separated upwards from the said road surface can be recognized.

  In the vehicle periphery monitoring method according to the embodiment, the information acquisition unit emits a wave to the imaging data provided from the imaging unit provided in the vehicle and imaging the periphery of the vehicle, and the imaging region provided in the vehicle. Then, a step of obtaining a detection result output from the distance measuring unit that detects a reflected wave of the wave, and a determination unit using the imaging data and the detection result from the road surface in the imaging region Determining whether there is an object separated upward. Therefore, according to the vehicle periphery monitoring method, as an example, the presence / absence of an object separated upward from the road surface can be grasped.

  The program according to the embodiment includes imaging data output from an imaging unit provided in a vehicle and imaging the periphery of the vehicle, and a reflected wave of the wave emitted from an imaging region of the imaging unit provided in the vehicle. A step of obtaining a detection result output from a distance measuring unit to be detected, and a step of determining whether or not there is an object located above the road surface in the imaging region using the imaging data and the detection result Is a program for causing a computer to execute. Therefore, according to the program, according to the vehicle periphery monitoring method, as an example, it is possible to grasp the presence or absence of an object separated upward from the road surface.

FIG. 1 is a perspective view showing an example of a vehicle according to the embodiment with a part of the vehicle cut away. FIG. 2 is a plan view illustrating an example of the vehicle according to the embodiment. FIG. 3 is a block diagram illustrating an example of a configuration of the vehicle according to the embodiment. FIG. 4 is a block diagram illustrating an example of a functional configuration of the vehicle according to the embodiment. FIG. 5 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the embodiment. FIG. 6 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the embodiment. FIG. 7 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the embodiment. FIG. 8 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the embodiment. FIG. 9 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the embodiment. FIG. 10 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the embodiment. FIG. 11 is a side view schematically showing the vehicle according to the embodiment and other vehicles. FIG. 12 is a flowchart illustrating an example of processing executed by the CPU of the vehicle according to the embodiment. FIG. 13 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the first modification of the embodiment. FIG. 14 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the first modified example of the embodiment. FIG. 15 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the second modification example of the embodiment. FIG. 16 is a diagram illustrating an example of an image displayed on the display device of the vehicle according to the third modification example of the embodiment.

  In the present embodiment, the vehicle 1 may be, for example, an automobile (an internal combustion engine automobile) using an internal combustion engine (engine, not shown) as a drive source, or an electric motor (motor, not shown) as a drive source. The vehicle may be a vehicle (electric vehicle, fuel cell vehicle, etc.) or a vehicle (hybrid vehicle) using both of them as drive sources. The vehicle 1 can be mounted with various transmissions, and various devices (systems, components, etc.) necessary for driving the internal combustion engine and the electric motor. In addition, the method, number, layout, and the like of the device related to driving of the wheels 3 in the vehicle 1 can be variously set.

  As shown in FIG. 1, the vehicle body 2 constitutes a passenger compartment 2a in which an occupant (not shown) rides. In the passenger compartment 2a, a steering section 4, an acceleration operation section 5, a braking operation section 6, a shift operation section 7 and the like are provided in a state facing the driver's seat 2b as a passenger. In the present embodiment, as an example, the steering unit 4 is a steering wheel protruding from a dashboard (instrument panel), the acceleration operation unit 5 is an accelerator pedal positioned under the driver's feet, and a braking operation unit Reference numeral 6 denotes a brake pedal positioned under the driver's feet, and the shift operation unit 7 is a shift lever protruding from the center console, but is not limited thereto.

  Further, a display device 8 (display output unit) and a sound output device 9 (sound output unit) are provided in the passenger compartment 2a. The display device 8 is, for example, an LCD (liquid crystal display) or an OELD (organic electroluminescent display). The audio output device 9 is a speaker as an example. In the present embodiment, as an example, the display device 8 is covered with a transparent operation input unit 10 (for example, a touch panel). An occupant or the like can visually recognize a video (image) displayed on the display screen of the display device 8 via the operation input unit 10. A passenger or the like operates an operation input (instruction) by touching, pushing, or moving the operation input unit 10 with a finger or the like at a position corresponding to an image (image) displayed on the display screen of the display device 8. Input). In the present embodiment, as an example, the display device 8, the audio output device 9, the operation input unit 10, and the like are provided in the monitor device 11 that is located in the center of the dashboard in the vehicle width direction (left-right direction). ing. The monitor device 11 can include an operation input unit (not shown) such as a switch, a dial, a joystick, and a push button. In addition, an audio output device (not shown) can be provided at another position in the passenger compartment 2a different from the monitor device 11, and audio is output from the audio output device 9 of the monitor device 11 and other audio output devices. Can be output. In the present embodiment, as an example, the monitor device 11 is also used as a navigation system or an audio system. However, a monitor device for a parking assistance device may be provided separately from these systems. Further, in addition to the audio output device 9, an alarm sound or the like can be output from an audio output unit such as the buzzer 24 (see FIG. 3).

  As shown in FIGS. 1 and 2, in the present embodiment, as an example, the vehicle 1 is a four-wheeled vehicle (four-wheeled vehicle), and includes two left and right front wheels 3F and two left and right rear wheels 3R. Have. Furthermore, in the present embodiment, these four wheels 3 are configured so that any of them can be steered (turnable). Specifically, as shown in FIG. 3, the vehicle 1 includes a front wheel steering system 12 that steers the front wheels 3F and a rear wheel steering system 13 that steers the rear wheels 3R. The front wheel steering system 12 and the rear wheel steering system 13 are electrically controlled by a parking assist ECU 14 (electronic control unit) or the like to operate the actuators 12a and 13a. The front wheel steering system 12 and the rear wheel steering system 13 are, for example, an electric power steering system, an SBW (steer by wire) system, or the like. The front wheel steering system 12 and the rear wheel steering system 13 add torque (assist torque) to the steering unit 4 by the actuators 12a and 13a to supplement the steering force, or steer the corresponding wheel 3 (front wheel 3F or rear wheel 3R). (Automatic steering). The actuators 12a and 13a may steer one wheel 3 or may steer a plurality of wheels 3. In the present embodiment, as an example, the two front wheels 3F are steered in the same phase and substantially parallel, and the two rear wheels 3R are steered in the same phase and substantially parallel. The drive wheels can be set in various ways.

  In the present embodiment, as an example, a two-wheel steering mode (first steering mode) in which only the front wheels 3F are steered and a four-wheel steering mode (second steering mode) in which the front wheels 3F and the rear wheels 3R are steered are exemplified. And are set. The steering mode is switched by an operation of the operation input unit 10 by a passenger, an operation of a steering mode switching operation unit 15 (for example, a switch or the like, see FIG. 3) provided in the passenger compartment 2a, and the like. In the present embodiment, as an example, with respect to parking assistance, in the four-wheel steering mode, the front wheels 3F and the rear wheels 3R are steered in opposite phases. The front wheel 3F can be mechanically steered by the operation of the steering unit 4.

  In the present embodiment, as an example, as shown in FIG. 2, the vehicle 1 (the vehicle body 2) is provided with a plurality of (in the present embodiment, four as an example) imaging units 16 (16 a to 16 d). It has been. The imaging unit 16 is a digital camera including an imaging element such as a CCD (charge coupled device) or a CIS (CMOS image sensor). The imaging unit 16 is a monocular camera as an example. The imaging unit 16 is an example of a passive distance sensor. The imaging unit 16 can output image data (moving image data, frame data) at a predetermined frame rate. Each of the imaging units 16 includes a wide-angle lens, and can capture a range (viewing angle) of 140 ° to 190 ° in the horizontal direction. Further, the optical axis of the imaging unit 16 is set downward (obliquely downward). Therefore, the imaging unit 16 can photograph the periphery of the vehicle body 2 (vehicle 1) including the road surface. In the present embodiment, as an example, the imaging unit 16a is located at the end 2c (the end in a plan view) on the front side (the front side in the vehicle front-rear direction) of the vehicle body 2, and is provided on a front bumper or the like. The imaging unit 16b is positioned at the left end 2d (left side in the vehicle width direction) of the vehicle body 2, and is provided on the left door mirror 2g (projecting portion). The imaging unit 16c is located at an end 2e on the rear side (rear side in the vehicle front-rear direction) of the vehicle body 2, and is provided on a wall portion below the door 2h of the rear trunk. The imaging unit 16d is positioned on the right end (figure width direction) end 2f of the vehicle body 2 and is provided on the right door mirror 2g (projection). The parking assist ECU 14 performs arithmetic processing and image processing based on the image data obtained by the plurality of imaging units 16 to generate an image with a wider viewing angle, or viewed the vehicle 1 (vehicle body 2) from above. A virtual overhead view image (planar image) can be generated. In addition, the parking assist ECU 14 can detect (extract) an object (obstacle) that may interfere with the moving vehicle 1 located around the vehicle 1 from the image of the imaging unit 16d. Therefore, the imaging unit 16 is an example of an object detection unit.

  In the present embodiment, as an example, as shown in FIGS. 1 and 2, the vehicle 1 (the vehicle body 2) includes a plurality of (in the present embodiment, four as an example) distance measuring units 17 (17 a to 17 d). , A detection unit) is provided. The distance measuring unit 17 is, for example, a sonar (sonar sensor, ultrasonic detector, active distance sensor) that emits (radiates) ultrasonic waves as waves and captures (detects) the reflected waves. The parking assist ECU 14 can measure the presence / absence and distance of an object (obstacle) located behind the vehicle 1 (vehicle body 2) based on the detection result of the distance measuring unit 17. That is, the distance measurement unit 17 is an example of an object detection unit.

  In the present embodiment, as an example, as shown in FIG. 3, in the parking assistance system 100, the parking assistance ECU 14, the monitor device 11, the front wheel steering system 12, the rear wheel steering system 13, the steering mode switching operation unit 15. In addition to the distance measuring unit 17 and the like, the brake system 18, the steering angle sensor 19 (angle sensor), the accelerator sensor 20, the shift sensor 21, the wheel speed sensor 22 and the like are electrically connected via the in-vehicle network 23 (electric communication line). It is connected to the. The in-vehicle network 23 is configured as a CAN (controller area network) as an example. The parking assist ECU 14 can control the front wheel steering system 12, the rear wheel steering system 13, the brake system 18 and the like by sending a control signal through the in-vehicle network 23. The parking assist ECU 14 also has a torque sensor 12b, a rudder angle sensor 13b (for the rear wheel 3R), a distance measuring unit 17, a brake sensor 18b, a rudder angle sensor 19 (for the front wheel 3F), an accelerator sensor via the in-vehicle network 23. 20, the detection results of the shift sensor 21, the wheel speed sensor 22, and the like, and the instruction signals (control signal, switching signal, operation signal, input signal, data) of the operation input unit 10 and the steering mode switching operation unit 15 are received. be able to.

  As an example, the parking assist ECU 14 includes a CPU 14a (central processing unit), a ROM 14b (read only memory), a RAM 14c (random access memory), a display control unit 14d, an audio control unit 14e, an SSD 14f (solid state drive, flash memory), and the like. have. For example, the CPU 14a can execute various kinds of calculation processing such as image processing related to an image displayed on the display device 8, calculation of a moving route of the vehicle 1, and determination of presence / absence of interference with an object. The CPU 14a can read a program stored (installed) in a nonvolatile storage device such as the ROM 14b and execute arithmetic processing according to the program. The RAM 14c temporarily stores various types of data used in computations by the CPU 14a. In addition, the display control unit 14d mainly performs image processing using image data obtained by the imaging unit 16 and image processing of image data displayed on the display device 8 (one example). And so on). In addition, the voice control unit 14 e mainly performs processing of voice data output from the voice output device 9 among the calculation processes in the parking assistance ECU 14. Further, the SSD 14f is a rewritable nonvolatile storage unit, and can store data even when the parking assist ECU 14 is powered off. The CPU 14a, the ROM 14b, the RAM 14c, and the like can be integrated in the same package. Further, the parking assist ECU 14 may have a configuration in which another logical operation processor such as a DSP (digital signal processor) or a logic circuit is used instead of the CPU 14a. Further, an HDD (hard disk drive) may be provided in place of the SSD 14f, and the SSD 14f and the HDD may be provided separately from the parking assist ECU 14. As an example, the parking assist ECU 14 constitutes the vehicle periphery monitoring device 101 together with the imaging unit 16 and the distance measuring unit 17.

  The brake system 18 is an ABS (anti-lock brake system) that suppresses the locking of the brake, a skid prevention device (ESC: electronic stability control) that suppresses the skidding of the vehicle 1 during cornering, and increases the braking force (brake assist). An electric brake system, BBW (brake by wire), etc. The brake system 18 gives a braking force to the wheel 3 (vehicle 1) via the actuator 18a.

  The rudder angle sensor 19 is a sensor that detects a steering amount (rotation angle) of the steering unit 4 (in this embodiment, a steering wheel as an example), and is configured by using a Hall element or the like as an example. The rudder angle sensor 13b is a sensor that detects the steering amount (rotation angle) of the rear wheel 3R, and is configured using a Hall element as an example. The parking assistance ECU 14 acquires the steering amount of the steering unit 4 by the driver, the steering amount of each wheel 3 during automatic steering, and the like from the steering angle sensors 19 and 13b and executes various controls. The torque sensor 12b detects torque that the driver gives to the steering unit 4.

  The wheel speed sensor 22 is a sensor that detects the amount of rotation of the wheel 3 and the number of rotations per unit time, and is configured using a Hall element as an example. The parking assist ECU 14 calculates the amount of movement of the vehicle 1 based on the data acquired from the wheel speed sensor 22 and executes various controls. The wheel speed sensor 22 may be provided in the brake system 18. The brake system 18 can execute various controls by detecting brake lock, idle rotation of the wheels 3, signs of skidding, and the like from the difference in rotation between the left and right wheels 3. When the wheel speed sensor 22 is provided in the brake system 18, the parking assistance ECU 14 acquires data via the brake system 18. The brake sensor 18 b is a sensor that detects the amount of operation of the brake pedal, and the parking assist ECU 14 acquires information via the brake system 18. For example, when the braking operation unit 6 is operated during automatic steering, the parking assist ECU 14 can interrupt or cancel the automatic steering because it is not suitable for automatic steering.

  As an example, the shift sensor 21 is a sensor (switch) that detects the position of a movable part (a lever, an arm, or a button) of the speed change operation unit 7, and is configured using a displacement sensor or the like. For example, the parking assist ECU 14 can start the assist control when the movable part is set to reverse, or can terminate the assist control when changed from reverse to forward.

  The configuration, arrangement, electrical connection form, and the like of the various sensors and actuators described above are merely examples, and can be set (changed) in various ways.

  Next, a vehicle rear monitoring process (vehicle rear monitoring method) executed by the CPU 14a will be described. The vehicle rear monitoring process (vehicle rear monitoring method) is an example of a vehicle periphery monitoring process (vehicle periphery monitoring method). As an example, the vehicle rear monitoring process is started when the movable portion of the speed change operation unit 7 is set to reverse. In the vehicle rear monitoring process, the CPU 14a functions as an information acquisition unit 51, a route guide unit 52, a determination unit 53, and a notification control unit 54 as shown in FIG.

  The information acquisition unit 51 acquires information from each unit of the vehicle 1. As an example, the information acquisition unit 51 acquires (receives) the imaging data output from the imaging unit 16c and the detection results output from the distance measurement unit 17 (17a to 17d). Here, the imaging unit 16 c is an example of an imaging unit that is provided in the vehicle 1 and images the periphery of the vehicle 1. The distance measuring unit 17 is a distance measuring unit that is provided in the vehicle 1 and that detects (receives and captures) a reflected wave of the wave by emitting a wave (ultrasonic wave in the present embodiment) to the imaging region of the imaging unit 16c. It is an example. Note that the distance measurement unit may detect a reflected wave by emitting a radio wave as a wave. The detection result of the distance measuring unit 17 is, for example, a time until the reflected wave is captured after the distance measuring unit 17 emits an ultrasonic wave. The information acquisition unit 51 acquires (receives) the detection results of the steering angle sensors 13b and 19.

  5, 6, and 7 show examples of images (videos captured by the imaging unit 16 c) displayed on the display screen of the display device 8 based on the imaging data output from the imaging unit 16 c. The image (video) shown in FIG. 5 displays the rear end 2e of the vehicle body 2, the road surface 201 (surface, ground, reference surface), white line 202, car stop 203, wall 204 (object), and the like. ing. In the image shown in FIG. 6, a rear end 2 e of the vehicle body 2, a road surface 201, a white line 202, a front end 205 a (object) of another vehicle 205, and the like are displayed. A bumper 205 b is included in the front end portion 205 a of the other vehicle 205. In the image shown in FIG. 7, a rear end 2e of the vehicle body 2, a road surface 201, a guard rail 206, a sign 208 (plate), and the like are displayed. The guard rail 206 includes a strip-shaped plate portion 206a and a support column 206b that supports the plate portion 206a. As an example, the sign 208 is supported by an inclined column 207.

  The route guide unit 52 calculates the expected trajectory of the vehicle 1 using the detection results of the steering angle sensors 13b and 19 acquired by the information acquisition unit 51. The route guide unit 52 superimposes (combines) the calculated predicted trajectory of the vehicle 1 on the image captured by the image capturing unit 16c and displayed on the display device 8. Here, FIGS. 8, 9, and 10 show images in which the expected trajectory 301 is superimposed on the images of FIGS. In addition, a distance guide line 302 is displayed in the images of FIGS. The distance guide line 302 is displayed on the display device 8 by the route guide unit 52. A distance guide line 302 indicates a guide of the distance from the rear end 2e of the vehicle body 2.

  As an example, the determination unit 53 uses an imaging data of the imaging unit 16c and a detection result of the distance measuring unit 17 to provide an object (object, obstacle) behind the vehicle 1 (periphery, traveling direction). , Structure) is determined (detected). In the present embodiment, the determination unit 53 uses the imaging data of the imaging unit 16c and the detection result of the distance measuring unit 17 to determine whether there is an object away from the road surface in the imaging region of the imaging unit 16c. judge. For example, the determination unit 53 determines whether or not there is an object away from the road surface within the detectable range of the distance measurement unit 17. The determination unit 53 determines whether there is an object grounded on the road surface in the imaging region of the imaging unit 16c. For example, the determination unit 53 determines whether there is an object grounded on the road surface within the detectable range of the distance measurement unit 17.

  The determination unit 53 acquires the first distance (distance) using the detection result of the distance measurement unit 17. The first distance is the distance from the vehicle 1 of the object to be determined calculated using the detection result of the distance measuring unit 17. For example, the first distance is calculated using the detection time included in the detection result of the distance measuring unit 17 and the sound speed. Further, the determination unit 53 detects (extracts) the object to be determined from the image data of the image capturing unit 16c, and determines the second distance that is the distance from the vehicle 1 of the object to be determined as the image data of the image capturing unit 16c. Use to obtain (calculate). If there is a difference between the first distance and the second distance, the determination unit 53 determines that there is an object that is away from the road surface.

  A method for calculating the second distance will be described. The second distance is a distance (number of pixels) from an end 2e of the vehicle 1 (vehicle body 2) on the image to a certain position (coordinates) on the road surface in the image, and an actual distance corresponding to the distance (number of pixels). It can be calculated using the corresponding distance information indicating the relationship with the actual distance. At this time, feature points (contours as an example) of the object in the image data are extracted (detected). For example, the feature points of the object can be extracted from the luminance difference of each pixel. Then, the position (coordinates) of the road surface at the position in contact with the lower end of the feature point (contour) of the object is specified. Next, a distance on the image between the position of the specified road surface and the end 2e of the vehicle 1 (vehicle body 2) is calculated, and a second distance is obtained from the calculated distance and the corresponding distance information.

  In the second distance calculation method described above, the object grounded on the road surface (for example, the wall 204 shown in FIG. 5) has a relatively high accuracy because the lower end of the object is grounded on the road surface. Thus, the second distance can be obtained, and the second distance and the first distance substantially coincide with each other. On the other hand, the accuracy of the second distance is low with respect to an object away from the road surface (for example, the front end portion 205a of the other vehicle 205 shown in FIG. 6 or the plate portion 206a of the guard rail 206 shown in FIG. 7). . This is because the portion of the road surface that is in contact with the lower end portion of the object that is separated from the road surface on the image is actually the portion that is further away from the end portion 2e of the vehicle 1 than the lower end portion of the object. For example, as shown in FIG. 11, when the rear of the vehicle is imaged by the imaging unit 16c, the road surface 201 at the position P1 behind the lower end 205b is connected to the lower end 205b of the front end 205a of the other vehicle 205. The image is taken as it is. For this reason, when the second distance of the front end 205a (object) away from the road surface is calculated by the second distance calculation method, the position P1 is behind the lower end 205b of the front end 205a (object). Is calculated. This calculated distance is longer than the actual distance, and a relatively large error occurs with respect to the actual distance. In this embodiment, paying attention to this error, it is determined whether or not the object is in contact with the ground. That is, if the second distance and the first distance substantially match, it is determined that the object is in contact with the road surface. If the second distance and the first distance are different, the object is separated from the road surface. Determine that you are away. At this time, in the present embodiment, as an example, the determination unit 53 determines that there is an object that is separated upward from the road surface when the difference between the first distance and the second distance is equal to or greater than the threshold value. The threshold value is a value larger than zero. In the above method, the determination unit 53 acquires the second distance on the assumption that the feature point of the object to be determined is located on the road surface.

  If the determination unit 53 determines that there is an object away from the road surface, the determination unit 53 estimates the distance from the road surface of the object away from the road surface using the first distance and the second distance ( calculate. At this time, in this embodiment, a projection (perspective projection) method is used as an example.

  A method of calculating the distance from the road surface of an object that is away from the road surface will be described with reference to FIG. When viewed from the side of the imaging unit 16c (vehicle 1), a straight line L1 that passes from the imaging unit 16c (lens) through the lower end 205c of the front end 205a (object) of another vehicle 205 to the road surface 201, and the straight line A straight line L2 (a straight line having a length X1) obtained by projecting L1 onto the road surface 201 and a straight line L3 connecting an end of the straight line L1 on the imaging unit 16c side and an end of the straight line L2 on the imaging unit 16c side are a right triangle M1. Is generated. The intersection point P1 between the straight line L1 and the road surface 201 corresponds to a position away from the rear end 2e of the vehicle 1 by a first distance. Then, a vertical line L4 with respect to the road surface 201 is generated at the position P2 of the object based on the measurement result of the distance measuring unit 17 (a position away from the end 2e of the vehicle 1 by a second distance backward). The vertical line L4 intersects with the straight line L1. As a result, a right triangle M2 is generated by the straight lines L1, L2 and the vertical line L4. These right triangles M1 and M2 have similar shapes. Then, assuming that the length X1 of the straight line L2 is the second distance, the height of the straight line L3 is obtained. Further, it is assumed that the length X2 between the straight line L3 and the vertical line L4 is the first distance. Using the conditions that the height of these straight lines L3, the lengths X1 and X2, and the right triangles M1 and M2 are similar, the height of the portion L4a between the straight lines L1 and L2 in the straight lines L4. (Length) can be obtained. The height of the portion L4a is the distance from the road surface 201 of the object (front end portion 205a) that is spaced upward from the road surface. A region indicated by a line L5 in FIG. 11 schematically shows a detectable range of the distance measuring unit 17.

  When the determination unit 53 determines (detects) an object such as an obstacle behind the vehicle 1 (periphery, traveling direction), the notification control unit 54 has an object such as an obstacle behind the vehicle 1. The display device 8 is notified of this. Specifically, when the determination unit 53 determines that there is an object that is separated upward from the road surface of the vehicle 1, the notification control unit 54 is an object that is separated upward from the road surface behind the vehicle 1 (periphery, traveling direction). The display device 8 provided in the vehicle 1 is notified that there is. Further, when the determination unit 53 determines that there is an object grounded on the road surface, the notification control unit 54 indicates that there is an object away from the road surface behind the vehicle 1 (periphery, traveling direction). 1 is notified to the display device 8 provided in 1. That is, the notification control unit 54 displays different notifications when the determination unit 53 determines that there is an object that is grounded on the road surface and when the determination unit determines that there is an object that is away from the road surface. Let the device 8 do. The display device 8 is an example of a notification unit.

  Specifically, when the determination unit 53 determines that there is an object grounded on the road surface, the notification control unit 54 uses the image data of the imaging unit 16c on the display device 8 as illustrated in FIG. A panel display 303 (highlighted display) is superimposed and displayed on an object (the wall 204 in FIG. 8) that is grounded on the road surface. As an example, the panel display 303 has a rectangular shape and is colored. As an example, the color of the panel display 303 is different from the predicted trajectory 301 and the distance guide line 302. Moreover, the panel display 303 has transparency, and the part of the object (wall 204) on which the panel display 303 is superimposed is visible. The panel display 303 is a set of two (plural) sheets, and the two sets are arranged at intervals in the left-right direction. The two panel displays 303 in one set are arranged so as to be shifted from each other so that a part of each other overlaps and another part does not overlap each other. The color of the overlapping portion of the two panel displays 303 is darker than the color of the single panel display 303.

  On the other hand, when the determination unit 53 determines that there is an object separated upward from the road surface, the notification control unit 54 uses the imaging data of the imaging unit 16c on the display device 8 as shown in FIGS. A panel display 303 (highlighted display) is superimposed on an object away from the road surface in the displayed image, and an alarm 304 (warning, display) indicating that there is an object away from the road surface is displayed in the image. To display. The alarm 304 is an example of information for a non-grounded object. 9 is the front end portion 205a of another vehicle 205 in FIG. 9, and the plate portion 206a of the guard rail 206 in FIG. The panel display 303 is the same as that described in FIG. The alarm 304 includes a symbol as an example. The symbol is, for example, an exclamation mark or the like. As an example, the alarm 304 is configured by arranging symbols in a triangular display. For example, one alarm 304 is arranged in each set of the panel display 303. For example, the alarm 304 is displayed so as to overlap with an object (a front end portion 205a of another vehicle 205 or a plate portion 206a of a guard rail 206). In the present embodiment, the panel display 303 without the alarm 304 indicates that there is an object whose determination unit 53 is grounded on the road surface. The panel display 303 without the alarm 304 is an example of information for grounded objects.

  Next, the flow of the vehicle rear monitoring process will be described with reference to the flowchart shown in FIG. First, the information acquisition part 51 acquires the information from each part of the vehicle 1 containing the detection result of the ranging part 17, and the imaging data of the imaging part 16c (step S1).

  Next, the determination unit 53 calculates a first distance that is the distance from the vehicle 1 (vehicle body 2) of the object to be determined from the detection result of the distance measuring unit 17 acquired in step S1 (step S2). The second distance, which is the distance from the vehicle 1 (vehicle body 2) of the object to be determined, is calculated from the imaging data of the imaging unit 16c acquired in step S1 (step S3). Next, the determination unit 53 determines whether or not the object to be determined is in contact with the road surface (step S4).

  When the determination unit 53 determines that the object is in contact with the road surface (Yes in step S4), the notification control unit 54 displays information for the grounded object (panel display 303 without the alarm 304) on the display device 8. (Step S5). On the other hand, when the determination unit 53 determines that the object is away from the road surface and the object is not in contact with the ground (No in step S4), the notification control unit 54 displays the information (alarm 304) for the non-grounded object on the display device. 8 is displayed (step S6). The above processing is repeated at a specified cycle.

  As described above, in the present embodiment, the determination unit 53 uses the imaging data output from the imaging unit 16c and the detection result of the distance measuring unit 17 to move upward in the imaging area of the imaging unit 16c from the road surface. Determine if there is a distant object. Therefore, according to the present embodiment, it is possible to grasp the presence / absence of an object separated upward from the road surface.

  Further, in the present embodiment, when the determination unit 53 determines that there is an object separated upward from the road surface, the notification control unit 54 indicates that there is an object separated upward from the road surface. The device 8 (notification unit) is informed. Therefore, according to the present embodiment, it is possible to notify the driver that there is an object separated upward from the road surface. Therefore, the driving assistance of the driver can be performed. Note that the notification unit is not limited to the display device 8. For example, the notification unit may be an audio output unit such as the audio output device 9 or the buzzer 24. In this case, the fact that there is an object away from the road surface is notified by voice (sound).

  In the present embodiment, the notification control unit 54 includes a case where the determination unit 53 determines that there is an object grounded on the road surface, and a case where the determination unit 53 determines that there is an object separated from the road surface. Then, the display device 8 is caused to make a different notification. Therefore, the driver can distinguish and recognize these notifications well.

  Next, a modification of this embodiment will be described. In addition, the following several modified examples include the component similar to the said embodiment. Therefore, in the following, common reference numerals are given to those similar components, and redundant description is omitted.

  In the first modification shown in FIGS. 13 and 14, when the determination unit 53 determines that there is an object away from the road surface, the notification control unit 54 uses the image data on the display device 8. A panel display 303A different from the panel display 303 (see FIG. 8) is displayed on an object that is distant from the inside road surface (in FIG. 13, the front end portion 205a of another vehicle 205, and in FIG. 14, the plate portion 206a of the guard rail 206). Is different from the above-described embodiment in that they are displayed in an overlapping manner. The panel display 303 is an example of a first panel display, and the panel display 303A is an example of a second panel display. As an example, the panel display 303A has a different color from the panel display 303. The arrangement of the panel display 303A is the same as that of the panel display 303. In the present modification, the panel display 303A indicates that there is an object separated upward from the road surface.

  In the second modification shown in FIG. 15, when the determination unit 53 determines that there is an object away from the road surface, the notification control unit 54 displays panel displays 303 and 303B and an alarm 304. Let In this modification, the panel display 303 has a rectangular shape having a longitudinal direction in the vertical direction. The panel display 303B extends from the lower end of the panel display 303 toward the rear end 2e of the vehicle body 2 in the image. The panel display 303B has a rectangular shape as an example, and is displayed on the road surface 201 in an overlapping manner. Further, an emphasis line 305 (line, emphasis part) is displayed on the edge (end part) of the panel displays 303 and 303B. The highlight line 305 makes it easier for the driver to visually recognize the panel displays 303 and 303B. In this modification, a go line 306 (frame) is displayed. A surrounding line 306 surrounds a sign 208 positioned above the guardrail 206. In this modification, a distance (for example, 1.0 m) is displayed for one distance guide line 302 (for example, the middle distance guide line 302 of the three).

  In the third modified example shown in FIG. 16, when the determination unit 53 determines that there is an object away from the road surface, the notification control unit 54 displays one panel display 303 and an alarm 304. . In this modification, the panel display 303 has a rectangular shape having a longitudinal direction along the plate portion 206a of the guard rail 206. In the present modification, as in the second modification, a distance (as an example, 1.0 m) with respect to one distance guide line 302 (as an example, the middle distance guide line 302 among the three). Is displayed.

  Even in each of the above-described modifications, the same effects as those of the above embodiment can be obtained.

  Note that the present invention is not limited to the above-described embodiments and modifications, and various other embodiments and modifications can be adopted without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Vehicle, 8 ... Display apparatus (notification part), 9 ... Audio | voice output apparatus (notification part), 16c ... Imaging part, 17 ... Distance measuring part, 24 ... Buzzer (notification part), 51 ... Information acquisition part, 53 ... Determining section 54 ... Notification control section 101 101 Vehicle periphery monitoring device 201 Road surface 204 Wall (object) 205a Front end of other vehicle (object) 206a Guardrail plate section (object) 303 Panel display (first panel display), 303A ... Panel display (second panel display), 304 ... Alarm.

Claims (11)

From imaging data output from an imaging unit that is provided in a vehicle and images the periphery of the vehicle, and a ranging unit that is provided in the vehicle and detects a reflected wave of the wave by emitting a wave to an imaging region of the imaging unit An information acquisition unit for acquiring the output detection result;
Using the imaging data and the detection result, a determination unit that determines whether there is an object separated upward from a road surface in the imaging area;
A vehicle periphery monitoring device.   And a notification control unit configured to notify a notification unit provided in the vehicle that there is an object separated upward from the road surface when the determination unit determines that there is an object separated upward from the road surface. Item 4. The vehicle periphery monitoring device according to Item 1. The determination unit determines whether there is an object grounded on a road surface in the imaging region,
The notification control unit notifies different notifications when the determination unit determines that there is an object grounded on the road surface and when the determination unit determines that there is an object separated upward from the road surface. The vehicle periphery monitoring device according to claim 2, wherein the vehicle periphery monitoring device is caused to be a notification unit. The notification unit is a display device,
When the determination unit determines that there is an object grounded on the road surface, the notification control unit superimposes a panel display on the object grounded on the road surface in the image using the imaging data. When the determination unit determines that there is an object separated upward from the road surface, the display device displays the panel display on the object distant from the road surface in the image using the imaging data. The vehicle periphery monitoring device according to claim 3, wherein the vehicle periphery monitoring device displays the alarm in the image while indicating that there is an object separated upward from the road surface. The notification unit is a display device,
When the determination unit determines that there is an object grounded on the road surface, the notification control unit displays a first panel on the object grounded on the road surface in the image using the imaging data. When the display is overlapped and the determination unit determines that there is an object that is separated upward from the road surface, the display device displays the object that is separated upward from the road surface in the image using the imaging data. The vehicle periphery monitoring apparatus according to claim 3, wherein a second panel display different from the first panel display is displayed in an overlapping manner.   The determination unit acquires a first distance using the detection result, detects an object to be determined from the imaging data, and uses a second distance from the vehicle of the object to be determined using the imaging data. 6. The vehicle periphery according to claim 1, wherein when there is a difference between the first distance and the second distance, it is determined that there is an object separated upward from the road surface. Monitoring device.   The vehicle periphery monitoring device according to claim 6, wherein the determination unit acquires the second distance on the assumption that a feature point of the determination target object is located on the road surface.   The vehicle periphery monitoring device according to claim 6 or 7, wherein the determination unit determines that there is an object separated upward from the road surface when a difference between the first distance and the second distance is a threshold value or more. .   When the determination unit determines that there is an object separated upward from the road surface, the distance from the road surface of the object separated upward from the road surface using the first distance and the second distance. The vehicle periphery monitoring device according to any one of claims 6 to 8, wherein An information acquisition unit detects the reflected data of the wave by emitting the wave to the imaging area of the imaging unit provided in the vehicle and the imaging data output from the imaging unit provided in the vehicle and imaging the periphery of the vehicle A detection result output from the distance measuring unit to obtain,
A step of determining, by using the imaging data and the detection result, whether or not there is an object separated upward from a road surface in the imaging area;
A vehicle periphery monitoring method including: From imaging data output from an imaging unit that is provided in a vehicle and images the periphery of the vehicle, and a ranging unit that is provided in the vehicle and detects a reflected wave of the wave by emitting a wave to an imaging region of the imaging unit Obtaining an output detection result; and
Using the imaging data and the detection result to determine whether or not there is an object away from the road surface in the imaging area;
A program that causes a computer to execute.

Images