JP2023120523A - Periphery monitoring device and periphery monitoring method - Google Patents

Abstract

To provide a periphery monitoring device and a periphery monitoring method that can more reliably alert a driver to an obstacle even during driving support.SOLUTION: A periphery monitoring device of an embodiment monitors a periphery of a vehicle having a sensor which collects peripheral information, wherein a display control unit: displays a first mark including a first pointer pointing to an obstacle, a first signal disposed within the first pointer, and a second signal disposed around the first pointer, superimposed on the obstacle in a peripheral image; displays a second mark including a second pointer of the same color as the first pointer, superimposed on a ground position of the obstacle in an overhead image; dynamically displays the first signal such that a transparency of the first signal changes periodically; and dynamically displays the second signal such that the second signal rotates around the first pointer.SELECTED DRAWING: Figure 4A

Description

The present invention relates to a perimeter monitoring device and a perimeter monitoring method.

2. Description of the Related Art There is a driving support device that provides driving support when a vehicle is parked and when it leaves the garage. In the driving support device, the moving route of the vehicle to entering (parking) or leaving the garage is calculated and presented to the driver. While the vehicle is moving, a sensor attached to the vehicle detects obstacles and the like around the vehicle, and the surroundings are monitored by, for example, visually alerting the driver.

JP 2017-161060 A JP 2017-129896 A JP 2017-122453 A JP 2018-148597 A JP 2015-187806 A

In the perimeter monitoring technology, obstacles are displayed in yellow, red, or other colors to make them visually stand out. However, it may be difficult to pay attention to a display device or the like in the vehicle and determine whether or not there is an obstacle during complicated driving operations such as entering or exiting the vehicle.

SUMMARY OF THE INVENTION The present invention has been made in view of the above, and provides a surroundings monitoring device and a surroundings monitoring method that can more reliably call the driver's attention to obstacles even during driving assistance.

In order to solve the above-described problems and achieve the object, the surroundings monitoring device of the embodiment is a surroundings monitoring device for monitoring the surroundings of a vehicle having a sensor that collects surrounding information. a detection unit for detecting an obstacle around the vehicle based on the surrounding information from the sensor when a user of the vehicle instructs to start driving assistance; and the vehicle including the obstacle. and a display control unit that causes a display device in the vehicle to display a peripheral image and an overhead image of the vehicle, wherein the display control unit includes a first pointer that points to the obstacle, and is arranged within the first pointer. and a second signal arranged around the first pointer, superimposed on the obstacle in the peripheral image, displaying the first mark A second mark including a second pointer having the same color as the pointer is superimposed on the ground position of the obstacle in the bird's-eye view image, and the transmission of the first signal is changed periodically. A first signal is dynamically displayed and the second signal is dynamically displayed such that the second signal rotates around the first pointer.

According to this configuration, it is possible to more reliably call the driver's attention to the obstacle even during driving assistance.

Further, in the above-described surroundings monitoring device, when there are a plurality of obstacles, the detection unit calculates the order of distance from the vehicle for each of the obstacles, and the display control unit controls the surrounding image wherein, for the obstacle closest to the vehicle, the first mark is displayed in a first color and the first and second signals are dynamically displayed, and in the peripheral image, For the obstacles other than the one closest to the vehicle, the first mark is displayed in a color different from the first color, and different for each obstacle, and the The first signal is statically displayed without changing the transparency, and the second signal is statically displayed without rotation. According to this configuration, the obstacle closest to the vehicle can be made more conspicuous than other obstacles.

Further, in the above-described surroundings monitoring device, the display device is provided on an instrument panel of the vehicle. According to this configuration, even if it is an image on the display device provided on the instrument panel, it is possible to visually make the obstacle stand out and call the attention of the driver.

Further, in the surroundings monitoring apparatus described above, the surroundings image is an image of the front of the vehicle in the traveling direction. According to this configuration, regardless of whether the vehicle is moving forward or backward, an image of the direction in which the vehicle is traveling is displayed, and obstacles on the image are marked to drive the vehicle. It is possible to call attention to people.

A surroundings monitoring method according to an embodiment is a surroundings monitoring method for monitoring the surroundings of a vehicle having a sensor that collects surrounding information, wherein an instruction to start driving assistance for assisting parking or exiting the vehicle is received from a user of the vehicle. If there is, an obstacle around the vehicle is detected based on the peripheral information from the sensor, and a peripheral image of the vehicle including the obstacle and a bird's-eye view image are displayed on a display device in the vehicle. , on the display device, a first pointer pointing to the obstacle, a first signal arranged within the first pointer, and a second signal arranged around the first pointer; is superimposed on the obstacle in the peripheral image, and a second mark including a second pointer having the same color as the first pointer is displayed on the obstacle in the bird's-eye view image dynamically displaying the first signal such that the transparency of the first signal changes periodically, and the second signal moves around the first pointer; Dynamically displaying the second signal to rotate.

According to this configuration, it is possible to more reliably call the driver's attention to the obstacle even during driving assistance.

1 is a top view of a vehicle equipped with a driving support device according to an embodiment; FIG. FIG. 2 is a block diagram illustrating an example of the overall configuration of the driving assistance system according to the embodiment; FIG. 3 is a block diagram illustrating an example of a functional configuration of the driving assistance device according to the embodiment; FIG. 4A is a schematic diagram illustrating an example of display on a monitor device during parking assistance by the driving assistance device according to the embodiment; FIG. 4B is a schematic diagram illustrating an example of display on the monitor device during parking assistance by the driving assistance device according to the embodiment; FIG. 4C is a schematic diagram illustrating an example of display on the monitor device during parking assistance by the driving assistance device according to the embodiment; FIG. 5 is a flowchart illustrating an example of a procedure of driving assistance processing by the driving assistance device according to the embodiment;

Similar components in the following exemplary embodiments are assigned common reference numerals, and overlapping descriptions are omitted as appropriate.

(Vehicle configuration example)
FIG. 1 is a top view of a vehicle 10 equipped with a driving assistance device 20 according to an embodiment. Front, back, left, and right of the vehicle 10 in FIG. 1 indicate directions when viewed from the driver's seat of the vehicle 10 .

The vehicle 10 of the embodiment may be, for example, an internal combustion engine vehicle using an internal combustion engine as a drive source, an electric vehicle or a fuel cell vehicle using an electric motor as a drive source, or both. It may be a hybrid vehicle that is the source.

In addition, the vehicle 10 can be equipped with various transmissions, and can be equipped with devices such as various systems and parts necessary for driving the internal combustion engine or the electric motor. Further, the system, number, layout, etc. of the devices related to the driving of the wheels 13 in the vehicle 10 can be set variously.

As shown in FIG. 1, the vehicle 10 includes a vehicle body 12, a plurality of wheels 13, a plurality of distance measurement units 14a-14l, and a plurality of imaging units 16a-16d. Note that the distance measurement units 14a to 14l are simply referred to as the distance measurement unit 14 when there is no need to distinguish between them. Further, when there is no need to distinguish between the imaging units 16a to 16d, they are simply referred to as the imaging unit 16. FIG.

The vehicle body 12 constitutes a vehicle compartment in which passengers ride. A plurality of wheels 13 , a plurality of distance measuring units 14 , and a plurality of imaging units 16 are attached to the vehicle body 12 . In the example of FIG. 1 , the vehicle body 12 includes four wheels 13 , twelve distance measuring units 14 and four imaging units 16 . However, the number of distance measuring units 14 and imaging units 16 attached to the vehicle body 12 is arbitrary.

The four wheels 13 are provided on the front, rear, left, and right sides of the vehicle body 12 . The two wheels 13 on the front side function, for example, as steered wheels, and the two wheels 13 on the rear side function, for example, as drive wheels.

The distance measuring unit 14 as a sensor is provided, for example, on the outer periphery of the vehicle 10, transmits sound waves such as ultrasonic waves as detection waves, and detects detection waves reflected by objects such as obstacles existing around the vehicle 10. It is a sonar that captures Note that the distance measurement unit 14 may be a radar that transmits a detection wave such as a laser beam, a millimeter wave radar, or the like.

The distance measuring unit 14 collects surrounding information, which is information about the surroundings of the vehicle 10 , and outputs the information to the driving support device 20 . The distance measurement unit 14 collects, for example, a response time, which is the time from transmission to reception of the detection wave, as peripheral information for specifying the distance between the object and the vehicle 10 . The driving assistance device 20 can detect the presence or absence of an obstacle or the like around the vehicle 10 and the distance to the obstacle based on the peripheral information collected by the distance measuring unit 14 .

Note that when the distance measuring unit 14 receives a plurality of detection waves reflected by a plurality of points of the object for one transmission of the detection wave, only the response time of the earliest received detection wave is measured. May be included in surrounding information.

Distance measuring units 14 a to 14 d are provided at the front portion of vehicle body 12 . Among these distance measuring units 14 a to 14 d, distance measuring units 14 b and 14 c are also called front sonar and are provided at the front end of the vehicle 10 . The distance measuring units 14 b and 14 c detect objects in front of the vehicle 10 and collect peripheral information in front of the vehicle 10 . Further, the distance measurement units 14a and 14d are also called corner sonars, and are provided at the front corner portions of the vehicle 10. As shown in FIG. The distance measuring units 14 a and 14 d detect objects on the front and outside of the vehicle 10 and collect peripheral information on the front and outside of the vehicle 10 .

The distance measuring units 14e to 14h are provided at the rear portion of the vehicle body 12. As shown in FIG. Among these distance measuring units 14e to 14h, the distance measuring units 14f and 14g are also called rear sonar and are provided at the rear end of the vehicle 10. FIG. The distance measurement units 14f and 14g detect objects behind the vehicle 10 and collect peripheral information behind the vehicle 10 . Further, the distance measuring units 14e and 14h are also called corner sonars, and are provided at the corners of the rear portion of the vehicle 10. As shown in FIG. The distance measurement units 14 e and 14 h detect objects on the rear outer side of the vehicle 10 and collect peripheral information on the rear outer side of the vehicle 10 .

The range finding units 14i to 14l are also called side sonars and are provided on the sides of the vehicle body 12. As shown in FIG. Among the distance measuring units 14i to 14l, the distance measuring units 14i and 14j are provided on the front side of the vehicle 10. As shown in FIG. Distance measuring units 14 k and 14 l are provided on the rear side of vehicle 10 . The distance measuring units 14i to 14l detect objects on the sides of the vehicle 10 and collect information on the surroundings on the sides of the vehicle 10 .

The imaging unit 16 as a sensor is a digital camera incorporating an imaging device such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging unit 16 generates a captured image of a moving image or a still image including a plurality of frame images captured at a predetermined frame rate.

The imaging unit 16 is provided on the outer periphery of the vehicle body 12 and has a wide-angle lens or a fish-eye lens, and can photograph a horizontal range of 140° to 190°, for example. The optical axis of the imaging unit 16 is set to point obliquely downward.

As a result, the imaging unit 16 collects surrounding information obtained by imaging the surroundings of the vehicle 10 including the road surface, and outputs the surrounding information to the driving support device 20 . The driving support device 20 can detect the presence or absence of an obstacle or the like around the vehicle 10 and the position of the obstacle based on the peripheral information collected by the imaging unit 16 . In addition, the driving support device 20 can detect parking lots around the vehicle 10 and the positions of the parking lots based on the peripheral information collected by the imaging unit 16 .

The imaging unit 16a is provided at the center of the front end of the vehicle body 12 in the left-right direction, for example, at the front bumper. The imaging unit 16a collects captured images of the front of the vehicle 10 as peripheral information. The imaging unit 16b is provided in the center of the rear end of the vehicle body 12 in the left-right direction, for example, in the rear bumper. The imaging unit 16b collects captured images of the rear of the vehicle 10 as peripheral information.

The imaging unit 16c is provided in the front-rear direction central portion of the left end portion of the vehicle body 12, for example, in the left side mirror. The imaging unit 16c collects captured images of the left side of the vehicle 10 as peripheral information. The imaging unit 16d is provided in the front-rear direction central portion of the right end portion of the vehicle body 12, for example, in the right side mirror. The imaging unit 16d collects captured images of the surroundings on the right side of the vehicle 10 as peripheral information.

(Configuration example of driving support system)
FIG. 2 is a block diagram showing an example of the overall configuration of the driving assistance system 200 according to the embodiment. The driving assistance system 200 is mounted on the vehicle 10, for example, and assists the driver in entering (parking) the vehicle 10 into a parking space or leaving the parking space.

As shown in FIG. 2, the driving support system 200 includes a driving support device 20, a monitor device 30, a braking system 140, an acceleration system 150, a steering system 160, a transmission system 170, a vehicle speed sensor 183, a distance measuring unit 14, and an imaging unit. 16. These components are connected by an in-vehicle network NT so as to be able to transmit and receive information to each other.

The in-vehicle network NT includes, for example, CAN (Controller Area Network) and LIN (Local Interconnect Network). In-vehicle network NT may be included as part of driving assistance system 200 .

The driving assistance device 20 is configured as a microcomputer such as an ECU (Electronic Control Unit), and assists the driver in driving the vehicle 10 .

The driving assistance device 20 includes a CPU (Central Processing Unit) 21 , a display control circuit 23 , an SSD (Solid State Drive) 24 , a ROM (Read Only Memory) 25 and a RAM (Random Access Memory) 26 . CPU 21, ROM 25, and RAM 26 may be integrated in the same package.

The CPU 21 is an example of a hardware processor, reads programs stored in a non-volatile storage device such as the ROM 25, and executes various arithmetic processes and controls according to the programs.

The ROM 25 stores various programs and parameters necessary for executing the programs. The RAM 26 temporarily stores various data used in calculations by the CPU 21 . The SSD 24 is a rewritable non-volatile storage device, and maintains data even when the driving support device 20 is powered off.

The display control circuit 23 mainly processes the image obtained by the imaging unit 16 among the arithmetic processing performed by the driving support device 20, and performs image data for display to be displayed on the display unit 31 provided in the monitor device 30, which will be described later. Execute conversion, etc.

The braking system 140 has a braking section 141 , a braking control section 142 and a braking section sensor 143 and controls deceleration of the vehicle 10 .

The braking unit 141 is a device including, for example, a brake and a brake pedal, and decelerates the vehicle 10 . The braking control unit 142 is, for example, a microcomputer having a hardware processor such as a CPU. The braking control unit 142 decelerates the vehicle 10 based on the operation of the brake pedal or the like by the driver. The braking section sensor 143 is, for example, a position sensor and detects the position of the brake pedal included in the braking section 141 . Braking unit sensor 143 outputs the detected brake pedal position to in-vehicle network NT.

The acceleration system 150 has an acceleration section 151 , an acceleration control section 152 and an acceleration section sensor 153 and controls acceleration of the vehicle 10 .

The acceleration unit 151 is a device including, for example, an accelerator pedal, etc., and accelerates the vehicle 10 . The acceleration control unit 152 is, for example, a microcomputer having a hardware processor such as a CPU. The acceleration control unit 152 accelerates the vehicle 10 based on the operation of the accelerator pedal or the like by the driver. The acceleration section sensor 153 is a position sensor for example, and detects the position of the accelerator pedal included in the acceleration section 151 . The acceleration unit sensor 153 outputs the detected position of the accelerator pedal to the in-vehicle network NT.

The steering system 160 has a steering section 161 , a steering control section 162 and a steering section sensor 163 and controls the traveling direction of the vehicle 10 .

The steering unit 161 is a device including, for example, a steering wheel or a steering wheel, and steers the traveling direction of the vehicle 10 by turning the steered wheels of the vehicle 10 . The steering control unit 162 is, for example, a microcomputer having a hardware processor such as a CPU. The steering control unit 162 controls the traveling direction of the vehicle 10 based on the driver's operation of the steering wheel or the like. The steering section sensor 163 is an angle sensor including, for example, a Hall element or the like, and detects the steering angle, which is the rotation angle of the steering section 161 . The steering unit sensor 163 outputs the detected steering angle of the steering unit 161 to the in-vehicle network NT.

The transmission system 170 has a transmission section 171 , a transmission control section 172 and a transmission section sensor 173 and controls the transmission ratio of the vehicle 10 .

The transmission unit 171 is a device including, for example, a shift lever and the like, and changes the gear ratio of the vehicle 10 . The shift control unit 172 is a microcomputer having a hardware processor such as a CPU. The gear shift control unit 172 controls the gear ratio of the vehicle 10 based on the driver's operation of a shift lever or the like. The shift section sensor 173 is, for example, a position sensor and detects the position of the shift lever included in the shift section 171 . The shift section sensor 173 outputs the detected position of the shift lever to the in-vehicle network NT.

The vehicle speed sensor 183 has, for example, a Hall element provided in the vicinity of the wheel 13 of the vehicle 10, and detects the amount of rotation of the wheel 13 or the number of revolutions per unit time. The vehicle speed sensor 183 outputs the wheel speed pulse number indicating the detected rotation amount or rotation speed to the in-vehicle network NT as a sensor value for calculating the speed of the vehicle 10 (vehicle speed). The driving support device 20 can calculate the speed, movement amount, etc. of the vehicle 10 based on the sensor value acquired from the vehicle speed sensor 183 .

The monitor device 30 is provided on an instrument panel or the like inside the vehicle 10 and has a display section 31 and an input section 32 .

The display unit 31 is a display device such as a liquid crystal display (LCD) or an organic EL display (OELD). The display unit 31 displays, for example, an image based on the image data transmitted by the driving assistance device 20, an image for receiving an operation instruction for instructing switching between the normal operation mode and the driving assistance mode for entering or exiting the vehicle.

The input unit 32 is, for example, a touch panel provided on the display screen of the display unit 31 . The input unit 32 is configured to be able to transmit the contents displayed on the display screen by the display unit 31 . Thereby, the input unit 32 allows the passenger to visually recognize the display content of the display unit 31 .

The input unit 32 receives an instruction input by the driver or the like by touching a position corresponding to the display content of the display unit 31, and transmits the instruction to the driving support device 20 via the in-vehicle network NT. Note that the input unit 32 is not limited to a touch panel, and may be a hard switch such as a push button type.

FIG. 3 is a block diagram showing an example of the functional configuration of the driving assistance device 20 according to the embodiment. As shown in FIG. 3, the driving support device 20 includes a display control unit 201, an acquisition unit 204, a detection unit 205, and a route calculation unit 206 as functional units.

These functional units are realized, for example, by the above-described CPU 21 reading a program stored in a storage device such as the ROM 25 and executing it. Alternatively, it is realized by operating the display control circuit 23 and the like under the control of the CPU 21 according to the program.

A part or all of these functional units may be configured by hardware such as a circuit including ASIC (Application Specific Integrated Circuit).

The display control unit 201 generates content to be displayed on the display unit 31 of the monitor device 30 and causes the display unit 31 to display the content. That is, the display control unit 201 displays, for example, a notification screen indicating that a parking area in which the vehicle 10 can be parked has been detected, a selection screen for starting, interrupting, or canceling driving assistance, a peripheral image of the vehicle 10 during driving assistance, and the like. is displayed on the display unit 31 .

When the display control unit 201 causes the display unit 31 to display an image of the surroundings of the vehicle 10 during driving assistance, the display control unit 201 also displays the , in the peripheral image of the vehicle 10, a process for visually highlighting obstacles and the like is performed. Details of such processing will be described later.

Acquisition unit 204 acquires transmission/reception wave information of sound waves from distance measurement unit 14 and captured images around vehicle 10 from imaging unit 16 as surrounding information of vehicle 10 .

The detection unit 205 detects obstacles, parking lots, parking areas, and the like around the vehicle 10 based on the surrounding information acquired by the acquisition unit 204 .

Obstacles are various objects such as other vehicles, walls, pillars, fences, protrusions, steps, and wheel chocks. Obstacles can also be, for example, people walking in a parking lot.

A parking section is provided to park the vehicle 10, and is a region defined by, for example, a division line, a frame line, a straight line, a band, a step, or the like. The parking area is a parking area in which the vehicle 10 can be parked, that is, a parking area in which there are no obstacles such as other vehicles obstructing the parking of the vehicle 10 .

The detection unit 205 detects the presence or absence of an obstacle, the distance from the vehicle 10 to the obstacle, and the like, based on the detection result of the distance measurement unit 14, for example. In addition, the detection unit 205 detects the presence or absence of obstacles, parking spaces, their positions (orientation) with respect to the vehicle 10, and their shapes, sizes, and heights, for example, by image processing based on images captured by the imaging unit 16. etc. is detected.

By combining these detection results, the detection unit 205 determines whether or not there are obstacles around the vehicle 10, how far each obstacle is from the vehicle 10 when there are multiple obstacles, It extracts information such as whether there is a parking space in the parking space, whether the vehicle 10 can be parked in the parking space, that is, whether the parking space can be used as a parking area.

The route calculation unit 206 calculates a target position to guide the vehicle 10 based on the detection result of the detection unit 205, and calculates a moving route for moving the vehicle 10 to the target position.

In the case of parking (entering) the vehicle 10, the parking area detected by the detection unit 205 becomes the target position. In the case of leaving the vehicle 10 from a parking space, the target position is a predetermined place where the vehicle 10 can safely start traveling, such as a passage provided between a plurality of parking spaces.

In addition, the route calculation unit 206 may calculate the turnaround position as necessary. One or more turning positions are set when it is difficult to park or leave the parking lot with one reverse or forward movement.

The route calculation unit 206 calculates the moving route of the vehicle 10 from the current position of the vehicle 10 to the target position in order to guide the vehicle 10 to the target position calculated as described above. If the travel route includes a turn-around position, the route calculation unit 206 calculates a travel route from the current position of the vehicle 10 to the target position via the turn-around position.

As described above, the route calculation unit 206 calculates a movement route for parking such as reverse parking, forward parking, reverse parallel parking, and forward parallel parking. Further, the route calculation unit 206 calculates a movement route for forward parking, backward parking, parallel parking by moving forward, parallel parking by moving backward, and the like.

Further, as described above, the driving support device 20 also functions as a surroundings monitoring device that monitors the surroundings of the vehicle 10 by detecting obstacles around the vehicle 10 during driving support. A driving support system 200, which includes the driving support device 20 and the range finding unit 14, imaging unit 16, monitor device 30, etc., also functions as a surroundings monitoring system.

(Operation example of driving support device)
Next, an operation example of the driving assistance device 20 according to the embodiment will be described with reference to FIGS. 4A to 4C. 4A to 4C are schematic diagrams illustrating an example of the display of the monitor device 30 during parking assistance by the driving assistance device 20 according to the embodiment.

FIG. 4A shows an example of an image displayed on the display unit 31 of the monitor device 30 when the driver issues an instruction to start driving assistance from the input unit 32 of the monitor device 30 or the like.

As shown in FIG. 4A, when driving assistance is started, the display control unit 201 of the driving assistance device 20 causes the display unit 31 of the monitor device 30 to display, for example, a peripheral image 31F and an overhead image 31B.

As the peripheral image 31F, a predetermined image is selected from among the images captured by the plurality of imaging units 16 provided on the outer periphery of the vehicle 10 . Specifically, the display control unit 201 selects, for example, an image of the traveling direction of the vehicle 10 from among the images captured by the plurality of imaging units 16 and displays the selected image on the display unit 31 .

The bird's-eye view image 31B is an image obtained by synthesizing the images captured by the plurality of imaging units 16 provided on the outer periphery of the vehicle 10 with a bird's-eye view of the vehicle 10 viewed from above by the display control unit 201 . The display control unit 201 also displays a vehicle icon 10ic indicating the vehicle 10 in the center of the overhead image 31B.

Further, in the example of FIG. 4A, in these peripheral image 31F and bird's-eye view image 31B, a rectangular mark 40 indicating the parking area detected based on the peripheral information from the imaging unit 16 is displayed superimposed on each image. ing.

Further, in the example of FIG. 4A , a mark 41 indicating the moving route of the vehicle 10 is superimposed on the captured image from the imaging unit 16 and displayed in the peripheral image 31F. The mark 41 is, for example, substantially equal to the width of the vehicle 10 and includes a pair of lines extending in the direction of travel of the vehicle 10, such as the front or rear of the vehicle 10, and the distances between the pair of lines from the vehicle 10, respectively. and a plurality of horizontal lines drawn at intervals.

Note that the marks 40 and 41 are displayed translucent so that the images themselves are not hidden by these marks 40 and 41, and the images behind them can be seen through the marks 40 and 41. FIG.

In the example of FIG. 4A, the person OBa in the parking lot detected based on the surrounding information from the imaging unit 16 is reflected in the surrounding image 31F and the overhead image 31B. The person OBa is positioned on the movement path of the vehicle 10 and can become an obstacle for the vehicle 10 that is trying to park in the parking area.

As described above, the detection unit 205 of the driving assistance device 20 detects the person OBa, who may be an obstacle, around the vehicle 10 during driving assistance, based on the peripheral information from the distance measuring unit 14 and the imaging unit 16, and detects the person OBa. The size and height of the person OBa, the orientation with respect to the vehicle 10, the distance from the vehicle 10, and the like are calculated.

Based on the detection result of the detection unit 205, the display control unit 201 displays the marks 50F and 50B superimposed on the image of the person OBa in the peripheral image 31F and the overhead image 31B. These marks 50F and 50B are also displayed semi-transparently so that the image behind them can be seen through the marks 50F and 50B.

Note that the display control unit 201 uses the same color for the marks 50F and 50B that are superimposed on the image of the same person OBa. In the driving support device 20, a plurality of colors of the marks 50F and 50B used for each obstacle are prepared in preparation for the case where a plurality of obstacles are detected.

A mark 50F as a first mark includes a pointer 51 and signals 52, 53, for example.

A pointer 51 as a first pointer is, for example, a circular mark and has a function of pointing to an obstacle. The pointer 51 is displayed at a position offset from the contact position of the obstacle detected by the detection unit 205, that is, near the center position of the obstacle in the height direction, for example. As in the example of FIG. 4A, when the obstacle is the person OBa, the pointer 51 is displayed near the center of the length of the person OBa, offset from the feet of the person OBa, for example.

A signal 52 as a first signal is, for example, a cross mark placed in a circular pointer 51 . When displaying the mark 50F on the peripheral image 31F, the display control unit 201 dynamically displays the signal 52 on the peripheral image 31F by, for example, changing the transmittance of the signal 52 at a predetermined cycle.

In this case, in the mark 50F originally displayed semi-transparently, the transparency of the signal 52 varies from a transparency equal to or less than the transparency of the pointer 51 whose transparency does not change, to a transparency of a predetermined value or more. It may change in steps or gradually through steps finer than two steps.

Further, when the transmittance of the signal 52 is changed periodically, there may be a timing when the transmittance of the signal 52 becomes 100%, that is, the signal 52 disappears from the screen. In this case, the signal 52 should appear blinking on the peripheral image 31F.

The signal 53 as the second signal is, for example, marks such as a plurality of line segments, dots, or polygons arranged to surround the circular pointer 51 at predetermined intervals. In the example of FIG. 4A, signal 53 includes four triangular marks surrounding pointer 51 from four directions. Each of these triangles surrounding the pointer 51 is arranged so that one vertex faces toward the center of the pointer 51 and the side corresponding to the base of the vertex faces the outside of the pointer 51 .

In displaying the mark 50F on the peripheral image 31F, the display control unit 201 dynamically displays the signal 53 on the peripheral image 31F by, for example, rotating the periphery of the pointer 51 at a predetermined speed. In the example of FIG. 4A, the arrangement of two pairs of triangular marks that face each other vertically and horizontally with the pointer 51 interposed therebetween is moved clockwise or counterclockwise. As a result, the four triangular marks are visually recognized as rotating around the pointer 51 .

The mark 50B as the second mark includes at least the pointer 54, for example.

A pointer 54 as a second pointer is, for example, a circular mark having four protrusions and has a function of pointing to an obstacle. Thus, the pointer 54 preferably has a shape similar to that of the pointer 51 of the mark 50F pointing to the same obstacle, for example.

Also, the pointer 54 is displayed near the contact position of the obstacle detected by the detection unit 205 . As in the example of FIG. 4A, when the obstacle is the person OBa, the pointer 54 is displayed near the feet of the person OBa, for example.

When an obstacle such as the person OBa is detected around the vehicle 10, the display control unit 201 displays the marks 50F and 50B and displays a message such as "Please directly check the vehicle surroundings." It may be displayed on the section 31 .

FIG. 4B shows an example of an image displayed on the display section 31 of the monitor device 30 after a predetermined period of time has elapsed from the state shown in FIG. 4A.

As shown in FIG. 4B, for example, the person OBa has not moved from the position in FIG. It has not changed from the position of FIG. 4A.

Note that when an obstacle such as the person OBa is moving, the pointers 51 and 54 of the respective marks 50F and 50b move accordingly on the peripheral image 31F or the overhead image 31B so as to follow the obstacle. Moving.

In addition, after a predetermined period of time from FIG. 4A, the transmittance of the signal 52 of the mark 50F has increased in the peripheral image 31F. That is, the signal 52 is displayed lighter than in the state of FIG. 4A. Also, the arrangement of the four triangular marks is changed in the signal 53 of the mark 50F. That is, in FIG. 4A, the four triangular marks, which were arranged to substantially coincide with the vertices of the cross of the signal 52, are rotated around the pointer 51, and in FIG. are placed at positions between

While the person OBa is detected around the vehicle 10, the display control unit 201 continues the above-described dynamic display of the signals 52 and 53 of the mark 50F at a predetermined cycle and at a predetermined speed.

FIG. 4C shows an example of an image displayed on the display section 31 of the monitor device 30 after a predetermined period of time has elapsed from the state shown in FIG. 4B.

As shown in FIG. 4C, for example, the person OBa has not moved from the position shown in FIG. 4B, and the marks 50F and 50B are still superimposed on the images of the person OBa in the peripheral image 31F and the overhead image 31B.

Further, in the example of FIG. 4A, a new person OBb detected based on the surrounding information from the imaging unit 16 is reflected in the surrounding image 31F and the overhead image 31B. The person OBb is positioned closer to the vehicle 10 than the person OBa is on the moving route of the vehicle 10, and can become an obstacle for the vehicle 10 trying to park in the parking area.

The detection unit 205 detects the new person OBb as an obstacle based on the peripheral information from the distance measurement unit 14 and the image pickup unit 16, and detects the size, height, direction of the person OBb with respect to the vehicle 10, and the position of the vehicle 10. Calculate the distance from

Further, as described above, when a plurality of obstacles such as the persons OBa and OBb are detected, the detection unit 205 calculates the order of distance from the vehicle 10 for each of these obstacles. In the example of FIG. 4C, the detection unit 205 calculates that the person OBb is the closest obstacle from the vehicle 10 among the two persons OBa and OBb serving as obstacles, and that the person OBa is the second closest obstacle to the vehicle 10 after the person OBb. It is calculated as an obstacle.

Based on the detection result of the detection unit 205, the display control unit 201 changes the display of the marks 50F and 50B, which have been superimposed on the image of the person OBa, to the second closest to the vehicle 10 in the peripheral image 31F and the overhead image 31B. Switch to the mode corresponding to the obstacle.

As described above, the driving support device 20 is set to display the marks 50F, 60F, . . . and the marks 50B, 60B, . These colors are assigned according to the distance from the vehicle 10 to these obstacles, for example.

At this time, for example, an obstacle closest to the vehicle 10 may be assigned a color such as red, which has the highest visibility, and then may be assigned a color such as orange, and then a moderate color such as yellow. 4A and 4B, when only one obstacle is detected, the color assigned to the closest obstacle from the vehicle 10 is used. may be

In the example of FIG. 4C, the display control unit 201 assigns the marks 50F and 50B attached to the image of the person OBa from the color assigned to the obstacle closest to the vehicle 10 to the obstacle closest to the vehicle 10. change to color. Also in this case, the marks 50F and 50B are displayed in the same color.

Further, the display control unit 201 stops changing the transparency of the signal 52 in the mark 50F, and switches the display mode of the signal 52 to static display. Further, the display control unit 201 stops the rotation of the signal 53 at the mark 50F, and switches the display mode of the signal 53 to static display.

The display control unit 201 also displays the marks 60F and 60B superimposed on the image of the newly detected person OBb in the peripheral image 31F and the overhead image 31B based on the detection result and determination result of the detection unit 205 . These marks 60F and 60B are also displayed translucent in the same color. At this time. Instead of the mark 50F, the marks 60F and 60B are given the color assigned to the closest obstacle from the vehicle 10, which was assigned to the marks 50F and 50B in FIGS. 4A and 4B, for example.

A mark 60F as a first mark includes, for example, a pointer 61 and signals 62 and 63, and is superimposed on the image of the person OBb in the same manner as the mark 50F in FIGS. 4A and 4B in place of the mark 50F.

A pointer 61 as a first pointer is displayed, for example, near the center position of the body length of the person OBb offset from the position of the feet of the person OBb, and points to the person OBb. When an obstacle such as the person OBb is moving, the pointer 61 moves on the peripheral image 31F so as to follow the obstacle accordingly.

A signal 62 as a first signal is arranged, for example, in a circular pointer 61, and its transmittance changes at predetermined intervals.

The signal 63 as the second signal is arranged, for example, so as to surround the circular pointer 61 at predetermined intervals, and rotates around the pointer 51 at a predetermined speed.

The mark 60B as the second mark includes at least a pointer 64, for example, and is superimposed on the image of the person OBb in the same manner as the mark 50B in FIGS. 4A and 4B instead of the mark 50B.

A pointer 64 as a second pointer is displayed near the feet of the person OBb, for example, and points to the person OBb.

As described above, the display control unit 201 continues to display the peripheral image 31F and the overhead image 31B based on the captured image captured by the imaging device 16 on the display unit 31 while driving assistance is being performed. Further, the display control unit 201 superimposes the marks 50F, 60F, . . . and the marks 50B, 60B, . displayed.

(Processing example of the driving support device)
Next, an example of driving assistance processing by the driving assistance device 20 of the embodiment will be described with reference to FIG. FIG. 5 is a flow diagram showing an example of a procedure of driving assistance processing by the driving assistance device 20 according to the embodiment. Note that the surroundings monitoring process by the driving support device 20 is performed in parallel with the driving support process.

As shown in FIG. 5, the driving assistance device 20 waits for an input from the driver of the vehicle 10 or the like to instruct the start of driving assistance (step S101: No).

When the driver or the like inputs an instruction to start driving assistance (step S101: Yes), the detection unit 205 determines the target position of the vehicle 10 for driving assistance based on the surrounding information from the distance measuring unit 14 and the imaging unit 16. Detection of a surrounding parking area or the like and detection of an obstacle such as a person around the vehicle 10 are started (step S102).

The detection unit 205 continues to detect the situation around the vehicle 10 until the vehicle 10 reaches the target position and the driving assistance ends.

The route calculation unit 206 calculates a target position to guide the vehicle 10, a turning position if necessary, and a movement route based on the detection result of the detection unit 205 (step S103).

The display control unit 201 displays an image in front of the vehicle 10 in the traveling direction of the captured images from the plurality of imaging devices 16 as a peripheral image 31F of the vehicle 10 on the display unit 31 of the monitor device 30 . Further, the display control unit 201 synthesizes the captured images from the plurality of imaging devices 16 and displays the bird's-eye view image 31B with the vehicle icon 10ic attached thereto on the display unit 31 of the monitor device 30 (step S104).

At this time, the display control unit 201 superimposes the mark 40 indicating the target position such as the parking area calculated by the route calculation unit 206 on the surrounding image 31F and the bird's-eye view image 31B, and also displays the mark indicating the moving route of the vehicle 10. 41 is superimposed on the peripheral image 31F and displayed.

The detection unit 205 determines whether or not there is an obstacle around the vehicle 10 (step S105). If there are no obstacles around the vehicle 10 (step S105: No), the processes of steps S106 to S111 are skipped.

If there are obstacles around the vehicle 10 (step S105: Yes), the detection unit 205 determines whether there are a plurality of obstacles around the vehicle 10 (step S106).

If there is only one obstacle around the vehicle 10 (step S106: No), the display control unit 201 periodically changes the transmittance of the symbol 52 of the mark 50F to be displayed in the peripheral image 31F. is subjected to dynamic processing such as rotating around the pointer 51 (step S109).

The display control unit 201 places the mark 50F, which is the image of the obstacle in the peripheral image 31F, near the central position in the height direction offset from the grounding position of the obstacle. Overlap display. Further, the display control unit 201 displays a mark 50B having the same color as the mark 50F, which is an image of the obstacle in the bird's-eye view image 31B, superimposed on the ground position of the obstacle (step S110).

When there are a plurality of obstacles around the vehicle 10 (step S106: Yes), the detection unit 205 calculates the order of distance from the vehicle 10 to each of these obstacles (step S107).

Based on the detection result of the detection unit 205, the display control unit 201 selects the color assigned to the nearest obstacle as the display color of the mark 60F for the obstacle closest to the vehicle. Dynamic processing is applied to the signals 62 and 63 in (step S108).

The display control unit 201 displays a mark 60F obtained by dynamically processing the symbols 62 and 63 in the image of the obstacle in the peripheral image 31F, near the central position in the height direction offset from the grounding position of the obstacle. Overlap display. Further, the display control unit 201 displays a mark 60B having the same color as the mark 60F, which is an image of the obstacle in the bird's-eye view image 31B, superimposed on the grounding position of the obstacle (step S110).

The display control unit 201 determines whether or not the displayed obstacle is the last one among the detected obstacles (step S111).

If unprocessed obstacles remain (step S111: No), the display control unit 201 repeats the process of step S110 by the number of obstacles. However, the display control unit 201 displays the marks 50F and 50B without dynamically processing the symbols 52 and 53 except for the nearest obstacle.

After displaying the above for all obstacles (step S111: Yes), the driving assistance device 20 determines whether or not the driving assistance has ended, for example, when the vehicle 10 reaches the target position (step S112).

The driving assistance device 20 repeats the processing of steps S105 to S111 until the driving assistance is completed (step S112: No), and when the driving assistance is completed (step S112), the peripheral monitoring processing is completed.

With the above, the driving support processing by the driving support device 20 of the embodiment ends.

(Overview)
The driving assistance device assists the driver in entering and exiting the vehicle based on information about the surroundings of the vehicle from various sensors such as the distance measuring unit and the imaging unit. During driving assistance, the driving assistance device monitors the surroundings of the vehicle, and when an obstacle or the like is detected, displays a highly visible colored mark on the monitor device or the like to alert the driver.

According to the technology disclosed in Patent Document 1, the passengers on the train platform are detected, each passenger is given a color-coded frame according to the risk level of the passenger's behavior, and dangerous behavior on the platform is monitored. . According to the technique of Patent Document 2, in a vehicle that is normally traveling on a road, a marker is attached to a pedestrian with a short time to collision calculated based on the walking speed of the pedestrian, and the marker is displayed on the driver's head-up display. indicate.

According to the technique disclosed in Patent Literature 3 described above, road facilities are efficiently monitored by displaying a warning according to the content of a warning such as a fire on an expressway or an intrusion of a person. According to the technique of Patent Document 4 described above, in order to monitor the surroundings of an excavator that is working in a depressed portion of the ground, etc., the color and brightness of features are changed according to the distance from the excavator, and the surroundings of the excavator are monitored. Improve visibility in

According to the technique disclosed in Patent Document 5, a vehicle that is normally traveling on a road rotates a plurality of arc-shaped index images surrounding the preceding vehicle and expands and contracts a linear gauge image so that the risk potential The height of the vehicle is displayed on the driver's head-up display.

However, as in Patent Documents 1 and 2, it may not be possible to sufficiently call the driver's attention simply by using different display colors. When the driver is elderly, it may be difficult for the driver to distinguish between colors due to aging of color vision, and the risk of overlooking the display informing the detection of the obstacle further increases.

In addition, when monitoring the surroundings during driving assistance for assisting the driver in entering or exiting the vehicle, unlike the case of displaying on the driver's head-up display during normal driving as in Patent Documents 2 and 5, complicated driving operations are required. While driving, it may be difficult for the driver to pay attention to the display of the monitor device arranged on the instrument panel of the vehicle.

Under such circumstances, it may not be possible to adequately call attention by simply changing the display color or rotating the mark alone as in Patent Document 5.

According to the driving support device 20 of the embodiment, the display control unit 201 includes a pointer 51 pointing to an obstacle, and a signal 52 arranged in the pointer 51 and dynamically displayed such that the degree of transparency changes periodically. , and a dynamically displayed signal 53 positioned around and rotating around the pointer 51 .

In this way, the signals 52 and 53 are dynamically displayed in different modes of periodic change in transmittance and rotation. Also, the signals 52 and 53 are dynamically displayed inside and outside the pointer 51, respectively.

For this reason, even a driver, such as an elderly driver who tends to overlook changes in the degree of transmission of the signal 52, may easily notice the rotation of the other signal 53. Complementary oversight by the driver can be suppressed.

With such a configuration, the attention-calling power of the mark 50F attached to the obstacle is greatly improved, and the driver's attention to the obstacle can be more reliably called even during driving assistance. Also, even if the obstacle is stationary, for example, the driver's attention to the obstacle can be more reliably called.

According to the driving support device 20 of the embodiment, the display control unit 201 displays the mark 50F superimposed on the obstacle in the peripheral image 31F. This makes it easier to grasp the position of the obstacle with respect to the vehicle 10 in the peripheral image 31F.

According to the driving support device 20 of the embodiment, the display control unit 201 displays the mark 50B superimposed on the contact position of the obstacle in the bird's-eye view image 31B. The bird's-eye view image 31B is a composite image of a plurality of captured images. In the bird's-eye view image 31B, three-dimensional obstacles such as persons OBa and OBb are displayed by being stretched in the height direction. With the above configuration, it becomes easier to grasp the position of the obstacle with respect to the vehicle 10 even in such a bird's-eye view image 31B.

According to the driving assistance device 20 of the embodiment, the mark 50B includes the pointer 54 of the same color as the pointer 51. As shown in FIG. In this way, by using the marks 50F and 50B of the same color for the same obstacle, it is possible to easily grasp whether the obstacles displayed in the peripheral image 31F and the bird's-eye view image 31B are the same or different. be able to.

According to the driving support device 20 of the embodiment, the display control unit 201 displays the mark 60F in a predetermined color for the obstacle closest to the vehicle 10 in the peripheral image 31F, and the signal 62 of the mark 60F. , 63 are dynamically displayed, and for other obstacles, the marks 50F . 52 . . . are statically displayed without changing their transmittance, and signals 53 .

As a result, the mark 60F attached to the obstacle closest to the vehicle 10 and likely to hinder the movement of the vehicle 10 can be made more conspicuous than the other marks 50F, and the driver's attention can be further aroused. Further, by dynamically displaying the mark 60F, it is possible to easily determine the obstacle closest to the vehicle 10 among the plurality of obstacles.

In the above-described embodiment, for example, the marks 50F and 60F have the same size, and the marks 50B and 60B have the same size. However, the marks 50F, 60F, . . . and the marks 50B, 60B, . In this case, the closer the distance from the vehicle 10 is, the larger the mark may be displayed.

Further, in the above-described embodiment, for example, the marks 50F and 60F have the same shape, and the marks 50B and 60B have the same shape. However, the marks 50F, 60F, . . . and the marks 50B, 60B, .

Even in this case, it is preferable that the marks 50F and 50B indicating the same obstacle in the peripheral image 31F and the bird's-eye view image 31B, and the marks 60F and 60B, etc. have similar shapes. This facilitates matching of the same obstacles displayed in the peripheral image 31F and the overhead image 31B.

In the above-described embodiment, when an obstacle is detected around the vehicle 10, the driving support device 20 puts a mark on the monitor device 30 to alert the driver. However, the driving assistance device 20 may have a function of alerting the driver and controlling the vehicle 10 to avoid contact with the detected obstacle.

In this case, for example, the driving assistance device 20 may be configured to be able to control the braking system 140, and the vehicle 10 may be braked by the braking system 140 to avoid contact with the obstacle.

Further, in the above-described embodiment, the driving assistance device 20 assists the driver who drives the vehicle 10 by, for example, showing the movement route of the vehicle 10 . However, the driving assistance device 20 may have a function of performing driving assistance by controlling part or all of the driving operation on behalf of the driver.

In this case, for example, the driving support device 20 may be configured to be able to control the steering system 160 , and the steering system 160 may control the moving direction of the vehicle 10 . Alternatively, the driving assistance device 20 may be configured to be able to control not only the steering system 160 but also the braking system 140, the acceleration system 150, and the like, and perform driving assistance through automatic driving.

However, in any of the above cases, if the driver visually confirms the obstacle and deems it necessary, the driving assistance device 20 performs driving operation of the vehicle 10 to avoid contact with the obstacle. can be performed by the driver himself/herself.

DESCRIPTION OF SYMBOLS 10... Vehicle 14... Ranging part 16... Imaging part 20... Driving assistance device 206... Route calculation part 30... Monitor device 31... Display part 31B... Bird's-eye view image 31F... Peripheral image 32... Input part 50B, 50F, 60B, 60F... Marks 51, 54, 61, 64 Pointers 52, 53, 62, 63 Signal 201 Display control unit 204 Acquisition unit 205 Detection unit

Claims (5)

A surroundings monitoring device for monitoring the surroundings of a vehicle having a sensor that collects surrounding information,
a detection unit that detects obstacles around the vehicle based on the peripheral information from the sensor when a user of the vehicle instructs to start driving assistance for assisting parking or exiting the vehicle;
a display control unit for displaying a peripheral image of the vehicle including the obstacle and a bird's-eye view image on a display device in the vehicle;
The display control unit
a first pointer pointing to the obstacle;
a first signal positioned within the first pointer;
and displaying a first mark including a second signal arranged around the first pointer superimposed on the obstacle in the peripheral image;
displaying a second mark including a second pointer having the same color as the first pointer so as to be superimposed on the ground position of the obstacle in the bird's-eye view image;
dynamically displaying the first signal such that the transmittance of the first signal changes periodically;
dynamically displaying the second signal such that the second signal rotates around the first pointer;
Perimeter monitoring device. The detection unit is
when there are a plurality of obstacles, calculating the order of distance from the vehicle for each of the obstacles,
The display control unit
displaying the first mark in a first color and dynamically displaying the first and second signals for the obstacle closest to the vehicle in the peripheral image;
In the peripheral image, for the obstacles other than the obstacle closest to the vehicle, the first mark is a color different from the first color, and is a different color for each obstacle. Displaying and statically displaying without changing the transparency of the first signal and statically displaying the second signal without rotation;
A perimeter monitoring device according to claim 1 . The display device is provided on an instrument panel of the vehicle,
The perimeter monitoring device according to claim 1 or 2. The peripheral image is an image in front of the traveling direction of the vehicle,
The perimeter monitoring device according to any one of claims 1 to 3. A surroundings monitoring method for monitoring the surroundings of a vehicle having a sensor that collects surrounding information,
detecting obstacles around the vehicle based on the peripheral information from the sensor when the user of the vehicle instructs to start driving assistance for assisting parking or exiting the vehicle;
displaying a peripheral image of the vehicle including the obstacle and a bird's-eye view image on a display device in the vehicle;
on the display device,
a first pointer pointing to the obstacle;
a first signal positioned within the first pointer;
and displaying a first mark including a second signal arranged around the first pointer superimposed on the obstacle in the peripheral image;
displaying a second mark including a second pointer having the same color as the first pointer, superimposed on the ground position of the obstacle in the bird's-eye view image;
dynamically displaying the first signal such that the transmittance of the first signal changes periodically;
dynamically displaying the second signal such that the second signal rotates around the first pointer;
Perimeter monitoring method.

Images