WO2022091194A1

WO2022091194A1 - Display control device, display system, display control method, and display control program

Info

Publication number: WO2022091194A1
Application number: PCT/JP2020/040178
Authority: WO
Inventors: 修平鈴木; 新作福▲高▼; 宗貴西平; 晶子今石; 美里湯浅
Original assignee: 三菱電機株式会社
Priority date: 2020-10-27
Filing date: 2020-10-27
Publication date: 2022-05-05
Also published as: JP7439949B2; JPWO2022091194A1

Abstract

Prior-art display control devices have had a problem in that if a blind spot image is displayed to a pedestrian, it is difficult for the pedestrian to recognize what range is being displayed.　This display control device (20) is provided with a blind spot image acquisition unit (21) that acquires a blind spot image of a blind spot caused by a host vehicle when viewed from a subject present outside the host vehicle, a display image generation unit (23) that generates a display image in which the image of the host vehicle is superimposed over the blind spot image, and a control unit (26) that displays a display image on display devices (32–35) provided in the host vehicle.

Description

Display control device, display system, display control method and display control program

The present disclosure relates to a display control device, a display system, a display control method, and a display control program that control when displaying a blind spot image showing a situation in a blind spot caused by the own vehicle on the display device.

When driving a vehicle such as a car, a blind spot may occur for a target person such as a pedestrian who exists in the vicinity of the own vehicle due to the own vehicle.

Therefore, a device that captures an image of the periphery of the own vehicle with a camera and displays information on a field of view that becomes a blind spot for a target person such as a pedestrian in the vicinity due to the presence of the own vehicle on the periphery of the own vehicle is a patent document. It is disclosed in 1. The device disclosed in Patent Document 1 is provided in the own vehicle and is imaged by an image pickup means for capturing an image of the periphery of the own vehicle and an image display means provided in at least a part of the own vehicle. It is provided with a display control means for displaying an image of a field of view that becomes a blind spot at a position where an image display means is provided.

Japanese Unexamined Patent Publication No. 2002-87188

However, the above-mentioned conventional display control device has a problem that it is difficult to recognize which range the pedestrian is displaying even if the blind spot image is displayed to the pedestrian.

This disclosure is made to solve the above-mentioned problems, and when displaying a blind spot image, it is possible to intuitively and easily display to the subject which blind spot image it is. It is an object of the present invention to provide a display control device, a display system, a display control method, and a display control program.

The display control device according to the present disclosure includes a blind spot image acquisition unit that acquires a blind spot image showing a blind spot generated by the own vehicle when viewed from a target person existing outside the own vehicle, and a display in which an image of the own vehicle is superimposed on the blind spot image. It includes a display image generation unit that generates an image and a control unit that displays a display image on a display device provided in the own vehicle.

The display system according to the present disclosure includes a display device provided in the own vehicle, a blind spot image acquisition unit that acquires a blind spot image showing a blind spot generated by the own vehicle when viewed from a target person outside the own vehicle, and the own vehicle. It is provided with a display image generation unit that generates a display image in which the image of the above is superimposed on a blind spot image, and a control unit that displays the display image on the display device.

The display control method according to the present disclosure includes a step of acquiring a blind spot image showing a blind spot generated by the own vehicle when viewed from a target person existing outside the own vehicle, and generating a display image in which the image of the own vehicle is superimposed on the blind spot image. It has a step of displaying a display image on a display device provided in the own vehicle.

The display control program according to the present disclosure processes to acquire a blind spot image showing a blind spot generated by the own vehicle when viewed from a target person existing outside the own vehicle, and generates a display image in which the image of the own vehicle is superimposed on the blind spot image. The process of displaying the displayed image on the display device provided in the own vehicle is executed.

According to the present disclosure, the display image generation unit generates a display image in which the image of the own vehicle is superimposed on the blind spot image, and the control unit displays the display image on the display device provided in the own vehicle to display the blind spot image. When displaying, it is possible to intuitively and easily display to the target person what range of the blind spot image is.

It is a figure which showed an example of the situation where there is a pedestrian crossing on a road with two lanes on each side. It is a block diagram of the display system including the display control device which concerns on Embodiment 1. FIG. It is a front view of the own vehicle which concerns on Embodiment 1. FIG. It is a rear view of the own vehicle which concerns on Embodiment 1. FIG. It is a left side view of the own vehicle which concerns on Embodiment 1. FIG. It is a right side view of the own vehicle which concerns on Embodiment 1. FIG. It is a hardware block diagram of the in-vehicle system of the own vehicle which concerns on Embodiment 1. FIG. It is a flowchart which shows the operation of the display system which concerns on Embodiment 1. This is an example of a display image before processing the image of the own vehicle according to the first embodiment. This is an example of a display image after processing the image of the own vehicle according to the first embodiment. It is an example of the figure when the own vehicle is seen from the subject person which concerns on Embodiment 1. It is an example of the figure when the own vehicle which concerns on Embodiment 1 is seen from the front. It is a flowchart which shows the operation of the display system which concerns on Embodiment 2.

Embodiment 1.
FIG. 1 is a diagram showing an example of a situation where a pedestrian crossing 4 is located on a road having two lanes on each side. In the first embodiment, as an example, consider a case where the own vehicle 1 which is an automobile is traveling on a road having two lanes on each side, as shown in FIG. The own vehicle 1 is stopped in front of the pedestrian crossing 4, and the pedestrian 3 is about to cross in front of the own vehicle 1 where the pedestrian 3 is stopped as a target person. Further, another vehicle 2 which is a vehicle is trying to overtake the own vehicle 1 from behind the lane next to the own vehicle 1 which is behind the own vehicle 1 with respect to the pedestrian 3. The case where this disclosure is applied will be described below in a situation where the range where the other vehicle 2 is present becomes a blind spot due to the own vehicle 1 from the pedestrian 3 and the other vehicle 2 cannot be seen.

FIG. 2 is a block diagram of a display system 30 including a display control device 20 according to the first embodiment. The display system 30 is information to be notified using the notification device 31 provided in the own vehicle 1 and the inside / outside information of the vehicle acquired from the overall control ECU (Electronic Control Unit) 10 that controls the entire vehicle of the own vehicle 1. It is provided with a display control device 20 that determines the notification information and controls the notification information to be notified from the notification device 31. Here, the vehicle interior / external information includes blind spot information or blind spot images. The blind spot information and the blind spot image will be described later.

The notification device 31 is a device that notifies notification information from the own vehicle 1 to a target person existing outside the own vehicle 1. The notification device 31 notifies, for example, the state of the own vehicle 1, the notice and intention of the current or future operation of the own vehicle 1, or a warning to the target person existing outside the own vehicle 1 around the own vehicle 1. .. The notification device 31 includes display devices 32 to 35 and a speaker 36. However, the notification device 31 may include at least one display device.

FIG. 3 is a front view of the own vehicle 1 according to the first embodiment. In the first embodiment, the display device 32 is provided on the front surface of the own vehicle 1, and the speaker 36 is provided on the upper side of the display device 32 on the front surface of the own vehicle 1.

FIG. 4 is a rear view of the own vehicle 1 according to the first embodiment. In the first embodiment, the display device 33 is provided on the rear surface of the own vehicle 1.

FIG. 5 is a left side view of the own vehicle 1 according to the first embodiment. In the first embodiment, the display device 34 is provided on the left side surface of the own vehicle 1.

FIG. 6 is a right side view of the own vehicle 1 according to the first embodiment. In the first embodiment, the display device 35 is provided on the right side surface of the own vehicle 1.

Returning to FIG. 2, the overall control ECU 10 acquires blind spot information or vehicle internal / external information including blind spot images from various sensors inside and outside the vehicle 1 of the own vehicle 1, and based on the acquired information inside and outside the vehicle, the inside of the own vehicle 1 Send instructions to the hardware and control the hardware.

The display control device 20 includes a vehicle interior / external information acquisition unit 27 that acquires vehicle interior / external information including blind spot information or blind spot images from the overall control ECU 10. Further, the display control device 20 uses the blind spot image DB (Database) 24b in which the parts of the image for generating the blind spot image from the blind spot information are stored, and the blind spot image DB 24b to generate the blind spot image from the blind spot information. It includes an image generation unit 28.

Further, the display control device 20 includes a blind spot image acquisition unit 21 for acquiring a blind spot image, and a determination unit 22 for determining whether or not to notify the blind spot image, vehicle interior / external information, and the like from the notification device 31. Further, when the display control device 20 determines that the own vehicle image DB (Database) 24a in which the image of the own vehicle 1 is stored and the determination unit 22 notifies the notification, the image of the own vehicle 1 acquired from the own vehicle image DB 24a. It is provided with a display image generation unit 23 that generates notification information including a display image in which the above is superimposed on the blind spot image.

Further, the display control device 20 stores notification device information in which the vehicle interior / external information and the notification device 31 are associated with each other, notification pattern information such as processing information of the display image, and condition information such as notification timing information. It is provided with a condition information DB (Database) 25. Further, the display control device 20 controls the condition giving unit 29 for imparting the condition information acquired from the condition information DB 25 to the display image and the notification device 31 associated with the condition information DB 25 to notify the notification information. It is equipped with a part 26. Each configuration will be described below.

The vehicle interior / external information acquisition unit 27 acquires the vehicle interior / external information including the blind spot information or the blind spot image from the overall control ECU 10. The vehicle interior / external information includes, for example, the position information of the own vehicle 1, the position information of the target person 3, the line-of-sight information of the target person 3, the blind spot information of the target person 3, the blind spot image of the target person 3, and the position information of the other vehicle 2. Engine rotation amount information, engine ON / OFF switching information, gear change information by shift lever, door opening / closing information, own vehicle 1 operation information, steering angle information of steering wheel, winker operation information, surrounding congestion status Information, detection information and recognition information of objects such as surrounding people and other vehicles, information about users in the vehicle of own vehicle 1, and the like. The vehicle interior / external information acquisition unit 27 corresponds to the line-of-sight information acquisition unit that acquires the line-of-sight information of the target person 3.

Here, the blind spot information is information indicating the situation in the blind spot caused by the own vehicle 1 as seen from the target person 3 existing outside the own vehicle 1. Specifically, for example, the blind spot information is information from an ultrasonic sensor, a radar sensor, a millimeter wave radar sensor, a lidar, an infrared laser sensor, or the like.

The blind spot image is an image of the blind spot generated by the own vehicle 1 as seen from the target person 3 existing outside the own vehicle 1. Specifically, for example, the blind spot image is an image of an outside camera or the like.

The blind spot image DB 24b stores in advance images of various objects such as other vehicles viewed from various angles. Further, the blind spot image DB 24b stores images of various backgrounds viewed from various angles in advance. An image such as an object and a background is an actual image such as a photograph, an illustration, or the like, and is a part of an image when a blind spot image is generated from blind spot information.

The blind spot image generation unit 28 generates a blind spot image from the blind spot information using the image parts stored in the blind spot image DB 24b. Further, when the range of the blind spot image acquired by the vehicle interior / external information acquisition unit 27 is too wide, the blind spot image generation unit 28 extracts an appropriate range and uses it as a subsequent blind spot image. Specifically, the blind spot image generation unit 28 acquires image parts from the blind spot image DB 24b using the blind spot information, for example, when the blind spot information is radar information or the like, and generates a blind spot image from the image parts. do. Further, the blind spot image generation unit 28 extracts, for example, a blind spot image according to the blind spot when the image is an image of a camera outside the vehicle in a range wider than the blind spot.

The blind spot image acquisition unit 21 acquires the blind spot image acquired by the vehicle interior / external information acquisition unit 27 or the blind spot image generated by the blind spot image generation unit 28. The blind spot image shows the blind spot caused by the own vehicle 1 when viewed from the subject 3 existing outside the own vehicle 1.

The determination unit 22 determines the state of the vehicle of the own vehicle 1, the traffic environment outside the own vehicle 1, and the like based on the information inside and outside the vehicle. The determination unit 22 determines, for example, whether the own vehicle 1 is still stopped, is about to move forward, or is about to move backward, based on the gear change information. Further, the determination unit 22 determines whether or not the target person is in a dangerous state based on, for example, detection information and recognition information of objects such as surrounding people and cars.

The determination unit 22 determines whether to notify the target person of the notification information based on the determined vehicle condition of the own vehicle 1, the traffic environment outside the own vehicle 1, and the like. Specifically, the determination unit 22 determines whether or not to display the display image on the target person 3. For example, in the case of FIG. 1, the determination unit 22 determines that the pedestrian 3 is notified of the presence of the other vehicle 2.

The own vehicle image DB 24a stores images of the own vehicle 1 viewed from various angles in advance. The image of the own vehicle 1 is an actual image such as a photograph, an illustration, or the like.

When the determination unit 22 determines that the display image generation unit 23 notifies, the display image generation unit 23 generates notification information from the information inside and outside the vehicle. When generating the notification information, the display image generation unit 23 generates notification information including a display image in which at least the image of the own vehicle 1 acquired from the own vehicle image DB 24a is superimposed on the blind spot image.

The condition information DB 25 stores the condition information. Specifically, for example, the condition information DB 25 stores information on the devices of the display devices 32 to 35 and the speaker 36, and the notification device in which the identifier of the vehicle interior / external information and the identifier of the notification device 31 are associated with each other. I remember the information. Further, the condition information DB 25 stores the notification pattern information associated with the vehicle interior / external information. The notification pattern information includes, for example, information such as the shape, color, size, brightness, position, etc. of the image displayed on the display devices 32 to 35, the loudness of the sound to be notified to the speaker 36, the pitch of the sound, the type of sound, and the like. Information. Further, the condition information DB 25 stores the notification timing information associated with the vehicle interior / external information. The notification timing information is, for example, information such as timing and time for notification to the notification device 31.

The condition giving unit 29 determines the notification device 31 to be notified from the condition information DB 25, the notification pattern, the timing to notify, the time, and the like, and adds the condition information to the notification information including the display image. Specifically, for example, when the condition giving unit 29 acquires the processing information for reducing the image of the own vehicle in the display image as the notification pattern information, the condition giving unit 29 reduces the image of the own vehicle and processes the display image.

The control unit 26 controls the notification device 31 associated with the condition information DB 25 to notify the notification information including the display image when the determination unit 22 determines to perform notification. Specifically, when the determination unit 22 determines that the display image is to be displayed, the control unit 26 controls the display device associated with the condition information DB 25 to display the display image.

Next, a part of the hardware configuration of the own vehicle 1 in the first embodiment will be described.

FIG. 7 is a hardware configuration diagram of the in-vehicle system 100 of the own vehicle 1 according to the first embodiment. The configuration of the display system 30 including the display control device 20 according to the first embodiment will be described with reference to FIG. 7.

As an example, the in-vehicle system 100 includes a communication line 65, a sensor 40, a communication device 60, an overall control ECU 10, a notification control ECU (Electronic Control Unit) 15, a driving device 70, and a notification device 80. To.

The communication line 65 is a signal path that electrically connects between devices and devices and exchanges data. In the first embodiment, data is exchanged via CAN (Control Area Network). In addition, each device of the in-vehicle system 100 may be connected by a bus. The devices and devices of the in-vehicle system 100 are connected via communication, and the communication to be used may be wired communication or wireless communication.

The sensors 40 include a vehicle speed sensor 41, a steering angle sensor 42, an accelerator sensor 43, a brake sensor 44, a shift sensor 45, a winker sensor 46, a hazard sensor 47, a wiper sensor 48, a light sensor 49, a door open / close sensor 50, a driver camera 51, and the like. Sensors that detect the inside and outside information of the own vehicle 1, such as the seating sensor 52, the acceleration sensor 53, the angular speed sensor 54, the GPS (Global Positioning System) device 55, the navigation system 56, the outside camera 57, the outside sensor 58, and the illuminance sensor 59. Is.

The vehicle speed sensor 41 is a sensor that detects the speed of the own vehicle 1, and outputs an electric signal (vehicle speed pulse) according to the wheel speed to the overall control ECU 10. The electric signal according to the wheel speed becomes the vehicle speed information.

The steering angle sensor 42 is a sensor that detects the steering angle of the own vehicle 1, and outputs an electric signal corresponding to the steering angle to the overall control ECU 10. The electric signal corresponding to the steering angle becomes the steering angle information.

The accelerator sensor 43 is a sensor that detects the opening degree of the accelerator of the own vehicle 1, that is, the operation amount of the accelerator pedal, and outputs the operation amount information of the accelerator pedal to the overall control ECU 10.

The brake sensor 44 is a sensor that detects the operation amount of the brake pedal of the own vehicle 1, and outputs the operation amount information of the brake pedal to the overall control ECU 10.

The shift sensor 45 is a sensor that detects the current state or change of the shift lever of the own vehicle 1, and outputs the operation information of the shift lever such as the shift change by the user of the own vehicle 1 to the overall control ECU 10.

The winker sensor 46 is a sensor that detects the operation of the winker (direction indicator) of the own vehicle 1, and when the user operates the winker, the information of the winker operation instruction is output to the overall control ECU 10.

The hazard sensor 47 is a sensor that detects the operation of the hazard switch of the own vehicle 1, detects the operation of the user's hazard switch, and outputs the operation information to the overall control ECU 10.

The wiper sensor 48 is a sensor that detects the wiper operation of the own vehicle 1, and when the user operates the wiper, this operation instruction information is output to the overall control ECU 10.

The light sensor 49 is a sensor that detects the operation of the light lever of the user's own vehicle 1, and outputs the operation information of the user's light to the overall control ECU 10.

The door open / close sensor 50 is a sensor that detects the open / close of the door of the own vehicle 1, and outputs the door open / close information to the overall control ECU 10.

The driver camera 51 is a camera (imaging device) provided facing the driver seat of the own vehicle 1 and has a function of photographing a user sitting in the driver seat. The driver camera 51 captures an image of the user's face and upper body, and outputs the captured image to the overall control ECU 10.

The seating sensor 52 is provided on the seat of the own vehicle 1 and is a sensor that detects the seating state of the user, and is realized by, for example, a pressing sensor or the like. The seating sensor 52 outputs this information to the overall control ECU 10 when the user sits down or leaves the seat. A plurality of seating sensors 52 may be provided on the seat, and the overall control ECU 10 estimates the posture of the user or the like based on the information of the plurality of pressing sensors.

The acceleration sensor 53 is a sensor that detects the acceleration of the own vehicle 1, and is composed of, for example, a 3-axis acceleration sensor. The acceleration sensor 53 outputs the acceleration information of the own vehicle 1 to the overall control ECU 10.

The angular velocity sensor 54 is a sensor that detects the angular velocity (gyro) of the own vehicle 1, and the overall control ECU 10 detects the turning speed and the like of the own vehicle 1 based on this angular velocity information. The angular velocity sensor 54 outputs the detected angular velocity information to the overall control ECU 10.

The GPS device 55 is a device that detects the position information of the own vehicle 1 by using the radio wave transmitted by the artificial satellite using the global positioning system (Global Positioning System), and is the information of the coordinates which is the position information of the own vehicle 1. Is output to the overall control ECU 10 and the navigation system 56.

The navigation system 56 has map information, and has a function of calculating a recommended route to the destination of the own vehicle 1 based on the position information and the map information of the own vehicle 1. Further, the navigation system 56 has a communication function, and may acquire external information such as traffic congestion information and traffic closure information from a server (not shown) and calculate a recommended route based on these. Alternatively, the navigation system 56 has a function of sending the position information and the destination information of the own vehicle 1 to the server, and is configured as a system that calculates the recommended route on the server side and receives the information of the recommended route. You may be. The navigation system 56 outputs the calculated route information to the overall control ECU 10.

The out-of-vehicle camera 57 is a camera (imaging device) provided for taking an image of the outside of the own vehicle 1. The outside camera 57 is provided, for example, in front of, behind, and to the left and right of the own vehicle 1, and outputs each captured image to the overall control ECU 10. The overall control ECU 10 can detect and recognize the target person and detect and recognize objects such as other vehicles and obstacles based on the input captured image.

The vehicle exterior sensor 58 is a sensor capable of detecting an object around the vehicle exterior of the own vehicle 1, and is composed of, for example, an ultrasonic sensor, a radar sensor, a millimeter wave radar sensor, a rider, an infrared laser sensor, and the like. The vehicle exterior sensor 58 outputs this detection information to the overall control ECU 10. The overall control ECU 10 detects the distance information to the object and the position information of the object based on the detection information of the object outside the vehicle input from the vehicle outside sensor 58. The detection of the distance information and the position information may be detected by the overall control ECU 10 as in the first embodiment, or the distance information and the position information are calculated based on the information detected by the vehicle exterior sensor 58, and the overall control is performed. This information may be output to the ECU 10.

The illuminance sensor 59 is provided toward the outside of the own vehicle 1 and is a sensor that detects the illuminance (brightness) outside the vehicle. The illuminance sensor 59 outputs the detected illuminance information to the overall control ECU 10.

The communication device 60 performs wireless communication with another communication device existing outside the vehicle of the own vehicle 1. The communication device 60 uses a specific frequency band to perform vehicle-to-vehicle communication with another vehicle, road-to-vehicle communication with a roadside unit, or communication with a communicable electronic device such as a smartphone owned by a person. The communication performed by the communication device 60 is a communication using a unique communication using a specific fixed frequency band, or a communication standard standardized for performing communication between an in-vehicle communication device and an external communication device. It may be communication using existing communication standards such as wireless LAN (Local Area Network), Bluetooth (registered trademark), Zigbee (registered trademark) and the like.

The communication device 60 includes an antenna 61, a transmitting unit 62, and a receiving unit 63. The communication device 60 transmits a wireless signal from the transmitting unit 62 to another communication device via the antenna 61, and transmits a wireless signal such as surrounding congestion status information from the receiving unit 63 from the receiving unit 63 via the antenna 61. Receive.

For example, the overall control ECU 10 has information on whether or not another vehicle 2 exists around the own vehicle 1 from the outside camera 57, the outside sensor 58, and the like, and features such as the color, shape, and model number of the other surrounding vehicles 2. Information is acquired, and the position information of the own vehicle 1, the speed regulation, the straight road, the number of lanes, and the shape information of the road such as a curved road are acquired from the GPS device 55, the navigation system 56, the communication device 60, etc., and the outside camera 57 The situation around the own vehicle 1 including the blind spot information or the blind spot image is acquired from. The display system 30 acquires such information from the overall control ECU 10. At least one of the sensor 40 and the communication device 60 acquires the blind spot information or the blind spot image, and transmits the blind spot information or the blind spot image to the overall control ECU 10.

The overall control ECU 10 is an ECU having a function of controlling the entire vehicle of the own vehicle 1. By acquiring the information detected from the sensor 40 and sending instructions and information to properly operate each part of the own vehicle 1 based on this information, the control of the entire vehicle of the own vehicle 1 is executed. The overall control ECU 10 includes a processor 11 and a memory 12.

The processor 11 is a CPU (Central Processing Unit) that executes calculation processing in the overall control ECU 10 by reading and executing a program stored in the memory 12. Specifically, the processor 11 loads at least a part of the OS (Operating System) stored in the memory 12 and executes the program while executing the OS. The processor 11 connects to each device and device via the communication line 65, and controls each device and device. Since the processor 11 may be an IC (Integrated Circuit) that performs processing, it may be a calculation processing circuit, an electric circuit, a controller, a central processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like. May be a combination of.

It should be noted that various functions of the overall control ECU 10 may be realized by another processor, or various functions may be collectively realized by one processor.

The memory 12 stores software, firmware, a program in which a combination of software and firmware is described, an OS, various information, and the like. The memory 12 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EPROM (Electrically Memory) Non-volatile or volatile semiconductor memory such as (Solid State Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versaille Disc) and the like.

The memory 12 may realize the functions of each part in separate memories, or may collectively realize the functions of each part in one memory.

The overall control ECU 10 has, for example, a ROM as a memory 12 which is a non-volatile storage device for storing one or more programs, and a RAM which is a volatile storage device used by the processor 11 as an expansion area for programs and various information. The processor 11 reads a program from the ROM, expands the read program on the RAM, and executes the calculation process.

The function of the overall control ECU 10 can be realized by hardware, software, firmware, or a combination of hardware, software, and firmware. The functions of each part of the overall control ECU 10 may be partially realized by dedicated hardware and partially realized by software or firmware. For example, even if a part of the overall control ECU 10 realizes its function by a processing circuit as dedicated hardware, and the rest realizes its function by reading and executing a program stored in the memory by the CPU. good.

The driving device 70 is a device for driving the own vehicle 1. The driving device 70 includes a headlight driver 71, a headlight 72, an engine 73, a transmission 74, a brake actuator 75, a steering actuator 76, a winker 77, and a HUD (Head-Up Display) 78. ing.

The headlight driver 71 is a drive device that drives the headlight 72. The headlight driver 71 drives the headride 72 based on the instruction from the overall control ECU 10, and controls operations such as turning on / off the headlight 72 and switching between high beam and low beam.

The headlight 72 is an irradiation device provided on the front side of the vehicle body of the own vehicle 1 and irradiates the front of the vehicle body with light. The headlights 72 are provided on the left and right sides of the front side of the vehicle body of the own vehicle 1, respectively, and a high beam that illuminates a farther distance by the structure of a light guide portion that guides the light emitted from the light or by switching a plurality of lights. And the low beam that illuminates a shorter distance than the high beam can be switched.

The engine 73 is an internal combustion engine that generates power to drive the own vehicle 1. The engine 73 generates power for rotating the wheels by burning fuel such as gasoline. The engine 73 operates based on an instruction from the overall control ECU 10.

The transmission 74 is composed of gears, shafts, etc., and has a function of transmitting power to the wheels. The transmission 74 changes the torque transmitted to the wheels of the own vehicle 1 by changing the gear based on the instruction from the overall control ECU 10.

The brake actuator 75 is a mechanism for operating a brake (reducer) for decelerating the own vehicle 1. The brake actuator 75 operates the brake based on the instruction from the overall control ECU 10 to decelerate the own vehicle 1.

The steering actuator 76 is a mechanism that changes the direction of the wheels of the own vehicle 1 and operates the steering (steering device) that controls the traveling direction of the own vehicle 1. The steering actuator 76 controls the steering based on the instruction from the overall control ECU 10, and controls the traveling direction of the own vehicle 1.

The winker 77 is a direction indicator for indicating the traveling direction of the own vehicle 1 to the outside of the vehicle by light emission. The winker 77 blinks based on an instruction from the overall control ECU 10, and indicates the traveling direction of the own vehicle 1 to the outside of the vehicle.

The HUD 78 is a transmissive image display device provided so as to be superimposed on the windshield of the own vehicle 1. The HUD 78 displays various images based on instructions from the overall control ECU 10. The HUD 78 presents various information to the user in the vehicle 1 of the own vehicle 1 by displaying an image. The HUD 78 may be used as a display for the navigation system 56.

The notification control ECU 15 is an ECU having a function of controlling the notification device 80 of the own vehicle 1. The notification control ECU 15 acquires vehicle inside / outside information of the own vehicle 1 from the overall control ECU 10, determines the status and state of the own vehicle 1 based on the acquired vehicle inside / outside information, and informs the target person of the blind spot image by the notification device 80. It is a device having a function of notifying notification information including. The notification control ECU 15 includes a processor 16 and a memory 17.

The processor 16 is a CPU (Central Processing Unit) that executes a calculation process in the notification control ECU 15 by reading and executing a program stored in the memory 17. Specifically, the processor 16 loads at least a part of the OS (Operating System) stored in the memory 17, and executes the program while executing the OS. The processor 16 connects to each device and device via the communication line 65, and controls each device and device. Since the processor 16 may be an IC (Integrated Circuit) that performs processing, it may be a calculation processing circuit, an electric circuit, a central processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or a combination thereof. It may be the one.

Control of the vehicle interior / external information acquisition unit 27, the blind spot image generation unit 28, the blind spot image acquisition unit 21, the determination unit 22, the display image generation unit 23, and the condition giving unit 29 included in the display control device 20 in FIG. The unit 26 is realized by reading and executing the program loaded in the memory 17 by the processor 16.

It should be noted that various functions of the notification control ECU 15 may be realized by another processor, or various functions may be collectively realized by one processor.

The memory 17 stores software, firmware, a program in which a combination of software and firmware is described, an OS, various information, and the like. The memory 17 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EPROM (Electrically Memory) Non-volatile or volatile semiconductor memory such as (Solid State Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versaille Disc) and the like.

Control of the vehicle interior / external information acquisition unit 27, the blind spot image generation unit 28, the blind spot image acquisition unit 21, the determination unit 22, the display image generation unit 23, and the condition giving unit 29 included in the display control device 20 in FIG. The unit 26 is realized by a program stored in the memory 17. Further, the blind spot image DB 24b, the own vehicle image DB 24a, and the condition information DB 25 are realized by the memory 17.

The memory 17 may realize the functions of each part in separate memories, or may collectively realize the functions of each part in one memory.

The notification control ECU 15 has, for example, a ROM as a memory 17 which is a non-volatile storage device for storing one or more programs, and a RAM which is a volatile storage device used by the processor 16 as an expansion area for programs and various information. The processor 16 reads a program from the ROM, expands the read program on the RAM, and executes the calculation process.

The function of the notification control ECU 15 can be realized by hardware, software, firmware, or a combination of hardware, software, and firmware. Some of the functions of each part of the notification control ECU 15 may be realized by dedicated hardware, and some may be realized by software or firmware. For example, even if a part of the notification control ECU 15 realizes its function by a processing circuit as dedicated hardware, and the rest realizes its function by reading and executing a program stored in the memory by the CPU. good.

The notification device 80 receives notification information including at least a display image from the notification control ECU 15, and notifies the target person of the notification information including displaying at least the display image. The notification device 80 is an external light device 81, a projector device 86, a body light device 89, a sound device 92, and the like. However, the notification device 80 includes at least one device capable of displaying an image, such as an external light device 81, a projector device 86, and a body light device 89. The notification device 80 corresponds to the notification device 31 in FIG.

The external light device 81 is an irradiation device attached to the outside from the vehicle body of the own vehicle 1. The external light device 81 irradiates the road surface or a nearby wall surface with light, and the target person existing outside the own vehicle 1 around the own vehicle 1 is informed of the state of the own vehicle 1 and the current or future operation of the own vehicle 1. Notify the intention, the notice of the future operation of the own vehicle 1, or the warning. In order to notify the target person existing outside the own vehicle 1 of these notification information, the external light device 81 illuminates with a shape, color, position, size, brightness, position, timing, time, etc. suitable for the notification information. Irradiates the road surface and the like. The external light device 81 includes an external light driver 82 and an external light set 83.

The external light driver 82 is a drive device for driving the external light set 83. The external light driver 82 controls the external light set 83 to irradiate a predetermined light. The external light driver 82 controls the combination of the irradiation timing, the irradiation time, and the like between the external light 84 and the external light 85 in the external light set 83. Further, the external light driver 82 operates a color filter, a shade, a light guide mechanism, etc. provided on the external light 84 and the external light 85, and has a predetermined shape at a position relative to the vehicle body of the predetermined own vehicle 1. Light of a predetermined color, a predetermined size, and a predetermined brightness is irradiated at a predetermined timing and at a predetermined time.

The external light set 83 includes a plurality of external lights (irradiation devices). Specifically, in the first embodiment, the external light set 83 includes an external light 84 and an external light 85. The external light 84 and the external light 85 are turned on based on the external light driver 82.

The projector device 86 is an image projection device attached to the outside from the vehicle body of the own vehicle 1. The projector device 86 irradiates the road surface or a nearby wall surface with light, and the state of the own vehicle 1 and the intention of the current or future operation of the own vehicle 1 to the target person existing outside the own vehicle 1 around the own vehicle 1. , Notice of future operation of own vehicle 1, warning, etc. In order to convey these notification information to the target person existing outside the own vehicle 1, the projector device 86 has a suitable shape, color, position, size, brightness, position, timing, time, etc. Irradiates (projects) light onto the road surface, etc. The projector device 86 includes a projector driver 87 and a projector 88.

The projector driver 87 is a driving device that drives the projector 88, and causes the projector 88 to irradiate a predetermined light. The projector driver 87 has a function of controlling the shape, color, position, size, brightness, timing, time, and the like of the light emitted by the projector 88.

The projector 88 is an irradiation (projection) device that irradiates (projects) light (image) to the outside of the own vehicle 1. The projector 88 irradiates a road surface or a wall surface outside the vehicle with light (image) based on the operation of the projector driver 87.

The body light device 89 is a light emitting device such as a display provided on the vehicle body of the own vehicle 1 and a device that shines in a line along the contour of the vehicle body. The body light device 89 informs the target person who exists outside the own vehicle 1 around the own vehicle 1 the state of the own vehicle 1, the intention of the current or future operation of the own vehicle 1, and the advance notice of the future operation of the own vehicle 1. , Or give a warning. Further, the body light device 89 emits light having a predetermined shape, a predetermined color, a predetermined size, and a predetermined brightness at a predetermined position on the vehicle body surface of the own vehicle 1 at a predetermined timing and at a predetermined time. It emits light. The body light device 89 includes a body light driver 90 and a body light 91.

The body light driver 90 is a drive device that drives the body light 91. The body light driver 90 irradiates the body light 91 with a predetermined light. The body light driver 90 controls the shape, color, position, size, brightness, timing, time, etc. of the light emitted by the body light 91.

The body light 91 is a light emitting device provided so as to expose the light emitted from the outer surface of the vehicle body of the own vehicle 1. The body light 91 in the first embodiment is composed of an LCD (Liquid Crystal Display) and an LED (Light Emitting Diode), and the light emitted by the LED is transmitted through the LCD to have a predetermined shape, color, and position. , The size and the brightness of the light are radiated to the outside of the own vehicle 1. In the first embodiment, an example in which the body light 91 is composed of an LCD and an LED is shown, but other light emission such as an organic EL (Electroluminescence) monitor, a dot LED, a liquid crystal monitor, an EL panel, and a rear projection type display is shown. It may be a device or a display device. Further, as for the place where the light is emitted, in the first embodiment, the display device 32 is provided on the front surface of the own vehicle 1, the display device 33 is provided on the rear surface of the own vehicle 1, and the display device 34 is on the left side of the own vehicle 1. Although it is provided on the surface and the display device 35 is provided on the right side surface of the own vehicle 1, it may be displayed on a window glass or the like.

The display devices 32 to 35 included in the notification device 31 in FIG. 2 are realized by an external light device 81, a projector device 86, a body light device 89, or a combination of the external light device 81, the projector device 86, and the body light device 89. In the first embodiment, the display devices 32 to 35 are body light devices 89, respectively.

The sound device 92 is an acoustic device provided on the vehicle body of the own vehicle 1. The sound device 92 tells the target person who exists outside the own vehicle 1 around the own vehicle 1 the state of the own vehicle 1, the intention of the current or future operation of the own vehicle 1, and the advance notice of the future operation of the own vehicle 1. Or notify a warning or the like. Further, the sound device 92 outputs a predetermined sound of a predetermined size, a predetermined timing, and a predetermined time at a predetermined position on the surface of the vehicle body of the own vehicle 1. The sound device 92 includes a sound driver 93 and a sound device 94.

The sound driver 93 is a drive device that drives the sound device 94. The sound driver 93 causes the sound device 94 to output a predetermined sound at a predetermined position at a predetermined time, at a predetermined timing, and at a predetermined time. The sound device 94 is a device that generates sound from the vehicle body of the own vehicle 1 toward a target person existing outside the own vehicle 1. The speaker 36 included in the notification device 31 in FIG. 2 is realized by the sound device 92.

Next, the operation of the display system 30 will be described. As a specific example, the situation as shown in FIG. 1 will be described.

FIG. 8 is a flowchart showing the operation of the display system 30 according to the first embodiment. The operation of the display system 30 will be described below with reference to FIG.

In step S1, the vehicle interior / external information acquisition unit 27 acquires vehicle interior / external information including blind spot information or blind spot image from the overall control ECU 10. Specifically, in the first embodiment, the vehicle inside / outside information acquisition unit 27 includes the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the target person as the inside / outside information of the vehicle. The blind spot information or the blind spot image of 3 is acquired.

In step S2, the vehicle inside / outside information acquisition unit 27 determines whether or not the blind spot information or the blind spot image included in the vehicle inside / outside information can be used as it is as a blind spot image. If the vehicle interior / external information acquisition unit 27 determines that the blind spot information or the blind spot image included in the vehicle interior / external information can be used as the blind spot image as it is, step S2: Yes. Specifically, for example, when the blind spot information or the blind spot image included in the inside / outside information of the vehicle is a blind spot image acquired from the camera outside the vehicle and which is in the line of sight of the subject 3, step S2: Yes. Become. The fact that the blind spot information or the blind spot image included in the vehicle interior / external information can be used as the blind spot image as it is means that the information included in the vehicle interior / external information is a blind spot image. The vehicle interior / external information acquisition unit 27 transmits the position information of the own vehicle 1, the position information of the target person 3, the line-of-sight information of the target person 3, and the blind spot image to the blind spot image acquisition unit 21, and proceeds to step S4.

On the other hand, if the vehicle interior / external information acquisition unit 27 determines that the blind spot information or the blind spot image included in the vehicle interior / external information cannot be used as the blind spot image as it is, step S2: No. Specifically, for example, when the blind spot information or the blind spot image included in the vehicle interior / external information is the radar information of the blind spot, step S2: No. Further, for example, when the blind spot information or the blind spot image included in the vehicle inside / outside information is an image of the vehicle outside camera in a range wider than the blind spot, step S2: No. For example, if the blind spot information or the blind spot image included in the vehicle interior / external information does not match the line of sight of the subject 3, step S2: No. The vehicle interior / external information acquisition unit 27 transmits the position information of the own vehicle 1, the position information of the target person 3, the line-of-sight information of the target person 3, and the blind spot information or the blind spot image of the target person 3 to the blind spot image generation unit 28. Then, the process proceeds to step S3.

In step S3, the blind spot image generation unit 28 receives the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the blind spot information or the blind spot of the target person 3 from the vehicle inside / outside information acquisition unit 27. Receive images and. The blind spot image generation unit 28 generates or extracts a blind spot image from the blind spot information or the blind spot image of the subject 3.

Specifically, for example, when the blind spot image or the blind spot image included in the vehicle interior / external information is the blind spot radar information, the blind spot image generation unit 28 uses the image parts of the blind spot image DB 24b from the blind spot radar information. Generate a blind spot image. Further, for example, the blind spot image generation unit 28 extracts the blind spot image according to the blind spot when the blind spot information or the blind spot image included in the vehicle inside / outside information is an image of the vehicle outside camera in a range wider than the blind spot. For example, when the blind spot information or the blind spot image included in the vehicle interior / external information does not match the line of sight of the target person 3, the blind spot image generation unit 28 generates a blind spot image that matches the line of sight of the target person 3. As a method for the blind spot image generation unit 28 to generate or extract a blind spot image from the blind spot information or the blind spot image of the subject 3, the prior art may be used. The blind spot image generation unit 28 transmits the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the blind spot image to the blind spot image acquisition unit 21, and proceeds to step S4.

In step S4, the blind spot image acquisition unit 21 receives the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the blind spot image from the vehicle inside / outside information acquisition unit 27 or the blind spot image generation unit 28. And get. The blind spot image acquisition unit 21 transmits the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the blind spot image to the determination unit 22, and proceeds to step S5.

In step S5, the determination unit 22 receives the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the blind spot image from the blind spot image acquisition unit 21. The determination unit 22 stores predetermined range information. In the first embodiment, the range information is within 5 meters from the own vehicle 1. The determination unit 22 determines whether or not the subject 3 exists within a predetermined range within 5 meters from the own vehicle 1. Specifically, for example, the determination unit 22 determines whether the target person 3 exists within a predetermined range of 5 meters from the own vehicle 1 based on the position information of the own vehicle 1 and the position information of the target person 3. Judge whether or not. Alternatively, the determination unit 22 determines whether or not the subject 3 exists in the predetermined range within 5 meters from the own vehicle 1 from the image within the range within 5 meters from the predetermined own vehicle 1. As a method for determining whether or not the subject 3 exists within a range of 5 meters from the own vehicle 1 determined in advance by the determination unit 22, the prior art may be used.

If the determination unit 22 determines that the target person 3 exists within a range of 5 meters from the own vehicle 1 determined in advance, step S5: Yes and the process proceeds to step S6. If the determination unit 22 determines that the target person 3 does not exist within a predetermined range of 5 meters from the own vehicle 1, step S5: No and the operation ends.

In step S6, the determination unit 22 determines whether or not an object such as another vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the target person 3. Specifically, for example, the determination unit 22 determines whether an object such as another vehicle 2 is shown in the blind spot image, which is a conventional technique. Or, for example, when the display system 30 acquires the position information and the map information of the other vehicle 2 and the position information is included in the blind spot image, the determination unit 22 obtains the position information and the map information of the other vehicle 2. From the blind spot image, it is determined whether an object such as another vehicle 2 exists in the blind spot. For example, in the determination unit 22, the display system 30 acquires the position information and the map information of the other vehicle 2, the position information and the map information of the other vehicle 2, the position information of the own vehicle 1, and the position information of the target person 3. It is determined from the above whether or not an object such as another vehicle 2 exists in the blind spot. As a method for the determination unit 22 to determine whether or not an object such as another vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the target person 3, the prior art may be used.

If the determination unit 22 determines that an object such as another vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the target person 3, step S6: Yes. The determination unit 22 transmits the position information of the own vehicle 1, the position information of the target person 3, the line-of-sight information of the target person 3, and the blind spot image to the display image generation unit 23, and proceeds to step S7. When the determination unit 22 determines that the object such as the other vehicle 2 does not exist in the blind spot of the own vehicle 1 as seen from the target person 3, step S6: No and the operation ends.

In step S7, the display image generation unit 23 receives the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the blind spot image from the determination unit 22. The display image generation unit 23 uses the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the blind spot image to match the line of sight of the target person 3 from the own vehicle image DB 24a. The image of the own vehicle 1 is acquired, and the process proceeds to step S8.

In step S8, the display image generation unit 23 generates a display image in which the image of the own vehicle 1 is superimposed on the blind spot image. The display image generation unit 23 generates a display image so that the target person 3 has the same position and the same line of sight as when he / she actually sees the own vehicle 1. Further, the display image generation unit 23 changes the display image to an easy-to-see magnification according to the distance between the target person 3 and the own vehicle 1. As a method for the display image generation unit 23 to generate a display image, a conventional technique such as machine learning may be used.

FIG. 9 is an example of the display image 7 before processing the image 8 of the own vehicle 1 according to the first embodiment. In the first embodiment, the display image generation unit 23 generates the display image 7 by superimposing the image 8 of the own vehicle 1 on the blind spot image which is the image of the other vehicle 2. The blind spot image may include not only the image of the other vehicle 2 but also another vehicle, an object such as a person, a background image, and the like. At the stage of step S8, the blind spot image, which is an image of the other vehicle 2, is not visible because it is hidden by the image 8 of the own vehicle 1.

Returning to FIG. 8, the display image generation unit 23 transmits the position information of the own vehicle 1, the position information of the target person 3, the line-of-sight information of the target person 3, and the display image to the condition giving unit 29, and steps are taken. Proceed to S9.

In step S9, the condition giving unit 29 receives the position information of the own vehicle 1, the position information of the target person 3, the line of sight information of the target person 3, and the display image from the display image generation unit 23. The condition giving unit 29 uses the position information of the own vehicle 1, the position information of the target person 3, the line-of-sight information of the target person 3, and the display image, and the notification device to be notified from the condition information DB 25 including the display device. 31 notification device information, notification pattern information, and notification timing information are acquired.

The condition giving unit 29 includes an identifier of the position information of the own vehicle 1, which is information inside and outside the vehicle, an identifier of the position information of the target person 3, an identifier of the line-of-sight information of the target person 3, and an identifier of the display image from the condition information DB 25. The notification device information, the notification pattern information, and the notification timing information corresponding to the above are acquired. Alternatively, the condition giving unit 29 uses machine learning to obtain an identifier of the position information of the own vehicle 1, an identifier of the position information of the target person 3, an identifier of the line-of-sight information of the target person 3, and an identifier of the blind spot image from the condition information DB 25. The notification device information, the notification pattern information, and the notification timing information corresponding to the above are acquired. As a method for the condition giving unit 29 to acquire the notification device information, the notification pattern information, and the notification timing information from the condition information DB 25, the prior art may be used. The notification device information, the notification pattern information, and the notification timing information do not have to be one set, and may be one set or a plurality of sets.

In the first embodiment, the condition giving unit 29 reduces the notification device information corresponding to the identifier of the display device 32 provided in front of the own vehicle 1 from the condition information DB 25 and the image of the own vehicle 1 to make a blind spot. Assuming that the notification pattern information that makes the image easier to see and the notification timing information that the display device 32 immediately starts the notification are acquired, the process proceeds to step S10.

In step S10, the condition giving unit 29 determines whether or not the notification pattern information is for processing the displayed image. Specifically, for example, the notification pattern information reduces the image of the own vehicle 1 to make the blind spot image easier to see, the notification pattern information that performs transmission processing on the image of the own vehicle 1, and a little image of the own vehicle 1. If the notification pattern information is an image as seen from a shifted viewpoint, or the notification pattern information is an image in which only the outline of the own vehicle 1 is a solid line or a dotted line, the condition giving unit 29 displays the notification pattern information. It is determined that the image is processed. When the notification pattern information is for processing the display image, the condition giving unit 29 goes to step S10: Yes and proceeds to step S11. In the first embodiment, since the notification pattern information is the notification pattern information that reduces the image of the own vehicle 1 to make the blind spot image easier to see, the condition giving unit 29 processes the display image by the notification pattern information. Is determined, step S10: Yes, and the process proceeds to step S11.

On the other hand, for example, the notification pattern information causes the control unit 26 to display the display so as to emphasize the display image, the notification pattern information for blinking the frame of the display device 32 in red, the notification pattern information for issuing an alarm sound together with the display, and the like. If this is the case, the condition giving unit 29 determines that the notification pattern information does not process the displayed image. If the notification pattern information does not process the displayed image, step S10: No. The condition giving unit 29 transmits the notification device information, the notification pattern information, the notification timing information, and the display image to the control unit 26, and proceeds to step S12.

In step S11, the condition giving unit 29 processes the display image. Specifically, in the first embodiment, the condition giving unit 29 acquires the notification pattern information from the condition information DB 25 by reducing the image of the own vehicle 1 to make the blind spot image easier to see, so that the image of the own vehicle 1 is obtained. Process. The condition giving unit 29 reduces the image of the own vehicle 1 according to the line of sight of the target person 3 acquired from the own vehicle image DB 24a by using the notification pattern information that reduces the image of the own vehicle 1 to make the blind spot image easier to see. .. For example, the condition giving unit 29 reduces the image of the own vehicle 1 based on the degree of reduction given in advance to the notification pattern information that reduces the image of the own vehicle 1 to make the blind spot image easier to see. Alternatively, the condition giving unit 29 determines the degree of reduction by machine learning, and reduces the image of the own vehicle 1 based on the degree of reduction. As a method for determining the degree of reduction by the condition-imparting unit 29, the prior art may be used.

FIG. 10 is an example of a display image 7 after processing the image 8 of the own vehicle 1 according to the first embodiment. Since the condition-imparting unit 29 reduces the image 8 of the own vehicle 1, the blind spot image, which is the image 9 of the other vehicle 2, becomes easy to see.

Returning to FIG. 8, the condition giving unit 29 transmits the notification device information, the notification timing information, and the display information to the control unit 26. In the first embodiment, the condition giving unit 29 transmits the notification device information corresponding to the identifier of the display device 32, the notification timing information at which the display device 32 immediately starts notification, and the display image to the control unit 26. Then, the process proceeds to step S12.

In the first embodiment, the condition-imparting unit 29 reduces the image of the own vehicle 1 to acquire only the notification pattern information that makes the blind spot image easier to see, but reduces the image of the own vehicle 1 to obtain the blind spot image. In addition to the notification pattern information that makes it easier to see, for example, the notification pattern information that emits an alarm sound together with the display may be acquired. In such a case, the condition giving unit 29 transmits the notification pattern information to the control unit 26 together with the notification device information, the notification timing information, and the display information.

Further, for example, the condition giving unit 29 is a notification device which is an identifier of the speaker 36 provided in front of the own vehicle 1 from the condition information DB 25 regardless of whether or not the notification pattern information is for processing the display image. Information, notification pattern information of the sound of the speaker 36, and notification timing information in which the speaker 36 immediately starts notification may be acquired. In such a case, the condition giving unit 29 generates sound information which is notification information generated from the speaker 36 and transmits it to the control unit 26. The broadcast information is display image, sound information, etc., and may be one or a plurality.

In step S12, the control unit 26 receives the notification device information, the notification pattern information, the notification timing information, and the display information from the condition giving unit 29. If necessary, the control unit 26 receives the notification pattern information and the notification information other than the display image such as sound information from the condition giving unit 29. In the first embodiment, the control unit 26 receives the notification device information which is an identifier of the display device 32, the notification timing information at which the display device 32 immediately starts notification, and the display image. The control unit 26 uses the notification device information which is an identifier of the display device 32, the notification timing information at which the display device 32 immediately starts notification, and the display image processed in step S11, and immediately receives the information. Controls to start displaying the display image on the display device 32.

FIG. 11 is an example of a diagram when the own vehicle 1 is viewed from the target person 3 according to the first embodiment. Specifically, FIG. 11 is a view taken from the dotted line arrow 5 when the own vehicle 1 is viewed from the subject 3 in FIG. 1. The display device 32 of the own vehicle 1 immediately starts displaying the display image 7 under the control of the control unit 26.

FIG. 12 is an example of a diagram when the own vehicle 1 according to the first embodiment is viewed from the front. Specifically, FIG. 12 is a view taken from the dotted line arrow 6 when the own vehicle 1 is viewed from directly in front of FIG. 1. Similar to FIG. 11, the display device 32 of the own vehicle 1 immediately starts displaying the display image 7 under the control of the control unit 26. However, since the displayed image 7 is an image when the own vehicle 1 is viewed from the target person 3, the displayed image 7 is different from the scenery when the own vehicle 1 is viewed from the front.

Returning to FIG. 8, when the control unit 26 receives the sound information of the speaker 36 as the notification information, it may be notified together with the display. For example, when the other vehicle 2 approaches the target person 3, the control unit 26 controls the speaker 36 so that the approaching running sound of the other vehicle 2 is emphasized and output. Further, when the control unit 26 receives the notification pattern information that causes the frame of the display device 32 to blink in red, the control unit 26 displays the display device 32 together with the display so that the display device displays the display so as to emphasize the display image. You may control to blink the frame of.

After the display device 32 displays the display image under the control of the control unit 26, the display system 30 ends the operation.

Step S5: No, step S6: No, and after executing step S12, the process returns to step S1, the power is turned off, or the end operation is performed. The above process is repeated until there is a trigger for termination. Each step may be repeated for each step as an independent process.

As described above, in the display system 30 of the first embodiment, the display image generation unit 23 generates a display image in which the blind spot image is superimposed on the image of the own vehicle 1, and the control unit 26 is provided in the own vehicle 1. Display the displayed image on the display device. Therefore, when the display system 30 of the first embodiment displays the blind spot image, the target person 3 can intuitively display the positional relationship of the blind spot image in an easy-to-understand manner.

Further, in the first embodiment, since the condition-imparting unit 29 reduces the image 8 of the own vehicle 1, the blind spot image which is the image 9 of the other vehicle 2 becomes easy to see, and the other vehicle 2 and the like existing in the blind spot can be easily seen. The visibility of the object can be increased.

In the first embodiment, the notification pattern information is the notification pattern information that reduces the image of the own vehicle 1 to make the blind spot image easier to see, and the condition giving unit 29 performs processing to reduce the image of the own vehicle 1. However, the condition-imparting unit 29 is processed to perform transparency processing on the image of the own vehicle 1 by the notification pattern information for performing transmission processing on the image of the own vehicle 1, as seen from a viewpoint in which the image of the own vehicle 1 is slightly shifted. Processing as if the image of the own vehicle 1 is viewed from a viewpoint shifted by the notification pattern information to be made into an image, and the image is made into an image in which only the outline of the own vehicle 1 is a solid line or a dotted line. You may perform the processing or the like. Even in this way, the same effect as that of the first embodiment can be obtained.

In the first embodiment, after the display system 30 generates the display image, the condition giving unit 29 notifies the notification device information, the notification pattern information, and the notification of the notification device 31 including the display device from the condition information DB 25. The timing information and the image of the own vehicle 1 were processed. However, in the display system 30, the condition giving unit 29 acquires the notification device information, the notification pattern information, and the notification timing information of the notification device 31 including the display device from the condition information DB 25, and the image of the own vehicle 1 is obtained. The display image may be generated after processing. Even in this way, the same effect as that of the first embodiment can be obtained.

Embodiment 2.
In the first embodiment, the notification pattern information is the notification pattern information that reduces the image of the own vehicle 1 to make the blind spot image easier to see, and the condition giving unit 29 reduces the image of the own vehicle 1 in the display image. In the second embodiment, the notification pattern information is the notification pattern information that enlarges the image of the other vehicle 2 to make the blind spot image easier to see, and the condition giving unit 29 enlarges the image of the other vehicle 2 of the blind spot image. By performing this processing, when displaying the blind spot image, it is possible to intuitively and easily display the range of the blind spot image to the target person, and at the same time, an object such as another vehicle 2 existing in the blind spot can be displayed. The visibility of the object can be increased. Other than that, it is the same as the first embodiment. In the following description, the configurations and operations described in the first embodiment are designated by the same reference numerals, and duplicate description will be omitted.

As in the case of the first embodiment, the case where the present disclosure is applied to the situation shown in FIG. 1 will be described below as an example of the second embodiment.

The block diagram of the display system 30 including the display control device 20 according to the second embodiment of the present disclosure is the same as that of FIG. 2 of the first embodiment. Further, the hardware configuration diagram of the in-vehicle system 100 of the own vehicle 1 according to the second embodiment is the same as that of FIG. 7.

Next, the operation of the display system 30 according to the second embodiment will be described.

FIG. 13 is a flowchart showing the operation of the display system 30 according to the second embodiment of the present disclosure. The operation of the display system 30 will be described below with reference to FIG.

Steps S1 to S8 are the same as steps S1 to S8 of the first embodiment.

In step S19, in the second embodiment, the condition giving unit 29 enlarges the image of the other vehicle 2 from the condition information DB 25, instead of the notification pattern information that reduces the image of the own vehicle 1 to make the blind spot image easier to see. The notification pattern information that makes the blind spot image easier to see is acquired. Other than that, step S19 is the same as step S9 of the first embodiment.

In step S20, in the second embodiment, since the notification pattern information is the notification pattern information that enlarges the image of the other vehicle 2 to make the blind spot image easier to see, the condition giving unit 29 processes the display image by the notification pattern information. It is determined that the image is to be used, step S20: Yes, and the process proceeds to step S21. Other than that, step S20 is the same as step S10 of the first embodiment.

In step S21, in the second embodiment, the condition giving unit 29 obtains the notification pattern information from the condition information DB 25 by enlarging the image of the other vehicle 2 to make the blind spot image easier to see, so that the image of the other vehicle 2 is processed. do. The condition giving unit 29 enlarges the image of the other vehicle 2 which is the blind spot image by using the notification pattern information which enlarges the image of the other vehicle 2 to make the blind spot image easier to see. For example, the condition giving unit 29 enlarges the image of the other vehicle 2 based on the degree of enlargement given in advance to the notification pattern information that enlarges the image of the other vehicle 2 to make the blind spot image easier to see. Alternatively, the condition giving unit 29 determines the degree of enlargement by machine learning, and enlarges the image of the other vehicle 2 based on the degree of enlargement. As a method for determining the degree of enlargement by the condition-imparting unit 29, the prior art may be used.

In the second embodiment, the condition giving unit 29 obtains only the notification pattern information that enlarges the image of the other vehicle 2 to make the blind spot image easier to see, but enlarges the image of the other vehicle 2 to obtain the blind spot image. In addition to the notification pattern information that makes it easier to see, for example, the notification pattern information that emits an alarm sound together with the display may be acquired. In such a case, the condition giving unit 29 transmits the notification pattern information to the control unit 26 together with the notification device information, the notification timing information, and the display information.

In this way, the condition giving unit 29 enlarges the image of the other vehicle 2, so that the blind spot image, which is the image of the other vehicle 2, becomes easier to see. Other than that, step S21 is the same as step S11 of the first embodiment.

Step S12 is the same as step S12 of the first embodiment.

As described above, in the display system 30 of the second embodiment, the display image generation unit 23 generates a display image in which the blind spot image is superimposed on the image of the own vehicle 1, and the control unit 26 is provided in the own vehicle 1. Display the displayed image on the display device. Therefore, when the display system 30 of the second embodiment displays the blind spot image, the target person 3 can intuitively display the positional relationship of the blind spot image in an easy-to-understand manner.

Further, in the second embodiment, the condition-imparting unit 29 enlarges the image of the other vehicle 2, so that the blind spot image, which is an image of the other vehicle 2, becomes easier to see, and the object of the other vehicle 2 or the like existing in the blind spot becomes easier to see. Visibility can be improved.

In the second embodiment, the notification pattern information is the notification pattern information that enlarges the image of the other vehicle 2 to make the blind spot image easier to see, and the condition-imparting unit 29 processes to enlarge the image of the other vehicle 2 that is the blind spot image. Was done. However, the condition-imparting unit 29 is processed to enlarge the size of at least one part of the blind spot image by the notification pattern information that enlarges the size of at least one part of the blind spot image, and the own vehicle is used for at least one part of the blind spot image. The position may be shifted so that the overlap with the image of the own vehicle 1 is reduced for at least one portion of the blind spot image by the notification pattern information for shifting the position so as to reduce the overlap with the image of the own vehicle 1. Even in this way, the same effect as that of the second embodiment can be obtained. The condition giving unit 29 attaches an arrow to the image of the object such as the other vehicle 2, emphasizes the outline color of the object such as the other vehicle 2 by adding a conspicuous color such as red or yellow, and emphasizes the other vehicle 2 or the like. Processing may be performed to emphasize the image of the other vehicle 2, which is a blind spot image such as changing the color of the object.

In the second embodiment, after the display system 30 generates the display image, the condition giving unit 29 notifies the notification device information, the notification pattern information, and the notification of the notification device 31 including the display device from the condition information DB 25. The timing information was acquired, and the image of the other vehicle 2, which is a blind spot image, was processed. However, in the display system 30, the condition giving unit 29 acquires the notification device information, the notification pattern information, and the notification timing information of the notification device 31 including the display device from the condition information DB 25, and the image of the other vehicle 2 is obtained. The display image may be generated after processing. Even in this way, the same effect as that of the second embodiment can be obtained.

In the first and second embodiments, as an example in which the control unit 26 causes the display device to display the display so as to emphasize the display image, the control unit 26 receives the notification pattern information that causes the frame of the display device 32 to blink in red. Shown an example of controlling the frame of the display device 32 to blink in red together with the display. However, the control unit 26 may blink at least one part of the display image in order to make the display device display so as to emphasize the display image. An example of controlling the frame of the display device 32 to blink in red together with the display is an example of blinking at least one part of the display image. In this way, the control unit 26 causes a display device that blinks the frame of the display device 32 or controls to blink at least one part of the display image to display the display image in an emphasized manner. As a result, the attention of the subject 3 can be further attracted.

In the first and second embodiments, in step S5, the determination unit 22 determines whether or not the target person 3 exists in the predetermined range of the own vehicle 1, and within the predetermined range of the own vehicle 1. When the target person 3 exists, it is determined that the display image is displayed on the target person 3. By doing so, the display system 30 can contribute to energy saving by not displaying the display when the target person 3 does not exist. In addition, when the target person 3 does not exist, the display system 30 uses characters such as "please go first", "thank you", and "pause" instead of displaying the displayed image to communicate with the target person 3. Display, advertisement display, reservation, vacancy, etc. can be displayed when the own vehicle 1 is a taxi. Further, when the own vehicle 1 is parked or stopped, the display system 30 may display an image on the opposite side of each display device on each display device in order to eliminate a blind spot caused by the own vehicle 1. good.

In the first and second embodiments, in step S6, the determination unit 22 determines whether or not an object such as another vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the subject 3, and is viewed from the subject 3. When an object such as another vehicle 2 exists in the blind spot of the own vehicle 1, it is determined that the display image is displayed on the target person 3. By doing so, the subject 3 can predict some change in the traffic environment in the direction of the own vehicle 1 because the display image is displayed on the display device. Specifically, for example, since the subject 3 can recognize that the display has started as a warning to the other vehicle 2, the situation can be grasped earlier.

In the first and second embodiments, the display system 30 uses the notification pattern information to notify the speaker 36 of the sound volume such as the shape, color, size, brightness, position, etc. of the image displayed on the display devices 32 to 35. By performing additional processing such as pitch, pitch, and type of sound, the visual and auditory senses of the subject 3 can be impacted, so that the situation can be grasped more quickly.

In the first and second embodiments, when the notification pattern information includes the notification pattern information that causes the frame of the display device 32 to blink in red, the display image generation unit 23 generates the display image and then the control unit 26 displays the display. The frame of the display device 32 is blinked in red on the device 32, but the notification pattern information is used as notification pattern information such as blinking an image of an object such as a vehicle 2 in addition to the blind spot image, and the display image generation unit 23 generates a display image. After that, the control unit 26 may blink an image of an object such as the vehicle 2 in addition to the blind spot image.

In the first and second embodiments, the display image generation unit 23 acquires the image of the own vehicle 1 that matches the line of sight of the target person 3, and superimposes the image of the own vehicle 1 on the blind spot image that matches the line of sight of the target person 3. And generated a display image. By doing so, the display system 30 can intuitively and easily display the range of the blind spot image by the subject 3. However, when the vehicle interior / external information acquisition unit 27 can acquire the eye height information of the target person 3 from the overall control ECU 10, the display image generation unit 23 receives the position information of the own vehicle 1 which is the vehicle interior / external information and the position information of the own vehicle 1. Using the position information of the target person 3, the line-of-sight information of the target person 3, the eye height information of the target person 3, and the blind spot image, the target person 3 is aligned with the line of sight of the target person 3 from the own vehicle image DB 24a. An image of the own vehicle 1 as seen from the height of the eyes of the own vehicle 1 may be acquired. Then, the display image generation unit 23 superimposes the image of the own vehicle 1 on the blind spot image of the own vehicle 1 viewed from the height of the eyes of the target person 3 in accordance with the line of sight of the target person 3, and the line of sight of the target person 3. A display image of the own vehicle 1 may be generated from the height of the eyes of the subject 3. Here, the vehicle interior / external information acquisition unit 27 corresponds to the eye height information acquisition unit.

Specifically, for example, it is assumed that the own vehicle image DB 24a stores an image of the own vehicle 1 for each line of sight of the target person 3 and the height of the eyes of the target person 3. The display image generation unit 23 obtains the position information of the own vehicle 1, which is the information inside and outside the vehicle, the position information of the target person 3, the line-of-sight information of the target person 3, the eye height information of the target person 3, and the blind spot image. It is used to acquire an image of the own vehicle 1 from the own vehicle image DB 24a according to the line of sight of the target person 3 and the height of the eyes of the target person 3.

Alternatively, it is assumed that the own vehicle image DB 24a stores the image of the own vehicle 1 for each attribute such as the line of sight of the target person 3 and the male, female, and children of the target person 3. The display image generation unit 23 determines the attribute from the eye height information of the target person 3, and the position information of the own vehicle 1 which is the information inside and outside the vehicle, the position information of the target person 3, and the line of sight information of the target person 3. , The blind spot image and the attribute information are used to acquire an image of the own vehicle 1 from the own vehicle image DB 24a according to the line of sight of the target person 3 and the height of the eyes of the target person 3.

Alternatively, the display image generation unit 23 uses the position information of the own vehicle 1 which is the information inside and outside the vehicle, the position information of the target person 3, the line of sight information of the target person 3, and the blind spot image from the own vehicle image DB 24a. The image of the own vehicle 1 according to the line of sight of the target person 3 is acquired. The display image generation unit 23 may process the acquired image of the own vehicle 1 by using the eye height information of the target person 3. As described above, by using the image of the own vehicle 1 that matches not only the line of sight of the target person 3 but also the height of the eyes of the target person 3, there is an effect that the target person 3 can more easily recognize the broadcast information. For example, if the display image of a male eye is displayed on the display device, it may be difficult to recognize because the display image is from a height that the child has never seen, but the eye height is taken into consideration. This makes it easier to recognize.

In the first and second embodiments, the line-of-sight information of the target person 3 is acquired from the overall control ECU 10 by the vehicle inside / outside information acquisition unit 27, but it is assumed that the target person 3 is looking at the own vehicle 1, and the determination unit 22. May calculate the line-of-sight information of the target person 3 from the position information of the own vehicle 1 and the position information of the target person 3.

In the first and second embodiments, the determination unit 22 has the target person 3 within a range of 5 meters from the own vehicle 1 determined in advance from the position information of the own vehicle 1 and the position information of the target person 3. Although it is determined whether or not the range is 5 meters, the range may be any range such as 3 meters and 10 meters. Further, although the range is set within 5 meters from the own vehicle 1 determined in advance, the shape of a predetermined rectangular equal range surrounding the own vehicle 1 with a radius of 5 meters from the display device 32 may be any shape.

In the first and second embodiments, in step S5, the determination unit 22 uses the position information of the own vehicle 1 and the position information of the target person 3 within a predetermined range of the target person within 5 meters from the own vehicle 1. It was determined whether or not 3 was present. Further, in step S6, the determination unit 22 determines whether or not an object such as another vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the target person 3, and the determination unit 22 displays a display image on the target person 3. It was judged whether or not to do it. However, the determination unit 22 predicts the operation of the own vehicle 1 from the gear change information, the steering angle information, etc., and the other vehicle from the speed information of the other vehicle 2, the position information of the other vehicle 2, the operation information of the winker, and the like. The operation of 2 may be predicted, and it may be determined whether or not the determination unit 22 displays the display image on the target person 3. For example, the determination unit 22 may determine whether or not to display the display image on the target person 3 by using the speed information of the other vehicle 2 and the information on the traveling direction. As a method of determining whether or not the determination unit 22 displays the display image on the target person 3, the prior art may be used. Further, the condition giving unit 29 may change the display image generated by the information inside and outside the vehicle, or process the display image. By doing so, the subject 3 can easily recognize the situation of the object such as the other vehicle 2 in the blind spot and its change.

In the first and second embodiments, in step S5, the determination unit 22 uses the position information of the own vehicle 1 and the position information of the target person 3 within a predetermined range of the target person within 5 meters from the own vehicle 1. It was determined whether or not 3 was present, and if the subject 3 was present, it was determined that the display image would be displayed on the subject 3. However, when the target person 3 goes out of the predetermined range within 5 meters from the own vehicle 1 and disappears in the middle of the display, or the target person 3 does not cross the pedestrian crossing 4. If the display is stopped at, the display may be terminated and it may be determined that the displayed image is not displayed. The control unit 26 controls not to display the display image on the display device when the determination unit 22 determines that the display is not displayed.

In the first and second embodiments, in step S5, the determination unit 22 uses the position information of the own vehicle 1 and the position information of the target person 3 within a predetermined range of the target person within 5 meters from the own vehicle 1. It was determined whether or not 3 was present, and if the subject 3 was present, it was determined that the display image would be displayed on the subject 3. However, the determination unit 22 may display a display image according to a predetermined line of sight even when the target person 3 is not present. By doing so, even if the subject 3 is in a position where the determination unit 22 cannot determine the existence of the subject 3, the subject 3 can grasp the situation of the blind spot early.

In the first and second embodiments, in step S6, the determination unit 22 determines whether or not an object such as another vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the target person 3, and displays an image on the target person 3. Was determined whether or not to display. However, even if an object such as another vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the target person 3, the determination unit 22 is not dangerous to the target person 3 from the prediction of the behavior of the object such as the other vehicle 2. If it is determined, it may be determined that the displayed image is not displayed. Further, when the display image is displayed, the determination unit 22 may end the display and determine that the display image is not displayed. For example, when the other vehicle 2 stops behind the own vehicle 1, right beside, or the like, or when the other vehicle 2 leaves the blind spot and passes the target person 3, the determination unit 22 ends the display and the target person 3 It is determined that the display image is not displayed on. The control unit 26 controls not to display the display image on the display device when the determination unit 22 determines that the display is not displayed. By doing so, it becomes easy for the subject 3 to confirm that there is no danger in the surroundings, which leads to the peace of mind of the subject 3.

In the first and second embodiments, in step S6, the determination unit 22 determines whether or not an object such as another vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the subject 3, and is viewed from the subject 3. When an object such as another vehicle 2 exists in the blind spot of the own vehicle 1, it is determined that the display image is displayed to the target person 3. However, the determination unit 22 may display the display image even when the object such as the other vehicle 2 does not exist in the blind spot of the own vehicle 1 as seen from the target person 3. By doing so, it becomes easy for the subject 3 to confirm that there is no danger in the surroundings, which leads to the peace of mind of the subject 3.

In the first and second embodiments, the determination unit 22 determines the existence of the target person 3 in step S5, and then the object such as the other vehicle 2 exists in the blind spot of the own vehicle 1 as seen from the target person 3 in step S6. It was determined whether or not to display the display image on the target person 3. However, the display system 30 may take either step S5 or step S6 first. Further, the determination unit 22 may determine anywhere until the display image is displayed on the display device in step S12.

In the first and second embodiments, the blind spot image acquisition unit 21 acquires the blind spot image in step S4, but the blind spot image acquisition unit 21 acquires the blind spot image after step S5, after step S6, after step S7, and the like. Images may be acquired. In the display system 30, the condition giving unit 29 acquires the notification device information, the notification pattern information, and the notification timing information of the notification device 31 including the display device from the condition information DB 25, and images of the other vehicle 2. When the display image is generated after the processing of the above, the condition giving unit 29 obtains the notification device information, the notification pattern information, and the notification timing information of the notification device 31 including the display device from the condition information DB 25. After the acquisition, the blind spot image acquisition unit 21 may acquire the blind spot image.

Although the display system 30 when the own vehicle 1 is stopped has been described in the first and second embodiments, the present disclosure can be applied even when the own vehicle 1 is parked or moving. For example, the display system 30 may start displaying while the own vehicle 1 is traveling several tens of meters before the pedestrian crossing 4. Further, the case where the pedestrian 3 is trying to cross in front of the own vehicle 1 which is stopped as the target person has been described, but when the target person 3 is moving, even if the target person 3 is stopped, the book Disclosure is applicable. For example, even if the pedestrian 3 does not want to cross the pedestrian crossing 4 and stops, another vehicle 2 may plunge into it. In that case, the subject 3 can be alerted by displaying the display image of the present disclosure.

In this case, the vehicle interior / external information acquisition unit 27 acquires other vehicle information indicating the information of the other vehicle 2 existing in the blind spot, and the condition giving unit 29 can be further targeted if the processing is changed according to the other vehicle information. Person 3 can be alerted. Specifically, the vehicle inside / outside information acquisition unit 27 acquires speed information, traveling direction information, and the like of the other vehicle 2 existing in the blind spot, which is information on the other vehicle. The determination unit 22 predicts the future operation of the other vehicle 2 from the speed information, the traveling direction information, and the like, and the condition giving unit 29 changes the processing according to the other vehicle information. Here, the vehicle inside / outside information acquisition unit 27 corresponds to another vehicle information acquisition unit.

In the first and second embodiments, the case where the subject 3 is one pedestrian has been described, but the present disclosure can be applied even when the subject 3 is a plurality of people. For example, as in the first and second embodiments, the own vehicle 1 is provided with display devices 32 to 35 on the front, rear, left and right sides of the vehicle body, one pedestrian is on the front side of the own vehicle 1, and another is on the right side of the own vehicle 1. When there is one pedestrian, the determination unit 22 determines whether or not to display the display image for each pedestrian. When the determination unit 22 determines that the display is to be displayed for each pedestrian, the display image generation unit 23 generates a display image for each pedestrian. The control unit 26 causes a display device 32 provided on the front surface of the own vehicle 1 to display a display image generated for a pedestrian on the front side of the own vehicle 1, and a display provided on the right side surface of the own vehicle 1. The device 35 causes the pedestrian on the right side of the own vehicle 1 to display the generated display image.

Further, for example, when the own vehicle 1 is provided with display devices 32 to 35 on the front, rear, left and right of the vehicle body as in the first and second embodiments, and there are a plurality of pedestrians on the front side of the own vehicle 1, the determination unit 22 may be used. , Determine whether to display the display image for each pedestrian. The determination unit 2 determines that the displayed image is to be displayed, and determines one pedestrian who is likely to take the most dangerous action when trying to cross the pedestrian crossing 4 even though there is another vehicle 2 in the blind spot. The display image generation unit 23 determines that the determination unit 22 displays the display image, and generates a display image for the pedestrian who is likely to cause the most dangerous behavior. The control unit 26 displays a display image on a display device 32 provided on the front surface of the own vehicle 1.

For example, if it is determined that the displayed image is to be displayed and there are a plurality of pedestrians who are likely to take the most dangerous action when trying to cross the pedestrian crossing 4 even though there is another vehicle 2 in the blind spot, the determination is made. The unit 22 decides to display the displayed image to the pedestrian who is the earliest to be dangerous. In that case, the display image that matches the line of sight of the pedestrian, which is the earliest danger, is displayed on the display device, but since the display itself can be confirmed by other pedestrians, it also alerts other pedestrians. Can be done. Further, if the speaker 36 is used to notify the display together with the display, more attention can be given.

Further, when there are a plurality of pedestrians on the front side of the own vehicle 1, the determination unit 22 determines the priority for each pedestrian, for example, determining the pedestrian to display the display image by machine learning. The pedestrian with the highest priority may be determined as the pedestrian to be displayed. Further, when there are a plurality of pedestrians on the front side of the own vehicle 1, the display system 30 displays a display image according to the line of sight of the average position of the plurality of pedestrians, directly in front of, directly behind, and on the left side of the own vehicle 1. The display image viewed from the center, the center of the right side surface, the top, etc. may be displayed.

In the first and second embodiments, the display system 30 determines whether the target person 3 is aware of the display image displayed on the display device from the line-of-sight information of the target person 3, and the target person 3 displays the image. If the image is not noticed, the condition-imparting unit 29 may process at least one part of the displayed image. The control unit 26 may display the processed display image on the display device. By doing so, the probability that the target person 3 confirms the displayed image is high, and the visibility of the target person 3 can be improved.

In the first and second embodiments, the display system 30 has described the case of displaying the display image, but the display system 30 outputs a horn (not shown) together with the display, outputs light from the headlight 72, and sounds from the speaker 36. Or may be combined with one or more display devices. Specifically, for example, when the display system 30 displays the display image, if the target person 3 is not aware of the display image, the determination unit 22 determines whether or not to output the horn provided in the own vehicle. It is determined whether or not light is output from the headlight 72 provided in the own vehicle, whether or not sound is output from the speaker 36 provided in the own vehicle, and the like. The control unit 26 displays a display image on the display device, such as when the determination unit 22 determines to output the horn, determines to output light from the headlight 72, determines to output sound from the speaker 36, and the like. At the same time, the horn is output, the light is output from the headlight 72, the sound is output from the speaker 36, and the like. Further, the display system 30 continuously outputs the horn until the subject 3 notices the displayed image, outputs light from the headlight 72, outputs sound from the speaker 36, and the like, so that the subject 3 outputs the displayed image. The probability of confirmation is high, and the visibility of the subject 3 can be improved.

Further, for example, when the display system 30 displays a display image, if the target person 3 wants to be alerted to the display, the display system 30 outputs light from the headlight 72, which outputs a horn together with the display. , The sound may be output from the speaker 36, and the like. By doing so, the probability that the target person 3 confirms the displayed image is high, and the visibility of the target person 3 can be improved.

In the case of the headlight 72, the display system 30 links the lighting and extinguishing of the headlight 72 with the display, for example, by performing passing to make the target person 3 notice the display, to call attention, and the like. May be good. Further, for example, when the display of the displayed image is difficult to see because the light is output from the headlight 72 when viewed from the pedestrian 3, the display system 30 weakens or turns off the light of the headlight 72. You may.

Although the case where the own vehicle 1 is in the vicinity of the pedestrian crossing has been described in the first and second embodiments, the present disclosure can be applied to a crowded parking lot, parking on the shoulder, and the like.

In the first and second embodiments, the vehicle interior / exterior information acquisition unit 21 acquires vehicle interior / exterior information from the overall control ECU 11, but the vehicle interior / exterior information acquisition unit 21 may acquire vehicle interior / exterior information directly from each sensor. ..

In the first and second embodiments, the own vehicle image DB 24a, the blind spot image DB 24b, and the condition information DB 25 are provided by the display control device 20, but the own vehicle image DB 24a, the blind spot image DB 24b, and the condition information DB 25 are different from each other. It is stored in a server or the like, and the display control device 20 may be acquired from another server or the like by using the communication device 60 or the like.

In the first and second embodiments, the own vehicle 1 is a vehicle, but it may be an autonomous vehicle or a vehicle driven by a person. In the case of an autonomous driving vehicle, the driver may not always know the driving situation, so that it becomes possible to support the target person 3 so that he / she can act safely on behalf of the driver. Further, the own vehicle 1 is not limited to an automobile, but may be a forklift, a ship, an airplane, a bicycle, a motorcycle, or the like.

In the first and second embodiments, the target person 3 is a pedestrian, but a driver of a vehicle such as a bicycle, a motorcycle, an automobile, or a forklift may be used.

By the way, the display control device, the display system, the display control method, and the display control program shown in the above-described embodiment are merely examples, and can be appropriately configured in combination with other devices. It is not limited to the configuration of a single form.

1 own vehicle, 2 other vehicles, 3 target person, 4 pedestrian crossing,
5 Dotted arrow when the target person 3 sees the own vehicle 1,
6 Dotted arrow when looking at own vehicle 1 from the front,
7 Display image, 8 Image of own vehicle 1,
9 Blind spot image, which is an image of another vehicle 2.
10 Overall control ECU, 11 processor, 12 memory,
15 notification control ECU, 16 processor, 17 memory,
20 Display control device, 21 Blind spot image acquisition unit, 22 Judgment unit,
23 Display image generator, 24a Own vehicle image DB,
24b blind spot image DB, 25 condition information DB,
26 Control unit, 27 Vehicle interior / external information acquisition unit, 28 Blind spot image generation unit,
29 Conditioning unit, 30 Display system, 31 Notification device,
32 to 35 display devices, 36 speakers, 40 sensors,
41 Vehicle speed sensor, 42 Steering angle sensor, 43 Accelerator sensor,
44 brake sensor, 45 shift sensor,
46 turn signal sensor, 47 hazard sensor,
48 wiper sensor, 49 light sensor,
50 Door open / close sensor, 51 Driver camera,
52 Seating sensor, 53 Accelerometer, 54 Angular velocity sensor,
55 GPS device, 56 navigation system,
57 outside camera, 58 outside sensor, 59 illuminance sensor,
60 communication equipment, 61 antenna, 62 transmitter,
63 receiver, 65 communication line, 70 driving device,
71 headlight driver, 72 headlight,
73 engine, 74 transmission, 75 brake actuator,
76 Steering actuator,
77 Turn Signal, 78 HUD, 80 Notifier,
81 external light device, 82 external light driver,
83 external light set, 84-85 external light,
86 projector device, 87 projector driver,
88 projector, 89 body light device,
90 body light driver, 91 body light,
92 Sound Devices, 93 Sound Drivers,
94 sound equipment, 95 display device 32 frame,
100 In-vehicle system.

Claims

A blind spot image acquisition unit that acquires a blind spot image showing a blind spot caused by the own vehicle when viewed from a target person existing outside the own vehicle, and a blind spot image acquisition unit.
A display image generation unit that generates a display image in which an image of the own vehicle is superimposed on the blind spot image,
A display control device including a control unit for displaying the display image on a display device provided in the own vehicle.
A condition-imparting unit for processing the display image is provided.
The display control device according to claim 1, wherein the control unit displays the display image processed on the display device.
The display control device according to claim 2, wherein the condition-imparting unit processes an image of the own vehicle.
The condition-imparting unit performs processing for reducing the image of the own vehicle, processing for performing transmission processing on the image of the own vehicle, processing as seen from a viewpoint of shifting the image of the own vehicle, or processing for the own vehicle. The display control device according to claim 3, wherein at least one of the processing of forming only the contour as a solid line or a dotted line is performed.
The display control device according to any one of claims 2 to 4, wherein the condition-imparting unit processes the blind spot image.
The condition-imparting portion is processed to enlarge the size of at least one portion of the blind spot image, or to shift the position of at least one portion of the blind spot image so as to reduce the overlap with the image of the own vehicle. The display control device according to claim 5, wherein at least one of the above is processed.
The display control device according to any one of claims 1 to 6, wherein the control unit causes the display device to display a display that emphasizes the display image.
The display control device according to claim 7, wherein the display that emphasizes the display image means blinking the frame of the display device or blinking at least one part of the display image.
It is provided with another vehicle information acquisition unit that acquires other vehicle information indicating information on other vehicles existing in the blind spot.
The display control device according to any one of claims 2 to 8, wherein the condition-imparting unit changes processing according to the other vehicle information.
It is equipped with a line-of-sight information acquisition unit that acquires the line-of-sight information of the target person.
The display control device according to any one of claims 1 to 9, wherein the display image generation unit generates the display image according to the line of sight of the target person based on the line-of-sight information.
It is equipped with an eye height information acquisition unit that acquires eye height information of the target person.
The display control device according to claim 10, wherein the display image generation unit generates the display image of the own vehicle viewed from the height of the eyes of the target person based on the height information of the eyes.
A determination unit for determining whether or not to display the display image on the target person is provided.
The display control device according to any one of claims 1 to 11, wherein the control unit displays the display image on the display device when the determination unit determines that the display is to be displayed.
When the determination unit determines that the target person exists in a predetermined range, the determination unit determines that the display image is displayed on the target person, and the target person exists in the predetermined range from the position information. The display control device according to claim 12, wherein if it is determined that the display image is not displayed, the display image is not displayed to the target person.
The determination unit determines that the display image is displayed to the target person when the object exists in the blind spot, and the object does not exist in the blind spot and the object is stopped in the blind spot. Is the display control device according to any one of claims 12 to 13, which determines that the display image is not displayed on the target person.
The determination unit determines whether or not the subject is aware of the display image displayed on the display device.
When the determination unit determines that the subject is not aware of the display image, the condition-imparting unit processes at least one part of the display image.
The display control device according to any one of claims 12 to 14, wherein the control unit displays the display image processed on the display device.
The determination unit determines whether or not to output sound from the speaker provided in the own vehicle.
Any of claims 12 to 15, wherein when the determination unit determines that the sound is output from the speaker, the control unit displays the display image on the display device and outputs the sound from the speaker. The display control device according to item 1.
The display device installed in the own vehicle and
A blind spot image acquisition unit that acquires a blind spot image showing a blind spot caused by the own vehicle when viewed from a target person existing outside the own vehicle, and a blind spot image acquisition unit.
A display image generation unit that generates a display image in which an image of the own vehicle is superimposed on the blind spot image,
A control unit that displays the displayed image on the display device,
Display system with.
A step of acquiring a blind spot image showing a blind spot caused by the own vehicle when viewed from a target person existing outside the own vehicle, and a step of acquiring the blind spot image.
A step of generating a display image in which the image of the own vehicle is superimposed on the blind spot image,
A display control method including a step of displaying the display image on a display device provided in the own vehicle.
A process of acquiring a blind spot image showing a blind spot caused by the own vehicle when viewed from a target person existing outside the own vehicle, and a process of acquiring a blind spot image.
A process of generating a display image in which the image of the own vehicle is superimposed on the blind spot image,
A display control program that causes a display device provided in the own vehicle to execute a process of displaying the display image.