JP2022006291A - On-vehicle device - Google Patents

Abstract

To appropriately display a video.SOLUTION: An on-vehicle device includes: an output section 111 for outputting video data of an exterior video based on a captured video in the periphery of a vehicle 1 to a display device; a delay monitoring section 113 for monitoring delay of displaying the exterior video in the display device based on a response from the display device with respect to an output of video data of the exterior video; and a video data size conversion section 114 for allowing a data size of the video data of the exterior video to be output to the display device to be smaller than a data size when delay occurs in displaying the exterior video, when delay occurs in displaying the exterior video in the display device.SELECTED DRAWING: Figure 1

Description

The present invention relates to an in-vehicle device.

Patent Document 1 is a technique related to parking support.
In the subject column of the abstract of Patent Document 1, "when the occurrence of a situation that requires the driver's attention during transmission of the camera image is detected, the situation can be quickly notified to the driver. Realize a driving support device. "
In the column of the solution means of the abstract of Patent Document 1, "The driving support device (10) has a captured image analysis unit (12) for detecting the occurrence of a situation requiring attention to the driver, and a camera ( The transmission processing unit (13) is provided with a transmission processing unit (13) for transmitting the video data of the captured image output by the 102) to the smartphone (20). The transmission processing unit (13) corresponds to the content that the driver should pay attention to. The identifier is transmitted to the smartphone (20) immediately after the above situation is detected. "

Japanese Unexamined Patent Publication No. 2016-66213

However, when the processing load of the smartphone is high, there is a possibility that the display of the image captured by the camera on the smartphone may be delayed.

An object of the present invention is to provide an in-vehicle device that displays an image more appropriately.

One aspect of the present invention is based on an output unit that outputs video data of an outside vehicle image based on a captured image of the surroundings of a vehicle to a display device and a response from the display device to the output of the image data of the outside vehicle image. A delay monitoring unit that monitors the delay in displaying the outside image on the display device, and data of the image data of the outside image output to the display device when the display of the outside image is delayed on the display device. It is an in-vehicle device including a video data size conversion unit that makes the size smaller than the data size when the display of the video outside the vehicle is delayed.

According to the present invention, an image can be displayed more appropriately.

It is a figure which shows the structure of the vehicle which mounted the driving support device which concerns on embodiment of this invention. It is explanatory drawing of the detection method of the image display delay in a portable display device. It is a flowchart of a display control process. It is a schematic diagram of the image outside the vehicle. It is a flowchart of the display control processing which concerns on the modification of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a configuration of a vehicle 1 equipped with a driving support device 100 according to the present embodiment.
The vehicle 1 includes an in-vehicle camera 10, an in-vehicle display device 20, a communication unit 30, and a driving support device 100, which are connected so as to be capable of data communication through an in-vehicle network 5 such as a CAN (Controller Area Network) bus. ing.

The vehicle-mounted camera 10, the vehicle-mounted display device 20, and the driving support device 100 are devices that realize a so-called electronic mirror (also referred to as a digital mirror or a camera monitoring system).
The in-vehicle camera 10 is an in-vehicle camera that outputs a photographed image of the periphery of the vehicle 1 (that is, outside the vehicle) to the driving support device 100. The angle of view and the shooting direction of the vehicle-mounted camera 10 can be appropriately set based on the specifications of the electronic mirror.

The in-vehicle display device 20 is a display device fixedly installed in the vehicle 1. The in-vehicle display device 20 is mounted on a general vehicle by sequentially inputting data of an out-of-vehicle image (moving image) based on an image taken by the in-vehicle camera 10 from the driving support device 100 and displaying the out-of-vehicle image. It acts as a substitute for existing mirrors (room mirrors, side mirrors, etc.).
The vehicle-mounted display device 20 is installed in the vehicle interior of the vehicle 1 at a position corresponding to the mirror to be replaced.

The communication unit 30 is a device that is connected to the portable display device 50 so as to be able to communicate wirelessly or by wire, and performs data communication with the portable display device 50 by using an appropriate communication protocol capable of transmitting a moving image. .. The communication unit 30 includes a receiving device that receives data from the portable display device 50 and a transmitting device that transmits data to the portable display device 50.

The portable display device 50 is a portable device capable of displaying an image outside the vehicle in place of the in-vehicle display device 20.
More specifically, the portable display device 50 includes a communication unit 51 and a display unit 52.
The communication unit 51 is a device that is wirelessly or wiredly connected to the communication unit 30 included in the vehicle 1 and communicates data with the communication unit 30. The communication unit 51 includes a receiving device and a transmitting device for transmitting and receiving data, similarly to the communication unit 30 of the vehicle 1.
The display unit 52 displays the outside image received from the vehicle 1 through the communication unit 51. The display unit 52 is, for example, a flat panel display such as a liquid crystal display or an organic EL (Electro-Luminescence) display.

The portable display device 50 is used as an alternative display means in an emergency when an image outside the vehicle is not displayed on the vehicle-mounted display device 20 due to a failure, damage, or the like. As the portable display device 50, for example, a portable electronic device having a display function carried by an occupant, such as a smartphone, a tablet computer, a portable personal computer, or a mobile display, is assumed.

The driving support device 100 has a function of controlling the output of video data of the video outside the vehicle, and is an in-vehicle device that supports driving by displaying the video outside the vehicle on the vehicle-mounted display device 20 and the portable display device 50.
The operation support device 100 includes a processor such as a CPU (Central Processing Unit) and an MPU (Microtechnology Unit), a memory device (also referred to as a main storage device) such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and an HDD. Storage devices (also called auxiliary storage devices) such as (Hard Disk Drive) and SSD (Solid State Drive), interface circuits for connecting sensors and peripheral devices, and other in-vehicle devices via the in-vehicle network 5. It is equipped with a computer equipped with an in-vehicle network communication circuit that communicates with. An ECU (Electronic Control Unit) is used as such a computer.
In the driving support device 100, various functional configurations shown in FIG. 1 are realized by the processor executing a computer program stored in the memory device or the storage device.

That is, the driving support device 100 has, as a functional configuration, a captured image acquisition unit 110, an output unit 111, a connection monitoring unit 112, a delay monitoring unit 113, a video data size conversion unit 114, and a caution display processing unit 115. And have.

The photographed image acquisition unit 110 acquires the photographed image outside the vehicle photographed by the vehicle-mounted camera 10.
The output unit 111 switches and outputs the video data of the vehicle exterior video based on the captured video between the vehicle-mounted display device 20 and the portable display device 50.
More specifically, the output unit 111 outputs the video data of the vehicle exterior video to the vehicle-mounted display device 20 as the default output destination. In this case, the output unit 111 outputs the video data of the vehicle-external video to the vehicle-mounted display device 20 via the vehicle-mounted network 5.
Further, when the output unit 111 is instructed to switch to the portable display device 50 by, for example, an operation of a occupant for an HMI (Human Machine Interface), the output destination is switched to the portable display device 50 and the portable display device 50 is used. The video data of the video outside the vehicle is output to 50. In this case, the output unit 111 outputs the video data of the video outside the vehicle to the portable display device 50 via the communication unit 30.
When the vehicle-mounted display device 20 is not normally displayed on the vehicle-mounted display device 20 due to a failure or damage of the vehicle-mounted display device 20, the occupant repairs the electronic mirror by performing an instruction operation to switch the output destination to the portable display device 50. The portable display device 50 can display the image outside the vehicle as a substitute for the vehicle-mounted display device 20 until the vehicle 1 is moved to a facility (repair shop or dealer) that accepts the vehicle.

When the output destination of the video data of the video outside the vehicle is the portable display device 50, the connection monitoring unit 112 disconnects the communication connection between the driving support device 100 (more accurately, the communication unit 30) and the portable display device 50. Monitor if it is not done.
More specifically, the connection monitoring unit 112 transmits a confirmation signal for confirming the communication connection status to the portable display device 50 at regular intervals, and determines whether or not the portable display device 50 responds to the confirmation signal. Judgment of disconnection of communication connection based on.

When the output destination of the video data of the video outside the vehicle is the portable display device 50, the delay monitoring unit 113 monitors the delay of the video display outside the vehicle in the portable display device 50.
More specifically, the display performance of the portable display device 50, which is carried by an occupant, is generally lower than that of the in-vehicle display device 20 specialized for an electronic mirror. The performance related to display is, for example, the resolution, the number of display colors, the processing speed of a processor (for example, Graphics Processing Unit) that executes drawing processing, and the like.
When the video data of the vehicle exterior image output by the driving support device 100 to the in-vehicle display device 20 is output to the portable display device 50 as it is, the performance difference related to the above-mentioned display and the driving support device 100 (more accurately, the communication unit 30) ) And the portable display device 50, a display delay may occur in the portable display device 50 due to a processing load generated in the portable display device 50 and the like.

Therefore, the delay monitoring unit 113 monitors such a delay.
Specifically, as shown in FIG. 2, the delay monitoring unit 113 requests the portable display device 50 to transmit a reception response for each transmission of the video frame constituting the video data of the video outside the vehicle. The reception response is a response signal indicating that the display based on the video frame is completed in the portable display device 50.
The delay monitoring unit 113 detects the delay time ΔTa by the time difference between the timing Ta1 for transmitting the transmission request of the reception response to the portable display device 50 and the timing Ta2 for receiving the reception response for the transmission request from the portable display device 50. do. Then, when the delay time ΔTa exceeds the specified value, the delay monitoring unit 113 determines that the display of the image outside the vehicle is delayed in the portable display device 50.
The delay in displaying the image outside the vehicle is called "image display delay".
Further, the delay monitoring unit 113 may request the transmission of the reception response not for each video frame but for each predetermined number of video frames.

The video data size conversion unit 114 executes the data size conversion process when the video display delay occurs in the portable display device 50.
In the data size conversion process, the data size of the video data of the outside vehicle is changed from the data size when the video display delay occurs to the data size at which the video display delay is eliminated, that is, to the data size in which the delay time ΔTa is equal to or less than the specified value. It is a process to make it smaller.
Specifically, the video data size conversion unit 114 gradually reduces the amount of data in each video frame until the delay time ΔTa is equal to or less than the specified value without changing the frame rate of each video frame constituting the video outside the vehicle. By doing so, the data size of the video data of the video outside the vehicle is reduced.
As a method for reducing the amount of data in the video frame, a known or well-known appropriate method is used. For example, the amount of data in a video frame can be reduced by lowering the resolution of the video frame or reducing the number of display colors by displaying in gray scale or the like.
In such data size conversion processing, since the frame rate of the outside vehicle image is maintained unchanged, that is, the image frames are not thinned out, the same time resolution as the outside vehicle image display when the in-vehicle display device 20 is used is used. The image outside the car is displayed with. As a result, even when the image outside the vehicle is displayed on the portable display device 50, the real-time property of the display outside the vehicle is not impaired.

The caution display processing unit 115 determines that the attention object 120 (FIG. 4) is displayed in the outside image while the data size of the image data (more accurately, the image frame) of the outside image is reduced. A process for calling attention to the caution object 120 is executed.

The object of caution 120 is an object that the driver should pay attention to.
Specifically, the object of caution 120 is, for example, an appropriate object (including another vehicle or a pedestrian) approaching the vehicle 1 or an obstacle. An obstacle is an object that hinders the running of the vehicle 1. Obstacles are, for example, walls, stanchions, structures such as road signs, other vehicles running or stopped, or pedestrians.
Further, in the present embodiment, the "process for calling attention" is a highlighting process for highlighting and conspicuous the attention object 120 in the image outside the vehicle.
The specific mode of highlighting is appropriate as long as it can draw the driver's attention. For example, there may be a highlighting mode in which a frame surrounding the attention object 120 is superimposed and displayed, or the display color in the range in which the attention object 120 exists is changed.

The attention display processing unit 115 detects the attention object 120 whether it is reflected in the image outside the vehicle by using a known or well-known technique such as image recognition processing. Then, when the attention object 120 is reflected in the image outside the vehicle, the attention display processing unit 115 performs image processing on the image frame of the image outside the vehicle so that the object 120 to be watched is highlighted.
In this way, even if the amount of information displayed on the outside vehicle image is reduced due to the reduction in the resolution and the number of display colors, the attention object 120 is highlighted so that the occupant (especially the driver) pays attention. You can accurately grasp the object to be.

Next, the operation of this embodiment will be described.
When the vehicle-mounted image is not normally displayed on the vehicle-mounted display device 20, the occupant performs an instruction operation for switching the output destination of the video data of the vehicle-mounted image to the portable display device 50. When the instruction operation is input to the driving support device 100, the output unit 111 transmits the video data of the vehicle outside video to the portable display device 50 through the communication unit 30. When the portable display device 50 receives the video data of the video outside the vehicle, the portable display device 50 displays the video outside the vehicle on the display unit 52 based on the video data.
As a result, the image outside the vehicle is displayed on the portable display device 50 instead of the vehicle-mounted display device 20, and the occupant can confirm the situation outside the vehicle by such a display.

Further, the driving support device 100 displays an appropriate out-of-vehicle image on the portable display device 50 while the video data of the out-of-vehicle image is output to the portable display device 50 (while the portable display device 50 is the output destination). The following display control process is executed to realize this.

FIG. 3 is a flowchart of the display control process.
In the operation support device 100, the connection monitoring unit 112 determines whether or not the communication connection with the portable display device 50 is disconnected (step Sa1).
For example, when the battery level of the portable display device 50 is exhausted, the communication connection is disconnected. When the communication connection is disconnected in this way (step Sa1: Yes), the output unit 111 switches the output destination of the video data of the external video to the vehicle-mounted display device 20 (step Sa2), and ends this process.
In this case, when the vehicle-mounted display device 20 is capable of displaying, the vehicle-mounted display device 20 displays the image outside the vehicle, although there is a possibility that the display is not normal. Therefore, for example, when the visibility of the display of the image outside the vehicle is hindered by the crack of the cover glass of the in-vehicle display device 20, and the alternative portable display device 50 is not connected to the communication. Will display the image outside the vehicle on the vehicle-mounted display device 20 even if there is some difficulty in visibility.

In the determination of step Sa1, when the communication connection of the portable display device 50 is not disconnected (step Sa1: Yes), the delay monitoring unit 113 determines whether or not the image display delay has occurred in the portable display device 50. (Step: Sa3).
Then, when the video display delay occurs (step Sa3: Yes), the video data size conversion unit 114 executes the video data size conversion process (step Sa4). As a result, the data size of the video data of the video outside the vehicle is reduced, the video display delay is eliminated, and the real-time property of the video display outside the vehicle is maintained.

In this way, while the data size of the video data of the video outside the vehicle is reduced by the video data size conversion unit 114, the caution display processing unit 115 determines whether or not the object 120 to be watched is reflected in the video outside the vehicle. When the determination is made (step Sa5) and the attention object 120 is displayed (step Sa5), the process of highlighting the attention object 120 is executed (step Sa6).

FIG. 4 is a diagram schematically showing an image outside the vehicle.
When the data size of the video data of the video outside the vehicle is reduced due to the video display delay in the portable display device 50, as shown in FIG. 4, the video outside the vehicle displayed on the portable display device 50 is displayed on the vehicle. The object of caution 120 is less visible than what is displayed on the device 20. Even in such a case, the driver can see the presence of the attention object 120 by highlighting the attention object 120 by the frame 130 surrounding the attention object 120 and the superimposed display of the icon 131 calling attention. And you can accurately grasp the position.
At the time of highlighting, the caution display processing unit 115 displays characters 132 indicating matters to be noted about the caution target object 120, so that the driver can display the caution target object 120 with respect to the caution target object 120. You can understand what kind of attention you should pay.
Further, the distance from the vehicle 1 to the object of caution 120 may be superimposed and displayed.

According to this embodiment, the following effects are obtained.

The driving support device 100 of the present embodiment includes a delay monitoring unit 113 that monitors the video display delay of the vehicle exterior image in the portable display device 50 when the video data of the vehicle exterior video is output to the portable display device 50. When the image display delay occurs in the portable display device 50, the data size of the video data of the outside image output to the portable display device 50 is made smaller than the data size when the image display delay of the outside image occurs. It includes a video data size conversion unit 114.
According to this configuration, when a portable electronic device such as a smartphone carried by an occupant is used as a portable display device 50 for displaying an image outside the vehicle, when the image display delay occurs in the portable display device 50, the image display is delayed. The data size of the video data of the outside vehicle image transmitted to the portable display device 50 is made smaller than the data size when the video display delay occurs.
As a result, the load of the display processing in the portable display device 50 is reduced, so that the image display delay can be suppressed and the real-time display can be maintained, and the image outside the vehicle can be displayed more appropriately.

In the driving support device 100 of the present embodiment, the video data size conversion unit 114 can easily reduce the amount of video data by lowering the resolution of the video data of the vehicle outside the vehicle to reduce the data size of the video data.

In the driving support device 100 of the present embodiment, the video data size conversion unit 114 can easily reduce the amount of video data by reducing the display color of the video data of the video outside the vehicle to reduce the data size of the video data. ..

In the driving support device 100 of the present embodiment, the video data size conversion unit 114 reduces the data amount of the video frame of the video data without changing the frame rate of the video data of the video outside the vehicle to reduce the data size of the video data. Make it smaller.
As a result, even if the data size of the video data is reduced, the video frames are not thinned out, so that the vehicle exterior video can be displayed with the same time resolution as the vehicle exterior video display using the vehicle-mounted display device 20. As a result, even when the image outside the vehicle is displayed on the portable display device 50, the real-time property of the display outside the vehicle is not impaired.

When the data size of the video data of the vehicle exterior video is reduced by the video data size conversion unit 114, the driving support device 100 of the present embodiment determines whether or not the caution object 120 is reflected in the vehicle exterior video, and cautions the driver. It is provided with a caution display processing unit 115 that executes a process for calling attention to the object 120 when the object 120 is displayed.
As a result, the data size of the video data is reduced, so that even if the amount of information displayed in the video outside the vehicle is reduced, the occupant, especially the driver, can accurately grasp the object to be paid attention to.
The caution display processing unit 115 does not execute the processing for highlighting the image outside the vehicle displayed on the vehicle-mounted display device 20. As a result, it is possible to prevent the detailed information in the image outside the vehicle (for example, the license plate of an approaching vehicle, the face of a pedestrian, etc.) from being impaired by the highlighting.
However, as described in Modification 2 described later, when the data size is reduced with respect to the data size of the vehicle-mounted image displayed on the vehicle-mounted display device 20, the caution display processing unit 115 highlights the data. I do.

The above-described embodiment is merely an example of one aspect of the present invention, and can be arbitrarily modified and applied without departing from the spirit of the present invention.

(Modification 1)
The driving support device 100 may include a resolution determination unit that determines whether or not the resolution that the portable display device 50 can display is lower than the resolution of the vehicle-mounted display device 20.
In this case, in the display control process, as shown in FIG. 5, the resolution of the portable display device 50 is the resolution of the in-vehicle display device 20 prior to the image display delay detection process (step Sa3) in the portable display device 50. The resolution determination unit determines whether or not the resolution is lower than (step Sb1). When it is determined that the resolution of the portable display device 50 is lower than the resolution of the in-vehicle display device 20 (step Sb1: Yes), the video data size conversion unit 114 does not monitor the video display delay in the portable display device 50. Decreases the data size of the video data of the video outside the vehicle according to the resolution of the portable display device 50 (step Sa4).
According to this process, since the video data of the video outside the vehicle is converted into a data size suitable for the resolution of the portable display device 50, it is possible to prevent a video display delay in the portable display device 50.

The method by which the resolution determination unit specifies the resolution of the portable display device 50 is arbitrary.
For example, the resolution determination unit may acquire information on the resolution from the portable display device 50, or may display information (for example, product name, model number, etc.) that can specify the product specifications of the portable display device 50 in the portable display device 50. The resolution of the portable display device 50 may be specified by searching the product specification database acquired from the device 50 and holding it in advance based on the information, or by making an inquiry to an external computer.

(Modification 2)
In the driving support device 100, the delay monitoring unit 113 monitors the video display delay in the vehicle-mounted display device 20 even when the video data of the vehicle-external video is output to the vehicle-mounted display device 20, and displays the video on the vehicle-mounted display device 20. When there is a delay, the video data size conversion unit 114 may reduce the data size of the video data of the vehicle-mounted video output to the vehicle-mounted display device 20.
As a result, even if a video display delay occurs in the vehicle-mounted display device 20 due to some trouble, the video display delay can be suppressed and real-time display can be maintained. As an example of the cause of such trouble, there is an example in which a process having a very high load is executed in the in-vehicle display device 20 due to a software bug or a malicious third party (so-called hacking).

(Modification 3)
The driving support device 100 may include an in-vehicle camera 10 and a communication unit 30 in the device.

(Other variants)
In the above-described embodiment, the functional block shown in FIG. 1 is a schematic diagram showing the components of the vehicle 1 and the driving support device 100 classified according to the main processing contents in order to facilitate the understanding of the present invention. Yes, these components can be further classified into more components according to the processing content. It can also be categorized so that one component performs more processing.
Further, the processing of each component of the driving support device 100 may be executed by one hardware or may be executed by a plurality of hardware. Further, the processing of each component may be realized by one program or may be realized by a plurality of programs.

1 Vehicle 10 In-vehicle camera 20 In-vehicle display device 30, 51 Communication unit 50 Portable display device 100 Driving support device (in-vehicle device)
110 Shooting video acquisition unit 111 Output unit 112 Connection monitoring unit 113 Delay monitoring unit 114 Video data size conversion unit 115 Caution display processing unit 120 Caution object ΔTa delay time

Claims (6)

An output unit that outputs video data of the outside video based on the shot video of the surroundings of the vehicle to the display device,
A delay monitoring unit that monitors the delay in displaying the outside image on the display device based on the response from the display device to the output of the image data of the outside image.
When the display of the outside image is delayed in the display device, the data size of the image data of the outside image output to the display device is larger than the data size when the display of the outside image is delayed. A video data size converter to make it smaller,
An in-vehicle device characterized by being equipped with. The vehicle-mounted device according to claim 1, wherein the video data size conversion unit lowers the resolution of the video data of the video outside the vehicle. The vehicle-mounted device according to claim 1 or 2, wherein the video data size conversion unit reduces the display color of video data of the video outside the vehicle. The video data size conversion unit is characterized in that the data amount of the video frame of the video data is reduced and the data size of the video data is reduced without changing the frame rate of the video data of the video outside the vehicle. The vehicle-mounted device according to any one of claims 1 to 3. When the data size of the video data of the outside vehicle image is reduced by the video data size conversion unit, it is determined whether or not the attention object is reflected in the outside vehicle image, and when the caution object is displayed. , A caution display processing unit that executes processing to call attention to the caution object,
The vehicle-mounted device according to any one of claims 1 to 4, wherein the in-vehicle device is provided. The display device is
An in-vehicle display device provided on the vehicle and a portable portable display device.
The output unit is
The vehicle-mounted device according to any one of claims 1 to 5, wherein the output destination of the video data of the outside vehicle image is switched between the vehicle-mounted display device and the portable display device.

Images