JP2016066213A - Operation support device, operation presentation terminal, operation support method, operation support system, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve an operation support device capable of quickly informing an operator of a situation when detecting occurrence of the situation in which the operator is urged to pay attention during transmission of a camera image.SOLUTION: An operation support device (10) includes an imaged video analyzer (12) for detecting generation of a situation in which an operator is urged to pay attention and a transmission processor (13) for transmitting video data of an imaged video output by a camera (102) to a smartphone (20). A transmission processor (13) transmits an identifier to the smartphone (20) according to what the operator is urged to pay attention to immediately after the situation is detected.SELECTED DRAWING: Figure 1

Description

  The present invention should mainly be noted by a driver when a driver assistance device installed in a vehicle and the occurrence of a situation that requires the driver to be alerted are detected by such a driver assistance device. The present invention relates to an information presentation terminal for presenting contents.

  In recent years, automobiles equipped with driving assistance devices (for example, a car navigation device or a collision avoidance device disclosed in Patent Document 1) are rapidly spreading.

  Some driving assistance devices detect the occurrence of a situation that requires the driver to be alerted by analyzing captured images generated by the built-in camera photographing the surroundings. Some of such driving support devices have a function of cooperating with a smartphone.

JP 2014-13148 A (published January 23, 2014)

  As an example of the driving support apparatus as described above, for example, to realize a driving support apparatus that transmits an image generated by a built-in camera to a smartphone for the purpose of functioning the smartphone as a drive recorder.

  When such a driving support device detects the occurrence of a situation that requires the driver to be alerted during the transmission of the camera video to the smartphone, the driver is alerted as an example of how to inform the driver of the situation A method may be considered in which a message is embedded in a video frame of a camera video to be transmitted and the video frame is transmitted to a smartphone.

However, when such a method is used, there is a problem that a time lag from when the driving support device detects the above situation until the driver grasps the situation becomes large for the following reason.
・ It takes time to embed the above message in the video frame.
Since the data size of the video frame is generally larger than the size of the text data, it takes a long time for the smartphone to receive the video frame after the driving support device transmits the video frame including the message.
・ It takes time to decode video frames on smartphones.

  And the magnitude | size of such a time lag can also become a hindrance to safe driving.

  The present invention has been made in view of the above problems, and its main purpose is to drive the situation when the occurrence of a situation requiring the driver to be alerted during the transmission of the camera image is detected. The object is to realize a driving support device that can be quickly transmitted to a person.

  In order to solve the above-described problem, a driving support device according to an aspect of the present invention includes a camera, a detection unit that detects occurrence of a situation that requires a driver to be alerted, and the situation by the detection unit. The identifier transmission that transmits the identifier according to the content to be noted by the driver to the information presentation terminal having the function of presenting the content to the driver immediately after receiving the identifier And video data transmission means for transmitting video data indicating the captured video output from the camera to the information presentation terminal.

  The driving assistance apparatus according to one aspect of the present invention can quickly notify the driver of the situation when the occurrence of the situation that requires the driver to be alerted during the transmission of the camera video is detected.

It is a block diagram which shows the structure of the driving assistance apparatus and smart phone which are included in the driving assistance system which concerns on Embodiment 1 of this invention. It is the figure which showed roughly the external appearance of the driving assistance apparatus and smart phone which concern on Embodiment 1 and 2 of this invention. It is a figure regarding Embodiments 1-3 of this invention, and is a figure for demonstrating the identifier transmitted when the condition which needs to call a driver | operator's attention generate | occur | produces. It is a figure regarding Embodiments 1-3, and is a figure for demonstrating the content of the operation | movement which a notification device should perform according to the condition classification value and attention level value which comprise the said identifier. It is a flowchart which shows the operation | movement which the driving assistance apparatus which concerns on Embodiments 1-2 (and the smart phone which concerns on Embodiment 3) repeats during the action | operation of a built-in camera. It is two flowcharts which show two operation | movement performed in parallel by the smart phone which concerns on Embodiments 1-2 (and the display audio which concerns on Embodiment 3). It is a figure for demonstrating the advantage of the driving assistance device which concerns on Embodiment 1-2. It is another figure for demonstrating the advantage of the driving assistance apparatus which concerns on Embodiment 1-2. It is a block diagram which shows the structure of the driving assistance apparatus and smart phone which are included in the driving assistance system which concerns on Embodiment 2. FIG. It is a block diagram which shows the structure of the smart phone and display audio which are included in the driving assistance system which concerns on Embodiment 3. FIG.

Embodiment 1
A driving support system according to an embodiment of the present invention will be described with reference to FIGS.

(Overview of driving support system)
An outline of the driving support system will be described with reference to FIGS. 1 and 2.

  FIG. 1 is a block diagram illustrating configurations of a driving support device 10 and a smartphone 20 included in the driving support system. FIG. 2 is a block diagram schematically showing the external appearance of the driving support device 10 and the smartphone 20.

  As shown in FIGS. 1 and 2, the driving support system according to the present embodiment includes a driving support device 10 and a smartphone 20.

  The driving assistance device 10 is a dedicated driving assistance device attached to the automobile, and can obtain information (vehicle speed information and the like) related to the running of the automobile.

  The smartphone 20 is an information presentation terminal registered in the driving support device 10. The smartphone 20 is possessed by the driver and is temporarily installed in the automobile only while the driver is in the automobile. In addition, in this embodiment, although the motor vehicle is employ | adopted as an example of the vehicle in which a driving assistance apparatus is installed, the driving assistance apparatus which concerns on this invention is not restricted to the driving assistance apparatus for motor vehicles. That is, the range of the driving support device according to the present invention includes driving support devices for motorcycles, bicycles, ships, and other driving support devices for vehicles.

  When the smartphone 20 is installed near the driving support device 10, the driving support device 10 and the smartphone 20 can perform wireless communication (in this embodiment, WiFi (registered trademark) communication) as illustrated in FIG. 2. Are connected to each other.

  While the engine of the automobile is operating, the driving support device 10 captures an image of a built-in camera that is transmitted to a connection information presentation terminal in order to detect the occurrence of a situation that requires the driver to be alerted. The video is analyzed.

  In the present embodiment, the driving support device 10 transmits a 32-bit identifier corresponding to the content that the driver should pay attention to to the connection destination information presentation terminal immediately after detecting the occurrence of such a situation. This feature is described in detail later. In this embodiment, the identifier is composed of 32 bits, but the number of bits of the identifier is not limited to 32 bits. In other words, the identifier only needs to be composed of a somewhat small number of bits.

  Hereinafter, the configuration of the driving support device 10 and the smartphone 20 included in the driving system will be described with reference to FIG.

(Configuration of the driving support device 10)
As illustrated in FIG. 1, the driving support device 10 includes a control unit 101, a camera 102, a microphone 103, a RAM 104, and a communication unit 105.

  The control unit 101 is a CPU that comprehensively controls the driving support apparatus 10 as a whole. The control unit 101 functions as a captured moving image data generation unit 11, a captured video analysis unit 12, and a transmission processing unit 13.

  The captured moving image data generation unit 11 generates a video frame based on a captured video signal input from the camera 102 and generates corresponding audio data based on a recording signal input from the microphone 103. The captured moving image data generation unit 11 generates captured moving image data for streaming transmission by repeating the generation of the video frame and the audio data.

  The captured video analysis unit 12 immediately analyzes the video frame every time a captured video data generation unit 11 generates a video frame, and thus, for various situations that require the driver to be alerted. Specify the attention level for.

  When the captured video analysis unit 12 (detection means) detects that a corresponding situation has occurred in the above-described various situations by analyzing the video frame (the attention level for the situation is no longer “safe”), it is relevant. When the severity of the situation has changed, or when it is detected that the situation has been removed (attention level for the situation has become “safe”), the specified identifier (described in detail later) ) Is supplied to the transmission processing unit 13.

  In addition, when the captured video analysis unit 12 determines that there is a situation that requires the driver to be alerted, a message corresponding to the content that the driver should pay attention to is included in the video frame that is the subject of analysis. Update the captured video data to be embedded.

  As soon as the identifier is supplied, the transmission processing unit 13 (identifier transmission means) executes a packet generation process and a transmission process for the identifier.

  In addition, the transmission processing unit 13 (video data transmission means) sequentially transmits captured moving image data packets.

  The camera 102 is a moving image-capable camera that captures the vicinity of the automobile on which the driver is riding (in this embodiment, the front of the automobile). Note that the camera 102 may be configured to take an image of the side or rear of the automobile, not the front of the automobile.

  The microphone 103 captures sounds around the automobile. The microphone 103 is preferably a microphone with low directivity.

  The RAM 104 is a recording medium on which captured moving image data that is being streamed is recorded.

  The communication unit 105 is a WiFi communication module used for communication with the smartphone 20.

(Configuration of smartphone 20)
As illustrated in FIG. 1, the smartphone 20 includes a control unit 201, a communication unit 202, a buffer 203, a display 204, a flash memory 205, a speaker 206, and an LED unit 207.

  The control unit 201 is a CPU that comprehensively controls the entire smartphone 20. The control unit 201 executes the installed application provided by the manufacturer of the driving support device 10, so that the reception processing unit 21, the video processing unit 22, the audio processing unit 23, the superimposed message determination unit 24, and the LED emission It functions as the control unit 25. In the present embodiment, the application includes both a real-time playback mode and a recording / playback mode, but the present invention is not limited to such a configuration. For example, the application may have only a real-time playback mode.

  When the reception processing unit 21 (video data receiving unit) receives the packet of the captured moving image data, it temporarily records the received packet in the buffer 203. The reception processing unit 21 reads out the packet recorded in the buffer 203, forms a video frame and audio data (audio data to be reproduced in synchronization with the video frame), and supplies the video frame to the video processing unit 22. The audio data is supplied to the audio processing unit 23.

  Also, the reception processing unit 21 (identifier receiving means) supplies the identifier to the voice processing unit 23, the superimposed message determination unit 24, and the LED light emission control unit 25 as soon as the identifier packet is received.

  Further, the reception processing unit 21 records the captured moving image data in the flash memory 205.

  In the real-time playback mode, the video processing unit 22 (display processing unit) displays the video frame on the display 204 every time a video frame is supplied from the reception processing unit 21. Further, in the recording / playback mode, the video processing unit 22 sequentially displays video frames included in the captured moving image data on the display 204 when the captured moving image data is read from the flash memory 205 by a data reading processing unit (not shown). To do.

  In particular, when a message is supplied from the superimposition message determination unit 24, the video processing unit 22 (presentation means) superimposes the message on each video frame to be displayed thereafter. Note that the video processing unit 22 continues the superimposed display of the message until receiving an instruction from the superimposed message determination unit 24 to stop the superimposed display of the message.

  In the real-time playback mode, the audio processing unit 23 outputs the audio represented by the audio data from the speaker 206 every time audio data is supplied from the reception processing unit 21. In the recording / playback mode, the audio processing unit 23 reads the audio represented by the audio data included in the captured moving image data when the captured moving image data is read from the flash memory 205 by a data read processing unit (not shown). The video is output in sync with the video displayed on the screen.

  The voice processing unit 23 (presentation means) is supplied from the reception processing unit 21 with an identifier (specifically, an identifier other than an identifier indicating that the driver has deviated from a specific situation that needs to call attention to the driver). As soon as the message corresponding to the value of the identifier is repeatedly output by voice, the process starts. The voice processing unit 23 continues this process until a new identifier is supplied from the reception processing unit 21.

  As soon as the identifier (specifically, an identifier other than an identifier indicating that the driver has escaped from a specific situation that needs to be alerted) is supplied from the reception processing unit 21, the superimposed message determination unit 24 receives the identifier. The message corresponding to the identifier is supplied to the video processing unit 22.

  As soon as the identifier indicating that the driver is out of the situation that needs to be alerted to the driver is supplied from the reception processing unit 21, the superimposition message determination unit 24 provides an instruction to stop the superimposition display. Supplied to the unit 22.

  As soon as the identifier is supplied from the reception processing unit 21, the LED light emission control unit 25 (presenting means) controls the light emission of the LED unit 207 based on the supplied identifier.

  The communication unit 202 is a WiFi communication module used for communication with the driving support device 10.

  The buffer 203 is a buffer for temporarily recording packets of captured moving image data.

  The display 204 (display screen) is a video display device that displays a captured video output from the camera 102 and a message indicating contents to be noted by the driver.

  The flash memory 205 is a recording medium in which the above-described application is installed.

  The speaker 206 is a voice output device that outputs a voice captured by the microphone 103 and a voice message indicating contents to be noted by the driver.

  The LED unit 207 is an LED device composed of a plurality of LED elements having different emission colors, and emits light in such a manner that the driver can recognize contents to be noted by the driver.

(About identifier details)
Next, the details of the identifier transmitted when a specific situation requiring attention to the driver occurs will be described below with reference to FIG. FIG. 3 is a diagram for explaining the identifier.

  As described above, the identifier in this embodiment is expressed by 32 bits.

  The first 8 bits (first octet) of the identifier is a reserved value. As the reserved value, an arbitrary value from “0x00” to “0xff” can be set.

  The next 8 bits (second octet) of the identifier is a situation type value indicating the type of situation that requires the driver to be alerted. For example, as can be seen from FIG. 3A, when the inter-vehicle distance between the driver's car and the car traveling ahead is reduced, "0x01" is set as the status type value. . In addition, when the distance between the driver's car and a pedestrian in front of the road is shortened, this situation type value is set to “0x02”, and the car "0x04" is set to this situation type value when the vehicle approaches or runs off a line or lane boundary line.

  The next 8 bits (third octet) of the identifier is an attention level value indicating the level of attention that the driver should pay. When the driver needs to be alerted, the attention level value is set to “0x01”.

  As can be seen from FIG. 3B, when the severity of the above situation is low, “0x02” is set to this attention level value, and when the severity of the above situation is high, this "0x04" is set for the attention level value. For example, when the driver's car approaches the lane line, this caution level value is set to "0x02", and when the driver's car runs over the road, this situation type value is set to "0x04" Is done.

  The last 8 bits (fourth octet) of the identifier is a notification device type value indicating the type of a device (notification device) that should notify the driver of a situation that requires the driver to be alerted. In the driving support apparatus 10, when user settings are made such that the video display device, the audio output device, and the LED device are notification devices, as shown in FIG. "0x07" is set as the device type value.

(Operation of driving support system)
Next, an outline of the operation of the driving support system will be described with reference to FIGS. FIG. 4 is a diagram for explaining the contents of the operation to be performed by the notification device according to the situation type value and the attention level value. FIG. 5 is a flowchart illustrating an operation that the driving support apparatus 10 repeatedly performs (that is, every time a video frame is generated) while the camera 102 is operating. FIG. 6 is two flowcharts showing two operations performed in parallel by the smartphone 20.

  First, the operation that the driving support device 10 repeatedly performs will be described.

(Step S1)
The photographed video analysis unit 12 analyzes the generated video frame, so that various situations that require the driver to be alerted (for example, approaching a car that is traveling ahead) are appropriate for the situation. Determine the level. For this determination process, an algorithm employed in a known driving support system may be employed.

  When the captured video analysis unit 12 determines that the attention level for the corresponding situation is not “safe”, the captured video analysis unit 12 embeds an object corresponding to the type of the situation in the video frame to be analyzed. For example, if the captured video analysis unit 12 determines that “the distance between the driver's vehicle and the vehicle traveling ahead has decreased,” the object indicating the image of the vehicle in the analysis target video frame is Embed in the video frame.

(Step S2)
Subsequently, the captured video analysis unit 12 determines whether or not the current analysis result (the analysis result in the latest step S1) is different from the previous analysis result. If it is determined that the current analysis result is different from the previous analysis result, the driving support device 10 proceeds to step S3, and if it is determined that the current analysis result is the same as the previous analysis result, the driving support device 10 proceeds to step S4. .

(Step S3)
The captured video analysis unit 12 generates an identifier packet corresponding to the analysis result of this time.

  For example, the previous analysis determines that the driver's car is approaching the lane line, and the current analysis determines that the car is not approaching the section line. When it is determined that the distance between the vehicle and the traveling vehicle has decreased, the captured video analysis unit 12 generates, for example, an identifier “0x00010207” and an identifier “0x00040107”.

  Then, the transmission processing unit 13 transmits the packet generated by the captured video analysis unit 12 to the smartphone 20 via the communication unit 105.

  The driving assistance apparatus 10 proceeds to step S4 after step S3.

(Step S4)
The transmission processing unit 13 generates a packet group of the video frame to be analyzed and a packet group of the corresponding recorded audio, and transmits these packet groups to the smartphone 20 via the communication unit 105.

  Next, two operations performed in parallel by the smartphone 20 executing the above-described application will be described with reference to FIGS. 4 and 6.

  First, the operation of the first eye (the operation of outputting captured video and recorded audio) will be described with reference to FIG. FIG. 6A is a flowchart showing the operation.

(Step S11)
The reception processing unit 21 waits for the packet group transmitted in step S4 to be transmitted. That is, the reception processing unit 21 repeats step S11 until reception of all packets transmitted in step S4 and recording in the buffer 203 are completed, and reception of all packets transmitted in step S4 and reception to the buffer 203. When the recording is completed, the process proceeds to step S12.

(Step S12)
The reception processing unit 21 reads out the packet group recorded in the buffer 203 in the immediately preceding step S11 at a timing in accordance with a FIFO (First In First Out) rule. The reception processing unit 21 configures a video frame of a captured video and audio data representing recorded audio from the read packet group, supplies the video frame to the video processing unit 22, and sends the audio data to the audio processing unit 23. Supply.

  The video processing unit 22 immediately displays the video frame supplied from the reception processing unit 21 (or the video frame obtained by superimposing the message supplied from the superimposition message determination unit 24 on the video frame) on the display 204. To do. At the same time, the audio processing unit 23 outputs the recorded audio represented by the audio data supplied from the reception processing unit 21 from the speaker 206.

  Next, the second operation (operation related to the identifier) will be described with reference to FIG. FIG. 6B is a flowchart showing the operation.

(Step S21)
The reception processing unit 21 waits for the packet (identifier) transmitted in step S3 to be transmitted. That is, the reception processing unit 21 repeats step S21 until receiving the identifier transmitted in step S3, and proceeds to step S22 when receiving the identifier transmitted in step S3.

(Step S22)
As soon as the reception processing unit 21 receives the identifier, the reception processing unit 21 supplies the received identifier to the voice processing unit 23, the superimposed message determination unit 24, and the LED light emission control unit 25.

(Step S23)
As soon as each of the voice processing unit 23, the superimposed message determination unit 24, and the LED light emission control unit 25 receives the identifier, it determines whether or not there is a need for processing according to the identifier.

  Specifically, when the lower 2nd bit of the notification device type value is “0”, the voice processing unit 23 determines that there is no need for processing according to the identifier, and the lower 2 of the notification device type value. When the bit is “1”, it is determined that there is a need for processing according to the identifier.

  Similarly, when the first bit of the notification device type value is “0”, the superimposition message determination unit 24 determines that there is no need for processing according to the identifier, and the lower one bit of the notification device type value. If the eye is “1”, it is determined that there is a need for processing according to the identifier. In addition, when the lower third bit of the notification device type value is “0”, the LED light emission control unit 25 determines that there is no need for processing according to the identifier, and the lower third bit of the notification device type value. Is “1”, it is determined that there is a need for processing according to the identifier.

(Step S24)
Each unit that has determined that there is a need for the process according to the identifier specifies the content of the process to be executed based on the situation type value and the attention level value.

  For example, as shown in FIGS. 4A and 4B, the audio processing unit 23 that has received the identifier “0x00010207” should repeatedly reproduce the audio “Please be careful ahead” at a standard volume. And the sound of the standard volume “Please pay attention to the front” is repeatedly output to the speaker 206. In addition, for example, the audio processing unit 23 that has received the identifier “0x00040107” specifies that the process of repeatedly reproducing the audio “please check the running position” should be terminated, and The output to the speaker 206 is terminated.

  In addition, for example, the superimposition message determination unit 24 that has received the identifier “0x00010207” specifies that a message with a standard character size “please be careful ahead” should be supplied to the video processing unit 22, The message is supplied to the video processing unit 22. Further, for example, the superimposition message determination unit 24 that has received the identifier “0x00040107” specifies that the superimposition display of the message “please check the driving position” should be stopped, and superimposes such a message. An instruction to stop the display is supplied to the video processing unit 22.

  For example, the LED light emission control unit 25 that has received the identifier “0x00010207” specifies that the orange LED elements should be continuously lit, and controls the light emission of the LED unit 207 so that the orange LED elements emit light continuously. To do. Further, for example, the LED light emission control unit 25 that has received the identifier “0x00040107” specifies that the yellow LED element should be turned off, and controls the light emission of the LED unit 207 so that the light emission of the yellow LED element is completed. .

  As can be seen from FIG. 4, the contents of processing according to the identifiers of the voice processing unit 23, the superimposed message determination unit 24, and the LED light emission control unit 25 are the status type value and the attention level included in the identifier. Each combination with a value is different. The specific contents of the above processing for each combination of the situation type value and the attention level value can be sufficiently understood by those skilled in the art with reference to the explanation of step S24 and FIG.

(Advantages of driving support system)
With reference to FIG. 7 and FIG. 8, the advantages of the driving support system according to the present embodiment will be described in comparison with the driving support system according to the comparative example. 7 and 8 are diagrams for explaining the advantages of the driving support apparatus according to the present embodiment.

  The driving support device of the driving support system according to the comparative example will transmit a message to call attention when it detects the occurrence of a situation that needs to call the driver's attention during the transmission of the camera video to the smartphone. After being embedded in the video frame to be transmitted, the video frame is transmitted to the smartphone.

  FIG. 7 and FIG. 8 (a) show the time lag from detection of the occurrence of a situation that requires the driver to be alerted until the driver is notified of the situation, with respect to the driving support system according to the comparative example. Show. Moreover, (b) of FIG.7 and FIG.8 has shown this time lag about the driving assistance system which concerns on this embodiment. In FIG. 8, each block labeled “A” or the like schematically represents a video frame. Further, the position occupied by each block on the time axis in FIG. 8 is roughly the time when the driving support device generates a video frame corresponding to the block and the time when the smartphone displays the video frame corresponding to the block. Show.

  As can be seen by comparing (a) and (b) of FIG. 7 ((a) and (b) of FIG. 8), the above time lag (t3-t0 in FIG. T (e ')-t (e) in FIG. 8 is the time lag (t3'-t0 in FIG. 7 and t (h) -t (e) in FIG. 8 in the driving support system according to the comparative example. )) Shorter than.

  The time lag in the driving support system according to the present embodiment is mainly due to the transmission latency of small data called an identifier, and the driving support system according to the present embodiment detects the above situation. This is because the driver can be notified of the above situation before completing the time-consuming processes such as encoding, transmission and decoding of the referenced video frame.

(Advantages of the driving support device 10)
The following can be said from the description of the first embodiment.

  That is, in the driving support apparatus 10, the above situation is detected by the camera 102, the photographed video analysis unit 12 (detection means) that detects the occurrence of a situation that requires the driver to be alerted, and the photographed video analysis unit 12. Immediately after that, an identifier corresponding to the content that the driver should pay attention to is transmitted to a smartphone 20 (information presentation terminal) having a function of presenting the content to the driver immediately after receiving the identifier. And a processing unit 13 (identifier transmission means).

  The transmission processing unit 13 (video data transmission unit) transmits the video data of the captured video output from the camera 102 to the smartphone 20.

  As can be seen from the above description regarding the advantages of the driving support system, when the driving support device 10 detects the occurrence of a situation that requires the driver to be alerted during the transmission of the camera image, the driving support device 10 determines the situation. The effect is that it can be conveyed quickly.

[Embodiment 2]
Hereinafter, with reference to FIG. 9, the driving assistance system which concerns on another one Embodiment of this invention is demonstrated. FIG. 9 is a block diagram illustrating configurations of the driving support device 10 ′ and the smartphone 20 ′ included in the driving support system according to the present embodiment. For convenience of explanation, members having the same functions as those explained in the first embodiment are basically given the same reference numerals and explanation thereof is omitted.

(Outline of the driving support system according to the present embodiment)
The driving support system according to the present embodiment is similar to the driving support system according to the first embodiment, but differs from the driving support system according to the first embodiment in the following points.

  That is, the driving support device according to the present embodiment transmits the identifier packet using a communication channel different from the channel used for transmitting the captured moving image data packet.

(Configuration of driving support device 10 ')
As illustrated in FIG. 9, the driving support device 10 ′ includes a control unit 101, a camera 102, a microphone 103, a RAM 104, a communication unit 105A, and a communication unit 105B.

  The control unit 101 is a CPU that controls the entire driving support apparatus 10 ′. The control unit 101 functions as a captured moving image data generation unit 11, a captured video analysis unit 12 ′, and a transmission processing unit 13.

  The captured video analysis unit 12 ′ functions in the same manner as the captured video analysis unit 12 except that the captured video data is not updated.

  The communication unit 105A is a wireless LAN (WiFi) communication module used to transmit a packet of captured moving image data to the smartphone 20 '.

  The communication unit 105B is a Bluetooth (registered trademark) communication module used to transmit an identifier packet to the smartphone 20 'and has a frequency hopping function.

(Configuration of smartphone 20 ')
The communication unit 202A is a WiFi communication module used for receiving a packet of captured moving image data from the driving support device 10 ′.

  The communication unit 202B is a Bluetooth communication module used to receive an identifier packet from the driving support device 10 ', and has a frequency hopping function.

(Operation of driving support system)
The driving support system according to the present embodiment basically operates according to the flowcharts of FIGS. 5 and 6 similarly to the driving support system according to the first embodiment. The operation of the driving support system according to the present embodiment is as follows. This is different from the operation of the driving support system according to the first embodiment.
In step S <b> 3, the transmission processing unit 13 transmits the identifier packet generated by the captured video analysis unit 12 ′ to the smartphone 20 via the communication unit 105 </ b> B instead of the communication unit 105.
In step S <b> 4, the transmission processing unit 13 transmits the packet group of the video frame to be analyzed and the packet group of the corresponding recorded audio to the smartphone 20 via the communication unit 105 </ b> A instead of the communication unit 105.
In step S <b> 11, the reception processing unit 21 receives the video frame packet group and the corresponding audio packet group transmitted in step S <b> 4 via the communication unit 202 </ b> A instead of the communication unit 202.
In step S21, the reception processing unit 21 receives the identifier packet via the communication unit 202B instead of the communication unit 202.

(Advantages of the driving support system according to the present embodiment)
When a large number of packets press the communication band, the frequency of packet loss increases, and congestion due to packet loss can frequently occur. In this regard, in the driving support system according to the present embodiment, the influence on the arrival time of the identifier packet on the smartphone is reduced under a situation where a large amount of packets of captured moving image data is compressing the communication band of WiFi communication. can do. This is because the identifier packet is transmitted by frequency-hopping Bluetooth communication.

  Therefore, the driving support system according to the present embodiment detects the state of the WiFi communication used for transmission / reception of the captured moving image data when the occurrence of a situation that requires the driver to be alerted during the transmission of the camera video is detected. Regardless of the situation, it is possible to quickly convey the situation to the driver.

(Appendix 1 regarding Embodiment 2)
Similar to the driving support apparatus according to the first embodiment, the driving support apparatus according to the present embodiment may include only one communication unit for near field communication. Similarly, similarly to the smartphone according to the first embodiment, the smartphone according to the present embodiment may include only one short-distance communication module (for example, a WiFi communication module).

  In this case, the driving support apparatus transmits the identifier packet to the smartphone through a certain communication channel, and transmits the captured moving image data packet to the smartphone through another communication channel logically different from the communication channel.

  For example, a driving support device including a WiFi communication module as a communication unit for short-range communication transmits an identifier packet through a certain band (for example, the first channel of 2.4 GHz band (center frequency: 2412 MHz)) A packet of captured moving image data may be transmitted through a band (for example, an 11th channel (center frequency: 2462 MHz) in the 2.4 GHz band).

(Appendix 2 regarding Embodiment 2)
The transmission processing unit 13 may transmit a packet of captured moving image data via the communication unit 105B, and may transmit an identifier packet via the communication unit 105A. However, as shown in FIG. 9, it is preferable that the transmission processing unit 13 transmits a packet of captured moving image data via the communication unit 105A and transmits an identifier packet via the communication unit 105B.

  This is because, since Bluetooth communication is a communication method that employs frequency hopping, transmission of an identifier packet is not easily affected even when a specific frequency band is being compressed by WiFi communication.

(Appendix 3 regarding Embodiment 2)
The driving support apparatus according to the present embodiment switches the communication unit used for transmitting the identifier packet from one communication unit to the other communication unit, and the communication unit used for transmitting the captured video data packet is the other communication unit. The process of switching from one communication unit to the other communication unit may be performed dynamically.

  For example, the transmission processing unit compares the current effective speed of Bluetooth communication with the current effective speed of WiFi communication, and the effective speed of communication used for transmitting the packet of the identifier is used for transmitting the packet of the captured video data. The above switching process may be performed at a timing when it is detected that the speed is lower than the effective communication speed.

(Appendix 4 regarding Embodiment 2)
The application installed in the smartphone according to the present embodiment may include a power saving mode in which only the identifier packet is received without establishing a WiFi connection with the driving support device.

[Embodiment 3]
Hereinafter, a system according to still another embodiment of the present invention will be described with reference to FIG. FIG. 10 is a block diagram illustrating configurations of the smartphone 1000 and the display audio 2000 included in the driving support system according to the present embodiment. For convenience of explanation, members having the same functions as those explained in the first embodiment are basically given the same reference numerals and explanation thereof is omitted.

(Outline of the driving support system according to the present embodiment)
An application that causes the smartphone to function in substantially the same manner as the driving support device 10 is installed in the smartphone 1000.

  The display audio 2000 is mounted on a car and is configured to execute a program having a function similar to the function of an application installed in the smartphone 20 and a display audio function at the time of startup. .

(Configuration of smartphone 1000)
As illustrated in FIG. 10, the smartphone 1000 includes a control unit 101, a camera 102, a RAM 104, and a communication unit 105.

  The control unit 101 is a CPU that comprehensively controls the entire smartphone 1000. The control unit 101 functions as the captured moving image data generation unit 11, the captured video analysis unit 12, and the transmission processing unit 13 by executing the above-described application.

  The captured moving image data generation unit 11 generates a video frame based on the captured video signal input from the camera 102. The captured moving image data generation unit 11 generates captured moving image data for streaming transmission by repeating the generation of the video frame.

(Configuration of Display Audio 2000)
As shown in FIG. 10, the display audio 2000 includes a control unit 201, a communication unit 202, a buffer 203, a display 204, a flash memory 205, and a speaker 206.

  The control unit 201 is a CPU that comprehensively controls the entire display audio 2000. The control unit 201 functions as the reception processing unit 21 ′, the video processing unit 22 ′, the audio processing unit 23 ′, and the superimposed message determination unit 24 by executing the above-described program when the display audio 2000 is activated.

  The reception processing unit 21 ′ supplies the received identifier only to the audio processing unit 23 and the superimposed message determination unit 24, and does not record the captured moving image data in the flash memory 205, according to the first embodiment. Functions in the same manner as the unit 21.

  The video processing unit 22 ′ displays a UI screen for playing back music recorded in the flash memory 205 on the display 204. Each time a video frame is supplied from the reception processing unit 21 ′, the video processing unit 22 ′ displays the video frame next to the UI screen.

  Similar to the video processing unit 22, when the message is supplied from the superimposition message determination unit 24, the video processing unit 22 'superimposes the message on each video frame to be displayed thereafter.

  The audio processing unit 23 ′ reproduces music recorded in the flash memory 205. Each time audio data is supplied from the reception processing unit 21 ′, the audio processing unit 23 ′ decreases the volume of the music being played and outputs the sound represented by the audio data from the speaker 206.

  Similar to the voice processing unit 23, the voice processing unit 23 ′ has an identifier (specifically, an identifier other than an identifier indicating that the driver needs to be alerted) in the reception processing unit 21 ′. As soon as it is supplied from, the process of repeatedly outputting a message according to the value of the identifier is started.

(Operation of driving support system)
The driving support system according to the present embodiment basically operates according to the flowcharts of FIGS. 5 and 6 similarly to the driving support system according to the first embodiment. The operation of the driving support system according to the present embodiment is as follows. This is different from the operation of the driving support system according to the first embodiment.
In step S <b> 3, the transmission processing unit 13 transmits the identifier packet generated by the captured video analysis unit 12 to the display audio 2000.
In step S <b> 4, the transmission processing unit 13 transmits only the packet group of the analysis target video frame to the display audio 2000.
In step S <b> 11, the reception processing unit 21 ′ receives only the video frame packet group from the smartphone 1000.
The identifier supply destination in step S22 is the voice processing unit 23 ′ and the superimposed message determination unit 24. That is, the processing of step S23 is performed by the voice processing unit 23 ′ and the superimposed message determination unit 24, and the processing of step S24 is performed by the voice processing unit 23 ′ and / or the superimposed message determination unit. 24.

(Advantages of the driving support system according to the present embodiment)
As can be seen from the above description, the driving support system according to the present embodiment can call the driver's attention during the transmission of the camera video without requiring an expensive dedicated driving support device 10, 10 ′. When the occurrence of a necessary situation is detected, the situation can be quickly communicated to the driver.

(Other supplementary notes 1)
Software for live streaming distribution may be installed in the driving support device as software for transmitting the captured moving image data to the smartphone.

  Also, instead of the configuration in which the driving support device transmits the captured video data to the smartphone and the smartphone plays back the captured video data, the driving support device plays back the captured video data, and the captured video data playback screen and playback audio data You may employ | adopt the structure which transmits to a smart phone.

  When the latter configuration is adopted, software for realizing the latter configuration may be VNC (Virtual Network Computing) software or Miracast (Miracast).

(Other appendix 2)
Devices capable of notifying a situation that needs to call the driver's attention are not limited to video display devices, audio output devices, and LED devices.

  For example, the smartphone may include a flash light emission control unit (not shown) that controls light emission of a built-in flash light (not shown).

  The flashlight emission control unit determines a flashlight emission pattern based on the situation type value when an identifier including a notification device type value indicating that the flashlight is set as a notification device is supplied from the reception processing unit. At the same time, the light emission intensity of the flashlight may be determined based on the attention level value.

  The flash light emission control unit may cause the flash light to emit light based on the determined light emission pattern and light emission intensity.

  For example, the smartphone may include a vibration unit controller (not shown) that controls the operation of a built-in vibration unit (not shown).

  The vibration unit control unit determines the vibration pattern of the vibration unit based on the situation type value when an identifier including a notification device type value indicating that the vibration unit is set as a notification device is supplied from the reception processing unit. The vibration intensity of the vibration unit may be determined based on the attention level value.

  Then, the vibration unit control unit may vibrate the vibration unit based on the determined vibration pattern and vibration intensity.

[Example of software implementation]
Control blocks (especially, the captured moving image data generation unit 11, the captured video analysis units 12, 12 ′, and the transmission processing unit 13) of the driving support devices 10 and 10 ′, and the control blocks of the smartphones 20 and 20 ′ (particularly reception). The processing units 21 and 21 ′, the video processing units 22 and 22 ′, the audio processing units 23 and 23 ′, the superimposition message determination unit 24, and the LED light emission control unit 25) are logics formed in an integrated circuit (IC chip) or the like. It may be realized by a circuit (hardware) or may be realized by software using a CPU (Central Processing Unit).

  In the latter case, the driving support devices 10 and 10 'and the smartphones 20 and 20' are read by a computer (or CPU) that executes a command of a program that is software that realizes each function, and the program and various data. A ROM (Read Only Memory) or a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) in which the program is expanded, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

[Summary]
The driving assistance apparatus (for example, the driving assistance apparatus 10, 10 ′) according to the first aspect of the present invention includes a camera (for example, the camera 102) and detection means for detecting the occurrence of a situation that requires the driver to be alerted. Immediately after the situation is detected by the detection means (for example, the captured video analysis units 12 and 12 '), an identifier corresponding to the content that the driver should pay attention to is displayed immediately after receiving the identifier. Identifier transmitting means (for example, transmission processing unit 13) that transmits information to an information presentation terminal (for example, smartphone 20, 20 ′) having a function of presenting the above content, and video data of the captured video output by the camera Is transmitted to the information presentation terminal (for example, transmission processing unit 13).

  According to the above configuration, the driving support device generates a situation that requires the driver to be alerted while transmitting the video data of the captured video output from the camera toward the information presentation terminal. Is detected, the identifier is immediately transmitted to the information presentation terminal. Since the data size of the identifier is very small, the time from when the driving support apparatus transmits the identifier until the information presentation terminal receives the identifier is usually very short.

  Therefore, as soon as the driving assistance device detects the occurrence of a situation that requires the driver to be alerted while transmitting the camera video to the information presentation terminal, the information presentation terminal The driver should be advised of the contents to be noted.

  As a result, the driving support apparatus has an effect that when the occurrence of a situation that requires the driver to be alerted during the transmission of the camera video is detected, the situation can be quickly communicated to the driver. .

  In the driving assistance apparatus (for example, driving assistance apparatus 10 ′) according to aspect 2 of the present invention, in the aspect 1, the identifier transmission unit (for example, the captured video analysis unit 12 ′) is configured such that the video data transmission unit includes the video. The identifier may be transmitted using a communication channel (for example, a channel for Bluetooth communication) different from a communication channel (for example, a channel for WiFi communication) used for data transmission.

  According to the above configuration, the driving support device is not affected at all or is not so affected even when the bandwidth of the communication channel used for transmission of the video data is compressed. The identifier can be sent without receiving it.

  Therefore, the driving support device has an additional effect that it can quickly inform the driver of a situation that requires the driver to be alerted.

  In the driving support device according to aspect 3 of the present invention, in the aspect 2, the identifier transmitting unit may transmit the identifier via a communication module (for example, the communication unit 105B) having a frequency hopping function. .

  According to said structure, the said driving assistance device switches the zone | band used for transmission of the said identifier, whenever very short time passes.

  Accordingly, when the band of the communication channel used for transmitting the video data is compressed, the band overlaps with the band of the communication channel used for transmitting the identifier, which affects the transmission of the identifier. Even if there is, the effect is temporary.

  As a result, the driving support device is in a situation where it is necessary to call the driver's attention without much influence even when a specific band is being compressed around the driving support device. It has the further effect of being able to communicate quickly to the driver.

  In the driving support apparatus according to aspect 4 of the present invention, in the aspect 2, the identifier transmission unit uses a band different from the band used by the communication module used by the video data transmission unit to transmit the video data. The identifier may be transmitted via another communication module.

  According to the above configuration, since the bandwidth of the communication channel used for transmitting the identifier does not overlap with the bandwidth of the communication channel used for transmitting the video data, the bandwidth is being pressed. However, the transmission of the identifier is not affected.

  Therefore, the driving assistance device is driven in a situation where it is necessary to call the driver's attention without any influence even when a specific band is being compressed around the driving assistance device. The effect is that it can be quickly communicated to the person.

  The information presentation terminal (for example, smart phone 20, 20 ') which concerns on aspect 5 of this invention gives the identifier according to the content which a driver | operator should be careful from an external driving assistance device (for example, driving assistance device 10, 10'). An identifier receiving means (for example, the reception processing unit 21) for receiving and a video data receiving means (for example, for receiving image data of the captured video output from the camera (for example, the camera 102) of the driving support device from the driving support device (for example, A reception processing unit 21), a display processing unit (for example, the video processing unit 22) for displaying the captured image on a display screen (for example, the display 204), and the operation immediately after the identifier receiving unit receives the identifier. Presenting means (for example, a video processing unit 22, an audio processing unit 23, and an LED light emission control unit 25) for presenting the above contents to a person.

  According to the above configuration, the information presentation terminal promptly informs the driver of a situation in which it is necessary to call the driver's attention while displaying the camera image being received from the driving support device. This is advantageous in that it is possible to convey a simple situation (before the timing at which the camera image can be recognized).

  In the information presentation terminal (for example, the smartphone 20 ′) according to the sixth aspect of the present invention, in the fifth aspect, the identifier receiving unit includes a communication channel (for example, WiFi) used by the video data receiving unit to receive the video data. The identifier may be received using a communication channel (for example, a Bluetooth communication channel) different from the communication channel.

  According to the above configuration, the driving support device is not affected at all or is not so affected even when the bandwidth of the communication channel used for receiving the video data is compressed. The identifier can be received without receiving it.

  Therefore, the information presenting terminal has an additional effect that it can quickly inform the driver of a situation that requires the driver to be alerted.

  In the information presentation terminal according to aspect 7 of the present invention, in the aspect 6, the identifier receiving unit may receive the identifier via a communication module (for example, the communication unit 202B) having a frequency hopping function. .

  According to said structure, the said information presentation terminal switches the zone | band used for reception of the said identifier, whenever very short time passes.

  Accordingly, when the band of the communication channel used for transmitting the video data is compressed, the band overlaps with the band of the communication channel used for transmitting the identifier, which affects the reception of the identifier. Even if there is, the effect is temporary.

  Therefore, the information presenting terminal is driven by a situation where it is necessary to call the driver's attention without being affected so much even when a specific band is being compressed around the driving support device. The effect is that it can be quickly communicated to the person.

  In the information presentation terminal according to aspect 8 of the present invention, in the aspect 6, the identifier receiving unit uses a band different from a band used by the communication module used by the video data receiving unit to receive the video data. The identifier may be received via another communication module.

  According to the above configuration, since the bandwidth of the communication channel used for receiving the identifier does not overlap with the bandwidth of the communication channel used for receiving the video data, the bandwidth is being pressed. However, the reception of the identifier is not affected.

  Therefore, the information presenting terminal is not affected at all even when a specific band is being compressed around the information presenting terminal, and the driver needs to be alerted to the driver. The effect is that it can be quickly communicated to the person.

  In the information presentation terminal according to aspect 9 of the present invention, in any of the aspects 5 to 8, the presentation unit displays the video immediately after the identifier reception unit receives the identifier. A message corresponding to the identifier may be displayed on the display screen.

  The driving support system according to aspect 10 of the present invention includes the driving support device according to aspect 1 and the information presentation terminal according to aspect 5.

  According to said structure, the said driving assistance system has an effect similar to the driving assistance apparatus which concerns on the said aspect 1. FIG.

  The driving support method according to aspect 11 of the present invention includes a detection step (for example, step S1) for detecting the occurrence of a situation that requires the driver to be alerted in the driving support method by the driving support device having a camera. Immediately after the situation is detected in the detecting step, an identifier according to the content to be noted by the driver, and an information presentation function for presenting the content to the driver immediately after receiving the identifier An identifier transmission step (e.g., step S3) for transmission toward the terminal, and a video data transmission step (e.g., step S4) for transmitting the video data of the captured video output from the camera toward the information presentation terminal. Contains.

  According to said structure, the said driving assistance method has an effect similar to the driving assistance apparatus which concerns on the said aspect 1. FIG.

  The driving support method according to the twelfth aspect of the present invention is a driving support method using an information presentation terminal, and receives video data of a captured image output from the camera of an external driving support device from the driving support device. (For example, step S11), a display processing step for displaying the captured image on the display screen (for example, step S12), and an identifier receiving step for receiving an identifier corresponding to the content to be noted by the driver from the driving support device. (For example, step S21) and a presentation step (for example, step S12) of presenting the content to the driver immediately after receiving the identifier in the identifier receiving step.

  According to said structure, the said driving assistance method has an effect similar to the information presentation terminal which concerns on the said aspect 4. FIG.

  The driving support apparatus according to each aspect of the present invention may be realized by a computer. In this case, the driving support apparatus is realized by a computer by causing the computer to function as each unit included in the driving support apparatus. A program and a computer-readable recording medium on which the program is recorded also fall within the scope of the present invention.

  Similarly, the information presentation terminal according to each aspect of the present invention may be realized by a computer. In this case, the information presentation terminal is made to function as each unit included in the information presentation terminal. The program to be realized and the computer-readable recording medium on which the program is recorded also fall within the scope of the present invention.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

  The present invention can be suitably applied to a driving support device including a camera and a smartphone.

10 Driving Support Device 101 Control Unit (Computer)
11 shooting moving picture data generation unit 12 shooting video analysis unit (detection means)
13 Transmission processing unit (identifier transmission means, video data transmission means)
105 Communication unit (communication module)
20 Smartphone (information presentation terminal)
201 Control unit (computer)
21 Reception processing unit (identifier receiving means, video data receiving means)
22 Video processing unit (display processing means, presentation means)
23 Voice processing unit (presentation means)
24 Superimposition message determination unit 25 LED light emission control unit (presentation means)
202 Communication unit (communication module)
1000 Smartphone (driving support device)
2000 Display audio (information presentation terminal)

Claims (16)

A camera,
Detection means for detecting the occurrence of a situation that needs to call the driver's attention;
Immediately after the situation is detected by the detecting means, an identifier corresponding to the content to be noted by the driver is displayed on an information presentation terminal having a function of presenting the content to the driver immediately after receiving the identifier. An identifier transmission means for transmitting
A driving support device, comprising: video data transmitting means for transmitting video data indicating a captured video output from the camera to the information presentation terminal.   2. The operation according to claim 1, wherein the identifier transmitting unit transmits the identifier using a communication channel different from a communication channel used by the video data transmitting unit to transmit the video data. Support device.   The driving support apparatus according to claim 2, wherein the identifier transmitting unit transmits the identifier via a communication module having a frequency hopping function.   The identifier transmitting means transmits the identifier via another communication module that uses a band different from the band used by the communication module used by the video data transmitting means to transmit the video data. The driving support apparatus according to claim 2, wherein the driving support apparatus is characterized. Identifier receiving means for receiving an identifier from an external driving support device according to the content to be noticed by the driver;
Video data receiving means for receiving, from the driving support device, video data indicating a captured video output by the camera of the driving support device;
Display processing means for displaying the captured image on a display screen;
An information presentation terminal comprising: presentation means for presenting the content to the driver immediately after the identifier receiving means receives the identifier.   6. The information according to claim 5, wherein the identifier receiving unit receives the identifier using a communication channel different from a communication channel used by the video data receiving unit to receive the video data. Presenting terminal.   The information presentation terminal according to claim 6, wherein the identifier receiving means receives the identifier via a communication module having a frequency hopping function.   The identifier receiving means receives the identifier via another communication module that uses a band different from the band used by the communication module used by the video data receiving means to receive the video data. The information presentation terminal according to claim 6, wherein the terminal is an information presentation terminal.   The presenting means displays a message corresponding to the identifier on the display screen on which the captured video is displayed immediately after the identifier receiving means receives the identifier. The information presentation terminal according to any one of 8.   A driving support system comprising: the driving support device according to claim 1; and the information presentation terminal according to claim 5. In a driving support method by a driving support device equipped with a camera,
A detection step for detecting the occurrence of a situation that needs to call the driver's attention;
Immediately after the situation is detected in the detection step, an information presentation terminal having a function for presenting the content to the driver immediately after receiving the identifier according to the content to be noted by the driver. An identifier sending step for sending to
And a video data transmission step of transmitting video data indicating the captured video output from the camera to the information presentation terminal. A driving support method using an information presentation terminal,
A video data receiving step for receiving video data indicating the captured video output by the camera of the external driving support device from the driving support device;
A display processing step for displaying the captured image on a display screen;
An identifier receiving step for receiving an identifier according to the content to be noted by the driver from the driving support device;
Immediately after receiving the identifier in the identifier receiving step, a presentation step for presenting the content to the driver is included.   A program for causing a computer to function as the driving support device according to any one of claims 1 to 4, wherein the program causes the computer to function as each of the means.   A computer-readable recording medium on which the program according to claim 13 is recorded.   A program for causing a computer to function as the information presentation terminal according to any one of claims 5 to 9, wherein the program causes the computer to function as each of the above means.   The computer-readable recording medium which recorded the program of Claim 15.

Images